Method and audio decoder for downscaling decoding

Abstract

The present application provides a method and an audio decoder for downscaling decoding. If a synthesis window used for downscaling audio decoding is a downsampled version of a reference synthesis window involved in a non-downscaling audio decoding process, the downsampled version of the audio decoding process can be implemented more efficiently and / or with improved compatibility maintenance, wherein the downsampled version is downsampled by a downsampling factor and obtained using segmented interpolation with 1 / 4 of the frame length as segments, and the downsampling factor represents the deviation of the downsampled sampling rate from the original sampling rate.

Description

[0001] This application is a divisional application of invention patent application No. 201680047160.9 entitled “Reduced Coding”, which entered the Chinese national phase of PCT international application PCT / EP2016 / 063371 with an application date of June 10, 2016. Technical Field

[0002] The present application relates to downscaled decoding concepts. Background Art

[0003] MPEG-4 Enhanced Low Delay AAC (AAC-ELD) typically operates at a sampling rate of up to 48kHz, resulting in an algorithmic delay of 15ms. For certain applications, such as lip-sync transmission of audio, lower latency is desirable. AAC-ELD has been designed to provide such an option by operating at a higher sampling rate (e.g., 96kHz), thereby providing an operating mode with even lower latency (e.g., 7.5ms). However, this operating mode comes with unnecessarily high complexity due to the high sampling rate.

[0004] The solution to this problem is to apply a downscaled version of the filter bank and thereby render the audio signal at a lower sampling rate (e.g. 48kHz instead of 96kHz). The downscaling operation is already part of AAC-ELD, as it is inherited from the MPEG-4 AAC-LD codec that forms the basis of AAC-ELD.

[0005] However, the problem that remains is how to find a reduced version of a specific filter bank. That is, the only uncertainty is the way to derive the window coefficients while supporting a clear conformance test for the reduced mode of operation of the AAC-ELD decoder.

[0006] In the following, the principle of the reduced operation mode of the AAC-(E)LD codec is described.

[0007] The reduced mode of operation for AAC-LD, or AAC-LD, is described in ISO / IEC 14496-3:2009, section 4.6.17.2.7, "Adaptation to systems with lower sampling rates," as follows:

[0008] “In some applications, it may be necessary to integrate a low-latency decoder into an audio system that operates at a lower sampling rate (e.g., 16 kHz) but has a much higher nominal sampling rate of the bitstream payload (e.g., 48 kHz, corresponding to an arithmetic codec delay of approximately 20 ms). In this case, it is advantageous to decode the output of the low-latency codec directly at the target sampling rate, rather than using an additional sample rate conversion operation after decoding.

[0009] This can be approximated by reducing both the frame size and the sampling rate appropriately by integer factors (e.g. 2, 3) to yield the same time / frequency resolution of the codec. For example, by retaining only the lowest third of the spectral coefficients (i.e. 480 / 3 = 160) before the synthesis filter bank, and reducing the inverse transform size to one-third (i.e. a window size of 960 / 3 = 320), the codec output can be generated at a 16 kHz sampling rate instead of the nominal 48 kHz.

[0010] Thus, lower sample rate decoding reduces memory requirements and computational requirements, but may not produce exactly the same output as would be obtained by full bandwidth decoding followed by band limiting and sample rate conversion.

[0011] Note that, as mentioned above, decoding at lower sampling rates does not affect the interpretation of the levels, which refer to the nominal sampling rate of the AAC low-delay bitstream payload.

[0012] Note that AAC-LD uses the standard MDCT framework and two window shapes: a sine window and a low overlap window. Both windows are fully described by formulas, so the window coefficients can be determined for any transform length.

[0013] The AAC-ELD codec shows two main differences compared to AAC-LD:

[0014] Low-delay MDCT window (LD-MDCT)

[0015] Possibility to exploit low-latency SBR tools

[0016] The IMDCT algorithm using a low-delay MDCT window is described in 4.6.20.2 of [1], which is very similar to the standard IMDCT version using, for example, a sine window. The coefficients for the low-delay MDCT window (frame sizes of 480 and 512 samples) are given in Tables 4.A.15 and 4.A.16 of [1]. Note that the coefficients cannot be determined by formulas because they are the result of an optimization algorithm. Figure 9 A diagram showing the window shape for a frame size of 512.

[0017] When the Low Delay SBR (LD-SBR) tool is used in conjunction with the AAC-ELD encoder, the filter bank of the LD-SBR module is also reduced. This ensures that the SBR module operates with the same frequency resolution, so no further adaptation is required.

[0018] The above description therefore reveals the need to reduce the decoding operation, for example the decoding at AAC-ELD. It is feasible to find the coefficients of the reduced synthesis window function again, but this is a tedious task, requires additional storage space to store the reduced version, and the consistency check between the non-reduced decoding and the reduced decoding is more complicated, or from another point of view, for example does not comply with the reduction method required by AAC-ELD. Depending on the reduction ratio, i.e. the ratio between the original sampling rate and the reduced sampling rate, the reduced synthesis window function can be derived simply by downsampling (i.e. extracting one out of every two, three... window coefficients of the original synthesis window function), but this process does not accordingly produce sufficient consistency between the non-reduced decoding and the reduced decoding. The use of more complex extraction processes applied to the synthesis window function leads to unacceptable deviations from the shape of the original synthesis window function. Therefore, there is a need in the art for an improved reduced decoding concept. Summary of the Invention

[0019] It is therefore an object of the present invention to provide an audio decoding scheme that implements such improved downscaling decoding.

[0020] This object is achieved by an audio decoder. The audio decoder is configured to decode an audio signal from a data stream at a first sampling rate, the audio signal being transform-encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate. The audio decoder comprises: a receiver configured to receive N spectral coefficients of each frame of the audio signal, wherein the length of a frame is N; a grabber configured to grab a low-frequency component of length N / F from the N spectral coefficients for each frame; a spectrum-time modulator configured to subject the low-frequency component to an inverse transform for each frame to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F extending over the corresponding frame and E+1 previous frames in time; and a windower configured to window the time portion using a synthesis window of length (E+2)·N / F for each frame, the synthesis window including a zero portion of length 1 / 4·N / F at its front end and having a peak value within a time interval of the synthesis window, the time portion The interval is after the zero part and has a length of 7 / 4·N / F, so that the window adder obtains a windowed time part with a length of (E+2)·N / F; and a time domain aliasing eliminator is configured to subject the windowed time part of the frame to overlap-addition processing so that the tail component of the windowed time part of the current frame with a length of (E+1) / (E+2) overlaps with the front end of the windowed time part of the previous frame with a length of (E+1) / (E+2), wherein the inverse transform is an inverse MDCT or an inverse MDST, and wherein the synthesis window is a down-sampled version obtained by down-sampling a reference synthesis window with a length of (E+2)·N according to a factor F and performing segmented interpolation according to segments with a length of 1 / 4·N, wherein the synthesis window is a cascade of spline functions with a length of 1 / 4·N / F.

[0021] The present invention is based on the following discovery: if the synthesis window used for reduced audio decoding is the following downsampled version of the reference synthesis window involved in the non-reduced audio decoding process, then the reduced version of the audio decoding process can be implemented more efficiently and / or with improved compatibility maintenance, wherein the downsampled version is downsampled according to a downsampling factor and obtained using segmented interpolation with 1 / 4 of the frame length as segment, and the downsampling factor represents the deviation of the downsampled sampling rate from the original sampling rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The advantageous aspects of the present application are the subject of preferred embodiments. The preferred embodiments of the present application are described below with reference to the accompanying drawings, in which:

[0023] Figure 1 is a schematic diagram illustrating perfect reconstruction requirements that need to be followed during downscaling decoding in order to maintain perfect reconstruction;

[0024] Figure 2 shows a block diagram of an audio decoder for downscaling decoding according to an embodiment;

[0025] Figure 3 A schematic diagram is shown which shows in the upper part how an audio signal has been encoded into a data stream at the original sampling rate and in the lower part, separated from the upper part by a horizontal dashed line, a downscaling decoding operation for reconstructing an audio signal from the data stream at a reduced or downscaled sampling rate, in order to illustrate Figure 2 The operating mode of the audio decoder;

[0026] Figure 4 Shown Figure 2 Schematic diagram of the collaboration between the windower and the time domain aliasing eliminator;

[0027] Figure 5 The use of a special treatment of the zero-weighted portion of the spectral-temporally modulated temporal portion to achieve the Figure 4 the possibility of reconstruction;

[0028] Figure 6 A schematic diagram of downsampling for obtaining a downsampled synthesis window is shown;

[0029] Figure 7 A block diagram illustrating the downscaling operation of AAC-ELD including the low-delay SBR tool is shown;

[0030] Figure 8 shows a block diagram of an audio decoder for downscaling decoding according to an embodiment, wherein the modulator, windower and canceller are implemented according to a lifting implementation; and

[0031] Figure 9 A graph showing window coefficients for a low delay window according to AAC-ELD for a frame size of 512 samples as an example of a reference synthesis window to be downsampled is shown. DETAILED DESCRIPTION

[0032] The following description begins with a schematic illustration of an embodiment of reduced decoding for an AAC-ELD codec. Specifically, the following description begins with an embodiment that can implement a reduced mode for AAC-ELD. This description also provides an explanation of the motivation for the embodiments of this application. This description is then summarized, thereby completing the description of the audio decoder and audio decoding method according to embodiments of this application.

[0033] As mentioned in the introduction to this specification, AAC-ELD uses a low-delay MDCT window. To generate its reduced version, the reduced low-delay window, the proposal for forming the reduced mode of AAC-ELD explained later uses a piecewise spline interpolation algorithm that maintains the perfect reconstruction property (PR) of the LD-MDCT window with very high accuracy. Therefore, this algorithm allows the window coefficients to be generated in a compatible manner, both in the direct form as described in ISO / IEC 14496-3:2009 and in the lifted form as described in [2]. This means that both implementations will generate a 16-bit compliant output.

[0034] Interpolation of the low-delay MDCT window is performed as follows.

[0035] Typically, spline interpolation will be used to generate reduced window coefficients to preserve the frequency response and mostly perfect reconstruction properties (approximately 170 dB SNR). The interpolation needs to be constrained to be within certain segments to preserve the perfect reconstruction properties. For the window coefficients c covering the DCT kernel of the transform (see also Figure 1 , c(1024)...c(2048)), the following constraints are required,

[0036] 1=|(sgn·c(i)·c(2N-1-i)+c(N+i)·c(N-1-i)|,

[0037] where i=0,…,N / 2-1 (1)

[0038] where N is the frame size. Some implementations may use different notations to optimize complexity, which is represented here by sgn. The requirements in (1) can be expressed as Figure 1 It should be recalled that even in the case of F=2 (ie half the sampling rate), omitting one out of every two window coefficients of the reference synthesis window in order to obtain a reduced synthesis window is not sufficient.

[0039] The coefficients c(0)…c(2N-1) are listed along the diamond shape. Bold arrows are used to mark the / 4 zeros in the window coefficients that are responsible for the delay reduction of the filter bank. Figure 1 The dependencies between coefficients due to the folding involved in the MDCT are shown, and the points at which the interpolation needs to be constrained in order to avoid any undesirable dependencies are shown.

[0040] Every N / 2 coefficients, interpolation needs to stop to maintain (1)

[0041] Furthermore, due to the inserted zeros, the interpolation algorithm needs to stop every N / 4. This ensures that the zeros are maintained and the interpolation error does not propagate, thus maintaining the PR.

[0042] The second constraint is necessary not only for the segments containing zeros but also for the other segments. Knowing that some coefficients in the DCT kernel are not determined by the optimization algorithm but by formula (1) in order to achieve PR, we can explain the Figure 1 There are several discontinuities in the window shape around c(1536+128) in

[1536] . To minimize the PR error, interpolation needs to stop at these points that appear in the N / 4 grid.

[0043] For this reason, a segment size of N / 4 is chosen for piecewise spline interpolation to generate the reduced window coefficients. The source window coefficients are always given by the coefficients for N=512, which are also used for the reduction operation resulting in a frame size of N=240 or N=120. The basic algorithm is briefly summarized as MATLAB code below:

[0044]

[0045] Since the spline function may not be completely deterministic, the complete algorithm is detailed in the following section, which may be included in ISO / IEC 14496-3:2009 to form an improved reduction mode in AAC-ELD.

[0046] In other words, the following section provides a proposal on how to apply the above ideas to ERAAC ELD, that is, how a low-complexity decoder can decode an ERAAC ELD bitstream encoded at a first data rate at a second data rate lower than the first data rate. However, it should be emphasized that the definition of N used below conforms to the standard. Here, N corresponds to the length of the DCT kernel, while in the above, in the claims, and in the generalized embodiments described later, N corresponds to the frame length, that is, the length of the mutual overlap of the DCT kernels, that is, half the DCT kernel length. Thus, for example, where N is indicated as 512 above, it will be indicated as 1024 below.

[0047] The following paragraphs are proposed for inclusion by amendment in 14496-3:2009.

[0048] A.0 Adaptation to Systems Using Lower Sampling Rates

[0049] For certain applications, ER AAC LD can change the broadcast sampling rate to avoid an additional resampling step (see 4.6.17.2.7). ER AAC ELD can apply a similar reduction step using a low-delay MDCT window and the LD-SBR tool. In the case of AAC-ELD operating with the LD-SBR tool, the reduction factor is limited to multiples of 2. In the absence of LD-SBR, the reduced frame size needs to be an integer.

[0050] A.1 Low-delay MDCT window reduction

[0051] LD-MDCT window w with N=1024 LD The window coefficients are reduced by a factor of F using piecewise spline interpolation. The number of leading zeros in the window coefficients (i.e. N / 8) determines the segment size. The reduced window coefficients w LD_d is used for the inverse MDCT (as described in 4.6.20.2), but with a reduced window length N d = N / F. Note that this algorithm can also generate the downscaled lifting coefficients of LD-MDCT.

[0052]

[0053]

[0054]

[0055] A.2 Reduction of Low-Latency SBR Tools

[0056] In case the low-latency SBR tool is used in conjunction with ELD, the tool can be scaled down to a lower sampling rate, at least for a reduction factor that is a multiple of 2. The reduction factor F controls the number of frequency bands used for the CLDFB analysis and synthesis filterbank. The following two paragraphs describe the reduced CLDFB analysis and synthesis filterbank, see also 4.6.19.4.

[0057] 4.6.20.5.2.1 CLDFB Filter Reduction Analysis

[0058] ● Define the number of reduced CLDFB bands B = 32 / F.

[0059] Shift the samples in array x by B positions. Discard the oldest B samples and store B new samples in positions 0 to B-1.

[0060] Multiply the samples of array x by the window coefficient ci to obtain array z. The window coefficient ci is obtained by linear interpolation of the coefficient c, that is, by the following equation

[0061]

[0062] The window coefficient c can be found in Table 4.A.90.

[0063] ● Sum the samples to create a 2B-element array u:

[0064] u(n)=z(n)+z(n+2B)+z(n+4B)+z(n+6B)+z(n+8B), 0≤n<(2B).

[0065] · Calculate B new subband samples through matrix operation Mu, where

[0066]

[0067] In the equation, exp() represents the complex exponential function, and j is the imaginary unit.

[0068] 4.6.20.5.2.2 CLDFB Filter Bank Reduction Analysis

[0069] • Define the number of reduced CLDFB bands B = 64 / F.

[0070] Shift the samples in array v by 2B positions. Discard the oldest 2B samples.

[0071] ● Multiply the B new complex-valued subband samples by the matrix N, where

[0072]

[0073] In the equation, exp( ) represents the complex exponential function and is the imaginary unit. The real part of the output from this operation is stored in positions 0 to 2B-1 of the array v.

[0074] ● Extract samples from v to create a 10B-element array g.

[0075]

[0076] ● Multiply the samples of array g by the window coefficient ci to generate array w. The window coefficient ci is obtained by linear interpolation of the coefficient c, that is, by the following equation

[0077]

[0078] The window coefficient c can be found in Table 4.A.90.

[0079] Compute B new output samples by summing the samples from array w according to the following formula:

[0080]

[0081] Note that setting F=2 provides a downsampled synthesis filterbank according to 4.6.19.4.3. Hence, in order to process a downsampled LD-SBR bitstream with an additional reduction factor F, F needs to be multiplied by two.

[0082] 4.6.20.5.2.3 Downsampled Real-Valued CLDFB Filter Bank

[0083] Downsampling of CLDFB can also be used for the real-valued version of low-power SBR mode. For illustration purposes, also consider 4.6.19.5.

[0084] For the reduced real-valued analysis and synthesis filterbank, follow the description in 4.6.20.5.2.1 and 4.6.20.2.2 and exchange the exp() modulators in M ​​by cos() modulators.

[0085] A.3 Low-Delay MDCT Analysis

[0086] This subsection describes the low-delay MDCT filter bank used in the AAC ELD encoder. The core MDCT algorithm is mostly unchanged, but the window is longer so that n now runs from -N to N-1 (instead of from 0 to N-1).

[0087] Spectral coefficient X i,k The definition is as follows:

[0088] where 0≤k <N / 2

[0089] in:

[0090] z in = Windowed input sequence

[0091] N = sample index

[0092] K = frequency coefficient index

[0093] I = Block Index

[0094] N = window length

[0095] n0=(-N / 2+1) / 2

[0096] The window length N (based on a sine window) is 1024 or 960.

[0097] The window length of the low-latency window is 2 × N. The windowing is extended into the past in the following way:

[0098] z i,n =w LD (N-1-n)·x' i,n

[0099] For n=-N, ..., N-1, the synthesis window w is used as the analysis window by reversing the order.

[0100] A.4 Low-Delay MDCT Synthesis

[0101] The synthesis filterbank is modified to employ a low-delay filterbank compared to the standard IMDCT algorithm using a sine window. The core IMDCT algorithm is largely unchanged, but the window is longer so that n now runs as high as 2N-1 (instead of N-1).

[0102] where 0≤n<2N

[0103] in:

[0104] n = sampling index

[0105] i = window index

[0106] k = spectral coefficient index

[0107] N = window length / twice the frame length

[0108] n0=(-N / 2+1) / 2

[0109] Where N=960 or 1024.

[0110] Windowing and overlap-add are performed as follows:

[0111] The window of length N is replaced by a window of length 2N, which overlaps more with the past and less with the future (N / 8 values ​​are effectively zero).

[0112] Windowing for low-latency windows:

[0113] z i,n =w LD (n)·x i,n

[0114] The window is now of length 2N, so n = 0,…,2N-1.

[0115] Overlap and add:

[0116]

[0117] Where 0 <= n <N / 2

[0118] Here, it is proposed that these paragraphs be incorporated into 14496-3:2009 by amendment.

[0119] Of course, the above description of possible reduction modes for AAC-ELD represents only one embodiment of the present application, and some modifications are possible. In general, embodiments of the present application are not limited to audio decoders that perform a reduced version of AAC-ELD decoding. In other words, embodiments of the present application can be obtained, for example, by forming an audio decoder that can perform only the inverse transform processing in a reduced manner, without supporting or using various AAC-ELD-specific further tasks, such as scaling factor-based transmission of the spectral envelope, TNS (temporal noise shaping) filtering, spectral band replication (SBR), etc.

[0120] Subsequently, a more general embodiment for an audio decoder is described. The above example of an AAC-ELD audio decoder supporting the reduced mode may therefore represent one implementation of the audio decoder described subsequently. In particular, the decoder explained subsequently is described in Figure 2 As shown in Figure 3 Shown by Figure 2 The decoder performs the following steps.

[0121] Figure 2 The audio decoder (generally indicated by reference numeral 10) comprises a receiver 12, a grabber 14, a spectrum-time modulator 16, a windower 18 and a time-domain aliasing canceller 20, all of which are connected in series with each other in the order mentioned. Figure 3 The interaction and functionality of blocks 12 to 20 of the audio decoder 10 are described. As described at the end of the description of this application, blocks 12 to 20 may be implemented in software, programmable hardware or hardware (e.g. in the form of a computer program, an FPGA or a suitably programmed computer), a programmed microprocessor or an application-specific integrated circuit (where blocks 12 to 20 represent corresponding subroutines, circuit paths, etc.).

[0122] In the manner outlined in more detail below, Figure 2 The audio decoder 10 is configured (and the elements of the audio decoder 10 are configured to cooperate appropriately) to decode an audio signal 22 from a data stream 24. It should be noted that the sampling rate used by the audio decoder 10 to decode the audio signal 22 is 1 / F of the sampling rate used when the audio signal 22 was transform-encoded into the data stream 24 on the encoding side. For example, F can be any rational number greater than 1. The audio decoder can be configured to operate with different or variable reduction factors F or with a fixed reduction factor F. Alternative solutions are described in more detail below.

[0123] exist Figure 3 The upper part of FIG shows how the audio signal 22 is transcoded into the data stream at the encoded or original sampling rate. At 26, Figure 3The spectral coefficients are shown using small boxes or blocks 28 arranged in a spectro-temporal manner along a time axis 30 and a frequency axis 32, respectively, wherein the time axis 30 is in the Figure 3 The frequency axis 32 extends horizontally in Figure 3 The spectral coefficients 28 are transmitted in the data stream 24. Figure 3 The manner in which the spectral coefficients 28 have been obtained and thus the manner in which the spectral coefficients 28 represent the audio signal 22 is shown at 34 in FIG. Figure 3 34 in FIG. 3 shows, for a portion of the time axis 30 , how spectral coefficients 28 belonging to or representing the corresponding time portion are obtained from the audio signal.

[0124] In particular, the coefficients 28 transmitted within data stream 24 are the coefficients of an overlapped transform of audio signal 22, such that audio signal 22, sampled at the original or encoded sampling rate, is divided into temporally consecutive, non-overlapping frames of a predetermined length N, with N spectral coefficients being transmitted in data stream 24 for each frame 36. In other words, transform coefficients 28 are obtained from audio signal 22 using a critically sampled overlapped transform. In the spectro-temporal spectrogram representation 26, each column of the temporal sequence of columns of spectral coefficients 28 corresponds to a respective frame 36 of the sequence of frames. For each frame 36, N spectral coefficients 28 are obtained by a spectral decomposition transform or time-spectral modulation, the modulation function of which, however, extends temporally not only over the frame 36 to which the resulting spectral coefficients 28 belong, but also over E+1 preceding frames, where E can be any integer greater than zero or any even integer. That is, the spectral coefficients 28 of a column belonging to a certain frame 36 in the spectrogram at 26 are obtained by applying the transformation to a transformation window, wherein the transformation window includes, in addition to the corresponding frame, E+1 frames located past the current frame. The spectral decomposition of the samples of the audio signal within the transformation window 38 is achieved using a low-latency unimodal analysis window function 40 (which is in Figure 334 ), wherein the spectral samples within the transform window 38 are weighted before being subjected to the MDCT or MDST or other spectral decomposition transform using the low-delay unimodal analysis window function 40. To reduce encoder-side latency, the analysis window 40 includes a zero-interval 42 at its temporal front end, so that the encoder does not need to wait for the corresponding portion of the latest samples within the current frame 36 to calculate the spectral coefficients 28 of the current frame 36. That is, within the zero-interval 42, the low-delay window function 40 is zero or has a zero window coefficient, so that the collocated audio samples of the current frame 36 do not contribute to the transform coefficients 28 and data stream 24 transmitted for that frame due to the window weighting 40. That is, to summarize the above, the transform coefficients 28 belonging to the current frame 36 are obtained by windowing and spectral decomposition of the audio signal samples within the transform window 38, wherein the transform window 38 includes the current frame and the temporally previous frame, and the transform window 38 temporally overlaps with the corresponding transform window used to determine the spectral coefficients 28 belonging to the temporally adjacent frame.

[0125] Before resuming the description of the audio decoder 10, it should be noted that the description provided so far of the transmission of the spectral coefficients 28 within the data stream 24 has been simplified with respect to the manner in which the spectral coefficients 28 are quantized or encoded into the data stream 24 and / or the manner in which the audio signal 22 is preprocessed before the audio signal is subjected to an overlap transform. For example, the audio encoder that transform-encodes the audio signal 22 into the data stream 24 may be controlled via a psychoacoustic model, or may use a psychoacoustic model to keep the quantization noise and the quantization of the spectral coefficients 28 imperceptible to the listener and / or below a masking threshold function, thereby determining scaling factors for spectral bands that are used to scale the quantized and transmitted spectral coefficients 28. The scaling factors will also be signaled in the data stream 24. Alternatively, the audio encoder may be a TCX (Transform Coded Excitation) type encoder. The audio signal will then have been subjected to linear prediction analysis filtering before forming the spectrotemporal representation 26 of the spectral coefficients 28 by applying an overlap transform to the excitation signal (i.e., the linear prediction residual signal). For example, linear prediction coefficients may also be signaled in the data stream 24 and spectrally uniform quantization may be applied to obtain spectral coefficients 28 .

[0126] Furthermore, the description presented so far has also been simplified with respect to the frame length of the frames 36 and / or with respect to the low-delay window function 40. In practice, the audio signal 22 may have been encoded into the data stream 24 in a manner that uses varying frame sizes and / or different windows 40. However, the following description focuses on one window 40 and one frame length, although the subsequent description can easily be extended to the case where the entropy encoder varies these parameters during encoding of the audio signal into the data stream.

[0127] Return to Figure 2 The audio decoder 10 and its description, the receiver 12 receives the data stream 24 and thus receives N spectral coefficients 28 for each frame 36, namely Figure 3 The corresponding columns of coefficients 28 are shown. It will be recalled that the temporal length of the frame 36 measured in samples at the original or encoded sampling rate is N, as Figure 3 As shown in 34, but Figure 2 The audio decoder 10 is configured to decode the audio signal 22 at a reduced sampling rate. The audio decoder 10, for example, only supports the reduced decoding function described below. Alternatively, the audio decoder 10 will be able to reconstruct the audio signal at the original or encoded sampling rate, but can switch between a reduced decoding mode and a non-reduced decoding mode, wherein the reduced decoding mode is consistent with the operating mode of the audio decoder 10 as described below. For example, in the event of low battery power, reduced reproduction environment capabilities, etc., the audio encoder 10 can switch to the reduced decoding mode. Whenever the situation changes, the audio decoder 10 can, for example, switch back from the reduced decoding mode to the non-reduced decoding mode. In any case, according to the reduced decoding process of the decoder 10 as described below, the audio signal 22 is reconstructed at a sampling rate at which, at the reduced sampling rate, the frame 36 has a shorter length measured by the samples of the reduced sampling rate, i.e., the sample length is N / F at the reduced sampling rate.

[0128] The output of the receiver 12 is a sequence of N spectral coefficients per frame 36, i.e. a set of N spectral coefficients, i.e. Figure 3. As can be seen from the above brief description of the transform coding process used to form the data stream 24, the receiver 12 can apply various tasks when obtaining the N spectral coefficients for each frame 36. For example, the receiver 12 can use entropy decoding to read the spectral coefficients 28 from the data stream 24. The receiver 12 can also perform spectral shaping on the spectral coefficients read from the data stream using scaling factors provided in the data stream and / or scaling factors derived from linear prediction coefficients transmitted within the data stream 24. For example, the receiver 12 can obtain scaling factors from the data stream 24 (i.e., on a per-frame and per-subband basis) and use these scaling factors to scale the scaling factors transmitted within the data stream 24. Alternatively, the receiver 12 can derive scaling factors from the linear prediction coefficients transmitted within the data stream 24 for each frame 36 and use these scaling factors to scale the transmitted spectral coefficients 28. Optionally, the receiver 12 can perform gap filling to synthetically fill zero-quantized portions within the set of N spectral coefficients 18 per frame. Additionally or alternatively, the receiver 12 may apply a TNS synthesis filter to the transmitted TNS filter coefficients for each frame to assist in reconstructing the spectral coefficients 28 from the data stream using the TNS coefficients also transmitted within the data stream 24. The possible tasks of the receiver 12 just outlined should be understood as a non-exclusive list of possible measures, and the receiver 12 may perform further or other tasks related to reading the spectral coefficients 28 from the data stream 24.

[0129] Therefore, the grabber 14 receives the spectrogram 26 of spectral coefficients 28 from the receiver 12 and grabs, for each frame 36 , the low frequency components 44 , ie, the N / F lowest frequency spectral coefficients, of the N spectral coefficients of the corresponding frame 36 .

[0130] That is, the spectrum-time modulator 16 receives from the grabber 14 a stream or sequence 46 of N / F spectral coefficients 28 for each frame 36, which are spectrally aligned to the low frequency slices in the spectrogram 26. Figure 3 The lowest spectral coefficient in the image (denoted by index "0") corresponds to the spectral coefficient with index N / F-1.

[0131] The spectrum-time modulator 16 subjects the corresponding low-frequency components 44 of the spectral coefficients 28 to an inverse transformation 48 for each frame 36 of a modulation function having a length of (E+2)·N / F, said modulation function extending in time over the corresponding frame and E+1 previous frames (e.g. Figure 350 in ), thereby obtaining a time portion of length (E+2)·N / F, i.e., a time segment 52 that has not yet been windowed. That is, the spectrum-time modulator can obtain time segments of (E+2)·N / F samples with a reduced sampling rate by weighting the modulation function of the same length using, for example, the first formula of the proposed alternative section A.4 as indicated above and summing them. The most recent N / F samples of the time segment 52 belong to the current frame 36. For example, as indicated, the modulation function can be a cosine function in the case where the inverse transform is an inverse MDCT, or a sine function in the case where the inverse transform is an inverse MDCT.

[0132] Window adder 18 is used for the unimodal synthesis window 54 pairs of time portions 52 of (E+2) N / F that window adder 18 uses length to carry out windowing, and described unimodal synthesis window 54 is included in the zero portion 56 (i.e. 1 / F N / F zero value window coefficient) that the length at its front end is 1 / 4 N / F, and in its time, in the time interval (i.e. time interval not covered by zero portion 52 of time portion 52) after zero portion 56, there is peak value 58. The time interval of back can be referred to as the non-zero part of window 58, and has the length 7 / 4 N / F of the sampling measurement that reduces with sampling rate, i.e. 7 / 4 N / F window coefficient.Window adder 18 for example uses window 58 pairs of time portions 52 weightings. The weighting or multiplication 58 of each temporal portion 52 by the window 54 results in a windowed temporal portion 60 (one for each frame 36) which is identical to the corresponding temporal portion 52 as far as temporal overlap is concerned. In Section A.4 presented above, the windowing process that may be used by the window 18 is described by dividing z by i,n with x i,n The associated formula is used to describe, where x i,n corresponds to the above time portion 52 which has not been windowed, and z i,n corresponds to the windowed time portions 60, where i indexes the sequence of frames / windows and n indexes the samples or values ​​of the respective portion 52 / 60 according to the reduced sampling rate within each time portion 52 / 60.

[0133] Thus, the time-domain aliasing canceller 20 receives a series of windowed time portions 60 from the windower 18, one for each frame 36. The canceller 20 subjects the windowed time portions 60 of the frames 36 to an overlap-add process 62 by aligning each windowed time portion 60 with the N / F value of its front end to coincide with the corresponding frame 36. By this measure, the tail component of the windowed time portion 60 of the current frame (i.e., the remainder of length (E+1)·N / F) of length (E+1) / (E+2) overlaps with the corresponding, equal-length front end of the time portion of the immediately preceding frame. In terms of formulae, the time-domain aliasing canceller 20 may operate as shown in the last formula of the version of Section A.4 presented above, where out i,n corresponds to the audio samples of the reconstructed audio signal 22 at the reduced sampling rate.

[0134] The following references Figure 4 The processes of windowing 58 and overlap-adding 62 performed by the windower 18 and the time-domain aliasing canceller 20 are shown in greater detail. Figure 4 Using the nomenclature applied in Section A.4 above and in Figure 3 and 4 Reference numerals used in the 0,0 to x 0,(E+2)·N / F-1 represents the 0th time portion 52 obtained by the space-time modulator 16 for the 0th frame 36. The first index of x indexes the frame 36 in time order, and the second index of x orders the time samples in time order, with the inter-sample pitch belonging to the reduced sampling rate. Then, in Figure 4 In the middle, w0 to w (E+2)·N / F-1 Indicates the window coefficient of the window 54. Similar to the second index of x, i.e., the time portion 52 output by the modulator 16, when the window 54 is applied to the corresponding time portion 52, the index of w is such that index 0 corresponds to the oldest sample value and (E+2)·N / F-1 corresponds to the newest sample value. The window adder 18 uses the window 54 to window the time portion 52 to obtain a windowed time portion 60, such that according to z 0,0 =x 0,0 w0,…,z 0,(E+2)·N / F-1 =x 0,(E+2)·N / F-1 w (E+2)·N / F-1 To obtain z representing the windowed temporal portion 60 for frame 0 0,0 to z 0,(E+2)·N / F-1The index of z has the same meaning as the index of x. In this way, the modulator 16 and the windower 18 act on each frame indexed by the first index of x and z. The canceller 20 adds the E+2 windowed time portions 60 of E+2 consecutive frames, wherein the samples of the windowed time portions 60 are offset relative to each other by one frame (i.e., the number of samples per frame 36, i.e., N / F), thereby obtaining a sample u of the current frame, here u -(E+1),0 …u -(E+1),N / F-1) Here again, the first index of u represents the frame number and the second index orders the samples of the frame in time order. The canceller connects the reconstructed frames thus obtained so that the samples of the reconstructed audio signal 22 in consecutive frames 36 are sorted according to u. -(E+1),0 …u -(E+1),N / F-1 ,u -E,0 ,…u -E,N / F-1 ,u -(E-1),0 ,… and follow each other. The canceller 20 is based on u -(E+1),0 =z 0,0 +z -1,N / F +…z -(E+1),(E+1)·N / F ,…,u -(E+1)·N / F-1 =z 0,N / F-1 +z -1,2·N / F-1 +…+z -(E+1),(E+2)·N / F-1 (ie, summing (e+2) addends for each sample u of the current frame) is performed, and each sample of the audio signal 22 in the (E+1)th frame is calculated.

[0135] Figure 5 One possible development is shown, namely that among the just windowed samples contributing to the audio sample u of frame (E+1), the zero portion 56 of the window 54 (i.e., z -(E+1),(E+7 / 4)·N / F …z -(E+1),(E+2)·N / F-1 ) corresponds to or the samples windowed using these zero parts 56 are zero values. Therefore, the canceller 20 can be configured according to u -(E+1),(E+7 / 4)·N / F =z 0,3 / 4·N / F +z -1,7 / 4·N / F +…+z -E,(E+3 / 4)·N / F ,…,u -(E+1),(E+2)·N / F-1 =z 0,N / F-1 +z -1,2·N / F-1 +…+z -E,(E+1)·N / F-1 , to calculate the leading quarter of N / F samples in the (E+1)th frame 36 of the audio signal u (i.e., u -(E+1),(E+7 / 4)·N / F …u -(E+1),(E+2)·N / F-1 ), instead of using E+2 addends to obtain all N / F samples in the (E+1)th frame 36 of the audio signal u. In this way, the window adder can even effectively omit the performance of the weight 58 relative to the zero part 56. Therefore, the sample u of the current (E+1)th frame -(E+1),(E+7 / 4)·N / F …u -(E+1),(E+2)·N / F-1can be obtained by using only E+1 addends, and u -(E+1),(E+1)·N / F …u -(E+1),(E+7 / 4)·N / F-1 This will be obtained using E+2 addends.

[0136] Therefore, in the above manner, Figure 2 The audio decoder 10 reproduces the audio signal encoded into the data stream 24 in a reduced manner. To this end, the audio decoder 10 uses a window function 54, which itself is a downsampled version of a reference synthesis window of length (E+2)·N. Figure 6 As explained, the downsampled version (i.e., window 54) is obtained by downsampling the reference synthesis window by a factor F (i.e., the downsampling factor), using segmented interpolation (i.e., segments of length 1 / 4·N measured in the version that has not been downsampled, segments of length 1 / 4·N / F in the downsampled version, segments of one quarter of the frame length of frame 36, measured in time and represented independently of the sampling rate). Thus, interpolation is performed in 4·(E+2), resulting in 4·(E+2) times 1 / 4·N / F long segments, which, when concatenated, represent the downsampled version of the reference synthesis window of length (E+2)·N. Figure 6 Provide explanation. Figure 6 Below the reference synthesis window 70 of length (E+2)·N, the synthesis window 54 is shown, which is unimodal and is used by the audio decoder 10 according to the downsampling audio decoding process. That is, the number of window coefficients is reduced by a factor F through the downsampling process 72 leading from the reference synthesis window 70 to the synthesis window 54 actually used by the audio decoder 10 for downsampling decoding. Figure 6 In, you can apply Figure 5 and Figure 6 , ie, w is used to denote the window coefficients of the downsampled version window 54 , and w′ is used to denote the window coefficients of the reference synthesis window 70 .

[0137] As just mentioned, to perform downsampling 72, the reference synthesis window 70 is processed into segments 74 of equal length. In number, there are (E+2)·4 such segments 74. Measured at the original sampling rate (i.e., the number of window coefficients of the reference synthesis window 70), the length of each segment 74 is 1 / 4·N window coefficients w', while measured at the reduced or downsampled sampling rate, the length of each segment 74 is 1 / 4·N / F window coefficients w.

[0138] Naturally, by simply setting w i =w′ j (where w i The sampling time and w′ j and / or by linear interpolation between two window coefficients w′ in time.j and w′ j+2 Linearly interpolate any window coefficient w between i , can be used for any window coefficient w′ with reference synthesis window 70 j The window coefficient w of each downsample is consistent by chance i Downsampling 72 is performed, but this process results in a poor approximation of the reference synthesis window 70, i.e. the synthesis window 54 used by the audio decoder 10 for downsampling decoding will present a poor approximation to the reference synthesis window 70, and thus will not meet the requirements of the conformance test for guaranteeing downsampling decoding compared to non-downsampling decoding of the audio signal from the data stream 24. Therefore, downsampling 72 involves an interpolation process according to which most of the window coefficients w of the downsampling window 54 are, by the downsampling process 72, i (ie, window coefficients whose positions deviate from the boundaries of the segments 74) depend on more than two window coefficients w' of the reference window 70. Specifically, although most of the window coefficients w of the downsampling window 54 i More than two window coefficients w′ depending on the reference window 70 j , so that for each window coefficient w of the downsampled version 54 i Improves the quality of the interpolation / downsampling result (ie the quality of the approximation), but the fact that the window coefficients do not depend on the window coefficients w′ belonging to different segments 74 j In contrast, the downsampling process 72 is a piecewise interpolation process.

[0139] For example, the synthesis window 54 can be a cascade of spline functions of length 1 / 4·N / F. A cubic spline function can be used. Such an example is outlined above in Section A.1, where an outer for-next (for next) loop sequentially loops around segments 74, where in each segment 74, downsampling or interpolation 72 involves a mathematical combination of consecutive window coefficients w' within the current segment 74, such as in the first for next statement in the "Vector r Required for Calculating Coefficients c" section. However, the interpolation applied to the segments can also be selected in different ways. That is, interpolation is not limited to splines or cubic splines. Instead, linear interpolation or any other interpolation method can also be used. In any case, the segmented implementation of interpolation will result in the calculation of the samples of the reduced synthesis window (i.e., the outermost samples of the segment of the reduced synthesis window that are adjacent to another segment) not depending on the window coefficients of the reference synthesis window located in a different segment.

[0140] It is possible that the window adder 18 stores the window coefficients w of the downsampled synthesis window 54 i The downsampled synthesis window 54 is obtained in memory (which is stored after having been obtained using downsampling 72). Alternatively, as Figure 2As shown, the audio decoder 10 may include performing Figure 6 The downsampling 72 is performed by a segmented downsampler 76 .

[0141] It should be noted that Figure 2 The audio decoder 10 may be configured to support only one fixed downsampling factor F or may support different values. In this case, the audio decoder 10 may respond to Figure 2 78 of . For example, the grabber 14 may be responsive to the value F so as to grab N / F spectral values ​​of each frame spectrum as described above. In a similar manner, the optional piecewise downsampler 76 may also operate in response to the value F as described above. The S / T modulator 16 may be responsive to F so as to, for example, compute a downsampled / down-sampled version of the derived modulation function that is downsampled / down-sampled compared to the version used in the un-downsampled mode of operation (where the reconstruction results in the full audio sampling rate).

[0142] Naturally, the modulator 16 will also be responsive to the F input 78, as the modulator 16 will use a suitably downsampled version of the modulation function and this also applies to the adaptation of the windower 18 and canceller 20 relative to the actual length of the frame in the reduced or downsampled sampling rate.

[0143] For example, F may be between 1.5 and 10, inclusive.

[0144] It should be noted that Figure 2 and Figure 3 The decoder of , or any modification thereof outlined herein, may be implemented such that the spectro-temporal transform is performed using a lifted implementation of the low-delay MDCT, as taught in, for example, EP 2 378 516 B1.

[0145] Figure 8 An implementation of a decoder using the lifting concept is shown. The S / T modulator 16 exemplarily performs an inverse DCT-IV and is shown as being followed by a block representing a cascade of a windower 18 and a time domain aliasing canceller 20. Figure 8 In the example, E is 2, that is, E=2.

[0146] The modulator 16 comprises an inverse type-iv discrete cosine transform frequency / time converter. Instead of outputting a sequence of (E+2)N / F long time portions 52, it outputs only time portions 52 of length 2·N / F, which are all derived from a sequence of N / F long spectra 46. These shortened portions 52 correspond to the 2·N / F most recent samples in the DCT kernel, i.e., the portion described previously.

[0147] Window adder 18 operates as previously described and generates the windowed time portion 60 for each time portion 52, but it only operates on the DCT kernel. For this reason, window adder 18 uses the windowing function ωi with kernel size, where i=0, ..., 2N / F-1. The relationship between it and wi (where i=0, ..., (E+2)·N / F-1) will be described later, as will be mentioned subsequently in relation to the lifting coefficient and w i (where i=0,…,(E+2)·N / F-1)) The relationship is the same.

[0148] Using the nomenclature applied above, the processing described so far produces:

[0149] zk,n=ωn·xk,nwhere n=0,…,2M-1,

[0150] Redefine M = N / F so that M corresponds to the frame size expressed in the reduced domain and use Figures 2 to 6 Nomenclature, however, z k,n and x k,n shall contain only samples of the windowed temporal portion of the DCT kernel of size 2·M and of the temporal portion that has not been windowed and corresponds in time to Figure 4 The samples in E·N / F ··(E+2)·N / F-1 are: That is, n is an integer indicating a sample index, and ωn is a real-number window function coefficient corresponding to the sample index n.

[0151] The overlap / add process of canceller 20 operates in a different manner compared to the above description. It generates the intermediate time portion m based on the following equation or expression: k (0)…m k (M-1):

[0152] m k,n =z k,n +z k-1,n+M Where n=0,…,M-1.

[0153] exist Figure 8 In the implementation of , the apparatus further comprises a lifter 80 which can be interpreted as part of the modulator 16 and the windower 18, since the lifter 80 compensates for the fact that the modulator and the windower restrict their processing to the DCT kernel, rather than processing the extension of the modulation function and the synthesis window beyond the kernel towards the past, which extension is introduced to compensate for the zero portion 56. The lifter 80 uses a framework consisting of delays and multipliers 82 and adders 84 to produce the final reconstructed time portions or frames of length M in the form of pairs of immediately consecutive frames based on the following equation or expression:

[0154] u k,n =mk,n +l n-M / 2 ·m k-1,M-1-n Where n=M / 2,…,M-1,

[0155] as well as

[0156] u k,n =m k,n +l M-1-n out k-1,M-1-n Where n=0,…,M / 2-1,

[0157] where l n (where n=0, ..., M-1) are real-valued lifting coefficients that are associated with the reduced synthesis window in a manner that will be described in more detail below.

[0158] In other words, to extend the overlap to the past E frames, only M additional multiplier addition operations are required, as can be seen in the framework of lifter 80. These additional operations are sometimes referred to as "zero delay matrices." Sometimes these operations are also referred to as "lifting steps." Figure 8 The efficient implementation shown may be more efficient in some cases than a straightforward implementation. More specifically, depending on the specific implementation, this more efficient implementation may result in a saving of M operations, since in principle, in the case of a straightforward implementation for M operations, the proposal would require 2M operations in the framework of module 820 and M operations in the framework of lifter 830.

[0159] As for the synthesis window w i (where i = 0, ..., (E + 2) M - 1, (please recall that E = 2 here)) on ωn (where n = 0, ..., 2M - 1) and l n (where n = 0, ..., M-1), the following formulas describe their relationship to the displacement, however, the subscripts used so far are placed in brackets following the corresponding variables:

[0160]

[0161] w(M / 2+i)=l(n)·l(M / 2+n)·ω(3M / 2+n)

[0162]

[0163] w(3M / 2+i)=-l(n)·ω(3M / 2+n)

[0164] w(2M+i)=-ω(M+n)-l(M-1-n)·ω(n)

[0165] w(5M / 2+i)=-ω(3M / 2+n)-l(M / 2+n)·ω(M / 2+n)

[0166] w(3M+i)=-ω(n)

[0167] w(7M / 2+i)=ω(M+n)

[0168] in,

[0169] Please note that the window w i The right side in this formula (i.e. between indices 2M and 4M-1) includes the peak value. n (n=0,…,M-1) and ω n (n=0,…,2M-1) and the coefficient w of the reduced synthesis window n (n=0,…,(E+2)M-1) is associated. It can be seen that l n (n=0,…,M-1) actually depends only on 3 / 4 of the coefficients of the downsampled synthesis window, that is, on w n (n=0,…,(E+1)M-1), and ωn(n=0,…,2M-1) depends on all wn(n=0,…,(E+2)M-1).

[0170] As mentioned above, it is possible that the window adder 18 obtains the downsampled synthesis window 54w from the memory n (n=0,…,(E+2)M-1), where the window coefficient w of the downsampled synthesis window 54 is i is stored in a memory after being obtained using downsampling 72, and the window coefficients are read from the memory to calculate the coefficients l using the above relationship n (n=0,…,M-1) and ω n (n=0, ..., 2M-1), but alternatively, the windower 18 may retrieve the coefficients l directly from the memory n (n=0, ..., M-1) and ωn(n=0, ..., 2M-1), thereby calculating the pre-downsampled synthesis window. Alternatively, as described above, the audio decoder 10 may include performing Figure 6 The segmented downsampler 76 of the downsampler 72 calculates the coefficient l based on the windower 18 using the above relationship / formula n Even with a boosted implementation, more than one F value can be supported.

[0171] To briefly summarize the improved implementation, the same result in an audio decoder 10 is configured to decode an audio signal 22 transform-encoded into a data stream at a second sampling rate from a data stream 24 at a first sampling rate, the first sampling rate being 1 / F of the second sampling rate, the audio decoder 10 comprising: a receiver 12 receiving N spectral coefficients 28 of length N for each frame of the audio signal; a grabber 14 grabbing a low-frequency component of length N / F from the N spectral coefficients 28 for each frame; a spectrum-time modulator 16 configured to subject the low-frequency component to an inverse transform for each frame 36 to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length 2·N / F extending in time over the corresponding frame and the previous frame; and a windower 18 windowing the time portion x according to zk,n=ωn·xk,n (n=0,…,2M-1) for each frame 36. k,n Windowing is performed to obtain the windowed time portion x k,n (n=0, ..., 2M-1). The time domain aliasing canceller 20 is configured according to m k,n =z k,n +z k-1,n+M (n=0,…,M-1) Generate the intermediate time part m k (0)…m k (M-1). Finally, the lifter 80 is based on u k,n =m k,n +l n-M / 2 ·m k-1,M-1-n (n=M / 2,…,M-1) and u k,n =m k,n +l M-1-n out k-1,M-1-n (n=0,…,M / 2-1) Calculate the frame u of the audio signal k,n (n=0, ..., M-1), wherein the inverse transform is an inverse MDCT or an inverse MDST, and wherein l n (n=0,…,M-1) and ω n (n=0,…,2M-1) depends on the coefficient w of the synthesis window n (n=0,…,(E+2)M-1), and the synthesis window is a downsampled version of a reference synthesis window of length 4·N obtained by downsampling by a factor F and performing segment interpolation according to segments of length 1 / 4·N.

[0172] It has been concluded from the above discussion of the proposal for extension of AAC-ELD regarding the reduced decoding mode that Figure 2The audio decoder can be used together with the low-delay SBR tool. The following is an overview of how an AAC-ELD encoder, for example, extended to support the reduced operating mode proposed above, operates when using the low-delay SBR tool. As already mentioned in the introduction to the specification of this application, when the low-delay SBR tool is used in conjunction with the AAC-ELD encoder, the filter bank of the low-delay SBR module is also reduced. This ensures that the SBR module operates with the same frequency resolution, so no additional adaptation is required. Figure 7 The signal path of an AAC-ELD decoder operating at 96 kHz with a frame size of 480 samples and a reduction factor F of 2 in downsampled SBR mode is summarized.

[0173] exist Figure 7 The bitstream is processed by a series of blocks, namely AAC decoder, inverse LD-MDCT block, CLDFB analysis block, SBR decoder and CLDFB synthesis block (CLDFB = Complex Low Delay Filter Bank). The bitstream is equivalent to the previous reference Figures 3 to 6 The data stream 24 in question is additionally accompanied by parameterized SBR data for assisting in the spectral shaping of the spectral replication of the spectral extension band that extends the spectral frequencies of the audio signal obtained by downscaling audio decoding at the output of the inverse low-delay MDCT block, said spectral shaping being performed by the SBR decoder. In particular, the AAC decoder retrieves all the necessary syntax elements by appropriate parsing and entropy decoding. The AAC decoder can partially overlap with the receiver 12 of the audio decoder 10, in Figure 7 In , the audio decoder 10 is implemented by an inverse low-delay MDCT block. Figure 7 In this example, F is equal to 2. That is, as Figure 2 An example of a reconstructed audio signal 22, Figure 7 The inverse low delay MDCT block outputs a 48 kHz time signal that is downsampled at half the sampling rate at which the audio signal was originally encoded into the arriving bitstream. The CLDFB analysis block subdivides the 48 kHz time signal (i.e. the audio signal obtained by downscaling audio decoding) into N frequency bands (here N=16), and the SBR decoder calculates the reshaping coefficients for these frequency bands, reshapes the N frequency bands accordingly (this is controlled by the SBR data in the input bitstream arriving at the input of the AAC decoder), and the CLDFB synthesis block reconverts from the spectral domain to the time domain, thereby obtaining a high frequency extension signal to be added to the original decoded audio signal output by the inverse low delay MDCT block.

[0174] Note that the standard operation of SBR uses a 32-band CLDFB. The 32-band CLDFB window coefficients ci 32The interpolation algorithm for is given in Section 4.6.19.4.1 of [1],

[0175]

[0176] where c 64 are the window coefficients for the 64-band window given in Table 4.A.90 of [1]. This formula can be further generalized to also define the window coefficients for a smaller number of bands B,

[0177]

[0178] where F denotes the reduction factor F = 32 / B. With this definition of the window coefficients, the CLDFB analysis and synthesis filterbank can be completely described as outlined in the example of Section A.2 above.

[0179] Therefore, the above example provides some missing definitions for the AAC-ELD codec to adapt the codec to systems with lower sampling rates. These definitions can be included in the ISO / IEC 14496-3:2009 standard.

[0180] Therefore, in the above discussion, it has been described that:

[0181] An audio decoder can be configured to decode an audio signal transform-encoded into the data stream at a second sampling rate from a data stream at a first sampling rate, wherein the first sampling rate is 1 / F of the second sampling rate, and the audio decoder comprises: a receiver configured to receive N spectral coefficients of each frame of the audio signal, wherein the length of the frame is N; a grabber configured to grab a low-frequency component of a length of N / F from the N spectral coefficients for each frame; a spectrum-time modulator configured to, for each frame, subject the low-frequency component to an inverse transform to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F extending in time over the corresponding frame and E+1 previous frames; a windower configured to window the time portion using a unimodal synthesis window of length (E2)·N / F for each frame, wherein the unimodal synthesis window comprises The invention relates to a time domain aliasing eliminator configured to perform overlap-addition processing on the windowed time portion of the frame so that the tail component of the windowed time portion of the current frame with a length of (E+1) / (E+2) overlaps with the front end of the windowed time portion of the previous frame with a length of (E+1) / (E+2), wherein the inverse transform is an inverse MDCT or an inverse MDST, and wherein the unimodal synthesis window is a downsampled version of a reference unimodal synthesis window with a length of (E+2)·N that is downsampled by a factor F and segmentally interpolated according to segments with a length of 1 / 4·N / F.

[0182] The audio decoder according to an embodiment wherein the unimodal synthesis window is a concatenation of spline functions of length 1 / 4·N / F.

[0183] The audio decoder according to an embodiment wherein the unimodal synthesis window is a cascade of cubic spline functions of length 1 / 4·N / F.

[0184] The audio decoder according to any preceding embodiment, wherein E=2.

[0185] The audio decoder according to any preceding embodiment, wherein the inverse transform is an inverse MDCT.

[0186] The audio decoder according to any preceding embodiment, wherein more than 80% of the size of the unimodal synthesis window is included in a time interval following the zero portion and having a length of 7 / 4·N / F.

[0187] The audio decoder according to any preceding embodiment, wherein the audio decoder is configured to perform the interpolation or derive the unimodal synthesis window from a memory.

[0188] The audio decoder according to any preceding embodiment, wherein the audio decoder is configured to support different values ​​of F.

[0189] The audio decoder as in any preceding embodiment, wherein F is between 1.5 and 10, inclusive.

[0190] A method performed by an audio decoder according to any one of the preceding embodiments.

[0191] A computer program having a program code for executing the method according to the embodiment when the program code is run on a computer.

[0192] With respect to the term "length", it should be noted that this term is to be interpreted as the length measured in samples. With respect to the length of the zero portion and the segment, it should be noted that the length can be an integer value. Alternatively, the length can be a non-integer value.

[0193] Regarding the time interval of the peak, it should be noted that Figure 1 As an illustration, the peak value and the time interval are shown for an example of a reference unimodal synthesis window with E=2 and N=512: the peak value has a maximum value at approximately sample number 1408, and the time interval extends from sample number 1024 to sample number 1920. The length of the time interval is therefore 7 / 8 of the DCT kernel length.

[0194] Regarding the term "downsampled version", it should be noted that in the above description, "reduced version" may be used synonymously as an alternative to this term.

[0195] Regarding the term "the size of a function within a certain time interval", it should be noted that the size should represent the definite integral of the corresponding function within the corresponding interval.

[0196] In the case that the audio decoder supports different values ​​of F, the audio decoder may include a memory with corresponding piecewise interpolated versions of the reference unimodal synthesis window, or may perform piecewise interpolation on the currently active value of F. The different piecewise interpolated versions have in common that the interpolation does not adversely affect discontinuities at segment boundaries. As mentioned above, these may be spline functions.

[0197] By the above Figure 1The reference unimodal synthesis window shown is started from and segmented interpolated to obtain a unimodal synthesis window, which can be formed into 4·(E+2) segments by spline approximation (e.g., cubic spline), and regardless of whether interpolation is performed or not, due to the zero part introduced by the synthesis as a means of reducing delay, the discontinuity that the unimodal synthesis window should present at the pitch of 1 / 4·N / F is retained.

[0198] The solution of this application can also be expressed by the following supplementary notes.

[0199] 1. An audio decoder (10) configured to decode an audio signal (22) from a data stream (24) at a first sampling rate, the audio signal (22) being transform-encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the audio decoder (10) comprising:

[0200] a receiver (12) configured to receive N spectral coefficients (28) of each frame of the audio signal, wherein the length of the frame is N;

[0201] A grabber (14) configured to grab a low-frequency component of length N / F from the N spectral coefficients (28) for each frame;

[0202] a spectrum-time modulator (16) configured to subject the low-frequency component to an inverse transform for each frame (36) to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F extending in time over the corresponding frame and E+1 previous frames;

[0203] a windower (18) configured to window the time portion for each frame (36) using a synthesis window of length (E+2)·N / F, the synthesis window including a zero portion of length 1 / 4·N / F at its front end and having a peak value within a time interval of the synthesis window, the time interval being after the zero portion and having a length 7 / 4·N / F, so that the windower obtains a windowed time portion of length (E+2)·N / F; and

[0204] A time domain aliasing eliminator (20) is configured to subject the windowed time portions of the frames to an overlap-add process such that a tail component of the windowed time portion of the current frame having a length of (E+1) / (E+2) overlaps a front component of the windowed time portion of the previous frame having a length of (E+1) / (E+2),

[0205] wherein the inverse transform is an inverse MDCT or an inverse MDST, and

[0206] The synthesis window is a downsampled version obtained by downsampling a reference synthesis window of length (E+2)·N by a factor F and performing segmented interpolation according to segments of length 1 / 4·N.

[0207] 2. The audio decoder (10) of embodiment 1, wherein the synthesis window is a concatenation of spline functions of length 1 / 4·N / F.

[0208] 3. The audio decoder (10) according to embodiment 1 or 2, wherein the synthesis window is a cascade of cubic spline functions of length 1 / 4·N / F.

[0209] 4. The audio decoder (10) according to any one of the preceding embodiments, wherein E=2.

[0210] 5. The audio decoder (10) according to any one of the preceding embodiments, wherein the inverse transform is an inverse MDCT.

[0211] 6. The audio decoder (10) according to any one of the preceding embodiments, wherein more than 80% of the size of the synthesis window is included in the time interval after the zero portion and having a length of 7 / 4·N / F.

[0212] 7. The audio decoder (10) according to any one of the preceding embodiments, wherein the audio decoder (10) is configured to perform the interpolation or to derive the synthesis window from a memory.

[0213] 8. The audio decoder (10) according to any one of the preceding embodiments, wherein the audio decoder (10) is configured to support different values ​​of F.

[0214] 9. The audio decoder (10) of any preceding embodiment, wherein F is between 1.5 and 10, inclusive.

[0215] 10. The audio decoder (10) according to any one of the preceding embodiments, wherein the reference synthesis window is unimodal.

[0216] 11. An audio decoder (10) according to any of the preceding embodiments, wherein the audio decoder (10) is configured to perform the interpolation in such a way that a majority of the coefficients of the synthesis window depend on more than two of the coefficients of the reference synthesis window.

[0217] 12. An audio decoder (10) according to any of the preceding embodiments, wherein the audio decoder (10) is configured to perform the interpolation in such a way that each coefficient of the synthesis window that is separated by more than two coefficients from a segment boundary depends on more than two of the coefficients of the reference synthesis window.

[0218] 13. An audio decoder (10) according to any one of the preceding embodiments, wherein the windower (18) and the time domain aliasing eliminator cooperate so that the windower skips the zero portion when weighting the time portion using the synthesis window, and the time domain aliasing eliminator (20) does not consider the corresponding non-weighted portion of the windowed time portion in the overlap-addition process, so that only E+1 windowed time portions are summed, resulting in the corresponding non-weighted portion of the corresponding frame and E+2 windowed portions are summed within the remaining portion of the corresponding frame.

[0219] 14. An audio decoder for generating a reduced version of a synthesis window of an audio decoder (10) according to any one of the preceding embodiments, wherein E=2, so that the synthesis window function comprises a kernel-related half of length 2·N / F, which is preceded by another half of length 2·N / F, and wherein the spectrum-time modulator (16), the windower (18) and the time-domain aliasing canceller (20) are implemented to cooperate in a lifting implementation according to which:

[0220] The spectrum-time modulator (16) will subject the low-frequency components to an inverse transformation for each frame (36) restricted to a transformation kernel consistent with the corresponding frame and one previous frame, thereby obtaining the temporal portion x k,n , where n=0, ..., 2M-1, and M=N / F is a sample index and k is a frame index, wherein the inverse transform has a modulation function of length (E+2)·N / F extending temporally over the corresponding frame and E+1 previous frames;

[0221] The window adder (18) is configured to calculate the window size of each frame (36) according to z k,n =ω n ·x k,n , n=0,…,2M-1, for the time portion x k,n Windowing is performed to obtain the windowed time portion z k,n , n=0,…,2M-1;

[0222] The time domain aliasing canceller (20) is configured according to m k,n =z k,n +z k-1,n+M , n=0,…,M-1, generate the intermediate time part m k (0)…mk (M-1),

[0223] The audio decoder comprises a booster (80) configured to obtain a frame u according to the following equation k,n , where n = 0,…,M-1:

[0224] u k,n =m k,n +l n-M / 2 ·m k-1,M-1-n Where n=M / 2,…,M-1,

[0225] as well as

[0226] u k,n =m k,n +l M-1-n out k-1,M-1-n Where n=0,…,M / 2-1,

[0227] where l n , n=0,…,M-1, is the lifting coefficient, and where l n , n=0,…,M-1 and ω n , n=0,…,2M-1 depends on the coefficient w of the synthesis window n , n=0,…,(E+2)M-1.

[0228] 15. An audio decoder (10) configured to decode an audio signal (22) from a data stream (24) at a first sampling rate, the audio signal (22) being transform-encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the audio decoder (10) comprising:

[0229] a receiver (12) configured to receive N spectral coefficients (28) of each frame of the audio signal, wherein the length of the frame is N;

[0230] A grabber (14) configured to grab a low-frequency component of length N / F from the N spectral coefficients (28) for each frame;

[0231] a spectrum-time modulator (16) configured to subject the low-frequency component to an inverse transform for each frame (36) to obtain a time portion of length 2·N / F, wherein the inverse transform has a modulation function of length 2·N / F extending in time over the corresponding frame and one previous frame;

[0232] The window adder (18) is configured to adjust the window size of each frame (36) according to z k,n =ω n ·x k,n , n=0,…,2M-1, for the time portion xk,n Windowing is performed to obtain the windowed time portion z k,n , n=0,…,2M-1;

[0233] The time domain aliasing canceller (20) is configured to k,n =z k,n +z k-1,n+M , n=0,…,M-1, generate the intermediate time part m k (0)…m k (M-1),

[0234] The enhancer (80) is configured to obtain the frame u of the audio signal according to the following formula k,n , where n = 0,…,M-1:

[0235] u k,n =m k,n +l n-M / 2 ·m k-1,M-1-n Where n=M / 2,…,M-1,

[0236] as well as

[0237] u k,n =m k,n +l M-1-n out k-1,M-1-n Where n=0,…,M / 2-1,

[0238] where l n , n=0,…,M-1, is the lifting coefficient,

[0239] wherein the inverse transform is an inverse MDCT or an inverse MDST, and

[0240] Among them, l n , n=0,…,M-1, and ω n , n=0,…,2M-1, depends on the coefficient w of the synthesis window n , n=0,…,(E+2)M-1, and the synthesis window is a downsampled version obtained by downsampling a reference synthesis window of length 4·N by a factor F and performing segmented interpolation according to segments of length 1 / 4·N.

[0241] 16. A device for generating a reduced version of a synthesis window of an audio decoder (10) according to any one of the preceding embodiments, wherein the device is configured to downsample a reference synthesis window of length (E+2)·N by a factor F and to perform segmented interpolation in 4·(E+2) segments of equal length.

[0242] 17. A method for generating a reduced version of a synthesis window of an audio decoder (10) according to any one of embodiments 1 to 16, wherein the method comprises downsampling a reference synthesis window of length (E+2)·N by a factor F and performing segmented interpolation in 4·(E+2) segments of equal length.

[0243] 18. A method for decoding an audio signal (22) at a first sampling rate from a data stream (24), the audio signal (22) being transform-encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the method comprising:

[0244] receiving N spectral coefficients (28) of each frame of the audio signal, wherein the length of the frame is N;

[0245] For each frame, grab a low-frequency component with a length of N / F from the N spectral coefficients (28);

[0246] performing a spectro-temporal modulation by subjecting the low frequency component to an inverse transform for each frame (36) to obtain a temporal portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F extending in time over the corresponding frame and E+1 previous frames;

[0247] Windowing the time portion using a synthesis window of length (E+2)·N / F for each frame (36), the synthesis window including a zero portion of length 1 / 4·N / F at its front end and having a peak value within a time interval of the synthesis window, the time interval following the zero portion and having a length 7 / 4·N / F, so that the window adder obtains a windowed time portion of length (E+2)·N / F; and

[0248] Time domain aliasing removal is performed by subjecting the windowed time portions of the frames to an overlap-add process such that a tail component of the windowed time portion of the current frame having a length of (E+1) / (E+2) overlaps a leading component of the windowed time portion of the previous frame having a length of (E+1) / (E+2),

[0249] wherein the inverse transform is an inverse MDCT or an inverse MDST, and

[0250] The synthesis window is a downsampled version obtained by downsampling a reference synthesis window of length (E+2)·N by a factor F and performing segmented interpolation according to segments of length 1 / 4·N.

[0251] 19. A computer program having a program code for executing the method according to embodiment 16 or 18 when the program code is run on a computer.

[0252] References

[0253] [1]ISO / IEC 14496-3:2009

[0254] [2]M13958, "Proposal for an Enhanced Low Delay Coding Mode", October 2006, Hangzhou, China.

Claims

1. An audio decoder (10) configured to decode an audio signal (22) from a data stream (24) at a first sampling rate, the audio signal (22) being transform-encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the audio decoder (10) comprising: a receiver (12) configured to receive N spectral coefficients (28) of each frame of the audio signal, wherein the length of the frame is N; A grabber (14) configured to grab a low-frequency component of length N / F from the N spectral coefficients (28) for each frame; a spectrum-time modulator (16) configured to subject the low-frequency component to an inverse transform for each frame (36) to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F extending in time over the corresponding frame and E+1 previous frames; a windower (18) configured to window the time portion using a synthesis window of length (E+2)·N / F for each frame (36), the synthesis window including a zero portion of length 1 / 4·N / F at its front end and having a peak value within a time interval of the synthesis window, the time interval being after the zero portion and having a length 7 / 4·N / F, so that the windower obtains a windowed time portion of length (E+2)·N / F; as well as A time domain aliasing eliminator (20) is configured to subject the windowed time portions of the frames to an overlap-add process such that a tail component of the windowed time portion of the current frame having a length of (E+1) / (E+2) overlaps a front component of the windowed time portion of the previous frame having a length of (E+1) / (E+2), wherein the inverse transform is an inverse MDCT or an inverse MDST, and The synthesis window is a downsampled version of a reference synthesis window of length (E+2)·N obtained by downsampling by a factor F and performing segmented interpolation according to segments of length 1 / 4·N. The synthesis window is a cascade of spline functions with a length of 1 / 4·N / F. More than 80% of the size of the synthesis window is included in the time interval after the zero portion and having a length of 7 / 4·N / F.

2. The audio decoder (10) according to claim 1, wherein The synthesis window is a cascade of cubic spline functions of length 1 / 4·N / F.

3. The audio decoder (10) of claim 1, wherein the inverse transform is an inverse MDCT.

4. The audio decoder (10) according to claim 1, wherein The audio decoder (10) is configured to perform the interpolation or to derive the synthesis window from a memory.

5. The audio decoder (10) according to claim 1, wherein The audio decoder (10) is configured to support different values ​​of F.

6. The audio decoder (10) of claim 1, wherein F is between 1.5 and 10, inclusive.

7. The audio decoder (10) according to claim 1, wherein The reference synthesis window is unimodal.

8. The audio decoder (10) according to claim 1, wherein The audio decoder (10) is configured to perform the interpolation in such a way that a majority of the coefficients of the synthesis window depend on more than two of the coefficients of the reference synthesis window.

9. The audio decoder (10) according to claim 1, wherein The audio decoder (10) is configured to perform the interpolation in such a way that each coefficient of the synthesis window separated by more than two coefficients from a segment boundary depends on more than two of the coefficients of the reference synthesis window.

10. The audio decoder (10) according to claim 1, wherein The windower (18) and the time domain aliasing eliminator cooperate such that the windower skips the zero portion when weighting the time portion using the synthesis window, and the time domain aliasing eliminator (20) does not consider the corresponding non-weighted portion of the windowed time portion in the overlap-add process, so that only E+1 windowed time portions are summed to result in the corresponding non-weighted portion of the corresponding frame, and E+2 windowed portions are summed within the remaining portion of the corresponding frame.

11. A device for generating a reduced version of a synthesis window of an audio decoder (10) according to claim 1, wherein The apparatus is configured to downsample a reference synthesis window of length (E+2)·N by a factor F and to perform segment-wise interpolation in 4·(E+2) segments of equal length.

12. A method for generating a reduced version of a synthesis window of an audio decoder (10) according to claim 1, wherein The method comprises downsampling a reference synthesis window of length (E+2)·N by a factor F and performing a segment-wise interpolation in 4·(E+2) segments of equal length.

13. A method for decoding an audio signal (22) from a data stream (24) at a first sampling rate, the audio signal (22) being transform-encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the method comprising: Receiving N spectral coefficients (28) of each frame of the audio signal, wherein the length of the frame is N; For each frame, grab a low-frequency component with a length of N / F from the N spectral coefficients (28); Spectro-temporal modulation is performed by subjecting the low frequency component to an inverse transform for each frame (36) to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F extending in time over the corresponding frame and E+1 previous frames; Windowing the time portion using a synthesis window of length (E+2)·N / F for each frame (36), the synthesis window including a zero portion of length 1 / 4·N / F at its front end and having a peak value within a time interval of the synthesis window, the time interval being after the zero portion and having a length 7 / 4·N / F, so as to obtain a windowed time portion of length (E+2)·N / F; as well as Time domain aliasing removal is performed by subjecting the windowed time portions of the frames to an overlap-add process such that a tail component of the windowed time portion of the current frame having a length of (E+1) / (E+2) overlaps a leading component of the windowed time portion of the previous frame having a length of (E+1) / (E+2), wherein the inverse transform is an inverse MDCT or an inverse MDST, and The synthesis window is a downsampled version of a reference synthesis window of length (E+2)·N obtained by downsampling by a factor F and performing segmented interpolation according to segments of length 1 / 4·N. The synthesis window is a cascade of spline functions with a length of 1 / 4·N / F.

14. A computer-readable storage medium having a computer program stored thereon, the computer program having a program code for executing the method according to claim 12 or 13 when the computer program is run on the computer.

Images