Method and audio decoder for reduced decoding

Abstract

The present application provides a method and an audio decoder for downmixing decoding. If the synthesis window used for downmixing audio decoding is a downsampled version of a reference synthesis window involved in a non-downmixing audio decoding process, wherein the downsampled version is downsampled by a downsample factor representing a deviation of a downsampled sampling rate from an original sampling rate and obtained using a piecewise interpolation segmented in quarters of a frame length, a downmixing version of the audio decoding process can be more efficiently and / or with improved compatibility maintained implemented.

Description

[0001] This application is a divisional application of PCT international application PCT / EP2016 / 063371, filed on June 10, 2016, which entered the Chinese national phase and is entitled “Reduced Code”. Technical Field

[0002] This application relates to a downscaled decoding concept. Background Technology

[0003] MPEG-4 Enhanced Low Latency AAC (AAC-ELD) typically operates at sampling rates up to 48kHz, resulting in an algorithm latency of 15ms. For some applications, such as lip-sync audio transmission, lower latency is desired. AAC-ELD has offered this option by operating at higher sampling rates (e.g., 96kHz), thus providing an operating mode with lower latency (e.g., 7.5ms). However, this operating mode introduces unnecessarily high complexity due to the high sampling rate.

[0004] The solution to this problem is to apply a reduced version of the filter bank, thereby rendering the audio signal at a lower sampling rate (e.g., 48kHz instead of 96kHz). This reduction operation is already part of AAC-ELD, as it is inherited from the MPEG-4 AAC-LD codec, which forms the basis of AAC-ELD.

[0005] However, the remaining problem is how to find a reduced version of a specific filter bank. In other words, the only uncertainty lies in how to derive the window coefficients while supporting a clear conformance test of the reduced operating mode of the AAC-ELD decoder.

[0006] The principle of the reduction operation mode of the AAC-(E)LD codec is described below.

[0007] Section 4.6.17.2.7, “Adaptation to systems employing lower sampling rates,” of ISO / IEC 14496-3:2009 describes the reduced operating mode or AAC-LD for AAC-LD as follows:

[0008] "In some applications, it may be necessary to integrate a low-latency decoder into an audio system that operates at a lower sampling rate (e.g., 16 kHz) but has a much higher nominal sampling rate for the bitstream payload (e.g., 48 kHz, corresponding to an algorithm codec latency of approximately 20 milliseconds). In such cases, it is advantageous to decode the output of the low-latency codec directly at the target sampling rate, rather than using an additional sample rate conversion operation after decoding."

[0009] This can be approximated by appropriately reducing both the frame size and sampling rate by an integer factor (e.g., 2, 3), thus achieving the same time-frequency resolution for the codec. For example, by retaining only the lowest third of the spectral coefficients before the synthesis filter bank (i.e., 480 / 3 = 160) and reducing the inverse transform size to one-third (i.e., a window size of 960 / 3 = 320), a codec output can be generated at a sampling rate of 16 kHz instead of the nominal 48 kHz.

[0010] Therefore, decoding at a lower sampling rate reduces the requirements for memory and computation, but may not produce the exact same output as that obtained by full-bandwidth decoding followed by band limiting and sampling rate conversion.

[0011] Please note that, as mentioned above, decoding at a lower sampling rate does not affect the interpretation of the rating, which refers to the nominal sampling rate of the AAC low-latency bitstream payload.

[0012] Please note that AAC-LD uses the standard MDCT framework and two window shapes: a sinusoidal window and a low-overlap window. Both windows are fully formulaic, allowing window coefficients to be determined for arbitrary transform lengths.

[0013] Compared to AAC-LD, the AAC-ELD codec exhibits two main differences:

[0014] ●Low-delay MDCT window (LD-MDCT)

[0015] ●Possibility of utilizing low-latency SBR tools

[0016] The IMDCT algorithm using a low-latency MDCT window is described in 4.6.20.2 of [1], which is very similar to the standard IMDCT version using, for example, a sinusoidal window. The coefficients of the low-latency MDCT window (frame sizes of 480 and 512 samples) are given in Tables 4.A.15 and 4.A.16 of [1]. Note that since the coefficients are the result of an optimization algorithm, they cannot be determined by formulas. Figure 9 A diagram showing the shape of a window with a frame size of 512 is provided.

[0017] When the low-latency SBR (LD-SBR) tool is used in conjunction with the AAC-ELD encoder, the filter bank of the LD-SBR module is also reduced. This ensures that the SBR module operates at the same frequency resolution, thus eliminating the need for additional adaptations.

[0018] Therefore, the above description reveals the need for reduction in decoding operations, such as reduction of decoding at AAC-ELD. Re-finding the coefficients of the reduced synthesis window function is feasible, but this is a tedious task, requiring additional storage space to store the reduced version, and is more complex in checking consistency between the unreduced and reduced decoding, or, from another perspective, does not conform to the reduction method required by AAC-ELD. Depending on the reduction ratio, i.e., the ratio between the original sampling rate and the reduced sampling rate, the reduced synthesis window function can be simply derived by downsampling (i.e., extracting one from every two, three... window coefficients of the original synthesis window function), but this process does not correspondingly produce sufficient consistency between the unreduced and reduced decoding. Using a more complex extraction process applied to the synthesis window function results in unacceptable deviations from the shape of the original synthesis window function. Therefore, an improved reduction decoding concept is needed in the art. Summary of the Invention

[0019] Therefore, one object of the present invention is to provide an audio decoding scheme that implements such improved reduction decoding.

[0020] This objective is achieved through the subject matter of the independent claims.

[0021] This invention is based on the discovery that if the synthesis window used for reducing audio decoding is a downsampled version of the reference synthesis window involved in the non-reduced audio decoding process, the reduced version of the audio decoding process can be implemented more efficiently and / or with improved compatibility maintenance, wherein the downsampled version is obtained by downsampling according to a downsampling factor and using segmented interpolation with segments of 1 / 4 of the frame length, and the downsampling factor represents the deviation of the downsampled sampling rate from the original sampling rate. Attached Figure Description

[0022] The advantageous aspects of this application are the subject matter of the dependent claims. Preferred embodiments of this application are described below with reference to the accompanying drawings, in which:

[0023] Figure 1 This is a schematic diagram illustrating the perfect reconstruction requirements that must be followed during reduction decoding in order to maintain perfect reconstruction;

[0024] Figure 2 A block diagram of an audio decoder for reduction decoding according to an embodiment is shown;

[0025] Figure 3 The following diagram illustrates how the audio signal has been encoded into the data stream at its original sampling rate in the upper half, and how the downsampling decoding operation, separated from the upper half by a horizontal dashed line, reconstructs the audio signal from the data stream at a reduced or downsampled sampling rate, for illustrative purposes. Figure 2 The operating mode of the audio decoder;

[0026] Figure 4 It shows Figure 2 A schematic diagram of the collaboration between the windower and the temporal aliasing eliminater;

[0027] Figure 5 This demonstrates the use of special processing on the zero-weighted portion of the time part after spectral-time modulation to achieve the desired result. Figure 4 The possibility of reconstruction;

[0028] Figure 6 A schematic diagram of downsampling for obtaining a synthesis window is shown;

[0029] Figure 7 A block diagram of the AAC-ELD reduction operation, including the low-latency SBR tool, is shown.

[0030] Figure 8 A block diagram of an audio decoder for reduction decoding according to an embodiment is shown, wherein the modulator, windower, and canceller are implemented according to a boosting implementation; and

[0031] Figure 9 A graph showing the window coefficients for a frame size of 512 samples, as an example of a reference synthesis window to be downsampled, according to the low-latency window of AAC-ELD. Detailed Implementation

[0032] The following description begins with an illustrative description of an embodiment of reduced decoding with respect to the AAC-ELD codec. That is, the following description begins with an embodiment that can form a reduced mode of AAC-ELD. This description also forms an explanation of the motivation for the embodiments of this application. Thereafter, the description is summarized, thereby providing a description of an audio decoder and audio decoding method according to embodiments of this application.

[0033] As described in the preamble of this application, AAC-ELD uses a low-latency MDCT window. To generate its reduced version, the reduced low-latency window, the proposed method for forming the reduced pattern of AAC-ELD, explained subsequently, uses a piecewise spline interpolation algorithm that preserves the perfect reconstruction property (PR) of the LD-MDCT window with very high accuracy. Thus, this algorithm allows for the generation of window coefficients in a compatible manner, in the direct form as described in ISO / IEC 14496-3:2009 and the elevated form as described in [2]. This means that both implementations will generate 16-bit compliant outputs.

[0034] Interpolation of the low-latency MDCT window is performed as follows.

[0035] Generally, spline interpolation will be used to generate reduced window coefficients to preserve the frequency response and most of the perfect reconstruction properties (approximately 170 dB SNR). Interpolation needs to be constrained within certain segments to maintain perfect reconstruction properties. For the window coefficients c of the DCT kernel of the overlay transform (see also...),... Figure 1 Given c(1024), ..., c(2048), the following constraints are required:

[0036] 1=|(sgn·c(i)·c(2N-1-i)+c(N+i)·c(N-1-i))|,

[0037] Where i = 0, ..., N / 2-1(1)

[0038] Where N represents the frame size. Some implementations can use different notations to optimize complexity; here, sgn is used. The requirement in (1) can be... Figure 1 To illustrate, it should be remembered that even with F=2 (i.e., half the sampling rate), omitting one of every two window coefficients in the reference synthesis window is insufficient to obtain a reduced synthesis window.

[0039] The coefficients c(0)...c(2N-1) are listed along a diamond shape. Bold arrows are used to mark the N / 4 zeros in the window coefficients, which are responsible for reducing the delay of the filter bank. Figure 1 The dependencies between coefficients caused by the folding involved in MDCT are shown, and the points where interpolation needs to be constrained to avoid any unwanted dependencies are also shown.

[0040] For every N / 2 coefficients, interpolation needs to stop to maintain (1).

[0041] Furthermore, due to the inserted zeros, the interpolation algorithm needs to stop every N / 4 zeros. This ensures that the zeros are preserved and the interpolation error does not propagate, thus maintaining PR.

[0042] The second constraint is necessary not only for segments containing zeros but also for all other segments. Knowing that some coefficients in the DCT kernel are not determined by optimization algorithms but by equation (1) to achieve PR, it can be explained that... Figure 1 There are several discontinuities in the window shape around c(1536+128). To minimize the PR error, interpolation needs to be stopped at these points that appear in the N / 4 grid.

[0043] For this reason, a piecewise size of N / 4 is chosen for piecewise spline interpolation to generate the reduced window coefficients. The source window coefficients are always given by coefficients used for N=512, which are also used for the reduction operation resulting in frame sizes of N=240 or N=120. The basic algorithm is briefly summarized in MATLAB code below:

[0044]

[0045] Since spline functions may not be completely deterministic, the complete algorithm is described in detail in the following sections. This algorithm can be included in ISO / IEC 14496-3:2009 to form an improved reduction pattern in AAC-ELD.

[0046] In other words, the following section provides a proposal on how to apply the above ideas to ERAAC ELD, specifically how a low-complexity decoder can decode an ERAAC ELD bitstream encoded at the first data rate at a second data rate lower than the first data rate. However, it should be emphasized that the definition of N used below conforms to the standard. Here, N corresponds to the length of the DCT kernel, whereas in the above, in the claims, and in the generalized embodiment described below, N corresponds to the frame length, that is, the overlap length of the DCT kernels, i.e., half the length of the DCT kernels. Thus, for example, where N is indicated as 512 above, it is indicated as 1024 below.

[0047] It is proposed that the following paragraph be incorporated into 14496-3:2009 by amendment.

[0048] A.0 Adaptation to systems using lower color rates

[0049] For some applications, ERAAC LD can change the broadcast sampling rate to avoid additional resampling steps (see 4.6.17.2.7). ERAAC ELD can apply similar reduction steps using a low-latency MDCT window and the LD-SBR tool. When AAC-ELD operates with the LD-SBR tool, the reduction factor is limited to a multiple of 2. Without LD-SBR, the reduced frame size must be an integer.

[0050] A.1 Reduction of the low-latency MDCT window

[0051] LD-MDCT window w with N=1024 LD The window size is determined by reducing the window size using piecewise spline interpolation with a factor F. The number of leading zeros in the window coefficient (i.e., N / 8) determines the piecewise size. The reduced window coefficient w LD_d It was used for inverse MDCT (as described in 4.6.20.2), but the reduced window length N d=N / F.

[0052] Please note that this algorithm can also generate the reduction boost factor for LD-MDCT.

[0053]

[0054]

[0055]

[0056] A.2 Reduction of Low-Latency SBR Tools

[0057] When the low-latency SBR tool is used in conjunction with ELD, the tool can be reduced to a lower sampling rate, at least for a reduction factor that is a multiple of 2. The reduction factor F controls the number of frequency bands used for CLDFB analysis and synthesis filter banks. The following two paragraphs describe the reduced CLDFB analysis and synthesis filter banks; see also 4.6.19.4.

[0058] 4.6.20.5.2.1 Reduction Analysis of CLDFB Filter

[0059] ● Define the number of reduced CLDFB bands as B = 32 / F.

[0060] ● Shift the samples in array x by B positions. Discard the oldest B samples and store the B new samples in positions 0 to B-1.

[0061] ● Samples of array x are multiplied by the window coefficient ci to obtain array z. The window coefficient ci is obtained by linear interpolation of the coefficient c, i.e., by the following equation.

[0062]

[0063] The window coefficient c can be found in Table 4.A.90.

[0064] ● Sum the samples to create a 2B-element array u:

[0065] u(n)=z(n)+z(n+2B)+z(n+4B)+z(n+6B)+z(n+8B), 0≤n<(2B).

[0066] ● The B new sub-band samples are calculated using matrix operations Mu, where

[0067]

[0068] In the equation, exp() represents a complex exponential function, and j is the imaginary unit.

[0069] 4.6.20.5.2.2 Reduction Analysis of CLDFB Filter Bank ● Define the number of reduced CLDFB bands as B = 64 / F.

[0070] ● Shift the samples in array v by 2B positions. Discard the oldest 2B samples.

[0071] ● Multiply the B new complex subband samples by matrix N, where

[0072]

[0073] In the equation, exp() represents the complex exponential function, and j is the imaginary unit. The real part of the output from this operation is stored in positions 0 to 2B-1 of the array v.

[0074] ● Extract samples from v to create a 10B-element array g.

[0075]

[0076] ● Samples of array g are multiplied by the window coefficient ci to produce array w. The window coefficient ci is obtained by linear interpolation of the coefficient c, i.e., by the following equation.

[0077]

[0078] The window coefficient c can be found in Table 4.A.90.

[0079] ●The following formula is used to calculate B new output samples by summing the samples from array w:

[0080]

[0081] Please note that setting F=2 provides a downsampled synthesis filter bank according to 4.6.19.4.3. Therefore, in order to process the downsampled LD-SBR bitstream with an additional reduction factor F, F needs to be multiplied by 2.

[0082] 4.6.20.5.2.3 Downsampled Real-Value CLDFB Filter Bank

[0083] The downsampling of CLDFB can also be used for the real-valued version of the low-power SBR mode. For illustrative purposes, please also consider 4.6.19.5.

[0084] For the reduced real-valued analysis and synthesis filter bank, as described in 4.6.20.5.2.1 and 4.6.20.2.2, the exp() modulator in M ​​is replaced by the cos() modulator.

[0085] A.3 Low-latency MDCT analysis

[0086] This section describes the low-latency MDCT filter bank used in the AAC ELD encoder. The core MDCT algorithm remains largely unchanged, but the window is longer, causing n to now run from -N to N-1 (instead of from 0 to N-1).

[0087] Spectral coefficient X i，k The definition is as follows:

[0088] Where 0≤k <N / 2

[0089] in:

[0090] zi n =Windowed input sequence

[0091] N = Sampling Index

[0092] K = Frequency coefficient index

[0093] I = block index

[0094] N = Window length

[0095] n0 = (-N / 2 + 1) / 2

[0096] The window length N (based on a sine window) is 1024 or 960.

[0097] The window length for the low-latency window is 2×N. Windowing extends to the past in the following way:

[0098] z i，n =w LD (N-1-n)·x′ i，n

[0099] For n = -N, ..., N-1, the synthesis window w is used as the analysis window by reversing the order.

[0100] A.4 Low-delay MDCT synthesis

[0101] Compared to the standard IMDCT algorithm which uses a sinusoidal window, the synthetic filter bank is modified to employ a low-delay filter bank. The core IMDCT algorithm remains largely unchanged, but the window is longer, causing n to now run as high as 2N-1 (instead of N-1).

[0102] Where 0 ≤ n < 2N

[0103] in:

[0104] n = sampling index

[0105] i = window index

[0106] k = Spectral coefficient index

[0107] N = twice the window length / frame length

[0108] n0 = (-N / 2 + 1) / 2

[0109] Where N = 960 or 1024.

[0110] Adding windows and overlapping elements is done as follows:

[0111] A window of length N is replaced by a window of length 2N, which overlaps more with the past and less with the future (N / 8 values ​​are actually zero).

[0112] Add a window to the low-latency window:

[0113] z i，n =w LD (n)·x i，n

[0114] The window is now 2N in length, therefore n = 0, ..., 2N-1.

[0115] Overlap and add:

[0116]

[0117] Where 0 <= n <N / 2

[0118] Here, it is proposed that these paragraphs be incorporated into 14496-3:2009 by amendment.

[0119] Of course, the above description of possible reduction modes for AAC-ELD represents only one embodiment of this application, and some modifications are possible. Generally, embodiments of this application are not limited to a reduced version of the audio decoder performing AAC-ELD decoding. In other words, embodiments of this application can be obtained, for example, by forming an audio decoder capable of performing inverse transform processing only in a reduced manner, without needing to support or use various AAC-ELD-specific further tasks, such as scaling factor-based transmission of the spectral envelope, TNS (Time Noise Shaping) filtering, spectral band copying (SBR), etc.

[0120] The following describes a more general embodiment for the audio decoder. The above example of an AAC-ELD audio decoder supporting the reduction mode can therefore represent an implementation of the audio decoder described subsequently. Specifically, the decoder explained below... Figure 2 As shown in the middle, Figure 3 Shown by Figure 2 The steps performed by the decoder.

[0121] Figure 2 The audio decoder (generally indicated by reference numeral 10) includes a receiver 12, a grabber 14, a spectrum-time modulator 16, a windower 18, and a time-domain aliasing canceller 20, all of which are connected in series in the order mentioned. See below for reference. Figure 3 The interaction and functionality of blocks 12 to 20 of the audio decoder 10 are described. As described at the end of this application, blocks 12 to 20 may be implemented in software, programmable hardware or hardware (e.g., in the form of a computer program, FPGA, or a suitably programmed computer), a programmable microprocessor, or an application-specific integrated circuit (where blocks 12 to 20 represent corresponding subroutines, circuit paths, etc.).

[0122] The following is a more detailed overview. Figure 2 The audio decoder 10 is configured (and the elements of the audio decoder 10 are configured to cooperate appropriately) to decode the audio signal 22 from the data stream 24. It should be noted that the sampling rate used by the audio decoder 10 to decode the signal 22 is 1 / F of the sampling rate used when the audio signal 22 was transformed and encoded into the data stream 24 on the encoding side. For example, F can be any rational number greater than 1. The audio decoder can be configured to operate with different or variable reduction factors F or with a fixed reduction factor F. Alternatives are described in more detail below.

[0123] exist Figure 3 The upper half shows how the audio signal 22 is transformed and encoded into the data stream at the coded or original sampling rate. At 26, Figure 3 The spectral coefficients are shown using small boxes or squares 28 arranged in a spectral-time manner along the time axis 30 and the frequency axis 32, respectively, where the time axis 30 is... Figure 3 Mid-horizontal extension, frequency axis 32 in Figure 3 Vertical extension in the middle. Spectral coefficient 28 is transmitted within data stream 24. Figure 3 Point 34 in the diagram illustrates how the spectral coefficients 28 have been obtained, and thus how the spectral coefficients 28 represent the audio signal 22. Figure 3 Point 34 in the diagram, for a portion of the time axis 30, illustrates how to obtain the spectral coefficients 28 belonging to or representing the corresponding time portion from the audio signal.

[0124] Specifically, the coefficients 28 transmitted within data stream 24 are the coefficients of the overlap transform of audio signal 22, such that audio signal 22 sampled at the original or coded sampling rate is divided into frames of a predetermined length N that are temporally contiguous and non-overlapping, wherein N spectral coefficients are transmitted in data stream 24 for each frame 36. That is, the transform coefficients 28 are obtained from audio signal 22 using a critically sampled overlap transform. In the spectrum-time spectrum representation 26, each column of the time sequence in the columns of spectral coefficients 28 corresponds to a corresponding frame in frame 36 of the frame sequence. For the corresponding frame 36, N spectral coefficients 28 are obtained through spectrum decomposition transform or time-spectral modulation, whereby the modulation function of the spectrum decomposition transform or time-spectral modulation extends not only temporally over the frame 36 to which the resulting spectral coefficients 28 belong, but also across E+1 previous frames, where E can be any integer greater than zero or any even integer. That is, by applying a transformation to a transformation window, the spectral coefficients 28 belonging to a column of a certain frame 36 in the spectrum located at 26 are obtained. The transformation window includes not only the corresponding frame but also E+1 frames past the current frame. A low-latency single-peak analysis window function 40 is used to perform spectral decomposition of the audio signal samples within this transformation window 38 (which... Figure 3 The table below shows the transform coefficients 28 for the intermediate frame 36 (shown at 34), where the spectral samples within the transform window 38 are weighted before undergoing MDCT, MDST, or other spectral decomposition transforms using the low-latency single-peak analysis window function 40. To reduce encoder-side latency, the analysis window 40 includes a zero-interval 42 at its temporal front, so that the encoder does not need to wait for the corresponding portion of the latest sample within the current frame 36 to calculate the spectral coefficients 28 for that current frame 36. That is, within the zero-interval 42, the low-latency window function 40 is zero or has a zero window coefficient, such that the co-bit audio samples of the current frame 36 do not contribute to the transform coefficients 28 and data stream 24 transmitted for that frame due to window weighting 40. In other words, to summarize the above, the transform coefficients 28 belonging to the current frame 36 are obtained by windowing and spectral decomposition of the audio signal sampled within the transform window 38, which includes the current frame and the temporally earlier frame, and the transform window 38 overlaps temporally with the corresponding transform window used to determine the spectral coefficients 28 belonging to the temporally adjacent frame.

[0125] Before resuming the description of the audio decoder 10, it should be noted that the description of the transmission of spectral coefficients 28 within data stream 24 provided so far has been simplified relative to the manner in which spectral coefficients 28 are quantized or encoded into data stream 24 and / or the manner in which audio signal 22 is preprocessed before the audio signal undergoes an overlap transform. For example, the audio encoder that transforms and encodes audio signal 22 into data stream 24 can be controlled via a psychoacoustic model, or a psychoacoustic model can be used to determine a scaling factor for the spectral bands used to scale the quantized and transmitted spectral coefficients 28, keeping the quantization noise and the quantization of spectral coefficients 28 imperceptible to the listener and / or below a masking threshold function. The scaling factor will also be signaled in data stream 24. Alternatively, the audio encoder may be a TCX (Transform Coded Excitation) type encoder. The audio signal will then be subjected to linear predictive analysis filtering before forming the spectral-time representation 26 of spectral coefficients 28 by applying an overlap transform to the excitation signal (i.e., the linear prediction residual signal). For example, linear prediction coefficients can also be signaled in data stream 24, and spectral uniform quantization can be applied to obtain spectral coefficients 28.

[0126] Furthermore, the descriptions presented so far have been simplified relative to the frame length of frame 36 and / or relative to the low-latency window function 40. In reality, the audio signal 22 may have already been encoded into the data stream 24 using varying frame sizes and / or different windows 40. However, the following description focuses on a window 40 and a frame length, although the subsequent description can be readily extended to cases where the entropy encoder changes these parameters during the encoding of the audio signal into the data stream.

[0127] Return to Figure 2 The audio decoder 10 and its description, the receiver 12 receives the data stream 24 and thereby receives N spectral coefficients 28 for each frame 36, i.e. Figure 3 The corresponding column for coefficient 28 is shown. It should be remembered that the time length of frame 36 measured in the original or encoded sampling rate is N, as... Figure 3 As shown in 34, but Figure 2The audio decoder 10 is configured to decode the audio signal 22 at a reduced sampling rate. The audio decoder 10, for example, only supports the downsampling decoding function described below. Alternatively, the audio decoder 10 will be able to reconstruct the audio signal at the original or encoded sampling rate, but can switch between a downsampling decoding mode and a non-downsampling decoding mode, wherein the downsampling decoding mode is consistent with the operating mode of the audio decoder 10 as described below. For example, in situations such as low battery power, reduced reproduction environment capability, etc., the audio decoder 10 can switch to the downsampling decoding mode. Whenever the situation changes, the audio decoder 10 can, for example, switch back from the downsampling decoding mode to the non-downsampling decoding mode. In any case, according to the downsampling decoding processing of the decoder 10 as described below, the audio signal 22 is reconstructed at a sampling rate at which, at the reduced sampling rate, the frame 36 has a shorter length measured at the reduced sampling rate, i.e., the sample length is N / F at the reduced sampling rate.

[0128] The output of receiver 12 is a sequence of N spectral coefficients for each frame of 36, that is, a set consisting of N spectral coefficients. Figure 3 One of the columns in the [data stream 24]. From the above brief description of the transform coding process used to form data stream 24, it has become clear that receiver 12 can apply various tasks when obtaining N spectral coefficients for each frame 36. For example, receiver 12 can use entropy decoding to read spectral coefficients 28 from data stream 24. Receiver 12 can also perform spectral shaping on the spectral coefficients read from the data stream using scaling factors provided in the data stream and / or scaling factors derived from linear prediction coefficients transmitted within data stream 24. For example, receiver 12 can obtain scaling factors from data stream 24 (i.e., on a per-frame and per-subband basis) and use these scaling factors to scale the scaling factors transmitted within data stream 24. Alternatively, receiver 12 can derive scaling factors from linear prediction coefficients transmitted within data stream 24 for each frame 36 and use these scaling factors to scale the transmitted spectral coefficients 28. Optionally, receiver 12 can perform gap padding to synthetically fill the zero-quantized portion within the set of N spectral coefficients 18 per frame. Alternatively or concurrently, receiver 12 may apply a TNS synthesis filter to the transmitted TNS filter coefficients for each frame to assist in reconstructing the spectral coefficients 28 from the data stream using the TNS coefficients also transmitted within data stream 24. The possible tasks of receiver 12 just outlined should be understood as a non-exclusive list of possible measures, and receiver 12 may perform further or other tasks related to reading the spectral coefficients 28 from data stream 24.

[0129] Therefore, the grabber 14 receives the spectrum diagram 26 of the spectrum coefficients 28 from the receiver 12, and grabs the low frequency components 44, i.e., N / F lowest frequency spectrum coefficients, from the N spectrum coefficients of each frame 36 for each frame 36.

[0130] In other words, the spectrum-time modulator 16 receives a stream or sequence 46 of N / F spectral coefficients 28 for each frame 36 from the capturer 14, the stream or sequence 46 of the N / F spectral coefficients 28 being matched with a low-frequency slice in the spectrogram 26 (which is spectrally registered to...). Figure 3 The lowest spectral coefficient (represented by index "0") corresponds to the spectral coefficients at index "N / F-1" and extends to the spectral coefficients at index "N / F-1".

[0131] For each frame 36, the spectrum-time modulator 16 performs an inverse transform 48 on the corresponding low-frequency component 44 of the spectrum coefficients 28 through a modulation function of length (E+2)·N / F, which is temporally extended over the corresponding frame and E+1 previous frames (e.g., ...). Figure 3 As shown in 50), a time segment of length (E+2)·N / F is obtained, namely, the unwindowed time segment 52. In other words, the spectrum-time modulator can obtain a time segment of (E+2)·N / F samples with a reduced sampling rate by weighting and summing the modulation functions of the same length using, for example, the first formula in the proposed alternative section A.4 as indicated above. The latest N / F samples of time segment 52 belong to the current frame 36. For example, as indicated, the modulation function can be a cosine function if the inverse transform is inverse MDCT, or a sine function if the inverse transform is inverse MDCT.

[0132] Therefore, windower 52 receives a time portion 52 for each frame, where the first N / F samples of the time portion 52 correspond temporally to the corresponding frame, and the remaining samples of the corresponding time portion 52 belong to the corresponding earlier frame. For each frame 36, windower 18 uses a single-peak synthesis window 54 of length (E+2)·N / F to window the time portion 52. The single-peak synthesis window 54 includes a zero portion 56 of length 1 / 4·N / F at its beginning (i.e., 1 / F·N / F zero-value window coefficients) and has a peak 58 temporally within the time interval following the zero portion 56 (i.e., the time interval of the time portion 52 not covered by the zero portion 52). The subsequent time interval can be referred to as the non-zero portion of window 58 and has a length of 7 / 4·N / F, i.e., 7 / 4·N / F window coefficients, measured at a reduced sampling rate. Windower 18 uses window 58, for example, to weight the time portion 52. The windowed time portion 60 (one per frame 36) is obtained by weighting or multiplying each time portion 52 using window 54 (or multiplying by itself 58), and is consistent with the corresponding time portion 52, provided that temporal coverage is taken into account. In the above-mentioned section A.4, the windowing process that can be used by window 18 is determined by... i，n With x i，n The related formula describes this, where x i，nCorresponding to the aforementioned time portion 52 before windowing, and z i，n Corresponding to the windowed time portion 60, where i indexes the sequence of frames / windows, and n indexes the samples or values ​​of the corresponding portion 52 / 60 according to the reduced sampling rate within each time portion 52 / 60.

[0133] Therefore, the temporal aliasing canceller 20 receives a series of windowed time portions 60 from the windower 18, one for each frame 36. The canceller 20 performs an overlap-addition process 62 on the windowed time portions 60 of frame 36 by registering each windowed time portion 60 with its leading N / F value to match the corresponding frame 36. Through this measure, the trailing component of the current frame's windowed time portion 60, with a length of (E+1) / (E+2) (i.e., the remaining portion with a length of (E+1)·N / F), overlaps with the corresponding equally long leading edge of the time portion of the immediately preceding frame. In terms of formulation, the temporal aliasing canceller 20 can operate as shown in the last formula of the version of section A.4 above, where out i，n This corresponds to the audio sampling of the audio signal 22 reconstructed at a reduced sampling rate.

[0134] The following is for reference Figure 4 The windowing 58 and overlap addition 62 processes performed by windower 18 and temporal aliasing canceller 20 are shown in more detail. Figure 4 Using the naming conventions applied in Section A.4 above and in Figure 3 and 4 The reference numerals used in the accompanying drawings. 0，0 To x 0，(E+2)·N / F-1 This represents the 0th time portion 52 obtained by the spatial-temporal modulator 16 for frame 36. The first index of x indexes frame 36 in temporal order, and the second index of x sorts the temporal samples in temporal order, with the inter-sample pitch belonging to a reduced sampling rate. Then, in... Figure 4 In the middle, w0 to w (E+2)·N / F-1 The window coefficient of indicator window 54. Similar to the second index of x, i.e., the time portion 52 output by modulator 16, when window 54 is applied to the corresponding time portion 52, the index of w is such that index 0 corresponds to the oldest sample value, and (E+2)·N / F-1 corresponds to the newest sample value. Windower 18 uses window 54 to window the time portion 52 to obtain the windowed time portion 60, such that according to z 0，0 =x 0.0 ·w0,...,z 0，(E+2)·N / F-1 =x 0，(E+2)·N / F-1 ·w (E+2)·N / F-1 To obtain the z-axis representing the time portion of the windowing for frame 0, 60. 0，0 To z 0，(E+2)·N / F-1The index of z has the same meaning as the index of x. In this way, modulator 16 and windower 18 operate on each frame indexed by the first indices of x and z. Canceller 20 adds the E+2 windowed time portions 60 of E+2 consecutive frames, where the samples of each windowed time portion 60 are offset from each other by one frame (i.e., the number of samples of 36 per frame, i.e., N / F), to obtain a sample u of the current frame, here u -(E+1)，0 ...u -(E+1)，N / F-1) Here, the first index of u again represents the frame number, and the second index sorts the samples of that frame chronologically. The canceller connects the reconstructed frames thus obtained, such that the samples of the reconstructed audio signal 22 within consecutive frames 36 are ordered according to u. -(E+1)，0 ...u -(E+1)，N / F-1 u -E，0 ,...u -E，N / F-1 u -(E-1)，0 ...and they follow each other. Eliminator 20 according to u -(E+1)，0 =z 0，0 +z -1，N / F +...z -(E+1)，(E+1)·N / F , ..., u -(E+1)·N / F-1 =z 0，N / F-1 +z -1，2·N / F-1 +...+z -(E+1)，(E+2)·N / F-1 (That is, summing (e+2) addends for each sample u of the current frame), calculate each sample of the audio signal 22 in the (E+1)th frame.

[0135] Figure 5 One possible development is shown, namely, in the sampling of the window that contributes to the audio sample u of frame (E+1), the zero part 56 of window 54 (i.e., z) is compared with that of window 54. -(E+1)，(E+7 / 4)·N / F ...z -(E+1)，(E+2)·N / F-1 The corresponding or windowed samples using these zero parts 56 are zero values. Therefore, the canceller 20 can be based on u -(E+1)，(E+7 / 4)·N / F =z 0，3 / 4·N / F +z -1，7 / 4·N / F +...+z -E，(E+3 / 4)·N / F , ..., u -(E+1)，(E+2)·N / F-1 =z 0，N / F-1 +z -1，2·N / F-1 +...+z -E，(E+1)·N / F-1 To calculate the first quarter of the N / F samples within the (E+1)th frame 36 of the audio signal u using only E+1 addends (i.e., u) -(E+1)，(E+7 / 4)·N / F ...u -(E+1)，(E+2)·N / F-1 Instead of using E+2 addends to obtain all N / F samples within the (E+1)th frame 36 of the audio signal u, the windower can effectively omit the performance of weight 58 relative to the zero part 56. Therefore, the sample u of the current (E+1)th frame... -(E+1)，(E+7 / 4)·N / F...u -(E+1)，(E+2)·N / F-1 This can be obtained by using only E+1 addends, while u -(E+1)，(E+1)·N / F ...u -(E+1)，(E+7 / 4)·N / F-1 It will be obtained using E+2 addends.

[0136] Therefore, in the above manner, Figure 2 The audio decoder 10 reproduces the audio signal encoded into the data stream 24 in a reduced manner. For this purpose, the audio decoder 10 uses a window function 54, which is itself a downsampled version of a reference synthesis window of length (E+2)·N. (See reference...) Figure 6 The downsampled version (i.e., window 54) is obtained by downsampling the reference synthesis window by a factor F (i.e., the downsampling factor) using segmented interpolation (i.e., segmenting by a length of 1 / 4·N measured in the un-downsampled version, segmenting by a length of 1 / 4·N / F in the downsampled version, and segmenting by a quarter of the frame length of frame 36, which is measured temporally and represented independently of the sampling rate). Therefore, interpolation is performed in 4·(E+2), resulting in segments of length 4·(E+2) multiplied by 1 / 4·N / F, which are concatenated to represent the downsampled version of the reference synthesis window of length (E+2)·N. (Refer to...) Figure 6 Please provide an explanation. Figure 6 Below the reference synthesis window 70 of length (E+2)·N, a synthesis window 54 is shown. This synthesis window 54 is unimodal and is used by the audio decoder 10 according to the downsampled audio decoding process. That is, through the downsampling process 72 leading from the reference synthesis window 70 to the synthesis window 54 actually used for downsampling decoding by the audio decoder 10, the number of window coefficients is reduced by a factor F. Figure 6 In, it can be applied Figure 5 and Figure 6 The naming convention is as follows: w is used to represent the downsampled version window 54, while w′ is used to represent the window coefficient of the reference synthesis window 70.

[0137] As just mentioned, in order to perform downsampling 72, the reference synthesis window 70 is processed into segments 74 of equal length. There are (E+2)·4 such segments 74. Measured by the original sampling rate (i.e., the number of window coefficients in the reference synthesis window 70), the length of each segment 74 is 1 / 4·N window coefficients w′, while measured by the reduced or downsampled sampling rate, the length of each segment 74 is 1 / 4·N / F window coefficients w.

[0138] Naturally, by simply setting w i =w j (where w) i Sampling time and w j(Sampling times are consistent), and / or located temporally at two window coefficients w via linear interpolation. j and w j+2 Linear interpolation of arbitrary window coefficients w between the positions i It can target any window coefficient w relative to the reference synthesis window 70. j The window coefficient w of each downsampling is consistent by chance i Downsampling 72 is performed, but this process causes an approximation of the difference in the reference synthesis window 70. That is, the synthesis window 54 used by the audio decoder 10 for downsampling decoding will exhibit an approximation of the difference in the reference synthesis window 70, thus failing to meet the requirement of ensuring consistency between reduced decoding and non-reduced decoding of the audio signal from the data stream 24. Therefore, downsampling 72 involves an interpolation process, according to which most of the window coefficients w of the downsampling window 54 are adjusted through downsampling process 72. i (i.e., the window coefficients that deviate from the boundary of segment 74) depend on more than two window coefficients w' of the reference window 70. Specifically, although most of the window coefficients w' of the downsampling window 54 i More than two window coefficients w′ depending on reference window 70 j So that for each window coefficient w of downsampling version 54 i This improves the quality (i.e., the approximate quality) of the interpolation / downsampling results, but the fact is that the window coefficients do not depend on the window coefficients w belonging to different segments 74. j Conversely, downsampling process 72 is a piecewise interpolation process.

[0139] For example, the synthesis window 54 can be a cascade of spline functions of length 1 / 4·N / F. Cubic spline functions can be used. An example of this is outlined above in Section A.1, where an outer for-next loop sequentially loops around segment 74, where in each segment 74, downsampling or interpolation 72 involves a mathematical combination of the continuous window coefficients w′ within the current segment 74, e.g., the first for-next statement in the “Vector r Required to Calculate Coefficient c” section. However, the interpolation applied to the segments can also be chosen in a different way. That is, interpolation is not limited to splines or cubic splines. Instead, linear interpolation or any other interpolation method can be used. In any case, the segmented implementation of interpolation will result in the computation of the samples of the reduced synthesis window (i.e., the outermost samples of the segments of the reduced synthesis window adjacent to another segment) not depending on the window coefficients of the reference synthesis window located in different segments.

[0140] It is possible that windower 18 stores the window coefficients w of the downsampled synthesis window 54. i The downsampled synthesis window 54 is obtained in memory (after it has been obtained using downsampled 72 and stored). Alternatively, such as Figure 2 As shown, the audio decoder 10 may include functions based on the reference synthesis window 70. Figure 6 The segmented downsampler 76 of the downsampling 72.

[0141] It should be noted that, Figure 2 The audio decoder 10 can be configured to support only a fixed downsampling factor F or to support different values. In this case, the audio decoder 10 can respond to... Figure 2 The input value for F is shown at point 78. For example, the grabber 14 can respond to this value F to grab N / F spectral values ​​for each frame spectrum as described above. In a similar manner, the optional segmented downsampler 76 can also operate in response to this value F as described above. The S / T modulator 16 can respond to F to, for example, compute a downsampled / downsampled version of the derived modulation function, which is downsampled / downsampled compared to the version used in the undownsampled operating mode (where reconstruction results in the full audio sample rate).

[0142] Naturally, modulator 16 will also respond to input F 78, as modulator 16 will use an appropriate downsampled version of the modulation function, and it is also suitable for windower 18 and canceller 20 to adapt to the actual length of the frame in the reduced or downsampled sampling rate.

[0143] For example, F can be between 1.5 and 10 (inclusive).

[0144] It should be noted that, Figure 2 and Figure 3 The decoder or any of its modifications outlined herein can be implemented to enable the use of a boosted implementation of low-latency MDCT to perform spectral-time transformation, as taught, for example, in EP2378516B1.

[0145] Figure 8 An implementation of a decoder using a boosting concept is shown. The S / T modulator 16 exemplarily performs inverse DCT-IV and is shown as a block followed by a cascaded representation of a windower 18 and a time-domain aliasing canceller 20. Figure 8 In the example, E is 2, that is, E = 2.

[0146] Modulator 16 includes an inverse-type-IV discrete cosine transform frequency / time converter. Instead of outputting a sequence of (E+2)N / F long time portions 52, it outputs only time portions 52 of length 2·N / F, all derived from a sequence of N / F long spectrum 46. These shortened portions 52 correspond to the DCT kernel, i.e., the 2·N / F most recent samples in the previously described portion.

[0147] Windower 18 operates as previously described and generates a windowed time portion 60 for each time portion 52, but it only operates on the DCT kernel. For this purpose, windower 18 uses a windowing function ω with a kernel size. i , where i = 0, ..., 2N / F⁻¹. Its relationship with w i The relationship between (where i = 0, ..., (E+2)·N / F-1) will be described later, as will be mentioned subsequently regarding the lifting factor and w. i (where i = 0, ..., (E+2)·N / F-1) has the same relationship.

[0148] Using the naming conventions applied above, the processes described so far produce:

[0149] z k，n =ω n ·x k，n Where n = 0, ..., 2M-1,

[0150] Redefine M = N / F such that M corresponds to the frame size represented in the reduction domain, and use... Figures 2 to 6 The naming convention, however, includes z k，n and x k，n It should only include the windowed temporal portion and the unwindowed temporal portion of the DCT kernel with a size of 2M, and correspond temporally to Figure 4 The sample size is E·N / F...(E+2)·N / F-1. That is, n is an integer indicating the sample index, and ω... n These are the real window function coefficients corresponding to the sampling index n.

[0151] Compared to the description above, the overlap / addition processing of canceller 20 operates in a different manner. It generates the intermediate time portion m based on the following equation or expression. k (0)...m k (M-1):

[0152] m k，n =z k，n +z k-1，n+M Where n = 0, ..., M-1.

[0153] exist Figure 8In its implementation, the device also includes a lifter 80, which can be interpreted as part of the modulator 16 and the windower 18, because the lifter 80 compensates for the fact that the modulator and windower limit their processing to the DCT kernel, rather than processing the modulation function and the synthesis window beyond the kernel's past extension range, which is to compensate for the zero part 56 introduced. The lifter 80 uses a framework consisting of a delay unit and a multiplier 82, and an adder 84, to produce a final reconstructed temporal portion or frame of length M in the form of frame pairs consisting of immediately following consecutive frames, based on the following equation or expression:

[0154] u k，n =m k，n +l n-M / 2 ·m k-1，M-1-n Where n = M / 2, ..., M-1,

[0155] as well as

[0156] u k，n =m k，n +l M-1-n ·out k-1，M-1-n Where n = 0, ..., M / 2-1,

[0157] Among them l n (where n = 0, ..., M-1) are real-valued boosting factors associated with reducing the synthesis window, as will be described in more detail below.

[0158] In other words, for the extended overlap to the past E frames, only M additional multiplier additions are needed, as can be seen in the frame of lifter 80. These additional operations are sometimes referred to as “zero-delay matrices”. Sometimes these operations are also referred to as “lifting steps”. Figure 8 The efficient implementation shown may be more efficient as a direct implementation in some cases. More specifically, depending on the specific implementation, this more efficient implementation may save M operations, since in the case of a direct implementation for M operations, it is recommended in principle that 2M operations are required in the framework of module 820 and M operations are required in the framework of lifter 830.

[0159] As for the composition window w i (where i = 0, ..., (E+2)M-1, (recall that E = 2 here)) on ω n (where n = 0, ..., 2M-1) and l n The dependence of (where n = 0, ..., M-1) on the displacement is described in the following formula; however, the subscripts used so far are placed in parentheses following the corresponding variables:

[0160]

[0161] w(M / 2+i)=l(n)·l(M / 2+n)·ω(3M / 2+n)

[0162]

[0163] w(3M / 2+i)=-l(n)·ω(3M / 2+n)

[0164] w(2M+i)=-ω(M+n)-l(M-1-n)·ω(n)

[0165] w(5M / 2+i)=-ω(3M / 2+n)-l(M / 2+n)·ω(M / 2+n)

[0166] w(3M+i)=-ω(n)

[0167] w(7M / 2+i)=ω(M+n)

[0168] in,

[0169] Please note that window w i The right side of this formula (i.e., between indices 2M and 4M-1) includes the peak value. The above formula includes the coefficient l. n (n = 0, ..., M-1) and ω n (n = 0, ..., 2M-1) and the coefficient w of the reduced synthesis window n (n = 0, ..., (E+2)M-1) are associated. It can be seen that l n (n = 0, ..., M-1) actually depends only on 3 / 4 of the coefficients of the downsampling synthesis window, i.e., on w n (n = 0, ..., (E+1)M-1), and ω n (n = 0, ..., 2M-1) depends on all w n (n = 0, ..., (E+2)M-1).

[0170] As described above, it is possible that the windower 18 obtains the downsampled synthesis window 54w from the memory. n (n = 0, ..., (E+2)M-1), where the window coefficient w of the synthesis window 54 of the downsampling is... i The window coefficient is obtained using downsampling 72 and stored in memory, and is read from the memory to calculate coefficient l using the above relationship. n (n = 0, ..., M-1) and ω n(n = 0, ..., 2M-1), but alternatively, windower 18 can directly retrieve coefficients ln(n = 0, ..., M-1) and ω from the memory. n (n = 0, ..., 2M-1), thereby calculating the pre-sampling synthesis window. Alternatively, as described above, the audio decoder 10 may include execution based on the reference synthesis window 70. Figure 6 The segmented downsampler 76 of the downsampler 72 calculates the coefficient l based on the above relation / formula using the windower 18. n (n = 0, ..., M-1) and ω n (n = 0, ..., 2M-1) and obtain w n (n = 0, ..., (E+2)M-1). Even with a promotion implementation, more than one F value can be supported.

[0171] In brief, to summarize the implementation improvement, the same result in the audio decoder 10 is configured as follows: decoding the audio signal 22 encoded into the data stream 24 at a first sampling rate of 1 / F, where the first sampling rate is 1 / F of the second sampling rate. The audio decoder 10 includes: a receiver 12 that receives N spectral coefficients 28 of length N for each frame of the audio signal; a grabber 14 that grabs a low-frequency component of length N / F from the N spectral coefficients 28 for each frame; a spectrum-time modulator 16 configured to, for each frame 36, perform an inverse transform on the low-frequency component to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length 2·N / F that extends temporally over the corresponding frame and the previous frame; and a windower 18 that, for each frame 36, according to z k，n =ω n ·x k，n (n = 0, ..., 2M-1) for the time component x k，n Windowing is applied to obtain the time component x of the windowed process. k，n (n = 0, ..., 2M-1). The time-domain aliasing canceller 20 is based on m... k，n =z k，n +z k-1，n+M (n = 0, ..., M-1) Generate intermediate time components m k (0)...m k (M-1). Finally, the lifter 80 according to u k，n =m k，n +l n-M / 2 ·m k-1，M-1-n (n = M / 2, ..., M-1) and u k，n =m k，n +l M-1-n ·out k-1，M-1-n(n = 0, ..., M / 2-1) Calculate the frame u of the audio signal. k，n (n = 0, ..., M-1), wherein the inverse transform is inverse MDCT or inverse MDST, and where l n (n = 0, ..., M-1) and ω n (n = 0, ..., 2M-1) depends on the coefficient w of the synthesis window. n (n = 0, ..., (E+2)M-1), and the synthesis window is a downsampled version obtained by downsampling a reference synthesis window of length 4·N according to a factor F and interpolating it piecewise according to segments of length 1 / 4·N.

[0172] From the above discussion of the proposals for extending AAC-ELD with reduced decoding modes, it has been concluded that... Figure 2 The audio decoder can be used with the low-latency SBR tool. The following outlines how an AAC-ELD encoder, for example, extended to support the reduced operating mode proposed above, operates when using the low-latency SBR tool. As already mentioned in the introductory section of this application, when the low-latency SBR tool is used in conjunction with the AAC-ELD encoder, the filter bank of the low-latency SBR module is also reduced. This ensures that the SBR module operates at the same frequency resolution, thus eliminating the need for additional adaptation. Figure 7 The signal path of the AAC-ELD decoder operating at 96kHz is summarized, with a frame size of 480 samples and a reduction factor F of 2 in the downsampled SBR mode.

[0173] exist Figure 7 The bitstream arriving is processed by a series of blocks (i.e., AAC decoder, inverse LD-MDCT block, CLDFB analysis block, SBR decoder, and CLDFB synthesis block (CLDFB = Complex Low-Latency Filter Bank)). The bitstream is equivalent to the previously referenced... Figures 3 to 6 The discussed data stream 24, but additionally includes parameterized SBR data, which assists in spectral shaping of the spectral copy of the spectral extension band. This extended spectral extension band extends the spectral frequencies of the audio signal obtained by audio decoding at the output of the inverse low-latency MDCT block. This spectral shaping is performed by the SBR decoder. Specifically, the AAC decoder retrieves all necessary syntax elements through appropriate parsing and entropy decoding. The AAC decoder may overlap with the receiver 12 portion of the audio decoder 10. Figure 7 In this implementation, the audio decoder 10 is implemented using an inverse low-latency MDCT block. Figure 7 In this context, F is exemplarily equal to 2. That is, as... Figure 2 An example of the reconstructed audio signal 22, Figure 7The inverse low-latency MDCT block outputs a 48kHz time signal, which is downsampled at half the sampling rate used when the audio signal was initially encoded into the arriving bitstream. The CLDFB analysis block subdivides this 48kHz time signal (i.e., the audio signal obtained by downsampling audio decoding) into N frequency bands (here, N = 16), and the SBR decoder calculates the reshaping coefficients for these frequency bands, reshaping the N frequency bands accordingly (this is controlled by the SBR data in the input bitstream arriving at the AAC decoder's input). The CLDFB synthesis block then reconverts from the spectral domain back to the time domain, thereby obtaining a high-frequency extended signal to be added to the original decoded audio signal output by the inverse low-latency MDCT block.

[0174] Please note that SBR standard operation uses 32-band CLDFB. The 32-band CLDFB window coefficient ci 32 The interpolation algorithm is given in Section 4.6.19.4.1 of [1].

[0175]

[0176] Where c 64 The window coefficients for the 64-band window are given in Table 4.A.90 of [1]. This formula can be further generalized to define window coefficients for a smaller number of bands B.

[0177]

[0178] Where F represents the reduction factor F = 32 / B. Using this definition of the window coefficient, CLDFB analysis and synthesis filter banks can be fully described, as outlined in the example in Section A.2 above.

[0179] Therefore, the example above provides some missing definitions for the AAC-ELD codec to adapt it to systems with lower sampling rates. These definitions can be included in the ISO / IEC 14496-3:2009 standard.

[0180] Therefore, as described in the discussion above:

[0181] An audio decoder is configured to decode an audio signal encoded into the data stream at a second sampling rate at a first sampling rate of 1 / F, the audio decoder comprising: a receiver configured to receive N spectral coefficients for each frame of the audio signal, wherein the frame length is N; a grabber configured to grab a low-frequency component of length N / F from the N spectral coefficients for each frame; a spectrum-time modulator configured to, for each frame, inversely transform the low-frequency component to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F extending temporally over the corresponding frame and E+1 previous frames; and a windower configured to window the time portion for each frame using a single-peak synthesis window of length (E2)·N / F, the single-peak synthesis window containing... The windower includes a zero portion of length 1 / 4·N / F at its front end and has a peak value within a time interval of the single-peak synthesis window, the time interval being after the zero portion and having a length of 7 / 4·N / F, such that the windower obtains a windowed time portion of length (E+2)·N / F; and a temporal aliasing canceller is configured to perform an overlap-addition process on the windowed time portion of the frame such that the tail component of the windowed time portion of the current frame of length (E+1) / (E+2) overlaps with the front end of the windowed time portion of the previous frame of length (E+1) / (E+2), wherein the inverse transform is inverse MDCT or inverse MDST, and wherein the single-peak synthesis window is a downsampled version obtained by downsampling a reference single-peak synthesis window of length (E+2)·N by a factor F and by segmented interpolation according to segments of length 1 / 4·N / F.

[0182] According to the audio decoder of the embodiment, the single-peak synthesis window is a cascade of spline functions of length 1 / 4·N / F.

[0183] According to the audio decoder of the embodiment, the single-peak synthesis window is a cascade of cubic spline functions of length 1 / 4·N / F.

[0184] The audio decoder according to any one of the foregoing embodiments, wherein E = 2.

[0185] The audio decoder according to any one of the foregoing embodiments, wherein the inverse transform is inverse MDCT.

[0186] According to any one of the preceding embodiments, in the audio decoder, more than 80% of the size of the single-peak synthesis window is included after the zero portion and within a time interval of length 7 / 4·N / F.

[0187] The audio decoder according to any one of the foregoing embodiments is configured to perform the interpolation or derive the single-peak synthesis window from memory.

[0188] The audio decoder according to any one of the foregoing embodiments is configured to support different values ​​of F.

[0189] The audio decoder according to any one of the foregoing embodiments, wherein F is between 1.5 and 10, and includes 1.5 and 10.

[0190] A method performed by an audio decoder according to any one of the foregoing embodiments.

[0191] A computer program having program code for performing the method described according to an embodiment when run on a computer.

[0192] Regarding the term "length," it should be noted that it is interpreted as length measured by sampling. Regarding the length of zero parts and segments, it should be noted that the length can be an integer value. Alternatively, the length can be a non-integer value.

[0193] Regarding the time interval at which the peak occurs, attention should be paid to Figure 1 As an illustration, an example of a reference single-peak synthesis window with E=2 and N=512 shows the peak value and the time interval: the peak value has a maximum at approximately sample number 1408, and the time interval extends from sample number 1024 to sample number 1920. Therefore, the length of the time interval is 7 / 8 of the DCT kernel length.

[0194] Regarding the term "downsampled version," it should be noted that in the instructions above, "reduced version" can be used synonymously as an alternative to this term.

[0195] Regarding the term "the size of a function over a given time interval," it should be noted that this size should represent the definite integral of the corresponding function over the given interval.

[0196] If the audio decoder supports different values ​​of F, the audio decoder may include a memory for a corresponding piecewise interpolation version with a reference single-peak synthesis window, or it may perform piecewise interpolation on the currently active value of F. What the different piecewise interpolation versions have in common is that the interpolation does not adversely affect discontinuities at the piecewise boundaries. As mentioned above, they can be spline functions.

[0197] From the above Figure 1Starting from the reference single-peak synthesis window shown, a single-peak synthesis window is obtained by piecewise interpolation. It can be formed into 4·(E+2) segments by spline approximation (e.g., cubic splines). Regardless of whether interpolation is used, the discontinuity that the single-peak synthesis window should present at a pitch of 1 / 4·N / F is preserved due to the zero part introduced by synthesis as a means of reducing delay.

[0198] The scheme of this application can also be expressed by the following supplementary notes.

[0199] 1. An audio decoder (10) configured to decode an audio signal (22) from a data stream (24) at a first sampling rate, the audio signal (22) being transformed and encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the audio decoder (10) comprising:

[0200] The receiver (12) is configured to receive N spectral coefficients (28) of each frame of the audio signal, wherein the length of the frame is N;

[0201] The grabber (14) is configured to grab a low-frequency component of length N / F from the N spectral coefficients (28) for each frame;

[0202] A spectrum-time modulator (16) is configured to, for each frame (36), inversely transform the low-frequency component to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F that extends temporally over the corresponding frame and E+1 previous frames.

[0203] A windower (18) is configured to window the temporal portion for each frame (36) using a composition window of length (E+2)·N / F, the composition window including a zero portion of length 1 / 4·N / F at its beginning and having a peak within a time interval after the zero portion and having a length of 7 / 4·N / F, such that the windower obtains a windowed temporal portion of length (E+2)·N / F; and

[0204] The temporal aliasing canceller (20) is configured to perform an overlap-addition process on the windowed temporal portions of the frame, such that the tail component of the windowed temporal portion of the current frame with a length of (E+1) / (E+2) overlaps with the front component of the windowed temporal portion of the previous frame with a length of (E+1) / (E+2).

[0205] The inverse transform mentioned above is inverse MDCT or inverse MDST, and

[0206] The synthesis window is a downsampled version obtained by downsampling a reference synthesis window of length (E+2)·N according to a factor F and interpolating it piecewise according to segments of length 1 / 4·N.

[0207] 2. The audio decoder (10) according to Embodiment 1, wherein the synthesis window is a cascade of spline functions of length 1 / 4·N / F.

[0208] 3. The audio decoder (10) according to embodiment 1 or 2, wherein the synthesis window is a cascade of cubic spline functions of length 1 / 4·N / F.

[0209] 4. The audio decoder (10) according to any one of the foregoing embodiments, wherein E = 2.

[0210] 5. The audio decoder (10) according to any one of the foregoing embodiments, wherein the inverse transform is inverse MDCT.

[0211] 6. The audio decoder (10) according to any of the foregoing embodiments, wherein more than 80% of the size of the synthesis window is included after the zero portion and within the time interval of length 7 / 4·N / F.

[0212] 7. The audio decoder (10) according to any of the foregoing embodiments, wherein the audio decoder (10) is configured to perform the interpolation or export the synthesis window from memory.

[0213] 8. The audio decoder (10) according to any one of the foregoing embodiments, wherein the audio decoder (10) is configured to support different values ​​of F.

[0214] 9. The audio decoder (10) according to any one of the foregoing embodiments, wherein F is between 1.5 and 10, and includes 1.5 and 10.

[0215] 10. The audio decoder (10) according to any one of the foregoing embodiments, wherein the reference synthesis window is single-peaked.

[0216] 11. The audio decoder (10) according to any of the foregoing embodiments, wherein the audio decoder (10) is configured to perform the interpolation in such a way that most of the coefficients of the synthesis window depend on more than two of the coefficients of the reference synthesis window.

[0217] 12. The audio decoder (10) according to any of the foregoing embodiments, wherein the audio decoder (10) is configured to perform the interpolation in such a way that each coefficient of the synthesis window, which is separated by more than two coefficients from the segment boundary, depends on more than two coefficients of the reference synthesis window.

[0218] 13. The audio decoder (10) according to any one of the preceding embodiments, wherein the windower (18) and the temporal aliasing canceller cooperate such that the windower skips the zero portion when weighting the temporal portion using the synthesis window, and the temporal aliasing canceller (20) does not consider the corresponding unweighted portion of the windowed temporal portion in the overlap addition process, so only E+1 windowed temporal portions are summed, thereby causing the corresponding unweighted portion of the corresponding frame and E+2 windowed portions to be summed in the remainder of the corresponding frame.

[0219] 14. A scaled-down version of an audio decoder for generating a synthesis window of an audio decoder (10) according to any of the preceding embodiments, wherein E = 2, such that the synthesis window function includes a kernel-dependent half of length 2·N / F, which is preceded by another half of length 2·N / F, and wherein the spectral-time modulator (16), the windower (18), and the temporal aliasing canceller (20) are implemented to cooperate in a lifting implementation, according to which:

[0220] The spectrum-time modulator (16) will, for each frame (36), inversely transform the low-frequency components to a transform kernel consistent with the corresponding frame and a previous frame, thereby obtaining the time component x. k，n , where n = 0, ..., 2M-1, and M = N / F is the sampling index, k is the frame index, and the inverse transform has a modulation function that extends in time over the corresponding frame and E+1 previous frames with a length of (E+2)·N / F;

[0221] The windower (18) applies z to each frame (36) according to z k，n =ω n ·x k，n n = 0, ..., 2M-1, for the time portion x k，n Windowing is applied to obtain the windowed time component z. k，n n = 0, ..., 2M-1;

[0222] The temporal aliasing canceller (20) is based on m k，n =z k，n +z k-1，n+M n = 0, ..., M-1, generating intermediate time components m k(0)...m k (M-1),

[0223] The audio decoder includes a booster (80) configured to obtain frame u according to the following formula. k，n Where n = 0, ..., M-1:

[0224] u k，n =m k，n +l n-M / 2 ·m k-1，M-1-n Where n = M / 2, ..., M-1,

[0225] as well as

[0226] u k，n =m k，n +l M-1-n ·out k-1，M-1-n Where n = 0, ..., M / 2-1,

[0227] Among them l n , n = 0, ..., M-1, are lifting coefficients, and where l n n = 0, ..., M-1 and ω n The coefficients w of the synthesis window, n = 0, ..., 2M-1, depend on the coefficients w of the synthesis window. n , n=0,...,(E+2)M-1。

[0228] 15. An audio decoder (10) configured to decode an audio signal (22) from a data stream (24) at a first sampling rate, the audio signal (22) being transformed and encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the audio decoder (10) comprising:

[0229] The receiver (12) is configured to receive N spectral coefficients (28) of each frame of the audio signal, wherein the length of the frame is N;

[0230] The grabber (14) is configured to grab a low-frequency component of length N / F from the N spectral coefficients (28) for each frame;

[0231] A spectrum-time modulator (16) is configured to, for each frame (36), inversely transform the low-frequency component to obtain a time portion of length 2·N / F, wherein the inverse transform has a modulation function of length 2·N / F that extends temporally over the corresponding frame and a previous frame.

[0232] The windower (18) is configured to apply a window to each frame (36) based on z. k，n =ω n ·xk，n n = 0, ..., 2M-1, for the time portion x k，n Windowing is applied to obtain the windowed time component z. k，n n = 0, ..., 2M-1;

[0233] The temporal aliasing canceller (20) is configured to, according to m k，n =z k，n +z k-1，n+M n = 0, ..., M-1, generating intermediate time components m k (0)...m k (M-1),

[0234] The booster (80) is configured to obtain the frame u of the audio signal according to the following formula. k，n Where n = 0, ..., M-1:

[0235] u k，n =m k，n +l n-M / 2 ·m k-1，M-1-n Where n = M / 2, ..., M-1,

[0236] as well as

[0237] u k，n =m k，n +l M-1-n ·out k-1，M-1-n Where n = 0, ..., M / 2-1,

[0238] Among them l n n = 0, ..., M-1, are the lifting coefficients.

[0239] Wherein, the inverse transform is inverse MDCT or inverse MDST, and

[0240] Among them, l n n = 0, ..., M-1, and ω n n = 0, ..., 2M-1, depending on the coefficient w of the synthesis window. n , n=0,...,(E+2)M-1,and the synthesis window is a downsampled version obtained by downsampling a reference synthesis window of length 4·N according to a factor F and by segmenting and interpolating according to segments of length 1 / 4·N.

[0241] 16. An apparatus for generating a reduced version of a synthesis window of an audio decoder (10) according to any of the preceding embodiments, wherein the apparatus is configured to downsample a reference synthesis window of length (E+2)·N by a factor F and to perform segmented interpolation in segments of equal length 4·(E+2).

[0242] 17. A method for generating a reduced version of the synthesis window of an audio decoder (10) according to any one of embodiments 1 to 16, wherein the method comprises downsampling a reference synthesis window of length (E+2)·N by a factor F and performing segmented interpolation in segments of equal length 4·(E+2).

[0243] 18. A method for decoding an audio signal (22) from a data stream (24) at a first sampling rate, the audio signal (22) being transformed and encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the method comprising:

[0244] Receive N spectral coefficients (28) for each frame of the audio signal, where the frame length is N;

[0245] For each frame, extract a low-frequency component of length N / F from the N spectral coefficients (28);

[0246] Spectrum-time modulation is performed by inversely transforming the low-frequency component for each frame (36) to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F that extends temporally over the corresponding frame and E+1 previous frames.

[0247] For each frame (36), a composition window of length (E+2)·N / F is used to window the time portion, the composition window including a zero portion of length 1 / 4·N / F at its beginning and having a peak within a time interval after the zero portion and having a length of 7 / 4·N / F, such that the windower obtains a windowed time portion of length (E+2)·N / F; and

[0248] Temporal aliasing cancellation is performed as follows: the windowed temporal portions of the frame are overlapped and added together, such that the tail component of the windowed temporal portion of the current frame with a length of (E+1) / (E+2) overlaps with the front component of the windowed temporal portion of the previous frame with a length of (E+1) / (E+2).

[0249] The inverse transform mentioned above is inverse MDCT or inverse MDST, and

[0250] The synthesis window is a downsampled version obtained by downsampling a reference synthesis window of length (E+2)·N according to a factor F and interpolating it piecewise according to segments of length 1 / 4·N.

[0251] 19. A computer program having program code for performing the method according to embodiment 16 or 18 when run on a computer.

[0252] References

[0253] [1] ISO / IEC 14496-3:2009

[0254] [2]M13958, "Proposal for an Enhanced Low Delay Coding Mode", October 2006, Hangzhou, China.

Claims

1. An audio decoder (10) configured to decode an audio signal (22) from a data stream (24) at a first sampling rate, the audio signal (22) being transformed and encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the audio decoder (10) comprising: The receiver (12) is configured to receive N spectral coefficients (28) of each frame of the audio signal, wherein the length of the frame is N; The grabber (14) is configured to grab a low-frequency component of length N / F from the N spectral coefficients (28) for each frame; A spectrum-time modulator (16) is configured to, for each frame (36), inversely transform the low-frequency component to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F that extends temporally over the corresponding frame and E+1 previous frames, where E is any integer greater than zero or any even integer. A windower (18) is configured to window the time portion for each frame (36) using a composition window of length (E+2)·N / F, the composition window including a zero portion of length 1 / 4·N / F at its beginning and having a peak within a time interval of the composition window, the time interval being after the zero portion and having a length of 7 / 4·N / F, such that the windower obtains a windowed time portion of length (E+2)·N / F; as well as The temporal aliasing canceller (20) is configured to perform an overlap-addition process on the windowed temporal portions of the frame, such that the tail component of the windowed temporal portion of the current frame with a length of (E+1) / (E+2) overlaps with the front component of the windowed temporal portion of the previous frame with a length of (E+1) / (E+2). The inverse transform mentioned above is inverse MDCT or inverse MDST, and The synthesis window mentioned therein is a downsampled version obtained by downsampling a reference synthesis window of length (E+2)·N according to a factor F and then interpolating it piecewise according to segments of length 1 / 4·N. The audio decoder (10) is configured to perform the interpolation in such a way that most of the coefficients of the synthesis window depend on more than two coefficients of the reference synthesis window, and each coefficient of the synthesis window does not depend on the coefficients of the reference synthesis window that belong to a segment different from the segment in which the coefficient is located.

2. The audio decoder (10) according to claim 1, wherein the inverse transform is inverse MDCT.

3. The audio decoder (10) according to claim 1, wherein, More than 80% of the size of the synthesis window is included after the zero portion and within the time interval of length 7 / 4·N / F.

4. The audio decoder (10) according to claim 1, wherein, The audio decoder (10) is configured to perform the interpolation or export the synthesis window from memory.

5. The audio decoder (10) according to claim 1, wherein, The audio decoder (10) is configured to support different values ​​of F.

6. The audio decoder (10) according to claim 1, wherein F is between 1.5 and 10, and includes 1.5 and 10.

7. The audio decoder (10) according to claim 1, wherein, The reference synthesis window is unimodal.

8. The audio decoder (10) according to claim 1, wherein, The audio decoder (10) is configured to perform the interpolation in such a way that each coefficient of the synthesis window, which is separated by more than two coefficients from the segment boundary, depends on more than two coefficients of the reference synthesis window.

9. The audio decoder (10) according to claim 1, wherein, The windower (18) and the temporal aliasing canceller cooperate such that the windower skips the zero portion when weighting the temporal portion using the synthesis window, and the temporal aliasing canceller (20) does not consider the corresponding unweighted portion of the windowed temporal portion in the overlap summation process, so only E+1 windowed temporal portions are summed, resulting in the corresponding unweighted portion of the corresponding frame and E+2 windowed portions being summed in the remainder of the corresponding frame.

10. An apparatus for generating a reduced version of a synthesis window of the audio decoder (10) according to claim 1, wherein, The device is configured to downsample a reference synthesis window of length (E+2)·N according to a factor F and perform segmented interpolation in 4·(E+2) segments of equal length.

11. A method for generating a reduced version of the synthesis window of the audio decoder (10) according to claim 1, wherein, The method includes downsampling a reference synthesis window of length (E+2)·N according to a factor F and performing segmented interpolation in 4·(E+2) segments of equal length.

12. A method for decoding an audio signal (22) from a data stream (24) at a first sampling rate, the audio signal (22) being transformed and encoded into the data stream at a second sampling rate, the first sampling rate being 1 / F of the second sampling rate, the method comprising: Receive N spectral coefficients (28) for each frame of the audio signal, where the frame length is N; For each frame, extract a low-frequency component of length N / F from the N spectral coefficients (28); Spectrum-time modulation is performed as follows: for each frame (36), the low-frequency component is inversely transformed to obtain a time portion of length (E+2)·N / F, wherein the inverse transform has a modulation function of length (E+2)·N / F that extends temporally over the corresponding frame and E+1 previous frames, where E is any integer greater than zero or any even integer. For each frame (36), a synthesis window of length (E+2)·N / F is used to window the time portion, the synthesis window including a zero portion of length 1 / 4·N / F at its beginning and having a peak within a time interval of the synthesis window, the time interval being after the zero portion and having a length of 7 / 4·N / F, so that a windowed time portion of length (E+2)·N / F is obtained; as well as Temporal aliasing cancellation is performed as follows: the windowed temporal portions of the frame are overlapped and added together, such that the tail component of the windowed temporal portion of the current frame with a length of (E+1) / (E+2) overlaps with the front component of the windowed temporal portion of the previous frame with a length of (E+1) / (E+2). The inverse transform mentioned above is inverse MDCT or inverse MDST, and The synthesis window mentioned therein is a downsampled version obtained by downsampling a reference synthesis window of length (E+2)·N according to a factor F and then interpolating it piecewise according to segments of length 1 / 4·N. The segmented interpolation is performed in such a manner that most of the coefficients of the synthesis window depend on more than two coefficients of the reference synthesis window, and each coefficient of the synthesis window does not depend on the coefficients of the reference synthesis window that belong to a segment different from the segment in which the coefficient is located.

13. A computer-readable storage medium having a computer program having program code thereon for performing the method according to claim 11 or 12 when run on a computer.

Images