Audio decoder for interleaving signals

Abstract

This invention provides a method for decoding an encoded audio bitstream in an audio processing system. [Solution] The decoding method includes the steps of: extracting a first waveform encoded signal from an encoded audio bitstream in an audio processing system, including spectral coefficients up to a first crossover frequency with respect to a time frame; performing parametric decoding above a second crossover frequency in a restoration range with respect to a time frame and generating a restored signal using the acquired restoration parameters; extracting a second waveform encoded signal from an encoded audio bitstream, including spectral coefficients corresponding to a subset of frequencies above the first crossover frequency with respect to a time frame; and interleaving the second waveform encoded signal with the restored signal to generate an interleaved signal with respect to a time frame.

Description

[Technical Field]

[0001] The disclosures herein generally relate to multi-channel audio coding. In particular, these disclosures relate to encoders and decoders for hybrid coding, including parametric coding and discrete multi-channel coding.

[0002] "Cross-reference to related applications" This application is a continuation of U.S. Patent Application No. 14 / 772,001, filed on September 1, 2015, which is a Section 371 national phase application of PCT Application No. PCT / EP2014 / 056852, filed on April 4, 2014, claiming similar priority to U.S. Provisional Patent Application No. 61 / 808,680, filed on April 5, 2013. As a result, each of these applications is incorporated herein by reference to its whole. [Background technology]

[0003] Conventional multi-channel audio coding includes discrete multi-channel coding such as MPEG Surround® and parametric coding. The scheme used depends on the bandwidth of the audio system. Parametric coding methods are known to be scalable and efficient with respect to listening quality, which makes them particularly attractive in low-bitrate applications. Discrete multi-channel coding is often used in high-bitrate applications. In particular, for applications with bitrates between low and high bitrates, existing distribution or processing formats and associated coding techniques can be improved in terms of their bandwidth efficiency. [Overview of the project] [Problems that the invention aims to solve]

[0004] U.S. Patent No. 7,292901 (by "Kroon," et al.) relates to a hybrid encoding method in which a hybrid audio signal is formed from at least one downmixed spectral component and at least one pure (unmixed) spectral component. The method disclosed in its application may increase the capacity of applications with a particular bitrate, however, further improvements may be needed to further increase the efficiency of the audio processing system.

[0005] An example of a real-world implementation will be described here with reference to the attached drawings. [Brief explanation of the drawing]

[0006] [Figure 1] This is a generalized configuration diagram of a decoding system according to one example embodiment. [Figure 2] This figure illustrates the first part of the decoding system shown in Figure 1. [Figure 3] This figure illustrates the second part of the decoding system shown in Figure 1. [Figure 4] This figure illustrates the third part of the decoding system in Figure 1. [Figure 5] This is a generalized configuration diagram of an encoding system based on one example embodiment. [Figure 6] This is a generalized configuration diagram of a decoding system according to one example embodiment. [Figure 7] This figure illustrates the third part of the decoding system shown in Figure 6. [Figure 8] This is a generalized configuration diagram of an encoding system based on one example embodiment. [Modes for carrying out the invention]

[0007] All drawings are schematic and generally show only the elements necessary to explain the present disclosure, while other elements may be omitted or merely suggested. Unless otherwise indicated, like reference numerals refer to like elements in different drawings.

[0008] "Overview of the Decoder" As used herein, an audio signal can be a pure audio signal, the audio portion of an audiovisual signal or a multimedia signal, or any of these combined with metadata.

[0009] As used herein, downmixing of a plurality of signals means combining the plurality of signals, for example, by forming a primary combination such that a smaller number of signals are obtained. The reverse operation to downmixing is called upmixing, i.e., operating on a smaller number of signals to obtain a larger number of signals.

[0010] According to a first aspect, example embodiments propose a method, an apparatus, and a computer program product for restoring a multi-channel audio signal based on an input signal. The proposed method, apparatus, and computer program product may generally have the same features and advantages.

[0011] According to an example embodiment, a decoder suitable for a multi-channel audio processing system for restoring M (M > 2) encoded channels is provided. The decoder includes a first receiving stage configured to receive N (1 < N < M) waveform-encoded downmix signals including spectral coefficients corresponding to frequencies between a first crossover frequency and a second crossover frequency.

[0012] The decoder is a second receiving stage configured to receive M waveform-coded signals including spectral coefficients corresponding to frequencies up to a first crossover frequency, each of the M waveform-coded signals corresponding to a respective one of M coded channels, and further includes the second receiving stage.

[0013] The decoder further includes a downmixing stage downstream of the second receiving stage, configured to downmix the M waveform-coded signals into N downmix signals including spectral coefficients corresponding to frequencies up to the first crossover frequency.

[0014] The decoder further includes a first combining stage downstream of the first receiving stage and the downmixing stage, configured to combine each of the N waveform-coded downmix signals received by the first receiving stage with a corresponding one of the N downmix signals from the downmixing stage to form N combined downmix signals.

[0015] The decoder further includes a high-frequency restoration stage downstream of the first combining stage, configured to perform high-frequency restoration to expand each of the N combined downmix signals from the first combining stage to a frequency range above a second crossover frequency.

[0016] The decoder is an upmixing stage downstream of the high-frequency restoration stage, configured to perform parametric upmixing of the N frequency-expanded combined downmix signals from the high-frequency restoration stage into M upmix signals including spectral coefficients corresponding to frequencies above the first crossover frequency, each of the M upmix signals corresponding to one of M coded channels, and further includes the upmixing stage.

[0017] The decoder further includes an upmix stage and a second coupling stage downstream of the second receiving stage, configured to couple M upmix signals from the upmix stage with M waveform coded signals received by a second receiving stage.

[0018] The M waveform-encoded signals are purely waveform-encoded signals without any parametric signal mixing; that is, they are undownmixed discrete representations of the processed multi-channel audio signal. The advantage of having lower frequencies represented by these waveform-encoded signals may be that the human ear is more sensitive to the low-frequency portions of the audio signal. Furthermore, encoding this portion with better quality can enhance the overall impression of the decoded audio.

[0019] The advantage of having at least two downmix signals is that this embodiment provides increased dimensionality of the downmix signal compared to a system with only one downmix channel. According to this embodiment, better decoded audio quality may be provided, which may exceed the gain at the bitrate offered by a single downmix signal system.

[0020] The advantage of using hybrid coding, which includes parametric downmixing and discrete multichannel coding, is that it can improve the quality of the decoded audio signal for a given bitrate compared to the conventional parametric coding approach, i.e., MPEG Surround with HE-AAC. At a bitrate of approximately 72 kilobits per second (kbps), the conventional parametric coding model can saturate, meaning that the quality of the decoded audio signal is limited not by a lack of bits for coding, but by the shortcomings of the parametric model. Therefore, for bitrates from approximately 72 kbps, it may be more beneficial to use bits to discretely waveform code lower frequencies. At the same time, the hybrid approach using parametric downmixing and discrete multichannel coding can improve the quality of the decoded audio signal for a given bitrate, for example, below 128 kbps, compared to using an approach where all bits are used to waveform code lower frequencies and spectral band replication (SBR) is used for the remaining frequencies.

[0021] The advantage of having N waveform-coded downmix signals containing only spectral data corresponding to frequencies between a first crossover frequency and a second crossover frequency is that the bit communication speed required for the audio signal processing system can be reduced. Instead, the bits saved by having band-pass filtered downmix signals can be used to waveform encode lower frequencies, for example, by increasing the sample frequency for those frequencies or by increasing the first crossover frequency.

[0022] As mentioned above, since the human ear is more sensitive to the low-frequency portion of an audio signal, the high frequencies, which are the portion of the audio signal with frequencies above the second crossover frequency, can be reproduced by high-frequency restoration without reducing the perceived audio quality of the decoded audio signal.

[0023] A further advantage of this embodiment may be that the complexity of the upmix is ​​reduced because the parametric upmix performed in the upmix stage processes only the spectral coefficients corresponding to frequencies above the first crossover frequency.

[0024] In another embodiment, the coupling performed in the first coupling stage is performed in the frequency domain, wherein each of N waveform-coded downmix signals, each containing spectral coefficients corresponding to frequencies between a first crossover frequency and a second crossover frequency, is coupled with a corresponding one of the N downmix signals, each containing spectral coefficients corresponding to frequencies up to the first crossover frequency, to form N coupled downmix signals.

[0025] An advantage of this embodiment is that M waveform-coded signals and N waveform-coded downmix signals can be encoded by a waveform coder using overlapping windowed transforms with independent windowing for the M waveform-coded signals and N waveform-coded downmix signals, respectively, and can still be decoded by a decoder.

[0026] In another embodiment, extending each of the N coupled downmix signals to a frequency range above a second crossover frequency in the high-frequency restoration stage is performed in the frequency domain.

[0027] In a further embodiment, the coupling performed in the second coupling stage, i.e., the coupling of M upmix signals containing spectral coefficients corresponding to frequencies above the first crossover frequency with M waveform-coded signals containing spectral coefficients corresponding to frequencies up to the first crossover frequency, is performed in the frequency domain. As mentioned above, the advantage of coupling the signals in the QMF domain is that independent windowing of the overlap windowing transform used to encode the signals in the MDCT domain can be used.

[0028] In another embodiment, the parametric upmixing of N frequency-extended coupled downmix signals into M upmix signals, performed in the upmix stage, is carried out in the frequency domain.

[0029] In yet another embodiment, the downmixing of M waveform-encoded signals into N downmix signals containing spectral coefficients corresponding to frequencies up to a first crossover frequency is performed in the frequency domain.

[0030] In one embodiment, the frequency domain is the quadrature mirror filter (QMF) domain.

[0031] In another embodiment, the downmixing performed in the downmixing stage is performed in the time domain, in which M waveform encoded signals are downmixed into N downmix signals containing spectral coefficients corresponding to frequencies up to a first crossover frequency.

[0032] In yet another embodiment, the first crossover frequency is determined by the bit transmission rate of the multi-channel audio processing system. This can result in the portion of the audio signal having frequencies below the first crossover frequency being simply waveform-coded, so the available bandwidth is used to improve the quality of the decoded audio signal.

[0033] In another embodiment, extending each of the N coupled downmix signals to a frequency range above a second crossover frequency by performing high-frequency restoration in a high-frequency restoration stage is performed using high-frequency restoration parameters. These parameters may be received by the decoder, for example, in the receiving stage and then transmitted to the high-frequency restoration stage. High-frequency restoration may include, for example, performing spectral band duplication (SBR).

[0034] In another embodiment, parametric upmixing in the upmixing stage is performed with the use of upmix parameters. The upmix parameters are received by an encoder, for example, in the receiving stage and transmitted to the upmixing stage. An uncorrelated version of the frequency-extended N coupled downmix signals is generated, and a matrix operation is performed on the frequency-extended N coupled downmix signals and the uncorrelated version of the frequency-extended N coupled downmix signals. The parameters of the matrix operation are given by the upmix parameters.

[0035] In another embodiment, N waveform-coded downmix signals received in a first receiving stage and M waveform-coded signals received in a second receiving stage are encoded using overlap windowing transformations with independent windowing processes for the N waveform-coded downmix signals and the M waveform-coded signals, respectively.

[0036] The advantage of this could be that it allows for improved encoding quality, and therefore, improved quality of the decoded multi-channel audio signal. For example, if a transient signal is detected in a higher frequency band at some point in time, the default window sequence may be retained for the lower frequency band, while the waveform encoder may encode this special time frame with a shorter window sequence.

[0037] In one embodiment, the decoder may include a third receiving stage configured to receive further waveform-coded signals, including spectral coefficients corresponding to a subset of frequencies above a first crossover frequency. The decoder may further include an interleaving stage downstream of the upmix stage. The interleaving stage may be configured to interleave the further waveform-coded signals with one of M upmix signals. The third receiving stage may be further configured to receive a plurality of further waveform-coded signals, and the interleaving stage may be further configured to interleave a plurality of further waveform-coded signals with a plurality of M upmix signals.

[0038] This is advantageous in that certain portions of the frequency range above a first crossover frequency, which are difficult to parametrically reconstruct from the downmix signal, can be provided in the waveform coding format as a result of interleaving with the parametrically reconstructed upmix signal.

[0039] In one representative embodiment, interleaving is performed by adding the additional waveform-coded signal to one of the M upmix signals. According to another representative embodiment, the step of interleaving the additional waveform-coded signal with one of the M upmix signals includes replacing one of the M upmix signals with the additional waveform-coded signal in a subset of frequencies above a first crossover frequency corresponding to the spectral coefficients of the additional waveform-coded signal.

[0040] In a typical embodiment, the decoder may be further configured to receive a control signal by, for example, a third receiving stage. The control signal may indicate how to interleave a further waveform-coded signal with one of M upmix signals, and the step of interleaving the further waveform-coded signal with one of M upmix signals is based on the control signal. Specifically, the control signal may indicate frequency ranges and time ranges, such as one or more time / frequency tiles in the QMF region, where the further waveform-coded signal should be interleaved with one of M upmix signals. Thus, the interleaving may occur in time and frequency within a single channel.

[0041] The advantage of this is that it allows for the selection of time and frequency ranges that are not plagued by aliasing of the overlap windowing transform used to encode the waveform encoded signal, or startup / fade-out issues.

[0042] According to several embodiments, a method for decoding an encoded audio bitstream in an audio processing system is disclosed. The method includes the steps of: extracting a first waveform-coded signal from the encoded audio bitstream, including spectral coefficients corresponding to frequencies up to a first crossover frequency; and performing parametric decoding at a second crossover frequency to generate a restored signal. The second crossover frequency is above the first crossover frequency, and the parametric decoding generates the restored signal using restoration parameters obtained from the encoded audio bitstream. The method further includes the steps of: extracting a second waveform-coded signal from the encoded audio bitstream, including spectral coefficients corresponding to a subset of frequencies above the first crossover frequency; and interleaving the second waveform-coded signal with the restored signal to generate an interleaved signal. The interleaved signal is then combined with the first waveform-coded signal.

[0043] Numerous variations exist. For example, the first crossover frequency may be determined by the bit transmission rate of the audio processing system, and the interleaving step may include (i) adding the second waveform-coded signal with the restored signal, (ii) combining the second waveform-coded signal with the restored signal, or (iii) replacing the second waveform-coded signal with the restored signal. The step of combining the interleaved signal with the first waveform-coded signal may be performed in the frequency domain, or the step of performing parametric decoding at the second crossover frequency to generate the restored signal may be performed in the frequency domain. Parametric decoding may include either (i) parametric upmixing using upmix parameters, or (ii) high-frequency restoration using high-frequency restoration parameters, such as spectral band duplication (SBR). The method may further include receiving control signals used during the interleaving step to generate the interleaved signal. The control signal may indicate how the second waveform-coded signal should be interleaved with the restored signal by specifying either a frequency range or a time range for the interleaving step. The first value of the control signal may indicate that the interleaving step is performed for each respective frequency range. The interleaving step may also be performed before the merging step. The interleaving step and the merging step may also be combined into a single stage or operation. The first and second waveform-coded signals may include signals representing the waveforms of audio signals in the frequency or time domain.

[0044] "An Overview of Encoders" According to a second aspect, the exemplary embodiment proposes a method, apparatus, and computer program product for encoding a multi-channel audio signal based on an input signal.

[0045] The proposed methods, apparatus, and computer program products may generally have the same characteristics and advantages.

[0046] The advantages regarding the features and configurations presented in the overview of the above decoder can generally be effective for corresponding features and configurations for an encoder.

[0047] According to an embodiment of the example, an encoder suitable for a multi-channel audio processing system for encoding M (M>2) channels is provided.

[0048] The encoder includes a receiving stage configured to receive M signals corresponding to the M channels to be encoded.

[0049] The encoder further includes a first waveform encoding stage configured to receive the M signals from the receiving stage and generate M waveform-encoded signals including spectral coefficients corresponding to frequencies up to a first crossover frequency by individually waveform-encoding the M signals with respect to a frequency range corresponding to frequencies up to the first crossover frequency.

[0050] The encoder further includes a downmixing stage configured to receive the M signals from the receiving stage and downmix the M signals into N (1 < N < M) downmix signals.

[0051] The encoder further includes a high-frequency restoration encoding stage configured to receive the N downmix signals from the downmixing stage and perform high-frequency restoration encoding on the N downmix signals, the high-frequency restoration encoding stage being configured to extract high-frequency restoration parameters that enable high-frequency restoration of the N downmix signals above a second crossover frequency.

[0052] The encoder is a parametric coding stage configured to receive M signals from a receiving stage and N downmix signals from a downmixing stage, and to parametric code M signals with respect to a frequency range corresponding to a frequency above a first crossover frequency, further comprising a parametric coding stage configured to extract upmix parameters that enable upmixing of N downmix signals to M restored signals corresponding to M channels with respect to a frequency range above a first crossover frequency.

[0053] The encoder further includes a second waveform coding stage configured to receive N downmix signals from a downmixing stage and generate N waveform coded downmix signals by waveform coding the N downmix signals with respect to a frequency range corresponding to the frequency between a first crossover frequency and a second crossover frequency, wherein the N waveform coded downmix signals include spectral coefficients corresponding to the frequency between the first crossover frequency and the second crossover frequency.

[0054] According to one embodiment, in the high-frequency restoration coding stage, high-frequency restoration coding is performed on N downmix signals in the frequency domain, preferably in the quadrature mirror filter (QMF) domain.

[0055] In a further embodiment, the parametric coding of M signals in the parametric coding stage is performed in the frequency domain, preferably in the quadrature mirror filter (QMF) domain.

[0056] In yet another embodiment, generating M waveform-coded signals by individually waveform-coding M signals in a first waveform coding stage involves applying an overlap windowing transform to the M signals, where different overlap window sequences are used for at least two of the M signals.

[0057] According to one embodiment, the encoder further includes a third waveform coding stage configured to generate a further waveform coded signal by waveform coding one of M signals with respect to a frequency range corresponding to a subset of frequency ranges above a first crossover frequency.

[0058] According to the embodiment, the encoder may include a control signal generation stage. The control signal generation stage is configured to generate a control signal indicating how the decoder should interleave a further waveform encoded signal with one of M parametric reconstructions. For example, the control signal may indicate a frequency range and a time range in which the further waveform encoded signal should be interleaved with one of the M upmix signals.

[0059] "Examples of Case Studies" FIG. 1 is a generalized block diagram of a decoder 100 in a multi-channel audio processing system for restoring M encoded channels. The decoder 100 comprises three conceptual elements 200, 300, 400 which will be described in more detail in connection with FIGS. 2 to 4. In a first conceptual element 200, the decoder receives N waveform-coded downmix signals and M waveform-coded signals representing the multi-channel audio signals to be decoded, where 1 < N < M. In the illustrated example, N is set to 2. In a second conceptual element 300, the M waveform-coded signals are downmixed and combined with the N waveform-coded downmix signals. High frequency restoration (HFR) is then performed for the combined downmix signals. In a third conceptual element 400, the high frequency restored signals are upmixed and the M waveform-coded signals are combined with the upmix signals to restore the M encoded channels.

[0060] In a representative embodiment described in connection with FIGS. 2 to 4, the restoration of encoded 5.1 surround audio is described. It may be noted that the low frequency effect signal is not mentioned in the described embodiment or drawings. This does not mean that any low frequency effects are ignored. The low frequency effect (Lfe) is added to the five restored channels in any suitable manner well known to those skilled in the art. It may also be noted that the described decoder is equally well suited for other types of encoded surround audio such as 7.1 or 9.1 surround audio.

[0061] Figure 2 illustrates the first conceptual element 200 of the decoder 100 in Figure 1. The decoder includes two receiving stages 212 and 241. In the first receiving stage 212, the bitstream 202 is decoded into two waveform-coded downmix signals 208a and 208b and then dequantized. Each of the two waveform-coded downmix signals 208a and 208b has a first crossover frequency k y and the second crossover frequency k x Includes spectral coefficients corresponding to frequencies between [a certain point] and [a certain point].

[0062] In the second receiving stage 214, the bitstream 202 is decoded into five waveform encoded signals 210a to e and then dequantized. Each of the five waveform encoded signals 210a to e has a first crossover frequency k y Includes spectral coefficients corresponding to frequencies up to [a certain point].

[0063] As an example, signals 210a-e include two channel pair components and one single channel component for the center. The channel pair components could be, for example, a combination of the left front signal and the left surround signal, and a combination of the right front signal and the right surround signal. Further examples include a combination of the left front signal and the right front signal, and a combination of the left surround signal and the right surround signal. These channel pair components can be encoded, for example, in a sum-and-difference format. All five signals 210a-e can be encoded using overlap windowing transformations with independent windowing, and are still decodeable by a decoder. This can allow for improved encoding quality, and therefore, improved quality of the decoded signal.

[0064] As an example, the first crossover frequency k y This is 1.1 kHz. As an example, the second crossover frequency k x It is within the range of 5.6~8kHz. The first crossover frequency k yThis can change, even if based on individual signals; that is, the encoder can detect that the signal components in a particular output signal may not be reproduced more faithfully than the stereo downmix signals 208a-b, and in order to perform proper waveform coding of the signal components, the bandwidth, i.e., the first crossover frequency k of the associated waveform coded signals, i.e., 210a-e, during that particular time instance. y It should be noted that this can be increased.

[0065] As will be explained later in this description, the remaining stages of the decoder 100 generally operate in the quadrature mirror filter (QMF) domain. For this reason, each of the signals 208a-b and 210a-e, received in modified discrete cosine transform (MDCT) form by the first and second receiving stages 212 and 214, is converted to the time domain by applying the inverse MDCT 216. Each signal is then converted back to the original frequency domain by applying the QMF transform 218.

[0066] In Figure 3, the five waveform encoded signals 210 are downmixed at the first crossover frequency k in the downmix stage 308. y The signals are downmixed into two downmix signals 310 and 312, which contain spectral coefficients corresponding to frequencies up to 1. These downmix signals 310 and 312 can be formed by downmixing the low-pass multi-channel signals 210a to e using the same downmixing scheme used in the encoder to create the two downmix signals 208a to b shown in Figure 2.

[0067] The two new downmix signals 310, 312 are then combined with the corresponding downmix signals 208a - b in the first combining stages 320, 322 to form the combined downmix signals 302a - b. Thus, each of the combined downmix signals 302a - b has spectral coefficients corresponding to frequencies up to the first crossover frequency k y from which the downmix signals 310, 312 originate, and spectral coefficients corresponding to frequencies between the first crossover frequency k y from which the two waveform - coded downmix signals 208a - b received in the first receiving stage 212 (shown in FIG. 2) originate, and the second crossover frequency k x .

[0068] The decoder further includes a high - frequency restoration (HFR) stage 314. The HFR stage is configured to extend each of the two combined downmix signals 302a - b from the combining stage to a frequency range above the second crossover frequency k x by performing high - frequency restoration. According to some embodiments, the high - frequency restoration performed includes performing spectral band replication (SBR). The high - frequency restoration can be performed by using high - frequency restoration parameters that can be received by the HFR stage 314 in any suitable manner.

[0069] The output from the high-frequency restoration stage 314 is two signals 304a-b, including downmix signals 208a-b with applied HFR extension portions 316, 318. As described above, the HFR stage 314 will perform high-frequency restoration based on the frequencies present in the input signals 210a-e from the second receiving stage 214 (shown in Figure 2), which are coupled with the two downmix signals 208a-b. Somewhat simply, the HFR ranges 316, 318 include portions of the spectral coefficients from the downmix signals 310, 312 copied up to the HFR ranges 316, 318. Thus, portions of the five waveform-coded signals 210a-e will appear in the HFR ranges 316, 318 of the output 304 from the HFR stage 314.

[0070] It should be noted that the downmixing in the downmixing stage 308 prior to the high-frequency restoration stage 314 and the coupling in the first coupling stages 320 and 322 can be performed in the time domain, that is, after each signal has been converted to the time domain by applying the inverse modified discrete cosine transform (MDCT) 216 (shown in Figure 2). However, if the waveform-coded signals 210a~e and the waveform-coded downmix signals 208a~b may be encoded by the waveform encoder using overlap windowing transforms with independent windowing, then signals 210a~e and signals 208a~b may not be seamlessly coupled in the time domain. Therefore, a better controlled scenario is achieved if at least the coupling in the first coupling stages 320 and 322 is performed in the QMF domain.

[0071] Figure 4 illustrates the third and final conceptual element 400 of the decoder 100. Output 304 from the HFR stage 314 constitutes the input to the upmix stage 402. The upmix stage 402 creates five signal outputs 404a-e by performing parametric upmixing on the frequency-extended signals 304a-b. Each of the five upmixed signals 404a-e has a first crossover frequency k y This corresponds to one of the five encoded channels in the encoded 5.1 surround sound for higher frequencies. According to a typical parametric upmixing procedure, the upmixing stage 402 first receives the parametric mixing parameters. The upmixing stage 402 further generates uncorrelated versions of the two frequency-extended combined downmix signals 304a-b. The upmixing stage 402 further performs matrix operations on the two frequency-extended combined downmix signals 304a-b and the uncorrelated versions of the two frequency-extended combined downmix signals 304a-b, where the parameters of the matrix operations are given by the upmixing parameters. Alternatively, any other parametric upmixing procedure known in the art may be applied. Applicable parametric upmixing procedures are described, for example, in “MPEG Surround - The ISO / MPEG Standard for Efficient and Compatible Multichannel Audio Coding” (“Herre” et al., Journal of the Audio Engineering Society, Vol. 56, No. 11, November 2008).

[0072] Therefore, the outputs 404a~e from the upmix stage 402 are the first crossover frequency k y It does not include frequencies below this. First crossover frequency k yThe spectral coefficients corresponding to the remaining frequencies up to are present in five waveform encoded signals 210a to e, which are delayed by the delay stage 412 to match the timing of the upmix signal 404.

[0073] The decoder 100 further includes second coupling stages 416, 418. The second coupling stages 416, 418 are configured to combine five upmix signals 404a to e with five waveform coded signals 210a to e received by the second receiving stage 214 (shown in Figure 2).

[0074] It should be noted that any current Lfe signal can be added as a separate signal to the resulting combined signal 422. Each of the signals 422 is then converted to the time domain by applying the inverse QMF transform 414. Thus, the output from the inverse QMF transform 414 is a fully decoded 5.1 channel audio signal.

[0075] Figure 6 illustrates decoding system 100', an improved version of decoding system 100. Decoding system 100' has conceptual elements 200', 300', and 400' corresponding to conceptual elements 200, 300, and 400 in Figure 1. The difference between decoding system 100' in Figure 6 and decoding system in Figure 1 is that conceptual element 200' has a third receiving stage 616, and the third conceptual element 400' has an interleaving stage 714.

[0076] A third receiving stage 616 is configured to receive a further waveform-coded signal. The further waveform-coded signal includes spectral coefficients corresponding to a subset of frequencies above the first crossover frequency. The further waveform-coded signal can be converted to the time domain by applying an inverse MDCT 216. In that case, it can be converted back to the original frequency domain by applying a QMF transform 218.

[0077] It should be understood that further waveform-coded signals may be received as separate signals. However, further waveform-coded signals may similarly form a part of one or more of the five waveform-coded signals 210a-e. In other words, further waveform-coded signals may be coded together with one or more of the five waveform-coded signals 210a-e, for example, using the same MDCT transform. If so, the third receiving stage 616 corresponds to the second receiving stage, i.e., the further waveform-coded signals are received together with the five waveform-coded signals 210a-e by the second receiving stage 214.

[0078] Figure 7 illustrates in more detail the third conceptual element 300' of the decoder 100' in Figure 6. In addition to the high-frequency extended downmix signals 304a-b and the five waveform-coded signals 210a-e, an additional waveform-coded signal 710 is input to the third conceptual element 400'. In the illustrated example, the additional waveform-coded signal 710 corresponds to the third channel of the five channels. The additional waveform-coded signal 710 has a first crossover frequency k y This further includes spectral coefficients corresponding to frequency intervals starting from . However, the form of the subset of frequency ranges above the first crossover frequency covered by the further waveform-coded signals 710 can, of course, vary in different embodiments. It should also be noted that multiple waveform-coded signals 710a to e can be received, and different waveform-coded signals can correspond to different output channels. The subset of frequency ranges covered by the multiple further waveform-coded signals 710a to e can vary among different signals among the multiple further waveform-coded signals 710a to e.

[0079] The further waveform-coded signal 710 may be delayed by the delay stage 712 to match the timing of the upmix signal 404 output from the upmix stage 402. The upmix signal 404 and the further waveform-coded signal 710 are then input to the interleaving stage 714. The interleaving stage 714 interleaves, or combines, the upmix signal 404 with the further waveform-coded signal 710 to generate the interleaved signal 704. In this example, the interleaving stage 714 therefore interleaves the third upmix signal 404c with the further waveform-coded signal 710. Interleaving can be performed by adding the two signals together. However, generally, interleaving is performed by exchanging the upmix signal 404 with the further waveform-coded signal 710 in the frequency and time ranges in which the signals overlap.

[0080] The interleaved signal 704 is then input to the second coupling stages 416, 418, where the interleaved signal 704 is coupled with the waveform-coded signals 201a-e in the same manner as described with reference to Figure 4 to generate the output signal 722. It should be noted that the order of the interleaving stage 714 and the second coupling stages 416, 418 may be reversed so that the coupling occurs before the interleaving.

[0081] Furthermore, in situations where an additional waveform-coded signal 710 forms a portion of one or more of the five waveform-coded signals 210a to e, the second coupling stages 416, 418 and the interleaved stage 714 can be coupled into a single stage. Specifically, such a coupled stage has a first crossover frequency k y For frequencies up to a certain point, the spectral components of five waveform-coded signals 210a to e will be used. For frequencies above the first crossover frequency, the combined stage will use an additional waveform-coded signal 710 and an interleaved upmix signal 404.

[0082] The interleaving stage 714 may operate under the control of a control signal. For this purpose, the decoder 100' may receive a control signal, for example through a third receiving stage 616, indicating how to interleave a further waveform-coded signal with one of the M upmix signals. For example, the control signal may indicate a frequency range and a time range in which the further waveform-coded signal 710 should be interleaved with one of the upmix signals 404. For example, the frequency range and time range may be expressed in terms of a time / frequency tile in which the interleaving should be performed. The time / frequency tile may be a time / frequency tile in terms of a time / frequency grid in the QMF region in which the interleaving is performed.

[0083] The control signal may use a vector, such as a binary vector, to indicate the time / frequency tile where interleaving should be performed. Specifically, there may be a first vector for frequency indication, indicating the frequency at which interleaving should be performed. This indication may be, for example, by indicating a logical 1 for the corresponding frequency interval in the first vector. Similarly, there may be a second vector for time indication, indicating the time interval at which interleaving should be performed. This indication may be, for example, by indicating a logical 1 for the corresponding time interval in the second vector. For this purpose, time frames are generally divided into multiple time slots so that time indications can be made on a subframe basis. A time / frequency matrix may be constructed by intersecting the first and second vectors. For example, the time / frequency matrix may be a binary matrix containing a logical 1 for each time / frequency tile at which the first and second vectors indicate a logical 1. The interleaving stage 714 may use the time / frequency matrix to perform interleaving such that, for example, with respect to a time / frequency tile indicated by logic 1 in the time / frequency matrix, one or more of the upmix signals 404 are replaced by further waveform encoded signals 710.

[0084] It should be noted that vectors may use schemes other than binary schemes to indicate the time / frequency tiles on which interleaving should be performed. For example, a vector might use a first value, such as zero, to indicate that interleaving should not be performed, and a second value to indicate that interleaving should be performed with respect to the specific channel identified by the second value.

[0085] Figure 5 shows, as an example, a generalized configuration diagram of an encoding system 500 suitable for a multi-channel audio processing system for encoding M channels, according to one embodiment.

[0086] In a typical embodiment described in Figure 5, the encoding of 5.1 surround sound audio is illustrated. Therefore, in the illustrated example, M is set to 5. It may be noted that in the described embodiment or in the drawings, low-frequency effect signals are not mentioned. This does not mean that all low-frequency effects are ignored. Low-frequency effects (Lfe) are added to the bitstream 552 in any appropriate way well known to those skilled in the art. It may also be noted that the described encoder is equally well suited to encoding other types of surround sound audio, such as 7.1 or 9.1 surround sound audio. In encoder 500, five signals 502, 504 are received in a receiving stage (not shown). Encoder 500 includes a first waveform coding stage 506 configured to receive the five signals 502, 504 from the receiving stage and generate five waveform coded signals 518 by waveform coding the five signals 502, 504 individually. The waveform coding stage 506 may, for example, perform an MDCT transform on each of the five received signals 502, 504. As discussed with respect to the decoder, the encoder may choose to encode each of the five received signals 502, 504 using an MDCT transform with independent window processing. This may allow for improved coding quality and, therefore, improved quality of the decoded signal.

[0087] The five waveform-coded signals 518 are waveform-coded with respect to a frequency range corresponding to frequencies up to a first crossover frequency. Thus, the five waveform-coded signals 518 contain spectral coefficients corresponding to frequencies up to the first crossover frequency. This can be obtained by applying a low-pass filter to each of the five waveform-coded signals 518. The five waveform-coded signals 518 are then quantized 520 according to a psychoacoustic model. The psychoacoustic model is configured to consider the bitrate available in the multi-channel audio processing system as accurately as possible and to reproduce the encoded signal as perceived by the listener when decoded on the decoder side of the system.

[0088] As discussed above, the encoder 500 performs hybrid coding, including discrete multichannel coding and parametric coding. Discrete multichannel coding is performed on each of the input signals 502 and 504 in the waveform coding stage 506, as described above, for frequencies up to a first crossover frequency. Parametric coding is performed on the decoder side so that the five input signals 502 and 504 can be reconstructed from N downmix signals for frequencies above the first crossover frequency. In the illustrated example in Figure 5, N is set to 2. Downmixing of the five input signals 502 and 504 is performed in the downmixing stage 534. The downmixing stage 534 operates favorably in the QMF region. Therefore, before being input to the downmixing stage 534, the five signals 502 and 504 are converted to the QMF region by the QMF analysis stage 526. The downmixing stage performs linear downmixing on five signals 502 and 504, and outputs two downmix signals 544 and 546.

[0089] These two downmix signals 544, 546 are received by a second waveform coding stage 508 after being converted back to the original time domain by an inverse QMF transform 554. The second waveform coding stage 508 generates two waveform coded downmix signals by waveform coding the two downmix signals 544, 546 with respect to a frequency range corresponding to the frequency between the first crossover frequency and the second crossover frequency. The waveform coding stage 508 may, for example, perform an MDCT transform on each of the two downmix signals. Thus, the two waveform coded downmix signals contain spectral coefficients corresponding to the frequency between the first crossover frequency and the second crossover frequency. The two waveform coded downmix signals are then quantized 522 according to a psychoacoustic model.

[0090] To enable the decoder to reconstruct frequencies above the second crossover frequency, high-frequency reconstruction (HFR) parameters 538 are extracted from the two downmix signals 544 and 546. These parameters are extracted in the HFR coding stage 532.

[0091] Five input signals 502, 504 are received by the parametric coding stage 530 to enable the decoder to reconstruct five signals from two downmix signals 544, 546. The five signals 502, 504 are parametric coded with respect to the frequency range corresponding to frequencies above a first crossover frequency. The parametric coding stage 530 is configured to extract an upmix parameter 536 that enables upmixing of the two downmix signals 544, 546 to five reconstructed signals corresponding to the five input signals 502, 504 (i.e., five channels in coded 5.1 surround sound) with respect to the frequency range above the first crossover frequency. It may be noted that the upmix parameter 536 is extracted only for frequencies above the first crossover frequency. This can reduce the complexity of the parametric coding stage 530 and the bitrate of the corresponding parametric data.

[0092] It may be noted that downmixing 534 can be achieved in the time domain. In such cases, since the HFR coding stage 532 generally operates in the QMF domain, the QMF analysis stage 526 should be placed downstream of the downmixing stage 534 and before the HFR coding stage 532. In this case, the inverse QMF stage 554 can be omitted.

[0093] The encoder 500 further includes a bitstream generation stage, i.e., a bitstream multiplexer 524. According to a typical embodiment of the encoder 500, the bitstream generation stage is configured to receive five encoded and quantized signals 548, two parameter signals 536 and 538, and two encoded and quantized downmix signals 550. These are converted into a bitstream 552 by the bitstream generation stage 524 so that they can be further distributed in a multichannel audio system.

[0094] In the described multi-channel audio system, the maximum available bitrate often exists, for example, when streaming audio over the internet. Because the characteristics of each time frame of the input signals 502, 504 are different, the exact same bit allocation may not be used between the five waveform-coded signals 548 and the two downmix waveform-coded signals 550. Furthermore, each individual signal 548 and 550 may require more or fewer allocated bits so that the signal can be reconstructed according to the psychoacoustic model. According to a typical embodiment, the first and second waveform coding stages 506, 508 share a common bit storage. The available bits per encoded frame are first distributed between the first and second waveform coding stages 506, 508, depending on the characteristics of the signal to be encoded and the current psychoacoustic model. As described above, the bits are then distributed between the individual signals 548, 550. The number of bits used for the high-frequency reconstruction parameter 538 and the upmix parameter 536 are, of course, taken into consideration when distributing the available bits. For perceptually smooth transitions around the first crossover frequency, care is taken to adjust the psychoacoustic models for the first and second waveform coding stages 506, 508 with respect to the number of bits allocated in a particular time frame.

[0095] Figure 8 illustrates an alternative embodiment of the encoding system 800. The difference between the encoding system 800 in Figure 8 and the encoding system 500 in Figure 5 is that the encoder 800 is prepared to generate further waveform-encoded signals by waveform encoding one or more of the input signals 502, 504 with respect to frequency ranges corresponding to a subset of the frequency range above the first crossover frequency.

[0096] For this purpose, the encoder 800 includes an interleaved detection stage 802. The interleaved detection stage 802 is configured to identify portions of the input signals 502, 504 that are not well reconstructed by the parametric reconstruction encoded by the parametric coding stage 530 and the high-frequency reconstruction coding stage 532. For example, the interleaved detection stage 802 may compare the input signals 502, 504 with the parametric reconstruction of the input signals 502, 504 as defined by the parametric coding stage 530 and the high-frequency reconstruction coding stage 532. Based on the comparison, the interleaved detection stage 802 may identify a subset 804 in the frequency range above a first crossover frequency that should be waveform encoded. The interleaved detection stage 802 may similarly identify the time range in which the identified subset 804 in the frequency range above the first crossover frequency should be waveform encoded. The identified frequency and time subsets 804 and 806 can be input to a first waveform coding stage 506. Based on the received frequency and time subsets 804 and 806, the first waveform coding stage 506 generates a further waveform coded signal 808 by waveform coding one or more of the input signals 502 and 504 with respect to the time and frequency ranges identified by subsets 804 and 806. The further waveform coded signal 808 can then be coded and quantized by stage 520 and added to bitstream 846.

[0097] The interleaving detection stage 802 may further include a control signal generation stage. The control signal generation stage is configured to generate a control signal 810 indicating how the decoder should interleave a further waveform-coded signal with one of the parametric reconstructors of the input signals 502, 504. As illustrated with reference to Figure 7, for example, the control signal may indicate the frequency range and time range in which the further waveform-coded signal should be interleaved with the parametric reconstructor. The control signal may be applied to the bitstream 846.

[0098] "Equivalents, extensions, substitutes, and other things" Further embodiments of this disclosure will be obvious to those skilled in the art after considering the above description. Where this description and drawings disclose examples and embodiments, this disclosure is not limited to these specific embodiments. Many modifications and changes can be made without exceeding the scope of this disclosure as defined by the appended claims. Reference numerals appearing in the claims should not be understood as limiting their scope.

[0099] Furthermore, variations of the disclosed embodiments can be understood and achieved by those skilled in the art in practicing the disclosure, based on a review of the drawings, the disclosure, and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude the plural. The mere fact that certain means are mentioned in different dependent claims does not indicate that combinations of these means cannot be effectively used.

[0100] The systems and methods disclosed above may be implemented as software, firmware, hardware, or a combination thereof. In hardware implementations, the division of tasks between the functional units referred to above does not necessarily correspond to a division into physical units; conversely, a single physical component may have multiple functions, and a single task may be performed by several collaborating physical components. Certain or all components may be implemented as software executed by a digital signal processor or microprocessor, or as hardware, or as an application-specific integrated circuit. Such software may be distributed on computer-readable media, which may include computer storage media (or non-temporary media) and communication media (or temporary media). As is well known to those skilled in the art, the term computer storage media includes both volatile and non-volatile media, removable and non-removable media, implemented in any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other media that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media generally include any information distribution media that embody computer-readable instructions, data structures, program modules or other data in modulated data signals such as carrier waves or other means of transfer.

Claims

1. A method for decoding in a multi-channel audio processing system, the method being: The steps include: multiplexing the parameters for frequency reconstruction from the bitstream; The steps include receiving a waveform-coded downmix signal that includes spectral coefficients corresponding to frequencies above a first fixed crossover frequency; A step of determining a reconstructed signal by performing frequency reconstruction based on the waveform coded downmix signal based on the parameters, wherein the reconstructed signal is above a second crossover frequency, and the second crossover frequency is different from the first crossover frequency; The steps include: performing a parametric upmix on the reconstructed signal to obtain M upmix signals; Methods that include...

2. The method according to claim 1, wherein the M upmix signals are interleaved with M waveform encoded signals.

3. The method according to claim 1, wherein M > 1.

4. The method according to claim 1, wherein the waveform-coded downmix signal is determined based on downmixing M waveform-coded signals.

5. A non-temporary computer-readable medium storing instructions that cause one or more processors to perform the method described in Claim 1 when executed by one or more processors.

6. A device for decoding in a multi-channel audio processing system, the device comprising: A demultiplexer that multiplexes and separates parameters for frequency reconstruction from the bitstream; A receiver configured to receive a waveform-coded downmix signal including spectral coefficients corresponding to frequencies above a first fixed crossover frequency; A frequency reconstructor for determining a reconstructed signal by performing frequency reconstruction based on the waveform-coded downmix signal based on the parameters, wherein the reconstructed signal is above a second crossover frequency, the second crossover frequency is different from the first crossover frequency, and the frequency reconstruction is based on the waveform-coded downmix signal, in steps; An upmixer that performs parametric upmixing of the reconstructed signal to obtain M upmix signals. A device having.

7. The apparatus according to claim 6, wherein M > 1.

8. The apparatus according to claim 6, wherein the waveform encoded downmix signal is determined based on downmixing M waveform encoded signals, and M > 1.

Images