Apparatus and method for audio decoding supporting two spectral band replication modes

Abstract

An apparatus for audio decoding according to one embodiment is provided. The apparatus comprises a decoding module for decoding a received encoded audio signal in order to obtain a decoded audio signal. Furthermore, the apparatus comprises a first spectral band replication module for performing spectral band replication in accordance with a decoded audio signal according to a first spectral band replication mode in order to obtain a spectral band replication audio signal. Furthermore, the apparatus comprises a second spectral band replication module for performing spectral band replication in accordance with a decoded audio signal according to a second spectral band replication mode in order to obtain a spectral band replication audio signal, the second spectral band replication mode being different from the first spectral band replication mode. In order to perform spectral band replication, the second spectral band replication module is configured to perform one or more first processing operations and one or more second processing operations. One or more second processing operations depend on one or more first processing operations. The apparatus is configured to receive side information. The second spectral replication module exhibits a state that depends on the side information, and the state is one of an inactive state and one or more activated states. The second spectral band replication module is configured not to perform any operations when the second spectral band replication module is in an inactive state. Furthermore, the second spectral band replication module is configured to perform at least one of one or more first processing operations and one or more second processing operations when the second spectral band replication module is not in an inactive state.

Description

Technical Field

[0001] The present invention relates to audio decoding, and more particularly, to an apparatus and method for audio decoding, and more particularly, to an apparatus and method for audio decoding that support two spectral band replication modes, and more specifically, to reducing the workload of HE-AAC decoding that supports MPEG-4 SBR extension.

[0002] ISO14496-3:2009 / AMD7[1] defines an optional extension of the MPEG-4 SBR algorithm called "SBR extension" (eSBR). This extension is signaled as the "esbr_data()" data field in the SBR extension mechanism.

[0003] By using this extension, the encoder can utilize two encoding tools that were originally standardized within the scope of MPEG-D USAC (ISO / IEC23003-3)[2]. One of these tools, the harmonic bandwidth extension (HBE), optionally replaces a relatively simple and low-cost SBR copy-up mechanism ("SBR legacy patch") with a more advanced and computationally expensive algorithm.

[0004] The SBR legacy patch functions in a stateless manner without additional algorithmic delay, but this is not the case with HBE. The algorithm introduces additional delay into the SBR / QMF signal region and requires a signal portion from the previous frame to generate the HBE output signal for the current frame. In general, (legacy) SBR copy-up patching functions in a stateless manner, but HBE requires one additional delay frame for decoding.

[0005] To make the concept of delay more precise, let frame(n) be the nth frame (also known as an access unit (AU)) from the bitstream, and let its decoding process be Decode(n). Here, we assume that frame(n) indicates that an HBE should be applied during Decode(n). The delay of one frame means that some processing steps of the HBE must have already been computed during the preceding iteration Decode(n-1). Thus, the delay corresponds to the n iterations of the process, rather than to a difference in time or samples.

[0006] Within the eSBR tool, it is possible to switch between harmonic SBR patching and legacy SBR patching on a frame-by-frame basis, which is controlled by an encoder with an "sbrPatchingMode" bit. To facilitate frame-by-frame switching between both patching modes, most of the HBE processing operates even during legacy patching, keeping the state up-to-date. Furthermore, legacy patching (if active) is delayed to match delayed harmonic patching.

[0007] However, introducing HBE to legacy HE-AACv2 decoders can significantly increase the workload, even if the bitstream does not include SBR extensions.

[0008] First, we will describe the situation with the MPEG-D USAC decoder, which avoids unnecessary workload increases by completely disabling the HBE tool in certain cases.

[0009] In the case of MPEG-D USAC, where the HBE tool is originally installed, setting the "harmonicSBR" bit to "0" will completely disable the tool as part of the audio configuration. In USAC, the "harmonicSBR" configuration information determines whether the HBE tool is enabled ("1") or completely disabled ("0").

[0010] Before decoding the first frame, it is guaranteed that the audio configuration exists. Therefore, the decoder knows in advance that there is no possibility of HBE activation throughout the entire stream, and that HBE processing and the additional delay of one frame can be completely avoided. The operating mode of the MPEG-D USAC decoder can be summarized as follows:

[0011] [Table 1] As a result, the increased complexity and delay of HBE patching can be controlled and avoided by setting harmonicSBR=0. If HBE patching is not used, no additional complexity or delay occurs in the decoder.

[0012] Next, I will explain the situation regarding the MPEG-4 SBR extension for legacy HE-AACv2. Legacy HE-AACv2 decoder implementations that do not support the MPEG-4 SBR extension essentially operate in a mode equivalent to the "harmonicSBR=0" configuration case of MPEG-D USAC.

[0013] Figure 2 shows the structure of such a legacy HE-AAC decoder (similar to a USAC decoder with harmonicSBR=0).

[0014] As specified in ISO14496-3:2009 / AMD7, SBR extensions cannot be explicitly signaled in the audio configuration, but are implicitly signaled only as part of the audio frame data by the SBR extension mechanism. This means that the decoder cannot distinguish in advance between a legacy HE-AACv2 bitstream without esbr_data() and a bitstream carrying the new esbr_data().

[0015] Due to the characteristic that the existence of esbr_data() is not known to the decoder in advance (e.g., at configuration), decoders supporting the MPEG-4 SBR extension cannot operate in a simple and workload-efficient structure comparable to the case of harmonicSBR=0 in USAC streams. If they attempt to operate and the esbr_data() extension is suddenly found in the stream, the HBE state and signal delay structure necessary to enable the HBE algorithm are missing. In particular, switching the delay structure cannot be done during decoding without significant audio dropouts.

[0016] As a result, state-of-the-art HE-AACv2 decoder implementations that support SBR extensions must operate in the same mode as in the USAC "harmonicSBR=1" configuration. This allows for instantaneous initiation of HBE processing when signaled in the bitstream.

[0017] Figure 3 shows the structure of such an HE-AAC decoder that supports SBR extension (similar to a USAC decoder with harmonicSBR=1).

[0018] In summary, the following operating modes can be distinguished:

[0019] [Table 2] This has the drawback that, because the decoder does not distinguish between the first two cases regarding HBE state updates, most existing legacy bitstreams without esbr_data() are decoded with significant workload overhead. Due to the delay involved in the HBE algorithm, most of the algorithm works even when legacy patching is active to facilitate a smooth switchover.

[0020] The state-of-the-art HE-AACv2 decoder implementation that supports eSBR operates in the same mode of operation as USAC with "harmonicSBR=1". Even in the case of legacy HE-AACv2 streams, the HBE algorithm operates continuously and prepares for switching when esbr_data() is found.

[0021] Especially in the case of battery-powered devices, for example, this is not desirable. As a result, with such a standard implementation approach, the complexity increases significantly even in the case of legacy bitstreams. Depending on the implementation, the overall increase in complexity can be more than double.

[0022] Existing implementations that exhibit this state-of-the-art behavior are the MPEG Reference Software [3] and the libxaac Open Source Software [4] by Ittiam.

[0023] It would be very beneficial to provide an improved concept for audio decoding using two SBR modes.

Summary of the Invention

Problems to be Solved by the Invention

[0024]

Means for Solving the Problems

[0025] An apparatus for audio decoding according to an embodiment is provided. The apparatus includes a decoding module for decoding the received encoded audio signal to obtain a decoded audio signal. Further, the apparatus includes a first spectral band replication module for performing spectral band replication in accordance with a first spectral band replication mode in response to the decoded audio signal to obtain a spectral band replicated audio signal. Further, the apparatus includes a second spectral band replication module for performing spectral band replication in accordance with a second spectral band replication mode in response to the decoded audio signal to obtain a spectral band replicated audio signal, the second spectral band replication mode being different from the first spectral band replication mode. To perform spectral band replication, the second spectral band replication module is configured to perform one or more first processing operations and one or more second processing operations. The one or more second processing operations are dependent on the one or more first processing operations. The apparatus is configured to receive side information. The second spectral replication module exhibits a state that depends on the side information, the state being one of a deactivated state and one or more activated states. The second spectral band replication module is configured not to perform any operations when the second spectral band replication module is in the deactivated state. Further, the second spectral band replication module is configured to perform at least one of the one or more first processing operations and the one or more second processing operations when the second spectral band replication module is not in the deactivated state.

[0026] Further, a method for audio decoding according to an embodiment is provided. The method includes decoding the received encoded audio signal to obtain a decoded audio signal, and performing spectral band replication by a first spectral band replication module in accordance with a first spectral band replication mode in response to the decoded audio signal to obtain a spectral band replicated audio signal. And To obtain a spectrally band duplicated audio signal, the procedure includes the step of performing spectral band duplication by a second spectral band duplication module in response to a decoded audio signal that follows a second spectral band duplication mode, wherein the second spectral band duplication mode differs from the first spectral band duplication mode, and the second spectral band duplication module is configured to perform spectral band duplication by performing one or more first processing operations and one or more second processing operations, the one or more second processing operations depending on one or more first processing operations.

[0027] The method includes the step of receiving side information, wherein the second spectral replication module exhibits a state dependent on the side information, which is one of an inactive state and one or more active states. When the second spectral replication module is in an inactive state, the second spectral replication module performs no operations. If the second spectral replication module is not in an inactive state, the second spectral replication module performs at least one of one or more first processing operations and one or more second processing operations.

[0028] Furthermore, a non-temporary computer-readable medium according to one embodiment is provided, which includes a computer program for implementing the above-described method when executed by a computer or signal processor.

[0029] The embodiments avoid the aforementioned increase in workload. In particular, they are embodiments that limit the complexity of legacy bitstreams.

[0030] Before describing embodiments of the present invention in detail using the accompanying drawings, it should be noted that identical or functionally equivalent elements are given the same reference numeral in the drawings, and repeated descriptions of elements with the same reference numeral are omitted. Therefore, the descriptions provided for elements with the same reference numeral are interchangeable. [Brief explanation of the drawing]

[0031] [Figure 1] This figure shows an apparatus for audio decoding according to one embodiment. [Figure 2] This diagram shows the structure of a legacy HE-AAC decoder without extended SBR support. [Figure 3] This figure shows the structure of such an HE-AAC decoder that supports SBR extensions. [Figure 4] This figure shows an HE-AAC decoder according to an embodiment that supports the MPEG-4 SBR extension and remains in the default operating mode until the first esbr_data() with sbrPatchingMode==0 is found. [Figure 5] This is a block diagram of an AAC decoder with reduced workload for eSBR decoding according to one embodiment. [Modes for carrying out the invention]

[0032] Figure 1 shows an apparatus for audio decoding according to one embodiment. The device includes a decoding module 105 that decodes the received encoded audio signal in order to obtain the decoded audio signal.

[0033] Furthermore, the device includes a first spectral band replication module 110 for performing spectral band replication in accordance with a decoded audio signal that follows a first spectral band replication mode in order to acquire a spectral band replication audio signal. For example, the first spectral band replication mode may be implemented to operate as described, for example, in Chapter [1a]4.6.18, “SBR Tools”.

[0034] Furthermore, the device includes a second spectral band replication module 120 for performing spectral band replication according to a decoded audio signal that follows a second spectral band replication mode in order to acquire a spectral band replication audio signal, wherein the second spectral band replication mode is different from the first spectral band replication mode. For example, the second spectral band replication mode may be implemented to operate as, for example, as described in Chapter 4 of [5], or as, for example, as described in Chapter 7.5.4 of [2], “QMF-based harmonic transposer”, or as, for example, as described in Chapter 7.5.3 of [2], “DFT-based harmonic transposer”, or as, for example, as described in “Appendix 8.A: Combination of SBR tools with parametric stereo tools and SBR extensions” of [1a].

[0035] To perform spectral band replication, the second spectral band replication module 120 is configured to perform one or more first processing operations and one or more second processing operations. The one or more second processing operations depend on one or more first processing operations.

[0036] The device is configured to receive side information. The second spectral replication module exhibits a state dependent on the side information, which is one of an inactive state and one or more active states. The second spectral band replication module 120 is configured not to perform any operations when the second spectral band replication module 120 is in an inactive state. Furthermore, the second spectral band replication module 120 is configured to perform at least one of one or more first processing operations and one or more second processing operations when the second spectral band replication module 120 is not in an inactive state.

[0037] According to one embodiment, one or more activation states may include, for example, a first activation state (e.g., a paused state) and a second activation state (e.g., an ON state). The second spectral band replication module 120 is configured to perform one or more first processing operations when the second spectral band replication module 120 is in the first activation state (e.g., a paused state), but not to perform one or more second processing operations. Furthermore, the second spectral band replication module 120 is configured to perform at least one or more second processing operations when the second spectral band replication module 120 is in the second activation state (e.g., an ON state).

[0038] In one embodiment, the second spectral band replication module 120 is configured to perform both one or more first processing operations and one or more second processing operations when the second spectral band replication module 120 is in a second activated state (e.g., the ON state).

[0039] According to one embodiment, if the received side information indicates that a spectrally band-reproduced audio signal according to a second spectrally band-reproduced mode should be output for subsequent frames of the current frame of the decoded audio data, the device may be configured to set, for example, the second spectrally band-reproduced module 120 to a first activated state, and the second spectrally band-reproduced module 120 may be configured to perform one or more first processing operations, for example, by determining the information necessary to perform spectrally band-reproduced according to the second spectrally band-reproduced mode for the subsequent frames.

[0040] In one embodiment, spectral band replication in the second processing mode may be performed, for example, with a delay of exactly one frame. For example, the device may be configured to switch the second spectral band replication mode from a first activation state (e.g., paused state) to a second activation state (e.g., ON state) at the start of processing the current frame of the decoded audio data. For example, the device may switch to the second activation state ("ON") immediately after parsing the esbr_data() side information before processing the current frame.

[0041] According to one embodiment, spectral band replication in the second processing mode may be performed with a delay of, for example, n frames, where n > 1. For example, the device may be configured to switch the second spectral band replication mode from a first activated state (e.g., paused state) to a second activated state (e.g., on state) at the start of processing the current frame, or after processing the current frame of the decoded audio data, for example, up to n-1 frames.

[0042] In one embodiment, the second spectral band replication module 120 may be configured to calculate one or more second processing operations within the current frame in response to one or more first processing operations in the previous frame, for example, the one immediately preceding the current frame.

[0043] According to one embodiment, one or more first processing operations may include, for example, a critical sampling operation. Details relating to the critical sampling operation are described, for example, in Chapters 4.1 and 4.2 of [5] (H. Zhong, L. Villemoes, P. Ekstrand, S. Disch, F. Nagel, S. Wilde, KO. SE. Chong, T. Norimatsu, "QMF-based harmonic spectral band replication," AES Convention Paper 8517, October 2011). The principle of the critical sampling operation is equally applicable to regions other than the QMF region, such as the DFT region. Further details are shown, for example, in Chapter 7.5.4, "QMF-based harmonic spectral band transposer," and Chapter 7.5.3, "DFT-based harmonic spectral band transposer," of [2]. Furthermore, referring to [1a] and Appendix 8.A, it is outlined that the harmonic transposers and SBR preprocessing defined in [2] (ISO / IEC 23003-3) may be used in combination with the SBR tools defined in Subsection 4.6.18. The bitstream element esbr_data() defined in Subsection 8.A.2 of [1] conveys the information required by these tools and is carried in the sbr_extension() container of the SBR bitstream.

[0044] In one embodiment, one or more second processing operations may include, for example, at least one of one or more time-stretching operations, one or more transposition operations, and overlap-adding operations. Details of time-stretching operations, transposition operations, and overlap-adding operations are described, for example, in Chapters 4.1, 4.3, 4.4, and 4.5 of [5] (H. Zhong, L. Villemoes, P. Ekstrand, S. Disch, F. Nagel, S. Wilde, KO. SE. Chong, T. Norimatsu, "QMF-based harmonic spectral band replication," AES Convention Paper 8517, October 2011). The principles of time-stretching operations, transposition operations, and overlap-adding operations are equally applicable to other regions other than the QMF region, such as the DFT region. Further details are provided, for example, in Chapter 7.5.4, “QMF-based harmonic transposers” and Chapter 7.5.3, “DFT-based harmonic transposers” of [2]. Furthermore, referring again to [1a] in Appendix 8.A, it outlines that the harmonic transposers and SBR preprocessing defined in [2] (ISO / IEC 23003-3) may be used in combination with the SBR tools defined in Subsection 4.6.18. The bitstream element esbr_data() defined in Subsection 8.A.2 of [1] conveys the information required by these tools and is carried in the sbr_extension() container of the SBR bitstream.

[0045] According to one embodiment, the device may be configured to set the second spectral band replication module 120 from an inactive state to one of one or more activated states, for example, depending on the presence of extended spectral band replication data (e.g., esbr_data()) in the side information.

[0046] In one embodiment, the device may be configured to set the second spectral band replication module 120 from an inactive state to one of one or more activated states, depending on, for example, the spectral band replication patching mode data of the side information (e.g., sbrPatchingMode).

[0047] According to one embodiment, the device may be configured to set the second spectral band replication module 120 from an inactive state to one of one or more activated states if, for example, the side information includes extended spectral band replication data (e.g., esbr_data()) and the spectral band replication patching mode data (e.g., sbrPatchingMode) exhibits a predetermined value (e.g., 0) among two or more values ​​(e.g., 0, 1).

[0048] In one embodiment, for example, if the received side information includes first side information indicating, for example, that spectral band replication should be performed using spectral band replication extension, the device may be configured to set the second spectral band replication module 120 to one of one or more activated states.

[0049] According to one embodiment, the first side information can be encoded, for example, with esbr_data() side information.

[0050] In one embodiment, the device may be configured to set the second spectral band replication module 120 to the first activated state if, for example, the received side information may include, for example, second side information, and the second side information indicates that spectral band replication should be set to the first activated state (e.g., paused state). Furthermore, the device may be configured to set the second spectral band replication module 120 to the second activated state (e.g., on state) if, for example, the received side information may include, for example, second side information, and the second side information indicates that spectral band replication should be set to the second activated state.

[0051] According to one embodiment, the second side information can be encoded, for example, with sbrPatchingMode side information.

[0052] In one embodiment, the first bit value in the sbrPatchingMode side information indicates that the first spectral band replication mode should be adopted. The second bit value in the sbrPatchingMode side information indicates that the second spectral band replication mode should be adopted.

[0053] According to one embodiment, the second spectral band replication module 120 may be configured to perform, for example, harmonic band replication.

[0054] In one embodiment, the second spectral band replication module 120 may be configured to perform harmonic band replication in, for example, the quadrature mirror filter (QMF) region.

[0055] According to one embodiment, the second spectral band replication module 120 may be configured to perform harmonic band replication in the discrete Fourier transform (DFT) region, for example.

[0056] In one embodiment, the device may be, for example, a device for HE-AAC decoding.

[0057] According to one embodiment, the decoding module 105 may include, for example, an AAC core decoder 106 for decoding an encoded audio signal in order to obtain a decoded audio signal.

[0058] In one embodiment, the decoding module 105 may include, for example, a QMF analysis module 107. The QMF analysis module 107 may be configured to process the output from the decoding module to obtain, for example, a decoded audio signal.

[0059] According to one embodiment, the apparatus may include, for example, a QMF synthesis module 130. The QMF synthesis module 130 may be configured, for example, to process spectral band duplicated audio signals in order to obtain a processed audio signal.

[0060] The following sections will describe specific embodiments in more detail.

[0061] As outlined above, the decoder has no prior knowledge (from the time of configuration) of whether frames with the MPEG-4 esbr_data() extension, in particular esbr_data() with sbrPatchingMode=0 ("harmonic patching"), exist in a given stream.

[0062] In MPEG-D USAC streams, eSBR side-info is embedded in the stream at a point that coincides with when SBR patching is actually applied. In contrast, in MPEG-4 HE-AAC streams with the esbr_data() extension embedded, the side information does not take into account the algorithmic delay introduced by the HBE.

[0063] This is intended to avoid increased workload for legacy bitstreams. The existence of esbr_data() requires a jump in the decoder delay, making it impossible to operate with legacy HE-AAC decoder structures.

[0064] According to the embodiment, the delay structure is implemented to prepare for the HBE, but the actual processing is disabled until esbr_data() is found. Until it is found for the first time, a fallback to the state-of-the-art (legacy) SBR decoder structure is implemented.

[0065] As a result, according to the embodiment, the decoder delays esbr_data() by one frame before applying it.

[0066] Figure 4 shows an HE-AAC decoder according to one embodiment, which supports the MPEG-4 SBR extension and operates in the default operating mode until the first esbr_data() with sbrPatchingMode==0 is found. In this operating mode, the HBE module is completely off (state="OFF"), thus completely avoiding any additional complexity required to decode legacy HE-AAC bitstreams.

[0067] When an esbr_data() with sbrPatchingMode==0 is found, the HE-AAC decoder switches to the structure shown in Figure 5. During this switch, the HBE module transitions to the "ON" state and performs a one-time recalculation of the missing state. From there, the HBE module switches between the state "ON" (harmonic patching in the next frame) and "PAUSE" (legacy patching in the next frame) depending on the side information. In both states, partial state updates incur a basic workload (avoided in the "OFF" state). However, full state updates and harmonic patching are calculated only when necessary. Otherwise, the HBE module operates in the "PAUSE" state with less workload. Assuming that HE-AAC bitstreams with eSBRs use legacy patching far more frequently than harmonic patching, it means that decoding can still be performed with reduced computational cost because a full update of the HBE state can be avoided.

[0068] In Figures 4 and 5, "sbrPatchingMode(t)" represents a bit that is transmitted in the current frame t and controls the patching algorithm for the next frame t+1. Alternatively, "sbrPatchingMode(t-1)" is the delay bit, and the delay bit controls the patching algorithm for frame t. The frame index t=[1, 2, 3, 4, ...] is used to enumerate consecutive frames in the input bitstream (= access unit (AU)) and is not likely to be used to directly infer the amount of samples processed or the delay between the input and output signals.

[0069] The operation of the eSBR module is summarized in the following state transition table, where transitions occur frame by frame and events are derived from the side information of the current frame. For example, updates may be performed depending on the current state, for example. Patching may be performed depending on the previous state, for example. More specifically, in certain embodiments, for example, the event "eSbrMode==harmonic" means that the current frame contains an esbr_data() with sbrPatchingMode==0.

[0070] [Table 3] The proposed implementation structure not only demonstrates the advantages of a “legacy bitstream” that does not carry the esbr_data() extension, but also the advantages of a stream where esbr_data() is present but sbrPatchingMode=1 (“legacy patching,” in other words, legacy spectral bandwidth replication / normal spectral bandwidth replication) is set for multiple consecutive frames. This is a common use case as shown in [1], namely, the use of the harmonic patching method (sbrPatchingMode==0) is generally preferred for encoding musical signals at very low bitrates, in which case the primary codec may be quite limited in audio bandwidth. This is especially true when these signals contain a significant harmonic structure. In contrast, for speech and mixed signals, the use of the normal SBR patching method is preferred because it allows for better preservation of the temporal structure of the speech.

[0071] For example, the embodiment is extended in Chapter 4, "QMF-based Harmonic SBRs," of [5] Chapter 4, in particular in Figure 3 of [5], as follows: Critical sampling can be performed, for example, always, in both the ON and PAUSE states. Therefore, critical sampling can be considered, for example, one of the one or more first processing operations described above.

[0072] However, stretching and transposing, determining the cross product, and performing overlap and addition are performed only in the ON state, for example, and not in the PAUSE state. Therefore, stretching and transposing, determining the cross product, and performing overlap and addition can be considered, for example, as the second processing operation described above.

[0073] However, in the OFF state, in one embodiment, none of these operations are performed, and neither stretching and transposition, cross product determination, overlapping and addition, nor critical sampling are performed. In the OFF state, the second spectral band replication module can be considered to be deactivated, for example.

[0074] The embodiments are equally applicable to DFT domains and SBRs of DFT domains, as well as to other domains and SBRs of such other domains.

[0075] While several embodiments have been described in the context of the apparatus, it is clear that these embodiments also represent descriptions of the corresponding methods, where blocks or devices correspond to method steps or features of method steps. Similarly, embodiments described in the context of method steps also represent descriptions of the corresponding blocks, items, or features of the corresponding apparatus. Some or all of the method steps may be performed by (or using) hardware devices such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such devices.

[0076] Depending on specific implementation requirements, embodiments of the present invention may be implemented in hardware or software, or at least partially in hardware or at least partially in software. Implementations may be carried out using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, which stores electronically readable control signals and which cooperates (or can cooperate) with a programmable computer system so that each method is performed. Thus, the digital storage medium may be computer-readable.

[0077] Some embodiments of the present invention include a data carrier having an electronically readable control signal that can cooperate with a programmable computer system so that one of the methods described herein can be performed.

[0078] Generally, embodiments of the present invention can be implemented as a computer program product having program code, the program code operates to perform one of the methods when the computer program product runs on a computer. The program code may be stored, for example, in a machine-readable carrier.

[0079] Other embodiments include a computer program stored in a machine-readable carrier for performing one of the methods described herein.

[0080] Therefore, in other words, one embodiment of the method of the present invention is a computer program having program code for performing one of the methods of the present invention when the computer program is running on a computer.

[0081] Accordingly, a further embodiment of the method of the present invention is a data carrier (or digital storage medium, or computer-readable medium) containing a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-temporary.

[0082] Therefore, a further embodiment of the method of the present invention is a data stream or signal sequence representing a computer program for performing one of the methods described herein. The data stream or signal sequence may be configured to be transmitted, for example, over a data communication connection, for example, over the Internet.

[0083] Further embodiments include processing means configured or adapted to perform one of the methods described herein, such as a computer or a programmable logic device.

[0084] Further embodiments include a computer on which a computer program for performing one of the methods described herein is installed.

[0085] Further embodiments of the present invention include an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0086] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to carry out some or all of the functionality of the methods described herein. In some embodiments, a field-programmable gate array can cooperate with a microprocessor to carry out one of the methods described herein. Generally, the methods are preferably carried out by any hardware device.

[0087] The apparatus described herein may be implemented using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0088] The methods described herein may be performed using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0089] The embodiments described above are merely illustrative of the principles of the present invention. Modifications and variations of the configurations and details described herein will be apparent to those skilled in the art. Therefore, it is intended to be limited only by the pending claims and not by the specific details presented as part of the description and explanation of the embodiments herein.

[0090] Although each claim refers to only one single claim, this disclosure also encompasses all possible combinations of claims.

[0091] References: [1] ISO 14496-3:2009 / AMD 7:2018 Information technology-Coding of audio-visual objects-Part 3:Audio,Amendment 7:SBR Enhancements [1a] ISO 14496-3:2019 Information technology-Coding of audio-visual objects-Part 3:Audio,5 th edition. [2] ISO / IEC FDIS 23003-3 Information technology-MPEG audio technologies-Part 3:Unified speech and audio coding [3] MPEG Referenz Decoder Software,https: / / mpeg.expert / software / MPEG / audio / [4] libxaac with eSBR support by Ittiam,https: / / github.com / ittiam-systems / libxaac),Dec 27,2022 [5] H.Zhong,L.Villemoes,P.Ekstrand,S.Disch,F.Nagel,S.Wilde,KO.SE.Chong,and T.Norimatsu,”QMF Based Harmonic Spectral Band Replication,” AES Convention Paper 8517,October 2011.

Claims

1. A device for audio decoding, To obtain the decoded audio signal, a decoding module is used to decode the received encoded audio signal, A first spectral band replication module for performing spectral band replication according to the decoded audio signal that follows a first spectral band replication mode in order to acquire a spectral band replication audio signal, The system includes a second spectral band replication module for performing spectral band replication according to the decoded audio signal in accordance with a second spectral band replication mode in order to acquire the spectral band replication audio signal, wherein the second spectral band replication mode differs from the first spectral band replication mode, and in order to perform spectral band replication, the second spectral band replication module is configured to perform one or more first processing operations and one or more second processing operations, and the one or more second processing operations depend on the one or more first processing operations. The apparatus is configured to receive side information, and the second spectral replication module exhibits a state dependent on the side information, the state being one of an inactive state and one or more activated states. The second spectral band replication module is configured not to perform any operations when the second spectral band replication module is in the deactivated state. The apparatus is configured such that the second spectral band replication module performs at least one of the one or more first processing operations and the one or more second processing operations when the second spectral band replication module is not in the deactivated state.

2. The one or more activation states include a first activation state (e.g., a paused state) and a second activation state (e.g., an on state), The second spectral band replication module is configured to perform one or more of the first processing operations but not the one or more of the second processing operations when the second spectral band replication module is in the first activated state (for example, the paused state). The second spectral band replication module is configured to perform at least one of the second processing operations when the second spectral band replication module is in the second activated state (e.g., the ON state). The apparatus according to claim 1.

3. The second spectral band replication module is configured to perform both the one or more first processing operations and the one or more second processing operations when the second spectral band replication module is in the second activated state (e.g., the ON state). The apparatus according to claim 2.

4. If the received side information indicates that a spectrally band-reproduced audio signal according to the second spectral band-reproduced mode should be output for the frame following the current frame of the decoded audio data, the second spectral band-reproduced module is configured to be set to the first activated state, and the second spectral band-reproduced module is configured to perform one or more of the first processing operations by determining the information necessary to perform spectral band reproduction according to the second spectral band-reproduced mode for the subsequent frame. The apparatus according to claim 2 or 3.

5. The one or more first processing operations include a critical sampling operation. The apparatus according to any one of claims 2 to 4.

6. The one or more second processing operations include at least one of one or more time-stretching operations, one or more transposition operations, and overlap-adding operations. The apparatus according to any one of claims 2 to 5.

7. If the received side information includes second side information, and the second side information indicates that spectral band replication should be set to the first activated state (e.g., the paused state), the second spectral band replication module is configured to set to the first activated state. If the received side information includes the second side information, and the second side information indicates that the spectral band replication should be set to the second activation state (e.g., the ON state), the second spectral band replication module is configured to set to the second activation state. The apparatus according to any one of claims 2 to 6.

8. The second side information is encoded using sbrPatchingMode side information. The apparatus according to claim 7.

9. The first bit value in the sbrPatchingMode side information indicates that the first spectral band replication mode should be adopted. The second bit value in the sbrPatchingMode side information indicates that the second spectral band replication mode should be adopted. The apparatus according to claim 8.

10. In the second processing mode, spectral band replication is performed with a delay of exactly one frame. The apparatus according to any one of claims 1 to 9.

11. In the second processing mode described above, spectral band replication is performed with a delay of n frames, where n > 1. The apparatus according to any one of claims 1 to 10.

12. The second spectral band replication module is configured to calculate one or more second processing operations in the current frame in accordance with one or more first processing operations in the previous frame. The apparatus according to any one of claims 1 to 11.

13. Depending on the presence of extended spectral band replication data (esbr_data()) in the side information, the second spectral band replication module is configured to change from the deactivated state to one of the one or more activated states. The apparatus according to any one of claims 1 to 12.

14. The second spectral band replication module is configured to set from the deactivated state to one of the one or more activated states, depending further on the spectral band replication patching mode data (sbrPatchingMode) of the side information. The apparatus according to claim 13.

15. If the side information includes the extended spectral band replication data (esbr_data()), and the spectral band replication patching mode data (sbrPatchingMode) exhibits a predetermined value (e.g., 0) among two or more values ​​(e.g., 0, 1), the second spectral band replication module is configured to change from the deactivated state to one of the one or more activated states. The apparatus according to claim 14.

16. If the received side information includes first side information indicating that spectral band replication should be performed using spectral band replication extension, the second spectral band replication module is configured to be set to one of the one or more activation states. The apparatus according to any one of claims 1 to 15.

17. The first side information is encoded using esbr_data() side information. The apparatus according to claim 16.

18. The second spectral band replication module is configured to perform harmonic band replication. The apparatus according to any one of claims 1 to 17.

19. The second spectral band replication module is configured to perform harmonic band replication in the quadrature mirror filter (QMF) region. The apparatus according to claim 18.

20. The second spectral band replication module is configured to perform harmonic band replication in the discrete Fourier transform (DFT) region. The apparatus according to claim 18 or 19.

21. This is a device for HE-AAC decoding. The apparatus according to any one of claims 1 to 20.

22. The decoding module includes an AAC core decoder for decoding the encoded audio signal in order to obtain the decoded audio signal. The apparatus according to any one of claims 1 to 21.

23. The decoding module includes a QMF analysis module, The QMF analysis module is configured to process the output from the decoding module to obtain the decoded audio signal. The apparatus according to claim 22.

24. Includes QMF synthesis module, The QMF synthesis module is configured to process the spectral band duplicated audio signal in order to obtain the processed audio signal. The apparatus according to claim 23.

25. A method for audio decoding, The steps include: decoding the received encoded audio signal to obtain the decoded audio signal, To obtain a spectral band duplicated audio signal, a first spectral band duplicate module performs spectral band duplicatement according to the decoded audio signal that follows a first spectral band duplicate mode, The process includes the step of performing spectral band replication in accordance with the decoded audio signal according to a second spectral band replication mode using a second spectral band replication module in order to obtain the spectral band replicated audio signal, wherein the second spectral band replication mode differs from the first spectral band replication mode, and the second spectral band replication module is configured to perform spectral band replication by performing one or more first processing operations and one or more second processing operations, wherein the one or more second processing operations depend on the one or more first processing operations. The method includes the step of receiving side information, wherein the second spectral replication module exhibits a state dependent on the side information, and the state is one of an inactive state and one or more activated states. When the second spectral band replication module is in the deactivated state, the second spectral band replication module does not perform any operations. A method wherein, if the second spectral band replication module is not in the deactivated state, the second spectral band replication module performs at least one of the one or more first processing operations and the one or more second processing operations.

26. A non-temporary computer-readable medium containing a computer program for implementing the method according to claim 25, when the computer program is executed by a computer or signal processor.

27. A computer program that implements the method according to claim 25 when the computer program is executed by a computer or signal processor.

Images