Method and apparatus for unified speech and audio decoding improvements

Abstract

Methods, apparatuses, and computer products for decoding an encoded MPEG-D USAC bitstream are described herein. Such methods, apparatuses, and computer products that reduce computational complexity are described herein.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to the following prior applications: U.S. Provisional Application 63 / 027,594 (reference number: D20046USP1), filed May 20, 2020, and European Application 20175652.5 (reference number: D20046EP), filed May 20, 2020, which are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to methods and apparatus for decoding encoded MPEG-D USAC bitstreams. This disclosure further relates to methods and apparatus for reducing computational complexity. In addition, this disclosure also relates to corresponding computer program products.

[0004] Although some embodiments will be described herein with particular reference to this disclosure, it will be understood that this disclosure is not limited to this field of use and can be applied to a broader context. Background Technology

[0005] Any discussion of the background art throughout the disclosure should not be construed as an admission that such art is well known in the art or forms part of common knowledge in the art.

[0006] As specified in the international standard ISO / IEC 23003-3 (hereinafter referred to as the MPEG-D USAC standard), decoders for Unified Speech and Audio Coding (USAC) consist of multiple modules (units) requiring several complex computational steps. Each of these computational steps can impose a burden on the hardware system implementing these decoders. Examples of these modules include the Forward Alias ​​Cancellation (FAC) module (or tool) and the Linear Predictive Coding (LPC) module.

[0007] In the context of adaptive streaming, when switching to different configurations (e.g., different bitrates, such as the bitrate configured within the adaptation set in MPEG-DASH), in order to accurately reproduce the signal from the beginning, it is necessary to supply the decoder with frames (AUs) representing the corresponding time segments of the program. n ), and supplies additional pre-rolled frames (AU) n-1 AU n-2 ..., AU s ) and frame AU n The previous configuration data. Otherwise, due to different encoding configurations (e.g., windowing data, SBR-related data, data related to stereo coding (MPS212)), it cannot be guaranteed that the decoder will only decode frame AU. n It produces the correct output.

[0008] Therefore, the first AU frame to be decoded using the new (current) configuration n It can carry new configuration data and utilize the new configuration to initialize all the pre-roll frames (in AU) required by the decoder. n-x The form, which represents AU n (Previous time period). This can be done, for example, by playing frames immediately (IPF).

[0009] In view of the above, there is a need for implementation methods of the MPEG-D USAC decoder process and modules that reduce computational complexity. Summary of the Invention

[0010] According to a first aspect of this disclosure, a decoder is provided for decoding an encoded MPEG-D USAC bitstream. The decoder may include a receiver configured to receive the encoded bitstream, wherein the bitstream represents a sequence of sample values ​​(hereinafter referred to as audio sample values) and includes multiple frames, wherein each frame includes associated encoded audio sample values, wherein the bitstream includes pre-roll elements comprising one or more pre-roll frames required by the decoder to build a complete signal so as to be able to output valid audio sample values ​​associated with the current frame, and wherein the bitstream further includes a USAC configuration element, the USAC configuration element including a current USAC configuration and a current bitstream identifier as payload. The decoder may further include a parser configured to parse the USAC configuration element to the current bitstream identifier and configured to store the start position of the USAC configuration element in the bitstream and the start position of the current bitstream identifier. The decoder may further include a determiner configured to determine whether the current USAC configuration differs from a previous USAC configuration, and if the current USAC configuration differs from a previous USAC configuration, to store the current USAC configuration. Furthermore, the decoder may include an initializer configured to initialize the decoder if the determiner determines that the current USAC configuration is different from the previous USAC configuration, wherein initializing the decoder may include decoding one or more pre-roll frames included in the pre-roll elements. Initializing the decoder may further include switching the decoder from the previous USAC configuration to the current USAC configuration if the determiner determines that the current USAC configuration is different from the previous USAC configuration, thereby configuring the decoder to use the current USAC configuration. The decoder may also be configured to discard the pre-roll elements and not decode them if the determiner determines that the current USAC configuration is the same as the previous USAC configuration.

[0011] In the case of adaptive streaming, processing the MPEG-D USAC bitstream can involve switching from a previous configuration to a different current configuration. This can be accomplished, for example, through immediately playable frames (IPFs). In this case, regardless of the configuration change, the pre-roll elements (i.e., including the pre-roll frames) can still be fully decoded each time. With the configuration described above, the decoder makes it possible to avoid such unnecessary decoding of the pre-roll elements.

[0012] In some embodiments, the determiner may be configured to determine whether the current USAC configuration is different from the previous USAC configuration by checking the current bitstream identifier against the previous bitstream identifier.

[0013] In some embodiments, the determiner may be configured to determine whether the current USAC configuration is different from the previous USAC configuration by checking the length of the current USAC configuration against the length of the previous USAC configuration.

[0014] In some embodiments, if it is determined that the current bitstream identifier is the same as the previous bitstream identifier, and / or if it is determined that the length of the current USAC configuration is the same as the length of the previous USAC configuration, the determiner may be configured to determine whether the current USAC configuration is different from the previous USAC configuration by comparing the current USAC configuration with the previous USAC configuration byte by byte.

[0015] In some embodiments, the decoder may be further configured to delay the output of valid audio sample values ​​associated with the current frame by one frame, wherein delaying the output of valid audio sample values ​​by one frame may include buffering each frame of audio samples before output, and wherein the decoder may be further configured to: if it is determined that the current USAC configuration is different from the previous USAC configuration, perform a crossfade between the frame of the previous USAC configuration buffered in the decoder and the current frame of the current USAC configuration.

[0016] According to a second aspect of this disclosure, a method is provided for decoding an encoded MPEG-D USAC bitstream using a decoder. The method may include receiving an encoded bitstream, wherein the bitstream represents a sequence of audio sample values ​​and includes multiple frames, wherein each frame includes associated encoded audio sample values, wherein the bitstream includes pre-roll elements comprising one or more pre-roll frames required by the decoder to build a complete signal so as to be able to output valid audio sample values ​​associated with the current frame, and wherein the bitstream further includes a USAC configuration element, which includes a current USAC configuration as a payload and a current bitstream identifier. The method may further include resolving the USAC configuration element to the current bitstream identifier and storing the start position of the USAC configuration element and the start position of the current bitstream identifier in the bitstream. The method may further include determining whether the current USAC configuration differs from a previous USAC configuration, and if the current USAC configuration differs from a previous USAC configuration, storing the current USAC configuration. The method may include: if it is determined that the current USAC configuration is different from the previous USAC configuration, initializing the decoder, wherein initializing the decoder may include decoding one or more pre-roll frames included in the pre-roll elements; and if it is determined that the current USAC configuration is different from the previous USAC configuration, switching the decoder from the previous USAC configuration to the current USAC configuration, thereby configuring the decoder to use the current USAC configuration. The method may further include: if it is determined that the current USAC configuration is the same as the previous USAC configuration, discarding the pre-roll elements through the decoder and not decoding the pre-roll elements.

[0017] In some embodiments, determining whether the current USAC configuration is different from the previous USAC configuration may include checking the current bitstream identifier against the previous bitstream identifier.

[0018] In some embodiments, determining whether the current USAC configuration is different from the previous USAC configuration may include checking the length of the current USAC configuration against the length of the previous USAC configuration.

[0019] In some embodiments, determining whether the current USAC configuration is different from the previous USAC configuration may include comparing the current USAC configuration with the previous USAC configuration byte by byte if it is determined that the current bitstream identifier is the same as the previous bitstream identifier, and / or if it is determined that the length of the current USAC configuration is the same as the length of the previous USAC configuration.

[0020] In some embodiments, the method may further include delaying the output of valid audio sample values ​​associated with the current frame by one frame, wherein delaying the output of valid audio sample values ​​by one frame may include buffering each frame of audio samples before output, and performing a crossfade of the frames of the previous USAC configuration buffered in the decoder with the current frame of the current USAC configuration if it is determined that the current USAC configuration is different from the previous USAC configuration.

[0021] According to a third aspect of this disclosure, a decoder is provided for decoding an encoded MPEG-D USAC bitstream, the encoded bitstream comprising multiple frames, each frame consisting of one or more subframes, wherein the encoded bitstream includes one or more line spectral frequency (LSF) sets for each subframe as a representation of linear prediction coefficients (LPCs). The decoder can be configured to decode the encoded bitstream, wherein decoding the encoded bitstream by the decoder can include decoding the LSF set from each subframe of the bitstream. Decoding the encoded bitstream by the decoder can also include converting the decoded LSF set into a linear spectral pair (LSP) representation for further processing. The decoder can be further configured to temporarily store the decoded LSF set for each frame for interpolation with subsequent frames.

[0022] With the above configuration, the decoder can directly use the last set stored in the LSF representation, thus avoiding the need to convert the last set stored in the LSP representation to LSF.

[0023] In some embodiments, the further processing may include determining the LPC based on the LSP representation by applying a root-finding algorithm, wherein applying the root-finding algorithm may involve scaling the coefficients of the LSP representation within the root-finding algorithm to avoid overflow in the fixed-point range.

[0024] In some embodiments, applying the root-finding algorithm may involve finding polynomials F1(z) and / or F2(z) from the LSP representation by expanding the corresponding product polynomials, wherein scaling is performed as a quadratic scaling of the polynomial coefficients. Scaling may involve or correspond to a left shift operation.

[0025] In some embodiments, the decoder may be configured to acquire a quantized LPC filter, and to compute its weighted version, and to compute the corresponding decimated spectrum, wherein modulation may be applied to the LPC before the decimated spectrum is computed based on pre-computed values ​​that may be obtained from one or more lookup tables.

[0026] According to a fourth aspect of this disclosure, a method is provided for decoding an encoded MPEG-D USAC bitstream, the encoded bitstream comprising multiple frames, each frame consisting of one or more subframes, wherein the encoded bitstream includes one or more line spectral frequency (LSF) sets for each subframe as a representation of linear prediction coefficients (LPCs). The method may include decoding the encoded bitstream, wherein decoding the encoded bitstream may include decoding the LSF set from each subframe of the bitstream. Decoding the encoded bitstream may further include converting the decoded LSF set into a linear spectral pair (LSP) representation for further processing. The method may further include temporarily storing the decoded LSF set for each frame for interpolation with subsequent frames.

[0027] In some embodiments, the further processing may include determining the LPC based on the LSP representation by applying a root-finding algorithm, wherein applying the root-finding algorithm may involve scaling the coefficients of the LSP representation within the root-finding algorithm to avoid overflow in the fixed-point range.

[0028] In some embodiments, applying the root-finding algorithm may involve finding polynomials F1(z) and / or F2(z) from the LSP representation by expanding the corresponding product polynomials, wherein scaling is performed as a quadratic scaling of the polynomial coefficients. Scaling may involve or correspond to a left shift operation.

[0029] According to a fifth aspect of this disclosure, a decoder is provided for decoding an encoded MPEG-D USAC bitstream. The decoder can be configured to implement a forward aliasing cancellation (FAC) tool for eliminating temporal aliasing and / or windowing within a linear prediction domain (LPD) codec when transitioning between ACELP-encoded frames and transform-encoded (TC) frames. The decoder can be further configured to perform a transition from LPD to frequency domain (FD) and apply the FAC tool if the previously decoded windowed signal is ACELP-encoded. The decoder can be further configured to perform a transition from FD to LPD and apply the FAC tool if the first decoded window is ACELP-encoded, wherein the same FAC tool can be used in both the LPD-to-FD and FD-to-LPD transitions.

[0030] With the above configuration, the decoder enables the use of forward aliasing elimination (FAC) tools in both LPD and FD codecs.

[0031] In some embodiments, when the FAC tool is used for the transition from FD to LPD, an ACELP zero-input response can be added.

[0032] According to a sixth aspect of this disclosure, a method is provided for decoding an encoded MPEG-D USAC bitstream using a decoder implementing a forward aliasing cancellation (FAC) tool, the FAC tool being used to eliminate temporal aliasing and / or windowing within a linear prediction domain (LPD) codec when transitioning between ACELP-encoded frames and transform-encoded (TC) frames. The method may include: if the previously decoded windowed signal is ACELP-encoded, performing a transition from LPD to frequency domain (FD) and applying the FAC tool. The method may further include: if a first decoded window is ACELP-encoded, performing a transition from FD to LPD and applying the FAC tool, wherein the same FAC tool can be used in both the LPD-to-FD and FD-to-LPD transitions.

[0033] In some embodiments, the method may further include adding an ACELP zero-input response when the FAC tool is used for the transition from FD to LPD.

[0034] According to a seventh aspect of this disclosure, a computer program product having instructions adapted to cause a processing-capable device to perform the following methods: a method for decoding an encoded MPEG-D USAC bitstream via a decoder; a method for decoding an encoded MPEG-D USAC bitstream comprising a plurality of frames, each frame consisting of one or more subframes, wherein the encoded bitstream includes one or more line spectral frequencies (LSFs) of each subframe as a representation of linear prediction coefficients (LPCs); or a method for decoding an encoded MPEG-D USAC bitstream via a decoder implementing a forward aliasing elimination (FAC) tool for eliminating temporal aliasing and / or windowing within a linear prediction domain (LPD) codec when transitioning between alpha-digital excited linear prediction (ACELP) encoded frames and transform-coded (TC) frames. Attached Figure Description

[0035] Exemplary embodiments of this disclosure will now be described by way of example only with reference to the accompanying drawings, in which:

[0036] Figure 1 An example of an MPEG-D USAC decoder is illustrated schematically.

[0037] Figure 2 The illustration shows an example of a method for decoding an encoded MPEG-D USAC bitstream using a decoder.

[0038] Figure 3 The illustration shows an example of an encoded MPEG-D USAC bitstream that includes pre-roll elements and USAC configuration elements.

[0039] Figure 4 The illustration shows an example of a decoder used to decode an encoded MPEG-D USAC bitstream.

[0040] Figure 5 The illustration shows an example of a method for decoding an encoded MPEG-D USAC bitstream, which comprises multiple frames, each consisting of one or more subframes, wherein the encoded bitstream includes one or more line spectral frequencies (LSFs) of each subframe as a representation of linear prediction coefficients (LPCs).

[0041] Figure 6 The illustration further illustrates a method for decoding an encoded MPEG-D USAC bitstream comprising multiple frames, each frame consisting of one or more subframes, wherein the encoded bitstream includes one or more line spectral frequency (LSF) sets for each subframe as a representation of linear prediction coefficients (LPCs), wherein the method includes temporarily storing the decoded LSF set for each frame for interpolation with subsequent frames.

[0042] Figure 7 The illustration shows another further example of a method for decoding an encoded MPEG-D USAC bitstream, which comprises multiple frames, each consisting of one or more subframes, wherein the encoded bitstream includes one or more line spectral frequencies (LSFs) of each subframe as a representation of linear prediction coefficients (LPCs).

[0043] Figure 8 The illustration shows an example of a method for decoding an encoded MPEG-D USAC bitstream using a decoder that implements a forward aliasing elimination (FAC) tool. This FAC tool is used to eliminate temporal aliasing and / or windowing within a linear prediction domain (LPD) codec when transitioning between algebraically excited linear prediction (ACELP) coded frames and transform-coded (TC) frames.

[0044] Figure 9 An example of a decoder for decoding an encoded MPEG-D USAC bitstream is illustrated, wherein the decoder is configured to implement a forward aliasing elimination (FAC) tool, which is used to eliminate temporal aliasing and / or windowing within a linear prediction domain (LPD) codec when transitioning between ACELP-encoded frames and transform-encoded (TC) frames.

[0045] Figure 10 An example of a device with processing capabilities is illustrated. Detailed Implementation

[0046] Processing of MPEG-D USAC bitstream

[0047] The processing of MPEG-D USAC bitstreams described herein involves different steps, such as decoding the encoded MPEG-D USAC bitstream by the application of a corresponding decoder. Herein and below, MPEG-D USAC bitstream may refer to a bitstream conforming to the standard specified in ISO / IEC 23003-3:2012, Information technology—MPEG audio technologies—Part 3: unified speech and audio coding, and subsequent versions, revisions, and corrections (“hereinafter MPEG-D USAC or USAC”).

[0048] refer to Figure 1The example illustrates an MPEG-D USAC decoder 1000. Decoder 1000 includes an MPEG surround function unit 1200 for stereo or multichannel processing. For example, the MPEG surround function unit 1200 may be described in Clause 7.11 of the USAC standard. This clause is incorporated herein by reference in its entirety. The MPEG surround function unit 1200 may include a one-to-two (OTT) box (OTT decoder block) as an example of an upmixing unit, which can perform mono-to-stereo upmixing.Decoder 1000 further includes: a bitstream payload demultiplexer tool 1400 that separates the bitstream payload into portions for each tool and provides bitstream payload information related to each tool; a scaling factor noise-free decoding tool 1500 that acquires information from the bitstream payload demultiplexer, parses the information, and decodes the scaling factors encoded by Huffman and Differential Pulse Code Modulation (DPCM); and a spectrum noise-free decoding tool 1500 that acquires information from the bitstream payload demultiplexer, parses the information, and decodes the arithmetic-coded data. The system includes: a decoder and reconstructed quantized spectrum; an inverse quantizer tool 1500 that acquires the quantized values ​​for the spectrum and converts the integer values ​​into an unscaled, reconstructed spectrum; the quantizer is preferably a companding quantizer, the companding factor of which depends on the selected core coding mode; a noise filling tool 1500 that fills spectral gaps in the decoded spectrum that occur when spectral values ​​are quantized to zero (e.g., due to strong bit requirements in the encoder); a rescaling tool 1500 that converts the integer representation of the scaling factor to the actual value and multiplies the unscaled inverse quantized spectrum by the relevant scaling factor; an M / S tool 1900, as described in ISO / IEC 14496-3; and a time-domain noise shaping (TNS) tool 1700, as described in ISO / IEC 14496-3. As described in 14496-3; a filter bank / block switching tool 1800, which applies the inverse of a frequency mapping performed in the encoder; preferably using the inverse modified discrete cosine transform (IMDCT) for the filter bank tool; a time-warped filter bank / block switching tool 1800, which replaces the normal filter bank / block switching tool when time-warped mode is enabled; the filter bank is preferably the same as that used for the normal filter bank (IMDCT), and additionally, windowed time-domain samples are mapped from the warped time domain to the linear time domain by time-varying resampling; a signal classifier tool that classifies signals. The instrument tool analyzes the raw input signal and generates control information that triggers the selection of different coding modes; the analysis of the input signal is typically dependent on the implementation method and will attempt to select the optimal core coding mode for a given input signal frame; optionally, the output of the signal classifier can also be used to influence the behavior of other tools, such as MPEG surround, enhanced spectral band replication (SBR), time-warped filter banks, etc.; and the Algebraic Excited Linear Prediction (ACELP) tool 1600, which provides a way to efficiently represent time-domain excitation signals by combining long-term predictors (adaptive codewords) with pulse-like sequences (innovative codewords).Decoder 1000 may further include an LPC filter tool 1300 that generates a time-domain signal from the excitation domain signal by filtering the reconstructed excitation signal using a linear predictive synthesis filter. Decoder 1000 may also include an enhanced spectral bandwidth replication (eSBR) unit 1100. For example, eSBR unit 1100 may be described in Clause 7.5 of the USAC standard. This clause is incorporated herein by reference in its entirety. eSBR unit 1100 receives an encoded audio bitstream or encoded signal from the encoder. eSBR unit 1100 can generate a high-frequency component of this signal, which is combined with the decoded low-frequency component to produce a decoded signal. In other words, eSBR unit 1100 can regenerate the high-frequency band of the audio signal.

[0049] Now for reference Figure 2 and Figure 4 The example illustrates a method and decoder for decoding an encoded MPEG-D USAC bitstream. In step S101, the encoded MPEG-D USAC bitstream is received by receiver 101. This bitstream represents a sequence of audio sample values ​​and comprises multiple frames, each including an associated encoded audio sample value. The bitstream includes pre-roll elements, which include one or more pre-roll frames required by decoder 100 to establish a complete signal so that it can output valid audio sample values ​​associated with the current frame. A complete signal (correct reproduction of audio samples) may, for example, refer to a signal established by the decoder during startup or restart. The bitstream further includes a USAC configuration element, which includes the current USAC configuration as a payload and a current bitstream identifier (ID_CONFIG_EXT_STREAM_ID). If a configuration change occurs, decoder 100 can use the USAC configuration included in the USAC configuration element as the current configuration. The USAC configuration element may be included in the bitstream as part of the pre-roll elements.

[0050] In step S102, the USAC configuration element (pre-volume element) is parsed by parser 102 to the current bitstream identifier. Further, the start position of the USAC configuration element and the start position of the current bitstream identifier in the bitstream are stored.

[0051] refer to Figure 3 The example schematically illustrates the location of USAC configuration element 1 in the MPEG-D USAC bitstream associated with pre-roll element 4. For example... Figure 3As shown in the example, and as mentioned above, USAC configuration element 1 (USACconfig element) includes the current USAC configuration 2 and the current bitstream identifier 3. Pre-roll element 4 includes pre-roll frames 5 and 6 (UsacFrame()[n-1], UsacFrame()[n-2]). The current frame is represented by UsacFrame()[n]. Figure 3 In the example, pre-roll element 4 further includes USAC configuration element 1. To determine a configuration change, pre-roll element 4 can be resolved to USAC configuration element 1, which itself can be resolved to the current bitstream identifier 3.

[0052] In step S103, the determiner 103 then determines whether the current USAC configuration differs from the previous USAC configuration, and if the current USAC configuration differs from the previous USAC configuration, the current USAC configuration is stored. The decoder 100 then uses the stored USAC configuration as the current configuration. Therefore, using the USAC configuration element as described herein makes it possible to avoid unnecessary decoding of pre-roll elements, particularly pre-roll frames included in the pre-roll elements (i.e., every time the configuration changes).

[0053] In an embodiment, a determiner 103 (which can be configured to determine) can determine whether the current USAC configuration is different from the previous USAC configuration by comparing the current bitstream identifier with the previous bitstream identifier. If the bitstream identifier is different, it can be determined that the USAC configuration has changed.

[0054] Alternatively or additionally, in an embodiment, if it is determined that the current bitstream identifier is the same as the previous bitstream identifier, the determiner 103 can determine whether the current USAC configuration is different from the previous USAC configuration by checking the length of the current USAC configuration (config_length_in_bits: length = start of bitstream identifier - start of USAC configuration) against the length of the previous USAC configuration. If it is determined that the lengths are different, it can be determined that the USAC configuration has changed.

[0055] If the current bitstream identifier and / or the length of the current USAC configuration indicate that the USAC configuration has changed, the current USAC configuration is stored. The stored current USAC configuration can then be used as the previous USAC configuration for comparison upon receiving the next USAC configuration element. For example, this can be done as follows:

[0056] a. Jump back to the beginning of the USAC config section in the bitstream;

[0057] b. Batch read (and store) ((config_length_in_bits+7) / 8) bytes of the USAC config payload (unparsed).

[0058] In an embodiment, if it is determined that the current bitstream identifier is the same as the previous bitstream identifier, and / or if it is determined that the length of the current USAC configuration is the same as the length of the previous USAC configuration, the determiner 103 can determine whether the current USAC configuration is different from the previous USAC configuration by comparing the current USAC configuration with the previous USAC configuration byte by byte. Exemplarily, this can be performed as follows:

[0059] a. Jump back to the beginning of the USAC config section in the bitstream;

[0060] b. Batch read ((config_length_in_bits+7) / 8) bytes of the USAC config payload (unparsed);

[0061] c. Compare each new payload byte with the corresponding byte of the previous payload;

[0062] d. If the bytes are different, replace the old (previous) bytes with the new (current) bytes;

[0063] e. If any replacements have been applied, the USAC configuration has changed.

[0064] Refer again Figure 2 and Figure 4 For example, in step S104, if it is determined that the current USAC configuration is different from the previous USAC configuration, the initializer 104 initializes the decoder 100. Initializing the decoder 100 includes: decoding one or more pre-roll frames included in the pre-roll elements, and if it is determined that the current USAC configuration is different from the previous USAC configuration, switching the decoder 100 from the previous USAC configuration to the current USAC configuration, thereby configuring the decoder 100 to use the current USAC configuration. In step S105, if it is determined that the current USAC configuration is the same as the previous USAC configuration, the decoder 100 discards the pre-roll elements and does not decode them. At this point, since the configuration change can be determined based on the USAC configuration elements (i.e., no need to decode the pre-roll elements), decoding the pre-roll elements every time regardless of a USAC configuration change can be avoided.

[0065] In one embodiment, the decoder 100 may delay the output of valid audio sample values ​​associated with the current frame by one frame. Delaying the output of valid audio sample values ​​by one frame may include buffering each frame of audio samples before output, wherein, if it is determined that the current USAC configuration is different from the previous USAC configuration, the decoder 100 performs a crossfade between the frame of the previous USAC configuration buffered in the decoder 100 and the current frame of the current USAC configuration.

[0066] In this regard, an error masking scheme could be enabled in decoder 100, which introduces an additional one-frame delay to the output of decoder 100. This additional delay means that the last output of the previously configured system (e.g., PCM) can still be accessed at the point in time when it is determined that the USAC configuration has changed. This allows crossfading to begin 128 samples earlier than described in the MPEG-D USAC standard, i.e., at the end of the previous frame rather than at the beginning of the flushed framestate. This means that a decoder refresh is not required at all.

[0067] Generally, the computational complexity of refreshing the decoder for one frame is comparable to that of decoding a normal frame. Therefore, this allows for a reduction of one frame's complexity at points where peak loads would occur if (number_of_pre-roll_frames+1)*(complexity of a single frame) were already incurred. Thus, crossfading (or fading in) of the output associated with the current (new) configuration can begin at the end of the last pre-roll frame. Generally, the decoder must be refreshed using the previous (old) configuration to obtain an additional 128 samples, which are used for crossfading to the first 128 samples of the first current (actual) frame (none of which are pre-roll frames) under the current (new) configuration.

[0068] Now for reference Figure 5 The example illustrates a method for decoding an encoded MPEG-D USAC bitstream, which comprises multiple frames, each consisting of one or more subframes, wherein the encoded bitstream includes one or more line spectral frequency (LSF) sets from each subframe as a representation of linear prediction coefficients (LPCs). In step S201, the encoded MPEG-D USAC bitstream is received. Then, in step S202, decoding the encoded bitstream includes decoding the LSF set from each subframe of the bitstream by a decoder (configured to decode). Then, in step S203, the decoded LSF set is converted into a linear spectral pair (LSP) representation by the decoder for further processing.

[0069] Generally, LSP has several characteristics that make it superior to LPC in direct quantization (e.g., less sensitivity to quantization noise).

[0070] refer to Figure 6 For example, in an embodiment, for each frame, the decoded LSF set can be temporarily stored by the decoder for interpolation with subsequent frames, S204a. In this regard, since the last set from the previous frame is needed for interpolation purposes, it may be sufficient to only store the last set in the LSF representation. Temporarily storing the LSF set allows for direct use of the LSF set:

[0071] if(!p_lpd_data->first_lpd_flag){

[0072] memcpy(lsf, h_lpd_dec->lsf_prev, LPD_ORDER*sizeof(DLB_LFRACT));

[0073] }

[0074] Without needing to convert the last set stored in the LSP representation to LSF:

[0075] if (!first_lpd_flag) {

[0076] ixheaacd_lsp_2_lsf_conversion(st->lspold, lsf_flt, ORDER); )

[0078] refer to Figure 7 As an example, alternatively or additionally, in embodiments, further processing may include determining the LPC based on the LSP representation by applying a root-finding algorithm, wherein applying the root-finding algorithm may involve scaling, S204b. The coefficients of the LSP representation may be scaled within the root-finding algorithm to avoid overflow in the fixed-point range.

[0079] In an embodiment, applying the root-finding algorithm may involve finding polynomials F1(z) and / or F2(z) from the LSP representation by expanding the corresponding product polynomials, where scaling can be performed as a quadratic scaling of the polynomial coefficients. This defines a left shift operation 1 << LPD_COEFF_SCALE, where the default value of LPD_COEFF_SCALE is 8. Exemplarily, this can be performed as follows.

[0080] The LSP representation of an LP polynomial is defined by roots (i.e., ω) of P and Q such that z = e iωThe positions of P(z) = 0 are used to find the roots. Since they appear in pairs, only half of the actual roots (usually between 0 and π) need to be transmitted. Therefore, the total number of coefficients for both P and Q is equal to p, which is the number of the original LP coefficients (excluding a0 = 1). A common algorithm for finding these coefficients is to evaluate the polynomial at a series of closely spaced points around the unit circle and observe when the sign of the result changes; when the sign of the result changes, the root must be located between the tested points. Because the roots of P and Q intersect, a single pass is sufficient to find the roots of both polynomials. Although the design range of LSP is [-1...1](cos()), this is not the case for LP coefficients. Therefore, scaling must be performed within the root-finding algorithm. The corresponding example code is given below:

[0081] Find polynomials F1(z) and / or F2(z) from the LSP.

[0082] This is done by extending the product polynomial:

[0083] F1(z)=product(1-2LSP_i z^-1+z^-2)i=0, 2, 4, 6, 8, 10, 12, 13

[0084] F2(z)=product(1-2LSP_i z^-1+z^-2)i=1, 3, 5, 7, 9, 11, 13, 15

[0085] Where LSP_i is the LSP in the cosine field.

[0086] RASalami, October 1990

[0087] As shown below, the pseudocode implements the scaling described above (and is not part of the RASalami algorithm):

[0088]

[0089] Scaling is performed within the root-finding algorithm to avoid overflow within the fixed-point range:

[0090]

[0091]

[0092] In some embodiments, the decoder may be configured to acquire a quantized LPC filter, compute its weighted version, and compute the corresponding decimation spectrum, wherein modulation may be applied to the LPC before the decimation spectrum is computed based on pre-computed values ​​that may be obtained from one or more lookup tables.

[0093] Generally, in Transform Codebook Excitation (TCX) gain calculation, before applying the Inverse Modified Discrete Cosine Transform (MDCT), two quantized LPC filters corresponding to the extremities (i.e., the left and right fold points) of the MDCT block can be obtained, their weighted versions can be calculated, and the corresponding decimated spectra can be calculated. These weighted LPC spectra can be calculated by applying the Odd Discrete Fourier Transform (ODFT) to the LPC filter coefficients. Before calculating the ODFT, composite modulation can be applied to the LPC coefficients so that the frequency bin of the ODFT is perfectly aligned with the frequency bin of the MDCT. This can be described, for example, in Clause 7.15.2 of the USAC standard, which is incorporated herein by reference in its entirety. Since the only possible values ​​of M(ccf1 / 16) are 64 and 48, a lookup table for this composite modulation can be used.

[0094] The following is an example of modulation using a lookup table:

[0095]

[0096] Now for reference Figure 8 and Figure 9 The example illustrates a method for decoding an encoded MPEG-D USAC bitstream using a decoder that implements a forward aliasing elimination (FAC) tool, which is used to eliminate temporal aliasing and / or windowing within a linear prediction domain (LPD) codec when transitioning between alpha-digital excited linear prediction (ACELP) encoded frames and transform-coded (TC) frames.

[0097] For example, the FAC tool can be described in Clause 7.16 of the USAC standard. This clause is incorporated herein by reference in its entirety. Generally, forward aliasing cancellation (FAC) is performed within the LPD codec during the transition between ACELP frames and TC frames to obtain the final synthesized signal. The goal of FAC is to eliminate temporal aliasing and windowing introduced by TC and which cannot be eliminated by preceding or following ACELP frames.

[0098] In step S301, the decoder 300 receives the encoded MPEG-D USAC bitstream. In step S302, if the previously decoded windowed signal was encoded using ACELP, a conversion from LPD to frequency domain FD is performed and FAC tool 301 is applied. Further, in step S303, if the first decoded window was encoded using ACELP, a conversion from FD to LPD is performed and (the same) FAC tool 301 is applied. The type of conversion to be performed can be determined during the decoding process, as this depends on how the MPEG-D USAC bitstream is encoded. Using only one function (lpd_fwd_alias_cancel_tool()) allows for less code and less memory, thus reducing computational complexity.

[0099] In an embodiment, further, when the FAC tool 301 is used for the transition from FD to LPD, an ACELP zero-input response (ACELP ZIR) can be added. The ACELP ZIR can be the actual synthesized output signal of the last ACELP-coded subframe, which is used by the FAC tool to generate the first new output sample after the codec switches from LPD to FD. Adding the ACELP ZIR to the FAC tool (e.g., as an input to the FAC tool) enables a seamless transition from FD to LPD, and / or uses the same FAC tool for transitions from LPD to FD and / or from FD to LPD.

[0100] As mentioned above, the same FAC tool can be applied to both the conversion from LPD to FD and the conversion from FD to LPD. Here, using the same tool can mean applying (or calling) the same function in the decoding application's code, regardless of the conversion between LPD and FD, or vice versa. For example, this function could be the lpd_fwd_alias_cancel_tool() function described below.

[0101] Functions implementing FAC tools (e.g., the function lpd_fwd_alias_cancel_tool()) can receive information as input related to filter coefficients, ZIR, subframe length, FAC length, and / or the FAC signal. In the example code presented below, this information can be represented by *lp_filt_coeff (filter coefficients), *zir (ZIR), len_subfrm (subframe length), fac_length (FAC length), and *fac_signal (FAC signal).

[0102] As described above, the function implementing the FAC tool (e.g., the function lpd_fwd_alias_cancel_tool()) can be designed to be invoked during any instance of decoding, regardless of the current encoding domain (e.g., LPD or FD). This means that the same function can be invoked when switching from FD to LPD, and vice versa. Therefore, the proposed FAC tool, or the function implementing the FAC tool, offers a technical advantage or improvement in code execution during decoding compared to previous implementations. Furthermore, the resulting decoding flexibility allows for code optimization, which was not feasible in previous implementations (e.g., implementations using different functions for FD and LPD to implement the FAC tool).

[0103] Below is sample code for the function that implements the FAC tool:

[0104]

[0105]

[0106] As can be seen from the example code above, the function lpd_fwd_alias_cancel_tool() that implements the FAC tool can be called regardless of the current encoding domain (e.g., FD or LPD), and the transition between encoding domains can be handled appropriately.

[0107] refer to Figure 10 As an example, it should be noted that the methods described herein can also be implemented by a corresponding computer program product having instructions adapted to cause a device 400 with processing capability 401 to perform the methods.

[0108] explain

[0109] Unless otherwise specifically stated, it is evident from the following discussion that, throughout this public discussion, terms such as “processing,” “computing,” “determining,” and “analyzing” are used to refer to the actions and / or processes of manipulating and / or transforming data represented as physical (e.g., electronic) quantities into other data similarly represented as physical quantities in a computer or computing system or similar electronic device.

[0110] In a similar manner, the term "processor" can refer to any device or part of a device that processes electronic data to transform that electronic data into other electronic data. A "computer," "computing machine," or "computing platform" can include one or more processors.

[0111] As described above, the methods described herein can be implemented as a computer program product with instructions adapted to cause a processing-capable device to execute the methods. This includes any processor capable of executing a set of instructions (sequential or otherwise) specifying the actions to be taken. Thus, an example could be a typical processing system that may include one or more processors. Each processor may include one or more of a CPU, a graphics processing unit, a tensor processing unit, and a programmable DSP unit. The processing system may further include a memory subsystem comprising main RAM and / or static RAM and / or ROM. A bus subsystem may be included for communication between components. The processing system may further be a distributed processing system, wherein processors are coupled via a network. If the processing system requires a display, it may include such a display as a liquid crystal display (LCD), any kind of light-emitting diode display (LED) (e.g., including OLED (organic light-emitting diode) displays), or a cathode ray tube (CRT) display. If manual data input is required, the processing system may also include one or more input devices such as alphanumeric input units (e.g., a keyboard), pointing control devices (e.g., a mouse), etc. The processing system may also encompass storage systems such as disk drive units. The processing system may include sound output devices (such as one or more speaker or headphone ports) and network interface devices.

[0112] Computer program products can be, for example, software. Software can be implemented in various ways. Software can be sent or received over a network via a network interface device, or it can be distributed via a carrier medium. The carrier medium can include, but is not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media can include, for example, optical discs, magnetic disks, and magneto-optical discs. Volatile media can include dynamic memory, such as main memory. Transmission media can include coaxial cables, copper wires, and optical fibers, including wires containing bus subsystems. Transmission media can also take the form of sound waves or light waves, such as sound waves or light waves generated during radio wave and infrared data communication. For example, the term "carrier medium" should therefore be considered to include, but is not limited to, solid-state memory; computer products embodied in optical and magnetic media; media carrying propagated signals that can be detected and represented by at least one or more processors, which implement the method when executed; and transmission media in a network that carry propagated signals that can be detected and represented by at least one of one or more processors, which implement the method.

[0113] It should be noted that when the method to be performed includes multiple elements (e.g., multiple steps), no order of these elements is implied unless otherwise specifically stated.

[0114] It will be understood that, in one exemplary embodiment, the steps of the method under discussion are performed by a suitable processor (or processors) of a processing (e.g., a computer) system that executes instructions (computer-readable code) stored in a storage device. It will also be understood that this disclosure is not limited to any particular implementation or programming technique, and that this disclosure can be implemented using any suitable technique for implementing the functions described herein. This disclosure is not limited to any particular programming language or operating system.

[0115] Throughout this disclosure, references to “one embodiment,” “some embodiments,” or “embodiment” mean that a particular feature described in connection with that embodiment is included in at least one embodiment of this disclosure. Therefore, the phrases “in one embodiment,” “in some embodiments,” or “in an embodiment” appearing throughout this disclosure do not necessarily refer to the same embodiment. Furthermore, in one or more embodiments, particular features may be combined in any suitable manner as will be apparent to those skilled in the art according to this disclosure.

[0116] In the claims and the description herein, the terms "comprising," "including," or any one of "comprising" are open-ended terms meaning that at least the following element / feature is included, but not excluding other elements / features. Therefore, when the term "comprising" is used in a claim, it should not be construed as limited to the modules, elements, or steps listed thereafter. As used herein, the terms "including," "including," or "including," are also open-ended terms meaning that at least the element / feature following the term is included, but not excluding other elements / features. Therefore, "including" is synonymous with "comprising" and means "including."

[0117] It should be understood that in the foregoing description of exemplary embodiments of this disclosure, various features of this disclosure are sometimes combined in a single exemplary embodiment, figure, or description thereof in order to simplify the disclosure and aid in understanding one or more of the inventive aspects. However, the approach of this disclosure should not be construed as reflecting an intention in the claims to require more features than expressly recited in each claim. Rather, as reflected in the appended claims, the inventive aspect lies in fewer than all features of a single foregoingly disclosed exemplary embodiment. Therefore, the claims appended to this specification are hereby expressly incorporated, wherein each claim is an independent exemplary embodiment of this disclosure.

[0118] Furthermore, while some of the exemplary embodiments described herein include features that are included in other exemplary embodiments but not others, as those skilled in the art will understand, combinations of features from different exemplary embodiments are intended to be within the scope of this disclosure and to form different exemplary embodiments. For example, any exemplary embodiment of the claimed embodiments in the appended claims can be used in any combination.

[0119] Numerous specific details are set forth in the description provided herein. However, it should be understood that exemplary embodiments of this disclosure may be practiced without these specific details. In other instances, well-known methods, apparatus structures, and techniques have not been shown in detail to avoid obscuring the understanding of this specification.

[0120] Therefore, although the mode considered to be the best mode of this disclosure has been described, those skilled in the art will recognize that other and further modifications can be made thereto without departing from the spirit of this disclosure, and all such changes and modifications falling within the scope of this disclosure are intended to be claimed. For example, steps may be added to or removed from the methods described within the scope of this disclosure.

Claims

1. A decoder for decoding an encoded MPEG-D USAC bitstream, the decoder comprising: a receiver configured to receive an encoded bitstream, wherein the bitstream represents a sequence of audio sample values and comprises a plurality of frames, wherein each frame comprises associated encoded audio sample values, wherein the bitstream comprises a pre-roll element comprising one or more pre-roll frames needed by the decoder to establish a complete signal in order to be able to output valid audio sample values associated with a current frame, and wherein the bitstream further comprises a USAC configuration element comprising as payload a current USAC configuration and a current bitstream identification; a parser configured to parse the USAC configuration element to the current bitstream identification and to store a start position of the USAC configuration element and a start position of the current bitstream identification in the bitstream; a determiner configured to determine whether the current USAC configuration is different from a previous USAC configuration by checking a length of the current USAC configuration, config_length_in_bits, against a length of the previous USAC configuration, and to store the current USAC configuration if the current USAC configuration is different from the previous USAC configuration, wherein the determiner is configured to jump back to the start position of the USAC configuration element in the bitstream and to bulk read and store (config_length_in_bits + 7) / 8 bytes of the unparsed USAC configuration payload; and an initializer configured to initialize the decoder if the determiner determines that the current USAC configuration is different from the previous USAC configuration, wherein initializing the decoder comprises: decoding the one or more pre-roll frames comprised in the pre-roll element, switching the decoder from the previous USAC configuration to the current USAC configuration if the determiner determines that the current USAC configuration is different from the previous USAC configuration, thereby configuring the decoder to use the current USAC configuration, and wherein the decoder is configured to discard and not decode the pre-roll element if the determiner determines that the current USAC configuration is the same as the previous USAC configuration.

2. The decoder of claim 1, wherein, the determiner is configured to determine whether the current USAC configuration is different from the previous USAC configuration by checking the current bitstream identification against a previous bitstream identification.

3. The decoder of claim 1 or 2, wherein, If it is determined that the current bitstream identification is the same as the previous bitstream identification and / or if it is determined that the length of the current USAC configuration is the same as the length of the previous USAC configuration, the determiner is configured to determine whether the current USAC configuration is different from the previous USAC configuration by comparing the current USAC configuration with the previous USAC configuration byte by byte.

4. The decoder of claim 1 or 2, wherein, The decoder is further configured to delay output of valid audio sample values associated with the current frame by one frame, wherein delaying output of valid audio sample values by one frame comprises buffering each frame of audio samples before output, and wherein the decoder is further configured to perform a cross-fade of frames of the previous USAC configuration buffered in the decoder with the current frame of the current USAC configuration if it is determined that the current USAC configuration is different from the previous USAC configuration.

5. A method of decoding an encoded MPEG-D USAC bitstream by a decoder, the method comprising: receiving an encoded bitstream, wherein the bitstream represents a sequence of audio sample values and comprises a plurality of frames, wherein each frame comprises associated encoded audio sample values, wherein the bitstream comprises a pre-roll element comprising one or more pre-roll frames needed by the decoder to establish a complete signal in order to be able to output valid audio sample values associated with a current frame, and wherein the bitstream further comprises a USAC configuration element comprising as payload a current USAC configuration and a current bitstream identification; parsing the USAC configuration element to the current bitstream identification and storing a start position of the USAC configuration element and a start position of the current bitstream identification in the bitstream; determining whether the current USAC configuration is different from a previous USAC configuration by checking a length of the current USAC configuration config_length_in_bits against a length of the previous USAC configuration, and storing the current USAC configuration if the current USAC configuration is different from the previous USAC configuration, wherein the determining comprises jumping back to the start position of the USAC configuration element in the bitstream and bulk reading and storing ((config_length_in_bits + 7) / 8) bytes of the unparsed USAC configuration payload; and initializing the decoder if it is determined that the current USAC configuration is different from the previous USAC configuration, wherein initializing the decoder comprises: decoding the one or more pre-roll frames comprised in the pre-roll element and switching the decoder from the previous USAC configuration to the current USAC configuration if it is determined that the current USAC configuration is different from the previous USAC configuration, thereby configuring the decoder to use the current USAC configuration, wherein the method further comprises: If it is determined that the current USAC configuration is the same as the previous USAC configuration, the decoder discards the pre-roll element and does not decode the pre-roll element.

6. A computer program product having instructions adapted to cause a processing-capable device to perform the method according to claim 5.

7. A computer-readable medium storing a computer program, the computer program including instructions adapted to cause a processing-capable device to perform the method according to claim 5.

Images