Method for parametric multi-channel encoding

Abstract

The present invention relates to a method for parametric multichannel coding. Specifically, this document relates to an efficient method and system for parametric multichannel audio coding. An audio coding system (500) is described, which is configured to generate a bitstream (564) indicating a downmixed signal and spatial metadata for generating a multichannel upmixed signal from the downmixed signal. The system (500) includes a downmix processing unit (510) configured to generate a downmixed signal from a multichannel input signal (561); wherein the downmixed signal includes m channels, and wherein the multichannel input signal (561) includes n channels; n, m are integers, where m < n. In addition, the system (500) includes a parameter processing unit (520) configured to determine spatial metadata from the multichannel input signal (561). Additionally, the system (500) includes a configuration unit (540) configured to determine one or more control settings for the parameter processing unit (520) based on one or more external settings; wherein the one or more external settings include the target data rate of the bitstream (564), and wherein the one or more control settings include the maximum data rate of the spatial metadata.

Description

[0001] This application is a divisional application of the patent application for invention titled "Method for Parametric Multichannel Coding" with the application number 202310791753.8, the application date of February 21, 2014.

[0002] Cross - reference to related applications

[0003] This application claims the priority of U.S. Provisional Patent Application No. 61 / 767,673 filed on February 21, 2013, the entire content of which is hereby incorporated by reference. Technical field

[0004] This document relates to audio coding systems. Specifically, this document relates to efficient methods and systems for parametric multichannel audio coding. Background art

[0005] Parametric multichannel audio coding systems can be used to provide improved listening quality at particularly low data rates. Nevertheless, further improvements to such parametric multichannel audio coding systems are still needed, especially with respect to bandwidth efficiency, computational efficiency, and / or robustness. Summary of the invention

[0006] According to one aspect, an audio coding system configured to generate a bitstream indicating a down - mixed signal and spatial metadata is described. The spatial metadata can be used by a corresponding decoding system to generate a multichannel up - mixed signal from the down - mixed signal. The down - mixed signal can include m channels, and the multichannel up - mixed signal can include n channels, where n, m are integers, and m < n. In an example, n = 6 and m = 2. The spatial metadata enables the corresponding decoding system to generate the n channels of the multichannel up - mixed signal from the m channels of the down - mixed signal.

[0007] The audio coding system can be configured to quantize and / or encode the down - mixed signal and the spatial metadata and insert the quantized / encoded data into the bitstream. Specifically, the down - mixed signal can be encoded using a Dolby Digital Plus encoder, and the bitstream can correspond to a Dolby Digital Plus bitstream. The quantized / encoded spatial metadata can be inserted into a data field of the Dolby Digital Plus bitstream.

[0008] An audio coding system may include a downmixing processing unit configured to generate a downmixed signal from a multichannel input signal. The downmixing processing unit is also referred to herein as a downmixing encoding unit. The multichannel input signal may include n channels, such as a multichannel upmixed signal regenerated based on the downmixed signal. Specifically, the multichannel upmixed signal can provide an approximation of the multichannel input signal. The downmixing unit may include the Dolby Digital Plus encoder mentioned above. The multichannel upmixed signal and the multichannel input signal may be 5.1 or 7.1 signals, and the downmixed signal may be a stereo signal.

[0009] Audio coding systems may include parameter processing units configured to determine spatial metadata from multichannel input signals. Specifically, the parameter processing unit (also referred to herein as a parameter encoding unit) may be configured to determine one or more spatial parameters, such as a set of spatial parameters, which may be determined based on different combinations of the channels of the multichannel input signal. The spatial parameters of the set of spatial parameters may indicate the cross-correlation between the different channels of the multichannel input signal. The parameter processing unit may be configured to determine the spatial metadata of a frame of the multichannel input signal, called a spatial metadata frame. A frame of the multichannel input signal typically includes a predetermined number (e.g., 1536) samples of the multichannel input signal. Each spatial metadata frame may include one or more sets of spatial parameters.

[0010] The audio encoding system may also include a configuration unit configured to determine one or more control settings for the parameter processing unit based on one or more external settings. The one or more external settings may include a target data rate for the bitstream. Alternatively or additionally, the one or more external settings may include one or more of the following: a sampling rate of the multichannel input signal, the number of channels m of the downmix signal, the number of channels n of the multichannel input signal, and / or an update period indicating the time required for the corresponding decoding system to synchronize with the bitstream. The one or more control settings may include a maximum data rate for spatial metadata. In the case of spatial metadata frames, the maximum data rate for spatial metadata may indicate the maximum number of metadata bits in the spatial metadata frame. Alternatively or additionally, the one or more control settings may include one or more of the following: a time resolution setting indicating the number of spatial parameter sets for each spatial metadata frame to be determined; a frequency resolution setting indicating the number of frequency bands for which spatial parameters will be determined; a quantizer setting indicating the type of quantizer to be used to quantize the spatial metadata; and an indication of whether the current frame of the multichannel input signal will be encoded as an independent frame.

[0011] The parameter processing unit can be configured to determine whether the number of bits in a spatial metadata frame, determined according to one or more control settings, exceeds the maximum number of metadata bits. Furthermore, the parameter processing unit can be configured to reduce the number of bits in a particular spatial metadata frame if the number of bits in that particular spatial metadata frame exceeds the maximum number of metadata bits. This bit reduction can be performed in a resource-efficient manner. Specifically, this bit reduction can be performed without recalculating the entire spatial metadata frame.

[0012] As indicated above, a spatial metadata frame may include one or more sets of spatial parameters. The one or more control settings may include a time resolution setting indicating the number of spatial parameter sets to be determined by the parameter processing unit for each spatial metadata frame. The parameter processing unit may be configured to determine a plurality of spatial parameter sets for the current spatial metadata frame, as indicated by the time resolution setting. Typically, the time resolution setting is a value of 1 or 2. Furthermore, the parameter processing unit may be configured to discard spatial parameter sets from the current spatial metadata frame if the current spatial metadata frame includes multiple sets of spatial parameters, and if the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits. The parameter processing unit may be configured to retain at least one set of spatial parameters for each spatial metadata frame. By discarding spatial parameter sets from spatial metadata frames, the number of bits in the spatial metadata frame can be reduced with minimal computational effort and without significantly affecting the perceived listening quality of the multi-channel upmixed signal.

[0013] The one or more spatial parameter sets are typically associated with one or more corresponding sampling points. These sampling points can indicate one or more corresponding moments in time. Specifically, a sampling point can indicate the moment when the decoding system should fully utilize the corresponding spatial parameter set. In other words, a sampling point can indicate the moment when the corresponding spatial parameter set has been determined.

[0014] The parameter processing unit can be configured to discard a first set of spatial parameters from the current spatial metadata frame if multiple sampling points of the current metadata frame are not associated with transients in the multichannel input signal, wherein the first set of spatial parameters is associated with a first sampling point preceding a second sampling point. Conversely, the parameter processing unit can be configured to discard a second (typically the last) set of spatial parameters from the current spatial metadata frame if multiple sampling points of the current metadata frame are associated with transients in the multichannel input signal. By doing so, the parameter processing unit can be configured to reduce the impact of discarding spatial parameter sets on the listening quality of the multichannel upmixed signal.

[0015] The one or more control settings may include quantizer settings indicating a first type of quantizer among a plurality of predetermined types. The plurality of predetermined types of quantizers may each provide different quantizer resolutions. Specifically, the plurality of predetermined types of quantizers may include fine quantization and coarse quantization. The parameter processing unit may be configured to quantize one or more sets of spatial parameters of the current spatial metadata frame according to the first type of quantizer. Furthermore, the parameter processing unit may be configured to, if it is determined that the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits, requantize one, some, or all of the spatial parameters of the one or more sets of spatial parameters according to a second type of quantizer with a resolution lower than that of the first type of quantizer. By doing so, the number of bits in the current spatial metadata frame can be reduced, while only a limited degree of impact on the quality of the upmixed signal, and without significantly increasing the computational complexity of the audio coding system.

[0016] The parameter processing unit can be configured to determine the time difference parameter set based on the difference between the current spatial parameter set and the immediately preceding spatial parameter set. Specifically, the time difference parameters can be determined by determining the difference between the parameters of the current spatial parameter set and the corresponding parameters of the immediately preceding spatial parameter set. The spatial parameter set may include, for example, the parameters described in this document. α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , g , k 1 , k 2 Typically, parameters k 1 , k 2 Only one of them might need to be sent, because these parameters can be used in relation to... Related. To give just one example, only parameters... k 1 Parameters can be sent. k 2 This can be calculated at the receiver. The time difference parameter can be correlated with the difference between the corresponding parameters mentioned above.

[0017] The parameter processing unit can be configured to encode the set of time difference parameters using entropy coding (e.g., using Huffman codes). Furthermore, the parameter processing unit can be configured to insert the encoded set of time difference parameters into the current spatial metadata frame. Additionally, the parameter processing unit can be configured to reduce the entropy of the set of time difference parameters if it is determined that the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits. As a result, the number of bits required for entropy coding of the time difference parameters can be reduced, thereby reducing the number of bits used in the current spatial metadata frame. For example, the parameter processing unit can be configured to set one, some, or all of the time difference parameters in the set of time difference parameters to a value with an increased (e.g., highest) probability among the possible values ​​of the time difference parameters, in order to reduce the entropy of the set of time difference parameters. Specifically, the probability can be increased compared to the probability of the time difference parameters before the setting operation. Typically, the value with the highest probability among the possible values ​​of the time difference parameter corresponds to zero.

[0018] It should be noted that time difference coding of spatial parameter sets generally cannot be used for independent frames. Therefore, the parameter processing unit can be configured to verify whether the current spatial metadata frame is an independent frame; time difference coding is only applied if the current spatial metadata frame is not an independent frame. On the other hand, frequency difference coding described below can also be used for independent frames.

[0019] The one or more control settings may include a frequency resolution setting, wherein the frequency resolution setting indicates the number of different frequency bands for which respective spatial parameters (referred to as band parameters) will be determined. The parameter processing unit can be configured to determine different corresponding spatial parameters (band parameters) for different frequency bands. Specifically, different parameters for different frequency bands can be determined. α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , g , k 1 , k 2 The spatial parameter set can therefore include corresponding band parameters for different frequency bands. For example, the spatial parameter set can include parameters for... T Each frequency band T Each corresponding parameter T It is an integer, for example, T =7, 9, 12 or 15.

[0020] The parameter processing unit can be configured to determine a set of frequency difference parameters based on the difference between one or more band parameters in a first frequency band and corresponding one or more band parameters in an adjacent second frequency band. Furthermore, the parameter processing unit can be configured to encode the set of frequency difference parameters using entropy coding (e.g., based on Huffman codes). Additionally, the parameter processing unit can be configured to insert the encoded set of frequency difference parameters into the current spatial metadata frame. Furthermore, the parameter processing unit can be configured to reduce the entropy of the set of frequency difference parameters if it is determined that the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits. Specifically, the parameter processing unit can be configured to set one, some, or all of the frequency difference parameters in the set of frequency difference parameters to a value with an increased probability (e.g., zero) among the possible values ​​of the frequency difference parameters, in order to reduce the entropy of the set of frequency difference parameters. Specifically, the probability can be increased compared to the probability of the frequency difference parameters before the setting operation.

[0021] Alternatively or additionally, the parameter processing unit can be configured to reduce the number of frequency bands if it is determined that the number of bits in the current spatial metadata frame exceeds the maximum number of metadata bits. Furthermore, the parameter processing unit can be configured to use the reduced number of frequency bands to redetermine some or all of the sets of one or more spatial parameters for the current spatial metadata frame. Typically, the change in the number of frequency bands primarily affects higher frequency bands. As a result, the band parameters of one of several frequencies may be unaffected, making it possible for the parameter processing unit to avoid recalculating all band parameters.

[0022] As indicated above, the one or more external settings may include an update period, which indicates the time period required for the corresponding decoding system to synchronize with the bitstream. Furthermore, the one or more control settings may include an indication of whether the current spatial metadata frame will be encoded as an independent frame. The parameter processing unit may be configured to determine a spatial metadata frame sequence for the corresponding frame sequence of the multi-channel input signal. The configuration unit may be configured to determine one or more spatial metadata frames to be encoded as independent frames from the spatial metadata frame sequence based on the update period.

[0023] Specifically, the one or more independent spatial metadata frames may be determined such that an update period is satisfied (on average). For this purpose, the configuration unit may be configured to determine whether a current frame of a frame sequence of a multichannel input signal includes a sample at a moment that is an integer multiple of the update period (relative to the start point of the multichannel input signal). In addition, the configuration unit may be configured to determine that the current spatial metadata frame corresponding to the current frame is an independent frame (since it includes samples at moments that are integers of the update period). The parameter processing unit may be configured to encode one or more sets of spatial parameters of the current spatial metadata frame independently of the data included in the previous (and / or future) spatial metadata frames if the current spatial metadata frame is to be encoded as an independent frame. Generally, if the current spatial metadata frame is to be encoded as an independent frame, all sets of spatial parameters of the current spatial metadata frame are encoded independently of the data included in the previous (and / or future) spatial metadata frames.

[0024] According to another aspect, a parameter processing unit is described, which is configured to determine a spatial metadata frame for a frame for generating a multichannel upmix signal from a corresponding frame of a downmix signal. The downmix signal may include m channels, and the multichannel upmix signal may include n channels; n and m are integers, where m < n. As outlined above, the spatial metadata frame may include one or more sets of spatial parameters.

[0025] The parameter processing unit may include a transformation unit, which is configured to determine a plurality of spectra from a current frame and a following frame (referred to as a look-ahead frame) of channels of a multichannel input signal. The transformation unit may use a filter bank, for example, a QMF filter bank. The spectra in the plurality of spectra may include a predetermined number of transform coefficients in respective predetermined number of frequency bins. The plurality of spectra may be associated with respective plurality of time intervals (or moments). Thus, the transformation unit may be configured to provide a time / frequency representation of the current frame and the look-ahead frame. For example, both the current frame and the look-ahead frame may include K samples. The transformation unit may be configured to determine 2 * K / Q spectra, each spectrum including Q transform coefficients.

[0026] The parameter processing unit may include a parameter determination unit, which is configured to determine a spatial metadata frame for a current frame of channels of a multichannel input signal by weighting the plurality of spectra using a window function. The window function may be used to adjust the influence of the spectra in the plurality of spectra on a particular spatial parameter or a particular set of spatial parameters. For example, the window function may take values between 0 and 1.

[0027] The window function can depend on one or more of the following: the number of spatial parameter sets included in the spatial metadata frame, the presence of one or more transients in the current or immediately following frame of the multichannel input signal, and / or the timing of the transients. In other words, the window function can be modified based on the properties of the current frame and / or the forward-looking frame. Specifically, the window function used to determine the set of spatial parameters (which is called the set-dependent window function) can depend on the properties of the current frame and / or the forward-looking frame.

[0028] Thus, the window function can include set-dependent window functions. Specifically, the window function used to determine the spatial parameters of the spatial metadata frame can include one or more set-dependent window functions (or can be composed of these set-dependent window functions) for one or more sets of spatial parameters respectively. The parameter determination unit can be configured to determine the set of spatial parameters for the current frame (i.e., for the current spatial metadata frame) of the channels of the multi-channel input signal by weighting the plurality of spectra using set-dependent window functions. As outlined above, the set-dependent window function can depend on one or more properties of the current frame. Specifically, the set-dependent window function can depend on whether the set of spatial parameters is associated with transients.

[0029] For example, if the spatial parameter set is not associated with a transient, a set-dependent window function can be configured to provide a phase-in of the plurality of spectra from the sampling points of the previous spatial parameter set to the sampling points of the next spatial parameter set. The phase-in can be provided by a window function that transitions from 0 to 1. Alternatively or additionally, if the spatial parameter set is not associated with a transient, the set-dependent window function can include multiple spectra from the sampling points of the spatial parameter set to the spectra preceding the sampling points of the next spatial parameter set (either these spectra can be fully considered, or these spectra can be left unaffected), if the next spatial parameter set is associated with a transient. This can be achieved using a window function with a value of 1. Alternatively or additionally, if the spatial parameter set is not associated with a transient, the set-dependent window function can cancel out the plurality of spectra from the sampling points of the next spatial parameter set (either these spectra can be excluded, or these spectra can be attenuated), if the next spatial parameter set is associated with a transient. This can be achieved using a window function with a value of 0. Alternatively or additionally, if the spatial parameter set is not associated with the transient, a set-dependent window function can phase-out the plurality of spectra from the sampling point of the spatial parameter set up to the spectra preceding the sampling point of the subsequent spatial parameter set, if the subsequent spatial parameter set is not associated with the transient. The phase-out can be provided by a window function transitioning from 1 to 0. On the other hand, if the spatial parameter set is associated with the transient, a set-dependent window function can eliminate (or exclude, or attenuate) the spectra preceding the sampling point of the spatial parameter set. Alternatively or additionally, if the spatial parameter set is associated with the transient, a set-dependent window function can include (or leave unaffected) the spectra from the sampling point of the spatial parameter set up to the spectra preceding the sampling point of the subsequent spatial parameter set, and can eliminate (or exclude, or attenuate) the spectra starting from the sampling point of the subsequent spatial parameter set, if the sampling point of the subsequent spatial parameter set is associated with the transient. Alternatively or additionally, if the set of spatial parameters is associated with a transient, the set-related window function may include the spectrum of the plurality of spectra from the sampling point of the spatial parameter set up to the spectrum of the plurality of spectra at the end of the current frame (or may make these spectra unaffected), and may provide the fading of the spectrum of the plurality of spectra from the beginning of the immediately following frame up to the sampling point of the next set of spatial parameters (or may make these spectra gradually decay), if the next set of spatial parameters is not associated with the transient.

[0030] According to another aspect, a parameter processing unit is described that is configured to determine a spatial metadata frame for a frame for generating a multi-channel upmix signal from a corresponding frame of a downmix signal. The downmix signal may include m channels, and the multi-channel upmix signal may include n channels; n and m are integers, where m < n. As discussed above, the spatial metadata frame may include a set of spatial parameters.

[0031] As outlined above, the parameter processing unit may include a transformation unit. The transformation unit may be configured to determine a first plurality of transformation coefficients from a frame of a first channel of a multi-channel input signal. Additionally, the transformation unit may be configured to determine a second plurality of transformation coefficients from a corresponding frame of a second channel of the multi-channel input signal. The first channel and the second channel may be different. Thus, the first plurality of transformation coefficients and the second plurality of transformation coefficients provide a first time / frequency representation and a second time / frequency representation of the corresponding frames of the first channel and the second channel, respectively. As outlined above, the first time / frequency representation and the second time / frequency representation include a plurality of frequency bins and a plurality of time intervals.

[0032] Furthermore, the parameter processing unit may include a parameter determination unit that is configured to use fixed-point arithmetic to determine a set of spatial parameters based on the first plurality of transformation coefficients and the second plurality of transformation coefficients. As indicated above, the set of spatial parameters typically includes corresponding band parameters for different frequency bands, where the different frequency bands may include different numbers of frequency bins. The specific band parameter for a particular frequency band may be determined based on the transformation coefficients among the first plurality of transformation coefficients and the second plurality of transformation coefficients for that particular frequency band (generally, not considering the transformation coefficients of other frequency bands). The parameter determination unit may be configured to determine the shift used by the fixed-point arithmetic for determining the specific band parameter that depends on a particular frequency band. In particular, the shift used by the fixed-point arithmetic for determining the specific band parameter for a particular frequency band may depend on the number of frequency bins included within that particular frequency band. Alternatively or additionally, the shift used by the fixed-point arithmetic for determining the specific band parameter for a particular frequency band may depend on the number of time intervals that will be considered for determining the specific band parameter.

[0033] The parameter determination unit may be configured to determine the shift for a particular frequency band such that the accuracy of the specific band parameter is maximized. This may be achieved by determining the shift required for each multiplication and addition operation of the determination process of the specific band parameter.

[0034] The parameter determination unit may be configured to determine a first energy (or energy estimate) by based on the transformation coefficients among the first plurality of transformation coefficients that fall within a particular frequency band p for determining the first energy (or energy estimate) E 1,1 ( p ) for a particular frequency band pSpecific parameters with arguments. Additionally, a second energy (or energy estimate) can be determined based on transform coefficients among the second plurality of transform coefficients that fall within a specific frequency band p among the transform coefficients E 2,2 ( p ). Additionally, a cross product or covariance can be determined based on transform coefficients among the first plurality of transform coefficients and the second plurality of transform coefficients that fall within a specific frequency band p among the transform coefficients E 1,2 ( p ). The parameter determination unit can be configured to determine a shift for a specific frequency band parameter E 1,1 ( p ) based on the maximum value among the absolute values of a first energy estimate E 2,2 ( p ), a second energy estimate E 1,2 ( p ) and a covariance p for the specific frequency band parameter z p .

[0035] ]>According to another aspect, an audio coding system is described that is configured to generate a bitstream that indicates a sequence of frames of a downmixed signal and a corresponding sequence of spatial metadata frames for generating a corresponding sequence of frames of a multichannel upmixed signal from the sequence of frames of the downmixed signal. The system can include a downmix processing unit that is configured to generate a sequence of frames of a downmixed signal from a corresponding sequence of frames of a multichannel input signal. As indicated above, the downmixed signal can include m channels, and the multichannel input signal can include n channels; n, m are integers, where m < n. Additionally, the audio coding system can include a parameter processing unit that is configured to determine a sequence of spatial metadata frames from the sequence of frames of the multichannel input signal.

[0036] Additionally, an audio coding system may include a bitstream generation unit configured to generate a bitstream including a sequence of bitstream frames, where the bitstream frames indicate frames of a downmixed signal corresponding to a first frame of a multichannel input signal and spatial metadata frames corresponding to a second frame of the multichannel input signal. The second frame may be different from the first frame. Specifically, the first frame may be in front of the second frame. By doing so, the spatial metadata frame for the current frame can be sent together with the frame of the subsequent frame. This ensures that the spatial metadata frame arrives at the corresponding decoding system only when it is needed. The decoding system typically decodes the current frame of the downmixed signal and generates a decorrelated frame based on the current frame of the downmixed signal. This process introduces algorithmic delay, and by delaying the spatial metadata frame for the current frame, it is ensured that the spatial metadata frame arrives at the decoding system only once the decoded current frame and the decorrelated frame are provided. As a result, the processing power and memory requirements of the decoding system can be reduced.

[0037] In other words, an audio coding system is described that is configured to generate a bitstream based on a multichannel input signal. As outlined above, the system may include a downmix processing unit configured to generate a sequence of frames of a downmixed signal from a corresponding sequence of first frames of the multichannel input signal. The downmixed signal may include m channels, and the multichannel input signal may include n channels; n, m are integers, where m < n. Additionally, the audio coding system may include a parameter processing unit configured to generate a sequence of spatial metadata frames from a second sequence of frames of the multichannel input signal. The sequence of frames of the downmixed signal and the sequence of spatial metadata frames may be used by a corresponding decoding system to generate a multichannel upmixed signal including n channels.

[0038] The audio coding system may further include a bitstream generation unit configured to generate a bitstream including a sequence of bitstream frames, where the bitstream frames may indicate frames of the downmixed signal corresponding to a first frame of a first sequence of frames of the multichannel input signal and spatial metadata frames corresponding to a second frame of a second sequence of frames of the multichannel input signal. The second frame may be different from the first frame. In other words, the framing for determining the spatial metadata frames and the framing for determining the frames of the downmixed signal may be different. As outlined above, different framings may be used to ensure data alignment at the corresponding decoding system.

[0039] The first and second frames typically consist of the same number of samples (e.g., 1536 samples). Some of the samples in the first frame may precede those in the second frame. Specifically, the first frame may precede the second frame by a predetermined number of samples. This predetermined number of samples may, for example, correspond to a small fraction of the total number of samples in the frame. For instance, the predetermined number of samples may correspond to 50% or more of the total number of samples in the frame. In a particular example, the predetermined number of samples corresponds to 928 samples. As illustrated in this document, this specific number of samples provides minimal total latency and optimal alignment for a particular implementation of the audio encoding and decoding system.

[0040] According to another aspect, an audio coding system is described, configured to generate a bitstream based on a multichannel input signal. The system may include a downmixing processing unit configured to determine a clip protection gain (also referred to in this document as clip-gain and / or DRC2 parameters) sequence for a corresponding frame sequence of the multichannel input signal. The current clip protection gain may indicate the current frame to be applied to the multichannel input signal to prevent attenuation of the corresponding current frame clipping of the downmixed signal. Similarly, the clip protection gain sequence may indicate the frames to be applied to the frame sequence of the multichannel input signal to prevent respective attenuation of the corresponding frame clipping of the frame sequence of the downmixed signal.

[0041] The downmixing unit can be configured to interpolate the current trim protection gain and the previous trim protection gain of the previous frame of the multichannel input signal to obtain a trim protection gain curve. This can be performed in a manner similar to a trim protection gain sequence. Furthermore, the downmixing unit can be configured to apply the trim protection gain curve to the current frame of the multichannel input signal to obtain the current frame of the attenuated multichannel input signal. Again, this can be performed in a manner similar to a frame sequence of the multichannel input signal. Additionally, the downmixing unit can be configured to generate the current frame of the downmixed signal frame sequence from the current frame of the attenuated multichannel input signal. Similarly, a frame sequence of the downmixed signal can be generated.

[0042] The audio processing system may further include a parameter processing unit configured to determine a spatial metadata frame sequence from the multichannel input signal. The downmixed signal frame sequence and the spatial metadata frame sequence can be used to generate a multichannel upmixed signal comprising n channels, such that the multichannel upmixed signal is an approximation of the multichannel input signal. Additionally, the audio processing system may include a bitstream generation unit configured to generate a bitstream indicating a trimmed guard gain sequence, a downmixed signal frame sequence, and a spatial metadata frame sequence, enabling a corresponding decoding system to generate the multichannel upmixed signal.

[0043] The trim protection gain curve may include a transition segment and a flat segment. The transition segment provides a smooth transition from the previous trim protection gain to the current trim protection gain, while the flat segment remains flat at the current trim protection gain. The transition segment may extend across a predetermined number of samples of the current frame of the multichannel input signal. The predetermined number of samples may be more than one but less than the total number of samples of the current frame of the multichannel input signal. Specifically, the predetermined number of samples may correspond to a sampling block (where a frame may include multiple blocks) or a frame. In a particular example, a frame may include 1536 samples, and a block may include 256 samples.

[0044] According to another aspect, an audio coding system is described, configured to generate a bitstream indicating a downmixed signal and spatial metadata for generating a multichannel upmixed signal from the downmixed signal. The system may include a downmixing processing unit configured to generate the downmixed signal from a multichannel input signal. Furthermore, the system may include a parameter processing unit configured to determine a spatial metadata frame sequence for a corresponding frame sequence of the multichannel input signal.

[0045] Furthermore, the audio encoding system may include a configuration unit configured to determine one or more control settings for the parameter processing unit based on one or more external settings. The one or more external settings may include an update period indicating the time period required for the corresponding decoding system to synchronize with the bitstream. The configuration unit may be configured to determine one or more independent spatial metadata frames to be encoded independently from a sequence of spatial metadata frames based on the update period.

[0046] According to another aspect, a method for generating a bitstream is described, the bitstream indicating a downmixing signal and spatial metadata for generating a multi-channel upmixing signal from the downmixing signal. The method can generate the downmixing signal from a multi-channel input signal. Furthermore, the method may include determining one or more control settings based on one or more external settings; wherein the one or more external settings include a target data rate for the bitstream, and wherein the one or more control settings include a maximum data rate for the spatial metadata. Additionally, the method may include determining the spatial metadata from the multi-channel input signal according to the one or more control settings.

[0047] According to another aspect, a method for determining a spatial metadata frame for generating a multichannel upmix signal from a corresponding frame of an downmix signal is described. The method may include determining multiple spectra from the current frame and the immediately following frame of a channel of the multichannel input signal. Furthermore, the method may include weighting the multiple spectra using a window function to obtain multiple weighted spectra. Additionally, the method may include determining a spatial metadata frame for the current frame of the channel of the multichannel input signal based on the multiple weighted spectra. The window function may depend on one or more of the following: the number of spatial parameter sets included within the spatial metadata frame, the presence of a transient in the current frame or the immediately following frame of the multichannel input signal, and / or the timing of that transient.

[0048] According to another aspect, a method for determining a spatial metadata frame for generating a multichannel upmix signal frame from a corresponding frame of an downmix signal is described. The method may include: determining a first plurality of transform coefficients from a frame of a first channel of the multichannel input signal, and determining a second plurality of transform coefficients from a corresponding frame of a second channel of the multichannel input signal. As outlined above, the first plurality of transform coefficients and the second plurality of transform coefficients typically provide a first time / frequency representation and a second time / frequency representation of the corresponding frames of the first and second channels, respectively. The first time / frequency representation and the second time / frequency representation may include a plurality of frequency intervals and a plurality of time intervals. The set of spatial parameters may include corresponding band parameters for different frequency bands comprising different numbers of frequency intervals. The method may further include determining a shift to be applied when determining a specific band parameter for a specific frequency band using fixed-point arithmetic. Furthermore, the shift may be determined based on the number of time intervals to be considered when determining the specific band parameter. Additionally, the method may include determining the specific band parameter using fixed-point arithmetic and the determined shift, based on the first plurality of transform coefficients and the second plurality of transform coefficients falling within the specific frequency band.

[0049] A method for generating a bitstream based on a multichannel input signal is described. The method may include generating a frame sequence of a downmixer from a corresponding first frame sequence of the multichannel input signal. Furthermore, the method may include determining a spatial metadata frame sequence from a second frame sequence of the multichannel input signal. The frame sequence of the downmixer and the spatial metadata frame sequence can be used to generate a multichannel upmixer. Additionally, the method may include generating a bitstream comprising a bitstream frame sequence. The bitstream frame may indicate a frame of the downmixer corresponding to a first frame of the first frame sequence of the multichannel input signal and a spatial metadata frame corresponding to a second frame of the second frame sequence of the multichannel input signal. The second frame may be different from the first frame.

[0050] According to another aspect, a method for generating a bitstream based on a multichannel input signal is described. The method may include determining a trim protection gain sequence for a corresponding frame sequence of the multichannel input signal. The current trim protection gain may indicate that the current frame of the multichannel input signal will be applied to prevent attenuation of the corresponding current frame of the downmixing signal. The method may continue to interpolate the current trim protection gain and a previous trim protection gain of the previous frame of the multichannel input signal to obtain a trim protection gain curve. Furthermore, the method may include applying the trim protection gain curve to the current frame of the multichannel input signal to obtain a current frame of attenuation of the multichannel input signal. The current frame of the downmixing signal's frame sequence can be generated from the current frame of the attenuation of the multichannel input signal. Additionally, the method may include determining a spatial metadata frame sequence from the multichannel input signal. The frame sequence of the downmixing signal and the spatial metadata frame sequence can be used to generate a multichannel upmixing signal. The bitstream may be generated such that it indicates the trim protection gain sequence, the frame sequence of the downmixing signal, and the spatial metadata frame sequence, enabling the generation of a multichannel upmixing signal based on the bitstream.

[0051] According to another aspect, a method for generating a bitstream is described, the bitstream indicating a downmixing signal and spatial metadata, the spatial metadata being used to generate a multi-channel upmixing signal from the downmixing signal. The method may include generating the downmixing signal from a multi-channel input signal. Furthermore, the method may include determining one or more control settings based on one or more external settings, wherein the one or more external settings include an update period indicating a time period required for the decoding system to synchronize with the bitstream. The method may also include determining a spatial metadata frame sequence for a corresponding frame sequence of the multi-channel input signal based on the one or more control settings. Additionally, the method may include encoding one or more spatial metadata frames in the spatial metadata frame sequence as independent frames according to the update period.

[0052] According to another aspect, a software program is described. This software program is adapted to execute on a processor, and is adapted to perform the method steps outlined in this document when executed on a processor.

[0053] According to another aspect, a storage medium is described. This storage medium may include a software program adapted to execute on a processor and, when executed on a processor, to perform the method steps outlined in this document.

[0054] According to another aspect, a computer program product is described. This computer program product may include executable instructions for performing the method steps outlined in this document when executed on a computer.

[0055] It should be noted that the methods and systems, including the preferred embodiments outlined in this patent application, can be used independently or in combination with other methods and systems disclosed in this document. Furthermore, all aspects of the methods and systems outlined in this patent application can be combined arbitrarily. Specifically, the features of the claims can be combined with each other in any manner. Attached Figure Description

[0056] The invention will now be described by way of example with reference to the accompanying drawings, wherein... Figure 1 A generalized block diagram of an example audio processing system for performing spatial synthesis is shown; Figure 2 Show Figure 1 Example details of the system; Figure 3 Similar to Figure 1 An example audio processing system for performing spatial synthesis is shown; Figure 4 An example audio processing system for performing spatial analysis is shown; Figure 5a A block diagram of an example parametric multichannel audio coding system is shown. Figure 5b A block diagram of an example spatial analysis and coding system is shown; Figure 5c Example time-frequency representation of frames for channels of a multichannel audio signal; Figure 5d Example time-frequency representation of multiple channels in a multi-channel audio signal; Figure 5e Show Figure 5b Example windowing applied to the transform unit of the spatial analysis and coding system shown; Figure 6 A flowchart illustrating an example method for reducing the data rate of spatial metadata; Figure 7a This example illustrates a transition scheme for spatial metadata that is executed at the decoding system. Figures 7b to 7d This example demonstrates a window function used to determine spatial metadata. Figure 8 A block diagram illustrating an example processing path for a parametric multichannel codec system; Figure 9a and 9b A block diagram of an example parametric multichannel audio coding system configured to perform trim protection and / or dynamic range control is shown. Figure 10 Example methods for compensating DRC parameters are shown; and Figure 11An example interpolation curve is shown for use in trimming protection. Detailed Implementation

[0057] As outlined in the introduction, this document relates to multichannel audio coding systems using parametric multichannel representation. The following describes an example multichannel audio coding and decoding (encoding / decoding) system. Figures 1 to 3 In the context of [the above], it describes how a decoder of an audio codec system can use the received parameterized multichannel representation to extract the received m-channel downmixed signal. X (For example, m=2) Generate an n-channel upmix signal Y (Typically, n>2). Next, the encoder-related processing of the multi-channel audio codec system is described. Specifically, it describes how a parameterized multi-channel representation and an m-channel downmix signal can be generated from an n-channel input signal.

[0058] Figure 1 Example configured to use a submixer signal X The mixing parameter set generates an upmixed signal. Y A block diagram of an example audio processing system 100. Specifically, the audio processing system 100 is configured to be based solely on the downmixed signal. X The set of mixing parameters generates an upmixed signal. The downmixed signal is extracted from the bitstream P by the audio decoder 140. X =[ l 0 r 0] T and a set of mixed parameters. In the illustrated example, the set of mixed parameters includes parameters α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , g , k 1 , k 2 The mixing parameters can be included in the individual mixing parameter data fields of the bitstream P in the form of quantization and / or entropy encoding. The mixing parameters can be referred to as metadata (or spatial metadata), which, along with the encoded downmixing signal,... X They are transmitted together. In some instances of this disclosure, it has been explicitly indicated that some connecting lines are adapted to transmit multi-channel signals, wherein these lines are provided as crossover lines adjacent to each number of channels. Figure 1 In the system 100 shown, the downmixed signal XIncludes m=2 channels, and the upmix signal will be defined below. Y Includes n=6 channels (e.g., 5.1 channels).

[0059] The upmixing stage 110, whose operation is parameterized to the mixing parameters, receives the downmixing signal. The downmixing modification processor 120 modifies the downmixing signal through nonlinear processing and by forming a linear combination of downmixing channels to obtain the modified downmixing signal. D =[ d 1 d 2] T The first mixing matrix 130 receives the downmixed signal. X and modified downmixing signal D And output the upmixed signal by forming the following linear combination. Y =[ l f l s r f r s c lfe ] T :

[0060] In the above linear combinations, the mixing parameters α 3 Control the intermediate type signal formed from the undermixed signal (and) l 0+ r The contribution of 0 (proportional to 0) to all channels in the overmix signal. Mixing parameters. β 3 Control side type signal (and) l 0- r The contribution of the mixing parameters (proportional to 0) to all channels in the overmix signal. Therefore, in practical applications, it is reasonable to expect that the mixing parameters will... α 3 and β 3 They will have different statistical properties, which allows for more efficient encoding. (For comparison, consider the reference parameterization (where independent mixing parameters control the left and right channel contributions of the downmix to the spatial left and right channels of the upmix), note that the statistical observables of such mixing parameters may not be significantly different.)

[0061] Returning to the linear combination shown in the equations above, further note the gain parameter. k 1 , k 2It can depend on a single, shared mixing parameter in the bitstream P. Furthermore, the gain parameter can be normalized to make... .

[0062] The contributions of the modified downmixer to the spatial left and spatial right channels of the upmixer can be determined by parameters. β 1 (The contribution of the first modified channel to the left channel) and β 2 (The contribution of the second modified channel to the right channel) is controlled. Furthermore, the contribution of each channel in the downmix to its spatially corresponding channel in the upmix can be individually controlled by changing the independent mixing parameters. g Control. Preferably, the gain parameter g It is quantized unevenly to avoid large quantization errors.

[0063] Now refer to another source Figure 2 The downmixing modifier 120 can perform the following linear combination (which is crossmixing) of the downmixing channels in the second mixing matrix 121:

[0064] As indicated by the formula, the gain of filling the second mixing matrix can be parameterized to depend on some of the mixing parameters encoded in the bitstream P. The processing performed by the second mixing matrix 121 yields the intermediate signal. Z =[ z 1 z 2] T The intermediate signal is supplied to the decorrelation unit 122. Figure 1 An example is shown where the decorcorrelator 122 includes two sub-decorcorrelators 123 and 124, which can be configured identically (i.e., providing the same output in response to the same input) or configured differently. As an alternative, Figure 2 An example is shown where all the solution-related operations are performed by a single unit 122, which outputs a preliminarily modified downmixed signal. D '. Figure 2 The downmixing modification processor 120 may also include an artifact attenuator 125. In an example embodiment, as outlined above, the artifact attenuator 125 is configured to detect intermediate signals. Z The signal is corrected by attenuating unwanted artifacts in the signal based on the detected tail note's position. This attenuation generates a modified downmixed signal. D This signal is output from the lower mixing modification processor 120.

[0065] Figure 3 It shows the relationship with Figure 1A first mixing matrix 130 of a similar type is shown, along with its associated transform stages 301, 302 and inverse transform stages 311, 312, 313, 314, 315, 316. The transform stages may include, for example, filter banks, such as quadrature mirror filter banks (QMF). Therefore, signals upstream of transform stages 301, 302 are represented in the time domain, just as signals downstream of inverse transform stages 311, 312, 313, 314, 315, 316 are represented in the frequency domain. The time dependence of other signals can be expressed, for example, as block values ​​or discrete values ​​related to the time blocks to which the signal is segmented. Note that... Figure 3 Using alternative notation compared to the matrix equations above; one that can, for example, have a correspondence. X L0 ~ l 0、 X R0 ~ r 0、 Y L ~ l f , Y LS ~ l S Etc. Furthermore, Figure 3 The notation in the text emphasizes the time-domain representation of the signal. X L0 ( t ) and the frequency domain representation of the same signal X L0 ( f The difference between them is that the frequency domain representation is divided into time frames; therefore, it is a function of both time and frequency variables.

[0066] Figure 4 An audio processing system 400 is shown, which is used to generate a downmixed signal. X And the mixing parameters applied to control the gain of the upper mixing stage 110. α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , g , k 1 , k 2 The audio processing system 400 is typically located on the encoder side, for example, in broadcast or recording equipment, while Figure 1The system 100 shown will typically be deployed on the decoder side, for example, in a playback device. The downmixer 410 generates an m-channel signal based on the n-channel signal Y. X Preferably, the downmixer 410 operates on the time-domain representation of these signals. The parameter extractor 420 can analyze the n-channel signals. Y Furthermore, the quantitative and qualitative properties of the mixed stage 410 should be considered to generate the mixing parameters. α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , g , k 1 , k 2 The value of the mixed parameter can be as follows: Figure 4 The notation in the diagram indicates a vector of frequency block values, which can be further subdivided into time blocks. In the example implementation, the downmixing stage 410 is time-invariant and / or frequency-invariant. Due to its time and / or frequency invariance, a communication connection is typically not required between the downmixing stage 410 and the parameter extractor 420, but parameter extraction can be performed independently. This provides considerable freedom for implementation. It also allows for the possibility of reducing the overall system latency, as several processing steps can be executed in parallel. As an example, the Dolby Digital Plus format (or Enhanced AC-3) can be used for downmixing signals. X Encode it.

[0067] The parameter extractor 420 can learn about the quantitative and / or qualitative properties of the downmixing stage 410 by accessing a downmixing specification, which may specify one of the following: a set of gain values, an index identifying a predefined downmixing mode for its predefined gain, etc. The downmixing specification may be data pre-loaded into the memory of each of the downmixing stage 410 and the parameter extractor 420. Alternatively or additionally, the downmixing specification can be sent from the downmixing stage 410 to the parameter extractor 420 via a communication line connecting these units. Alternatively, each of the downmixing stages 410 to the parameter extractor 420 may access the downmixing specification from a common data source, such as one in an audio processing system or related to the input signal. Y In the associated metadata stream (e.g., Figure 5a The memory of the configuration unit 540 shown.

[0068] Figure 5aShows an example multichannel encoding system 500 for encoding a multichannel audio input signal X using a downmixed signal Y (including m channels, where m < n) and a parametric representation. The system 500 includes a downmix encoding unit 510, which includes a downmix stage 410 such as Figure 4 . The downmix encoding unit 510 can be configured to provide an encoded version of the downmixed signal X . The downmix encoding unit 510 can encode the downmixed signal X using, for example, a Dolby Digital Plus encoder. In addition, the system 500 includes a parametric encoding unit 510, which can include a parameter extractor 420 such as Figure 4 . The parametric encoding unit 510 can be configured to quantize and encode the set of mixing parameters α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , g , k 1 (also referred to as spatial parameters) to obtain encoded spatial parameters 562. As indicated above, the parameter k 2 can be determined from the parameter k [[ID=4U]] 1 . Additionally, the system 500 can include a bitstream generation unit 530, which is configured to generate a bitstream P 564 from the encoded downmixed signal 563 and the encoded spatial parameters 562. The bitstream 564 can be encoded according to a predetermined bitstream syntax. Specifically, the bitstream 564 can be encoded in a format compliant with Dolby Digital Plus (DD+ or E-AC-3, Enhanced AC-3). [[ID=4Z]]

[0069] System 500 may include a configuration unit 540 configured to determine one or more control settings 552, 554 for parameter encoding unit 520 and / or downmixing encoding unit 510. The one or more control settings 552, 554 may be determined based on one or more external settings 551 of system 500. For example, the one or more external settings 551 may include the total (maximum or fixed) data rate of bitstream 564. Configuration unit 540 may be configured to determine one or more control settings 552 based on the one or more external settings 551. The one or more control settings 552 for parameter encoding unit 520 may include one or more of the following: The maximum data rate for the encoded spatial parameter 562. This control setting is referred to as the metadata data rate setting in this document.

[0070] The maximum and / or specific number of parameter sets determined by parameter encoding unit 520 for each frame of audio signal 561. This control setting is referred to herein as the temporal resolution setting because it allows for the temporal resolution to be affected by the spatial parameters.

[0071] The parameter encoding unit 520 will determine the number of parameter bands for the spatial parameters. This control setting is referred to herein as the frequency resolution setting because it allows the frequency resolution of the spatial parameters to be affected.

[0072] The resolution of the quantizer used to quantize spatial parameters. This control setting is referred to as the quantizer setting in this document.

[0073] The parameter encoding unit 520 may use one or more of the control settings 552 mentioned above for determining and / or encoding the spatial parameters to be included in the bitstream 564. Typically, the input audio signal... Y 561 was divided into a sequence of frames, where each frame includes the input audio signal. Y A predetermined number of samples 561. The metadata data rate setting can indicate the maximum number of bits available for encoding the spatial parameters of frames of the input audio signal 561. The actual number of bits used to encode the spatial parameters 562 of the frames can be less than the number of bits allocated by the metadata data rate setting. The parameter encoding unit 520 can be configured to notify the configuration unit 540 about the actual number of bits used 553, thereby enabling the configuration unit 540 to determine the number of bits available for encoding the downmixed signal. XThe number of bits to be encoded. This number of bits can be transmitted to the downmixing encoding unit 510 as a control setting 554. The downmixing encoding unit 510 can be configured (e.g., using a multi-channel encoder, such as Dolby Digital Plus) to downmix the signal based on the control setting 554. X Encoding is then performed. In this way, bits that have not yet been used to encode spatial parameters can be used to encode the downmixed signal.

[0074] Figure 5b A block diagram of an example parameter encoding unit 520 is shown. The parameter encoding unit 520 may include a transformation unit 521 configured to determine the frequency representation of the input signal 561. Specifically, the transformation unit 521 may be configured to transform frames of the input signal 561 into one or more spectra, each spectrum comprising multiple frequency intervals. For example, the transformation unit 521 may be configured to apply a filter bank (e.g., a QMF filter bank) to the input signal 561. The filter bank may be a critical sampling filter bank. The filter bank may include a predetermined number of Q filters (e.g., Q = 64 filters). Thus, the transformation unit 521 may be configured to determine Q sub-band signals from the input signal 561, wherein each sub-band signal is associated with a corresponding frequency interval 571. For example, K sampled frames of the input signal 561 may be transformed into Q sub-band signals, wherein each sub-band signal has K / Q frequency coefficients. In other words, K sampled frames of the input signal 561 are transformed into K / Q spectra, wherein each spectrum comprises Q frequency intervals. In a specific example, the frame length is K=1536, the number of frequency ranges is Q=64, and the number of spectrums is K / Q=24.

[0075] The parameter encoding unit 520 may include a banding unit 522 configured to group one or more frequency ranges 571 into frequency bands 572. The grouping of frequency ranges 571 to frequency bands 572 may depend on a frequency resolution setting 552. Table 1 illustrates example mappings of frequency ranges 571 to frequency bands 572, which may be applied by the banding unit 522 based on the frequency resolution setting 552. In the illustrated example, the frequency resolution setting 552 may indicate banding of frequency ranges 571 to 7, 9, 12, or 15 frequency bands. Banding typically models the psychoacoustic behavior of the human ear. As a result, the number of frequency ranges 571 for each frequency band 572 typically increases with increasing frequency.

[0076]

[0077] Table 1

[0078] The parameter determination unit 523 of the parameter encoding unit 520 (and specifically, the parameter extractor 420) can be configured to determine one or more sets of mixed parameters for each frequency band 572. α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , g , k 1 , k 2 Therefore, band 572 can also be referred to as the parameter band. The hybrid parameters used for band 572... α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , g , k 1 , k 2 This can be referred to as band parameters. Thus, the entire set of hybrid parameters typically includes band parameters for each 572 frequency band. Band parameters can be applied to... Figure 3 The mixing matrix 130 is used to determine the sub-band version of the decoded overmixed signal.

[0079] The number of mixed parameter sets for each frame determined by the parameter determination unit 523 can be indicated by the temporal resolution setting 552. For example, the temporal resolution setting 552 can indicate that one or two mixed parameter sets will be determined for each frame.

[0080] Figure 5c The example illustrates the determination of a mixed parameter set including parameters for multiple frequency bands 572. Figure 5c An example set of transform coefficients 580 derived from a frame of input signal 561 is illustrated. Transform coefficients 580 correspond to a specific time 582 and a specific frequency band 571. Frequency band 572 may include multiple transform coefficients 580 from one or more frequency bands 571. Figure 5c As can be seen, the transformation of the time-domain sampling of the input signal 561 provides a time-frequency representation of the frames of the input signal 561.

[0081] It should be noted that the set of hybrid parameters for the current frame can be determined based on the transform coefficients 580 of the current frame and possibly also based on the transform coefficients 580 of the immediately following frame (which is also referred to as the look-ahead frame).

[0082] The parameter determination unit 523 can be configured to determine the hybrid parameters for each frequency band 572. α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , g , k 1 , k 2 If the time resolution setting is set to 1, all transform coefficients 580 of a specific frequency band 572 (the current frame and the forward-looking frame) can be considered to determine the mixing parameters for the specific frequency band 572. Alternatively, the parameter determination unit 523 can be configured to determine two sets of mixing parameters for each frequency band 572 (e.g., when the time resolution setting is set to 2). In this case, the first half-time of the transform coefficients 580 of the specific frequency band 572 (corresponding to, for example, the transform coefficients 580 of the current frame) can be used to determine the first set of mixing parameters, while the second half-time of the transform coefficients 580 of the specific frequency band 572 (corresponding to, for example, the transform coefficients 580 of the forward-looking frame) can be considered to determine the second set of mixing parameters.

[0083] Generally, the parameter determination unit 523 can be configured to determine one or more sets of mixed parameters based on the transform coefficients 580 of the current frame and the forward-looking frame. A window function can be used to limit the effect of the transform coefficients 580 on the one or more sets of mixed parameters. The shape of the window function can depend on the number of sets of mixed parameters for each frequency band 572 and / or the nature of the current frame and / or the forward-looking frame (e.g., the presence of one or more transients). Figure 5e and Figures 7b to 7d The example window function is described in the context of [the example window function].

[0084] It should be noted that the above applies to cases where the frame of input signal 561 does not include a transient signal portion. System 500 (e.g., parameter determination unit 523) can be configured to perform transient detection based on input signal 561. In the event that one or more transients are detected, one or more transient indicators 583, 584 can be set, wherein transient indicators 583, 584 can identify the time 582 of the corresponding transient. Transient indicators 583, 584 can also be referred to as sampling points of each mixed parameter set. In the case of a transient, parameter determination unit 523 can be configured to determine the mixed parameter set based on the transform coefficients 580 starting from the time of the transient (this is determined by...). Figure 5c (Examples of areas with different shaded lines). On the other hand, the transformation coefficient 580 before the transient moment can be ignored, thus ensuring that the set of mixing parameters reflects the multi-channel situation after the transient.

[0085] Figure 5c Multi-channel input signals are illustrated. Y The transformation coefficients 580 for the channels of the multichannel input signal 561. The parameter encoding unit 520 is typically configured to determine the transformation coefficients 580 for the multiple channels of the multichannel input signal 561. Figure 5d Example transform coefficients for the first 561-1 channel and the second 561-2 channel of the input signal 561 are shown. Frequency band p 572 includes frequency index i to j The frequency range is 571. The first channel is 561-1 at time (or in the frequency spectrum). q In the frequency range i The transformation coefficient 580 in the figure can be called a q,i In a similar manner, the second channel 561-2 at time (or in the spectrum) q In the frequency range i The transformation coefficient 580 in the middle can be called b q,i The transform coefficient 580 can be a complex number. Used for frequency band. p Determining the mixing parameters may involve determining the energy and / or covariance of the first channel 561-1 and the second channel 561-2 based on the transform coefficients 580. For example, the first channel 561-1 and the second channel 561-2 in the frequency band... p In the middle, regarding the time interval [ q , v The covariance of the transformation coefficients 580 can be determined as:

[0086] The first channel 561-1 is in the frequency band p In the middle, regarding the time interval [q , v The energy estimate of the transformation coefficient 580 can be determined as follows:

[0087] The second channel 561-2 is in the frequency band p In the middle, regarding the time interval [ q , v Energy estimation of transformation coefficient 580. E 2,2 ( p ) can be determined in a similar way.

[0088] Thus, the parameter determination unit 523 can be configured to determine one or more band parameter sets 573 for different frequency bands 572. The number of frequency bands 572 typically depends on the frequency resolution setting 552, while the number of mixed parameter sets for each frame typically depends on the temporal resolution setting 552. For example, the frequency resolution setting 552 may indicate the use of 15 frequency bands 572, while the temporal resolution setting 552 may indicate the use of 2 mixed parameter sets. In this case, the parameter determination unit 523 can be configured to determine two temporally different mixed parameter sets, wherein each mixed parameter set includes 15 band parameter sets 573 (i.e., mixed parameters for different frequency bands 572).

[0089] As indicated above, the blending parameters for the current frame can be determined based on the transform coefficients 580 of the current frame and the transform coefficients 580 of the following preceding frame. The parameter determination unit 523 can apply a window to the transform coefficients 580 to ensure a smooth transition between the blending parameters of successive frames in the frame sequence, and / or to account for destructive portions (e.g., transients) within the input signal 561. This is in Figure 5e is exemplified in Figure 5e The K / Q spectra 589 of the current frame 585 and the immediately following frame 590 of the input audio signal 561 at corresponding K / Q consecutive moments 582 are shown. Furthermore, Figure 5e An example window 586 used by parameter determination unit 523 is shown. Window 586 reflects the effect of the K / Q spectra 589 of the current frame 585 and the immediately following frame 590 (referred to as the foreground frame) on the mixed parameters. As will be outlined in more detail below, window 586 reflects the case where the current frame 585 and the foreground frame 590 do not include any transients. In this case, window 586 ensures smooth inflation and deflation of the spectra 589 of the current frame 585 and the foreground frame 590, respectively, thereby allowing for smooth evolution of spatial parameters. Furthermore, Figure 5eExample windows 587 and 588 are shown. The dashed window 587 reflects the effect of the K / Q spectra 589 of the current frame 585 on the mixing parameters of the previous frame. Additionally, the dashed window 588 reflects the effect of the K / Q spectra 589 of the immediately following frame 590 on the mixing parameters of the immediately following frame 590 (in the case of smooth interpolation).

[0090] The one or more sets of mixed parameters can then be quantized and encoded using the encoding unit 524 of the parameter encoding unit 520. The encoding unit 524 can apply various encoding schemes. For example, the encoding unit 524 can be configured to perform differential encoding of the mixed parameters. Differential encoding can be based on a time difference (for the same frequency band 572, the time difference between the current mixed parameter and the corresponding previous mixed parameter) or a frequency difference (the frequency difference between the current mixed parameter of the first frequency band 572 and the corresponding current mixed parameter of the adjacent second frequency band 572).

[0091] Furthermore, encoding unit 524 can be configured to quantize the set of mixed parameters and / or the time difference or frequency difference of the mixed parameters. The quantization of the mixed parameters can depend on quantizer setting 552. For example, quantizer setting 552 can take two values, a first value indicating fine quantization and a second value indicating coarse quantization. Thus, encoding unit 524 can be configured to perform fine quantization (with relatively low quantization error) or coarse quantization (with relatively increased quantization error) based on the quantization type indicated by quantizer setting 552. The quantized parameters or parameter differences can then be encoded using entropy-based codes (such as Huffman codes). As a result, encoded spatial parameters 562 are obtained. The number of bits 553 used for encoding spatial parameters 562 can be transmitted to configuration unit 540.

[0092] In an embodiment, encoding unit 524 can be configured to first quantize different mixing parameters (within consideration of quantizer setting 552) to obtain quantized mixing parameters. The quantized mixing parameters can then be entropy-coded (using, for example, Huffman codes). Entropy coding can then encode the quantized mixing parameters of a frame (regardless of previous frames), the frequency difference of the quantized mixing parameters, or the time difference of the quantized mixing parameters. Encoding the time difference may not be used in the case of so-called independent frames, which are encoded independently of previous frames.

[0093] Therefore, parameter encoding unit 520 can use a combination of differential coding and Huffman coding to determine the encoded spatial parameters 562. As outlined above, the encoded spatial parameters 562 can be included as metadata (also referred to as spatial metadata) in the bitstream 564 along with the encoded downmix signal 563. Differential coding and Huffman coding can be used for transmitting the spatial metadata to reduce redundancy and thus increase the spare bit rate available for encoding the downmix signal 563. Because Huffman codes are variable-length codes, the size of the spatial metadata can vary considerably depending on the statistics of the encoded spatial parameters 562 to be transmitted. The data rate required to transmit the spatial metadata is subtracted from the data rate available to the core codec (e.g., Dolby Digital Plus) for encoding the stereo downmix signal. To avoid compromising the audio quality of the downmix signal, the number of bytes that may be spent transmitting spatial metadata for each frame is generally limited. This limit can be subject to encoder tuning considerations, which can be taken into account by configuration unit 540. However, due to the variable-length nature of the basic differential / Huffman coding of the spatial parameters, it is generally not guaranteed that the data rate limit (as reflected in the metadata data rate setting 552) will not be exceeded without any further means.

[0094] This document describes a method for post-processing encoded spatial parameter 562 and / or spatial metadata including encoded spatial parameter 562. Figure 6 The method 600 for post-processing spatial metadata is described in the context of [the relevant context]. Method 600 can be applied when the total size of a frame of spatial metadata exceeds a predefined limit, for example, indicated by metadata data rate setting 552. Method 600 aims to progressively reduce the amount of metadata. Reducing the size of spatial metadata typically also reduces the accuracy of the spatial metadata, and thus impairs the quality of the spatial image of the reproduced audio signal. However, Method 600 generally guarantees that the total amount of spatial metadata does not exceed a predefined limit, and thus allows determining the trade-off between the spatial metadata (used to regenerate the m-channel multichannel signal) and the audio codec metadata (used to decode the encoded downmixed signal 563) in terms of overall audio quality. Furthermore, the method 600 for post-processing spatial metadata can be implemented with relatively low computational complexity (compared to completely recalculating the encoded spatial parameters with modified control settings 552).

[0095] Method 600 for post-processing spatial metadata may include one or more of the following steps. As outlined above, each spatial metadata frame may include multiple (e.g., one or two) parameter sets, wherein the use of additional parameter sets allows for increased temporal resolution of the mixed parameters. The use of multiple parameter sets per frame can improve audio quality, especially in the case of attack-rich (i.e., transient) signals. Even in the case of audio signals with relatively slowly changing spatial images, spatial parameter updates with a grid twice the size of the sampling points can improve audio quality. However, sending multiple parameter sets per frame results in an approximately 2x increase in data rate. Therefore, if it is determined that the data rate of the spatial metadata exceeds the metadata data rate setting 552 (step 601), it can be checked whether the spatial metadata frame includes more than one mixed parameter set. Specifically, it can be checked whether the metadata frame includes two mixed parameter sets that should be sent (step 602). If it is determined that the spatial metadata includes multiple mixed parameter sets, one or more of the sets exceeding a single mixed parameter set can be discarded (step 603). The result is that the data rate of spatial metadata can be significantly reduced (typically by half in the case of two mixed parameter sets), while only relatively little loss to audio quality.

[0096] The decision of which of two (or more) mixed parameter sets to discard can depend on whether the encoding system 500 detects a transient location (“attack”) in the portion of the input signal 561 covered by the current frame: if multiple transients exist in the current frame, earlier transients are generally more significant than later transients due to the psychoacoustic backmasking effect of each individual attack. Therefore, if a transient exists, it may be recommended to discard the later mixed parameter set (e.g., the second of two). On the other hand, if no attack exists, the earlier mixed parameter set (e.g., the first of two) may be discarded. This may be due to the windowing used when calculating spatial parameters (e.g., ...). Figure 5e (As shown). The window 586 used for windowing the input signal 561 to calculate the spatial parameters for the second blending parameter set typically has its greatest influence at the time point when the sampling points for parameter reconstruction are placed in the upper mixing stage 130 (i.e., at the end of the current frame). On the other hand, the first blending parameter set typically receives a half-frame offset at that time point. Therefore, the error resulting from discarding the first blending parameter set is most likely to be lower than the error resulting from discarding the second blending parameter set. This is in Figure 5e As shown in, Figure 5eAs can be seen from the data, the second half of the spectrum 589 of the current frame 585 used to determine the second set of mixed parameters is more affected by the sampling of the current frame 585 than the first half of the spectrum 589 of the current frame 585 (for the first half, the value of the window function 586 is lower than the value for the second half of the spectrum 589).

[0097] The spatial cue (i.e., the mixing parameters) computed in the encoding system 500 is sent to the corresponding decoder 100 via a bitstream 562 (which may be part of a bitstream 564 in which the encoded stereo downmix signal 563 is delivered). Between the computation of the spatial cue and its representation in the bitstream 562, the encoding unit 524 typically applies a two-step encoding method: the first step, quantization, is a lossy step because it adds error to the spatial cue; the second step, differential / Huffman coding, is a lossless step. As outlined above, the encoder 500 can choose between different types of quantization (e.g., two types of quantization): a high-resolution quantization scheme, which adds relatively little error but results in a larger potential quantization index, thus requiring a larger Huffman codeword; and a low-resolution quantization scheme, which adds relatively more error but results in a smaller quantization index, thus not requiring such a large Huffman codeword. It should be noted that different types of quantization can be applied to some or all of the mixing parameters. For example, different types of quantization can be applied to the mixing parameters. α 1 , α 2 , α 3 , β 1 , β 2 , β 3 , k 1 On the other hand, gain g Quantization can be performed using a fixed type of quantization.

[0098] Method 600 may include a step 604 of verifying which type of quantization has been used to quantize the spatial parameters. If it is determined that a relatively fine quantization resolution was used, the encoding unit 524 may be configured to reduce the quantization resolution to a lower type of quantization 605. As a result, the spatial parameters are quantized again. However, this does not significantly increase computational overhead (compared to re-determining the spatial parameters using different control settings 552). It should be noted that different types of quantization can be used for different spatial parameters. α 1 , α 2 , α 3 , β1 , β 2 , β 3 , g , k 1 Therefore, the encoding unit 524 can be configured to select the quantizer resolution individually for each type of spatial parameter, thereby adjusting the data rate of the spatial metadata.

[0099] Method 600 may include the step of reducing the frequency resolution of the spatial parameters. Figure 6 (Not shown in the image). As outlined above, the set of mixed parameters for a frame is typically clustered into frequency bands or parameter bands 572. Each parameter band represents a frequency range, and for each band, a separate set of spatial clues is determined. The number of parameter bands 572 can be progressively changed (e.g., 7, 9, 12, or 15 bands) depending on the data rate available for transmitting spatial metadata. The number of parameter bands 572 is approximately linearly related to the data rate, and therefore a reduction in frequency resolution can significantly reduce the data rate of spatial metadata while only moderately affecting audio quality. However, such a reduction in frequency resolution typically requires recalculating the set of mixed parameters using the changed frequency resolution, thus increasing computational complexity.

[0100] As outlined above, encoding unit 524 can use differential coding of (quantized) spatial parameters. Configuration unit 551 can be configured to apply direct coding of the spatial parameters of the frames of the input audio signal 561 to ensure that transmission errors do not propagate over an infinite number of frames and to allow the decoder to synchronize with the received bitstream 562 at intermediate moments. Thus, a small portion of a frame can be used without differential coding along the timeline. Such frames without differential coding can be referred to as independent frames. Method 600 may include a step 606 to verify whether the current frame is an independent frame and / or whether the independent frame is a forced independent frame. The coding of the spatial parameters may depend on the result of step 606.

[0101] As outlined above, differential coding is typically designed to compute differences between temporal successors or between adjacent frequency bands of quantized spatial cues. In both cases, the statistics of spatial cues mean that smaller differences occur more frequently than larger differences; therefore, smaller differences are represented by shorter Huffman codewords compared to larger differences. This document proposes performing smoothing (temporally or frequency-wise) of the quantized spatial parameters. Smoothing spatial parameters temporally or frequency-wise generally results in smaller differences and thus a reduction in data rate. Due to psychoacoustic considerations, temporal smoothing is generally preferred over frequency-wise smoothing. If it is determined that the current frame is not a forced independent frame, method 600 can proceed with temporal differential coding (step 607), possibly combined with temporal smoothing. On the other hand, if the current frame is determined to be an independent frame, method 600 can proceed with frequency differential coding (step 608), possibly with frequency smoothing.

[0102] The differential coding in step 607 can be submitted to temporal smoothing to reduce the data rate. The degree of smoothing can be varied depending on the amount by which the data rate will be reduced. The most severe type of temporal "smoothing" corresponds to keeping the previous set of mixed parameters unchanged, which corresponds to sending only incremental values ​​equal to zero. Temporal smoothing of differential coding can be performed on one or more (e.g., all) of the spatial parameters.

[0103] Frequency smoothing can be performed in a similar manner to temporal smoothing. In its most extreme form, frequency smoothing corresponds to sending the same quantized spatial parameters over the entire frequency range of the input signal 561. While ensuring that the limits set by the metadata data rate settings are not exceeded, frequency smoothing can have a relatively high impact on the quality of the spatial image that can be reproduced using spatial metadata. Therefore, it may be preferable to apply frequency smoothing only when temporal smoothing is not permitted (e.g., if the current frame is a forced independent frame for which temporal differential coding of the previous frame is not available).

[0104] As outlined above, system 500 can operate subject to one or more external settings 551, such as the overall target data rate of bitstream 564 or the sampling rate of input audio signal 561. Typically, there is no single optimal operating point for all combinations of external settings. Configuration unit 540 can be configured to map valid combinations of external settings 551 to combinations of control settings 552, 554. For example, configuration unit 540 can rely on the results of psychoacoustic listening tests. Specifically, configuration unit 540 can be configured to determine combinations of control settings 552, 554 that ensure (on average) optimal psychoacoustic coding results for a particular combination of external settings 551.

[0105] As outlined above, the decoding system 100 should be able to synchronize with the received bitstream 564 within a given time period. To ensure this, the encoding system 500 can periodically encode so-called independent frames (i.e., frames that do not depend on knowledge about their predecessors). The average distance between two independent frames can be given by the ratio between the maximum latency for synchronization and the duration of a frame. This ratio does not necessarily have to be an integer, where the distance between two independent frames is always an integer.

[0106] Encoding system 500 (e.g., configuration unit 540) can be configured to receive the maximum latency or desired update time period for synchronization as an external setting 551. Furthermore, encoding system 500 (e.g., configuration unit 540) may include a timer module configured to track the absolute amount of time elapsed since the first encoded frame of bitstream 564. The first encoded frame of bitstream 564 is defined as an independent frame. Encoding system 500 (e.g., configuration unit 540) can be configured to determine whether the next encoded frame includes a sample corresponding to a time point that is an integer multiple of the desired update time period. Whenever the next encoded frame includes a sample at a time point that is an integer multiple of the desired update time period, encoding system 500 (e.g., configuration unit 540) can be configured to ensure that the next encoded frame is encoded as an independent frame. By doing so, it is ensured that the desired update time period is maintained even if the ratio of the desired update time period to the frame length is not an integer.

[0107] As outlined above, the parameter determination unit 523 is configured to calculate spatial cues based on the time / frequency representation of the multi-channel input signal 561. Spatial metadata frames can be determined based on K / Q (e.g., 24) spectra 589 (e.g., QMF spectra) of the current frame and / or K / Q (e.g., 24) spectra 589 (e.g., QMF spectra) of the forward-looking frame, where each spectrum 589 can have a frequency resolution of Q (e.g., 64) frequency intervals 571. Depending on whether the encoding system 500 detects transients in the input signal 561, the time length of the signal portion used to calculate a single set of spatial cues can include a varying number of spectra 589 (e.g., from 1 spectrum up to twice the number of K / Q spectra). Figure 5c As shown, each spectrum 589 is divided into a certain number of frequency bands 572 (e.g., 7, 9, 12, or 15 bands), which, due to psychoacoustic considerations, include a different number of frequency intervals 571 (e.g., 1 frequency interval up to 41 frequencies). Different frequency bands p 572 and different time segments [ q , vThe input signal 561 is defined by a grid on the time / frequency representation of the current frame and the forward frame. For different “boxes” within this grid, different sets of spatial cues can be calculated based on estimates of the energy and / or covariance of at least some of the input channels within each “box.” As outlined above, the energy estimate and / or covariance can be calculated by summing the squares of the transform coefficients 580 of a channel and / or by summing the products of the transform coefficients 580 of different channels, respectively (as indicated by the formulas provided above). The different transform coefficients 580 can be weighted according to a window function 586 used to determine the spatial parameters.

[0108] Energy estimation E 1,1 ( p ), E 2,2 ( p ) and / or covariance E 1,2 ( p The calculation of ) can be performed using fixed-point arithmetic. In this case, the different sizes of the "boxes" of the time / frequency grid can affect the arithmetic accuracy of the values ​​determined for the spatial parameters. As outlined above, each frequency band of 572 ( j - i +1) The quantity of 571 and / or the time interval of the "boxes" in the time / frequency grid [ q , v The length of the transformation coefficients can be significantly varied (e.g., between 1×1×2 and 48×41×2, i.e., the real and complex parts of the complex QMF coefficients). As a result, determining the energy... E 1,1 ( p ) / covariance E 1,2 ( p And the product that needs to be summed. Re { a t,f} Re { b t,f}and Im { a t,f} Im { b t,f The number of bits can be significantly changed. To prevent the calculation result from exceeding the range of numbers that can be expressed in fixed-point arithmetic, the signal can be scaled down by the maximum number of bits (e.g., due to 2^32 bits). 6 ·2 6=4096≥48·41·2, scaled down by 6 bits). However, for smaller “boxes” and / or for “boxes” that only include relatively low signal energy, this method results in a significant reduction in arithmetic accuracy.

[0109] In this document, it is proposed that each “box” of the time / frequency grid uses a separate scale. The separate scale can depend on the number of transform coefficients 580 included within the “box” of the time / frequency grid. Typically, specific “boxes” for the time-frequency grid (i.e., for a specific frequency band 572 and for a specific time interval […]) are scaled separately. q , v The spatial parameters of a spatial cue are determined solely based on the transform coefficients 580 from that particular "box" (and not depending on the transform coefficients 580 from other "boxes"). Furthermore, the spatial parameters are typically determined solely based on the energy estimate and / or covariance ratio (and are generally unaffected by the absolute energy estimate and / or covariance). In other words, a single spatial cue typically does not use the energy estimate and / or cross-channel product from a single time / frequency "box". Moreover, the spatial cue is generally unaffected by the absolute energy estimate / covariance, but only by the energy estimate / covariance ratio. Therefore, a separate scaling can be used for each individual "box". This scaling should be matched to the channels that contribute to a particular spatial cue.

[0110] For frequency band p 572 and for the time interval [ q , v Energy estimation of the first channel 561-1 and the second channel 561-2 E 1,1 ( p ), E 2,2 ( p The covariance between the first channel 561-1 and the second channel 561-2 E 1,2 ( p The energy estimate and covariance can be determined, for example, as indicated by the formula above. The energy estimate and covariance can be determined by scaling factors. s p Scaling is performed to provide scaling energy and covariance: s p · E 1,1 ( p ), s p · E 2,2 ( p )and s p · E 1,2 (p Based on energy estimation E 1,1 ( p ), E 2,2 ( p ) and covariance E 1,2 ( p Exported spatial parameters P ( p The spatial parameters typically depend on the ratio of energy and / or covariance. P ( p The value of ) is independent of the scaling factor. s p As a result, different scaling factors s p , s p+1 , s p+2 It can be used in different frequency bands p , p +1、 p +2.

[0111] It should be noted that one or more of the spatial parameters may depend on more than two different input channels (e.g., three different channels). In this case, energy estimation based on the different channels can be used. E 1,1 ( p ), E 2,2 ( p ... and based on the covariances between different vocal tracts (i.e., E 1,2 ( p ), E 1,3 ( p ), E 2,3 ( p The one or more spatial parameters are derived using methods such as [e.g., ...]]]]. Furthermore, in this case, the values ​​of the one or more spatial parameters are independent of the scaling factors applied to the energy estimation and

[0112] Specifically, for a specific frequency band p scaling factor s p =2 -zp (in, z p (A positive integer indicating a shift in fixed-point arithmetic) can be determined such that

[0113] And cause shift z p Minimum. By for each frequency band p And / or for each time interval in which the mixing parameters are determined [ q , v Ensuring this separately allows for increased (e.g., maximum) precision in fixed-point arithmetic while ensuring a valid range of values.

[0114] For example, individual scaling can be achieved by checking whether the result of each individual MAC (multiply-accumulate) operation exceeds + / - 1. Only if this is the case can the individual scaling for the "box" be increased by one bit. Once this is done for all channels, the maximum scaling for each "box" can be determined, and all deviations in scaling for the "box" can be adjusted accordingly.

[0115] As outlined above, spatial metadata may include one or more (e.g., two) sets of spatial parameters per frame. Thus, encoding system 500 may send one or more sets of spatial parameters per frame to the corresponding decoding system 100. Each of these sets of spatial parameters corresponds to a specific spectrum in the K / Q temporally consecutive spectra 289 of the spatial metadata frame. This specific spectrum corresponds to a specific time point, which may be referred to as a sampling point. Figure 5c Two example sampling points 583 and 584 are shown for two sets of spatial parameters, respectively. Sampling points 583 and 584 can be associated with specific events included in the input audio signal 561. Alternatively, the sampling points can be predetermined.

[0116] Sampling points 583 and 584 indicate the moments when the corresponding spatial parameters should be fully utilized by the decoding system 100. In other words, the decoding system 100 can be configured to update the spatial parameters at sampling points 583 and 584 based on the transmitted set of spatial parameters. Furthermore, the decoding system 100 can be configured to interpolate spatial parameters between two subsequent sampling points. Spatial metadata can indicate the type of transition to be performed between successive sets of spatial parameters. Examples of transition types are “smooth” and “steep” transitions between spatial parameters, meaning that the spatial parameters can be interpolated smoothly (e.g., linearly) or updated abruptly, respectively.

[0117] In the case of a “smooth” transition, the sampling point can be fixed (i.e., predetermined) and therefore does not need to be transmitted as a signal in the bitstream 564. If the spatial metadata frame delivers a single set of spatial parameters, the predetermined sampling point can be the position at the very end of the frame, i.e., the sampling point can correspond to the (K / Q)th spectrum 589. If the spatial metadata frame delivers two sets of spatial parameters, the first sampling point can correspond to the (K / 2Q)th spectrum 589, and the second sampling point can correspond to the (K / Q)th spectrum 589.

[0118] In the case of a "steep" transition, sampling points 583 and 584 can be variable and can be transmitted as signals in bitstream 562. The portion of bitstream 562 carrying the following information can be referred to as the "framing" portion of bitstream 562: information about the number of spatial parameter sets used in a frame, information about the choice between "smooth" and "steep" transitions, and information about the position of the sampling points in the case of a "steep" transition. Figure 7a An example transition scheme is shown that can be applied by the decoding system 100 based on the framing information included in the received bitstream 562.

[0119] For example, the framing information for a particular frame may indicate a “smooth” transition and a single spatial parameter set 711. In this case, the decoding system 100 (e.g., the first mixing matrix 130) may assume that the sampling points of the spatial parameter set 711 correspond to the last spectrum of the particular frame. Furthermore, the decoding system 100 may be configured to perform (e.g., linear) interpolation 701 between the last received spatial parameter set 710 for the immediately preceding frame and the spatial parameter set 711 for the particular frame. In another example, the framing information for a particular frame may indicate a “smooth” transition and two spatial parameter sets 711, 712. In this case, the decoding system 100 (e.g., the first mixing matrix 130) may assume that the sampling points of the first spatial parameter set 711 correspond to the last spectrum of the first half of the particular frame, and the sampling points of the second spatial parameter set 712 correspond to the last spectrum of the second half of the particular frame. Furthermore, the decoding system 100 can be configured to perform (e.g., linear) interpolation 702 between the last received set of spatial parameters 710 for the preceding frame and the first set of spatial parameters 711, and between the first set of spatial parameters 711 and the second set of spatial parameters 712.

[0120] In another example, the framing information for a particular frame may indicate a "steep" transition, a single spatial parameter set 711, and a sampling point 583 for that single spatial parameter set 711. In this case, the decoding system 100 (e.g., the first mixing matrix 130) may be configured to apply the last received spatial parameter set 710 to the immediately preceding frame up to sampling point 583, and to apply spatial parameter set 711 starting from sampling point 583 (as shown by curve 703). In another example, the framing information for a particular frame may indicate a "steep" transition, two spatial parameter sets 711, 712, and two corresponding sampling points 583, 584 for the two spatial parameter sets 711, 712, respectively. In this configuration, the decoding system 100 (e.g., the first mixing matrix 130) can be configured to apply the last received set of spatial parameters 710 to the immediately preceding frame up to the first sampling point 583, apply the first set of spatial parameters 711 from the first sampling point 583 up to the second sampling point 584, and apply the second set of spatial parameters 712 (as shown in curve 704) from the second sampling point 584 at least until the end of the particular frame.

[0121] The encoding system 500 should ensure that the framing information matches the signal characteristics, and that an appropriate portion of the input signal 561 is selected to calculate the one or more sets of spatial parameters 711, 712. For this purpose, the encoding system 500 may include a detector configured to detect signal locations in one or more channels where signal energy suddenly increases. If at least one such signal location is found, the encoding system 500 can be configured to switch from a “smooth” transition to a “steep” transition; otherwise, the encoding system 500 can continue with a “smooth” transition.

[0122] As outlined above, the encoding system 500 (e.g., parameter determination unit 523) can be configured to calculate spatial parameters for the current frame based on multiple frames 585, 590 of the input audio signal 561 (e.g., based on the current frame 585 and the immediately following frame 590 (i.e., the so-called forward frame)). Thus, the parameter determination unit 523 can be configured to determine spatial parameters (e.g., based on twice the K / Q spectra 589) Figure 5e As shown). Figure 5e As shown, spectrum 589 can be windowed using window 586. In this document, it is proposed to adapt window 586 based on the number of spatial parameter sets 711, 712 to be determined, based on the transition type, and / or based on the location of sampling points 583, 584. By doing so, it can be ensured that the framing information matches the signal characteristics, and that appropriate portions of the input signal 561 are selected to compute the one or more spatial parameter sets 711, 712.

[0123] The following describes example window functions for different encoder / signal scenarios: a) Case: Single spatial parameter set 711, smooth transition, no transients in forward-looking frame 590; Window function 586: Between the last spectrum of the previous frame and the (K / Q)th spectrum 589, window function 586 can linearly increase from 0 to 1. Between the (K / Q)th spectrum 589 and the 48th spectrum 589, window function 586 can linearly decrease from 1 to 0 (see [link to relevant documentation]). Figure 5e ).

[0124] b) Case: A single set of spatial parameters 711, smooth transition, transient in the Nth spectrum (N>K / Q), i.e., transient in the forward-looking frame 590; like Figure 7b The window function 721 shown: Between the last spectrum of the previous frame and the (K / Q)th spectrum, window function 721 linearly increases from 0 to 1. Between the (K / Q)th spectrum and the (N-1)th spectrum, window function 721 remains constant at 1. Between the Nth spectrum and the (2)th spectrum... Between the K / Q) spectrum segments, the window function remains constant at 0. The transient at the Nth spectrum segment is represented by the transient point 724 (which corresponds to the sampling point used for the set of spatial parameters immediately following frame 590). Furthermore, Figure 7b The diagram illustrates a complementary window function 722 (applied to the spectrum of the current frame 585 when determining the set of one or more spatial parameters for the previous frame) and a window function 723 (applied to the spectrum of the next frame 590 when determining the set of one or more spatial parameters for the next frame). In general, window function 721 ensures that, in the case of one or more transients in the forward-looking frame 590, the spectrum of the forward-looking frame preceding the first transient point 724 is sufficiently considered for determining the set of spatial parameters 711 for the current frame 585. Conversely, the spectrum of the forward-looking frame 590 following the transient point 724 is ignored.

[0125] c) Case: Single spatial parameter set 711, steep transition, transient in the Nth spectrum (N<=K / Q), no transient in subsequent frame 590.

[0126] like Figure 7c The window function 731 shown: Between the 1st spectrum and the (N-1)th spectrum, window function 731 remains constant at 0. Between the Nth spectrum and the (K / Q)th spectrum, window function 731 remains constant at 1. Between the (K / Q)th spectrum and the (2nd spectrum... Between the K / Q) spectrum, the window function 731 linearly decreases from 1 to 0. Figure 7cThis indicates the transient point 734 at the Nth frequency spectrum (which corresponds to a sampling point of a single spatial parameter set 711). Furthermore, Figure 7c Window functions 732 and 733 are shown. Window function 732 is applied to the spectrum of the current frame 585 when determining the set of one or more spatial parameters for the previous frame, and window function 733 is applied to the spectrum of the next frame 590 when determining the set of one or more spatial parameters for the next frame.

[0127] d) Cases: Single set of spatial parameters, steep transition, transients in the Nth and Mth spectra (N<=K / Q, M>K / Q); Figure 7d Window function 741 in the code: Between the 1st spectrum and the (N-1)th spectrum, window function 741 remains constant at 0. Between the Nth spectrum and the (M-1)th spectrum, window function 741 remains constant at 1. Between the Mth spectrum and the 48th spectrum, window function 741 remains constant at 0. Figure 7d This indicates transient point 744 at the Nth spectrum (i.e., the sampling point of the spatial parameter set) and transient point 745 at the Mth spectrum. Furthermore, Figure 7d Window functions 742 and 743 are shown. Window function 742 is applied to the spectrum of the current frame 585 when determining the set of one or more spatial parameters for the previous frame, and window function 743 is applied to the spectrum of the next frame 590 when determining the set of one or more spatial parameters for the next frame.

[0128] e) Case: Two sets of spatial parameters, smooth transition, no transients in subsequent frames; Window function: i.) The first set of spatial parameters: Between the last spectrum of the previous frame and the (K / 2Q)th spectrum, the window linearly increases from 0 to 1. Between the (K / 2Q)th spectrum and the (K / Q)th spectrum, the window linearly decreases from 1 to 0. Between the (K / Q)th spectrum and the (2... Between the K / Q) spectrums, the window remains constant at 0.

[0129] ii.) The second set of spatial parameters: Between the first and (K / 2Q)th spectra, the window remains constant at 0. Between the (K / 2Q)th and (K / Q)th spectra, the window linearly increases from 0 to 1. Between the (K / Q)th and (3)th spectra... Between the (K / 2Q)th spectrum, the window linearly decreases from 1 to 0. At the (3rd)th... The K / 2Q)th spectrum and the (2)th Between the K / Q) spectrums, the window remains constant at 0.

[0130] f) Case: Two sets of spatial parameters, smooth transition, transient in the Nth spectrum (N > K / Q); Window function: i.) The first set of spatial parameters: Between the last spectrum of the previous frame and the (K / 2Q)th spectrum, the window linearly rises from 0 to 1. Between the (K / 2Q)th spectrum and the (K / Q)th spectrum, the window linearly drops from 1 to 0. Between the (K / Q)th spectrum and the (2 K / Q)th spectrum, the window remains constantly 0.

[0131] ii.) The second set of spatial parameters: Between the 1st spectrum and the (K / 2Q)th spectrum, the window remains constantly 0. Between the (K / 2Q)th spectrum and the (K / Q)th spectrum, the window linearly rises from 0 to 1. Between the (K / Q)th spectrum and the (N - 1)th spectrum, the window remains constantly 1. Between the Nth spectrum and the (2 K / Q)th spectrum, the window remains constantly 0.

[0132] g) Case: Two sets of spatial parameters, steep transition, transient in the Nth and Mth spectra (N < M <= K / Q), no transient in subsequent frames; Window function: i.) The first set of spatial parameters: Between the 1st spectrum and the (N - 1)th spectrum, the window remains constantly 0. Between the Nth spectrum and the (M - 1)th spectrum, the window remains constantly 1. Between the Mth spectrum and the (2 K / Q)th spectrum, the window remains constantly 0.

[0133] ii.) The second set of spatial parameters: Between the 1st spectrum and the (M - 1)th spectrum, the window remains constantly 0. Between the Mth spectrum and the (K / Q)th spectrum, the window remains constantly 1. Between the (K / Q)th spectrum and the (2 K / Q)th spectrum, the window linearly drops from 1 to 0.

[0134] h) Case: Two sets of spatial parameters, steep transition, transient in the Nth, Mth and Oth spectra (N < M <= K / Q, O > K / Q); Window function: i.) The first set of spatial parameters: Between the 1st spectrum and the (N - 1)th spectrum, the window remains constantly 0. Between the Nth spectrum and the (M - 1)th spectrum, the window remains constantly 1. Between the Mth spectrum and the (2 K / Q)th spectrum, the window remains constantly 0.

[0135] ii.) The second set of spatial parameters: Between the first and (M-1)th spectra, the window remains constant at 0. Between the Mth and (O-1)th spectra, the window remains constant at 1. Between the Oth and (2)th spectra, the window remains constant at 1. Between the K / Q) spectrums, the window remains constant at 0.

[0136] In general, the following example rules can be specified for the window function used to determine the current set of spatial parameters: If the current set of spatial parameters is not associated with transients - Window functions provide a smooth, gradual increase in the spectrum from the sampling points of the previous set of spatial parameters to the sampling points of the current set of spatial parameters; - A window function provides a smooth decay of the spectrum from the sampling points of the current set of spatial parameters to the sampling points of the next set of spatial parameters, if the next set of spatial parameters is not associated with the transient. - The window function fully considers the spectrum from the sampling point of the current spatial parameter set to the sampling point of the next spatial parameter set, and eliminates the spectrum starting from the sampling point of the next spatial parameter set if the next spatial parameter set is associated with transients. If the current set of spatial parameters is associated with transients - Window functions eliminate the spectrum preceding the sampling points of the current spatial parameter set; - The window function fully considers the spectrum from the sampling point of the current spatial parameter set to the sampling point of the next spatial parameter set, and eliminates the spectrum starting from the sampling point of the next spatial parameter set if the sampling point of the next spatial parameter set is associated with transients. - The window function fully considers the spectrum from the sampling points of the current spatial parameter set to the end of the current frame, and provides a smooth fading of the spectrum from the beginning of the previous frame to the sampling points of the next spatial parameter set, if the next spatial parameter set is not associated with the transient.

[0137] The following describes a method for reducing latency in a parameterized multichannel codec system including an encoding system 500 and a decoding system 100. As outlined above, the encoding system 500 includes several processing paths, such as downmixing signal generation and encoding, and parameter determination and encoding. The decoding system 100 typically performs decoding of the encoded downmixing signal and generation of the decorrelated downmixing signal. Furthermore, the decoding system 100 performs decoding of the encoded spatial metadata. Subsequently, the decoded spatial metadata is applied to the decoded downmixing signal and the decorrelated downmixing signal to generate an upmixing signal in a first upmixing matrix 130.

[0138] The desired outcome is to provide an encoding system 500 configured to enable the decoding system 100 to generate an upmixed signal with reduced latency and / or reduced buffer memory. Y The bitstream 564. As outlined above, the encoding system 500 includes several distinct paths that can be aligned to ensure that the encoded data provided to the decoding system 100 within the bitstream 564 matches correctly during decoding. As outlined above, the encoding system 500 performs downmixing encoding of the PCM signal 561. Furthermore, the encoding system 500 determines spatial metadata from the PCM signal 561. Additionally, the encoding system 500 can be configured to determine one or more trimming gains (typically one trimming gain per frame). The trimming gain indication has been applied to the downmixed signal. X In order to ensure the downmixing signal X Untrimmed trimming prevents gain loss. The one or more trimmed gains can be transmitted within the bitstream 564 (typically within a spatial metadata frame) so that the decoding system 100 can regenerate the upmixed signal. Y Additionally, the encoding system 500 can be configured to determine one or more Dynamic Range Control (DRC) values ​​(e.g., one or more DRC values ​​per frame). These one or more DRC values ​​can be used by the decoding system 100 to perform upmixing. Y Dynamic range control. Specifically, the one or more DRC values ​​ensure that the DRC performance of the parameterized multichannel codec system described in this document is similar to (or equal to) the DRC performance of older multichannel codec systems such as Dolby Digital Plus. The one or more DRC values ​​can be transmitted within the downmixed audio frame (e.g., within an appropriate field of the Dolby Digital Plus bitstream).

[0139] Thus, the encoding system 500 may include at least four signal processing paths. To align these four paths, the encoding system 500 may also take into account the latency introduced into the system by different processing components that are not directly related to the encoding system 500, such as core encoder latency, core decoder latency, spatial metadata decoder latency, LFE filter latency (used to filter LFE channels), and / or QMF analysis latency.

[0140] To align different paths, the delay of the DRC processing path can be considered. The DRC processing delay can typically only be aligned to the frame level, not based on time-samples. Thus, the DRC processing delay usually depends only on the core encoder delay that can be rounded up to the next frame for alignment, i.e., DRC processing delay = round up(core encoder delay / frame size). Based on this, the downmixing processing delay used to generate the downmixed signal can be determined, since the downmixing processing delay can be delayed based on time samples, i.e., downmixing processing delay = DRC delay. Frame size - core encoder latency. For example... Figure 8 As shown, the remaining delays can be calculated by summing the individual delay lines and by ensuring that the delays match at the decoder level.

[0141] By considering the different processing delays when writing 564 bitstreams, the resulting spatial metadata is delayed by one frame (memory reduction of the number of input channels). 1536 Instead of delaying the encoded PCM data by 1536 samples (4 bytes - 245 bytes), this reduces the processing power of the decoding system by 100 (the copy operation reduces the number of input channels - 1). 1536) and memory. As a result of the delay, all signal paths are precisely aligned through time sampling, not just roughly matched.

[0142] As summarized above, Figure 8 The different delays caused by example encoding system 500 are illustrated. Figure 8 The numbers in parentheses indicate example delays based on the number of samples of the input signal 561. The encoding system 500 typically includes a delay 801 caused by filtering the LFE channels of the multi-channel input signal 561. Furthermore, a delay 802 (referred to as “clipgainpcmdelayline”) can be caused by determining a clipping gain (i.e., the DRC2 parameter below) that will be applied to the input signal 561 to prevent clipping of the downmixed signal. Specifically, this delay 802 can be introduced to synchronize the application of the clipping gain in the encoding system 500 with the application of the clipping gain in the decoding system 100. For this purpose, the input delay of the downmixing calculation (performed by the downmixing processing unit 510) can be made equal to the delay 811 (referred to as “coredecdelay”) of the decoder 140 of the downmixed signal. This means that, in the illustrated example, clipgainpcmdelayline = coredecdelay = 288 samples.

[0143] The downmixing processing unit 510 (which includes, for example, a Dolby Digital Plus encoder) delays the processing path of audio data (e.g., the downmix signal), but it does not delay the processing path of spatial metadata or the processing path for DRC / trimmed gain data. Therefore, the downmixing processing unit 510 should delay the calculated DRC gain, trimmed gain, and spatial metadata. For DRC gain, this delay typically needs to be a multiple of one frame. The delay 807 of the DRC delay line (referred to as "drcdelayline") can be calculated as drcdelayline = ceil((corencdelay + clipgainpcmdelayline) / frame_size) = 2 frames; where "corencdelay" refers to the delay 810 of the encoder of the downmix signal.

[0144] The delay of DRC gain is typically only a multiple of the frame size. Therefore, an additional delay may need to be added in the downmixing processing path to compensate for this and round up to the next multiple of the frame size. The additional downmixing delay 806 (which is called "dmxdelayline") can be calculated as dmxdelayline + coreencdelay + clipgainpcmdelayline = drcdelayline The frame_size is determined; and dmxdelayline = drcdelayline Set frame_size-coreencdelay-clipgainpcmdelayline to make dmxdelayline=100.

[0145] When spatial parameters are applied in the frequency domain (e.g., in the QMF domain) on the decoder side, the spatial parameters should be synchronized with the downmixing signal. To compensate for the fact that the encoder of the downmixing signal delays the downmixing processing path instead of the spatial metadata frame, the input of parameter extractor 420 should be delayed such that the following condition applies: dmxdelayline + coreencdelay + coredecdelay + aspdecanadelay = aspdelayline + qmfanadelay + framingdelay. In the above formula, "qmfanadelay" specifies the delay 804 caused by transform unit 521, and "framingdelay" specifies the delay 805 caused by windowing of transform coefficients 580 and determination of spatial parameters. As outlined above, framing calculation uses two frames (the current frame and the look-forward frame) as input. Due to the look-forward, framing introduces a delay 805 of exactly one frame length. Furthermore, a delay of 804 is known, such that the additional delay to be applied to the processing path used to determine spatial metadata is aspdelayline = dmxdelayline + coreencdelay + coredecdelay + aspdecanadelay - qmfanadelay - framingdelay = 1856. Because this delay is greater than one frame, the memory size of the delay line can be reduced by delaying the calculated bitstream instead of the input PCM data, thus providing aspbsdelayline = floor(aspdelayline / frame_size) = 1 frame (delay 809) and asppcmdelayline = aspdelayline - aspbsdelayline frame_size=320 (delay 803).

[0146] After calculating the one or more trimming gains, the one or more trimming gains are provided to the bitstream generation unit 530. Therefore, the one or more trimming gains undergo a delay applied to the final bitstream by aspbsdelayline 809. Thus, the additional delay 808 for the trimming gains should be: clipgainbsdelayline + aspbsdelayline = dmxdelayline + coreencdelay + coredecdelay, which provides: clipgainbsdelayline = dmxdelayline + coreencdelay + coredecdelay - aspbsdelayline = 1 frame. In other words, it should be ensured that the one or more trimming gains are provided to the decoding system 100 immediately after the corresponding frame of the downmixed signal is decoded, so that the one or more trimming gains can be applied to the downmixed signal before upmixing is performed in the upmixing stage 130.

[0147] Figure 8 Further delays induced at the decoding system 100 are shown, such as delay 812 (referred to as “aspdecanadelay”) caused by time-domain to frequency-domain transformations 301, 302 of the decoding system 100, delay 813 (referred to as “aspdecsyndelay”) caused by frequency-domain to time-domain transformations 311 to 316, and further delay 814.

[0148] If possible Figure 8 As can be seen, the different processing paths of the encoding / decoding system include processing-related delays or alignment delays, which ensure that different output data from different processing paths are available at the decoding system 100 when needed. Providing alignment delays (e.g., delays 803, 809, 807, 808, 806) within the encoding system 500 reduces the processing power and memory required at the decoding system 100. The total delay for the different processing paths (excluding the LFE filter delay 801 which can be applied to all processing paths) is as follows: Downmixing processing path: The sum of delays 802, 806, and 810 = 3072, that is, two frames; DRC processing path: delay 807 = 3072, that is, two frames; Trimmed gain processing path: The sum of delays 808, 809, and 802 = 3360, which, in addition to the 811 delay of the decoder for the downmixed signal, also corresponds to the delay of the downmixing processing path; Spatial metadata processing path: The sum of delays 802, 803, 804, 805, and 809 = 4000, which corresponds to the delay of the downmixing processing path, in addition to the delay 811 of the decoder of the downmixed signal and the delay 812 caused by the time-to-frequency transformation stages 301 and 302.

[0149] Therefore, it is ensured that DRC data is available at decoding system 100 at time 821, trimmed gain data is available at time 822, and spatial metadata is available at time 823.

[0150] In addition, from Figure 8 As can be seen, the bitstream generation unit 530 can combine encoded audio data and spatial metadata that may be associated with different segments of the input audio signal 561. Specifically, it can be seen that the downmixing processing path, the DRC processing path, and the trimmed gain processing path have a delay of exactly two frames (3072 samples) up to the output of the encoding system 500 (indicated by interfaces 831, 832, 833) (when delay 801 is ignored). The encoded downmixed signal is provided by interface 831, the DRC gain data is provided by interface 832, and the spatial metadata and trimmed gain data are provided by interface 833. Typically, the encoded downmixed signal and DRC gain data are provided in a conventional Dolby Digital Plus frame, while the trimmed gain data and spatial metadata can be provided in a spatial metadata frame (e.g., in an auxiliary field of a Dolby Digital Plus frame).

[0151] It can be seen that the spatial metadata processing path at interface 833 has a delay of 4000 samples (when delay 801 is ignored), which differs from the delay of other processing paths (3072 samples). This means that the spatial metadata frame may be related to a segment of the input signal 561 that is different from the downmixed signal. Specifically, it can be seen that, in order to ensure alignment at decoding system 100, bitstream generation unit 530 should be configured to generate a bitstream 564 comprising a sequence of bitstream frames, wherein the bitstream frames indicate the frame of the downmixed signal corresponding to the first frame of the multi-channel input signal 561 and the spatial metadata frame corresponding to the second frame of the multi-channel input signal 561. The first and second frames of the multi-channel input signal 561 may include the same number of samples. Nevertheless, the first and second frames of the multi-channel input signal 561 may be different from each other. Specifically, the first and second frames may correspond to different segments of the multi-channel input signal 561. More specifically, the first frame may include samples preceding the samples of the second frame. For example, the first frame may include samples of the multi-channel input signal 561 such that these samples precede the samples of the second frame of the multi-channel input signal 561 by a predetermined number of samples (e.g., 928 samples).

[0152] As outlined above, the encoding system 500 can be configured to determine dynamic range control (DRC) and / or trim gain data. Specifically, the encoding system 500 can be configured to ensure the downmixed signal... X No trimming. Furthermore, the encoding system 500 can be configured to provide dynamic range control (DRC) parameters, which ensure that multi-channel signals encoded using the aforementioned parametric encoding scheme are properly handled. Y The DRC behavior is similar to or equal to that of a multichannel signal encoded using a reference multichannel coding system such as Dolby Digital Plus. Y DRC behavior.

[0153] Figure 9a A block diagram of an example dual-mode coding system 900 is shown. It should be noted that portions 930 and 931 of the dual-mode coding system 900 are typically provided separately. (n-channel input signal) Y 561 is provided to each of the upper portion 930 and the lower portion 931, wherein the upper portion 930 is valid at least in the multi-channel encoding mode of the encoding system 900, and the lower portion 931 is valid at least in the parametric encoding mode of the system 900. The lower portion 931 of the encoding system 900 may correspond to or may include, for example, the encoding system 500. The upper portion 930 may correspond to a reference multi-channel encoder (such as a Dolby Digital Plus encoder). The upper portion 930 generally includes a discrete-mode DRC analyzer 910 arranged in parallel with the encoder 911, both of which receive audio signals as input. Y 561. Based on the input signal 561, the encoder 911 outputs the encoded n-channel signal. The DRC analyzer 910 outputs quantization, which applies one or more post-processing DRC parameters DRC1 to the decoder-side DRC. The DRC parameter DRC1 can be a "compr" gain (compressor gain) and / or a "dynrng" gain (dynamic range gain) parameter. The parallel outputs of the two units 910 and 911 are acquired by a discrete-mode multiplexer 912, which outputs a bitstream P. The bitstream P can have a predefined syntax, such as Dolby Digital Plus syntax.

[0154] The lower portion 931 of the encoding system 900 includes a parametric analysis stage 922 arranged in parallel with the parametric mode DRC analyzer 921. The parametric mode DRC analyzer 921, like the parametric analysis stage 922, receives n-channel input signals. Y The parameterized analysis stage 922 may include a parameter extractor 420. Based on n-channel audio signals. Y, the parametric analysis stage 922 outputs one or more hybrid parameters (as outlined above) (in Figure 9a and 9b jointly denoted in a and the m-channel (1 < m < n) downmixed signal X . This m-channel downmixed signal X is then processed by a core signal encoder 923 (e.g., a Dolby Digital Plus encoder), and the core signal encoder 923 outputs an encoded downmixed signal based on this. The parametric analysis stage 922 affects the dynamic range limitation in a time block or frame of the input signal, where it may be required. The possible conditions for controlling when to apply the dynamic range limitation can be "non-trim condition" or "in-range condition", which imply that in the time block or frame segment where the downmixed signal has a high amplitude, the signal is processed so that it fits within a defined range. This condition can be implemented based on a time block or a time frame including several time blocks. For example, a frame of the input signal 561 can include a predetermined number (e.g., 6) of blocks. Preferably, this condition is implemented by applying wide-spectrum gain reduction rather than just truncating the peaks or using similar methods.

[0155] Figure 9b shows a possible implementation of the parametric analysis stage 922, which includes a preprocessor 927 and a parametric analysis processor 928. The preprocessor 927 is responsible for performing dynamic range limitation on the n-channel input signal 561, and thus it outputs a dynamically range-limited n-channel signal, which is supplied to the parametric analysis processor 928. The preprocessor 527 further outputs the per-block or per-frame values of the preprocessed DRC parameter DRC2. Together with the m-channel downmixed signal X and the hybrid parameter α from the parametric analysis processor 928, the parameter DRC2 is included in the output from the parametric analysis stage 922.

[0156] The parameter DRC2 can also be referred to as the trim gain. The parameter DRC2 can indicate the gain that has been applied to the multi-channel input signal 561 to ensure that the downmixed signal X is not trimmed. The one or more channels of the downmixed signal Y can be determined from the channels of the input signal Y by determining a linear combination of some or all of the channels of the input signal. For example, the input signal X can be a 5.1 multi-channel signal, and the downmixed signal can be a stereo signal. The samples of the left and right channels of the downmixed signal can be generated based on different linear combinations of the samples of the 5.1 multi-channel input signal. Y

[0157] ​The DRC2 parameter can be determined such that the maximum amplitude of each channel of the downmixed signal does not exceed a predetermined threshold. This can be ensured block-by-block or frame-by-frame. A single gain (trimming gain) for each block or frame can be applied to the multi-channel input signal. Y The channels are configured to ensure that the conditions mentioned above are met. The DRC2 parameter can indicate this gain (e.g., the inverse of the gain).

[0158] Reference Figure 9a Note that the discrete-mode DRC analyzer 910 operates similarly to the parametric-mode DRC analyzer 921 because it outputs one or more post-processed DRC parameters DRC1 to which the decoder-side DRC will be quantized. Thus, the parametric-mode DRC analyzer 921 can be configured to simulate the DRC processing performed by the reference multichannel encoder 930. The parameters DRC1 provided by the parametric-mode DRC analyzer 921 are typically not included in the bitstream P in parametric coding mode, but are compensated so that the dynamic range limitations implemented by the parametric analysis level 922 are taken into account. For this purpose, the DRC up-compensator 924 receives the post-processed DRC parameters DRC1 and the pre-processed DRC parameters DRC2. For each block or frame, the DRC up-compensator 924 derives the value of one or more compensated post-processed DRC parameters DRC3, such that the combined effect of the compensated post-processed DRC parameters DRC3 and the pre-processed DRC parameters DRC2 is quantitatively equivalent to the DRC quantized by the post-processed DRC parameters DRC1. In other words, the DRC compensator 924 is configured to reduce the share (if any) of the post-processed DRC parameters output by the DRC analyzer 921, which has already been implemented by the parameterized analysis stage 922. This can be the compensated post-processed DRC parameter DRC3 that can be included in the bitstream P.

[0159] Referring to the lower part 931 of the reference system 900, the parameterized mode multiplexer 925 collects the compensated post-processing DRC parameter DRC3, the pre-processing DRC parameter DRC2, the mixing parameter α, and the encoded downmixed signal. X And based on this, a bitstream P is formed. Thus, the parameterized mode multiplexer 925 may include or correspond to the bitstream generation unit 530. In a possible implementation, the compensated post-processing DRC parameter DRC3 and the pre-processing DRC parameter DRC2 may be encoded in logarithmic form as dB values ​​that affect amplitude amplification or reduction on the decoder side. The compensated post-processing DRC parameter DRC3 may have any sign. However, the pre-processing DRC parameter DRC2, derived from implementing "unpruned conditions," will generally always be represented by a non-negative dB value.

[0160] Figure 10An example processing is shown that can be performed, for example, in the parameterized mode DRC analyzer 921 and the DRC compensator 924 to determine the modified DRC parameters DRC3 (e.g., the modified "dynrng gain" and / or "compr gain" parameters).

[0161] The DRC2 and DRC3 parameters can be used to ensure that the decoding system 100 plays back different audio bitstreams at a consistent loudness level. Furthermore, it can be ensured that the bitstream generated by the parameterized encoding system 500 has a consistent loudness level relative to the bitstream generated by legacy and / or reference encoding systems (such as Dolby Digital Plus). As outlined above, this can be ensured by generating an untrimmed downmix at the encoding system 500 (using the DRC2 parameter) and by providing the DRC2 parameter within the bitstream (e.g., the inverse of the attenuation applied to prevent downmix trimming) so that the decoding system 100 can recreate the original loudness (when an upmix is ​​generated).

[0162] As outlined above, the downmix signal is typically generated based on a linear combination of some or all of the channels of the multichannel input signal 561. Thus, the scaling factor (or attenuation) applied to the channels of the multichannel input signal 561 can depend on all channels of the multichannel input signal 561 that contribute to the downmix signal. Specifically, the one or more channels of the downmix signal can be determined based on the LFE channels of the multichannel input signal 561. Therefore, the scaling factor (or attenuation) applied to trim protection should also take into account the LFE channels. This differs from other multichannel coding systems (such as Dolby Digital Plus) where LFE channels are typically not considered for trim protection. By considering the LFE channels and / or all channels that contribute to the downmix signal, the quality of trim protection can be improved.

[0163] Thus, the one or more DRC2 parameters provided to the corresponding decoding system 100 can depend on all channels of the input signal 561 that contribute to the downmix signal; specifically, the DRC2 parameters can depend on the LFE channel. By doing so, the quality of trim protection can be improved.

[0164] It should be noted that the dialnorm parameter may not be considered in calculating the scaling factor and / or DRC2 parameter (e.g., Figure 10 (As shown).

[0165] As outlined above, encoding system 500 can be configured to write a so-called "trimmed gain" (i.e., DRC2 parameter) into a spatial metadata frame indicating which gains have been applied to the input signal 561 to prevent trimming in the undermixed signal. The corresponding decoding system 100 can be configured to accurately invert the trimmed gain applied in encoding system 500. However, only the sample points of the trimmed gain are transmitted in the bitstream. In other words, the trimmed gain parameter is typically determined only frame-by-frame or block-by-block. Decoding system 100 can be configured to interpolate the trimmed gain value (i.e., the received DRC2 parameter) between sample points from adjacent sample points.

[0166] Figure 11 Example interpolation curves for interpolating DRC2 parameters between adjacent frames are shown in the figure. Specifically, Figure 11 A first DRC2 parameter 953 for the first frame and a second DRC2 parameter 954 for the subsequent second frame 950 are shown. The decoding system 100 can be configured to interpolate between the first DRC2 parameter 953 and the second DRC2 parameter 954. Interpolation can be performed within a sample subset 951 of the second frame 950 (e.g., within the first block 951 of the second frame 950) (as shown by interpolation curve 952). Interpolation of the DRC2 parameters ensures a smooth transition between adjacent audio frames and thus avoids audible artifacts that may be caused by the difference between the subsequent DRC2 parameters 953 and 954.

[0167] Encoding system 500 (specifically, downmixing processing unit 510) can be configured to apply a corresponding trimmed gain interpolation to the DRC2 interpolation 952 performed by decoding system 100 when a downmixed signal is generated. This ensures that the trimmed gain protection of the downmixed signal is consistently removed when an upmixed signal is generated. In other words, encoding system 500 can be configured to simulate the curve of the DRC2 value derived from the DRC2 interpolation 952 applied by decoding system 100. Furthermore, encoding system 500 can be configured to accurately (i.e., sample-by-sample) inversely apply the curve of this DRC2 value to the multichannel input signal 561 when a downmixed signal is generated.

[0168] The methods and systems described in this document can be implemented as software, firmware, and / or hardware. Some components can be implemented, for example, as software running on a digital signal processor or microprocessor. Other components can be implemented, for example, as hardware or application-specific integrated circuits (ASICs). Signals encountered in the described methods and systems can be stored on media such as random access memory or optical storage media. They can be transmitted via networks such as radio networks, satellite networks, wireless networks, or wired networks (e.g., the Internet). Typical devices using the methods and systems described in this document are portable electronic devices or other consumer devices for storing and / or presenting audio signals.

Claims

1. A method comprising: The audio processor receives multi-channel input audio signals and downmixing instructions, including a set of gain values. A first dynamic range control (DRC) value set is determined, the first DRC value set being configured to control the dynamic range of the output audio signal; A second set of DRC values ​​is determined, the second set of DRC values ​​being configured to prevent the multi-channel input audio signal from being trimmed during downmixing by the audio processor; The second set of DRC values ​​is applied to the multi-channel input audio signal to obtain a decayed multi-channel input audio signal; In response to the downmixing command, the attenuated multi-channel input audio signal is downmixed to obtain a downmixed signal; as well as The output audio signal is generated from the first set of DRC values ​​and the downmixed audio signal.

2. An apparatus comprising: One or more processors; A memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including the following: Receives multi-channel input audio signals and downmixing commands including a set of gain values; A first dynamic range control (DRC) value set is determined, the first DRC value set being configured to control the dynamic range of the output audio signal; A second set of DRC values ​​is determined, the second set of DRC values ​​being configured to prevent the multi-channel input audio signal from being trimmed during downmixing by the device; The second set of DRC values ​​is applied to the multi-channel input audio signal to obtain a decayed multi-channel input audio signal; In response to the downmixing command, the attenuated multi-channel input audio signal is downmixed to obtain a downmixed signal; as well as The output audio signal is generated from the first set of DRC values ​​and the downmixed audio signal.

3. The apparatus according to claim 2, wherein, Generating the output audio signal includes applying the first set of DRC values ​​to the downmixed audio signal.

4. The apparatus according to claim 2, wherein, The first set of DRC values ​​and / or the second set of DRC values ​​are represented as dB values ​​in logarithmic form.

5. The apparatus according to claim 2, wherein, The multi-channel input audio signal is divided into a frame sequence of samples of the multi-channel audio signal, and determining the first DRC value set and / or the second DRC value set includes determining the DRC value of each sample of each frame in the frame sequence.

6. The apparatus according to claim 5, wherein, Determining the DRC value for each sample of a frame involves interpolation between the DRC value of the frame and the DRC value of the previous frame.

7. The apparatus according to claim 6, wherein, The interpolation is spline interpolation.

8. The apparatus according to claim 2, wherein, The downmixed signal is a stereo signal.

9. The apparatus according to claim 2, wherein, The left and right channels of the downmix signal are generated based on different linear combinations of the channels of the multichannel input audio signal.

10. A non-transitory computer-readable storage medium comprising a sequence of instructions, the sequence of instructions causing the audio signal processing device to perform a method when executed by the audio signal processing device, the method comprising: The audio processor receives multi-channel input audio signals and downmixing instructions, including a set of gain values. A first dynamic range control (DRC) value set is determined, the first DRC value set being configured to control the dynamic range of the output audio signal; Determine a second set of DRC values, the second set of DRC values being configured to prevent the multichannel input audio signal from being clipped during downmixing by the audio processor; Apply the second set of DRC values to the multichannel input audio signal to obtain a attenuated multichannel input audio signal; In response to the downmixing instruction, downmix the attenuated multichannel input audio signal to obtain a downmixed signal; And Generate the output audio signal from the first set of DRC values and the downmixed audio signal.

11. A parameter processing unit (520) configured to determine a spatial metadata frame for generating a multichannel upmix signal from a corresponding frame of an undermix signal; wherein, The downmixed signal includes m channels, and wherein, the multichannel upmixed signal includes n channels; n, m are integers, where m < n; wherein, the spatial metadata frame includes one or more sets of spatial parameters (711, 712); the parameter processing unit (520) includes: - A transformation unit (521), the transformation unit (521) being configured to determine a plurality of spectra (589) from a current frame (585) and a following frame (590) of channels of a multichannel input signal (561); and - A parameter determination unit (523), the parameter determination unit (523) being configured to determine a spatial metadata frame for the current frame of the channels of the multichannel input signal (561) by weighting the plurality of spectra (589) using a window function (586); Wherein, the window function (586) depends on one or more of the following: the number of sets of spatial parameters (711, 712) included within the spatial metadata frame, the presence of one or more transients in the current frame or the following frame of the multichannel input signal (561), and / or the moment of the transients.

12. A parameter processing unit (520) configured to determine a spatial metadata frame for generating a multichannel upmix signal from a corresponding frame of an undermix signal; wherein, The downmixed signal includes m channels, and wherein, the multichannel upmixed signal includes n channels; n, m are integers, where m < n; wherein, the spatial metadata frame includes a set of spatial parameters (711); the parameter processing unit (520) includes: - A transformation unit (521), the transformation unit (521) being configured to: determine a first plurality of transformation coefficients (580) from a frame (585) of a first channel (561-1) of a multichannel input signal (561), and determine a second plurality of transformation coefficients (580) from a corresponding frame of a second channel (561-2) of the multichannel input signal (561); wherein, the first channel (561-1) and the second channel (561-2) are different; wherein, the first plurality of transformation coefficients (580) and the second plurality of transformation coefficients (580) respectively provide a first time / frequency representation and a second time / frequency representation of the frames (585) of the first channel and the second channel; wherein, the first time / frequency representation and the second time / frequency representation include a plurality of frequency intervals (571) and a plurality of time intervals (582); and - A parameter determination unit (523) configured to determine the set of spatial parameters (711) using fixed-point arithmetic based on the first plurality of transform coefficients (580) and the second plurality of transform coefficients (580); wherein the set of spatial parameters (711) includes corresponding band parameters for different frequency bands (572) including different numbers of frequency intervals (571); wherein a specific band parameter for the specific frequency band (572) is determined based on the transform coefficients (580) of the first plurality of transform coefficients (580) and the second plurality of transform coefficients (580) from the specific frequency band (572); and wherein the shift used in the fixed-point arithmetic for determining the specific band parameter depends on the specific frequency band (572).

13. An audio coding system (500) configured to generate a bitstream (564) based on a multichannel input signal (561); the system (500) comprising: - A downmix processing unit (510) configured to generate a sequence of frames of a downmix signal from a corresponding first sequence of frames of the multichannel input signal (561); wherein the downmix signal includes m channels, and wherein the multichannel input signal (561) includes n channels; n, m are integers, where m < n; - A parameter processing unit (520) configured to determine a sequence of spatial metadata frames from a second sequence of frames of the multichannel input signal (561); wherein the sequence of frames of the downmix signal and the sequence of spatial metadata frames are used to generate a multichannel upmix signal including n channels; and - A bitstream generation unit (503) configured to generate a bitstream (564) including a sequence of bitstream frames, wherein the bitstream frames indicate the frame of the downmix signal corresponding to the first frame of the first sequence of frames of the multichannel input signal (561) and the spatial metadata frame corresponding to the second frame of the second sequence of frames of the multichannel input signal (561); wherein the second frame is different from the first frame.

14. A method for determining a spatial metadata frame, the spatial metadata frame being used to generate a frame of a multichannel upmix signal from a corresponding frame of an downmix signal; wherein, The downmix signal includes m channels, and wherein the multichannel upmix signal includes n channels; n, m are integers, where m < n; wherein the spatial metadata frame includes one or more sets of spatial parameters (711, 712); the method comprising: - Determining a plurality of spectra (589) from the current frame (585) and the following frame (590) of the channels of the multichannel input signal (561); - Weighting the plurality of spectra (589) using a window function (586) to obtain a plurality of weighted spectra; and - Determining a spatial metadata frame for the current frame of the channels of the multichannel input signal (561) based on the plurality of weighted spectra. The window function (586) depends on one or more of the following: the number of sets of spatial parameters (711, 712) included in the spatial metadata frame, the presence of one or more transients in the current frame or the immediately following frame of the multichannel input signal (561), and / or the moments of the transients.

15. A method for determining a spatial metadata frame, the spatial metadata frame being used to generate a frame of a multichannel upmix signal from a corresponding frame of an downmix signal; wherein, The downmixed signal includes m channels, and wherein, the multichannel upmixed signal includes n channels; n and m are integers, where m < n; wherein, the spatial metadata frame includes a set of spatial parameters (711); the method includes: - Determining a first plurality of transform coefficients (580) from a frame (585) of a first channel (561-1) of the multichannel input signal (561); - Determining a second plurality of transform coefficients (580) from a corresponding frame of a second (561-2) channel of the multichannel input signal (561); wherein, the first channel (561-1) and the second channel (561-2) are different; Wherein, the first plurality of transform coefficients (580) and the second plurality of transform coefficients (580) respectively provide a first time / frequency representation and a second time / frequency representation of the frames (585) of the first channel and the second channel; wherein, the first time / frequency representation and the second time / frequency representation include a plurality of frequency intervals (571) and a plurality of time intervals (582); wherein, the set of spatial parameters (711) includes corresponding band parameters for different frequency bands (572) including different numbers of frequency intervals (571); - Determining a shift to be applied when using fixed-point arithmetic to determine a specific band parameter for a specific frequency band (572); wherein, the shift is determined based on the specific frequency band (572); and - Using fixed-point arithmetic and the determined shift, determining the specific band parameter based on the first plurality of transform coefficients (580) and the second plurality of transform coefficients (580) falling within the specific frequency band (572).

16. A method for generating a bitstream (564) based on a multichannel input signal (561); the method includes: - Generating a sequence of frames of a downmixed signal from a corresponding first sequence of frames of the multichannel input signal (561); wherein, the downmixed signal includes m channels, and wherein, the multichannel input signal (561) includes n channels; n and m are integers, where m < n; - Determining a sequence of spatial metadata frames from a second sequence of frames of the multichannel input signal (561); wherein, the sequence of frames of the downmixed signal and the sequence of spatial metadata frames are used to generate a multichannel upmixed signal including n channels; and - Generating a bitstream (564) including a sequence of bitstream frames; Wherein, the bitstream frames indicate the frame of the downmixed signal corresponding to the first frame of the first sequence of frames of the multichannel input signal (561) and the spatial metadata frame corresponding to the second frame of the second sequence of frames of the multichannel input signal (561); wherein, the second frame is different from the first frame.

Images