Joint stereo coding in complex valued filter bank domain

Abstract

Systems and methods for joint stereo coding. A method for stereo audio signal encoding includes passing a left channel and a right channel of a stereo audio signal to a complex valued filter bank analysis section and calculating an energy of the left channel, an energy of the right channel, and a covariance of the left channel and the right channel. The method includes selecting a stereo encoding mode for encoding a left channel and a right channel. The method comprises: calculating a phase difference between a left channel and a right channel, adjusting phase alignment between the left channel and the right channel based on the phase difference, converting the aligned left channel and the aligned right channel into an intermediate signal and a side signal, and generating a residual signal based on side prediction data and the side signal; and encoding the intermediate signal, the residual signal, the phase difference, and the side prediction data in a bit stream, and transmitting the bit stream.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 559,764, filed February 29, 2024, and U.S. Provisional Application No. 63 / 491,840, filed March 23, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application generally relates to audio and speech coding, and more specifically, to center-side stereo coding. Background Technology

[0004] Unless otherwise stated herein, the materials described in this section are not prior art to the claims of this application and are not considered prior art simply because they are included in this section.

[0005] Joint coding of the left (L) and right (R) channels of a stereo signal achieves more efficient coding compared to independent coding of the L and R channels. One method of joint stereo coding is so-called center / side (M / S) coding. In center-side coding, the center (M) signal is formed as a combination of the L and R channel signals, while the side (S) signal can be formed as the difference between the L and R channel signals. For example, the M signal can be in the form M = (L + R) / 2; and the S signal can be in the form S = (LR) / 2. In the case of M / S coding, the M and S signals are encoded, rather than the L and R signals.

[0006] In some implementations, M / S stereo coding can be implemented in a time- and frequency-variable manner. For example, a stereo encoder can apply L / R coding to encode some frequency bands of the stereo signal, while M / S coding can be used to encode other frequency bands of the stereo signal (frequency variation). Furthermore, some encoders can switch between L / R and M / S coding (time-varying) methods over time.

[0007] Taking into account these and other factors, this disclosure is presented herein. Summary of the Invention

[0008] Techniques for processing audio signals are described. Various embodiments described herein provide systems, methods, and / or apparatuses extended to center-side stereo coding applied in the complex-valued filter domain. For example, by incorporating real-valued prediction of the side signal from the center signal in the encoder, embodiments provide adjusted phase alignment between the left and right audio channels prior to center-side conversion. Furthermore, embodiments described herein provide a novel method for performing phase alignment and transmitting complex audio data in the complex domain.

[0009] The inter-channel phase difference, which can be transmitted in the bitstream as metadata, is used to reconstruct the original inter-channel phase relationship at the decoder. Since the signal operates in the complex domain, a phase alignment operation can be applied without the risk of information loss or degradation, which is not typically the case when only real-valued data is encoded. Furthermore, different processing blocks (e.g., phase alignment, mid-side conversion, and side signal prediction blocks) are enabled or disabled based on a psychoacoustic model that relies on the level and, in some cases, on the parametric side-rate cost.

[0010] Compared to known mid-side encoding, the embodiments described herein improve spatial noise shaping (by preventing spatial unmasking) and improve energy compression, especially for signals with systematic inter-channel level differences and inter-channel phase shifts or time delays. This property is common for binaural signals, which are generated by filtering audio objects or channels with head-related transfer functions. Using the mid-side encoding extension described herein, the audio quality is improved at a given bit rate, or the bit rate can be reduced at a certain quality level.

[0011] In some aspects, the embodiments described herein include an encoder that receives a left channel (e.g., a left input signal) and a right channel (e.g., a right input signal) as binaural channels. A complex filter bank analysis is performed on the left channel and the right channel to convert the left channel and the right channel to a complex-valued filter bank domain. Converting the left channel and the right channel to the complex-valued filter bank domain prepares the left channel and the right channel for presentation by a head tracking device, such as processing by a head-related transfer function (HRTF). For example, the complex-valued filter bank domain signals can be one or more frequency bands.

[0012] In some aspects, a stereo analysis is performed on the complex-valued filter bank domain left channel and the complex-valued filter bank domain right channel. The stereo analysis can identify an energy of the complex-valued filter bank domain left channel and an energy of the complex-valued filter bank domain right channel. In addition, the stereo analysis can identify an energy of a potential mid signal and an energy of a potential side signal. The mid signal represents a sum of the left channel and the right channel. The side signal represents a difference between the left channel and the right channel. In some cases, an energy of a potential residual signal is determined.

[0013] In some aspects, the stereo analysis also generates stereo metadata based on the left channel and the right channel. For example, a covariance of the complex-valued filter bank domain left channel and the complex-valued filter bank domain right channel can be computed. A inter-channel phase difference of the left channel and the right channel is determined based on the covariance. In addition, real-valued prediction coefficients (e.g., side prediction coefficients) are computed based on the energy of the complex-valued filter bank domain left channel and the complex-valued filter bank domain right channel, and the energy of the potential mid signal and the potential side signal.

[0014] In some aspects, a single stereo coding mode is selected from a plurality of stereo coding modes to signal and encode the left and right channels. For example, for each possible stereo coding mode, a bit cost is determined for signaling the left and right channels in that stereo coding mode. The bit cost is based on energies of the complex-valued filterbank domain left channel, the complex-valued filterbank domain right channel, a possible mid signal, a possible side signal, and a possible residual signal. In some cases, a ratio of the energies of the signals involved in each stereo coding mode is determined. The stereo coding mode is selected based on the bit cost. In some cases, the stereo coding mode with the lowest bit cost is selected.

[0015] In some aspects, the left and right channels are processed according to the selected stereo coding mode. As one example, when the stereo coding mode is a separate coding mode, the stereo processor directly performs the signaling of the left and right channels (or the complex-valued filterbank domain left channel and the complex-valued filterbank domain right channel) without converting the signals into a mid signal and a side signal.

[0016] In another example, the selected stereo mode is a mid / side stereo mode with adjusted phase alignment. In this case, the left and right channels are phase aligned based on an inter-channel phase difference. After alignment, the left and right channels are converted into a mid signal and a side signal. The stereo processor performs the signaling of the mid signal and the side signal, and the phase difference is encoded along with the mid signal and the side signal.

[0017] In another example, the selected stereo mode is a mid / side stereo mode with adjusted phase alignment. In this case, the left and right channels are phase aligned based on an inter-channel phase difference. After alignment, the left and right channels are converted into a mid signal and a side signal. The stereo processor performs the signaling of the mid signal and the side signal, and the phase difference is encoded along with the mid signal and the side signal.

[0018] In another example, the selected stereo mode is a mid / side stereo mode with adjusted phase alignment. In this case, the left and right channels are phase aligned based on an inter-channel phase difference. After alignment, the left and right channels are converted into a mid signal and a side signal. The stereo processor performs the signaling of the mid signal and the side signal, and the phase difference is encoded along with the mid signal and the side signal.

[0019] In another example, the selected stereo mode is a mid / side stereo mode with adjusted phase alignment. In this case, the left and right channels are phase aligned based on an inter-channel phase difference. After alignment, the left and right channels are converted into a mid signal and a side signal. The stereo processor performs the signaling of the mid signal and the side signal, and the phase difference is encoded along with the mid signal and the side signal.

[0020] One example provides a method for encoding a stereo audio signal into a bitstream. The method includes passing a left channel and a right channel of the stereo audio signal to a complex-valued filter bank analysis block to responsively generate one or more frequency bands, and for each of the one or more frequency bands, computing an energy of the left channel, an energy of the right channel, and a covariance of the left channel and the right channel. The method includes selecting a stereo encoding mode for encoding the left channel and the right channel based on the computed energy of the left channel, the computed energy of the right channel, and the computed covariance of the left channel and the right channel. When the stereo encoding mode is an extended mid / side encoding mode, the method includes computing a phase difference between the left channel and the right channel, adjusting a phase alignment between the left channel and the right channel based on the computed phase difference to generate an aligned left channel and an aligned right channel, transforming the aligned left channel and the aligned right channel into a mid signal and a side signal, generating a residual signal based on side prediction data and the side signal, encoding the mid signal, the residual signal, the phase difference, and the side prediction data into the bitstream, and outputting the bitstream for the selected stereo encoding mode.

[0021] Another example provides an apparatus for encoding a stereo audio signal into a bitstream, the apparatus including an electronic processor. The electronic processor is configured to pass a left channel and a right channel of the stereo audio signal to a complex-valued filter bank analysis block to responsively generate one or more frequency bands, for each of the one or more frequency bands, compute an energy of the left channel, an energy of the right channel, and a covariance of the left channel and the right channel, and select a stereo encoding mode for encoding the left channel and the right channel based on the computed energy of the left channel, the computed energy of the right channel, and the computed covariance of the left channel and the right channel. When the stereo encoding mode is an extended mid / side encoding mode, the electronic processor is configured to compute a phase difference between the left channel and the right channel, adjust a phase alignment between the left channel and the right channel based on the computed phase difference to generate an aligned left channel and an aligned right channel, transform the aligned left channel and the aligned right channel into a mid signal and a side signal, generate a residual signal based on side prediction data and the side signal, encode the mid signal, the residual signal, the phase difference, and the side prediction data into the bitstream, and output the bitstream for the selected stereo encoding mode.

[0022] Another example provides a method for decoding a stereo audio signal. The method includes receiving an encoded bitstream, decoding a replica mid signal, a replica residual signal, and a replica stereo metadata from the bitstream, wherein the replica stereo metadata includes a phase difference and side prediction data, converting the replica mid signal and the replica residual signal into a replica left channel and a replica right channel using the replica stereo metadata, and passing the replica left channel and the replica right channel to a filter bank analysis block to reconstruct an original left channel and an original right channel.

[0023] Another example provides an apparatus for decoding a stereo audio signal, the apparatus comprising an electronic processor. The electronic processor is configured to receive an encoded bitstream, decode a replica mid signal, a replica residual signal, and a replica stereo metadata from the bitstream, wherein the replica stereo metadata includes a phase difference and side prediction data, convert the replica mid signal and the replica residual signal into a replica left channel and a replica right channel using the replica stereo metadata, and pass the replica left channel and the replica right channel to a filter bank analysis block to reconstruct an original left channel and an original right channel.

[0024] As such, various aspects of the present disclosure provide for processing of stereo audio signals and effect improvements in at least the technical fields of audio encoding, audio decoding, virtual reality, etc.

[0025] Embodiments described herein can be generally described as technology, where the term "technology" can refer to one or more systems, one or more devices, one or more methods, one or more computer-readable instructions, one or more modules, one or more components, hardware logic, and / or one or more operations as suggested by the context of application herein.

[0026] Features and technical benefits other than those specifically described herein, however, will become apparent to one of ordinary skill in the art upon reading the following detailed description and figures. This Summary is intended to introduce the reader to selected concepts related to techniques in a simplified form, and is not intended to identify key or essential features of the claimed subject matter. BRIEF DESCRIPTION OF DRAWINGS

[0027] These and other more detailed and specific features of various embodiments are disclosed in the following description and in association with the associated drawings, in which:

[0028] Figure 1 A block diagram of an example audio encoding system in which various aspects of the present disclosure can be incorporated is shown.

[0029] Figure 2 A block diagram of an example encoder is shown.

[0030] Figure 3 A block diagram of an example stereo processing process is shown.

[0031] Figure 4A An exemplary joint stereo processing transform is shown.

[0032] Figure 4B A table is shown providing each joint stereo coding type and its associated bitstream syntax element.

[0033] Figure 5A A block diagram of an exemplary mode-based stereo processing unit is shown, e.g., a mode-based stereo processing unit of Figure 3 .

[0034] Figure 5B A table is shown providing an exemplary bitstream syntax.

[0035] Figure 6A A chart is shown illustrating the stereo metadata rate for the extension mid / side coding mode of each audio frame, in accordance with various aspects of the present disclosure.

[0036] Figures 6B-6D An example of pseudo code is shown.

[0037] Figures 7A-7B A block diagram of various exemplary methods for encoding a stereo signal, in accordance with various aspects of the present disclosure, is shown, which can be performed by an encoder of Figure 2 .

[0038] Figure 8A A block diagram of an exemplary decoder is shown.

[0039] Figure 8B An example of pseudo code is shown.

[0040] Figure 9 A block diagram of various exemplary methods for decoding a stereo signal, in accordance with various aspects of the present disclosure, is shown, which can be performed by a decoder of Figure 8A .

[0041] Figure 10 A plot of a PEQ evaluation of twelve audio items is shown.

[0042] Figure 11A A schematic block diagram of an exemplary device architecture that can be used to implement various aspects of the present disclosure is shown.

[0043] Figure 11B A schematic block diagram of an exemplary CPU implemented in the device architecture of Figure 11A that can be used to implement various aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0044] In the following description, numerous specific details are set forth such as audio device configurations, timings, operations, etc., in order to provide a thorough understanding of one or more aspects of the present disclosure. It will be apparent, however, to one skilled in the art that these specific details need not be used to practice the present application.

[0045] As used herein, the term "includes" and its variants are to be read to be analogous to an open term denoting "including, but not limited to." The term "or" should be understood to be inclusive, i.e., "and / or" unless the context clearly indicates otherwise. The term "based on" should be understood as "based at least in part on." The terms "one example implementation" and "an example implementation" should be understood as "at least one example implementation." The term "another implementation" should be understood as "at least one other implementation." The term "determining" should be understood as obtaining, receiving, calculating, computing, estimating, predicting or deriving. Furthermore, all

[0046] Various acronyms can appear throughout the disclosure and in the related claims and / or drawings listed below. Other commonly used abbreviations and terms can be excluded from this list for brevity. Accordingly, a short list of acronyms is provided below as a simple reference for the reader.

[0047] ERB - Equivalent Rectangular Bandwidth

[0048] PEAQ - Perceptual Evaluation of Audio Quality

[0049] ODG - Objective Difference Grade

[0050] IVAS - Immersive Voice and Audio Services

[0051] MDFT - Modified Discrete Fourier Transform

[0052] MDCT - Modified Discrete Cosine Transform

[0053] HRTF - Head Related Transfer Function

[0054] Figure 1 A block diagram of an audio encoding system in which various aspects of the present application can be incorporated is shown. The example audio encoding system 100 includes an encoder 110 and a decoder 120. The input to the encoder 110 corresponds to a first signal path 105, while the output of the encoder 100 corresponds to a second signal path 115. The input to the decoder 120 corresponds to the second signal path 115, while the output of the decoder corresponds to a third signal path 125.

[0055] The encoder 110 is configured to receive one or more audio information streams representing one or more audio signal channels from the first signal path 105. The encoder 110 is further configured to process the received audio information streams and generate an encoded signal, which can be output to the second signal path 115. At the second signal path 115, the encoded signal can be stored (e.g., captured, buffered, and / or recorded), or transmitted (e.g., via a wired or wireless communication medium). The decoder 120 is configured to receive the encoded signal from the second signal path 115. The decoder 120 is further configured to process the received encoded signal and generate a decoded signal, which can be output to the third signal path 125. The decoded signal generated by the decoder 120 corresponds to a copy of the audio information previously received by the encoder 110 from the first signal path 105. At the third signal path, the decoded signal can be stored (e.g., captured and / or recorded), transmitted (e.g., via a wireless or wired electronic communication medium), or output to a listening device (e.g., an audio processing device such as a receiver, a speaker, a sound bar, etc.). The audio encoding system 100 can be an audio system capable of implementing an audio codec standard such as the Immersive Voice and Audio Services (IVAS) standard. In this case, the encoded signal at the second signal path 115 can correspond to an IVAS bitstream.

[0056] In various examples described herein, the terms “copy” and “copy signal” do not imply that the audio information streams are “identical.” Rather, the term “copy” can indicate that the audio information stream is substantially identical to the original audio information stream. For example, when the encoder 110 uses a lossless encoding technique to generate the encoded signal, the decoder 120 can in principle recover a lossless version of the original audio information from the stream that is substantially identical to the original audio information. However, in examples where the encoder 110 uses a lossy encoding technique (e.g., for perceptual coding), the content of the recovered copy signal is typically different from the content of the original stream, but it can be perceptually indistinguishable from the original content. Thus, the terms “copy” and “copy signal” are intended to encompass both lossless and lossy encoding techniques used herein.

[0057] Figure 2An example of an encoder 110 is shown. The encoder 110 includes a complex filterbank analysis block 205, a stereo processing block 210, an encoding block 215, and a bitstream writing block 220. The complex filterbank analysis block 205 is configured to receive left and right audio channels (e.g., from one or more microphones). The left and right audio channels can be binaural channels that have a small delay between the channels and have a correlation between the channels. The left and right audio channels can be processed by a head-related transfer function (HRTF). In some cases, the left and right audio channels are Ambisonics signals. The complex filterbank analysis block 205 is also configured to process the left and right audio channels to generate a complex-valued filterbank domain left signal and a complex-valued filterbank domain right signal. The complex filterbank analysis block 205 is configured to output the complex-valued filterbank domain left signal and the complex-valued filterbank domain right signal to the stereo processing block 210. The complex-valued filterbank domain signals can include additional metadata processed by a head-tracking device. In some examples, the complex-valued filterbank domain signals are one or more frequency bands associated with a left channel and a right channel. In some examples, the complex filterbank analysis can be a perfect reconstruction, such as for a modified discrete Fourier transform (MDFT), a modified discrete cosine / sine transform (MDCT), or a near perfect reconstruction, such as a complex modulated filterbank.

[0058] The stereo processing block 210 is configured to receive the complex-valued filterbank domain left signal and the complex-valued filterbank domain right signal from the complex filterbank analysis block 205. The stereo processing block 210 is configured to perform a stereo processing analysis by aggregating filterbank frequency bins associated with a frame into frequency bands according to an auditory frequency scale, such as an equivalent rectangular bandwidth (ERB) scale or a Bark scale. For each frequency band, the energy of the left channel and the right channel, as well as the covariance from the two channels, is computed. In addition, the energy of the mid and side signals is computed for each frequency band. A correlation coefficient for each frequency band is computed from the covariance. When the correlation coefficient is above a threshold, a phase difference between the left channel and the right channel can be computed and used to adjust the phase alignment. The covariance can be updated based on the quantized phase difference.

[0059] In addition, the stereo processing block 210 can compute real-valued prediction coefficients that can be used to remove redundancy in the side signal with respect to the mid signal. The adjusted phase alignment can result in an increase in the mid signal energy and a decrease in the side signal energy. In addition, the adjusted phase alignment can ensure that the side signal energy is always lower than the mid signal energy, which can result in a situation where the left signal and the right signal partially cancel each other out in the mid signal for certain frequencies. A residual energy of the side signal can be computed based on the quantized prediction coefficients. Further details regarding the operation of the stereo processing block 210 are provided below.

[0060] The stereo processing block 210 outputs the computed signal energies and the computed stereo processing metadata to an encoding block 215. The encoding block 215 is configured to receive the computed signal energies and the computed stereo processing metadata from the stereo processing block 210. The encoding block 215 is further configured to encode the computed signal energies and the stereo processing metadata into an encoded signal. The encoding block 215 outputs the encoded computed signal energies and the encoded stereo processing metadata to a bitstream writing block 220. The bitstream writing block 220 is configured to receive the encoded computed signal energies and the encoded stereo processing metadata. The bitstream writing block 220 is configured to convert the encoded computed signal energies and the encoded stereo processing metadata into a bitstream. The bitstream can be output by the bitstream writing block 220.

[0061] The stereo processing is based on the determined stereo encoding mode. Some exemplary stereo encoding modes or methods include at least: a left / right mode, a mid / side mode, and an extended mid / side mode. The selected stereo encoding mode for a certain frequency band can be determined based on an estimated or computed inter-channel energy difference of the signal pair for each possible mode. For example, the encoding mode decision can be based on the estimated number of bits required to encode each signal for the corresponding encoding mode pair. The estimate of the number of bits required can be based on a level-dependent psychoacoustic model that allocates more bits to louder signal portions than to quieter signal portions. Thus, the signal pair with the highest energy difference can require the least number of bits at a certain quality level. In some cases, the bits required to encode the stereo metadata can also be considered in determining the most efficient encoding method. Furthermore, in some cases, the same level-dependent psychoacoustic model can also be used to select the encoding method for the audio data. For the mid / side mode, the bit savings estimate relative to left / right encoding can be computed and added up to determine the total bit savings per frame. After deducting the corresponding metadata bits, the most efficient (e.g., having the lowest cost) stereo encoding mode in terms of required bits can be selected.

[0062] Figure 3An exemplary stereo processing process or method, such as the process or method performed by stereo processing block 210, is shown. Exemplary stereo processing block 210 includes a stereo analysis unit 305, a psychoacoustic model 310, a bit cost estimator unit 315, a mode decision unit 320, and a mode-based stereo processing unit 325. Stereo analysis unit 305 is configured to receive complex-valued filter bank domain left and right signals from complex filter bank analysis block 205. Stereo analysis unit 305 determines, by computation or estimation, the energy of the complex-valued filter bank domain left and right signals, the energy of the mid and side signals, and the residual energy. Stereo analysis unit 305 is also configured to provide this energy to bit cost estimator unit 315. Stereo analysis unit 305 also determines, by computation or estimation, stereo metadata based on the complex-valued filter bank domain left and right signals. The stereo metadata can include, for example, one or more of the phase difference of the complex-valued filter bank domain left and right signals, the mid and side signals, prediction coefficients, and / or covariances.

[0063] The stereo metadata is provided by stereo analysis unit 305 to mode-based stereo processing unit 325 and bit cost estimator unit 315. Bit cost estimator unit 315 estimates the number of bits required to encode the signals based on the determined energy provided by stereo analysis unit 305, and based on psychoacoustic model 310. For example, bit cost estimator unit 315 estimates the number of bits required to encode the signals for each candidate (or possible) stereo encoding mode. Bit cost estimator unit 315 is configured to send the estimated number of bits required to mode decision unit 320. Mode decision unit 320 is configured to receive the estimated number of bits required, and in response, determine a stereo encoding mode (e.g., select one of the candidate stereo encoding modes) based on the estimated number of bits required. Mode decision unit 320 is also configured to provide the selected (or determined) stereo encoding mode to mode-based stereo processing unit 325. Mode-based stereo processing unit 325 is configured to receive the selected stereo encoding mode from mode decision unit 320. Mode-based stereo processing unit 325 signals the processed left and right signals using, for example, two bits per frame (e.g., signal 0 and signal 1). The processed left and right signals can correspond to the left and right signals, the mid and side signals, or the mid and residual signals, as described in more detail below.

[0064] Figure 4AAn exemplary joint stereo process 400 performed by the stereo analysis unit 305 is shown. By performing the process 400, the stereo analysis unit 305 analyzes the complex filter bank domain left and right signals received from the complex filter bank analysis block 205. For example, the stereo analysis unit 305 receives a complex-valued filter bank domain left signal L and a complex-valued filter bank domain right signal R. The left input signal L is provided according to Equation 1:

[0065] L(l,n) = L r (l,n) + jL i (l,n) [Equation 1]

[0066] where l is a filter bank frequency bin, n is a filter bank time index, L r is the real part of the left input signal L, and L i is the imaginary part (e.g., complex part) of the left input signal L.

[0067] The right input signal R is provided according to Equation 2:

[0068] R(l,n) = R r (l,n) + jR i (l,n) [Equation 2]

[0069] where R r is the real part of the right input signal R, and R i is the imaginary part of the right input signal R.

[0070] R i

[0071] In some cases, the right input signal R is phase-rotated prior to calculating the mid signal M and the side signal S. The phase-rotated right signal R' is provided according to Equation 3:

[0072]

[0073] where is a phase rotation angle.

[0074] Next, the mid signal M is determined according to Equation 4:

[0075] M(l,n) = 0.5(L(l,n) + R'(l,n)) [Equation 4]

[0076] and the side signal S is determined according to Equation 5:

[0077] S(l,n) = 0.5(L(l,n) - R'(l,n)) [Equation 5]

[0078] A residual signal S' is also determined, as provided by Equation 6:

[0079] S′(l,n)=S(l,n)-pM(l,n) [Equation 6]

[0080] Where p is the real-valued prediction coefficient (and is provided in Equation 15 below).

[0081] Stereo processing parameters (provided as stereo metadata) are also generated by stereo analysis unit 305. These parameters are based on the energy and covariance metrics of each perceptual band k in the audio frame f (omitted in the calculations below for clarity). Therefore, energy compression estimates for various stereo coding types, referenced by mode determination unit 320, are calculated.

[0082] The energy of the left input signal L is provided by Equation 7:

[0083]

[0084] Where * denotes complex conjugation, N is the number of filter bank slots per frame, and n f It is the first filter bank slot index in audio frame f, and and These correspond to the first and last filter bank frequency bins associated with frequency band k. In some instances, the total number of frequency bands (K) is approximately 23, and the total number of filter bank frequency bins is greater than K (e.g., 60, 64, 256, or more filter bank frequency bins).

[0085] The energy for the right input signal R is provided by Equation 8:

[0086]

[0087] Equation 9 provides the covariance of the left input signal L and the right input signal R:

[0088]

[0089] The inter-channel phase difference that can be used to maximize the intermediate signal energy is provided by Equation 10:

[0090]

[0091] The threshold Thr can be greater than or equal to 0.5, but less than 1.0.

[0092] The real part of the covariance of the phase-rotated right signal R' according to the quantized phase can be derived from the initial covariance using Equation 11:

[0093]

[0094] in yes The quantized version is defined by Formula 12:

[0095]

[0096] wherein, and x is an integer greater than 0.

[0097] The energy of the mid signal M is calculated according to equation 13:

[0098]

[0099] The energy of the side signal S is calculated according to equation 14:

[0100]

[0101] The prediction coefficient p that minimizes the energy of the residual signal S' is calculated according to equation 15:

[0102]

[0103] The residual signal energy is calculated according to equation 16:

[0104]

[0105] where p k,Q is a quantized version of p k .

[0106] Now referring to the operations performed by the mode decision unit 320 of Figure 3 for each frequency band k, four different types of joint stereo coding (e.g., advanced stereo coding modes) are considered. Figure 4B Table 1 shown in FIG. 1 1 provides each joint stereo coding type and its associated bitstream syntax elements. In Table 1, type "0" corresponds to separate coding of the left and right signals and is not a joint stereo coding type. The joint coding types are associated with a corresponding stereo mode MSMode. The relevant bitstream elements are provided to the mode-based stereo processing unit 325.

[0107] The stereo coding type t=0 corresponds to separate coding of the left and right signals. The stereo coding type t=1 corresponds to standard mid / side coding. The stereo coding type t=2 corresponds to extended mid / side coding with adjusted phase alignment only. The stereo coding type t=3 corresponds to extended mid / side coding with side prediction only. The stereo coding type t=4 corresponds to extended mid / side coding with adjusted phase alignment and side prediction. The stereo coding type of any one of t=1, 2, 3, or 4 corresponds to a joint coding type.

[0108] To determine the joint stereo mode, the bit cost estimator unit 315 determines an energy ratio quantity for each potential joint coding type (e.g., t = 1, 2, 3, or 4) and the separate coding type (e.g., t = 0), as defined by Equations 17 and 18 below:

[0109]

[0110] When the ratio for a certain potential joint coding type t (= 1, 2, 3, 4) is greater than the ratio for the separate coding type t = 0, then that certain joint coding type is marked for use for the corresponding frequency band k (e.g., a flag is set), as shown in Equation 19:

[0111]

[0112] The reduction in required bits due to energy compression from joint coding is evaluated according to Equation 20:

[0113]

[0114] where smrSlope k is a factor (and is typically between the values of 0.25 and 0.4) provided by the psychoacoustic model 310 that relates levels above the quiet threshold in dB to the required signal-to-mask ratio for just noticeable noise in dB. In some cases, smrSlope k is a frequency band dependent factor. One example of a psychoacoustic model 310 can be found in U.S. Patent Publication No. 2022 / 0415334, “A Psychoacoustic Model for Audio Processing,” which is incorporated by reference herein in its entirety. The constant bitsPerdB is an estimate of the number of bits required to quantize a value to achieve a particular signal-to-noise ratio (SNR). The SNR depends on the codec used and can be, for example, approximately 0.3.

[0115] The total reduction in required bits for a certain joint coding type is computed by summing the relevant reductions in all frequency bands, as provided by Equation 21:

[0116]

[0117] Consideration is given to the signaling cost according to the bitstream syntax Reference is made to Table 2 below. The mode decision unit 320 selects the most efficient (e.g., lowest cost) joint stereo coding type t, as provided by Equation 22:

[0118] t best = argmax(B t) [Formula 22]

[0119] In some embodiments, the joint coding type can be selected to be more than a certain percentage of the bit reduction amount of another joint coding type.

[0120] The mode-based stereo processing unit 325 is configured to receive the stereo metadata (which includes the quantized phase difference and the quantized prediction coefficients) from the stereo analysis unit 305 and the selected joint stereo coding type t from the mode decision unit 320. The mode-based stereo processing unit 325 performs processing of the left input signal L and the right input signal R based on the selected joint stereo coding type t (e.g., the selected stereo mode).

[0121] In the case where the separate coding type t = 0 is selected, the mode-based stereo processing unit 325 can simply pass the left input signal L and the right input signal R to the encoding block 215 without changing or altering the signals. In other cases where the coding type t = 1, 2, 3, or 4 is selected, mid / side mode, the mode-based stereo processing unit 325 processes the left input signal L and the right input signal R according to the selected joint stereo coding type t and the stereo metadata.

[0122] Figure 5A A block diagram of an exemplary mode-based stereo processing unit (e.g., the mode-based stereo processing unit 325 of Figure 3 ) is shown. The illustrated example can be used when the mid / side mode is selected. The exemplary mode-based stereo processing unit 325 includes a phase alignment block 505, a mid / side conversion block 510, and a side prediction block 515. The phase alignment block 505 is configured to receive the left input signal L, the right input signal R, and the quantized phase difference. The phase alignment block 505 is only needed for stereo coding types t = 2 and t = 4. The phase alignment block 505 uses the quantized phase difference to perform a phase alignment adjustment between the left input signal L and the right input signal R. In some examples, the adjustment for phase alignment can be applied to one of the L or R signals, while in other examples, the adjustment for phase alignment can be applied to both the L and R signals. The phase-aligned left input signal L and the phase-aligned right input signal R are output by the phase alignment block 505 to the side conversion block 510. In other joint stereo coding types, e.g., t = 1 and t = 3, the phase alignment block 505 can be disabled or bypassed (e.g., the phase difference can be set to 0).

[0123] ​The mid / side conversion block 510 is configured to receive the left input signal L and the right input signal R, which can be phase aligned as described above, from the phase alignment block 505. The mid / side conversion block 510 converts the left input signal L and the right input signal R into a mid signal M and a side signal S (and, in some cases, a residual signal S'), as previously described with reference to Figure 4A The side conversion block 510 outputs the mid signal M and the side signal S to the side prediction block 515.

[0124] The side prediction block 515 can be implemented for stereo coding types t = 3 and t = 4. The side prediction block 515 receives the mid signal M and the side signal S from the side conversion block 510 and the quantized prediction coefficients p k,Q from the mode decision unit 320. The side prediction block 515 applies side prediction to the mid signal M and the side signal S. In other joint stereo coding types, such as t = 1 and t = 2, the side prediction block 515 is disabled or bypassed (e.g., p k,Q is set to 0).

[0125] The stereo mode can be signaled, for example, from the mode-based stereo processing unit 325 by 2 bits per frame. In this example, the mid / side encoding versus left / right encoding can be signaled by one bit per frequency band. Alternatively, mid / side encoding valid for all frequency bands can be indicated by one bit per frame. For the extended mid / side mode, the presence of the phase difference data can be signaled by one bit per frame, and the side prediction data can be signaled by one bit per frame. In this way, the metadata size can be reduced for different types of stereo signals. If the side prediction data is present, the side prediction data can be Huffman entropy coded according to the differences between the relevant frequency bands. If the phase difference data is present, the phase difference data can be entropy coded. Figure 5B Table 2, shown in FIG. 2, provides an example bitstream syntax.

[0126] During encoding (at the encoding block 215), the inter-channel phase difference data is linearly quantized. The scaling factor used for quantization is chosen such that the values π and -π are represented exactly as shown in Equation 12. In some implementations, the second order differences of the phase signs along the relevant frequency bands (e.g., the quantized inter-channel phase differences for each frequency band) can be calculated and wrapped into the range [-π, +π] such that jumps larger than π are not needed. The second order differences can be calculated by first calculating the difference between the phase signs in frequency bands 1 through N and the phase signs in frequency bands 0 through N-1, where N is the number of frequency bands in which the M / S processing with phase adjustment is active (e.g., see Table 2 in Figure 5B Figure 6B ​The variable "numMSBands" in pseudocode 1). In the second step, as... Figure 6C As shown in pseudocode 2, the second-order difference is calculated as the difference between the new values ​​in frequency bands 2 to N and the new values ​​in frequency bands 1 to N. Thus, the quantized values ​​in the first frequency band can be unmodified, the quantized values ​​in the second frequency band can represent the difference, and the quantized values ​​in the remaining frequency bands can correspond to the differences from the first difference. Entropy coding can be performed on the coiled phase symbols, where zero symbols can be encoded using a minimum number of bits. The signal model has inter-channel time delays or a constant phase shift. In some cases, for linear frequency bands, the second-order frequency difference is essentially zero, thus requiring only a small number of bits for encoding. In other cases, the second-order frequency difference is essentially non-zero. Furthermore, in some cases, only the first-order difference is used for encoding. In any case, the described entropy coding scheme for inter-channel phase differences using second-order frequency difference coding is more efficient (in the sense of using fewer bits for the same information) compared to ordinary frequency difference coding or non-differential coding for stereo coded content (e.g., binaural stereo).

[0127] Figure 6A This illustrates the stereo metadata rate for the extended middle / side coding mode per audio frame for a project with five samples (48kHz) of interchannel delay across multiple frames, when using first-order or second-order differential coding. Figure 6A In the diagram, the x-axis represents multiple frames, and the y-axis represents the encoded bit rate (kB / s) for each frame. For example... Figure 6A As shown, when encoding the quantization phase angle of the joint coding band, as... As shown, the quantization phase angles can first be arranged adjacently. As a concrete example of the operation, see... Figure 6B The pseudocode 1 is shown. Next, for the arranged quantization phase angles, the first-order difference and second-order difference are calculated. Finally, before Huffman coding, the data is wound into the quantized 2π range. Figure 6C The pseudocode shown in Figure 2 provides an example of calculating the first and second differences, presented in a C-like format. Figure 6D As shown, pseudocode 3 provides an example of wrapping the phase sign (variable phaseQ) within the 2π range.

[0128] Phase symbols represent absolute phase (first band), first-order difference (second band), or second-order difference between bands (other bands). If first-order difference is used to encode phase data, the same phase winding described above can be applied. Figure 6DIn the pseudo code 3 shown, the value of PHASE_MAX_VAL represents a value for +π and the value of PHASE_MIN_VAL represents a value for -π, where the phase sign is represented by an integer in the range [PHASE_MIN_VAL, PHASE_MAX_VAL], where PHASE_MIN_VAL is defined as the negative of PHASE_MAX_VAL, and PHASE_MAX_VAL is the integer x (e.g., x = 12) used for quantization in equation 12.

[0129] Figures 7A-7B A block diagram illustrating various example methods 700 for stereo signal encoding is shown, which can be performed by an encoder 110 of Figure 2 The method 700 can be performed by a processor, which can be configured to perform the method 700 via machine-executable instructions. The method 700 can be divided into various blocks or sections, such as blocks 705, 710, 715, 720, 725, 730, 735, 740, 745, 750, 755, 760, 765, and 770. The various process blocks shown in FIG. 7 provide examples of the various methods disclosed herein, and it should be understood that some blocks can be eliminated, added, combined, or modified without departing from the spirit of the disclosure. For some examples, processing of the various blocks that can be described as processes, methods, steps, blocks, operations, or functions can begin at block 705.

[0130] At block 705, “passing left and right channels of a stereo audio signal to filter bank analysis,” the example method 700 can include passing left and right channels of a stereo audio signal to filter bank analysis. The filter bank analysis responsively generates one or more frequency bands. Processing can continue from block 705 to block 710.

[0131] At block 710, “computing energy of left channel, energy of right channel, and covariance,” the example method 700 can include computing energy of a left channel, computing energy of a right channel, and computing covariance of the left and right channels, as previously described with respect to equations 7-16. Processing can continue from block 710 to block 715.

[0132] At block 715, “determining bit costs for a plurality of stereo encoding modes,” the example method 700 can include determining bit costs for a plurality of stereo encoding modes, as previously described with respect to equations 17-22 and Table 1. Processing can continue from block 715 to block 720. At block 720, the method 700 includes selecting a stereo encoding mode, as previously described with respect to Table 1.

[0133] When the individual encoding mode is selected at block 720, processing continues from block 720 to block 725. The example method 700 can include, at block 725, "encode left and right channels," encoding the left and right channels separately. For example, the stereo processing block 210 passes the left and right input signals to the encoding block 215.

[0134] When the mid / side encoding mode is selected at block 720, processing continues from block 720 to block 730. The example method 700 can include, at block 730, "convert to mid and side signals," converting (e.g., transforming) the left and right channels to mid and side signals (e.g., with the side conversion block 510). Processing can continue from block 730 to block 735. At block 735, "encode mid and side signals," the example method 700 can include encoding the mid and side signals.

[0135] When the extended mid / side encoding mode is selected at block 720, processing continues from block 720 to block 740. The example method 700 can include, at block 740, "adjust phase alignment based on phase difference," adjusting the phase alignment of the left and / or right channels based on the computed phase difference (e.g., with the phase alignment block 505). Processing can continue from block 740 to block 745.

[0136] At block 745, "convert to mid and side signals," the method 700 includes converting the phase-aligned left and right channels to mid and side signals (e.g., with the side conversion block 510). Processing can continue from block 745 to block 750.

[0137] At block 750, "generate residual signal using side prediction," the example method 700 can include generating a residual signal using side prediction coefficients (e.g., with the side prediction block 515). Processing can continue from block 750 to block 755.

[0138] At block 755, "encode mid and residual signals," the example method 700 can include encoding the mid and residual signals. In some cases, the phase difference and the side prediction coefficients are encoded with the mid and residual signals. Also, in some cases, only the phase alignment adjustment or the use of side prediction coefficients is performed.

[0139] Figure 8A A block diagram of an example decoder 120 is shown. The example decoder 120 reverses the encoding performed by the encoder 110. The example decoder 120 includes a bitstream read block 805, a decoding block 810, an inverse stereo processing block 815, and a filter combination block 820. The bitstream read block 805 receives a bitstream from the encoder 110. The bitstream read block 805 processes the bitstream and provides the processed bitstream to the decoding block 810.

[0140] The decode block 810 is configured to receive the processed bitstream from the bitstream reading block 805. The decode block 810 processes (e.g., decodes) the processed bitstream to substantially replicate the mid signal, the side signal, and the stereo metadata. The decode block 810 provides the replicated mid signal, the replicated side signal, and the replicated stereo metadata to the inverse stereo processing block 815.

[0141] The inverse stereo processing block 815 is configured to receive the replicated mid signal, the replicated side signal, and the replicated stereo metadata from the decode block 810. The inverse stereo processing block 815 is configured to process the replicated mid signal, the replicated side signal, and the replicated stereo metadata to generate a replicated complex filter bank domain left signal and a replicated complex filter bank domain right signal. For example, to invert the stereo processing, in the case where the residual signal was encoded, the inverse stereo processing block 815 first reconstructs the side signal using the side prediction information included in the stereo metadata. Then, the mid / side transform is inverted. Finally, the original phase relationship of the left and right signals is recovered based on the transmitted phase data. In some cases, the phase adjustment (or alignment) is applied to only one channel. In other cases, the phase adjustment (or alignment) is applied to both the left and right channels. Equation 23 provides an example of decoding the jointly encoded signals in matrix notation:

[0142]

[0143] The decoding of the second order frequency differential encoding of the transmitted phase signal is provided by the pseudo code 4 shown in Figure 8B The inverse stereo processing block 815 provides the replicated complex filter bank domain left signal and the replicated complex filter bank domain right signal to the filter bank synthesis block 820. The filter bank synthesis block 820 is configured to receive the replicated complex filter bank domain left signal and the replicated complex filter bank domain right signal. The filter bank synthesis block 820 converts the replicated complex filter bank domain left signal and the replicated complex filter bank domain right signal to a replicated left signal and a replicated right signal. The filter bank synthesis block 820 outputs the replicated original left signal and the replicated original right signal.

[0144] Figure 9 A block diagram illustrating various example methods 900 for decoding a stereo signal is shown, which can be performed by the decoder 120 of Figure 8A . The example methods 900 can be performed by a processor, which can be configured to perform the methods 900 via machine executable instructions. The methods 900 can be divided into various blocks or partitions, such as blocks 905, 910, 915, and 920. Figure 9The various process blocks shown in FIG. 9 provide examples of the various methods disclosed herein and it is understood that some blocks can be eliminated, added, combined, or modified, without departing from the spirit of the disclosure. In some examples of the method 900, the stereo signal can have been previously encoded using an extended side / mid mode. For some examples, the processing of the various blocks, which can be described as processes, methods, steps, blocks, operations, or functions, can begin at block 905.

[0145] At block 905, "receive encoded bitstream," the example method 900 can include receiving an encoded bitstream. For example, the decoder 120 receives the bitstream from the encoder 110. Processing can continue from block 905 to block 910. At block 910, "decode mid signal, residual signal, and stereo metadata from bitstream," the example method 900 can include decoding a copied mid signal, a copied residual signal, and copied stereo metadata from the bitstream. Processing can continue from block 910 to block 915.

[0146] At block 915, "convert mid signal and residual signal to left and right channels using stereo metadata," the example method 900 can include converting the copied mid signal and the copied residual signal to a copied left channel and a copied right channel using the copied stereo metadata. For example, a side signal is copied using side prediction information included in the copied stereo metadata. The copied mid signal and the copied side signal are converted to the copied left channel and the copied right channel. In the case where the left and right channels are misaligned, a phase adjustment is performed using a phase difference included in the stereo metadata. Processing can continue from block 915 to block 920.

[0147] At block 920, "pass left and right channels to filter bank analysis to reconstruct original signals," the example method 900 can include passing the copied left channel and the copied right channel to filter bank analysis to copy the original left channel signal and the original right channel signal.

[0148] Figure 10A graph of perceptual evaluation of audio quality (PEAQ) is shown for 12 audio items (provided along the x-axis). The audio items are audio events captured by a microphone for encoding by the encoder 110. An objective difference grade (ODG) value is provided for each audio item (along the y-axis). The graph shows the benefit of joint stereo encoding. For example, audio quality is improved at least in audio item 5 (which has a small inter-channel delay), as indicated by the ODG value for encoding item 5 with the extended mid / side mode being closer to zero than encoding item 5 with the mid / side mode. Audio item 9 (which is a sound image positioning speech item) shows a similar audio quality improvement, as indicated by the ODG for encoding item 9 with the extended mid / side mode being closer to zero than encoding item 9 with the mid / side mode. In the PEAQ evaluation, the codec was run at 256 kb / s

[0149] Figure 11A A schematic block diagram of an exemplary device architecture 1100 (e.g., apparatus 1100) that can be used to implement various aspects of the present disclosure is shown. The architecture 1100 includes, but is not limited to, the server and client devices, systems, and methods described with reference to Figures 1-10 The architecture 1100 includes a central processing unit (CPU) 1101 that is capable of executing various processes according to programs stored in, for example, a read only memory (ROM) 1102 or loaded from, for example, a storage unit 1108 to a random access memory (RAM) 1103. The CPU 1101 can be, for example, an electronic processor 1101. In the RAM 1103, data required when the CPU 1101 executes various processes is also stored as necessary. The CPU 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output interface 1105 is also connected to the bus 1104.

[0150] The following components are connected to the I / O interface 1105: an input unit 1106, which can include a keyboard, a mouse, and the like; an output unit 1107, which can include a display such as a liquid crystal display (LCD) and one or more speakers; a storage unit 1108, including a hard disk or other suitable storage device; and a communication unit 1109, including a network interface card (e.g., wired or wireless) such as a network card.

[0151] In some implementations, the input unit 1106 includes one or more microphones at different locations (depending on the host device) so that audio signals in various formats (e.g., mono, stereo, spatial, immersive, and other suitable formats) can be captured.

[0152] In some implementations, the output unit 1107 includes a system with various numbers of speakers. The output unit 1107 can present audio signals in various formats (e.g., mono, stereo, immersive, binaural, and other suitable formats), depending on the capabilities of the hose device.

[0153] In some embodiments, the communication unit 1109 is configured to communicate with other devices (e.g., via a network). The drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, optical disk, magneto-optical disk, flash memory drive, or other suitable removable medium, is installed in the drive 1110, so that a computer program read therefrom is installed into the storage unit 1108 as needed. Those skilled in the art will appreciate that, although the apparatus 1100 is described as including the above-mentioned components, some of these components can be added, removed, and / or replaced in actual applications, and all such modifications or changes fall within the scope of the present disclosure.

[0154] According to example embodiments of the present disclosure, the above-described processes can be implemented as a computer software program or on a computer-readable storage medium. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program including program code for executing methods. In such embodiments, the computer program can be downloaded and installed from a network via the communication unit 1109, and / or installed from the removable medium 1111, as shown. Figure 11A

[0155] Figure 11B A schematic block diagram of an exemplary CPU 1101 implemented in the device architecture 1100 of Figure 11A The exemplary CPU 1101 can be used to implement various aspects of the present disclosure. The CPU 1101 includes an electronic processor 1120 and a memory 1121. The electronic processor 1120 is electrically and / or communicatively connected to the memory 1121 for bidirectional communication. The memory 1121 stores encoding software 1122 and / or decoding software 1123. In some examples, the memory 1121 can be internal to the electronic processor 1120, such as for internal cache memory or some other internal located ROM, RAM, or flash memory. In other examples, the memory 1121 can be external to the electronic processor 1120, such as in the ROM 1102, RAM 1103, flash memory, or removable medium 1111, or another non-transitory computer readable medium contemplated for the device architecture 1100. In some cases, the electronic processor 1120 can implement the encoding software 1122 stored in the memory 1121 to perform Figures 7A-7B ​any of the methods 700, etc. Further, the electronic processor 1120 can implement decoding software 1123 stored in the memory 1121 to perform any of the methods 900, etc. Figure 9 any of the methods 900, etc.

[0156] Generally, the various example embodiments of the present disclosure can be implemented in hardware or special-purpose circuits (e.g., control circuits), software, logic or any combination thereof. For example, the units discussed above can be implemented by control circuits (e.g., the CPU 1101 in combination with the memory 1121 and other components of the control circuitry) executing software, and as such, the control circuits can perform the actions described in the present disclosure. Some aspects can be implemented in hardware, while other aspects can be implemented in software or firmware that is executed by control, microprocessors or other computing devices (e.g., control circuits). While the various aspects of the example embodiments of the present disclosure are illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein can be implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controllers, or other computing devices, or some combination thereof. Figure 11A

[0157] Further, the various blocks shown in the flowcharts can be viewed as method steps, and / or as operations that result from operation of computer program code, and / or as a plurality of coupled logic circuit elements constructed to carry out the associated function(s). For example, the embodiments of the present disclosure include a computer program product that comprises a computer program tangibly embodied on a machine-readable medium, the computer program including program code configured to perform the methods described above.

[0158] In the context of the present disclosure, a machine-readable medium can be any tangible medium that contains or stores programming for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can be non-transitory, and can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0159] ​Computer program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages. The computer program code can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor of the computer or other programmable data processing apparatus, enables the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can execute entirely on a computer, partly on a computer as a stand-alone software package, partly on a computer and partly on a remote computer or entirely on a remote computer or server, or across one or more remote computers and / or servers.

[0160] Those skilled in the art will realize that the present application is in no way limited to the embodiments described above. On the contrary, many modifications and variations are possible and are considered to be within the scope of the appended claims. Various aspects and implementations of the present disclosure can also be understood from the following enumerated example embodiments (EEEs), which are not claims and can represent systems, methods and apparatuses, all arranged in accordance with aspects of the present disclosure.

[0161] EEE1. A method for encoding a stereo audio signal into a bitstream, the method comprising: passing a left channel and a right channel of the stereo audio signal to a complex- valued filter bank analysis block to generate one or more frequency bands; for each of the one or more frequency bands, computing an energy of the left channel, an energy of the right channel, and a covariance of the left channel and the right channel; based on the computed energy of the left channel, the computed energy of the right channel, and the computed covariance of the left channel and the right channel, selecting a stereo encoding mode for encoding the left channel and the right channel; and in the case that the stereo encoding mode is an extended mid / side encoding mode: computing a phase difference between the left channel and the right channel; adjusting a phase alignment between the left channel and the right channel based on the computed phase difference to generate an aligned left channel and an aligned right channel; transforming the aligned left channel and the aligned right channel into a mid signal and a side signal; generating a residual signal based on side prediction data and the side signal; encoding the mid signal, the residual signal, the phase difference, and the side prediction data into the bitstream; and outputting the bitstream for the selected stereo encoding mode.

[0162] EEE2. The method of EEE1, wherein the method comprises: in the case that the stereo encoding mode is a separate encoding mode: encoding the left channel and the right channel into the bitstream.

[0163] EEE3. The method of any of EEE1-EEE2, wherein the method comprises, in the case that the stereo coding mode is mid / side coding mode: transforming the aligned left channel and the aligned right channel into a mid signal and a side signal; and encoding the mid signal and the side signal into the bitstream.

[0164] EEE4. The method of any of EEE1-EEE3, wherein the mid signal in the extended mid / side coding mode represents a sum of the left channel and the right channel, and wherein the side signal in the extended mid / side coding mode represents a difference between the left channel and the right channel.

[0165] EEE5. The method of any of EEE1-EEE4, wherein computing the phase difference comprises: comparing the computed covariance of the left channel and the right channel to an energy threshold, and setting the computed phase difference to zero in the case that the computed covariance of the left channel and the right channel is less than or equal to the energy threshold.

[0166] EEE6. The method of any of EEE1-EEE5, wherein selecting the stereo coding mode comprises: determining a bit cost associated with each of the plurality of stereo coding modes based on the computed energy of the left channel, the computed energy of the right channel, the computed covariance of the left channel and the right channel, and a cost of transmitting the stereo audio signal; and selecting the stereo coding mode based on the bit costs.

[0167] EEE7. The method of EEE6, wherein determining the bit cost associated with each of the plurality of stereo coding modes comprises: determining an energy ratio of signals included in each of the plurality of stereo coding modes; and comparing the energy ratio to a threshold, wherein the bit cost indicates a bit reduction between each of the plurality of stereo coding modes compared to encoding the left channel and the right channel, and wherein the threshold is based on the computed energy of the left channel and the computed energy of the right channel.

[0168] EEE8. The method of any of EEE1-EEE7, further comprising signaling the selected stereo coding mode using two bits per frame.

[0169] EEE9. The method of EEE8, further comprising signaling a presence of the phase difference or a presence of the side prediction data using one bit per frame.

[0170] EEE10. The method of any of EEE1-EEE9, wherein the phase difference and the side prediction data are quantized.

[0171] EEE 11. An apparatus for encoding a stereo audio signal into a bitstream, the apparatus comprising: an electronic processor configured to: pass a left channel and a right channel of a stereo audio signal to a complex-valued filterbank analysis block to generate one or more frequency bands; for each of the one or more frequency bands, compute an energy of the left channel, an energy of the right channel, and a covariance of the left channel and the right channel; select a stereo encoding mode for encoding the left channel and the right channel based on the computed energy of the left channel, the computed energy of the right channel, and the computed covariance of the left channel and the right channel; and in a case where the stereo encoding mode is an mid / side encoding mode: compute a phase difference between the left channel and the right channel; adjust a phase alignment between the left channel and the right channel based on the computed phase difference to generate an aligned left channel and an aligned right channel; transform the aligned left channel and the aligned right channel into a mid signal and a side signal; generate a residual signal based on side prediction data and the side signal; encode the mid signal, the residual signal, the phase difference, and the side prediction data into a bitstream; and output the bitstream for the selected stereo encoding mode.

[0172] EEE 12. The apparatus of EEE 11, wherein the electronic processor is configured to: in a case where the stereo encoding mode is a separate encoding mode: encode the left channel and the right channel in the bitstream.

[0173] EEE 13. The apparatus of any of EEEs 11-12, wherein the electronic processor is configured to: in a case where the stereo encoding mode is a mid / side encoding mode: transform the aligned left channel and the aligned right channel into a mid signal and a side signal; and encode the mid signal and the side signal into the bitstream.

[0174] EEE 14. The apparatus of any of EEEs 11-13, wherein adjusting the phase alignment between the left channel and the right channel comprises adjusting a phase of the right channel to align the left channel and the right channel.

[0175] EEE 15. The apparatus of any of EEEs 11-14, wherein, to compute the phase difference, the electronic processor is configured to: compare the computed covariance of the left channel and the right channel to an energy threshold, and in a case where the computed covariance of the left channel and the right channel is less than or equal to the energy threshold, set the computed phase difference to zero.

[0176] EEE16. The apparatus of any of EEEs 11-15, wherein to select the stereo coding mode, the electronic processor is configured to: determine a bit cost associated with each of the plurality of stereo coding modes based on the computed energy of the left channel, the computed energy of the right channel, the computed covariance of the left and right channels, and a cost of transmitting the stereo audio signal; and select the stereo coding mode based on the bit cost.

[0177] EEE17. The apparatus of EEE 16, wherein to determine the bit cost associated with each of the plurality of stereo coding modes, the electronic processor is configured to: determine an energy ratio of signals included in each of the plurality of stereo coding modes; and compare the energy ratio to a threshold, wherein the bit cost indicates a bit reduction between each of the plurality of stereo coding modes compared to encoding the left and right channels, and wherein the threshold is based on the computed energy of the left channel and the computed energy of the right channel.

[0178] EEE18. The apparatus of any of EEEs 11-17, wherein the electronic processor is configured to signal the selected stereo coding mode using two bits per frame.

[0179] EEE19. The apparatus of EEE 18, wherein the electronic processor is configured to signal a presence of a phase difference or a presence of side prediction data using one bit per frame.

[0180] EEE20. The apparatus of any of EEEs 11-19, wherein the phase difference and the side prediction data are quantized.

[0181] EEE21. The apparatus of any of EEEs 11-20, wherein the phase difference is a linearly quantized inter-channel phase difference, and wherein to encode the phase difference, the electronic processor is configured to: compute a second order difference of the linearly quantized inter-channel phase difference for each frequency band, wrap the second order difference to be within a range of 2p, and encode the wrapped second order difference.

[0182] EEE22. The method of any of EEEs 1-10, wherein the bitstream corresponds to an IVAS bitstream.

[0183] EEE23. The apparatus of any of EEEs 11-21, wherein the bitstream corresponds to an IVAS bitstream.

[0184] EEE24. A method for decoding a stereo audio signal, the method comprising: receiving an encoded bitstream; decoding a copied mid signal, a copied residual signal, and copied stereo metadata from the bitstream, wherein the copied stereo metadata comprises a phase difference and side prediction data; converting the copied mid signal and the copied residual signal into a copied left channel and a copied right channel using the copied stereo metadata; and passing the copied left channel and the copied right channel to a filter bank analysis block to reconstruct an original left channel and an original right channel.

[0185] EEE25. The method of EEE 24, wherein converting the copied mid signal and the copied residual signal comprises generating a side signal from the copied residual signal based on the side prediction data.

[0186] EEE26. The method of any of EEEs 24 to 25, further comprising aligning the copied left channel and the copied right channel using the phase difference.

[0187] EEE27. An apparatus for decoding a stereo audio signal, the apparatus comprising: an electronic processor configured to: receive an encoded bitstream; decode a copied mid signal, a copied residual signal, and copied stereo metadata from the bitstream, wherein the copied stereo metadata comprises a phase difference and side prediction data; convert the copied mid signal and the copied residual signal into a copied left channel and a copied right channel using the copied stereo metadata; and pass the copied left channel and the copied right channel to a filter bank analysis block to reconstruct an original left channel and an original right channel.

[0188] EEE28. The apparatus of EEE 27, wherein to convert the copied mid signal and the copied residual signal, the electronic processor is configured to generate a side signal from the copied residual signal based on the side prediction data.

[0189] EEE29. The apparatus of any of EEEs 27 to 28, wherein the electronic processor is configured to align the copied left channel and the copied right channel using the phase difference.

[0190] EEE30. A method for encoding a stereo audio signal, comprising: determining an advanced stereo coding mode for encoding the stereo audio signal, wherein the advanced stereo coding mode is selected from a group consisting of a left / right coding mode, a center / side coding mode, and an extended center / side coding mode; signaling the advanced stereo coding mode using two bits per frame; signaling the phase difference data using one bit per frame in response to the advanced stereo coding mode being an extended center / side coding mode; and signaling the prediction data using one bit per frame in response to the advanced stereo coding mode being an extended center / side coding mode.

[0191] EEE31. A non-transitory computer-readable storage medium recording an instruction program that can be executed by a device to perform the method according to any one of EEE1 to EEE10, EEE22, EEE24 to EEE26 or EEE30.

[0192] Regarding the processes, systems, methods, heuristics, etc., described herein, it should be understood that although the steps of these processes are described as occurring according to a specific ordered sequence, these processes can be practiced with the steps described being performed in a different order than those described herein. It should also be understood that some steps can be performed simultaneously, other steps can be added, or some steps described herein can be substituted, modified, or omitted. In other words, the process descriptions herein are provided for the purpose of illustrating certain embodiments and should not be construed as limiting the claims in any way.

[0193] Therefore, it should be understood that the above description is illustrative and not restrictive. Many embodiments and applications beyond the examples provided will become apparent upon reading the above description. The scope should not be determined by reference to the above description, but rather by the full scope of the appended claims and their equivalents. It is contemplated and intended that future developments will occur in the art discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In conclusion, it should be understood that modifications and variations are possible with this application.

[0194] All terms used in the claims are intended to be given their broadest reasonable interpretation and their ordinary meaning as understood by one of ordinary skill in the art described herein, unless expressly indicated otherwise herein. In particular, the use of singular articles such as “a,” “the,” “the,” etc., should be understood to refer to one or more of the indicated elements, unless the claims expressly limit this to the contrary.

[0195] The abstract of the disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or the meaning of the claims. Furthermore, in the preceding detailed description, it can be seen that various features are described in conjunction with various embodiments. This method of disclosure is not to be interpreted as reflecting a necessity to disclose all features in each embodiment. Rather, each embodiment is described in its preferred form of disclosure. Therefore, the following claims are hereby incorporated into this detailed description of the embodiments as specific means for carrying out the subject matter of the claims.

Claims

1. A method (700) for encoding a stereo audio signal into a bitstream, the method comprising: passing (705) a left channel and a right channel of the stereo audio signal to a complex-valued filterbank analysis block (205) to responsively generate one or more frequency bands; for each of the one or more frequency bands, computing (710) an energy of the left channel, an energy of the right channel, and a covariance of the left channel and the right channel; based on the computed energy of the left channel, the computed energy of the right channel, and the computed covariance of the left channel and the right channel, selecting (720) a stereo encoding mode for encoding the left channel and the right channel; and in the case that the stereo encoding mode is an extended mid / side encoding mode: computing a phase difference between the left channel and the right channel; based on the computed phase difference, adjusting (740) a phase alignment between the left channel and the right channel to generate an aligned left channel and an aligned right channel; transforming (745) the aligned left channel and the aligned right channel into a mid signal and a side signal; based on side prediction data and the side signal, generating (750) a residual signal; encoding (755) the mid signal, the residual signal, and the side prediction data into the bitstream; and outputting the bitstream for the selected stereo encoding mode. the bitstream corresponds to an IVAS bitstream.

2. The method (700) of claim 1, wherein the method (700) comprises:

3. The method (700) according to claim 1 or 2, wherein in the case that the stereo encoding mode is a separate encoding mode: encoding (725) the left channel and the right channel into the bitstream. the method (700) comprises:

4. The method (700) according to any one of claims 1 to 3, wherein in the case that the stereo encoding mode is a mid / side encoding mode: transforming (735) the aligned left channel and the aligned right channel into a mid signal and a side signal; and encoding (740) the mid signal and the side signal into the bitstream. the mid signal in the extended mid / side encoding mode represents a sum of the left channel and the right channel, and wherein the side signal in the mid / side encoding mode represents a difference between the left channel and the right channel.

5. The method (700) according to any one of claims 1 to 4, wherein computing the phase difference comprises:

6. The method (700) according to any one of claims 1 to 5, wherein comparing the computed covariance of the left channel and the right channel to an energy threshold, and in the case that the computed covariance of the left channel and the right channel is less than or equal to the energy threshold, setting the computed phase difference to zero. selecting the stereo encoding mode comprises:

7. The method (700) according to any one of claims 1 to 6, wherein based on the computed left channel energy, the computed right channel energy, the computed covariance of the left channel and the right channel, and a cost of transmitting the stereo audio signal, determining (715) a bit cost associated with each of a plurality of stereo encoding modes; and selecting the stereo encoding mode based on the bit costs. determining (715) the bit cost associated with each of the plurality of stereo encoding modes comprises:

8. The method (700) according to claim 7, wherein determining an energy ratio of signals included in each of the plurality of stereo encoding modes; and comparing the energy ratio to a threshold, wherein the bit cost indicates a bit reduction between each of the plurality of stereo encoding modes compared to encoding the left channel and the right channel, and ​ ​ wherein the threshold is based on the computed left channel energy and the computed right channel energy.

9. The method (700) of any of claims 1-8, further comprising signaling the selected stereo coding mode using two bits per frame.

10. The method (700) of claim 9, further comprising signaling the presence of a phase difference or the presence of side prediction data using one bit per frame.

11. The method (700) of any of claims 1-10, wherein the phase difference and the side prediction data are quantized.

12. A non-transitory computer-readable storage medium having recorded thereon a program of instructions executable by a device to perform the method of any of claims 1-11.

13. An apparatus (1100) for encoding a stereo audio signal into a bitstream, the apparatus (1100) comprising: an electronic processor (1101) configured to: pass (705) left and right channels of the stereo audio signal to a complex- valued filterbank analysis block (205) to responsively generate one or more frequency bands; for each of the one or more frequency bands, compute (710) an energy of the left channel, an energy of the right channel, and a covariance of the left and right channels; based on the computed energy of the left channel, the computed energy of the right channel, and the computed covariance of the left and right channels, select (720) a stereo coding mode for encoding the left and right channels; and in the case that the stereo coding mode is an extended mid / side coding mode (750): compute a phase difference between the left and right channels; adjust (740) a phase alignment between the left and right channels based on the computed phase difference to generate an aligned left channel and an aligned right channel; transform (745) the aligned left and right channels into a mid signal and a side signal; generate (750) a residual signal based on side prediction data and the side signal; encode (755) the mid signal, the residual signal, and the side prediction data into the bitstream; and output the bitstream for the selected stereo coding mode.

14. The apparatus (1100) of claim 13, wherein the electronic processor (1101) is configured to: in the case that the stereo coding mode is a separate coding mode: encode (725) the left and right channels into the bitstream. the electronic processor (1101) is configured to:

15. The apparatus (1100) according to any one of claims 13 to 14, wherein, in the case that the stereo coding mode is a mid / side coding mode: transform (730) the aligned left and right channels into a mid signal and a side signal; and encode (735) the mid signal and the side signal into the bitstream. Adjusting the phase alignment between the left and right channels includes adjusting a phase of the right channel to align the left and right channels.

16. The apparatus (1100) according to any one of claims 13 to 15, wherein To compute the phase difference, the electronic processor (1101) is configured to:

17. The apparatus (1100) according to any one of claims 13 to 16, wherein, compare the computed covariance of the left and right channels to an energy threshold, and in the case that the computed covariance of the left and right channels is greater than the energy threshold: compute a phase difference between the left and right channels; and adjust (740) a phase alignment between the left and right channels based on the computed phase difference to generate an aligned left channel and an aligned right channel. in the case that the computed covariance of the left and right channels is not greater than the energy threshold: output the bitstream for the selected stereo coding mode. In case the computed covariance of the left and right channels is less than or equal to an energy threshold, the computed phase difference is set to zero.

18. The apparatus (1100) according to any one of claims 13 to 17, wherein, To select a stereo coding mode, the electronic processor (1101) is configured to: determine (715) a bit cost associated with each of a plurality of stereo coding modes based on the computed left channel energy, the computed right channel energy, the computed covariance of the left and right channels, and a cost of transmitting the stereo audio signal; and select a stereo coding mode based on the bit cost.

19. The apparatus (1100) of claim 18, wherein To determine (715) the bit cost associated with each of the plurality of stereo coding modes, the electronic processor (1101) is configured to: determine an energy ratio of signals included in each of the plurality of stereo coding modes; and compare the energy ratio to a threshold, wherein the bit cost indicates a bit reduction between each of the plurality of stereo coding modes compared to encoding the left and right channels, and wherein the threshold is based on the computed left channel energy and the computed right channel energy.

20. The apparatus (1100) according to any one of claims 13 to 19, wherein The electronic processor (1101) is configured to signal the selected stereo coding mode using two bits per frame.

21. The apparatus (1100) of claim 20, wherein, The electronic processor (1101) is configured to signal the presence of phase difference or side prediction data using one bit per frame.

Images