Joint Stereo Coding in the Complex-Valued Filterbank Domain

Abstract

A system and method for joint stereo coding. One method for encoding a stereo audio signal includes passing the left and right channels of the stereo audio signal through a complex-valued filter bank analysis to calculate left channel energy, right channel energy, and covariance of the left and right channels. The method includes selecting a stereo coding mode for encoding the left and right channels. The method includes calculating a phase difference between the left and right channels, adjusting phase alignment between the left and right channels based on the phase difference, converting the aligned left and right channels into a mid signal and a side signal, generating a residual signal based on side prediction data and the side signal, encoding the mid signal, the residual signal, the phase difference, and the side prediction data into a bitstream, and transmitting the bitstream.

Description

[Technical Field]

[0001] REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 559,764, filed February 29, 2024, and U.S. Provisional Application No. 63 / 491,840, filed March 23, 2023, the disclosures of which are incorporated herein by reference in their entireties.

[0002] Technical Field This application relates generally to audio and speech coding, and more particularly to mid-side stereo coding.

[0003] background Unless otherwise specifically stated herein, the statements in this section are not prior art to the claims in this application, and are not admitted to be prior art by virtue of their inclusion in this section.

[0004] Joint coding of the left (L) and right (R) channels of a stereo signal allows for more efficient coding than coding the L and R channels independently. One approach to joint stereo coding is so-called mid / side (M / S) coding. In mid / side coding, the mid (M) signal is formed as a combination of the L and R channel signals, and the side (S) signal can be formed as the difference between the L and R channel signals. For example, the M signal can be of the form M = (L + R) / 2, and the S signal can be of the form S = (L + R) / 2. In M / S coding, the M and S signals are coded instead of the L and R signals.

[0005] In some implementations, M / S stereo coding can be implemented in a time-frequency-variant manner. For example, a stereo encoder may apply L / R coding to encode some frequency bands of a stereo signal and use M / S coding to encode other frequency bands of the stereo signal (frequency-variant). Furthermore, some encoders may switch between the L / R and M / S coding methods over time (time-variant).

[0006] It is with respect to these and other considerations that the present disclosure is presented. Summary of the Invention [Problem to be solved by the invention]

[0007] Disclosure summary Techniques for processing audio signals are described. Various embodiments described herein provide systems, methods, and / or devices with extensions to mid-side stereo coding applied in a complex-valued filter domain. For example, embodiments provide coordinated phase alignment between left and right audio channels prior to mid-side transform, in combination with real-valued prediction of the side signal from the mid signal at the encoder. Additionally, embodiments described herein provide novel methods for phase alignment in the complex domain and transmission of complex audio data.

[0008] The inter-channel phase differences, which can be transmitted as metadata in the bitstream, are used to reconstruct the original inter-channel phase relationships at the decoder. Because the signal is processed in the complex domain, the phase matching operation can be applied without risk of information loss or degradation (which is generally not the case when encoding only real-valued data). Furthermore, different processing blocks (e.g., phase matching, mid-side transform, side signal prediction blocks) are enabled or disabled based on level-dependent psychoacoustic models and, in some cases, on the side rate cost of the parameters.

[0009] The embodiments described herein provide improved spatial noise shaping (by preventing spatial unmasking) and improved energy compaction compared to known mid-side coding, especially for signals with systematic inter-channel level differences, inter-channel phase shifts, or time delays. Such characteristics are common to binaural signals generated by filtering audio objects or channels using head-related transfer functions. The described mid-side coding extensions can be used to improve audio quality at a given bitrate or to reduce the bitrate at a constant quality level.

[0010] In some aspects, the embodiments described herein include an encoder that receives binaural channels, a left channel (e.g., a left input signal) and a right channel (e.g., a right input signal). A complex filter bank analysis is performed on the left and right channels, converting the left and right channels to a complex-valued filter bank domain. By converting the left and right channels to the complex-valued filter bank domain, the left and right channels are ready for rendering by a head tracking device (e.g., processing with head-related transfer functions (HRTFs)). The signals in the complex-valued filter bank domain may be, for example, one or more frequency bands.

[0011] In some aspects, a stereo analysis is performed on a left channel of the complex-valued filter bank domain and a right channel of the complex-valued filter bank domain. The stereo analysis may identify the energy of the left channel of the complex-valued filter bank domain and the energy of the right channel of the complex-valued filter bank domain. Furthermore, the stereo analysis may identify the energy of a potential mid signal and a potential side signal. The mid signal represents the sum of the left channel and the right channel. The side signal represents the difference of the left channel and the right channel. In some cases, the energy of a potential residual signal is determined.

[0012] In some aspects, the stereo analysis also generates stereo metadata based on the left and right channels. For example, the covariance of the complex-valued filter bank domain left channel and the complex-valued filter bank domain right channel may be calculated. The inter-channel phase difference of the left and right channels is determined based on the covariance. Furthermore, real-valued prediction coefficients (e.g., side prediction coefficients) are calculated based on the energies of the complex-valued filter bank domain left channel and the complex-valued filter bank domain right channel, as well as the energies of the potential mid signal and potential side signal.

[0013] In some embodiments, one stereo coding mode is selected from a plurality of stereo coding modes to signal and encode the left and right channels. For example, for each possible stereo coding mode, a bit cost for signaling the left and right channels in that stereo coding mode is determined. The bit cost is based on the energy of the left channel in a complex-valued filter bank domain, the right channel in a complex-valued filter bank domain, a possible mid signal, a possible side signal, and a possible residual signal. In some cases, an energy ratio of the signals involved in each stereo coding mode is determined. The stereo coding mode is selected based on the bit cost. In some cases, the stereo coding mode with the lowest bit cost is selected.

[0014] In some aspects, the left channel and the right channel are processed according to a selected stereo coding mode. As an example, when the stereo coding mode is a separate coding mode, the stereo processor directly transfers the left channel and the right channel (or the left channel in a complex-valued filter bank domain and the right channel in a complex-valued filter bank domain) without converting the signal into a mid signal and a side signal.

[0015] In another example, the selected stereo mode is the basic mid / side mode, where the left and right channels are converted into mid and side signals, and the stereo processor performs the signal routing of the mid and side signals.

[0016] In another example, the selected stereo mode is a mid / side mode with adjusted phase alignment. In this case, the left and right channels are phase aligned based on the inter-channel phase difference. After alignment, the left and right channels are converted to a mid signal and a side signal. A stereo processor performs signal routing of the mid signal and the side signal, and the phase difference is encoded along with the mid signal and the side signal.

[0017] In another example, the selected stereo mode is a mid / side mode with side prediction. In this case, the left and right channels are converted into a mid signal and a side signal. A residual signal is generated based on the side signal and the prediction coefficients. The stereo processor performs signal transfer of the mid signal and the residual signal, and the prediction coefficients are encoded together with the mid signal and the residual signal.

[0018] In another example, the selected stereo mode is a mid / side stereo mode with both adjusted phase alignment and side prediction. In this case, the left and right channels are phase aligned based on the inter-channel phase difference. After adjustment, the left and right channels are converted into a mid signal and a side signal. A residual signal is generated based on the side signal and the prediction coefficients. A stereo processor performs signal transfer of the mid signal and the residual signal, and the prediction coefficients and the phase difference are encoded along with the mid signal and the residual signal.

[0019] As an example, a method for encoding a stereo audio signal into a bitstream is provided. The method includes passing left and right channels of the stereo audio signal to a complex-valued filter bank analysis block to responsively generate one or more frequency bands, and calculating a left channel energy, a right channel energy, and a left and right channel covariance for each frequency band. The method also includes selecting a stereo coding mode for encoding the left and right channels based on the calculated left channel energy, the calculated right channel energy, and the calculated left and right channel covariance. If the stereo coding mode is an extended mid / side coding mode, the method also includes calculating a phase difference between the left and right channels, adjusting phase alignment between the left and right channels based on the calculated phase difference to generate a matched left channel and a matched right channel, converting the matched left and right channels into a mid signal and a side signal, generating a residual signal based on the side prediction data and the side signal, encoding the mid signal, the residual signal, the phase difference, and the side prediction data into a bitstream, and outputting the bitstream for the selected stereo coding mode.

[0020] As another example, an apparatus for encoding a stereo audio signal into a bitstream is provided, the apparatus including an electronic processor configured to pass left and right channels of the stereo audio signal to a complex-valued filter bank analysis block to responsively generate one or more frequency bands, calculate a left channel energy, a right channel energy, and a covariance of the left and right channels for each of the one or more frequency bands, and select a stereo coding mode for encoding the left and right channels based on the calculated left channel energy, the calculated right channel energy, and the calculated covariance of the left and right channels. If the stereo coding mode is an extended mid / side coding mode, the decoder is configured to calculate a phase difference between the left channel and the right channel, adjust phase matching between the left channel and the right channel based on the calculated phase difference, thereby generating a matched left channel and a matched right channel, convert the matched left channel and the matched right channel into a mid signal and a side signal, generate a residual signal based on the side prediction data and the side signal, encode the mid signal, the residual signal, the phase difference, and the side prediction data into a bitstream, and output the selected stereo coding mode bitstream.

[0021] As another example, a method for decoding a stereo audio signal is provided, the method including receiving an encoded bitstream, decoding from the bitstream a replicated mid signal, a replicated residual signal, and replicated stereo metadata including phase difference and side prediction data, converting the replicated mid signal and the replicated residual signal into a replicated left channel and a replicated right channel using the stereo metadata, and passing the replicated left channel and the replicated right channel to a filter bank analysis block, thereby regenerating the original left channel and the original right channel.

[0022] As another example, provided is an apparatus for decoding a stereo audio signal, the apparatus including an electronic processor configured to receive an encoded bitstream, decode from the bitstream a replicated mid signal, a replicated residual signal, and replicated stereo metadata including phase difference and side prediction data, transform the replicated mid signal and the replicated residual signal into a replicated left channel and a replicated right channel using the replicated stereo metadata, and pass the replicated left channel and the replicated right channel to a filter bank analysis block, thereby regenerating the original left channel and the original right channel.

[0023] Thus, various aspects of the present disclosure provide for processing of stereo audio signals, leading to improvements in at least the fields of audio encoding, audio decoding, virtual reality, and the like.

[0024] The embodiments described herein may be broadly described as technology, although the term "technology" may refer to systems, apparatus, methods, computer-readable instructions, modules, components, hardware logic, and / or operations as the context in which it is applied suggests.

[0025] Features and technical advantages in addition to those expressly described above will become apparent upon reading the following detailed description and viewing the accompanying drawings. This Summary is intended to introduce some aspects of the technology in a simplified form and is not intended to identify key or essential features of the claimed subject matter defined by the appended claims. [Brief explanation of the drawings]

[0026] These and other more detailed and specific features of the various embodiments are more fully disclosed in the following description, which refers to the accompanying drawings, in which:

[0027] FIG. 1 illustrates a block diagram of an exemplary audio encoding system in which various aspects of the present invention may be incorporated.

[0028] FIG. 2 shows a block diagram illustrating an exemplary encoder.

[0029] FIG. 3 shows a block diagram illustrating an exemplary stereo processing process.

[0030] FIG. 4A illustrates an exemplary joint stereo processing transform.

[0031] FIG. 4B shows a table of each joint stereo coding type and its corresponding bitstream syntax elements.

[0032] FIG. 5A shows a block diagram of an exemplary mode-based stereo processing unit, such as the mode-based stereo processing unit of FIG.

[0033] FIG. 5B shows a table of an exemplary bitstream syntax.

[0034] FIG. 6A illustrates a graph illustrating stereo metadata rate per audio frame in an extended mid / side coding mode, in accordance with various aspects of the present disclosure.

[0035] 6B-6D show examples of pseudocode.

[0036] 7A-7B show block diagrams of various example methods for encoding a stereo signal that may be performed by the encoder of FIG. 2 according to various aspects of the present disclosure.

[0037] FIG. 8A shows a block diagram of an exemplary decoder.

[0038] FIG. 8B shows an example of pseudocode.

[0039] FIG. 9 illustrates block diagrams of various example methods for decoding a stereo signal that may be performed by the decoder of FIG. 8A, according to various aspects of the disclosure.

[0040] Figure 10 shows a graph of the PEAQ ratings for the 12 audio items.

[0041] FIG. 11A shows a schematic block diagram illustrating an example device architecture that can be used to implement various aspects of the present disclosure.

[0042] FIG. 11B is a schematic block diagram illustrating an example CPU implemented in the device architecture of FIG. 11A that can be used to implement various aspects of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0043] Detailed Description In the following description, numerous details are set forth regarding audio device configurations, timing, operation, etc., in order to provide an understanding of one or more aspects of the present specification. Those skilled in the art will readily appreciate that these specific details are merely examples and are not intended to limit the scope of the present application.

[0044] As used herein, the term "comprises" and words of similar meaning shall be interpreted as open-ended terms meaning "including, but not limited to." The term "or" shall be interpreted as "and / or" unless the context clearly indicates otherwise. The term "based on" shall be interpreted as "based at least in part on." The term "one implementation" shall be interpreted as "at least one implementation." The term "another implementation" shall be interpreted as "at least one other implementation." The term "determined" shall be interpreted as obtaining, receiving, calculating, computing, estimating, predicting, or deriving. Furthermore, in the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0045] Listed below are various abbreviations that may appear in this disclosure and the accompanying claims and / or drawings. For the sake of brevity, other commonly used abbreviations and terms may not be included in this list. Thus, the following brief list of abbreviations is provided for the convenience of the reader. ERB: Equivalent Rectangular Bandwidth PEAQ: Perceptual Evaluation of Audio Quality ODG: Objective Difference Grade IVAS: Immersive Voice and Audio Services MDFT: Modified Discrete Fourier Transform MDCT: Modified Discrete Cosine Transform HRTF: Head Related Transfer Function

[0046] 1 shows a block diagram of an audio encoding system in which various aspects of the present invention may be incorporated. The exemplary audio encoding system 100 includes an encoder 110 and a decoder 120. The input of the encoder 110 corresponds to a first signal path 105, and the output of the encoder 110 corresponds to a second signal path 115. The input of the decoder 120 corresponds to the second signal path 115, and the output of the decoder 120 corresponds to a third signal path 125.

[0047] Encoder 110 is configured to receive one or more audio information streams representing one or more audio signal channels from first signal path 105. Encoder 110 is further configured to process the received audio information streams to generate an encoded signal that can be output to second signal path 115. In second signal path 115, the encoded signal can be stored (e.g., captured, buffered, and / or recorded) or transmitted (e.g., via a wired or wireless communication medium). Decoder 120 is configured to receive the encoded signal from second signal path 115. Decoder 120 is further configured to process the received encoded signal to generate a decoded signal that can be output to third signal path 125. The decoded signal generated by decoder 120 corresponds to a copy of the audio information previously received by encoder 110 from first signal path 105. In the third signal path, the decoded signal may be stored (e.g., captured and / or recorded), transmitted (e.g., via a wireless or wired electronic communication medium), or output to a listening device (e.g., an audio processing device such as a receiver, speaker, sound bar, etc.). The audio encoding system 100 may be an audio system capable of implementing an audio codec standard such as the Intense Voice Acoustic Services (IVAS) standard. In such a case, the encoded signal in the second signal path 115 may correspond to an IVAS bitstream.

[0048] In various examples described herein, the terms “replica” and “replica signal” do not imply that the audio information stream is “identical.” The term “replica” may indicate that the audio information stream is substantially similar to the original audio information. For example, if the encoder 110 generates the encoded signal using a lossless encoding technique, the decoder 120 can, in principle, recover a substantially lossless version of the original audio information from the stream. However, in examples where the encoder 110 uses a lossy encoding technique, such as perceptual coding, the content of the recovered replica signal, while not generally identical to the content of the original stream, may be perceptually indistinguishable from the original content. Thus, the terms “replica” and “replica signal” are intended to encompass both lossless and lossy encoding techniques as used herein.

[0049] FIG. 2 illustrates an example of the encoder 110. The encoder 110 includes a complex filter bank analysis block 205, a stereo processing block 210, an encoding block 215, and a bitstream writing block 220. The complex filter bank analysis block 205 is configured to receive left and right audio channels (e.g., from one or more microphones). The left and right audio channels may be binaural channels with small delays and correlations between the channels. The left and right audio channels may be processed by head-related related transfer functions (HRTFs). In some cases, the left and right audio channels are Ambisonic signals. The complex filter bank analysis block 205 is further configured to process the left and right audio channels to generate complex-valued filter bank domain left and complex-valued filter bank domain right signals. The complex filter bank analysis block 205 is configured to output the complex-valued filter bank domain left and complex-valued filter bank domain right signals to the stereo processing block 210. The complex-valued filter bank domain signals may include additional metadata for processing by a head tracking device. In some cases, the complex-valued filter bank domain signal is one or more frequency bands associated with a left channel and a right channel. In some examples, the complex filter bank analysis can be perfect reconstruction, such as a modified discrete Fourier transform (MDFT), a modified discrete cosine / sine transform (MDCT), or near-perfect reconstruction, such as a complex-modulated filter bank.

[0050] The stereo processing block 210 is configured to receive a complex-valued filterbank domain left signal and a complex-valued filterbank domain right signal from the complex filterbank analysis 205. The stereo processing block 210 is configured to perform stereo processing analysis by assembling the filterbank frequency bins associated with a frame into frequency bands according to an auditory frequency scale, such as the equivalent rectangular bandwidth (ERB) scale or the Bark scale. For each frequency band, the energy of the left and right channels and the covariance between the two channels are calculated. Furthermore, the energies of the mid and side signals are calculated for each frequency band. A correlation coefficient for each band is calculated from the covariance. If the correlation coefficient is greater than a threshold, the phase difference between the left and right channels is calculated and may be used to adjust the phase alignment. The covariance may be updated based on the quantized phase difference.

[0051] Additionally, the stereo processing block 210 may calculate real-valued prediction coefficients and use them to remove redundancy in the side signal relative to the mid signal. The adjusted phase matching may result in an increase in mid signal energy and a decrease in side signal energy. Furthermore, the adjusted phase matching ensures that the side signal energy is always lower than the mid signal energy, which may result in a situation where the left and right signal components cancel each other out in the mid signal at certain frequencies. The side signal residual energy may be calculated based on the quantized prediction coefficients. Further details regarding the operation of the stereo processing block 210 are provided below.

[0052] The calculated signal energy and the calculated stereo processing metadata are output from the stereo processing block 210 to an encoding block 215. The encoding block 215 is configured to receive the calculated signal energy and the calculated stereo processing metadata from the stereo processing block 210. The encoding block 215 is further configured to encode the calculated signal energy and the stereo processing metadata as an encoded signal. The encoded calculated signal energy and the encoded stereo processing metadata are output from the encoding block 215 to a bitstream writing block 220. The bitstream writing block 220 is configured to convert the encoded calculated signal energy and the encoded stereo processing metadata into a bitstream. The bitstream may be output by the bitstream writing block 220.

[0053] The stereo processing is based on the determined stereo coding mode. Some exemplary stereo coding modes or methods include at least an L / R mode, a mid / side mode, and an extended mid / side mode. The selected stereo coding mode for a particular frequency band may be determined based on an estimated or calculated inter-channel energy difference for the signal pair in each possible mode. For example, the coding mode determination may be based on an estimated number of bits that would be required to encode each signal pair for the corresponding coding mode. The estimated number of bits may be based on a level-dependent psychoacoustic model that allocates more bits to louder signal portions than quieter portions. Thus, the signal pair with the largest energy difference may require the fewest bits at a particular quality level. In some cases, the bits required to encode the stereo metadata may also be considered when determining the most efficient coding method. Furthermore, in some cases, the same level-dependent psychoacoustic model may also be used to select an encoding method for the audio data. For the mid / side mode, bit savings estimates for L / R coding may be calculated and summed to determine the total bit savings per frame. After subtracting each metadata bit, the most efficient (eg, lowest cost) stereo encoding mode in terms of required bits may be selected.

[0054] 3 illustrates an exemplary stereo processing process or method as performed by the stereo processing block 210. The exemplary stereo processing block 210 includes a stereo analysis unit 305, a psychoacoustic model 310, a bit-cost estimation unit 315, a mode decision unit 320, and a mode-based stereo processing unit 325. The stereo analysis unit 305 is configured to receive the complex-valued filter bank domain L / R signal from the complex filter bank analysis block 205. The stereo analysis unit 305 determines, by calculation or estimation, the energy of the complex-valued filter bank domain L / R signal, the energies of the mid signal and the side signal, and the residual energy. The stereo analysis unit 305 is further configured to provide these energies to the bit-cost estimation unit 315. The stereo analysis unit 305 also determines, by calculation or estimation, stereo metadata based on the complex-valued filter bank domain L / R signal. The stereo metadata may include, for example, one or more of a phase difference, a prediction coefficient, and / or a covariance of the complex-valued filter bank domain L / R signal, the mid signal, and the side signal.

[0055] The stereo metadata is provided from the stereo analysis unit 305 to the mode-based stereo processing unit 325 and the bit cost estimation unit 315. The bit cost estimation unit 315 estimates the number of bits required to encode the signal based on the determined energy provided from the stereo analysis unit 305 and based on the psychoacoustic model 310. For example, the bit cost estimation unit 315 estimates the number of bits required to encode the signal for each candidate (or possible) stereo coding mode. The bit cost estimation unit 315 is configured to send the estimated number of required bits to the mode decision unit 320. The mode decision unit 320 receives the estimated number of required bits and responsively determines a stereo coding mode (e.g., selects one from among the candidate stereo coding modes) based on the estimated number of required bits. The mode decision unit 320 further provides the selected (or determined) stereo coding mode to the mode-based stereo processing unit 325. The mode-based stereo processing unit 325 is configured to receive the selected stereo coding mode from the mode decision unit 320. The mode-based stereo processing unit 325 signals the processed L / R signal, for example, using two bits per frame (e.g., signal 0 and signal 1). The processed L / R signal may correspond to the L / R signal, the mid / side signal, or the mid / residual signal, which are described in more detail below.

[0056] 4A illustrates an exemplary joint stereo process 400 performed by the stereo analysis unit 305. By performing process 400, the stereo analysis unit 305 analyzes the complex-valued filter bank domain L / R signals received from the complex filter bank analysis block 205. For example, the stereo analysis unit 305 receives a complex-valued filter bank domain left signal L and a complex-valued filter bank domain right signal R. The left input signal L is given by Equation 1:

number

[0057] The right input signal R is given by Equation 2.

number

[0058] In some cases, the right input signal R is phase rotated before calculating the mid signal M and the side signal S. The phase-rotated right signal R′ is given by Equation 3.

number

[0059] Next, the mid signal M is determined as in Equation 4.

number

number

[0060] The residual signal S' is also determined as in Equation 6.

number

[0061] Stereo processing parameters (given as stereo metadata) are also generated by the stereo analysis unit 305. The stereo processing parameters are based on energy and covariance measurements for each perceptual frequency bank k within an audio frame f (omitted in the following calculations for clarity). Thus, an energy compaction estimate is calculated for the stereo coding type referenced by the mode decision unit 320.

[0062] The energy of the left input signal L is given by Equation 7.

number

[0063] The energy of the right input signal R is given by Equation 8.

number

[0064] The covariance for the left input signal L and the right input signal R is given by Equation 9.

number

[0065] The inter-channel phase difference that can be used to maximize the mid signal energy is given by Equation 10.

number

[0066] The real part of the covariance with the phase-rotated signal R′ by the quantized phase can be derived from the initial covariance using Equation 11:

number

number

[0067] The energy of the mid signal M is calculated as in Equation 13.

number

[0068] The energy of the side signal S is calculated as in Equation 14.

number

[0069] The prediction coefficient p that minimizes the energy of the residual signal S' is calculated as shown in Equation 15.

number

[0070] The residual signal energy is calculated as in Equation 16.

number

[0071] Referring to the process performed by the mode decision unit 320 in FIG. 3, four types of joint stereo coding (e.g., advanced stereo coding modes) are considered for each frequency bank k. Table 1 shown in FIG. 4B lists each joint stereo coding type and its associated bitstream syntax element. In Table 1, type "0" corresponds to coding that separately codes the L / R signals and is not a joint stereo coding type. The joint coding type is associated with a corresponding stereo mode MSMmode. The associated bitstream element is provided to the mode-based stereo processing unit 325.

[0072] Stereo coding type t=0 corresponds to coding in which the L / R signals are coded separately. Stereo coding type t=1 corresponds to standard mid / side coding. Stereo coding type t=2 corresponds to extended mid / side coding with adjusted phase matching only. Stereo coding type t=3 corresponds to extended mid / side coding with side prediction only. Stereo coding type t=4 corresponds to extended mid / side coding with adjusted phase matching and side prediction. All stereo coding types t=1, 2, 3, and 4 correspond to joint coding types.

[0073] To determine the joint stereo mode, the bit cost estimator unit 315 determines an energy ratio metric for each potential joint coding type (e.g., t=1, 2, 3, or 4) and separate coding type (e.g., t=0), as defined in Equations 17 and 18 below.

number

number

[0074] If the proportion of a potential joint coding type t (=1, 2, 3, 4) is greater than the proportion of the separate coding type t=0, then that joint coding type is marked (e.g., a flag is set) for each frequency bank k, as shown in Equation 19.

number

[0075] The reduction in the number of required bits due to the energy compression achieved by joint coding is evaluated as shown in Equation 20.

number

[0076] The total bit reduction required for a particular joint coding type is calculated as a function of the signaling cost based on the bitstream syntax, as described below with respect to Table 2, as in Equation 21. It is calculated by summing the relevant reductions across all bands, taking into account JPEG2026508703000025.jpg78.

number

number

[0077] The mode-based stereo processing unit 325 is configured to receive stereo metadata (including quantized phase differences and quantized prediction coefficients) from the stereo analysis unit 305 and the selected joint stereo coding type t from the mode decision unit 320. Processing of the left input signal L and the right input signal R is performed by the mode-based stereo processing unit 325 based on the selected joint stereo coding scheme t (e.g., the selected stereo mode).

[0078] If separate coding type t=0 is selected, the mode-based stereo processing unit 325 may simply pass the left input signal L and the right input signal R to the encoding block 215 without modifying or processing the signals. Otherwise, if coding type t=1, 2, 3, or 4 is selected, which is mid / side mode, the mode-based stereo processing unit 325 processes the left input signal L and the right input signal R according to the selected joint stereo coding type t and stereo metadata.

[0079] FIG. 5A shows a block diagram of an example mode-based stereo processing unit, such as the mode-based stereo processing unit 325 of FIG. 3. The example shown may be used when mid / side mode is selected. The mode-based stereo processing unit 325 of this example includes a phase matching block 505, a mid / side transform block 510, and a side prediction block 515. The phase matching block 505 is configured to receive a left input signal L, a right input signal R, and a quantized phase difference. The phase matching block 505 is only required for stereo coding types t=2 and t=4. The phase matching block 505 calculates the quantized phase difference φ k,Q , is used to perform a phase matching adjustment between the left input signal L and the right input signal R. In some examples, the phase matching adjustment may be applied to either the L signal or the R signal, while in other examples, the phase matching adjustment may be applied to both the L signal or the R signal. The phase-matched left input signal L and the phase-matched right input signal R are output by the phase matching block 505 to the side transform block 510. For other joint stereo coding types (e.g., t=1 and t=3), the phase matching block 505 may be disabled or bypassed (e.g., φ k,Q can be set to 0).

[0080] The mid / side transform block 510 is configured to receive a left input signal L and a right input signal R from the phase alignment block 505, which may be phase aligned as described above. The mid / side transform block 510 converts the left input signal L and the right input signal R into a mid signal M and a side signal S (and possibly a residual signal S'), as described above in FIG. 4A. The mid signal M and the side signal S are output from the side transform block 510 to a side prediction block 515.

[0081] The side prediction block 515 may be implemented for stereo coding types t=3 and t=4. The side prediction block 515 receives the mid signal M and the side signal S from the side transform block 510 and the quantized prediction coefficients p k,QThe side prediction block 515 applies side prediction to the mid signal M and the side signal S. In other joint stereo coding types (e.g., t=1 and t=2), the side prediction block 515 is disabled or bypassed (e.g., p k,Q is set to 0).

[0082] The stereo mode may be signaled, for example, by two bits per frame from the mode-based stereo processing unit 325. In this example, mid / side coding versus left / right coding may be signaled by one bit per frequency band. Alternatively, mid / side coding valid for all frequency bands may be indicated by one bit per frame. In extended mid / side mode, the presence of phase difference data may be signaled by one bit per frame, and side prediction data may be signaled by one bit per frame. In this manner, metadata size may be reduced for different types of stereo signals. If side prediction data is present, it may be Huffman entropy coded for each difference between the associated frequency bands. If phase difference data is present, it may be entropy coded. Table 2, shown in FIG. 5B, illustrates an exemplary bitstream syntax.

[0083] During encoding (in encoding block 215), the inter-channel phase difference data is linearly quantized. The scale factor used for quantization is selected so that the values ​​of π and −π are accurately represented, as shown in Equation 12. In some implementations, the phase symbols φ along the associated frequency bands are k,QSecond-order differences (e.g., quantized inter-channel phase differences per band) are calculated and may be wrapped to the range [-π, +π] to avoid jumps larger than π. The second-order differences may be calculated by first calculating the difference between the phase symbols for bands 1 through N and the phase symbols for bands 0 through N-1, where N is the number of bands for which M / S processing with phase matching is enabled (see, for example, Table 2 in FIG. 5B and the variable "numMSBands" in Pseudo Code 1 in FIG. 6B). In a second step, as shown in Pseudo Code 2 in FIG. 6C, the second-order differences are calculated as the difference between the new values ​​for bands 2 through N and the new values ​​for bands 1 through N. In this way, the quantized values ​​in the first frequency band are unchanged, the quantized values ​​in the second frequency band represent the difference, and the quantized values ​​in the remaining frequency bands may correspond to differences from the first difference. The wrapped phase symbols may be entropy coded (where symbol zero may be encoded using the minimum number of bits). The signal model has a time delay or a constant phase shift between channels. In some cases, with linear frequency banding, the second-order frequency differences are substantially equal to zero, and therefore require fewer bits for encoding. In other cases, the second-order frequency differences are not substantially zero. Furthermore, in some cases, encoding is performed using only first-order differences. In any case, the above-described entropy coding scheme for inter-channel phase differences using second-order frequency differential coding is more efficient (in the sense of using fewer bits for the same information) than simple frequency differential or non-differential coding for stereo-coded content, such as binaural stereo.

[0084] Figure 6A shows the stereo metadata rate for each audio frame in the extended mid / side coding mode across multiple frames when encoded using first-order or second-order differentials for an item with an inter-channel time delay of 5 samples (48 kHz). In Figure 6A, the x-axis shows multiple frames, and the y-axis shows the bit rate (kB / s) of encoding each frame. As shown in Figure 6A, When encoding the quantized phase angles of the jointly coded bands shown in JPEG2026508703000028.jpg714, the quantized phase angles can first be aligned adjacently. For an example of this process, see Pseudocode 1 in Figure 6B. Next, first and second order differences are calculated for the aligned quantized phase angles. Finally, the data is wrapped to a quantized 2π range before Huffman encoding. Pseudocode 2 in Figure 6C shows an example of the calculation of first and second order differences, written in a C-like format. Pseudocode 3 in Figure 6D shows an example of wrapping the phase symbol (variable phaseQ) to a 2π range.

[0085] The phase symbols represent absolute phases (first frequency band), first-order differences (second frequency band), or second-order differences across frequency bands (other frequency bands). If the phase data is encoded using first-order differences, the same phase wrapping as described above can be applied. In the pseudocode 3 shown in FIG. 6D, the value of PHASE_MAX_VAL represents a value of +π and the value of PHASE_MIN_VAL represents a value of −π (where the phase symbols are represented by integers in the range [PHASE_MIN_VAL, PHASE_MAX_VAL], where PHASE_MIN_VAL is defined as negative PHASE_MAX_VAL, and PHASE_MAX_VAL is the integer x (e.g., x=12) used for quantization in Equation 12).

[0086] 7A-7B illustrate block diagrams of various exemplary methods 700 for encoding a stereo signal that may be performed by the encoder 110 of FIG. 2. The method 700 may be performed by a processor that may be configured to execute the method 700 via machine-executable instructions. The method 700 may be divided into various blocks or partitions, such as blocks 705, 710, 715, 720, 725, 730, 735, 740, 745, 750, 755, 760, 765, and 770. The various process blocks illustrated in FIG. 7 are illustrative of examples of various methods disclosed herein, and it will be understood that some blocks may be removed, added, combined, or modified without departing from the spirit of the present disclosure. In some examples, processing of the various blocks, which may be described as processes, methods, steps, blocks, operations, or functions, may begin at block 705.

[0087] At block 705, "Pass Left and Right Channels of Stereo Audio Signal to Filter Bank Analysis," the example method 700 may include passing the left and right channels of the stereo audio signal to a filter bank analysis. The filter bank analysis responsively generates one or more frequency bands. Processing proceeds from block 705 to block 710.

[0088] In block 710, "Calculate Left Channel Energy, Right Channel Energy, and Covariance," example method 700 may include calculating the left channel energy, calculating the right channel energy, and calculating the covariance of the left and right channels, as described above with respect to Equations 7 through 16. Processing proceeds from block 710 to block 715.

[0089] At block 715 "Determine bit costs of multiple stereo coding modes," example method 700 may include determining bit costs of multiple stereo coding modes, as described above with respect to Equations 17-22 and Table 1. Processing proceeds from block 715 to block 720. At block 720, method 700 includes selecting a stereo coding mode, as described above with respect to Table 1.

[0090] If separate encoding mode is selected at block 720, processing proceeds from block 720 to block 725. The example method 700 may include encoding the left and right channels separately at block 725, "Encode Left and Right Channels." For example, the stereo processing block 210 passes the left and right input signals to the encoding block 215.

[0091] If a mid / side coding mode is selected at block 720, processing proceeds from block 720 to block 730. The example method 700 may include converting (e.g., transforming) the left and right channels to a mid and side signal (e.g., at side transform block 510) at block 730, "Convert to mid and side signals." Processing proceeds from block 730 to block 735. At block 735, "Encode mid and side signals," the example method 700 may include encoding the mid and side signals.

[0092] If the extended mid / side coding mode is selected at block 720, processing may proceed from block 720 to block 740. The example method 700 may include, at block 740 "Adjust Phase Matching Based on Phase Difference," adjusting phase matching for the left channel and / or the right channel based on the calculated phase difference (e.g., at phase matching block 505). Processing may proceed from block 740 to block 745.

[0093] At block 745, "Convert to Mid and Side Signals," the method 700 includes converting the phase-aligned left and right channels to mid and side signals (e.g., in side conversion block 510). Processing may proceed from block 745 to block 750.

[0094] At block 750, "Generate a residual signal using side prediction," the example method 700 may include generating a residual signal using the side prediction coefficients (e.g., in side prediction block 515). Processing may proceed from block 750 to block 755.

[0095] At block 755 "Encode mid signal and residual signal," the example method 700 may include encoding the mid signal and residual signal. In some cases, phase differences and side prediction coefficients are encoded along with the mid signal and residual signal. Furthermore, in some cases, only phase matching adjustment or only side prediction coefficient use is performed.

[0096] 8A shows a block diagram of an exemplary decoder 120. The exemplary decoder 120 reverses the encoding performed by the encoder 110. The exemplary decoder 120 includes a bitstream read block 805, a decode block 810, an inverse stereo processing block 815, and a filter bank synthesis block 820. The bitstream read block 805 receives a bitstream from the encoder 110. The bitstream read block 805 processes the bitstream and provides the processed bitstream to the decode block 810.

[0097] The decode block 810 is configured to receive the processed bitstream from the bitstream read block 805. The decode block 810 processes (e.g., decodes) the processed bitstream to substantially replicate the mid signal, the side signal, and the stereo metadata. The replicated mid signal, the replicated side signal, and the replicated stereo metadata are provided from the decode block 810 to an inverse stereo processing block 815.

[0098] The inverse stereo processing block 815 is configured to receive the replicated mid signal, the replicated side signal, and the replicated stereo metadata from the decoding block 810. The inverse stereo processing block 815 processes the replicated mid signal, the replicated side signal, and the replicated stereo metadata to generate a replicated complex-valued filter bank domain left signal and a replicated complex-valued filter bank domain right signal. For example, to reverse the stereo processing, if the residual signal is encoded, the inverse stereo processing block 815 first reconstructs the side signal using the side prediction information included in the stereo metadata. Then, it performs an inverse transform of the mid / side transform. Finally, the original phase relationship between the left and right signals is restored based on the transmitted phase data. In some cases, phase adjustment (or phase alignment) is applied to only one channel. In other cases, phase adjustment (or phase alignment) is applied to both the left and right channels. Equation 23 shows an example of decoding a jointly encoded signal in matrix notation.

number

[0099] Decoding of the second-order frequency differential encoding of the transmitted phase signal is performed by pseudocode 4 shown in Figure 8B. The inverse stereo processing block 815 provides a replicated complex-valued filter bank domain left signal and a replicated complex-valued filter bank domain right signal to the filter bank synthesis block 820. The filter bank synthesis block 820 is configured to receive the replicated complex-valued filter bank domain left signal and the replicated complex-valued filter bank domain right signal. The filter bank synthesis block 820 converts the replicated complex-valued filter bank domain left signal and the replicated complex-valued filter bank domain right signal into a replicated left signal and a replicated right signal. The filter bank synthesis block 820 outputs the replicated original left signal and the replicated original right signal.

[0100] FIG. 9 shows a block diagram of various example methods 900 for decoding a stereo signal, which may be performed by the decoder 120 of FIG. 8A. The example method 900 may be performed by a processor, which may be configured to execute the method 900 via machine-executable instructions. The method 900 may be divided into various blocks or partitions, such as blocks 905, 910, 915, and 920. The various processing blocks shown in FIG. 9 provide examples of the various methods disclosed herein, and it should be understood that some blocks may be deleted, added, combined, or modified without departing from the spirit of the present disclosure. In some examples of the method 900, the stereo signal may have been pre-encoded using an extended side / mid mode. In some examples, processing of the various blocks, which may be described as processes, methods, steps, blocks, operations, or functions, may begin at block 905.

[0101] At block 905, "Receive Encoded Bitstream," the example method 900 may include receiving an encoded bitstream. For example, the decoder 120 receives the bitstream from the encoder 110. Processing may proceed from block 905 to block 910. At block 910, "Decode Mid Signal, Residual Signal, and Stereo Metadata from Bitstream," the example method 900 may include decoding a replicated mid signal, a replicated residual signal, and a replicated stereo metadata from the bitstream. Processing may proceed from block 910 to block 915.

[0102] At block 915, "Convert mid signal and residual signal to left and right channel using stereo metadata," the example method 900 may include converting the replicated mid signal and replicated residual signal to a replicated left channel and a replicated right channel using the replicated stereo metadata. For example, the side signal is replicated using side prediction information included in the replicated stereo metadata. The replicated mid signal and replicated side signal are converted to a replicated left channel and a replicated right channel. If the left channel and right channel are out of phase, a phase adjustment is performed using the phase difference included in the stereo metadata. Processing may proceed from block 915 to block 920.

[0103] At block 920, "Passing the Left and Right Channels through Filter Bank Analysis to Reproduce the Original Signal," the example method 900 may include passing the replicated left and right channels through filter bank analysis to replicate the original left and right channel signals.

[0104] FIG. 10 shows a graph of perceptual audio quality assessment (PEAQ) for 12 audio items (shown along the x-axis). An audio item is an audio event captured by a microphone for encoding by the encoder 110. For each audio item, an objective difference factor (ODF) value is shown (along the y-axis). This graph illustrates the benefits of joint stereo encoding. For example, audio quality is improved for at least audio item 5 (which has a small inter-channel time delay), as indicated by the ODG value being closer to zero when encoded in extended mid / side mode than when encoded in mid / side mode. A similar audio quality improvement is observed for audio item 9, a panned speech item, as indicated by the ODG value for item 9 encoded in extended mid / side mode being closer to zero than when encoded in mid / side mode. For the PEAQ evaluation, the codec was operated at 256 kb / s.

[0105] FIG. 11A illustrates a schematic block diagram of an exemplary device architecture 1100 (e.g., apparatus 1100) that may be used to implement various aspects of the present disclosure. The architecture 1100 includes, but is not limited to, the server and client devices, systems, and methods described in FIGS. 1-10. As illustrated, the architecture 1100 includes a central processing unit (CPU) 1101 that can execute various processes according to a program stored in, for example, a read-only memory (ROM) 1102 or loaded from, for example, a storage unit 1108 into a random access memory (RAM) 1103. The CPU 1101 may be, for example, an electronic processor 1101. The RAM 1103 also stores data required by the CPU 1101 to execute various processes, as needed. The CPU 1101, the ROM 1102, and the RAM 1103 are connected to one another via a bus 1104. An input / output interface 1105 is also connected to the bus 1104.

[0106] The following components are connected to the I / O interface 1105: an input unit 1106, which may include a keyboard, a mouse, etc.; an output unit 1107, which may include a display device such as a liquid crystal display (LCD) and one or more speakers; a storage unit 1108, which may include a hard disk or other suitable storage device; and a communication unit 1109, which may include a network interface card such as a network card (e.g., wired or wireless).

[0107] In some embodiments, the input unit 1106 includes one or more microphones positioned at different locations (depending on the host device) to enable capturing audio signals in various formats (e.g., mono, stereo, spatial, immersive, and other suitable formats).

[0108] In some embodiments, output unit 1107 includes a system with various numbers of speakers and may reproduce audio signals in various formats (e.g., mono, stereo, immersive, binaural, and other suitable formats) (depending on the capabilities of the host device).

[0109] In some embodiments, the communication unit 1109 is configured to communicate with other devices (e.g., via a network). A drive 1110 is also connected to the I / O interface 1105, as needed. A removable medium 1111, such as a magnetic disk, optical disk, magneto-optical disk, flash drive, or other suitable removable medium, is mounted on the drive 1110, whereby a computer program is read therefrom and installed in the storage device 1108, as needed. Although the device 1100 is described as including the above components, those skilled in the art will understand that in actual applications, some of these components may be added, removed, and / or substituted, and all such changes or modifications are within the scope of the present disclosure.

[0110] According to embodiments of the present disclosure, the processes described above may be implemented as a computer software program or on a computer-readable storage medium. For example, embodiments of the present disclosure include a computer program product including a computer program tangibly embodied in a machine-readable medium and including program code for performing various methods. In such embodiments, the computer program may be downloaded and mounted from a network via a communication unit 1109 and / or installed from a removable medium 1111, as shown in FIG. 11A.

[0111] FIG. 11B shows a schematic block diagram of an example CPU 1101 implemented in the device architecture 1100 of FIG. 11A that may be used to implement various aspects of the present disclosure. The CPU 1101 includes an electronic processor 1120 and a memory 1121. The electronic processor 1120 is electrically and / or communicatively coupled to the memory 1121 for bidirectional communication. The memory 1121 stores encoding software 1122 and / or decoding software 1123. In some examples, the memory 1121 may be located internal to the electronic processor 1120 (e.g., internal cache memory or other internally located ROM, RAM, or flash memory). In other examples, the memory 1121 may be located external to the electronic processor 1120 (e.g., ROM 1102, RAM 1103, flash memory, or removable media 1111 or other non-transitory computer-readable media contemplated for the device architecture 1100). In some cases, the electronic processor 1120 may implement encoding software 1122 stored in memory 1121 to, among other things, perform any of the methods 700 of Figures 7A-7B. Additionally, the electronic processor 1120 may implement decoding software 1123 stored in memory 1121 to, among other things, perform any of the methods 900 shown in Figure 9.

[0112] In general, various exemplary embodiments of the present disclosure may be implemented in hardware or dedicated circuitry (e.g., control circuitry), software, logic, or any combination thereof. For example, the above-described units may be executed by control circuitry (e.g., CPU 1101 in combination with other components of FIG. 11A ), such that the control circuitry performs the operations described in this disclosure. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device (e.g., control circuitry). While various aspects of exemplary embodiments of the present disclosure are described using block diagrams, flowcharts, or other graphical representations, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented by, by way of non-limiting example, hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers, or other computing devices, or combinations thereof.

[0113] Additionally, the various blocks illustrated in the flowcharts may be viewed as method steps and / or operations resulting from the operation of computer program code and / or as multiple coupled logic circuit elements configured to perform the associated functions. For example, embodiments of the present disclosure include computer program products including a computer program tangibly embodied in a machine-readable medium and including program code configured to perform the various methods described above.

[0114] In the context of this disclosure, a machine-readable medium refers to any tangible medium that can contain or store a program used by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may be non-transitory and may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples of machine-readable storage media include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), a rewritable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0115] Computer program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. The computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus with control circuitry such that, when executed by the processor of the computer or other programmable data processing apparatus, the program code implements the functions / operations specified in the flowcharts and / or block diagrams. The program code may be executed entirely on one computer, partially on one computer and operating as a stand-alone software package, partially on one computer and partially on a remote computer, entirely on a remote computer or server, or distributed across one or more remote computers and / or servers.

[0116] Those skilled in the art will appreciate that the present invention is not limited to the above-described embodiments. Rather, many modifications and variations are possible and are considered to be within the scope of the appended claims. Various aspects and embodiments of the present disclosure can also be understood from the following numbered enumerated example embodiments (EEE), which are not claims and may represent systems, methods, and apparatuses, all of which are constructed in accordance with aspects of the present disclosure.

[0117] EEE1: A method for encoding a stereo audio signal into a bitstream, comprising: passing a left channel and a right channel of the stereo audio signal to a complex-valued filter bank analysis block, thereby generating one or more frequency bands; calculating, for each of the one or more frequency bands, an energy of the left channel, an energy of the right channel, and a covariance of the left channel and the right channel; selecting a stereo coding mode for encoding the left channel and the right channel based on the calculated energy of the left channel, the calculated energy of the right channel, and the calculated covariance of the left channel and the right channel; If the stereo coding mode is an extended mid / side coding mode, the method includes: calculating a phase difference between the left channel and the right channel; adjusting phase matching between the left channel and the right channel based on the calculated phase difference, thereby generating a matched left channel and a matched right channel; converting the matched left channel and the matched right channel into a mid signal and a side signal; generating a residual signal based on side prediction data and the side signal; encoding the mid signal, the residual signal, the phase difference, and the side prediction data into a bitstream; and outputting the bitstream for the selected stereo coding mode.

[0118] EEE2: The method of EEE1, comprising, if the stereo coding mode is a separate coding mode, encoding the left channel and the right channel into the bitstream.

[0119] EEE3: The method according to any one of EEE1 to EEE2, comprising, if the stereo coding mode is a mid / side coding mode, converting the matched left channel and the matched right channel into the mid signal and the side signal, and encoding the mid signal and the side signal into the bitstream.

[0120] EEE4: The method described in any one of EEE1 to EEE3, wherein the mid signal in the extended mid / side coding mode represents the sum of the left channel and the right channel, and the side signal in the extended mid / side coding mode represents the difference between the left channel and the right channel.

[0121] EEE5: The method described in any of EEE1 to EEE4, wherein calculating the phase difference includes comparing the calculated covariance of the left channel and the right channel to an energy threshold, and setting the calculated phase difference to zero if the calculated covariance of the left channel and the right channel is less than or equal to the energy threshold.

[0122] EEE6: The method according to any one of EEE1 to EEE5, wherein selecting the stereo coding mode comprises determining a bit cost associated with each of a plurality of stereo coding modes based on the calculated energy of the left channel, the calculated energy of the right channel, the calculated covariance of the left channel and the right channel, and a transmission cost of the stereo audio signal, and selecting the stereo coding mode based on the bit cost.

[0123] EEE7: The method according to EEE6, wherein determining the bit cost associated with each of the plurality of stereo coding modes comprises determining an energy ratio of signals included in each of the plurality of stereo coding modes and comparing the energy ratio to a threshold, the bit cost indicating a bit reduction rate for each of the plurality of stereo coding modes compared to encoding the left channel and the right channel, and the threshold being based on the calculated energy of the left channel and the calculated energy of the right channel.

[0124] EEE8: The method of any of EEE1-EEE7, further comprising signaling the selected stereo coding mode using 2 bits per frame.

[0125] EEE9: The method of EEE8, further comprising signaling the presence of said phase difference or the presence of said side prediction data using one bit per frame.

[0126] EEE10: The method according to any one of EEE1 to EEE9, wherein the phase difference and the side prediction data are quantized.

[0127] EEE11: An apparatus for encoding a stereo audio signal into a bitstream, comprising: passing a left channel and a right channel of the stereo audio signal to a complex-valued filter bank analysis block, thereby generating one or more frequency bands; calculating, for each of the one or more frequency bands, an energy of the left channel, an energy of the right channel, and a covariance of the left channel and the right channel; selecting a stereo coding mode for encoding the left channel and the right channel based on the calculated energy of the left channel, the calculated energy of the right channel, and the calculated covariance of the left channel and the right channel; and selecting a stereo coding mode for encoding the left channel and the right channel based on the calculated energy of the left channel, the calculated energy of the right channel, and the calculated covariance of the left channel and the right channel. is an extended mid / side coding mode, calculating a phase difference between the left channel and the right channel, adjusting phase matching between the left channel and the right channel based on the calculated phase difference, thereby generating matched left and right channels, converting the matched left and right channels into a mid signal and a side signal, generating a residual signal based on side prediction data and the side signal, encoding the mid signal, the residual signal, the phase difference, and the side prediction data into the bitstream, and outputting the bitstream for the selected stereo coding mode.

[0128] EEE12: The apparatus described in EEE11, wherein the electronic processor is configured to: if the stereo coding mode is a separate coding mode, encode the left channel and the right channel into the bitstream.

[0129] EEE13: The apparatus described in any of EEE11 to EEE12, wherein the electronic processor is configured to, if the stereo coding mode is a mid / side coding mode, convert the matched left channel and the matched right channel into the mid signal and the side signal, and encode the mid signal and the side signal into the bitstream.

[0130] EEE14: The apparatus described in any of EEE11 to EEE13, wherein adjusting the phase matching between the left channel and the right channel comprises matching the left channel and the right channel by adjusting the phase of the right channel.

[0131] EEE15: The apparatus described in any of EEE11 to EEE14, wherein, to calculate the phase difference, the electronic processor is configured to compare the calculated covariance of the left channel and the right channel to an energy threshold, and to set the calculated phase difference to zero if the calculated covariance of the left channel and the right channel is less than or equal to the energy threshold.

[0132] EEE16: The apparatus described in any of EEE11 to EEE15, wherein, to select the stereo coding mode, the electronic processor is configured to: determine a bit cost associated with each of a plurality of stereo coding modes based on the calculated energy of the left channel, the calculated energy of the right channel, the calculated covariance of the left channel and the right channel, and a transmission cost of the stereo audio signal; and select the stereo coding mode based on the bit cost.

[0133] EEE17: The apparatus described in EEE16, wherein to determine the bit cost associated with each of the plurality of stereo coding modes, the electronic processor comprises determining an energy ratio of signals included in each of the plurality of stereo coding modes and comparing the energy ratio to a threshold, the bit cost indicating a bit reduction rate for each of the plurality of stereo coding modes compared to encoding the left channel and the right channel, and the threshold is based on the calculated energy of the left channel and the calculated energy of the right channel.

[0134] EEE18: The apparatus described in any one of EEE11 to EEE17, wherein the electronic processor is configured to signal the selected stereo coding mode using 2 bits per frame.

[0135] EEE19: The apparatus according to EEE18, configured to signal the presence of said phase difference or said side prediction data using one bit per frame.

[0136] EEE20: The apparatus according to any one of EEE11 to EEE19, wherein the phase difference and the side prediction data are quantized.

[0137] EEE21: The apparatus described in any of EEE11 to EEE20, wherein the phase differences are linearly quantized inter-channel phase differences, and to encode the phase differences, the electronic processor is configured to: calculate, for each frequency band, second order differences of the linearly quantized inter-channel phase differences; wrap the second order differences to a 2π range; and encode the wrapped second order differences.

[0138] EEE22: The method according to any one of EEE1 to EEE10, wherein said bitstream corresponds to an IVAS bitstream.

[0139] EEE23: The device according to any one of EEE11 to EEE21, wherein the bitstream corresponds to an IVAS bitstream.

[0140] EEE24: A method for decoding a stereo audio signal, comprising: receiving an encoded bitstream; decoding from the bitstream a replicated mid signal, a replicated residual signal, and replicated stereo metadata including phase difference and side prediction data; converting the replicated mid signal and the replicated residual signal using the replicated stereo metadata into a replicated left channel and a replicated right channel; and passing the replicated left channel and the replicated right channel to a filter bank analysis block, thereby regenerating the original left channel and the original right channel.

[0141] EEE25: The method according to EEE24, wherein converting the replicated mid signal and the replicated residual signal comprises generating a side signal from the replicated residual signal based on the side prediction data.

[0142] EEE26. The method of any one of EEE24 to EEE25, further comprising aligning the replicated left channel and the replicated right channel using the phase difference.

[0143] EEE27: An apparatus for decoding a stereo audio signal, the apparatus comprising an electronic processor configured to: receive an encoded bitstream; decode from the bitstream a replicated mid signal, a replicated residual signal, and replicated stereo metadata including phase difference and side prediction data; transform the replicated mid signal and the replicated residual signal using the replicated stereo metadata into a replicated left channel and a replicated right channel; and pass the replicated left channel and the replicated right channel to a filter bank analysis block, thereby regenerating the original left channel and the original right channel.

[0144] EEE28: The apparatus according to EEE27, wherein, to convert the replicated mid signal and the replicated residual signal, the electronic processor is configured to generate a side signal from the replicated residual signal based on the side prediction data.

[0145] EEE29: The apparatus of any of EEE27 to EEE28, wherein the electronic processor is configured to align the replicated left channel and the replicated right channel using the phase difference.

[0146] EEE30: A method for encoding a stereo audio signal, comprising: determining an advanced stereo coding mode selected from the group consisting of an L / R coding mode, a mid / side coding mode, and an extended mid / side coding mode for encoding the stereo audio signal; signaling the advanced stereo coding mode using two bits per frame; signaling phase difference data using one bit per frame if the advanced stereo coding mode is the extended mid / side coding mode; and signaling prediction data using one bit per frame if the advanced stereo coding mode is the extended mid / side coding mode.

[0147] EEE31: A non-transitory computer-readable storage medium having recorded thereon a program of instructions executable by an apparatus to perform the method described in any one of EEE1-EEE10, EEE22, EEE24-EEE26 or EEE30.

[0148] With respect to the processes, systems, methods, heuristics, etc. described herein, although steps of these processes, etc. are described as occurring in a particular order, it will be understood that these processes may be implemented with steps performed in an order different from that described. Furthermore, it will be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be substituted, modified, or omitted. In other words, the process descriptions herein are provided for the purpose of illustrating particular embodiments and are not intended to limit the scope of the claims in any way.

[0149] Therefore, it should be understood that the above description is illustrative and not limiting. Many embodiments and applications other than the described examples will become apparent from reading the above description. The scope should be determined not with reference to the above description, but with reference to the appended claims, along with the full scope of equivalents to which the appended claims are entitled. It is anticipated and intended that the technology described herein will evolve in the future, and the disclosed systems and methods will be incorporated into such future embodiments. In other words, it should be understood that the present application is subject to modification and variation.

[0150] All terms used in the claims, unless expressly stated otherwise herein, shall be given the broadest possible reasonable interpretation and ordinary meaning as understood by one skilled in the art to which they pertain. In particular, the use of singular articles such as "a," "the," "said," etc. shall refer to one or more elements unless the claim expressly states otherwise.

[0151] The Abstract of the Disclosure is provided to allow the reader to quickly grasp the nature of the technical disclosure. It is submitted not to be used to interpret or limit the scope or meaning of the claims. It may also be seen that in the above Detailed Description, various features are grouped together in various embodiments to simplify the disclosure. This method of disclosure does not reflect an intention that the claimed embodiments encompass more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Accordingly, the following claims are incorporated into the Detailed Description, with each claim standing on its own as separately claimed subject matter.

Claims

1. A method (700) for encoding a stereo audio signal into a bitstream, comprising: passing (705) the left and right channels of said stereo audio signal to a complex-valued filter bank analysis block (205), thereby responsively generating one or more frequency bands; calculating (710) for each of the one or more frequency bands the energy of the left channel, the energy of the right channel, and the covariance of the left and right channels; selecting (720) a stereo coding mode for encoding the left channel and the right channel based on the calculated energy of the left channel, the calculated energy of the right channel, and the calculated covariance of the left channel and the right channel; and If the stereo encoding mode is an extended mid / side encoding mode, calculating a phase difference between the left channel and the right channel; adjusting (740) the phase matching between the left channel and the right channel based on the calculated phase difference, thereby generating a matched left channel and a matched right channel; converting (745) the matched left channel and the matched right channel into a mid signal and a side signal; generating (750) a residual signal based on side prediction data and the side signal; and encoding (755) the mid signal, the residual signal, and the side prediction data into the bitstream; and outputting the bitstream for the selected stereo coding mode; The method includes:

2. 7. The method (700) of claim 1, wherein the bitstream corresponds to an IVAS bitstream.

3. When the stereo encoding mode is a separation encoding mode, encoding (725) the left channel and the right channel into the bitstream; The method (700) of claim 1 or 2, comprising:

4. If the stereo encoding mode is a mid / side encoding mode, converting (735) the matched left channel and the matched right channel into a mid signal and a side signal; and encoding (740) the mid-signal and the side-signal into the bitstream; The method (700) of any of claims 1 to 3, comprising:

5. The method (700) according to any one of claims 1 to 4, wherein the mid signal in the extended mid / side coding mode represents the sum of the left channel and the right channel, and the side signal in the mid / side coding mode represents the difference between the left channel and the right channel.

6. The phase difference is calculated as follows: comparing the calculated covariance of the left and right channels to an energy threshold; and setting the calculated phase difference to zero if the calculated covariance of the left channel and the right channel is less than or equal to the energy threshold; The method (700) of any of claims 1 to 5, comprising:

7. Selecting the stereo encoding mode comprises: determining (715) a bit cost associated with each of a plurality of stereo coding modes based on the calculated energy of the left channel, the calculated energy of the right channel, the calculated covariance of the left and right channels, and a transmission cost of the stereo audio signal; and selecting the stereo coding mode based on the bit cost; The method (700) of any of claims 1 to 6, comprising:

8. Determining (715) the bit costs associated with each of the plurality of stereo coding modes comprises: determining an energy ratio of signals included in each of said plurality of stereo coding modes; and comparing the energy ratio to a threshold; the bit cost indicates a bit reduction rate for each of the plurality of stereo coding modes compared to encoding the left channel and the right channel; the threshold is based on the calculated energy of the left channel and the calculated energy of the right channel.

8. The method (700) of claim 7.

9. The method (700) of any of claims 1 to 8, further comprising signaling the selected stereo coding mode using 2 bits per frame.

10. 10. The method (700) of claim 9, further comprising signaling the presence of the phase difference or the presence of the side prediction data using one bit per frame.

11. The method (700) according to any of the preceding claims, wherein the phase difference and the side prediction data are quantized.

12. A non-transitory computer-readable storage medium having recorded thereon a program of instructions executable by an apparatus to perform the method according to any one of claims 1 to 11.

13. An apparatus (1100) for encoding a stereo audio signal into a bitstream, comprising: passing (705) the left and right channels of said stereo audio signal to a complex-valued filter bank analysis block (205), thereby responsively generating one or more frequency bands; calculating (710) for each of the one or more frequency bands the energy of the left channel, the energy of the right channel, and the covariance of the left and right channels; selecting (720) a stereo coding mode for encoding the left channel and the right channel based on the calculated energy of the left channel, the calculated energy of the right channel, and the calculated covariance of the left channel and the right channel; and If the stereo encoding mode is an extended mid / side encoding mode (750), calculating a phase difference between the left channel and the right channel; adjusting (740) the phase matching between the left channel and the right channel based on the phase difference, thereby generating a matched left channel and a matched right channel; converting (745) the matched left channel and the matched right channel into a mid signal and a side signal; generating (750) a residual signal based on side prediction data and the side signal; and encoding (755) the mid signal, the residual signal, and the side prediction data into the bitstream; and outputting the bitstream for the selected stereo coding mode; An apparatus (1100) comprising an electronic processor (1101) configured to:

14. The electronic processor (1101) When the stereo encoding mode is a separation encoding mode, encoding (725) the left channel and the right channel into the bitstream; The apparatus (1100) of claim 13, configured to:

15. The electronic processor (1101) If the stereo encoding mode is a mid / side encoding mode, converting (730) the matched left channel and the matched right channel into the mid signal and the side signal; and encoding (735) the mid-signal and the side-signal into a bitstream; The apparatus (1100) according to any of claims 13 to 14, configured to:

16. 16. The apparatus (1100) of any of claims 13 to 15, wherein adjusting the phase matching between the left channel and the right channel comprises matching the left channel and the right channel by adjusting the phase of the right channel.

17. To calculate the phase difference, the electronic processor (1101) comparing the calculated covariance of the left and right channels to an energy threshold; and setting the calculated phase difference to zero if the calculated covariance of the left channel and the right channel is less than or equal to the energy threshold; The apparatus (1100) according to any of claims 13 to 16, configured to:

18. To select the stereo encoding mode, the electronic processor (1101) determining (715) a bit cost associated with each of a plurality of stereo coding modes based on the calculated energy of the left channel, the calculated energy of the right channel, the calculated covariance of the left and right channels, and a transmission cost of the stereo audio signal; and selecting the stereo coding mode based on the bit cost; The apparatus (1100) according to any of claims 13 to 17, configured to:

19. To determine (715) the bit costs associated with each of the plurality of stereo coding modes, the electronic processor (1101) determining an energy ratio of signals included in each of said plurality of stereo coding modes; and comparing the energy ratio to a threshold; the bit cost indicates a bit reduction rate for each of the plurality of stereo coding modes compared to encoding the left channel and the right channel; the threshold is based on the calculated energy of the left channel and the calculated energy of the right channel.

20. The apparatus (1100) of claim 18.

20. 20. The apparatus (1100) of any one of claims 13 to 19, wherein the electronic processor (1101) is configured to signal the selected stereo coding mode using 2 bits per frame.

21. 21. The apparatus (1100) of claim 20 configured to signal the phase difference or the presence of the side prediction data using one bit per frame.

Images