High-order Ambisonics spatial coding for low-latency immersive audio codecs

Abstract

A method for encoding Higher Order Ambisonics (HOA) audio is described herein. The method includes receiving an input HOA audio signal having more than four Ambisonics channels, encoding the HOA audio signal using a SPAR coding framework and a core audio encoder, and providing the encoded HOA audio signal including the core encoded SPAR downmix channel and encoded SPAR metadata to a downstream device. A method for decoding Higher Order Ambisonics (HOA) audio, a respective apparatus and a computer program product are further described herein.

Description

[Technical field]

[0001] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims priority to U.S. Provisional Patent Application No. 63 / 301,152, filed January 20, 2022, U.S. Provisional Patent Application No. 63 / 394,586, filed August 2, 2022, and U.S. Provisional Patent Application No. 63 / 476,518, filed December 21, 2022, each of which is incorporated by reference in its entirety herein.

[0002] The present disclosure generally relates to a method for encoding Higher Order Ambisonics (HOA) audio, in particular, the method includes encoding an HOA audio signal using a spatial reconstruction (SPAR) coding framework and a core audio encoder. The present disclosure further relates to a method for decoding HOA audio, a respective device, and a computer program product.

[0003] Although certain embodiments of the present disclosure are described herein with particularity, the present disclosure is not limited to such fields of use and is applicable in a broader context. [Background technology]

[0004] Although background art is described throughout this disclosure, this description should not be taken as an admission that such fields are widely known or form part of the common general knowledge in the art. Summary of the Invention [Problem to be solved by the invention]

[0005] SPAR is a technique for spatially coding Ambisonics and is used in the Immersive Voice and Audio Services (IVAS) codec standardized by the 3rd Generation Partnership Project (3GPP). The SPAR coding framework has been applied so far for first-order Ambisonics (FOA) at various bit rates. However, there is still a need to improve the SPAR algorithm and extend it to higher-order Ambisonics, especially to obtain good results within the IVAS framework. [Means for solving the problem]

[0006] According to a first aspect of the present disclosure, a method for encoding Higher Order Ambisonics (HOA) audio is provided. The method may include receiving an input HOA audio signal having more than four Ambisonics channels. The method may further include encoding the HOA audio signal using a SPAR coding framework and a core audio encoder. The method may include providing the encoded HOA audio signal, including the core encoded SPAR downmix channel and the encoded SPAR metadata, to a downstream device.

[0007] In some embodiments, the encoding is based on some or all of the Ambisonics channels and a representation of the W channel and a set of n total A prediction residual signal is generated by calculating respective prediction coefficients in the SPAR metadata, and a set of n total A subset of the prediction residual signal that is directly coded res By selecting a prediction residual signal, a number of n dmx =n res and obtaining a +1 downmix channel (+1 refers to including a representation of the W channel).

[0008] In some embodiments, a subset of n resThe selection of the prediction residual signal may be based on a threshold value for directly coded channels, which indicates the maximum number of directly coded channels.

[0009] In some embodiments, the threshold for the direct coded channel may be determined based on information indicative of one or more of a bit rate limit, metadata size, core codec capabilities, and audio quality.

[0010] In some embodiments, the threshold for the direct-coded channel may be selected from a predetermined set of thresholds for the direct-coded channel.

[0011] In some embodiments, a subset of n res The prediction residual signal may be selected according to a channel ranking of the Ambisonics channels from high-ranked channel to low-ranked channel.

[0012] In some embodiments, the channel ranking of the Ambisonics channels may be based on the perceptual importance of the Ambisonics channels, with Ambisonics channels having relatively higher channel rankings having relatively higher perceptual importance.

[0013] In some embodiments, the channel ranking of the Ambisonics channels may be based on an agreement of channel rankings between the encoder and decoder.

[0014] In some embodiments, spherical harmonics that have a relatively large overlap with the left-right front-back plane at a given order l are The Ambisonics channel corresponding to JPEG2025504862000002.jpg614 is a spherical harmonic function with a relatively large overlap with the height direction. It may be ranked as being of higher perceptual importance than the Ambisonics channel corresponding to JPEG2025504862000003.jpg614.

[0015] In some embodiments, spherical harmonics with relatively large overlap in the left and right directions The ambisonic channels corresponding to JPEG2025504862000004.jpg614 are spherical harmonic functions with relatively large overlap in the front - back direction Can be ranked as having a higher perceptual importance than the ambisonic channels corresponding to JPEG2025504862000005.jpg614.

[0016] In some embodiments, spherical harmonic functions of a given degree l where |m| = l Pairs formed by the ambisonic channels corresponding to JPEG2025504862000006.jpg614 can be ranked as having a higher perceptual importance than HOA channels of a given degree l where |m| < l.

[0017] In some embodiments, spherical harmonic functions of a given degree l The channel ranking of the ambisonic channels corresponding to JPEG2025504862000007.jpg614 is for spherical harmonic functions of degree (l + 1) Can form the channel ranking of a subset of the ambisonic channels corresponding to JPEG2025504862000008.jpg616, and the channel ranking of the ambisonic channels of degree (l + 1) can start from the channel ranking of the ambisonic channels of degree l.

[0018] In some embodiments, spherical harmonic functions with relatively large overlap in the left - right - front - back plane of a given degree l The ambisonic channels corresponding to JPEG2025504862000009.jpg614 are spherical harmonic functions of degree (l - 1) with relatively large overlap in the height direction Can be ranked as having a higher perceptual importance than the ambisonic channels corresponding to JPEG2025504862000010.jpg616.

[0019] In some embodiments, a subset of n res One or more prediction residual signals to be added later to the prediction residual signal are spherical harmonic functions Spherical harmonics function before the Ambisonics channel corresponding to JPEG2025504862000011.jpg614 Spherical harmonics rather than the Ambisonics channel corresponding to JPEG2025504862000012.jpg613 may be selected based on a ranking that highly ranks the Ambisonics channel corresponding to JPEG2025504862000013.jpg614, where 0<|m| <lである。

[0020] In some embodiments, the encoding is performed by encoding each coefficient in the SPAR metadata as the remaining n dec =n total -n res The method may further include representing a parametric channel based on computing from the prediction residual signal.

[0021] In some embodiments, the operations in the SPAR metadata may include calculating multiple inter-prediction coefficients for use by the decoder, such as n res From the directly coded prediction residual signal dec This may include reconstructing at least a portion of the parametric channels.

[0022] In some embodiments, the operations in the SPAR metadata may further include computing multiple decorrelator coefficients for use by the decoder to account for remaining energy not accounted for by the prediction and inter-prediction coefficients during reconstruction.

[0023] In some embodiments, the operations in the SPAR metadata may further include computing at least one of the prediction coefficients, the inter-prediction coefficients, and the decorrelator coefficients at a first temporal resolution t1 milliseconds that is longer than the second temporal resolution t2 milliseconds of the encoder filter bank.

[0024] In some embodiments, operations at the second time resolution t2 milliseconds may be performed only on the high frequency band.

[0025] In some embodiments, a calculation at the second time resolution t2 milliseconds may be performed if a transient is detected.

[0026] In some embodiments, the computation in the SPAR metadata may further include computing a normalization term for a channel corresponding to a given Ambisonics order l using only the covariance estimates for the channel corresponding to order l.

[0027] In some embodiments, the encoding may further include obtaining a bitrate limit value, selecting a SPAR quantization mode from a set of SPAR quantization modes that matches the bitrate limit value, and applying the selected SPAR quantization mode to the SPAR metadata.

[0028] In some embodiments, some or all of the modes of a set of SPAR quantization modes may include reallocating bits from coefficients associated with Ambisonics channels that are relatively lower ranked in the channel rankings to coefficients associated with Ambisonics channels that are relatively higher ranked in the channel rankings.

[0029] In some embodiments, some or all of the modes of a set of SPAR quantization modes may include selecting a subset of inter-prediction coefficients to exclude from the plurality of inter-prediction coefficients.

[0030] In some embodiments, some or all of the modes of a set of SPAR quantization modes may include selecting a subset of decorrelator coefficients to remove from the plurality of decorrelator coefficients.

[0031] In some embodiments, selecting a subset of coefficients may be based on a channel ranking of the Ambisonics channels.

[0032] In some embodiments, the received input HOA audio signal may be comprised of Ambisonics channels that are ranked as having relatively high perceptual importance.

[0033] According to a second aspect of the present disclosure, a method for decoding High Order Ambisonics (HOA) audio is provided. The method may include receiving an encoded HOA audio signal obtained by applying a SPAR coding framework and a core audio encoder to an input HOA audio signal having more than four Ambisonics channels. The method may further include decoding the encoded HOA audio signal to obtain a decoded HOA audio signal including a core decoded SPAR downmix channel and decoded SPAR metadata. The method may include reconstructing the input HOA audio signal based on the decoded HOA audio signal to obtain a reconstructed input HOA audio signal as an output HOA audio signal.

[0034] In some embodiments, the core decoded SPAR downmix channel is a representation of the W channel and a set of n res and a directly coded prediction residual signal, and the decoded SPAR metadata may include prediction coefficients, inter-prediction coefficients, and decorrelator coefficients.

[0035] In some embodiments, reconstructing the input HOA audio signal includes predicting a subset of Ambisonics channels of the HOA audio signal based on a representation of the W channel and a plurality of prediction coefficients; res and adding in a directly coded prediction residual signal.

[0036] In some embodiments, reconstructing the input HOA audio signal includes: a representation of the W channel; a plurality of prediction coefficients; a set of n resThe method may further include determining a remaining parametric channel based on the directly coded prediction residual signal and the plurality of inter-prediction coefficients.

[0037] In some embodiments, reconstructing the input HOA audio signal may further include calculating an indication of remaining energy not accounted for by the prediction coefficients and the multiple inter-prediction coefficients based on the multiple decorrelator coefficients and the multiple decorrelated versions of the W channel.

[0038] According to a third aspect of the present disclosure, an apparatus for encoding Higher Order Ambisonics (HOA) audio is provided. The apparatus may include one or more processors configured to perform a method including receiving an input HOA audio signal having more than four Ambisonics channels, encoding the HOA audio signal using a SPAR coding framework and a core audio encoder, and providing the encoded HOA audio signal including the core encoded SPAR downmix channel and the encoded SPAR metadata to a downstream device.

[0039] According to a fourth aspect of the present disclosure, an apparatus for decoding Higher Order Ambisonics (HOA) audio is provided. The apparatus may include one or more processors configured to perform a method including: receiving an encoded HOA audio signal obtained by applying a SPAR coding framework and a core audio encoder to an input HOA audio signal having more than four Ambisonics channels, decoding the encoded HOA audio signal to obtain a decoded HOA audio signal including core decoded SPAR downmix channels and decoded SPAR metadata, and reconstructing the input HOA audio signal based on the decoded HOA audio signal to obtain the reconstructed input HOA audio signal as an output HOA audio signal.

[0040] According to a fifth aspect of the present disclosure, there is provided an apparatus including a memory and one or more processors configured to perform a method for encoding Higher Order Ambisonics (HOA) audio or a method for decoding Higher Order Ambisonics (HOA) audio.

[0041] According to a sixth aspect of the present disclosure, there is provided a system of an apparatus for encoding Higher Order Ambisonics (HOA) audio and an apparatus for decoding Higher Order Ambisonics (HOA) audio.

[0042] According to a seventh aspect of the present disclosure, there is provided a program comprising instructions which, when executed by a processor, cause the processor to perform a method for encoding Higher Order Ambisonics (HOA) audio or a method for decoding Higher Order Ambisonics (HOA) audio.

[0043] According to an eighth aspect of the present disclosure, there is provided a computer-readable storage medium storing the above program. [Brief description of the drawings]

[0044] Example embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying drawings, in which: [Figure 1] FIG. 1 illustrates an example block diagram of a codec for encoding and decoding HOA audio signals according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 illustrates an example method for encoding Higher Order Ambisonics (HOA) audio according to an embodiment of the present disclosure. [Diagram 3] FIG. 3 illustrates an example method for decoding Higher Order Ambisonics (HOA) audio according to an embodiment of the present disclosure. [Figure 4] FIG. 4 illustrates an example block diagram of an HOA encoder including a SPAR HOA encoder and a core encoder, according to an embodiment of the present disclosure. [Diagram 5] FIG. 5 illustrates an example block diagram of a SPAR HOA encoder according to an embodiment of the present disclosure. [Figure 6] FIG. 6 illustrates an example block diagram of an HOA decoder including a SPAR HOA decoder and a core decoder, the SPAR HOA decoder including a metadata decoder, a predictor-1), a mutual predictor-1) configured to perform inverse encoder side operations, and a decorrelator, in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0045] Several embodiments are described in detail. Examples of these embodiments are illustrated in the accompanying drawings. However, whenever possible, similar or similar reference numerals may be used in the drawings to indicate similar or similar functionality. The drawings depict embodiments of the systems, devices, or methods disclosed herein, but for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods described herein may be employed without departing from the principles described herein.

[0046] Immersive Voice and Audio Services (IVAS) Framework First, a viable example of an IVAS framework will be described as an example of a framework to which the technology of the present disclosure can be applied and is not a limitation of the present disclosure.

[0047] IVAS provides a spatial audio experience for communication and entertainment applications. The basic spatial audio format is typically FOA. Four signals (W, Y, Z, X) are coded that allow rendering to the desired output format, e.g. immersive loudspeaker playback or binaural playback over headphones. Depending on the total available bitrate, 1, 2, 3 or 4 downmix channels can be delivered by the core audio codec with low latency. The W channel is delivered unmodified or modified (in case of active W) to allow a better prediction of the remaining channels. The downmix channels other than the W channel are the generated residual signal after prediction (prediction residual signal) together with the respective parameters (metadata), the so-called SPAR parameters. The SPAR parameters can be coded for perceptually motivated frequency bands, the number of bands is typically 12.

[0048] At the decoder, the four FOA signals are reconstructed by processing the downmix channels and their decorrelated versions with the transmitted parameters. This process may also be called upmixing, and the parameters (metadata) are called SPAR parameters. The IVAS decoding process includes core decoding and SPAR upmixing. The core decoded signals may be transformed by a complex-valued low-latency filter bank. After SPAR upmixing in the frequency domain, the FOA time-domain signals are generated by filter bank synthesis.

[0049] The methods and apparatus described herein may relate to extending the SPAR algorithm to higher order Ambisonics, and in particular to improving the SPAR algorithm to obtain good results within the IVAS framework.

[0050] HOA Audio Coding and Decoding Using SPAR 1 shows a schematic block diagram of an example of an encoder / decoder ("codec") 100 for encoding and decoding HOA audio signals. The audio coder / decoder may have one or more input and output channels and may be, for example, a mono or multi-channel codec.

[0051] In particular, the schematic example of Figure 1 shows an HOA encoder 101 and an HOA decoder 104 located downstream of the HOA encoder 101. The HOA codec 100 includes SPAR HOA codecs 102, 106 and respective core codecs 103, 105 for encoding and decoding HOA audio, e.g. generating and decoding an IVAS bitstream in HOA format. The core codecs 103, 105 may be low latency core codecs.

[0052] As shown in the example of FIG. 1, the HOA audio encoder 101 encodes the input HOA audio signal into more than four Ambisonics channels (W, Y, Z, X, A...), i.e. (N+1) 2 In some embodiments, the HOA audio encoder 101 receives more than four Ambisonics channels in a subset of (N+1) 2 It may also be an Ambisonics channel. The received input HOA audio signal is encoded using the SPAR HOA encoder 102 and the core encoder 103. The encoded HOA audio signal includes the core encoded SPAR downmix channel output by the core encoder 103 and the encoded SPAR metadata output by the SPAR HOA encoder 102. The encoded HOA audio signal is then provided to a corresponding downstream device, for example as an IVAS bitstream. The IVAS bitstream may include a corresponding audio bitstream including the core encoded downmix channel and a metadata bitstream including the encoded SPAR metadata. However, the HOA audio encoder 101 may be an IVAS encoder.

[0053] In the downstream device, the encoded HOA audio signal is received by a corresponding HOA audio decoder 104, for example as an IVAS bitstream. However, the HOA audio decoder 104 may be an IVAS decoder. The encoded HOA audio signal is decoded using the core decoder 105 to obtain a decoded HOA audio signal. The decoded HOA audio signal includes the core decoded SPAR downmix channel output by the core decoder 105 and the decoded SPAR metadata obtained in the SPAR HOA decoder 106. Based on the decoded HOA audio signal (downmix channel and SPAR metadata), the input HOA audio signal is reconstructed using the SPAR HOA decoder 106 to obtain the respective output HOA audio signal (W, Y, Z, X, A...). However, the output HOA audio signal can also be said to be a reconstruction of the HOA input signal (received by the HOA encoder).

[0054] The example of FIG. 2 illustrates a corresponding method 200 for encoding HOA audio, according to an embodiment of the present disclosure.

[0055] In step S201, an input HOA audio signal having more than four Ambisonics channels is received.

[0056] In step S202, the HOA audio signal is encoded using the SPAR coding framework and the core audio encoder.

[0057] In step S203, the encoded HOA audio signal, including the core encoded SPAR downmix channel and the encoded SPAR metadata, is provided to a downstream device.

[0058] In one embodiment, the received (input) HOA audio signal may consist of Ambisonics channels that are ranked as having relatively high perceptual importance, as described below.

[0059] The example of FIG. 3 illustrates a corresponding method 300 for decoding HOA audio, according to an embodiment of the present disclosure.

[0060] In step S301, an encoded HOA audio signal obtained by applying a SPAR coding framework and a core audio encoder to an input HOA audio signal having more than four Ambisonics channels is received.

[0061] In step S302, the encoded HOA audio signal is decoded to obtain a decoded HOA audio signal including the core decoded SPAR downmix channel and the decoded SPAR metadata.

[0062] In step S303, an output HOA audio signal is obtained by reconstructing the input HOA audio signal based on the decoded HOA audio signal.

[0063] In one embodiment, the core decoded SPAR downmix channel is a representation of the W channel and a set of n res and a directly coded prediction residual signal. The decoded SPAR metadata may include prediction coefficients, inter-prediction coefficients, and decorrelator coefficients.

[0064] Reconstructing the (input) HOA audio signal includes predicting a subset of Ambisonics channels of the HOA audio signal based on a representation of the W channels and a plurality of prediction coefficients, and predicting a set of n res and adding in the directly coded prediction residual signal. The add-in may include adding in the predicted Ambisonics channels to a set of n resIt can also be said to mean combining with the corresponding directly coded prediction residual signal.

[0065] Reconstructing the input HOA audio signal then involves obtaining a representation of W channels, a set of prediction coefficients, and a set of n res The method may further include determining a remaining parametric channel based on the directly coded prediction residual signal and the plurality of inter-prediction coefficients.

[0066] Reconstructing the (input) HOA audio signal may further include calculating an indication of remaining energy not accounted for by the prediction coefficients and the multiple inter-prediction coefficients based on the multiple decorrelator coefficients and the multiple decorrelated versions of the W channel.

[0067] 4 shows an example block diagram of an HOA encoder 400 including a SPAR HOA encoder and a core encoder, with reference thereto for describing the encoding in more detail.

[0068] The SPAR HOA encoder 401 converts the input HOA signal into a set of SPAR downmixes (n dmx ) channels (W channel and selected prediction residual signal) and SPAR metadata (parameters, coefficients). That is, in one embodiment, the encoding is configured to convert a representation of the W channel and a set of n based on some or all of the Ambisonics channels. total This may include generating a prediction residual signal and therewith computing the respective prediction coefficients in the SPAR metadata. The W channel may always be transmitted untouched.

[0069] The predictor 402 may receive an input HOA audio signal with more than four Ambisonics channels (W, Y, X, Z, A...). In the prediction step, the input channels are predicted as a representation of W and a set of n total =(N+1) 2-1) The prediction residual signals (Y', Z', X', A'...) can be converted into a prediction residual signal (Y', Z', X', A'...) and the respective prediction coefficients PR can be computed as SPAR metadata. These prediction coefficients can be used to calculate the downmix / transport channels.

[0070] However, as one skilled in the art will understand and appreciate, W may be a passive channel W or an active channel W'.

[0071] In the above embodiment, a set of n total Of the prediction residual signal, one subset of n res The prediction residual signal may be selected as the one to be directly coded. The selection may be made, for example, in the downmix selector 403. res The prediction residual signal is sent to the core encoder 405 as a set of downmix channels n dmx (n dmx =n res +1, where the additional 1 represents the W channel. In other words, n total Among the prediction residual signals, multiple n res The residual signal can be coded directly, e.g., from 0 to (N+1) 2 -1. The core encoder 405 may be a low latency core encoder.

[0072] In principle any channel structure (from fully coded residual signal to fully parametric) could be possible, but since HOA support is expected to be a relatively high bit rate, it is expected to have enough bits to transmit at least the primary residual signal via the core codec, i.e. directly coded n res There may be a constraint on the number of prediction residual signals.

[0073] In one embodiment, a subset of n resThe selection of the prediction residual signal may be based on a threshold for directly coded channels, which indicates a maximum number of directly coded channels. The threshold for directly coded channels may be determined based on information indicative of one or more of a bit rate limit, a metadata size, a core codec capability, and an audio quality. Thus, the bit rate limit, the metadata size, the core codec capability, and the audio quality may determine the maximum number of directly coded channels. res The number of prediction residual signals is constrained.

[0074] In one embodiment, the direct coded channel threshold may be selected from a predefined set of direct coded channel thresholds, which may be a reasonable number for the directly coded prediction residual signal given the respective constraints, where the number of direct coded channels always includes a representation of the W channel.

[0075] In the SPAR framework, the coefficients (parameters, e.g., in the metadata) for reconstructing the input HOA audio signal at the decoder may include some and all of the prediction coefficients, the inter-prediction coefficients, and the decorrelator coefficients.

[0076] Therefore, in one embodiment, the encoding is performed in the SPAR metadata for the remaining n dec =n total -n res The encoding further includes representing the parametric channels based on computing respective coefficients from the prediction residual signal. In other words, the encoding includes encoding n dec =n total -n res It may further include generating parametric (parameterized) channels, which may be done in, for example, the respective parameterizers 404.

[0077] The SPAR metadata may then be encoded in a respective metadata encoder 405 to generate a respective metadata bitstream. dmxThe downmix channels may be encoded in the core encoder 406 to generate respective audio bitstreams. The metadata bitstream and the audio bitstream may then be combined with the respective IVAS bitstreams output from the HOA encoder.

[0078] The SPAR HOA encoder 500 is shown in more detail with reference to the example of FIG. dmx is selected as 4.

[0079] In the predictor 501, a representation of the W channel and a set of n total A prediction residual signal may be generated with the respective prediction coefficients PR computed in the SPAR metadata.

[0080] 1 set of n total Of the prediction residual signal, a particular downmix, e.g., n dmx If you choose =4, then n of the 1 subset is directly coded. res A prediction residual signal (eg, Y', X', Z') may be selected.

[0081] The encoding is done by using the remaining n dec =n total -n res The method may further include representing the parametric channels based on computing respective coefficients from the prediction residual signal.

[0082] In the second prediction step, n dec n selected to be directly coded into the residual signal res From the residual signal, a set of inter-prediction coefficients, i.e., C coefficients, is obtained, dec In one embodiment, the SPAR metadata can be generated together with the inter-predicted residual signal (A″, ...) by computing multiple inter-prediction coefficients for use by the decoder. resFrom the directly coded prediction residual signal dec This may include reconstructing at least a portion of the parametric channels.

[0083] Finally, the remaining inter-predicted residual signal (A", ...) may be used to calculate decorrelator coefficients P via energy matching 503. That is, in one embodiment, the operations in the SPAR metadata may further include calculating multiple decorrelator coefficients for use by the decoder to account for remaining energy not accounted for by the prediction and inter-prediction coefficients during reconstruction. The coefficients may be calculated band-by-band from a covariance band matrix generated from the input channels.

[0084] Overall, (N+1) 2 -1 Prediction (PR) coefficient, n res * n dec The cross-prediction (C) coefficient, and n dec A decorrelation (P) coefficient can be generated for each band (e.g., computed in the SPAR metadata). In many intermediate structures, n res and n dec When is neither large nor small, the number of C coefficients can quickly decrease the number of PR and P coefficients.

[0085] In general, the HOA decoder 104, 600 may be configured to reverse the operations performed by the HOA encoder 101, 400 to obtain an output (reconstructed input) HOA audio signal.

[0086] The example of FIG. 6 shows an example block diagram of an HOA decoder 600 including a SPAR HOA decoder 602 and a core decoder 601. The SPAR HOA decoder 602 includes a metadata decoder 603 and a predictor 604. -1 604, and a mutual predictor configured to perform an inverse operation (inverse prediction) on the encoder side. -1The encoder-side inverse operations include a decoder 605 and a decorrelator 606. As described in more detail below, performing the inverse operations on the encoder side may include predicting from the reconstructed W (using prediction coefficients) and predicting from the reconstructed residual channel (using inter-prediction coefficients), and combining the predicted signal with either the residual channel or the decorrelator output signal.

[0087] The HOA decoder 600 may be configured to receive an encoded HOA audio signal obtained by applying a SPAR coding framework and a core audio encoder to an input HOA audio signal having more than four Ambisonics channels. The encoded HOA audio signal may be received, for example, in the form of an IVA bitstream or in the form of a core codec bitstream. The bitstream may include a metadata bitstream and an audio bitstream. In one embodiment, the encoded HOA audio signal includes a representation of the W channel and a set of n res The encoded HOA audio signal may include a core encoded SPAR downmix channel, which may be a directly coded prediction residual signal. The encoded HOA audio signal may further include encoded SPAR metadata, which may be some or all of the prediction coefficients, the inter-prediction coefficients, and the decorrelator coefficients. res The directly coded prediction residual signal may be encoded in the audio bitstream, while the prediction coefficients, the inter-prediction coefficients, and the decorrelator coefficients may be encoded in the metadata bitstream.

[0088] However, prediction coefficients may be used to minimize the predictable energy in the residual downmix channel, inter-prediction coefficients may be used to further aid in reconstructing a fully parametrized channel from the residual channel, and decorrelator coefficients may be used to fill in the remaining energy not explained by the prediction coefficients and decorrelator coefficients.

[0089] The core decoder 601 may be configured to obtain core-decoded SPAR downmix channels by core-decoding the audio bitstream. The core-decoded SPAR downmix channels are each a set of n res The prediction residual signal (Y', X', Z') and a representation of the W channel may include the W channel and each set of n res The prediction residual signal, together with a metadata bitstream, may be sent to a SPAR HOA decoder 602. In the metadata decoder 603, the metadata bitstream may be decoded to obtain decoded SPAR metadata. The decoded SPAR metadata may include some or all of the prediction coefficients, the inter-prediction coefficients, and the decorrelator coefficients.

[0090] The SPAR HOA decoder 602 may be configured to obtain an output HOA audio signal (reconstruction of the input HOA audio signal) by reconstructing the input HOA audio signal based on the decoded HOA audio signal, i.e., based on the core decoded SPAR downmix channel and the decoded SPAR metadata.

[0091] Reconstructing the input HOA audio signal by the SPAR HOA decoder 602 includes predicting Ambisonics channels of a subset of the HOA audio signal based on the representation of the W channel and a plurality of prediction coefficients. -1 604. Then, a set of n res A directly coded prediction residual signal can be added in.

[0092] Reconstructing the input HOA audio signal then comprises a set of n res The method may further include determining a remaining parametric channel based on the directly coded prediction residual signal and the plurality of inter-prediction coefficients. As shown in FIG. 6, the remaining parametric channel (n dec ) is predicted from W channels using the prediction coefficients, and n resThe directly coded prediction residual signal can be reconstructed by inter-prediction, as shown in FIG. -1 605. Reconstructing the input HOA audio signal may further include calculating an indication of (capture of) the remaining energy not explained by the prediction coefficients and the multiple cross-prediction coefficients based on the multiple decorrelator coefficients and the output of the multiple decorrelated versions of the W channel. This may be performed within the decorrelator 606. In other words, the input covariance / signal energy may be matched using the decorrelator coefficients and the decorrelated version of the W channel.

[0093] The overall steps used to reconstruct the residual and parametric channels can be effectively summarized as follows: -Residual channels: W and n res W and predictions from PR coefficients -Parametric channels: W and n dec Add prediction from PR coefficients, cross prediction from the residual signal and C coefficients, and decorrelation from P coefficients and a decorrelated version of W

[0094] As one skilled in the art would understand and appreciate, the HOA decoder 600 may include one or more decorrelator blocks, which may be used to generate a decorrelated version of the W channel using a time domain or frequency domain decorrelator. The downmix and decorrelated channels may be used in combination with metadata for parametric reconstruction by the SPAR HOA decoder.

[0095] As one skilled in the art will further understand and appreciate, the HOA encoder 400 may further include a mixer, and the HOA decoder 600 may further include an inverse mixer, to achieve preferred inner channel ordering and output channel ordering, respectively.

[0096] SPAR Channel Rankings Expansion The Ambisonics input to SPAR is assumed to be SN3D normalized and to use ACN channel ordering. SPAR utilizes a slightly different preferred inter-channel ranking than ACN, in order to give higher relative importance to channels that are more spatially perceptually relevant, and therefore higher priority when transmitted as residual channels rather than parametric channels.

[0097] Given an original input of channels {W=0, Y=1, Z=2, X-3,...}, the Ambisonics channels may be described by their channel letter (e.g. W, Y, Z, X,...) or ACN channel number (0, 1, 2, 3,...) or individually by their "mode" or order and degree (l (or n),m).

[0098]

number

[0099] The Ambisonics channels may be further described by the spherical harmonics shown in Table 1, where φ and θ are the azimuth and elevation directions of the source's angle of arrival, however, it is understood that the spherical harmonics definitions shown in Table 1 are only an example and other definitions, normalizations, etc. are possible in this disclosure.

[0100] [Table 1]

[0101] As noted above, in one embodiment, a set of n total Of the prediction residual signal, one subset of n res The prediction residual signal may be selected to be directly coded. resThe selection of the prediction residual signal may be based on a direct coded channel threshold value indicating a maximum number of directly coded channels, which may correspond to the number of downmix channels.

[0102] Referring again to Figure 4, one subset of n res The prediction residual signal may be selected according to a channel ranking of the Ambisonics channels from a high-ranked channel to a low-ranked channel. The channel ranking of the Ambisonics channels may be based on a channel ranking agreement between the encoder and the decoder. Alternatively or additionally, the channel ranking of the Ambisonics channels may be based on a perceptual importance of the Ambisonics channels. An Ambisonics channel with a relatively high channel ranking has a relatively high perceptual importance.

[0103] For first-order Ambisonics, assuming that sound direction in Y (left / right) is more perceptually relevant than sound direction in X or Z, the preferred SPAR FOA internal ranking is {0,1,3,2} or {W,Y,X,Z}. Similarly, sounds in the XY plane are more relevant than height information, with X coming before Z. Extending this logic to HOA is non-trivial, since many mutually contradictory options are possible.

[0104] Channels in the XY plane In one embodiment, spherical harmonics that have a relatively large overlap with the left-right front-back plane at a given order l (order l may correspond to order n used in Table 1, where 0≦l≦N for HOA order N) The Ambisonics channel corresponding to JPEG2025504862000016.jpg614 is a spherical harmonic function with a relatively large overlap with the height direction. It can be ranked as being more perceptually important than the Ambisonics channel corresponding to JPEG2025504862000017.jpg614. Spherical harmonics with relatively large overlap with the height direction For the ambisonic channels corresponding to JPEG2025504862000018.jpg614, those with relatively less overlap in the height direction can be ranked higher than those with relatively more overlap in the height direction.

[0105] Alternatively or in addition to this, spherical harmonic functions of a given degree l when |m| = l Pairs formed by the ambisonic channels corresponding to JPEG2025504862000019.jpg614 can be ranked as having a higher perceptual importance than the HOA channels of a given degree l when |m| < l.

[0106] [1] Ranking the channels in the X - Y (left - right, front - back) plane {4,8} higher than the weaker {5,7} ones can lead to a more dominant {6} Z (height) component in the pattern of {0,1,3,2,4,8,5,7,6,...}. This takes channel pairs from the outside to the center for each degree of the HOA pyramid. For example, refer to Table 1 for comparison.

[0107] However, the central channel of all even degrees, for example, the 2nd - order channel {6} (mode (2,0)) actually has lobes in the X - Y plane. Therefore, it can be said that this is more perceptually relevant than the {5,7} pair and thus can be ranked higher than this.

[0108] This result will be, for example, in the pattern of {0,1,3,2,4,8,6,5,7,...}.

[0109] However, if it is considered that some HOA2 (HOA second - order) microphone array provider chooses to leave this channel blank during the conversion to ambisonics, it also makes sense to apply the above - mentioned pattern, that is, to lower the rank of 6, or in other words, not to raise the rank of 6.

[0110] [2] One could argue that the order of the channel pairs is {4,8} and not {8,4}. Either is possible, and if we keep the first-order pattern of the Y channel (1,-1) coming before the X channel (1,1), we end up with an embodiment in which the (n,-m) mode comes before the (n,m) mode. In other words, in one embodiment, spherical harmonics with relatively large overlap with the left and right directions are used. The Ambisonics channel corresponding to JPEG2025504862000020.jpg614 is a spherical harmonic function with relatively large overlap in the front-to-back direction. It may be ranked as being of higher perceptual importance than the Ambisonics channel corresponding to JPEG2025504862000021.jpg614.

[0111] Alternatively, the choice of which to put first can be made adaptively, for example based on some energy criterion. dmx See the latter point regarding having more than four channels.

[0112] Order l-1 before order l In one embodiment, a spherical harmonic function of a given order l The channel ranking of the Ambisonics channel corresponding to JPEG2025504862000022.jpg614 is the (l+1)th order spherical harmonic function A channel ranking may be formed for a subset of the Ambisonics channels corresponding to JPEG2025504862000023.jpg616. The channel ranking for the (l+1)th Ambisonics channel starts from the channel ranking for the lth Ambisonics channel.

[0113] [3] According to bitrate switching, input audio of a particular (high) order may be coded at lower orders at some bitrates and at the original order at other bitrates. For the purposes of bitrate switching, it is useful that the internal SPAR channel ranking of a given order is a subset of the higher orders, e.g., FOA ⊂ HOA2 ⊂ HOA3. Therefore, it may be beneficial to ensure that all l-th order channels appear before the (l+1)-th order channels.

number

[0114] Planar channel before non-planar channel In one embodiment, spherical harmonics with relatively large overlap in the left-right front-back plane of a given order l The Ambisonics channel corresponding to JPEG2025504862000025.jpg614 is an (l-1)th order spherical harmonic function with relatively large overlap in the height direction. It may be ranked as having higher perceptual importance than the Ambisonics channel corresponding to JPEG2025504862000026.jpg616.

[0115] [4] It can also be said that the predominantly planar channels from higher orders are, up to a point, more perceptually relevant than the height channels from lower orders. The l-th order Z channel can be said to provide height information to a sufficiently low resolution that second and third order (or higher) planar information is more relevant and preferable to be residually coded than the second order height channel. For example, the HOA rankings can be:

number

[0116] The difference between the rankings in equation (2) and equation (3) may be more pronounced when more than five downmix channels are selected to be directly coded.

[0117] Downmix Channel n dmx Increase the number of to >4 In the case of FOA, there is a clear benefit to every additional downmix channel beyond the W channel. In many cases, it becomes increasingly difficult to motivate the addition of individual channels beyond the second order based on spatial relevance.

[0118] For example, n dmx For n = 5, it becomes difficult to select which of the {4, 8} channel pairs to transmit. Conversely, it may be preferable to add downmix channels in the (n, + / - m) pairs, i.e., to add both {4, 8}, if they exist. Thus, using the ranking from equation (2), n dmx Reasonable choices for might be 1, 2, 3, 4, 6, 8, 9, 11, 13, 15, 16, etc.

[0119] For other reasons, such as bitrate limitations as mentioned in [2] above, dmx If desired, the last transmitted residual signal may be adaptively selected, i.e., in one embodiment, a subset of n res One or more prediction residual signals to be added to the prediction residual signal are spherical harmonic functions Spherical harmonics function before the Ambisonics channel corresponding to JPEG2025504862000028.jpg614 Spherical harmonics rather than the Ambisonics channel corresponding to JPEG2025504862000029.jpg613 The Ambisonics channel corresponding to JPEG2025504862000030.jpg614 may be selected based on the ranking that highly ranks it. <lである。

[0120] Downmix Channel n dmx Selection SPAR algorithms 1 and (N+1) 2 Any n between dmxWhile we support the choice of whichever is selected, the choice of the number of downmix channels to transmit may depend on the available bitrate, the size of the coded metadata, and any other applicable real-world considerations, such as core codec performance, complexity and memory constraints.

[0121] If the HOA is considered to be operating in high quality mode, the HOAn dmx The lower limit of is the highest quality FOA mode, e.g., n dmx It stands to reason that with this many input channels, once the required metadata is taken into account, the dmx is (N+1) 2 Allowing n to approach 1 means that the average bitrate per channel will be very low. Trying to boost the bitrate for the lower order channels, e.g. W, X, Y, Z channels, to a level that is well suited for FOA operation can lead to the problem of trying to encode the higher order residual signal with a very low quality / low bitrate core codec instance. Given the constraints of the core codec, dmx <=8 might be a reasonable choice.

[0122] Taking into account the computational complexity / memory footprint, we recommend that the number of downmix channels be set to that of the highest quality FOA mode, e.g., max n dmx It might be reasonable to further restrict it to =4.

[0123] Attempting to operate in HOA mode in extremely bitrate-limited scenarios is dmx = 3, which may prove to be an acceptable tradeoff between audio quality and spatial metadata quality.

[0124] n dmx Given that ,= 4 is unlikely to be exceeded, the preferred SPAR HOA inter-channel ranking may be as shown in equation (2), which combines the logic from points [1] to [3] above.

[0125] This produces 3x5 C coefficients per band for HOA2 and 3x12 C coefficients per band for HOA3 mode. This is close to the maximum possible inter-prediction metadata for HOA2, a bit less than the maximum for HOA3, but far from the minimum. This means that most of the bitrate is saved for metadata (MD).

[0126] Computing prediction coefficients for high-order channels - Patents.com The computation of prediction coefficients for the FOA input in SPAR can be determined based on the input covariance matrix. For example,

[0127]

number

[0128] Extending normalization to higher channels

number

[0129] In the above, R AB represents the elements of the input covariance matrix of signals A and B, and pr iis the prediction coefficient corresponding to the i-th channel of the HOA input in the ACN ordering, where the i-th channel can be any Ambisonics channel except the 0th W channel, and N is the HOA order.

[0130] Normalize each order separately based on spherical harmonics normalization For point source inputs, the prediction coefficient normalization described in equation (5) tends to result in over-normalization. For a point source input with full SN3D normalization and unity power, the Ambisonics input R iW The covariance between the W channel of and any other input channel i can be approximated as the spherical harmonic response corresponding to channel i in Table 1. In this case, the ideal value of the prediction coefficient for the i-th channel should be:

[0131]

number

[0132] Here, Y i is the spherical harmonic response corresponding to ACN channel i of the Ambisonics input as shown in Table 1.

[0133] However, in equation (5), R iW =Y i After substituting pr i If equation (5) is used to calculate JPEG2025504862000034.jpg812, where l corresponds to the order of ACN channel i in the corresponding mode (l,m), because the SN3D normalized spherical harmonics corresponding to each order form a unit vector. So,

number

[0134] where order is the order N of the Ambisonics input. Normalization as shown in equation (7) results in under-prediction, which can lead to higher post-prediction errors and subsequently coding problems.

[0135] It is therefore desirable to normalize the prediction coefficients separately for each order, as shown in the implementation example below.

[0136]

number

[0137] Here, l,i are the prediction coefficients corresponding to the i-th input channel (ACN) corresponding to order l. a and b are the starting and ending channel indices for order l. The mapping of i and l and a=(l+1) 2 -2l-1 and b=(l+1) 2 An ACN value of -1 may be used in Table 1 (except that n is used instead of l in Table 1). An example of the computation of prediction coefficients corresponding to the secondary channel V of an Ambisonics input is shown below.

[0138]

number

[0139] It is important to note that the normalization terms in equations (5) and (8) are such that the prediction coefficients are always within the desired quantization range to minimize the post-prediction error.

[0140] Improved frequency-based time resolution for parametric channel prediction - Patents.com The variance of the error after prediction in ACN channel i of an Ambisonics input of order l can be shown by:

[0141]

Number

[0142] It is desirable to reduce E in order to minimize the coding artifacts. The higher the value of E, the higher the contribution to the decorrelation from the decorrelator, which may lead to audio artifacts.

[0143] Improving the calculation of the pr coefficient helps reduce the value of E and decreases the dependence on the decorrelator. One way to improve the value prediction coefficient is to improve the time resolution of the analysis window and the covariance estimate when calculating the prediction coefficient. The idea here is to improve the time resolution only for the parametric channel in a way that is not affected by the encoder filter bank and computational complexity. An implementation example is described below.

[0144] When the input is HOA3 and n dmx = 4, assuming the time resolution of the encoder filter bank, the window length is mutually faded as t1 milliseconds as needed. Two sets of covariance estimates are calculated. The first set is calculated with the same t1 - millisecond time resolution as the encoder filter bank, and the second set is calculated with a t2 - millisecond time resolution. t2 < t1. The selection of the t2 time resolution depends on the decoder filter bank.

[0145] n dmx Calculate the prediction coefficient value with the t1 - millisecond time - resolution covariance estimate for the channel. When the core coder codes the n dmx channels with complete waveform reconstruction for the ambisonics input, the n dmx channels can be completely reconstructed using the prediction coefficients at a t1 - millisecond time resolution.

[0146] For parametric channels, the prediction coefficient values ​​are computed with a time resolution covariance estimate of t2 milliseconds. This results in improved prediction coefficients, especially at high frequencies, and reduces the post-prediction error. For parametric channels, the post-prediction error signal is not coded by the core coder, but instead is estimated by the decorrelator at the decoder.

[0147] In one implementation, t1 is equal to 20 and t2 is equal to 5.

[0148] In one implementation, a time resolution covariance estimate of t ms is used only at higher frequencies, while in another implementation, a time resolution covariance estimate of t ms is used when a transient is detected.

[0149] The increased temporal resolution of the prediction coefficients for the parametric channels does not impact the computation of the downmix channels, thus maintaining low computational complexity at the encoder side. In one implementation, the increased temporal resolution of the prediction coefficients requires that additional metadata be coded in the IVA bitstream.

[0150] In one implementation, the increased time resolution of the prediction coefficients requires a filter bank with finer time resolution at the decoder in order to apply the prediction coefficients to the corresponding time-frequency tiles.

[0151] HOA Metadata Encoding PCT / US2021 / 036886 and US Provisional Patent Application No. 63 / 037,784 describe a looped approach to SPAR metadata encoding. It depends on a set of quantization strategies (which determine how the metadata is quantized), a target metadata bitrate, and a maximum metadata bitrate. The quantized metadata is encoded using various encoding schemes (non-differential, time differential (striped), frequency differential) and encoder models. If metadata encoding is possible under the target bitrate, the loop ends. If not, the loop continues to try other schemes and coding models. After trying all of these, if it is less than the maximum metadata bitrate specified, the most efficient coding is selected and the loop ends. If not, the loop proceeds to a second quantization strategy, then a third (final) quantization strategy. The final quantization strategy is sufficiently coarse to ensure that the base-2 coded MD fits within the maximum metadata bitrate budget.

[0152] The HOA metadata encoding may be subject to a bitrate constraint, which may be a target metadata bitrate to be met or a maximum bitrate for the metadata encoding. Thus, in one embodiment, the encoding may include obtaining a bitrate limit, selecting a SPAR quantization mode from a set of SPAR quantization modes that meets the bitrate limit, and applying the selected SPAR quantization mode to the SPAR metadata.

[0153] Metadata encoded below the target metadata bitrate means that there are excess bits that can be distributed among the core coders to encode the audio. Conversely, if the metadata is encoded above the target bitrate, the excess bits are taken away from the allocation to the individual core coders according to a distribution strategy.

[0154] Thus, in one embodiment, some or all of a set of SPAR quantization modes may include reallocating bits from coefficients associated with Ambisonics channels that are relatively lower ranked in the channel rankings to coefficients associated with Ambisonics channels that are relatively higher ranked in the channel rankings.

[0155] The relationship between the target and worst case / maximum metadata bitrates that drive the metadata coding, in turn, has a large impact on the actual bitrate that the core coder uses to perform the audio coding.

[0156] In FOA mode, there is relatively little metadata, i.e. coefficients, to process and the associated quantization schemes vary from acceptable quality (low bitrates) to high quality / sophisticated quantization at high bitrates. Typical target and worst case FOA metadata bitrates are 10kbps and 15kbps, respectively.

[0157] In HOA mode, it is expected that there are much more coefficients to code, and of higher quality, if possible. Using a similar approach as in FOA, the target bitrate may be on the order of 70 kbps for HOA3, with a worst-case bitrate of the order of 130 kbps (even if the metadata quality is relatively low). Encoding some finely quantized metadata close to the worst-case limit (rather than slightly reducing quality and using coarser quantization to code close to the target metadata bitrate) may force the audio channel to be coded at a much lower, and often highly variable, bitrate than the preferred bitrate. This has potential impacts on audio quality.

[0158] Furthermore, the core coder may have a preferred operating range within which the SPAR minimum, target, and maximum core coder bitrates should be placed, since switching between the two operating ranges may not be suitable for audio quality consistency. It may be difficult, or even impossible, to account for large variations in metadata bitrates within these constraints.

[0159] The only way to solve this is to find a way to reduce the worst-case metadata bitrate. Several approaches to reducing the worst-case metadata have been explored.

[0160] Consider the sparseness of the matrix to be coded, e.g. the matrix of C coefficients. An analysis of the C coefficients may be performed to determine whether cross-prediction was useful (i.e., whether the coefficient is zero) from a particular residual channel to another parametric channel or in a particular band, so that the C parameters can be coded more efficiently.

[0161] Artificially generate sparsity / remove low-relevance metadata In one embodiment, some or all of the SPAR quantization modes of a set of SPAR quantization modes may include selecting a subset of inter-prediction coefficients to exclude from the plurality of inter-prediction coefficients.

[0162] Alternatively or additionally, some or all of the modes of a set of SPAR quantization modes may include selecting a subset of decorrelator coefficients to remove from the plurality of decorrelator coefficients.

[0163] Selecting a subset of coefficients may be based on a channel ranking of the Ambisonics channels.

[0164] The largest contributors to the metadata bitrate in the SPAR HOA are the prediction coefficients, because they are known to be very important for audio quality and therefore are typically chosen to be finely quantized, requiring more bits to be coded. The prediction coefficients are also expected to do a large amount of work in reconstructing the parametrized signal at the decoder.

[0165] Expected n dmx At = 4, the C coefficients are by far the most numerous. The C coefficients correspond to cross-prediction between the FOA residual signal (Y', X', Z') and all higher order channels, and therefore can be reduced at any time.

[0166] Setting the C coefficient to zero all the time has a large impact on audio quality. Doing this over several frames when metadata encoding is difficult has a greatly limited effect.

[0167] Setting a certain subset (or all) of the C coefficients to zero can be considered as a very coarse quantization of these parameters, which may lead to the energy of the reconstructed signal being too low. In this case, we may or may not allow the decorrelator coefficients to compensate for the inter-prediction "quantization error".

[0168] Given the bitrate constraints on the metadata, it may become evident that in the worst case, and in order to meet the bitrate requirements, it is also necessary to remove the corresponding decorrelator coefficients P. Further embodiments may include the option of coding the C coefficients but excluding a subset of the P coefficients, or, in cases where bitrate is extremely limited, to also remove the associated prediction coefficients PR.

[0169] The C coefficients may be specified by their correspondence with a particular first-order residual signal, a particular higher-order parametric channel, and a band. Any pattern of sparseness may be imposed on the C coefficients, as long as both the encoder and the decoder know which coefficients have been removed. The choice to remove a subset of C and / or P coefficients may be perceptually motivated, e.g., similar to the reasoning behind the channel ranking points in [4]. For full parametrization (i.e., transmitting PR, C and P coefficients), a higher-order planar channel may be preferable over a partially parametrized (i.e., transmitting PR without C and / or P) non-planar channel. The specified n dmx Given that, this preference need not be imposed by the ordering of the signals.

[0170] Planar HOA C and P coefficients A reasonable assumption for eliminating a large number of coefficients is that the higher order planar channels, e.g., {5,9}, {10,16}, are most relevant, while the higher order height-related channels, e.g., {6-8}, {11-15}, are less relevant. Removing the height-related channels for HOA3 reduces the inter-prediction and decorrelator coefficients by 2 / 3. Similarly, for HOA2, where channels {6-8} can be removed, it reduces by 3 / 5. Many other structures are possible, depending on the bitrate constraints. To reduce the required equivalent metadata bitrate (for HOA3) without removing coefficients, the quantization level needs to be reduced far below the "good quality" level determined from the FOA tuning. This is not appropriate for HOA modes.

[0171] For FOA inputs, there is typically three rounds of slowly degrading quantization levels. For HOA, a similar result can be obtained by lowering the quantization levels from the original to the second instance, then keeping the same quantization levels but deliberately excluding non-planar coefficients in the third instance.

[0172] Using the previous example of HOA3, which originally had a maximum metadata bitrate of 130 kbps with coarse quantization using only the plane C and P coefficients, we were able to reduce the maximum bitrate to about 84 kbps using more sophisticated quantization for the included metadata.

[0173] MD PLC for worst case frame From initial observations, it is not seen that metadata enters this planar mode very often. Only the worst vectors do so for about 1-2 frames per second, so the audio degradation is not particularly noticeable. However, if SPAR is forced to enter this lower quality mode more frequently, a Packet Loss Concealment (PLC)-like approach can be applied. This approach treats the unsent non-planar metadata as "lost" by using the last frame transmitted. In this case, the non-planar metadata is sent as a starting point for interpolation.

[0174] PLC - Packet Loss Concealment refers to an algorithm that allows the decoder to fill in the blanks and construct some meaningful output, this is usually done when an entire cache (packet) of information is lost, e.g. all audio and metadata for a particular frame, often due to network problems.

[0175] In this example, one might not lose all the audio / MD information, but only a subset of the MD. In a similar manner, it is possible to make some reasonable inference from information received in previous frames to fill in the blanks for this deliberately omitted metadata.

[0176] Compensate for the removed decorrelator coefficients in another way The decorrelator coefficients are used to match the energy of a parametrized channel to its input after prediction and / or cross prediction. It may be possible to compensate for the lost energy in the higher order channels selected to exclude the P coefficients. This may be done by adjusting the coefficients of the lower order associated channels selected to fully parametrize. For example, excluding the P coefficient for channel {12} may be compensated for by boosting the P coefficients of channels {6} and / or {2}, if present.

[0177] interpretation Aspects of the system described herein may be implemented in any suitable computer-based sound processing network environment that processes digital or digitized audio files. Part of an adaptive audio system may include one or more networks that include any desired number of individual machines. The individual machines include one or more routers (not shown) that serve as buffers and route data communicated between the computers. Such networks may be built on a variety of network protocols and may be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof.

[0178] One or more of the components, blocks, processors, or other functional components may be implemented via a computer program that controls the execution of a processor-based computing device of the system. It should also be noted that the various functions disclosed herein may be described in terms of their behavior, register transfers, logic components, and / or other characteristics using any number of combinations of hardware, firmware, and / or data and / or instructions embedded in various machine-readable or computer-readable media. Computer-readable media in which such formatted data and / or instructions may be embedded include, but are not limited to, physical (non-transitory) and non-volatile storage media in various forms, such as optical, magnetic, or semiconductor storage media.

[0179] A computing device that executes the techniques described above may have the following example architecture. Other architectures are possible, including architectures with more or fewer components. In some implementations, the example architecture includes one or more processors (e.g., a dual-core Intel® processor), one or more output devices (e.g., LCD), one or more network interfaces, one or more input devices (e.g., mouse, keyboard, touch display), and one or more computer-readable media (e.g., RAM, ROM, SDRAM, hard disk, optical disk, flash memory, etc.). These components may communicate and exchange data via one or more communication channels (e.g., buses). The communication channels may utilize various hardware and software to facilitate the exchange of data and control signals between the components.

[0180] The term "computer-readable medium" refers to any medium that participates in providing instructions to a processor for execution, and includes, but is not limited to, non-volatile media (e.g., optical or magnetic disks), volatile media (e.g., memory), and transmission media, including, but not limited to, coaxial cables, copper wire, and fiber optics.

[0181] The computer readable medium may further include an operating system (e.g., a Linux operating system), a network communications module, an audio interface manager, an audio processing manager, and a live content distributor. The operating system may be multi-user, multi-programming, multi-tasking, multi-threading, real-time, etc. The operating system performs basic tasks, including, but not limited to, recognizing input from and providing output to network interfaces and / or devices, tracking and managing files and directories on the computer readable medium (e.g., memory or storage devices), and managing traffic on one or more communications channels. The network communications module includes various components that establish and maintain network connections (e.g., software implementing communications protocols such as TCP / IP, HTTP, etc.).

[0182] The architecture may be implemented in a parallel processing or peer-to-peer infrastructure, or on a single device having one or more processors. The software may include multiple software components or may be a single piece of code.

[0183] The above features may be advantageously implemented in one or more computer programs executable on a programmable system including at least one programmable processor, at least one input device, and at least one output device, coupled to receive data and instructions from and communicate data and instructions to the data storage system. A computer program is a set of instructions that can be used directly or indirectly in a computer to perform a certain activity or bring about a certain result. A computer program may be written in any form of programming language, including compiled or interpreted languages ​​(e.g., Object-C, Java). A computer program may be deployed as a stand-alone program or in any form including modules, components, subroutines, browser-based web applications, or other units suitable for use in a computing environment.

[0184] Processors suitable for executing a program of instructions include, for example, general purpose or special purpose processors of any kind of computer, and a single processor or one of multiple processors or cores. Typically, a processor receives instructions and data from a read-only memory, a random access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Typically, a computer also includes, or is operatively connected to communicate with, one or more mass storage devices for storing data files. Such devices include magnetic disks, such as internal hard disks and removable disks, magneto-optical disks, and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor memory devices, such as EPROM, EEPROM, flash memory, magnetic disks, such as internal hard disks and removable disks, magneto-optical disks, CD-ROM disks, and DVD-ROM disks. The processor and memory may be supplemented by or incorporated in ASICs (Application Specific Integrated Circuits).

[0185] To provide for interaction with a user, features may be implemented in a computer having a display device, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor or retina display device, that displays information to the user. The computer may have a touch surface input device (e.g., a touch screen) or keyboard, and a pointing device, such as a mouse or trackball, that allows the user to provide input to the computer. The computer may have a voice input device that receives spoken commands from the user.

[0186] Features may be implemented in a computer system that includes a back-end computer such as a data server, or includes middleware components such as an application server or Internet server, or includes a front-end computer such as a client computer having a graphical user interface or Internet browser, or any combination thereof. The components of the system may be connected to each other by any form or medium of digital data communication, such as a communications network. Examples of communications networks include, for example, LANs, WANs, and the computers and networks forming the Internet.

[0187] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of clients and servers is established by computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server communicates data (e.g., HTML pages) to client devices (e.g., to display the data and receive user input from a user interacting with the client device). Data generated at the client device (e.g., a result of a user interaction) may be received from the client device by the server.

[0188] A system including one or more computers may be configured to perform particular operations by having software, firmware, hardware, or a combination of these installed on the system and performing the operations or causing the system to perform the operations. One or more computer programs may be configured to perform particular operations by containing instructions that, when executed by a data processing device, cause the device to perform the operations.

[0189] Although the present specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of what is claimed, but rather as describing features specific to certain embodiments of a particular invention. Certain features described in separate embodiments herein can also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment can also be implemented separately or in any suitable subcombination in multiple embodiments. Furthermore, although features may be described above as acting in certain combinations, and may also be so described in the initial claims, one or more features from the combinations described in the claims may in some cases be separated from the combination, and the combinations described in the claims may be directed to subcombinations or subcombination variations.

[0190] Similarly, although the figures show operations in a particular order, it should not be understood that such operations need to be performed in the particular order or sequence shown, or that all of the operations shown need to be performed, to achieve desired results. In certain environments, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components in the above embodiments should not be understood as required for all embodiments. It should be understood that the program components and systems described herein may generally be integrated in a single software product or packaged in multiple software products.

[0191] Unless otherwise indicated, and as will be apparent from the description that follows, throughout this disclosure, descriptions made using terms such as "processing," "computing," "calculating," "determining," "analyzing," and the like, are understood to refer to operations and / or processes of a computer or computing system, or similar electronic computing device, that manipulates and / or converts data expressed as physical quantities, such as quantities of electrons, into other data similarly expressed as physical quantities.

[0192] References throughout this disclosure to "one example embodiment," "example embodiments," or "example embodiments" mean that a particular feature, structure, or characteristic described in connection with that example embodiment is included in at least one example embodiment of the disclosure. Thus, the appearances of the phrases "in one example embodiment," "in example embodiments," or "in example embodiments" in various places throughout this disclosure do not necessarily all refer to the same example embodiment. Furthermore, particular features, structures, or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more example embodiments.

[0193] As used herein, unless otherwise specified, the use of ordinal numbers such as "first," "second," "third," etc. to describe a common object merely indicates that different instances of a similar object are being referenced, and is not intended to imply that the objects so described must be in a given sequence, whether temporally, spatially, in ranking order, or in any other manner.

[0194] Further, it is to be understood that the phraseology and terminology used herein is for purposes of description and should not be construed as limiting the present disclosure. The use of "containing," "including," or "having" and variations thereof is intended to cover the items listed thereafter as well as additional items. Unless otherwise noted, the terms "mounted," "connected," "supported," and "coupled" and variations thereof are used in the broadest sense to cover direct and indirect mounting, connecting, supporting, and coupling.

[0195] In the following claims and in the description of this specification, the terms "comprise", "comprises", or "comprises" are all open terms meaning at least the element / feature following the term, but not excluding other elements / features. Thus, the term "comprises", when used in the claims, should not be interpreted as being limited to the means or elements or steps listed thereafter. For example, the scope of an expression of a device comprising A and B should not be limited to a device consisting of only elements A and B. As used herein, the terms "containing" or "comprises" are also open terms meaning at least the element / feature following the term, but not excluding other elements / features. Thus, "comprises" is synonymous with "comprises" and means "comprises".

[0196] In the above description of example embodiments of the present disclosure, it should be understood that various features of the present disclosure may be grouped together in a single example embodiment, figure, or description thereof in order to simplify the disclosure and aid in understanding one or more of the various inventive aspects. However, this method of disclosure should not be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects reside in fewer than all features of a single disclosed example embodiment above. Thus, the claims following the "Description of the Invention" are expressly incorporated into the "Description of the Invention", with each claim standing on its own as a separate example embodiment of the present disclosure.

[0197] Moreover, although some example embodiments described herein include some features and not others that are included in other example embodiments, combinations of features of different example embodiments are meant to be within the scope of the disclosure and form different example embodiments, as would be understood by one of ordinary skill in the art. For example, in the following claims, any of the example embodiments claimed can be used in any combination.

[0198] In the description provided herein, numerous specific details are set forth. However, it will be understood that example embodiments of the present disclosure may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

[0199] Thus, while what is believed to be the best mode of the disclosure has been described, those skilled in the art will recognize that other and further modifications may be made thereto without departing from the spirit of the disclosure, and it is intended to claim all such variations and modifications as falling within the scope of the disclosure. For example, any formulas above are merely representative of procedures that may be used. Functions may be added or deleted from the block diagrams, and operations may be interchanged between functional blocks. Steps may be added or deleted to methods within the scope of the disclosure.

[0200] Example embodiments and implementations Various aspects and implementations of the present disclosure can also be understood from the following enumerated example descriptions (EEE). The enumerated example descriptions (EEE) are not claims.

[0201] Example embodiments may include a method for encoding audio, executed by one or more processors. The method may include receiving an HOA audio signal including more than four HOA channels, encoding the HOA audio signal into a waveform and metadata using SPAR quantization, and providing the encoded waveform and metadata to a downstream device, such as a decoder. Encoding the HOA audio signal may optionally include selecting a SPAR quantization mode based on a bitrate limit.

[0202] Example embodiments may include a method, executed by one or more processors, for decoding audio. The method may include receiving a bitstream, determining a SPAR quantization mode for the bitstream, and SPAR decoding the bitstream according to the quantization mode.

[0203] Example embodiments may include a method for encoding audio, executed by one or more processors. The method may include receiving an HOA audio signal having more than four HOA channels in native order, which may be ACN but other formats are possible. The method may further include reordering the channels based on perceptual importance, SPAR downmixing at least one first set of HOA channels with relatively high perceptual importance in a first representation and at least one second set of HOA channels with lesser importance in a second representation, and providing the SPAR downmixed channels to a downstream device, e.g., a decoder.

[0204] Optionally, for a given Ambisonics order, the planar HOA channels have a higher ordering priority than the non-planar HOA channels, such that the planar HOA channels are assigned to a first set and the non-planar HOA channels are assigned to a second set.

[0205] Optionally, at least two HOA channels have the same or equivalent position in the ordering. The first representation may be a waveform representation. The second representation includes a parameterization. In particular, the second representation includes a pruned parameterization. In a pruned parameterization, certain parameters are removed. In some implementations, a particular channel from a pair or group of equally positioned channels is dynamically selected for transmission.

[0206] Example embodiments may include a method for encoding audio and metadata, executed by one or more processors. The method may include obtaining bitrate limit values ​​for the audio and metadata and selecting a quantization mode appropriate for the bitrate limit. In various quantization modes, (a) all information in the audio and metadata, which may be a residual channel and all associated metadata, may be selected, (b) at least all information in the metadata, e.g., a parametric channel with all associated metadata, may be selected, and (c) at least some coefficients are removed, e.g., a parametric channel with some associated metadata is selected and a parametric channel with some associated metadata is removed. The method may include SPAR downmixing the audio according to the selected quantization mode and metadata.

[0207] In some implementations, the removed coefficients include inter-predicted coefficients. The method may include adapting at least one of the selected prediction coefficients, the inter-predicted coefficients, or the decorrelator coefficients to compensate for the removed coefficients.

[0208] Example embodiments may include a method for decoding audio, executed by one or more processors. The method may include receiving encoded audio data, e.g., metadata. The audio data may include a representation of a quantization mode in which spatial metadata is encoded. The audio data may include a bitstream, the bitstream including the coded spatial metadata. The coded spatial metadata includes, together with the audio bitstream, an indicator that the quantization mode was used.

[0209] The method may include determining a padding value based on a quantization mode, inserting the padding value in place of missing SPAR metadata corresponding to the particular quantization mode for decoding, and SPAR decoding the audio data based on the non-missing SPAR metadata and the padding value, where the padding value includes zero or is from metadata of a previous frame.

[0210] EEE1. 1. A method for encoding audio, comprising the steps of: receiving an HOA audio signal including four or more HOA channels; encoding the HOA audio signal into a waveform and metadata using SPAR; Providing the encoded waveform and metadata to a downstream device; The method includes:

[0211] EEE2. The method of claim 1, wherein encoding the HOA audio signal includes selecting a SPAR metadata quantization mode based on a bit rate limit.

[0212] EEE3. 1. A method of decoding audio, comprising: receiving a bitstream; determining a SPAR quantization mode for the bitstream; SPAR decoding the bitstream according to a quantization mode; The method includes:

[0213] EEE4. 1. A method for encoding audio, comprising the steps of: receiving an HOA audio signal having more than four HOA channels in native order; reordering the channels based on their perceptual importance; SPAR downmixing at least one first set of HOA channels of relatively high perceptual importance in a first representation and representing at least one second set of HOA channels of lesser perceptual importance in a second representation; providing the SPAR downmixed channel to a downstream device; The method includes:

[0214] EEE5. The method according to EEE4, wherein for a given Ambisonics order, the planar HOA channels have a higher ordering priority than the non-planar HOA channels, such that the planar HOA channels are assigned to a first set and the non-planar HOA channels are assigned to a second set.

[0215] EEE6. The method of EEE4 or EEE5, wherein at least two HOA channels have the same or equivalent position in the ordering.

[0216] EEE7. The method of any one of EEE4 to 6, wherein the first representation is a waveform representation.

[0217] EEE8. The method of any one of EEE4 to 7, wherein the second representation includes a parameterization.

[0218] EEE9. The method of any one of EEE4 to 8, wherein the second representation includes a pruned parameterization.

[0219] EEE10. The method of any one of EEE4 to 9, wherein the downstream device is a decoder.

[0220] EEE11. The method of any one of EEE4 to 10, wherein a particular channel from a pair or group of comparably positioned channels is dynamically selected for transmission.

[0221] EEE12. 1. A method for encoding audio and metadata, comprising the steps of: obtaining bit rate limits for audio and metadata; Selecting a quantization mode appropriate for the bit rate limit; SPAR downmixing the audio according to a selected quantization mode and metadata; Including, In various quantization modes, (a) all information in the audio and metadata is selected; (b) At least all of the information in the metadata is selected, or (c) A method in which at least some of the coefficients are eliminated.

[0222] EEE13. The method of EEE9, wherein the removed coefficients include inter-prediction coefficients.

[0223] EEE14. The method of any one of EEE12-13 comprising adapting at least one of the selected prediction coefficients, inter-prediction coefficients or decorrelator coefficients to compensate for the removed coefficients.

[0224] EEE15. The method of any one of EEE1 to 4, comprising computing a normalization term for a channel corresponding to a given Ambisonics order l using only the covariance estimates for the channel corresponding to order l in computing the one or more sets of coefficients in the SPAR metadata.

[0225] EEE16. The method of any one of EEE1 to 4, comprising computing one or more sets of coefficients in SPAR metadata for the parametric channel with a first temporal resolution t1 milliseconds that is longer than the second temporal resolution t2 milliseconds of the encoder filter bank.

[0226] EEE17. The method according to EEE16, wherein the one or more sets of coefficients in the SPAR metadata are computed at a second time resolution t2 milliseconds only for the high frequency band.

[0227] EEE18. The method according to EEE17, wherein the one or more sets of coefficients in the SPAR metadata are calculated with a second time resolution t2 milliseconds when a transient is detected.

[0228] EEE19. 1. A method for decoding audio data, comprising the steps of: Receiving encoded audio data including a representation of a quantization mode in which spatial metadata is encoded; determining a padding value based on a quantization mode; inserting padding values ​​in place of missing SPAR metadata corresponding to a particular quantization mode for decoding; SPAR decoding the audio data based on the non-missing SPAR metadata and the padding values; The method includes:

[0229] EEE20. The method according to EEE15, wherein the padding values ​​contain zeros or are derived from metadata of the previous frame.

[0230] EEE21. 1. A system comprising: one or more processors; a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the operations recited in any one of EEE1 to EEE16; Including, the system.

[0231] EEE22. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the operations recited in any one of EEE1 to EEE16.

Claims

1. 1. A method for encoding Higher Order Ambisonics (HOA) audio, comprising the steps of: receiving an input HOA audio signal having more than four Ambisonics channels; encoding the HOA audio signal using a SPAR coding framework and a core audio encoder; providing the encoded HOA audio signal, including the core encoded SPAR downmix channel and the encoded SPAR metadata, to a downstream device; The method includes:

2. The encoding may include encoding a representation of the W channel and a set of n ｔｏｔａｌ a prediction residual signal while calculating respective prediction coefficients in the SPAR metadata; ｔｏｔａｌ A subset of the prediction residual signal that is directly coded, n ｒｅｓ A plurality of n prediction residual signals are selected to be supplied to the downstream device. ｄｍｘ = n ｒｅｓ and obtaining a +1 downmix channel.

3. The n of the subset ｒｅｓ The method of claim 2 , wherein the selection of the prediction residual signal is based on a threshold for directly coded channels, which threshold indicates a maximum number of directly coded channels.

4. The method of claim 3 , wherein the threshold for the direct-coded channel is determined based on information indicative of one or more of a bit rate limit, a metadata size, a core codec capability, and an audio quality.

5. 5. The method of claim 3 or 4, wherein the direct-coded channel threshold is selected from a predetermined set of direct-coded channel thresholds.

6. The n of the subset ｒｅｓ The method according to claim 2 , wherein the predicted residual signal is selected according to a channel ranking of the Ambisonics channels from a high-ranked channel to a low-ranked channel.

7. The method of claim 6 , wherein the channel rankings of the Ambisonics channels are based on a perceptual importance of the Ambisonics channels, with Ambisonics channels having higher channel rankings having higher perceptual importance.

8. The method of claim 6 or 7, wherein the channel ranking of the Ambisonics channels is based on an encoder-decoder channel ranking agreement.

9. Spherical harmonics that have a relatively large overlap with the left-right and front-back planes at a given order l The Ambisonics channel corresponding to is a spherical harmonic function with a relatively large overlap with the height direction.

9. The method of claim 7 or 8, wherein the corresponding Ambisonics channel is ranked as having higher perceptual importance than the corresponding Ambisonics channel.

10. Spherical harmonics with relatively large overlap in the left and right directions The Ambisonics channel corresponding to 10. The method of claim 7, wherein the corresponding Ambisonics channel is ranked as having higher perceptual importance than the corresponding Ambisonics channel.

11. Spherical harmonic function of a given degree l for |m|=l 11. The method according to claim 7, wherein a pair formed by an Ambisonics channel corresponding to |m| is ranked as having higher perceptual importance than the HOA channel of the given order l for |m|<l.

12. Spherical harmonic function of a given degree l The channel ranking of the Ambisonics channel corresponding to the (l+1)th order spherical harmonic function 12. The method of claim 7, further comprising forming the channel rankings of a subset of the Ambisonics channels corresponding to a (l+1)th order Ambisonics channel starting from the channel ranking of the lth order Ambisonics channel.

13. Spherical harmonics with relatively large overlap in the left-right-front-back plane of a given order l The Ambisonics channel corresponding to the (l-1)th order spherical harmonic function with relatively large overlap in the height direction is 13. The method of claim 7, wherein the corresponding Ambisonics channel is ranked as having higher perceptual importance than the corresponding Ambisonics channel.

14. The n of the subset ｒｅｓ One or more prediction residual signals to be added later to the prediction residual signal are spherical harmonic functions Spherical harmonics function in front of the Ambisonics channel corresponding to Spherical harmonics functions rather than the corresponding Ambisonics channels 14. The method of claim 7, wherein m is selected based on a ranking that highly ranks Ambisonics channels corresponding to m, where 0<|m|<l.

15. The encoding process encodes each coefficient in the SPAR metadata as the remaining n ｄｅｃ = n ｔｏｔａｌ -n ｒｅｓ The method of claim 2 , further comprising representing a parametric channel based on computing from a prediction residual signal.

16. The computation in the SPAR metadata involves computing a number of inter-prediction coefficients for use by the decoder, ｒｅｓ From the directly coded prediction residual signal, ｄｅｃ The method of claim 15 , comprising reconstructing at least a portion of the parametric channels.

17. 17. The method of claim 16, wherein the operation within the SPAR metadata further comprises operating a plurality of decorrelator coefficients for use by the decoder to account for remaining energy not accounted for by the prediction coefficients and the inter-prediction coefficients during reconstruction.

18. The SPAR metadata is operated on by the second temporal resolution t ２ A first time resolution t that is longer than a millisecond １ 18. The method of claim 15, further comprising: computing at least one of the prediction coefficients, the inter-prediction coefficients, and the decorrelator coefficients in milliseconds.

19. The second time resolution t ２ 20. The method of claim 18, wherein the calculation in milliseconds is performed only for high frequency bands.

20. The second time resolution t ２ 20. The method of claim 19, wherein the calculation in milliseconds occurs when a transient is detected.

21. 21. The method of claim 15, wherein the computing in the SPAR metadata further comprises computing a normalization term for a channel corresponding to a given Ambisonics order l using only covariance estimates for the channel corresponding to order l.

22. 22. The method of claim 15, wherein the encoding further comprises obtaining a bit rate limit value, selecting a SPAR quantization mode from a set of SPAR quantization modes that matches the bit rate limit value, and applying the selected SPAR quantization mode to the SPAR metadata.

23. 23. The method of claim 22, wherein some or all of the set of SPAR quantization modes include reallocating bits from coefficients associated with Ambisonics channels that are relatively lower ranked in the channel ranking to coefficients associated with Ambisonics channels that are relatively higher ranked in the channel ranking.

24. 24. A method according to claim 22 or 23 when dependent on claim 16, wherein some or all of the set of SPAR quantisation modes comprises selecting a subset of inter-prediction coefficients to exclude from the plurality of inter-prediction coefficients.

25. 25. A method according to any one of claims 22 to 24 when dependent on claim 17, wherein some or all of the set of SPAR quantisation modes include selecting a subset of decorrelator coefficients to exclude from a plurality of decorrelator coefficients.

26. A method according to claim 22 or 25 when dependent on claim 6, wherein selecting the subset of coefficients is based on the channel rankings of the Ambisonics channels.

27. 27. The method of claim 7, wherein the received input HOA audio signal consists of Ambisonics channels that are ranked as having relatively high perceptual importance.

28. 1. A method for decoding Higher Order Ambisonics (HOA) audio, comprising: receiving an encoded HOA audio signal obtained by applying a SPAR coding framework and a core audio encoder to an input HOA audio signal having more than four Ambisonics channels; decoding the encoded HOA audio signal to obtain a decoded HOA audio signal including a core decoded SPAR downmix channel and decoded SPAR metadata; reconstructing the input HOA audio signal based on the decoded HOA audio signal to obtain an output HOA audio signal; The method includes:

29. The core decoded SPAR downmix channel is a representation of the W channel and a set of n ｒｅｓ and a direct coded prediction residual signal, wherein the decoded SPAR metadata includes a plurality of prediction coefficients, a plurality of inter-prediction coefficients, and a plurality of decorrelator coefficients.

30. Reconstructing the input HOA audio signal includes predicting the Ambisonics channels of a subset of the HOA audio signal based on the representation of the W channel and the plurality of prediction coefficients; ｒｅｓ and adding in a directly coded prediction residual signal.

31. Reconstructing the input HOA audio signal includes: ｒｅｓ 31. The method of claim 30, further comprising determining a residual parametric channel based on a directly coded prediction residual signal and the plurality of inter-prediction coefficients.

32. The method of claim 31, wherein reconstructing the input HOA audio signal further comprises calculating an indication of remaining energy not explained by the prediction coefficients and the plurality of inter-prediction coefficients based on the plurality of decorrelator coefficients and a plurality of decorrelated versions of the W channel.

33. 1. An apparatus for encoding Higher Order Ambisonics (HOA) audio, comprising: receiving an input HOA audio signal having more than four Ambisonics channels; encoding the HOA audio signal using a SPAR coding framework and a core audio encoder; providing the encoded HOA audio signal, including the core encoded SPAR downmix channel and the encoded SPAR metadata, to a downstream device; 23. An apparatus comprising: one or more processors configured to perform a method comprising:

34. 1. An apparatus for decoding Higher Order Ambisonics (HOA) audio, comprising: receiving an encoded HOA audio signal obtained by applying a SPAR coding framework and a core audio encoder to an input HOA audio signal having more than four Ambisonics channels; decoding the encoded HOA audio signal to obtain a decoded HOA audio signal including a core decoded SPAR downmix channel and decoded SPAR metadata; reconstructing the input HOA audio signal based on the decoded HOA audio signal to obtain an output HOA audio signal; 23. An apparatus comprising: one or more processors configured to perform a method comprising:

35. 33. An apparatus comprising a memory and one or more processors configured to perform the method of any one of claims 1 to 32.

36. 35. A system comprising an apparatus for encoding Higher Order Ambisonics (HOA) audio as claimed in claim 33 and an apparatus for decoding Higher Order Ambisonics (HOA) audio as claimed in claim 34.

37. A program comprising instructions which, when executed by a processor, cause the processor to perform a method according to any one of claims 1 to 32.

38. A computer readable storage medium storing the program according to claim 37.

Images