Decoding method and apparatus for compressed hoa representation of a sound or soundfield

Abstract

The present disclosure relates to a method and apparatus for decoding a compressed HOA representation of a sound or soundfield. The compressed HOA representation comprises a plurality of transport signals. The method comprises: allocating the plurality of transport signals to a plurality of hierarchical layers, the plurality of hierarchical layers comprising a base layer and one or more hierarchical enhancement layers; generating, for each layer, a respective HOA extension payload comprising side information for parametrically enhancing a reconstructed HOA representation obtainable from the transport signals allocated to the respective layer and any layers lower than the respective layer, allocating the generated HOA extension payloads to their respective layers, and signaling the generated HOA extension payloads in an output bitstream. The present disclosure further relates to a method of decoding a frame of a compressed HOA representation of a sound or soundfield, an encoder and a decoder for hierarchical encoding of a compressed HOA representation, and a data structure representing a frame of a compressed HOA representation of a sound or soundfield.

Description

[0001] This application is a divisional application of the invention patent application with application number 201680057989.7, application date October 7, 2016, entitled "Hierarchical coding and data structure for compressing high-order high-fidelity stereo sound reproduction of sound or sound field representation".

[0002] Cross-reference to related applications

[0003] This application claims priority to European Patent Application No. 15306653.5, filed on 15 October 2015, the entire contents of which are incorporated herein by reference. Technical Field

[0004] This disclosure relates to methods and apparatus for layered audio coding. Specifically, this disclosure relates to methods and apparatus for layered audio coding of frames that compress high-order high-fidelity stereo reproduction (HOA) sound (or sound field) representations. This disclosure further relates to data structures (e.g., bitstreams) for representing frames of compressed HOA sound (or sound field) representations. Background Technology

[0005] In the current definition of hierarchical HOA coding, side information is created for HOA decoding tools such as spatial signal prediction, sub-band directional signal synthesis, and parametric environment copy (PAR) decoders to enhance specific HOA representations. That is, in the current definition of hierarchical HOA coding, the provided data only appropriately extends the HOA representation of the highest layer (e.g., the highest enhancement layer). For lower layers, including the base layer, these tools do not appropriately enhance the partially reconstructed HOA representation.

[0006] The tool "Subband Directional Signal Synthesis and Parametric Environment Replication Decoder" is specifically designed for low data rates, where only a few transmit signals are available. However, in HOA layered coding, adequate enhancement of the (partially) reconstructed HOA representation is not possible, especially for low bit-rate layers (such as the base layer). From the perspective of sound quality at low bit rates, this is clearly undesirable.

[0007] Additionally, it has been found that if `CodedVVecLength` equal to 1 is signaled in `HOADecoderConfig()` (i.e., if the vector encoding mode is active), the conventional way of processing the encoded V vector elements of the vector-based signal does not result in proper decoding. In this vector encoding mode, V vector elements are not sent for HOA coefficient indices included in the `ContAddHoaCoeff` set. This set includes all HOA coefficient indices `AmbCoeffIdx[i]` with an `AmbCoeffTransitionState` equal to zero. Normally, no additional weighted V vector signal is needed because the original HOA coefficient sequence used for these indices is explicitly sent (signaled). Therefore, for these indices, the V vector elements are set to zero.

[0008] However, in layered coding mode, the set of consecutive HOA coefficient indices depends on the transmission channels, which are part of the currently operating layer. Additional HOA coefficient indices transmitted in higher layers may be missing in lower layers. Therefore, the assumption that vector signals should not contribute to the HOA coefficient sequence is incorrect for HOA coefficient indices belonging to HOA coefficient sequences included in higher layers.

[0009] Therefore, the V vector in layered HOA encoding may not be suitable for decoding any layer below the highest layer.

[0010] Therefore, a layered coding scheme and bitstream suitable for compressed HOA representation of sound or sound field are needed.

[0011] This disclosure addresses the above-mentioned problems. Specifically, it describes a method and encoder / decoder for layered coding of frames for compressing HOA sound or sound field representations, as well as a data structure for representing frames of compressed HOA sound or sound field representations. Summary of the Invention

[0012] According to one aspect, a layered coding method for frames of a compressed high-order high-fidelity stereo echo (HOA) representation of sound or sound field is described. The compressed HOA representation conforms to the draft MPEG-H 3D audio standard and any other future adopted or draft standards. The compressed HOA representation may include multiple transport signals. The transport signals may be associated with a monoaural signal, such as a coefficient sequence or dominant sound signal representing the HOA representation. The method may include assigning multiple transport signals to multiple hierarchical layers. For example, the transport signals may be distributed across multiple layers. The multiple layers may include a base layer and one or more hierarchical enhancement layers. The multiple hierarchical layers may be ordered from the base layer to the first enhancement layer, the second enhancement layer, and so on, up to the overall highest enhancement layer (overall highest layer). The method may further include generating a corresponding HOA extension payload for each layer, the HOA extension payload including auxiliary information (e.g., enhancement auxiliary information) for parametrically enhancing the reconstructed HOA representation obtainable from the transport signals assigned to the corresponding layer and any layer below that corresponding layer. The HOA representation used for reconstruction of lower layers may be referred to as a partially reconstructed HOA representation. This method may further include assigning the generated HOA extended payloads to their respective layers. The method may also further include marking the generated HOA extended payloads in the output bitstream. The HOA extended payloads can be marked in the HOAEnhFrame() payload. Therefore, auxiliary information can be moved from HOAFrame() to HOAEnhFrame().

[0013] As configured above, the proposed method applies layered coding to the compressed HOA representation (frames) to enable high-quality decoding even at low bit rates. Specifically, the proposed method ensures that each layer includes a suitable HOA extension payload (e.g., enhancement auxiliary information) to enhance the (partially) reconstructed audio representation obtained from the transmitted signal in any layer up to the current layer. Here, "up to the current layer" is understood to include, for example, a base layer, a first enhancement layer, a second enhancement layer, etc. For example, the decoder is enabled to enhance the (partially) reconstructed audio representation obtained from the base layer by referencing the HOA extension payload assigned to the base layer. In conventional methods, only the reconstructed HOA representation of the highest enhancement layer can be enhanced by the HOA extension payload. Therefore, regardless of the actual highest available layer (e.g., a layer below the lowest layer that has not yet been effectively received, such that all layers below the highest available layer and the highest available layer itself have been effectively received), the decoder will be enabled to improve or enhance the reconstructed audio representation, even if the (partially) reconstructed audio representation may differ from the complete (e.g., the entire) audio representation. Specifically, regardless of the actual highest available layer, it is sufficient for the decoder to decode only the HOA extended payload for a single layer (i.e., the highest available layer) to improve or enhance the (partially) reconstructed audio representation, which can be obtained based on all transmitted signals included in the layers up to the actual highest available layer. Decoding the HOA extended payload for higher or lower layers is not necessary. On the other hand, the proposed method allows full utilization of the required bandwidth reduction achievable when layered coding is applied.

[0014] In an embodiment, the method may further include transmitting data payloads with corresponding error protection levels for multiple layers. The data payloads may include corresponding HOA extended payloads. The base layer may have the highest error protection, and one or more enhancement layers may have progressively decreasing error protection. Thus, it can be ensured that at least several lower layers are reliably transmitted, while on the other hand, the overall required bandwidth is reduced by not applying excessive error protection to higher layers.

[0015] In one embodiment, the HOA extended payload may include bitstream elements for a HOA spatial signal prediction decoding tool. Alternatively, the HOA extended payload may include bitstream elements for a HOA sub-band direction signal synthesis decoding tool. Alternatively, the HOA extended payload may include bitstream elements for a HOA parameterized environment replication decoding tool.

[0016] In an embodiment, the HOA extended payload may have a usacExtElementType of ID_EXT_ELE_HOA_ENH_LAYER.

[0017] In an embodiment, the method may further include generating a HOA configuration extension payload, which includes bitstream elements for configuring the HOA spatial signal prediction decoding tool, the HOA subband direction signal synthesis decoding tool, and / or the HOA parameterized environment replication decoding tool. The HOA configuration extension payload may be included in HOADecoderEnhConfig(). The method may further include identifying the HOA configuration extension payload in the output bitstream.

[0018] In an embodiment, the method may further include generating a HOA decoder configuration payload, the HOA decoder configuration payload including information indicating the allocation of HOA extended payloads to multiple layers. The method may further include identifying the HOA decoder configuration payload in the output bitstream.

[0019] In an embodiment, the method may further include determining whether a vector coding mode is active. The method may further include, if the vector coding mode is active, determining a set of consecutive HOA coefficient indices for each layer based on the transmission signals assigned to the corresponding layer. The HOA coefficient indices in the set of consecutive HOA coefficient indices may be HOA coefficient indices included in the set ContAddHOACoeff. The method may further include generating a V vector for each transmission signal based on the set of consecutive HOA coefficient indices determined for the layer to which the corresponding transmission signal was assigned, such that the generated V vector includes elements for any transmission signals assigned to layers higher than the layer to which the corresponding transmission signal was assigned. The method may further include indicating the generated V vector in the output bitstream.

[0020] According to another aspect, a hierarchical coding method for frames of a compressed high-fidelity stereo echo (HOA) representation of a sound or sound field is described. The compressed HOA representation may include multiple transmission signals. The transmission signals may be associated with monoaural signals, such as representing a sequence of coefficients or a dominant sound signal in the HOA representation. The method may include assigning multiple transmission signals to multiple hierarchical layers. For example, the transmission signals may be distributed across multiple layers. These multiple layers may include a base layer and one or more hierarchical enhancement layers. The method may further include determining whether a vector coding mode is in effect. The method may further include, if the vector coding mode is in effect, determining a set of consecutive HOA coefficient indices for each layer based on the transmission signals assigned to the corresponding layer. The HOA coefficient indices in the set of consecutive HOA coefficient indices may be HOA coefficient indices included in the set ContAddHOACoeff. The method may further include generating a V vector for each transmission signal based on the set of consecutive HOA coefficient indices determined for the layer to which the corresponding transmission signal was assigned, such that the generated V vector includes elements for any transmission signals assigned to layers higher than the layer to which the corresponding transmission signal was assigned. The method may further include indicating the generated V vector in the output bitstream.

[0021] With this configuration, the proposed method ensures that, in vector coding mode, the appropriate V vector is available for every transmitted signal belonging to the layer up to the highest available layer. Specifically, the proposed method eliminates cases where elements of the V vector corresponding to the transmitted signal in a higher layer are not explicitly labeled. Therefore, the information included in the layers up to the highest available layer is sufficient for decoding any transmitted signal belonging to that layer. Consequently, even if a higher layer may not yet be effectively received by the decoder, the HOA representation for the corresponding reconstruction of the lower layer (low bit rate layer) is properly decompressed. On the other hand, the proposed method allows full utilization of the bandwidth reduction that can be achieved when layered coding is applied.

[0022] According to another aspect, a method for decoding frames of a compressed high-order high-fidelity stereo reproduction (HOA) representation of a sound or sound field is described. The compressed HOA representation can be encoded in multiple hierarchical layers. The multiple hierarchical layers may include a base layer and one or more hierarchical enhancement layers. The method may include receiving a bitstream associated with a frame of the compressed HOA representation. The method may further include extracting payloads for the multiple layers. Each payload may include a transmission signal assigned to a corresponding layer. The method may further include determining the highest available layer for decoding among the multiple layers. The method may further include extracting an HOA extended payload assigned to the highest available layer. The HOA extended payload may include auxiliary information for parametrically enhancing the (partially) reconstructed HOA representation corresponding to the highest available layer. The (partially) reconstructed HOA representation corresponding to the highest available layer can be obtained based on transmission signals assigned to the highest available layer and any layers below the highest available layer. The method may further include generating the (partially) reconstructed HOA representation corresponding to the highest available layer based on transmission signals assigned to the highest available layer and any layers below the highest available layer. The method may further include using auxiliary information included in the HOA extended payload allocated to the highest available layer to enhance (e.g., parametrically enhance) the (partially) reconstructed HOA representation. As a result, an enhanced reconstructed HOA representation can be obtained.

[0023] With this configuration, by making the most of available (e.g., effectively received) information, the proposed method ensures that the final (e.g., enhanced) reconstructed HOA representation has the best quality.

[0024] In one embodiment, the HOA extended payload may include bitstream elements for a HOA spatial signal prediction decoding tool. Alternatively, the HOA extended payload may include bitstream elements for a HOA sub-band direction signal synthesis decoding tool. Alternatively, the HOA extended payload may include bitstream elements for a HOA parameterized environment replication decoding tool.

[0025] In an embodiment, the HOA extended payload may have a usacExtElementType of ID_EXT_ELE_HOA_ENH_LAYER.

[0026] In an embodiment, the method may further include extracting the HOA configuration extension payload by parsing the bitstream. The HOA configuration extension payload may include bitstream elements for configuring the HOA spatial signal prediction decoding tool, the HOA sub-band direction signal synthesis decoding tool, and / or the HOA parameterized environment replication decoding tool.

[0027] In an embodiment, the method may further include extracting HOA extended payloads allocated to multiple layers respectively. Each HOA extended payload may include auxiliary information for parametrically enhancing the (partially) reconstructed HOA representation corresponding to its corresponding allocated layer. The (partially) reconstructed HOA representation corresponding to its corresponding allocated layer can be obtained from the transmission signals allocated to that layer and any layers below it. The allocation of the HOA extended payload to the corresponding layer can be known from configuration information included in the bit stream.

[0028] In an embodiment, determining the highest available layer may involve identifying a set of invalid layer indices that indicate layers that have not yet been effectively received. It may further involve determining the highest available layer as the layer below the layer indicated by the smallest (lowest) index in the set of invalid layer indices. The base layer may have the lowest layer index (e.g., layer index 1), and hierarchical enhancement layers may have progressively higher layer indices. Thus, the proposed method ensures that the highest available layer is selected in a manner where all the information required to decode the (partially) reconstructed HOA representation from the highest available layer and any layers below it is available.

[0029] In an embodiment, determining the highest available layer may involve determining a set of invalid layer indices indicating layers that have not yet been effectively received. It may further involve determining the highest available layer of the previous frame preceding the current frame. It may also further involve determining the highest available layer as the lower of the following layers: the highest available layer of the previous frame, and a layer that is lower than the layer indicated by the smallest index in the set of invalid layer indices. Thus, even if the current frame has been encoded differently relative to the previous frame, the highest available layer for the current frame is selected in a manner that all the information required to decode the (partially) reconstructed HOA representation from the highest available layer and layers lower than the highest available layer is available.

[0030] In an embodiment, the method may further include: if the highest available layer of the current frame is lower than the highest available layer of the previous frame, and if the current frame has been encoded differently relative to the previous frame, then deciding not to use the auxiliary information included in the HOA extended payload allocated to the highest available layer to perform parameterized enhancement of the (partially) reconstructed HOA representation. Thus, in cases where the current frame (including the auxiliary information included in the HOA extended payload allocated to the highest available layer) has been encoded differently relative to the previous frame, the reconstructed HOA representation can be decoded error-free.

[0031] In this embodiment, the set of invalid layer indexes can be determined by evaluating the validity flag of the corresponding HOA extended payload. If the validity flag for the HOA extended payload assigned to the corresponding layer is not set, the layer index of the given layer can be added to the set of invalid layer indexes. Thus, the set of invalid layer indexes can be determined in an efficient manner.

[0032] According to another aspect, a data structure (e.g., a bitstream) is described for a frame representing a compressed high-fidelity stereo reproduction (HOA) representation of sound or a sound field. The compressed HOA representation may include multiple transmission signals. The data structure may include multiple HOA frame payloads corresponding to respective hierarchical layers in a plurality of hierarchical layers. The HOA frame payloads may include corresponding transmission signals. The multiple transmission signals may be assigned to (e.g., distributed to) multiple layers. The multiple layers may include a base layer and one or more hierarchical enhancement layers. The data structure may further include a corresponding HOA extension payload for each layer, which includes auxiliary information for parametrically enhancing the (partially) reconstructed HOA representation obtainable from the transmission signals assigned to the corresponding layer and any layers below the corresponding layer.

[0033] In this embodiment, the HOA frame payload and HOA extended payload for multiple layers may have corresponding error protection levels. The base layer may have the highest error protection, and one or more enhancement layers may have progressively decreasing error protection.

[0034] In one embodiment, the HOA extended payload may include bitstream elements for a HOA spatial signal prediction decoding tool. Alternatively, the HOA extended payload may include bitstream elements for a HOA sub-band direction signal synthesis decoding tool. Alternatively, the HOA extended payload may include bitstream elements for a HOA parameterized environment replication decoding tool.

[0035] In an embodiment, the HOA extended payload may have a usacExtElementType of ID_EXT_ELE_HOA_ENH_LAYER.

[0036] In an embodiment, the data structure may further include a HOA configuration extension payload, which includes bitstream elements for configuring the HOA spatial signal prediction decoding tool, the HOA sub-band direction signal synthesis decoding tool, and / or the HOA parameterized environment replication decoding tool.

[0037] In an embodiment, the data structure may further include a HOA decoder configuration payload, which includes information indicating the allocation of HOA extended payloads to multiple layers.

[0038] In embodiments, the method and apparatus relate to decoding a compressed high-order high-fidelity stereo reproduction (HOA) representation of a sound or sound field. The apparatus may be configured, or the method may include: receiving a bitstream containing compressed HOA representations corresponding to a plurality of hierarchical layers, the plurality of hierarchical layers including a base layer and one or more hierarchical enhancement layers, wherein the plurality of layers are assigned components of a basic compressed sound representation of a sound or sound field, these components being assigned to corresponding layers in corresponding component groups; determining the highest available layer for decoding among the plurality of layers; extracting an HOA extended payload assigned to the highest available layer, wherein the HOA extended payload includes auxiliary information for parametrically enhancing the reconstructed HOA representation corresponding to the highest available layer, wherein the reconstructed HOA representation corresponding to the highest available layer may be obtained based on transmission signals assigned to the highest available layer and any layers below the highest available layer; decoding the compressed HOA representation corresponding to the highest available layer based on layer information, transmission signals assigned to the highest available layer and any layers below the highest available layer; and parametrically enhancing the decoded HOA representation using the auxiliary information included in the HOA extended payload assigned to the highest available layer.

[0039] The HOA extended payload can include bitstream elements for HOA spatial signal prediction decoding tools. Layer information can indicate the number of action direction signals in the current frame of the enhancement layer.

[0040] Layer information can indicate the total number of additional environment HOA coefficients used for enhancement layers. Layer information can include the HOA coefficient index for each additional environment HOA coefficient used for enhancement layers. Layer information can include enhancement information, which includes at least one of spatial signal prediction, sub-band direction signal synthesis, and parameterized environment copy decoder. If CodedVVechLength equal to 1 is indicated in HOADecoderConfig(), then the compressed HOA representation is suitable for a hierarchical coding mode for HOA-based content. Furthermore, for indices equal to the indices of the additional HOA coefficients included in the ContAddHoaCoeff set, the v vector element may not be sent. The ContAddHoaCoeff set can be defined separately for each of multiple hierarchical layers. Layer information includes NumLayers elements, where each element indicates the number of transmitted signals included in all layers up to the i-th layer. Layer information can include indicators of all layers actually used in the k-th frame. Layer information can also indicate that all coefficients used for the dominant vector are specified. Layer information can indicate that coefficients for the dominant vector corresponding to values ​​greater than MinNumOfCoeffsForAmbHOA are specified. Layer information can indicate that all elements defined in MinNumOfCoeffsForAmbHOA and ContAddHoaCoeff[lay] are not sent, where lay is the index of the layer containing the vector-based signal corresponding to the vector.

[0041] According to another aspect, an encoder is described for layered encoding of frames of a compressed high-order high-fidelity stereo echo (HOA) representation of sound or a sound field. The compressed HOA representation may include multiple transmission signals. The encoder may include a processor configured to perform some or all of the method steps according to the first and second aspects described above.

[0042] According to another aspect, a decoder is described for decoding frames of a compressed high-order high-fidelity stereo echo (HOA) representation of sound or a sound field. The compressed HOA representation may be encoded in multiple hierarchical layers, including a base layer and one or more hierarchical enhancement layers. The decoder may include a processor configured to perform some or all of the method steps according to the third aspect above.

[0043] According to another aspect, a software program is described. This software program is adapted to execute on a processor and, when executed on a computing device, to perform some or all of the method steps outlined in this disclosure.

[0044] According to another aspect, a storage medium is described. This storage medium may include a software program adapted to execute on a processor and, when executed on a computing device, to perform some or all of the method steps outlined in this disclosure.

[0045] As those skilled in the art will recognize, statements made with respect to any of the above aspects or embodiments thereof also apply to the other corresponding aspects or embodiments thereof. For the sake of brevity, repetition of these statements for each aspect or embodiment has been omitted.

[0046] It should be noted that the methods and apparatuses, including their preferred embodiments, as outlined in this disclosure can be used independently or in combination with other methods and systems disclosed in this disclosure. Furthermore, all aspects of the methods and apparatuses outlined in this disclosure can be combined arbitrarily. Specifically, the features of the claims can be combined with each other in any manner.

[0047] It should be further noted that the method steps and apparatus features can be interchanged in many ways. Specifically, as those skilled in the art will recognize, the details of the disclosed method can be implemented as apparatus suitable for performing some or all of the steps of the method, and vice versa. Attached Figure Description

[0048] The invention will now be described by way of example with reference to the accompanying drawings, wherein:

[0049] Figure 1 This is a block diagram schematically showing the payload allocation at the encoder end for the basic layer and M-1 enhancement layers;

[0050] Figure 2 This is a block diagram schematically illustrating an example of a receiver and a decompression stage;

[0051] Figure 3 This is a flowchart illustrating an example of a layered coding method for frames in compressed HOA representation according to an embodiment of the present disclosure;

[0052] Figure 4 This is a flowchart illustrating another example of a layered coding method for frames in compressed HOA representation according to embodiments of the present disclosure;

[0053] Figure 5 This is a flowchart illustrating an example of a method for decoding a frame represented by a compressed HOA according to an embodiment of the present disclosure;

[0054] Figure 6 This is a block diagram schematically illustrating an example of a hardware implementation of an encoder according to an embodiment of the present disclosure; and

[0055] Figure 7This is a block diagram schematically illustrating an example of a hardware implementation of a decoder according to an embodiment of the present disclosure. Detailed Implementation

[0056] First, a compressed sound (or sound field) representation that can be applied according to the method and encoder / decoder of this disclosure will be described.

[0057] In order to stream compressed sound (or sound field) representations over transmission channels with time-varying conditions, layered coding is a means of adapting the quality of the received sound representation to the transmission conditions, and in particular, avoiding dropout of unwanted signals.

[0058] For layered coding, the compressed audio (or sound field) representation is typically subdivided into a high-priority base layer with a relatively small size and additional enhancement layers with decreasing priority and arbitrary size. Each enhancement layer is generally assumed to contain incremental information that complements all lower layers to improve the quality of the compressed audio (or sound field) representation. The concept then involves controlling the amount of error protection used for transmission of these layers based on their respective priorities. Specifically, the base layer is given high error protection, which is reasonable and affordable due to its low size.

[0059] The following assumes that a complete compressed sound (or sound field) generally includes the following three components:

[0060] 1. A basic compressed sound (or sound field) representation that includes several supplementary components, which constitute the largest distinguishing percentage of the complete compressed sound (or sound field) representation.

[0061] 2. Basic auxiliary information required for decoding the basic compressed sound representation, which is assumed to be much smaller than the basic compressed sound (or sound field) representation. It is further assumed to consist primarily of two components that specify the decompression of only one particular component of the basic compressed sound representation:

[0062] a) The first component contains auxiliary information that describes the individual supplementary components of the basic compressed sound (or sound field) representation independently of the other supplementary components.

[0063] (b) The second (optional) component contains auxiliary information that depends on the other supplementary components describing the basic compressed sound (or sound field) representation. Specifically, the dependency has the following properties:

[0064] • The relevant auxiliary information for each individual supplementary component used in the basic compressed sound (or sound field) representation is maximized without any other supplementary components being included in the basic compressed sound (or sound field) representation.

[0065] • When certain additional supplementary components are added to the basic compressed sound (or sound field) representation, the relevant auxiliary information for the individual supplementary component under consideration becomes a subset of the original relevant auxiliary information, thereby reducing its size.

[0066] 3. Optional enhancements to the basic compressed sound (or sound field) representation. These enhancements are also assumed to be much smaller than the basic compressed sound (or sound field) representation.

[0067] A prominent example of this type of complete compressed sound (or sound field) representation is given by the compressed HOA sound field representation specified in the initial version of the MPEG-H 3D audio standard.

[0068] 1. Its basic compressed sound field representation can be identified by several quantized monoaural signals, which represent the coefficient sequence of the so-called dominant sound signal or the so-called ambient HOA sound field component.

[0069] 2. The basic auxiliary information, among other things, describes how each of these monoaural signals contributes spatially to the sound field. This information can be further divided into two distinct components:

[0070] (a) Auxiliary information associated with each specific monoaural signal, which is independent of the existence of other monoaural signals. Such auxiliary information may, for example, specify a monoaural signal representing a directional signal (meaning a plane wave in general) with a certain incident direction. Alternatively, the monoaural signal may be specified as a sequence of coefficients of the original HOA representation with a certain index.

[0071] (b) Auxiliary information associated with specific individual monoaural signals, which depends on the presence of other monoaural signals. For example, such auxiliary information arises if the monoaural signals are designated as so-called vector-based signals (meaning they are oriented within the sound field, where the directional distribution is specified by means of a vector). In a certain mode (i.e., CodedVVecLength = 1), specific components of the vector are implicitly set to zero and are not part of the compressed vector representation. These components are those that are part of the basic compressed sound field representation and have indices equal to the indices of the coefficient sequences in the original HOA representation. This means that if the individual components of the vector are encoded, their total number depends on the basic compressed sound field representation, and in particular on which coefficient sequences of the original HOA representation it contains.

[0072] If none of the coefficient sequences of the original HOA representation are included in the basic compressed sound field representation, then the dependent basic auxiliary information for each vector-based signal includes all vector components and has its maximum size. When the coefficient sequences of the original HOA representation with certain indices are added to the basic compressed sound field representation, the vector components with those indices are removed from the auxiliary information for each vector-based signal, thereby reducing the size of the dependent basic auxiliary information for the vector-based signal.

[0073] 3. Enhanced auxiliary information includes the following components:

[0074] • Parameters related to so-called (broadband) spatial prediction, which is used to (linearly) predict the missing part of the sound field from the directional signal.

[0075] The parameters related to so-called subband directional signal synthesis and parameterized environment replication are compression tools that allow frequency-dependent parameterized predictions of additional monocular signals to be spatially distributed in order to supplement the currently spatially incomplete or defective compressed HOA representation. This prediction is based on a sequence of coefficients from the basic compressed sound field representation. Importantly, the aforementioned supplementary contribution to the sound field within the compressed HOA representation is represented not by an additional quantization signal, but by a relatively small amount of additional auxiliary information. Therefore, the two mentioned coding tools are particularly suitable for compressing HOA representations at low data rates.

[0076] A second example of a compressed representation of a monoaural signal with the above structure may include the following components:

[0077] 1. Encoded spectral information for a series of disjoint frequency bands up to a certain upper frequency, which can be considered as a basic compressed representation.

[0078] 2. (In accordance with, for example, the number and width of the encoded frequency bands) Specify some basic auxiliary information for the encoded spectral information.

[0079] 3. Includes some enhanced auxiliary information on the parameters of so-called band replication (SBR), which describes how to parameterize the spectral information of higher frequency bands that were not considered in the basic compressed representation based on the basic compressed representation.

[0080] Next, a layered coding method for a complete compressed sound (or sound field) representation with the above structure will be described.

[0081] Compression is assumed to be frame-based in the sense that it provides a compressed representation (e.g., in the form of packets or equivalent frame payloads) over consecutive time intervals (e.g., time intervals of equal size). These packets are assumed to contain validity flags (values ​​indicating their size) along with the actual compressed representation data. Throughout the following description, the focus will be primarily on the processing of individual frames; therefore, frame indexes will be omitted.

[0082] The payload of each frame of the complete compressed sound (or sound field) representation 1100 is assumed to contain J data packets, each packet representing a component 1110-1,…,1110-J of the basic compressed sound (or sound field) representation, which are represented by BSRC. j Let j = 1, ..., J be the denoting integers. Furthermore, it is assumed to contain elements using BSI. I The packet represents a component with independent basic auxiliary information 1120, which independently specifies a specific component of the basic compressed sound representation BSRC, separate from other components. j Optionally, it is also assumed to include BSI. D The representation includes packets that depend on basic auxiliary information, which in turn depend on other components that specify a particular component of the basic compressed sound representation, BSRC. j Two data packets BSI I and BSI D The information contained therein can optionally be grouped into a single data packet BSI.

[0083] Finally, it includes an enhanced auxiliary information payload, represented by ESI, which describes how to improve the reconstruction of the sound (or sound field) from the complete basic compressed representation.

[0084] The described layered coding scheme addresses the steps required to implement both the compression (including packet packing for transmission) and the receiver and decompression components. Each component will be described in detail below.

[0085] Next, the compression and packing used for transmission will be described. In the case of layered encoding (assuming a total of M layers, i.e., one base layer and M-1 enhancement layers), each component of the complete compressed sound (or sound field) representation 1100 is processed as follows:

[0086] • Subdivide the basic compressed sound (or sound field) representation into parts to be assigned to each layer. Without loss of generality, grouping can be done using M+1 numbers J. m ,m=0,…,m(where J0=1 and J M =J+1) is used to describe this, so that for J m-1 ≤j <J mBSRC j It is assigned to the m-th layer.

[0087] Due to its small size, it is reasonable to allocate complete basic auxiliary information to the basic layer to avoid unnecessary fragmentation. Although the independent basic auxiliary information (BSI)... I While the allocation remains unchanged, the dependent basic auxiliary information needs special processing for hierarchical coding to allow for correct decoding at the receiver while reducing the size of the relevant auxiliary information to be transmitted. This is proposed by decomposing it using BSI. D,m The m = 1, ..., M represents M parts 1130-1, ..., 1130-M, where the m-th part contains the BSRC component used to assign to the basic compressed sound representation of the m-th layer. j J m-1 ≤j <J m Each of the relevant auxiliary information in the BSI, if the corresponding relevant auxiliary information exists. If the corresponding relevant auxiliary information does not exist, BSI... D,m It is assumed to be empty. Auxiliary Information BSI D,m Depends on all components contained in all layers up to the m-th layer BSRC j ,1≤j <J m .

[0088] In the case of layered encoding, it is important to recognize the need to additionally compute enhancement auxiliary information for each layer, as its intention is to enhance the sound (or sound field) during initial decompression; however, this depends on the layers available for decompression. Therefore, compression needs to provide enhancement auxiliary information using ESI. m The m = 1, ..., M represents M individual enhanced auxiliary information data packets 1140-1, ..., 1140-M, where the ESI of the m-th data packet is calculated. m Enhancement information, such as sound (or sound field) representations, is obtained from all data contained in the base layer and enhancement layers with indices below m.

[0089] In summary, at the compression level, frame data packets represented by FRAME must be provided, containing the following components:

[0090] FRAME = [BSRC1…BSRC J BSI I BSI D,1 …BSI D,M ESI1…ESI M (1)

[0091] It is understood that the order of the payloads in a frame data packet is generally arbitrary.

[0092] The already described allocation of each payload to the base and enhancement layers is achieved through so-called transport layer packetizers, and... Figure 1 It is shown schematically in the middle.

[0093] Next, the process of receiving and decompressing will be described. Figure 2 The corresponding receiver and decompression stage are shown in the diagram.

[0094] First, the individual data packets 1200, 1300-1, ..., 1300-(M-1) are multiplexed to provide a complete compressed audio (or sound field) representation of the received frame packets:

[0095]

[0096] The frame packet is then passed to decompressor 2100. Assuming that the transmission of a single layer is error-free, the validity flag of at least the included enhancement auxiliary information payload is set to "true". In the event of an error due to the transmission of a single layer, the validity flag within at least the enhancement auxiliary information payload of that layer is set to "false". Therefore, the validity of the layer packet can be determined from the validity of the included enhancement auxiliary information payload.

[0097] In the decompressor 2100, the received frame packets are first demultiplexed. For this purpose, information about the size of each payload can be used to avoid unnecessarily parsing the data of each payload.

[0098] In the next step, the number N of the highest layer of decompression that will actually be used for the basic sound representation is selected. B The highest enhancement layer actually used for decompression in basic sound representation consists of N. B -1 is given. Because each layer contains exactly one enhancement auxiliary information payload, it is known from each enhancement auxiliary information payload whether the containing layer is effective. Therefore, all enhancement auxiliary information payloads can be used for ESI. m The selection is achieved by setting m = 1, ..., M. Additionally, the index N of the enhanced auxiliary information payload to be used for decompression is determined. E The index is always equal to N. B Or equal to zero. This means that enhancements are either always implemented based on the basic sound representation, or not implemented at all. A more detailed description of the choice is given below.

[0099] Continuously, the basic compressed sound representation components BSRC1,…,BSRC J The payload and all basic auxiliary information payloads (i.e., BSI) I and BSI D,m (m=1,…,M) and sum NB Together they are passed to the basic representation decompression processing unit 2200, which uses only the lowest N. B Layers (i.e., the basic layer and N) B The basic compressed sound representation components contained within the -1 enhancement layer are used to reconstruct the basic sound (or sound field) representation. The necessary information regarding which components of the basic compressed sound (or sound field) representation are included in each layer is assumed to be known to the decompressor 2100 from a data packet with configuration information, which is assumed to be sent and received before the frame data packet. Each individual dependent basic auxiliary information payload (BSI) D,m m=1,…,N B The actual decoding can be divided into two parts as follows:

[0100] 1. By utilizing each payload BSI D,m m=1,…,N B For the head J contained in the head m layers m -1 basic compressed sound representation component The dependencies are assumed at the encoding level to perform initial decoding of the payload.

[0101] 2. By taking into account the basic sound components, it is finally from the beginning N. B >The headers contained in m layers The basic compressed sound components Reconstruction of each payload BSI D,m m=1,…,N B Continuous correction is performed on these basic compressed sound representation components, which are more components than those taken for the initial decoding. Therefore, correction can be achieved by discarding outdated information, which is possible due to the initial assumption that depends on basic auxiliary information (i.e., if some supplementary components are added to the basic compressed sound (or sound field) representation, the basic auxiliary information dependent on each individual supplementary component becomes a subset of the original basic auxiliary information dependent on basic auxiliary information).

[0102] Finally, the reconstructed basic sound (or sound field) representation is combined with all the enhanced auxiliary information payloads ESI1,…,ESI. M Basic Auxiliary Information Payload (BSI) I and BSI D,m m = 1, ..., M and the value N E Together, they are provided to the enhanced representation decompression processing unit 2300, which uses only the enhanced auxiliary information payload. And discard all other enhancement auxiliary information payloads to compute the final enhanced sound (or sound field) representation. If N E If the value is zero, all augmentation payload information is discarded, and the final reconstructed augmented sound (or sound field) representation is equal to the reconstructed basic sound (or sound field) representation.

[0103] Next, the description layer is selected. Assuming all frame data packets can be decompressed independently of each other, the highest layer numbered N, which will actually be used for decompression of the basic sound representation, is selected. B and the index N of the enhanced auxiliary information payload to be used for decompression E These two are set as the highest number L for valid augmentation and auxiliary information payloads, which can be determined by evaluating the validity flags within the augmentation and auxiliary information payloads. By utilizing knowledge of the size of each augmentation and auxiliary information payload, complex analysis of their actual data can be avoided to determine the validity of the payloads.

[0104] When employing differential compression with inter-frame dependency, additional decisions regarding the data derived from previous frames need to be considered. In differential compression, independent frame packets are sent at regular time intervals to allow decompression to begin from these points, where the value N... B and N E The determination becomes frame-independent and is performed as described above.

[0105] To explain the frame dependency decision in detail, we first consider the k-th frame.

[0106] Let L(k) denote the highest number of the effective augmentation and auxiliary information payload.

[0107] ·Use N B (k) represents the highest layer number to be selected and used for decompression of the basic sound representation.

[0108] ·Use N E (k) indicates the number of the enhanced auxiliary information payload to be used for decompression.

[0109] By using this notation, the following equation can be used to calculate the result of N. B (k) represents the highest layer number to be used for decompression in the basic sound representation:

[0110] N B (k)=min(N B (k-1),L(k)) (3)

[0111] By selecting no greater than N B N of (k-1) and L(k) B(k) ensures that all the information required for differential compression of the basic sound representation is available.

[0112] The number N of the enhanced auxiliary information payload to be used for decompression is determined according to the following equation. E (k):

[0113]

[0114] This specifically means that, as long as the highest layer number N to be used for decompression of the basic sound representation... B (k) remains unchanged, and the same corresponding enhancement layer number is selected. However, in N B When (k) changes, by changing N E Setting (k) to zero disables enhancement. Since differential compression is used on the enhancement auxiliary information, it is based on N. B (k) Change is not possible because it would require decompressing the corresponding augmentation auxiliary information layer at the previous frame, which is assumed to have not yet been performed.

[0115] Alternatively, if decompression has up to N E All the enhanced auxiliary information payloads numbered (k) are decompressed in parallel, and the selection rule (4) can be replaced by the following equation:

[0116] N E (k)=N B (k) (5)

[0117] Finally, it should be noted that for differential decompression, the number of the highest used layer can only be increased at individual frame data packets, while it is possible to decrease at each frame.

[0118] Next, embodiments of the hierarchical coding of frames involving compressed audio representations and the data structures (e.g., bitstreams) representing the encoded compressed audio representations of the present disclosure will be described with regard to compressed HOA representations. Specifically, changes to the proposed hierarchical coding scheme for compressed HOA representations will be described.

[0119] As a correction to the layered coding mode used for HOA-based content, a new usacExtElementType is defined to better adapt the configuration and frame payload of the HOA decoding tools spatial signal prediction, subband direction signal synthesis, and parameterized environment copy (PAR) decoders to the corresponding HOA enhancement layers. If the layered coding mode for HOA-based content is activated (indicated by SingleLayer == 0), it is proposed to move the corresponding bitstream elements of these tools to an additional HOA extension payload of a new type for each layer (including the base layer and one or more enhancement layers).

[0120] Because the auxiliary information used in these tools is created to enhance specific HOA representations, it needs to be extended. In the current definition of hierarchical HOA encoding, the provided data only appropriately extends the HOA representation at the highest level. For lower levels, these tools do not adequately enhance the partially reconstructed HOA representation.

[0121] Therefore, it would be better to provide these tools with auxiliary information for each layer so that they can be better adapted to the HOA representation of the corresponding layer's reconstruction.

[0122] Furthermore, the tool subband directional signal synthesis and parameterized environment replication decoder are specifically designed for low data rates, where only a few transmission signals are available. The proposed extension will therefore provide the ability to optimally adapt the auxiliary information of these tools to the number of transmission signals in the layer. Consequently, the sound quality of the HOA representation for reconstruction of low bit-rate layers (e.g., the base layer) can be significantly improved compared to existing layering methods.

[0123] Furthermore, if `CodedVVecLength` is set to 1 in `HOADecoderConfig()`, the bitstream syntax of the V vector elements used for encoding vector-based signals needs to be adapted for HOA hierarchical encoding. In this vector encoding mode, V vector elements are not sent for HOA coefficient indices included in the `ContAddHoaCoeff` set. This set includes all HOA coefficient indices `AmbCoeffIdx[i]` with an `AmbCoeffTransitionState` equal to zero. Because the original HOA coefficient sequences used for these indices are explicitly sent, there is no need to add weighted V vector signals. Therefore, for these indices, the V vector elements in the normal approach are set to zero.

[0124] However, in layered coding mode, the set of consecutive HOA coefficient indices depends on the transmission channels, which are part of the currently operating layer. This means that additional HOA coefficient indices transmitted in higher layers are missing in lower layers. Therefore, the assumption that vector signals should not contribute to the HOA coefficient sequence is incorrect for HOA coefficient indices belonging to HOA coefficient sequences included in higher layers. Therefore, it is proposed to (explicitly) label these missing coefficient indices as V vector elements.

[0125] Therefore, it is proposed to define a ContAddHoaCoeff set for each layer and use the set of layers to which the V vector signal (to which the transmitted signal of the V vector signal belongs) has been added to select the V vector element to be used. However, it is proposed that the V vector data remain in HOAFrame() and not be moved to HOAEnhFrame().

[0126] Next, the integration into the MPEG-H bitstream syntax will be described. (Refer to...) Figure 3 The corresponding encoding methods according to embodiments of this disclosure (e.g., layered encoding methods for frames represented by HOA for compressed sound or sound fields) will be described below. The proposed changes to the MPEG-H 3D bitstream will be described in the appendix.

[0127] In layered encoding mode, the SingleLayer flag in HOADecoderConfig() is inactive (SingleLayer == 0), and it defines the number of layers and the corresponding number of HOA transmission signals assigned to them. Generally, compressed HOA representations can include multiple transmission signals.

[0128] Therefore, in Figure 3 In S3010 This involves distributing multiple transmission signals to multiple hierarchical layers. In other words, the transmission signals are distributed across multiple layers. Each layer can be described as including the corresponding transmission signals assigned to it. Each layer can have more than one transmission signal assigned to it. The multiple layers can include a base layer and one or more hierarchical enhancement layers. These layers can be ordered from the base layer, to the enhancement layers, all the way to the overall highest enhancement layer (the overall highest layer).

[0129] An additional HOA configuration extension payload and HOA frame extension payload with a newly defined usacExtElementType ID_EXT_ELE_HOA_ENH_LAYER are proposed to be added to the MPEG-H bitstream to send spatial signal prediction, subband direction signal synthesis, and PAR decoder data for each HOA enhancement layer (including the base layer). These additional payloads will follow directly after the payload of type ID_EXT_ELE_HOA in mpegh3daExtElementConfig() and correspondingly in mpegh3daFrame().

[0130] Therefore, it is proposed that, when SingleLayer == 0, the configuration elements used for spatial signal prediction, sub-band directional signal synthesis, and PAR decoder be moved from HOADecoderConfig() to the newly defined HOADecoderEnhConfig(), and correspondingly, HOARedictionInfo(), HOADirectionalPredictionInfo(), and HOAArInfo() be moved from HOAFrame() to the newly defined HOAEnhFrame().

[0131] Therefore, in S3020 For each layer, a corresponding HOA extended payload is generated. The generated HOA extended payload may include auxiliary information for parametrically enhancing the reconstructed HOA representation obtainable from the transmitted signals assigned to the corresponding layer and any layer below the corresponding layer (e.g., included in any layer below the corresponding layer). As indicated above, the HOA extended payload may include bitstream elements for one or more of the HOA spatial signal prediction decoding tool, the HOA sub-band direction signal synthesis decoding tool, and the HOA parametric environment replication decoding tool. Furthermore, the HOA extended payload may have a usacExtElementType of ID_EXT_ELE_HOA_ENH_LAYER.

[0132] exist S3030 The generated HOA extended payloads are then assigned to their respective layers.

[0133] In addition (in) Figure 3 (Not shown in the image), it can generate a HOA configuration extension payload that includes bitstream elements for configuring the HOA spatial signal prediction decoding tool, the HOA sub-band direction signal synthesis decoding tool, and / or the HOA parameterized environment copy decoding tool.

[0134] In addition (in) Figure 3 (Not shown in the image), can generate HOA decoder configuration payload, which includes information indicating the allocation of HOA extended payloads to multiple layers.

[0135] Next, the transmission of the layered bitstream (e.g., an MPEG-H bitstream) will be described. Because all extended payloads of the MPEG-H bitstream are byte-aligned and their sizes are explicitly indicated, the unpacker can parse the MPEG-H bitstream and extract the payloads for layers higher than one, assuming the elementLengthPresent flag is equal to 1, and transmit them separately through different transmission channels. The base layer consists of an MPEG-H bitstream excluding data for higher layers (e.g., composed of an MPEG-H bitstream excluding data for higher layers). Missing extended payloads are marked as empty or inactive. For payloads of types ID_USAC_SCE, ID_USAC_CPE, and ID_USAC_LFE, an empty payload is indicated by an elementLength of zero, where elementLengthPresent needs to be set to one. An empty payload of type ID_USAC_EXT can be indicated by setting the usacExtElementPresent flag to zero (false).

[0136] Therefore, in S3040 The generated HOA extended payload is indicated (e.g., transmitted or output) in the output bit stream. Generally, multiple layers and their assigned payloads are indicated (e.g., transmitted or output) in the output bit stream. Furthermore, HOA decoder configuration payloads and / or HOA configuration extended payloads can be indicated (e.g., transmitted or output) in the output bit stream.

[0137] Assume the HOA base layer (layer index equal to 1) is transmitted with the highest error protection and a relatively low bit rate. Error protection for subsequent layers (one or more HOA enhancement layers) steadily decreases as the bit rate of the enhancement layer increases. Due to poor transmission conditions and lower error protection, transmission of higher layers may fail, and in the worst case, only the base layer is transmitted correctly. Assume combined error protection is applied to all payloads of a layer. Therefore, if the transmission of a layer fails, all payloads for the corresponding layer are lost.

[0138] In other words, data payloads for multiple layers can be sent with corresponding error protection levels, where the base layer has the highest error protection and one or more enhancement layers have progressively lower error protection.

[0139] Unless a step requires certain other steps as prerequisites, the aforementioned steps can be performed in any order, and Figure 3 The exemplary order shown is to be understood as non-limiting.

[0140] As indicated above, if CodedVVecLength is set to 1 in HOADecoderConfig(), then the bitstream syntax of the V vector elements used for encoding vector-based signals needs to be adapted for HOA layered encoding. (Refer to...) Figure 4 To describe the corresponding encoding method according to embodiments of the present disclosure (e.g., a layered encoding method for frames represented by HOA for the compression of sound or sound field).

[0141] exist Figure 4 In S4010 This involves distributing multiple transmission signals to multiple hierarchical layers. This step can be performed in the same manner as S3010 described above.

[0142] exist S4020 This involves determining whether the vector encoding mode is in effect. This may involve determining whether CodedVVecLength == 1.

[0143] As indicated above, in the conventional approach, in vector encoding mode, V vector elements are not sent for the HOA coefficient indices included in the ContAddHoaCoeff set. This set includes all HOA coefficient indices AmbCoeffIdx[i] that have an AmbCoeffTransitionState equal to zero. Because the original HOA coefficient sequences used for these indices are explicitly sent, there is no need to add a weighted V vector signal. Therefore, for these indices, the V vector elements in the conventional approach are set to zero.

[0144] However, in layered coding mode, the set of consecutive HOA coefficient indices depends on the transmission channels, which are part of the currently operating layer. This means that additional HOA coefficient indices transmitted in higher layers are missing in lower layers. Therefore, the assumption that vector signals should not contribute to the HOA coefficient sequence is incorrect for HOA coefficient indices belonging to HOA coefficient sequences included in higher layers.

[0145] Therefore, if the vector encoding mode is active, then in S4030 For each layer, a set of continuous HOA coefficient indices (e.g., ContAddHoaCoeff) is determined (e.g., limited) based on the transmission signals assigned to the corresponding layer.

[0146] If the vector encoding mode is active, then in S4040For each transmission signal, a V vector is generated based on a determined set of consecutive HOA coefficient indices for the layer to which the corresponding transmission signal was assigned. Each generated V vector may include elements for any transmission signal assigned to a layer higher than the layer to which the corresponding transmission signal was assigned. This step may involve selecting the V vector elements to be used using a set of consecutive HOA coefficient indices already determined for the layer in which the V vector signal was added (the layer to which the transmission signal of the V vector signal belongs). However, it is proposed that the V vector data remain in HOAFrame() and is not moved to HOAEnhFrame().

[0147] Then, in S4050 The generated V vector (V vector signal) is identified in the output bit stream. This may involve (explicitly) identifying the V vector elements used for the aforementioned missing coefficient index.

[0148] Figure 4 Steps S4040 to S4050 can also be performed in Figure 3 The encoding method shown is used in the context of the example, for example, after S3010. In this case, S3040 and S4050 can be combined into a single marking step.

[0149] Unless a step requires certain other steps as prerequisites, the aforementioned steps can be performed in any order, and Figure 4 The exemplary order shown is to be understood as non-limiting.

[0150] At the receiver end, the MPEG-H bitstream packer can re-insert the correctly received payload into the base layer MPEG-H bitstream and pass it to the MPEG-H 3D audio decoder.

[0151] Next, the HOA decoding initialization (configuration) will be described. HOA configuration payloads (with their corresponding byte sizes) of types ID_EXT_ELE_HOA and ID_EXT_ELE_HOA_ENH_LAYER are input to the HOA decoder for its initialization. The HOA encoding tools are configured according to the bitstream elements defined in HOAConfig(), which is parsed from the payload of type ID_EXT_ELE_HOA. Furthermore, this payload contains the use of the layered coding mode, the number of layers, and the corresponding number of transmitted signals for each layer. Then, if layered coding is activated (SingleLayer == 0), HOAConfig() is parsed from the payload of type ID_EXT_ELE_HOA_ENH_LAYER to configure the corresponding spatial signal prediction, sub-band direction signal synthesis, and parameterized environment replication decoder for each layer.

[0152] The LayerIdx element from HOAEnhConfig(), along with the order of the HOA enhancement layer configuration payloads in mpegh3daExtElementConfig(), indicates the order of the HOA enhancement layers. The order of the HOA enhancement layer frame payloads of type ID_EXT_ELE_HOA_ENH_LAYER in mpegh3daFrame() is the same as the order of the configuration payloads in mpegh3daExtElementConfig() to clearly assign the frame payloads to the corresponding layers.

[0153] In the case of SingleLayer == 1 (single-layer encoding), the payload of type ID_EXT_ELE_HOA_ENH_LAYER is ignored, and the spatial signal prediction, sub-band direction signal synthesis, and parameterized environment copy decoder use the corresponding data from HOADecoderConfig() for their configuration.

[0154] Next, HOA frame decoding in layered mode will be described. (Refer to...) Figure 5 To describe a corresponding decoding method according to embodiments of the present disclosure (e.g., a method for decoding frames of compressed HOA representations of sound or sound fields). It is understood that compressed HOA representations (e.g., those described above) Figure 3 or Figure 4 The output of the method has been encoded in multiple hierarchical layers, which include a base layer and one or more enhancement layers.

[0155] exist Figure 5 In S5010 Receive bit streams associated with frames represented by compressed HOA.

[0156] The 3D audio core decoder decodes correctly transmitted HOA transport signals and recreates transport signals with zero samples for corresponding invalid payloads. The decoded transport signals, along with the usacExtElementPresent flag, the data and size of the HOA payload of type ID_EXT_ELE_HOA and ID_EXT_ELE_HOA_ENH_LAYER, are input to the HOA decoder. Extended payloads from type ID_USAC_EXT with the usacExtElementPresent flag set to false need to be flagged as missing payloads by the HOA decoder to ensure payload allocation to the corresponding layer.

[0157] exist S5020 Extract payloads for multiple layers. Each payload may include transmission signals assigned to the corresponding layer.

[0158] In this step, the HOA decoder can parse HOAFrame() from the payload of type ID_EXT_ELE_HOA.

[0159] Subsequently, valid payloads of type ID_EXT_ELE_HOA_ENH_LAYER and invalid payloads of type ID_EXT_ELE_HOA_ENH_LAYER are determined by evaluating the corresponding usacExtElementPresent flag of the payload, wherein invalid payloads are indicated by a usacExtElementPresent flag equal to false, and the allocation of HOA enhancement payloads to enhancement layer indexes is known from the HOA decoder configuration.

[0160] exist S5030 The highest available layer for decoding is determined among multiple layers.

[0161] Because layers are interdependent in terms of transmitted signals, the HOA decoder can only decode a layer if all layers with lower indices have been correctly received. The highest available layer can be selected at this step to ensure that all layers up to the highest available layer have been correctly received. Details of this step are described below.

[0162] exist S5040 The HOA extended payload allocated to the highest availability layer is extracted. As indicated above, the HOA extended payload may include auxiliary information for parametrically enhancing the reconstructed HOA representation corresponding to the highest availability layer. The reconstructed HOA representation corresponding to the highest availability layer can be obtained based on the transmission signals allocated to the highest availability layer and any layer below it.

[0163] Additionally, HOA extended payloads allocated to the remaining layers in the multiple layers can be extracted. Each HOA extended payload may include auxiliary information for parametrically enhancing the reconstructed HOA representation corresponding to its respective allocated layer. The reconstructed HOA representation corresponding to its respective allocated layer can be obtained from the transmission signals allocated to that layer and any layers below it.

[0164] In addition (in) Figure 5 (Not shown in the image), the decoding method may include the step of extracting the HOA configuration extension payload. This can be done by parsing the bitstream. The HOA configuration extension payload may include bitstream elements for configuring the HOA spatial signal prediction decoding tool, the HOA sub-band direction signal synthesis decoding tool, and / or the HOA parameterized environment replication decoding tool.

[0165] exist S5050The HOA representation corresponding to the highest availability layer is generated based on the transmission signals assigned to the highest availability layer and any layer below the highest availability layer.

[0166] The actual number of transmitted signals I ADD,LAY (k) is based on the index M of the highest available tier. LAY (k) is set, and the first initial HOA is decoded from HOAFrame() and the corresponding transmission signals from this layer and any lower layer.

[0167] Then, in S5060 The reconstructed HOA representation is enhanced (e.g., parametrically enhanced) by using auxiliary information included in the HOA extension payload assigned to the highest available layer.

[0168] In other words, the decoder then uses the currently acting layer M through spatial signal prediction, sub-band direction signal synthesis, and parameterized environment replication. LAY The HOA enhancement layer of type ID_EXT_ELE_HOA_ENH_LAYER (k) (i.e., the highest available layer) extends the HOA representation obtained in S5050 by resolving the HOAEnhFrame() data of the payload parsing.

[0169] The information used in steps S5020-S5060 can be referred to as layer information.

[0170] Unless a step requires certain other steps as prerequisites, the aforementioned steps can be performed in any order, and Figure 5 The exemplary order shown is to be understood as non-limiting.

[0171] Next, the details of determining (e.g., selecting) the highest available layer in S5030 will be described.

[0172] As indicated above, the HOA decoder can only decode a layer if all layers with lower indices have been correctly received, because the layers are interdependent in terms of the transmitted signals.

[0173] To select the highest decodable layer, the HOA decoder can create a set of invalid layer indices, where the index M of the highest decodable enhancement layer is obtained by subtracting one from the smallest index in this set. LAY The set of invalid layer indexes can be determined by evaluating the validity flags of the corresponding HOA extended payload.

[0174] In other words, determining the highest available layer can involve identifying a set of invalid layer indices that indicate layers that have not yet been effectively received. It can further involve determining the highest available layer as the layer below the layer indicated by the smallest index in the set of invalid layer indices. This ensures that all layers below the highest available layer have been effectively received.

[0175] When performing differential coding on frames, it is necessary to consider the index of the highest available layer of the previous frame (e.g., the immediately preceding frame). First, the case where the index of the highest available layer of the previous frame (e.g., the previous frame) is preserved will be described.

[0176] If the index used for the highest available layer (e.g., the highest decodeable layer) in the current frame is equal to the layer index M of the previous frame. LAY (k-1), then the layer index M of the current frame will be... LAY (k) is set to M LAY (k-1).

[0177] Then, as indicated above, according to M LAY (k) is used to set the actual number of transmitted signals I. ADD,LAY (k), and decode the first preliminary HOA representation from HOAFrame() and the corresponding transmission signals from this layer and any lower layers. As indicated above, then the decoder uses the currently acting layer M through spatial signal prediction, sub-band direction signal synthesis, and parameterized environment replication. LAY The HOA enhancement layer of type ID_EXT_ELE_HOA_ENH_LAYER (k) is used to enhance the HOA representation by parsing the HOAEnhFrame() data to extend the payload.

[0178] Next, we will describe the case where it is switched to an index lower than the highest available layer index of the previous frame (e.g., the previous frame). That is, the index of the highest decodeable layer in the current frame is less than the layer index M of the previous frame. LAY In the case of (k-1), the HOA decoder will M LAY (k) is set to the index of the highest decodeable layer in the current frame. Decoding of the payloads used for spatial signal prediction, subband direction signal synthesis, and parameterized environment copy decoder for the new layer can only begin at the next HOA frame with a hoaIndependencyFlag equal to 1. Reconstructing index M without performing spatial signal prediction, subband direction signal synthesis, and parameterized environment copy decoder operations. LAY The HOA representation of layer (k) continues until such a HOAFrame() has been received. This means that the actual number of transmitted signals used is I. ADD,LAY (k) is based on M LAY(k) is set, and only the first preliminary HOA representation is decoded from HOAFrame() and the corresponding transport signals from that layer and any lower layers. Then, if a HOAFrame() with a hoaIndependencyFlag equal to 1 has been received, the payloads for spatial signal prediction, sub-band direction signal synthesis, and parameterized environment replication decoder are parsed and decoded to enhance the preliminary HOA representation so that it provides the full quality of the currently acting layer for that frame.

[0179] Therefore, the proposed method may include (in Figure 5 (not shown in the image): If the highest available layer of the current frame is lower than the highest available layer of the previous frame (if the current frame has been encoded differently relative to the previous frame), it is decided not to use the auxiliary information included in the HOA extended payload allocated to the highest available layer to perform parameterized enhancement of the reconstructed HOA representation.

[0180] Generally, determining the highest available layer of the current frame may involve determining a set of invalid layer indices indicating that the current frame has not yet been effectively received. It may further include determining the highest available layer of the preceding frames. It may also further include determining the highest available layer as the lower of the following layers: the highest available layer of the preceding frames, and the layer below the layer indicated by the smallest index in the set of invalid layer indices (if the current frame has been encoded differently relative to the preceding frames).

[0181] An alternative solution can always resolve all valid enhancement layer payloads (e.g., HOA extended payloads) in parallel, even if they are not currently active. This would enable direct switching to layers with lower indices with full quality, where spatial signal prediction, sub-band directional signal synthesis, and parametric environment replication (PAR) decoders can be directly applied to the switched-to frame.

[0182] Next, we will describe the case where it is switched to an index higher than the highest available layer index of the previous frame (e.g., the previous frame). This switch to a layer with a higher index can only be applied if `mpegh3daFrame()` has a `usacIndependencyFlag` equal to 1 (e.g., the frame is an independent frame), because all corresponding payloads or decoder levels of the previous frame are missing. Therefore, the HOA decoder makes the HOA layer index M... LAY (k) remains equal to M LAY (k-1), until an mpegh3daFrame() (e.g., an independent frame) with a usacIndependencyFlag equal to 1, containing valid data for higher decodeable layers, has been received. Then, MLAY (k) is set to the highest decodable layer index of the current frame, and accordingly, the amount of transmitted signal I actually used is determined. ADD,LAY (k). Decode the initial HOA representation of this layer from HOAFrame() and the corresponding transmitted signal, and use the decoder from the currently acting layer M through spatial signal prediction, sub-band direction signal synthesis, and parameterized environment replication. LAY (k) The HOA enhancement layer of type ID_EXT_ELE_HOA_ENH_LAYER extends the payload resolution of HOAEnhFrame() to enhance the initial HOA representation.

[0183] It is understood that the proposed hierarchical coding method for compressed sound representation can be implemented by an encoder for hierarchical coding of compressed sound representation. Such an encoder may include units suitable for performing the steps described above. Figure 6 An example of such an encoder 6000 is schematically illustrated. For example, such an encoder 6000 may include a transmission signal distribution unit 6010 adapted to perform the aforementioned S3010, a HOA extension layer payload generation unit 6020 adapted to perform the aforementioned S3020, an HOA extension payload distribution unit 6030 adapted to perform the aforementioned S3030, and an identification unit or output unit 6040 adapted to perform the aforementioned S3040. Further understood, each unit of such an encoder can be implemented by a processor 6100 of a computing device, the processor 6100 being adapted to execute the processing performed by each of the units, i.e., adapted to execute... Figure 3 The diagram schematically illustrates some or all of the aforementioned steps of the proposed encoding method. Alternatively or additionally, the processor 6100 may be adapted to execute... Figure 4 Each step of the encoding method is schematically illustrated. For this purpose, the processor 6100 may be adapted to implement the various units of the encoder. The encoder or computing device may further include a memory 6200 accessible to the processor 6100.

[0184] Furthermore, the proposed method for decoding compressed audio representations encoded in multiple hierarchical layers can be implemented using a decoder for decoding compressed audio representations encoded in multiple hierarchical layers. Such a decoder may include units suitable for performing the steps described above. Figure 7An example of such a decoder 7000 is schematically illustrated. For example, such a decoder 7000 may include a receiving unit 7010 adapted to perform the aforementioned S5010, a payload extraction unit 7020 adapted to perform the aforementioned S5020, a highest available layer determination unit 7030 adapted to perform the aforementioned S5030, a HOA extended payload extraction unit 7040 adapted to perform the aforementioned S5040, a reconstructed HOA representation generation unit 7050 adapted to perform the aforementioned S5050, and an enhancement unit 7060 adapted to perform the aforementioned S5060. Further understood, each unit of such a decoder may be implemented by a processor 7100 of a computing device, the processor 7100 being adapted to perform the processing executed by each of the units, i.e., adapted to perform some or all of the aforementioned steps of the proposed decoding method. The decoder or computing device may further include a memory 7200 accessible to the processor 7100.

[0185] Next, a data structure (e.g., a bitstream) for accommodating (e.g., representing) a compressed HOA representation in a hierarchical encoding mode will be described. Such a data structure can originate from the proposed encoding method and can be decoded (e.g., decompressed) using the proposed decoding method.

[0186] This data structure may include multiple HOA frame payloads corresponding to corresponding hierarchical layers in a plurality of hierarchical layers. Multiple transmission signals may be assigned to (e.g., may belong to) corresponding layers in a plurality of layers. The data structure may include corresponding HOA extended payloads, which include auxiliary information for parametrically enhancing the reconstructed HOA representation obtainable from transmission signals assigned to the corresponding layer and any layer below the corresponding layer. As indicated above, the HOA frame payloads and HOA extended payloads for multiple layers may be provided with corresponding error protection levels. Furthermore, the HOA extended payloads may include the bitstream elements indicated above and may have a usacExtElementType with ID_EXT_ELE_HOA_ENH_LAYER. The data structure may further include HOA configuration extended payloads and / or HOA decoder configuration payloads, which include the bitstream elements indicated above.

[0187] It should be noted that the description and accompanying drawings only illustrate the principles of the proposed methods and apparatus. Therefore, it will be appreciated that those skilled in the art will be able to conceive of various arrangements that, although not explicitly described or shown herein, embody the principles of the invention and are included within the spirit and scope of the invention. Furthermore, all examples recorded herein are primarily and explicitly intended for educational purposes only to aid the reader in understanding the principles of the proposed methods and apparatus and the inventors' contributions to advancing the prior art, and are to be construed as not being limited to such specific examples and conditions. Moreover, all statements herein recounting the principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to cover their equivalents.

[0188] The methods and apparatus described in this disclosure can be implemented as software, firmware, and / or hardware. Some components can be implemented, for example, as software running on a digital signal processor or microprocessor. Other components can be implemented, for example, as hardware and / or application-specific integrated circuits (ASICs). Signals encountered in the described methods and apparatus can be stored on a medium (such as random access memory or optical storage medium). They can be transmitted via a network (such as a radio network, satellite network, wireless network, or wired network (e.g., the Internet)).

[0189] appendix:

[0190] The proposed MPEG-H 3D bitstream modification

[0191] Changes are marked by highlighting them in gray:

[0192] Table 1 – Syntax of mpegh3daExtElementConfig()

[0193]

[0194]

[0195] Table 2 – Values ​​of usacExtElementType

[0196]

[0197] Table 3 – Explanation of data blocks used for extended payload decoding

[0198]

[0199]

[0200] Table 4 – Syntax of HOADecoderConfig()

[0201]

[0202]

[0203]

[0204] New table? — Syntax of HOAEnhConfig()

[0205]

[0206] New table? — Syntax of HOADecoderEnhConfig()

[0207]

[0208]

[0209]

[0210] Table 5 – Syntax of HOAFrame

[0211]

[0212]

[0213]

[0214] The NumOfDirSigsPerLayer[lay] element determines the number of directional signals that are actually used in the current HOAFrame() in the HOA enhancement layer lay.

[0215] The AddHoaCoeffPerLayer[lay] array contains the HOA coefficient index for each additional environmental HOA coefficient actually used in the HOA enhancement layer lay.

[0216] The NumOfAddHoaChansPerLayer[lay] element indicates the total number of additional environmental HOA coefficients actually used in the HOA enhancement layer lay.

[0217] Add this table

[0218] New table? — Syntax of HOAEnhFrame

[0219]

[0220] Update the table:

[0221] Table 6 – Syntax of VVectorData()

[0222]

[0223]

[0224]

[0225] Table 7 – Syntax of HOAPredictionInfo(DirSigChannelIds, NumOfDirSigs)

[0226]

[0227]

[0228] Table 1.2 - Syntax of HOADirectionalPredictionInfo()

[0229]

[0230]

[0231]

[0232]

[0233] Table 8 – Definition of SingleLayer

[0234]

[0235] The `codedLayerCh` element indicates the number of transmitted signals included in the first (i.e., base) layer, given by `codedLayerCh+MinNumOfCoeffsForAmbHOA`. For higher (i.e., enhancement) layers, this element indicates the number of additional signals included in the enhancement layer compared to the next lower layer, given by `codedLayerCh+1`.

[0236] The HOALayerChBits element indicates the number of bits used to read codedLayerCh.

[0237] The NumLayers element indicates the total number of layers within the bitstream (after reading HOADecoderConfig()).

[0238] The NumHOAChannelsLayer element is an array of NumLayers elements, where the i-th element indicates the number of transmitted signals included in all layers up to the i-th layer.

[0239] 12.4.1x Frames and User-Dependent Parameters

[0240] M LAY (k) The number of layers actually used at the decoder for the k-th frame (to be specified). In the case of layered coding (indicated by SingleLayer == 0), this number must be less than or equal to the total number of layers present in the bitstream, i.e., M LAY ≤NumLayers. In the case of single-layer encoding (indicated by SingleLayer == 1), M LAY It is set to 1.

[0241] Dependent on M LAY The choice of (k) is calculated as follows: the additional transport channel actually used for spatial HOA decoding (i.e., in addition to O, which is always implicitly used) is calculated. MIM The number of channels (excluding) I ADD,LAY (k):

[0242]

[0243]

[0244] VVecLength and VVecCoeffId codedVVecLength word indicators:

[0245] 0) Full vector length (NumOfHoaCoeffs element). Indicates all coefficients (NumOfHoaCoeffs) used for the dominant vector that are specified.

[0246] 1) All elements defined in ContAddHoaCoeff[lay] for the currently active layer, from vector element 1 to MinNumOfCoeffsForAmbHOA and indexed as lay = 0…NumLayers-1, are not sent. For SingleLayer mode == 1, the variable NumLayers needs to be set to 1. Only those coefficients corresponding to the number greater than MinNumOfCoeffsForAmbHOA are specified to indicate the dominant vector. Furthermore, those NumOfContAddAmbHoaChan[lay] coefficients identified in ContAddAmbHoaChan[lay] are removed. The list ContAddAmbHoaChan[lay] specifies additional channels corresponding to orders exceeding the order of MinAmbHoaOrder.

[0247] 2) Vector elements 1 through MinNumOfCoeffsForAmbHOA were not sent. The coefficients corresponding to the number of dominant vectors greater than MinNumOfCoeffsForAmbHOA are specified.

[0248] When codedVVecLength == 1, both the VVecLength[i] array and the VVecCoeffId[i][m]2D array are valid for the VVector at index i. In other cases, both the VVecLength element and the VVecCoeffId[m] array are valid for all VVectors within the HOAFrame. For the following allocation algorithm, the helper function is defined as follows.

[0249]

[0250]

[0251]

[0252] The first switch statement, with three cases (cases 0-2), provides a way to determine the dominant vector length (VVecLength) in terms of quantity and the index of the coefficient (VVecCoeffId).

[0253] 12.4.1X to VVec element conversion

[0254] The type of dequantization for the V vector is indicated by the word NbitsQ. An NbitsQ value of 4 indicates vector quantization. When NbitsQ equals 5, uniform 8-bit scalar dequantization is performed. Conversely, an NbitsQ value greater than or equal to 6 indicates that Huffman decoding of the V vector with scalar quantization is applied.

[0255] The prediction pattern is represented as PFlag, while CbFlag represents the Huffman table information bits.

[0256]

[0257]

Claims

1. A method for decoding a compressed high-order high-fidelity stereo reproduction (HOA) representation of sound or sound field, the method comprising: Receive a bitstream containing a compressed HOA representation, wherein the bitstream includes a plurality of hierarchical layers, the plurality of hierarchical layers including a base layer and one or more hierarchical enhancement layers, wherein the plurality of hierarchical layers include components of a compressed HOA representation of a sound or sound field. Among the plurality of hierarchical layers, the highest available layer for decoding is determined, wherein the highest available layer is the layer below the lowest layer that has not yet been effectively received; Extract the HOA extended payload allocated to the highest available layer, wherein the HOA extended payload includes auxiliary information for parametrically enhancing the reconstructed HOA representation corresponding to the highest available layer, and wherein the HOA extended payload includes bitstream elements for HOA spatial signal prediction decoding, wherein the reconstructed HOA representation corresponding to the highest available layer can be obtained based on the transmission signals allocated to the highest available layer and any layer below the highest available layer; Based on layer information, transmission signals assigned to the highest available layer and any layers below the highest available layer, the compressed HOA representation corresponding to the highest available layer is decoded; and The decoded HOA representation is parametrically enhanced by using auxiliary information included in the HOA extension payload assigned to the highest available layer.

2. The method according to claim 1, wherein, Layer information indicates the total number of additional environmental HOA coefficients used to enhance the layer.

3. The method according to claim 1, wherein, The layer information includes enhancement information, which includes at least one of the following: spatial signal prediction, sub-band direction signal synthesis, and parameterized environment replication decoder.

4. The method of claim 1, further comprising a v vector element, wherein the v vector element is not sent for an index that is equal to the index of an additional HOA coefficient included in the ContAddHoaCoeff set.

5. The method according to claim 1, wherein, The layer information includes NumLayers elements, where each element indicates the number of transmitted signals included in all layers up to the i-th layer.

6. The method according to claim 1, wherein, The layer information includes indicators for all layers actually used in the k-th frame.

7. The method according to claim 1, wherein, Layer information indicates that coefficients used for the dominant vector are specified.

8. The method according to claim 1, wherein, Layer information indicates that the coefficients corresponding to the dominant vectors with values ​​greater than MinNumOfCoeffsForAmbHOA are specified.

9. The method according to claim 1, wherein, Layer information indication: MinNumOfCoeffsForAmbHOA and all elements defined in ContAddHoaCoeff are not sent, where lay is the index of the layer containing the vector-based signal corresponding to the vector.

10. A non-transitory computer-readable medium comprising instructions that, when executed by a processor, implement the method according to any one of claims 1-9.

11. An apparatus for decoding a compressed high-order high-fidelity stereo reproduction (HOA) representation of sound or sound field, the apparatus comprising: A receiver configured to receive a bitstream comprising a compressed HOA representation, wherein the bitstream comprises a plurality of hierarchical layers, the plurality of hierarchical layers comprising a base layer and one or more hierarchical enhancement layers, wherein the plurality of hierarchical layers comprise components of a compressed HOA representation of a sound or sound field. The decoder is configured as follows: Among the plurality of hierarchical layers, the highest available layer for decoding is determined, wherein the highest available layer is the layer below the lowest layer that has not yet been effectively received; Extract the HOA extended payload allocated to the highest available layer, wherein the HOA extended payload includes auxiliary information for parametrically enhancing the reconstructed HOA representation corresponding to the highest available layer, and wherein the HOA extended payload includes bitstream elements for HOA spatial signal prediction decoding, wherein the reconstructed HOA representation corresponding to the highest available layer can be obtained based on the transmission signals allocated to the highest available layer and any layer below the highest available layer; Based on layer information, transmission signals assigned to the highest available layer and any layers below the highest available layer, the compressed HOA representation corresponding to the highest available layer is decoded; and The decoded HOA representation is parametrically enhanced by using auxiliary information included in the HOA extension payload assigned to the highest available layer.

12. An apparatus for decoding a compressed high-order high-fidelity stereo reproduction (HOA) representation of sound or sound field, comprising: One or more processors, and The memory stores instructions that, when executed, cause the one or more processors to perform the steps of the method according to any one of claims 1-9.

13. An apparatus comprising components for performing the steps of the method according to any one of claims 1-9.

14. A computer program product having instructions that, when executed by a computing device or system, cause the computing device or system to perform the method according to any one of claims 1-9.

15. An encoding method for a compressed high-order high-fidelity stereo reproduction (HOA) representation of sound or sound field, wherein the compressed HOA representation comprises multiple transmission signals, and the method comprises: Distribute multiple transmission signals to multiple hierarchical layers; Determine whether the vector encoding mode is in effect; If the vector coding mode is in effect, then for each of the multiple hierarchical layers, the set of consecutive HOA coefficient indices is determined based on the transmission signals assigned to the corresponding layer. For each transmission signal, a V vector is generated based on a continuous set of HOA coefficient indices determined for the layer to which the corresponding transmission signal is assigned, such that the generated V vector includes elements for any transmission signal assigned to a layer higher than the layer to which the corresponding transmission signal is assigned. as well as The generated V vector is indicated in the output bit stream.

16. An encoding device for compressing a high-order high-fidelity stereo reproduction (HOA) representation of a sound or sound field, wherein the compressed HOA representation comprises a plurality of transmitted signals, and the device includes an encoder configured to: Distribute multiple transmission signals to multiple hierarchical layers; Determine whether the vector encoding mode is in effect; If the vector coding mode is in effect, then for each of the multiple hierarchical layers, the set of consecutive HOA coefficient indices is determined based on the transmission signals assigned to the corresponding layer. For each transmission signal, a V vector is generated based on a continuous set of HOA coefficient indices determined for the layer to which the corresponding transmission signal is assigned, such that the generated V vector includes elements for any transmission signal assigned to a layer higher than the layer to which the corresponding transmission signal is assigned. as well as The generated V vector is indicated in the output bit stream.

Images