Signaling layer characteristics in multi-layer coding

Abstract

To provide a video encoder that codes a video into a multi-layered video signal on the basis of inter-layer prediction, and provide a layer identification extender.SOLUTION: A layer identification extender 110 contained in a device 100 for processing a multi-layered data stream reads a maximum text element from a multi-layered video signal indicating the maximum assumption value of an expansion layer ID field 112 of a packet 108 containing the existed part 113 of a multi-layered video signal against the existed part of the multi-layered video signal containing a plurality of packets 106 such as a part 113. The existed part is a video sequence to be coded, a large lamp, and a group of images. A maximum text element 114 is contained in a specified packet of the part 113, and an extender determines a layer ID for individual packets on the basis of a layer ID text element formed by a coding 108 and a coding 112 with a condition.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to... for example in the field of multi-layer video coding. [Background technology]

[0002] Video encoder standards such as the HEVC base specification [Non-Patent Document 1] and its extensions such as the multi-view extension [Non-Patent Documents 2, 3] typically inherit mechanisms for signaling the layer to which each coded chunk of video data belongs in a layered coding scenario. The signaling mechanism needs to be readily available (compare the layer identifier in the HEVC NAL unit header (nuh_layer_id) given in the table in Figure 1).

[0003] Due to its finite size in bits, each HEVC syntax element allows for 64 values ​​that can be used to identify scalable layers, views, and / or depths. For example, for multi-view video using the current HEVC NAL unit header syntax, this implies a limit of a maximum of 64 views, or 32 view+depth combinations. While this is sufficient for many common stereo-view scenarios and other applications, huge camera arrays with more than 100 views for applications such as light-field image processing [Non-Patent Documents 4, 5] or holographic displays [Non-Patent Document 6] require an extensible mechanism to signal a larger space of layer identifiers.

[0004] Furthermore, it is advantageous to structure the space of layer identifiers by grouping certain layers or views in terms of their coding dependencies or spatial relationships to each other. Furthermore, since deriving such information from information already available in the coded video bitstream requires prohibitive computational resources, a compact representation of coding dependencies in a structured layer identifier space is also advantageous. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] B. Bross, W.-J. Han, J.-R. Ohm, G.J. Sullivan, and T. Wiegand (eds.), High Efficiency Video Coding (HEVC) Text Specification Draft 10, JCTVC-L1003, Geneva, CH, January 2013. [Non-patent document 2] G. Tek, K. Wegner, Y. Chen, M. Hannüksela, and J. Boyce (eds.), "MV-HEVC Draft Text 3 (ISO / IEC 23008-2 PDAM2)", JCT3V-C1004, Geneva, CH, January 2013. [Non-patent document 3] G. Tek, K. Wegner, Y. Chen, and S. Ia (eds.), "3D-HEVC Test Model Description, Draft Specification," JCT3V-C1005, Geneva, CH, January 2013. [Non-patent document 4] Wilburn, Bennett et al., High-Performance Imaging Using Large Camera Arrays, ACM Transactions on Graphics, 2005, 24. Jg., No. 3, S. 765-776 [Non-patent document 5] Wilburn, Bennett S. et al., Optical Field Video Camera, Electronic Image Processing 2002, International Society for Optics and Photonics, 2001. S.29-36 [Non-patent document 6] Horimai, H. et al., Full-color 3D display system with 360-degree horizontal viewing angle, Proc. Int. Symposium on 3D and Contents, 2010, S.7-10 Summary of the Invention [Problem to be solved by the invention]

[0006] It is therefore an object of the present application to provide a concept for a signal in connection with a multi-layer coding that is more efficient, e.g., with reduced side information overhead and / or improved backward compatibility. This object is achieved by the subject matter of the pending independent claims. [Means for solving the problem]

[0007] One aspect of this application relates to signaling at least one characteristic for layers of a multi-layer video signal (e.g., an indication for each layer of a dependent layer to which the respective layer is directly related via inter-layer prediction), or signaling the second interdependent syntax structure described above. According to this aspect, a maximum syntax element is signaled within the multi-layer video signal to indicate the maximum value used for the enhancement layer ID field of packets of the multi-layer video signal. For example, the range of the maximum syntax element is a predetermined portion of the multi-layer video signal, extending across several portions of the multi-layer video signal. Thus, according to this aspect, it is appropriate for a device, such as a decoder or network element, receiving the multi-layer video signal to have knowledge of the actually consumed portion of the possible domain of possible values ​​that can be signaled by the enhancement layer ID field for a relatively large predetermined portion of the multi-layer video signal. It is not necessary for at least one characteristic to be signaled for each combination of base layer ID field value and enhancement layer ID field value, but rather, it is important to signal at least one characteristic for the maximum value of the layer determined based on the maximum hypothesis value. Therefore, it is not necessary to transmit / signal at least one characteristic for layers with a layer ID. The enhancement layer ID does not occur within a given portion of the multi-layer video signal. Beyond this, according to another embodiment, knowledge of the maximum hypothesis value is used to reduce the side information overhead for signaling the layer IDs of the individual portions, i.e., to reduce the bits required to signal the enhancement layer ID field within packets of the multi-layer video signal.

[0008] One aspect of this application relates to signaling inter-layer dependencies between layers of a multi-layer data stream. According to this aspect, a good compromise between a very focused restriction of the potential diversity of inter-layer dependencies, on the one hand, and a very complex signaling of inter-layer dependencies, on the other hand, is found by describing the inter-layer dependencies by means of a first inter-dependence syntactic structure indicating the interdependence between pairs of different values ​​representable by base layer IDs and a second inter-dependence syntactic structure indicating the interdependence between pairs of different values ​​representable by extension layer IDs. Portions of a multi-layer data stream are associated with base layer IDs and extension layer IDs indexing the layers. According to this concept, the emphasis shifts between an increased diversity of signalable inter-layer dependencies, on the one hand, and a reduced side information overhead for signaling inter-layer dependencies, on the other hand. For example, sets of layers with a common base layer ID are each referred to as a "cluster." The same second inter-dependence syntactic structure is used separately to adjust the inter-dependencies within all clusters and between all clusters related to each other via the first inter-dependence syntactic structure. Instead, two instantiations of the second interdependent syntactic structure are used to illustrate the interdependence between layers within one cluster and layers of different clusters on the other. Regardless of the emphasis placed on increased diversity or reduced side information overhead, the interdependent signaling concept results in keeping signaling overhead low.

[0009] One aspect of this application relates to signaling the layer ID associated with each packet of a multi-layer video signal. This aspect, in particular, achieves an efficient method of signaling this layer relationship. Nevertheless, it maintains backward compatibility with codes where certain values ​​of the base layer ID field are restricted to be non-extensible, such as a base layer ID value of 0 in the base layer ID field. To specifically circumvent this restriction for non-extensible base layer ID values, instead, the layer IDs of portions of the multi-layer data stream are signaled in an extensible manner by partitioning the base layer ID field into a first subfield and a second subfield. An enhancement layer ID field is provided whenever the first subfield of the base layer ID field meets a predetermined criterion. If the first subfield of the base layer ID field does not meet the predetermined criterion, the enhancement layer ID field is omitted. The non-extensible base layer ID value is "hidden" within a group of base layer ID values ​​whose first subfield of the base layer ID field does not meet the predetermined criterion. Therefore, the non-extensible base layer ID value is not processed separately, but rather, only the former group is processed. Rather, if the first subfield of the base layer ID field satisfies a predetermined criterion, an extension value is derived from the enhancement layer ID field signaled in the multi-layer data stream such that it lies within a first subset of the domain of extension values. If the first subfield of the base layer ID field does not satisfy the predetermined criterion, the extension value is set to a value that falls within the first subset of the domain of extension values. The layers to which each individual portion pertains are then indexed using the extension value as well as the cluster value derived from the second subfield of the base layer ID field. Generally, signaling efficiency is not lost despite maintaining backward compatibility. [Brief explanation of the drawings]

[0010] The outlined aspects may be used in combination or individually, and preferred examples of the above outlined aspects are the subject of dependent claims and are discussed below with reference to the figures.

[0011] [Figure 1] Figure 1 shows the syntax of the HEVC NAL unit header. [Figure 2] 2 illustrates schematically an environment including a multi-layer video encoder, a network element, and a multi-layer video decoder, which may be improved by any or any combination of the concepts described in the following figures. [Figure 3] FIG. 3 illustrates diagrammatically the concept of signaling in a network device and layer interdependencies in a clustered manner within a multi-layer data stream arriving at the network device. [Figure 4] FIG. 4 illustrates schematically how inter-layer dependencies can be signaled according to the concept of FIG. [Figure 5] FIG. 5 illustrates a more specific example for deriving inter-layer dependencies when using the signal concept of FIG. [Figure 6] FIG. 6 illustrates a possible extension of the HEVC syntax for slice segment headers by showing a portion of the HEVC syntax extended by way of a syntax element indicating a layer ID extension. [Figure 7] FIG. 7 shows an example of a VPS syntax illustratively extended to implement the signaling concept of FIG. [Figure 8] FIG. 8 shows an alternative to the example of FIG. [Figure 9] FIG. 9 shows an alternative to the example of FIG. [Figure 10] FIG. 10 shows an alternative to the example of FIG. [Figure 11] FIG. 11 shows a schematic diagram of an apparatus for establishing a further concept of the present application in which the maximum hypothesized value of the enhancement layer ID field is signaled in the data stream. [Figure 12] FIG. 12 illustrates the concept of FIG. [Figure 13]FIG. 13 shows an example of a portion extracting an HEVC slice segment header extended to include an enhancement layer ID field. [Figure 14] FIG. 14 shows an example of the VPS syntax extended to include a syntax element to indicate the length of the extension layer ID field of FIG. [Figure 15] FIG. 15 shows an example of a VPS syntax extended to include a syntax element indicating the maximum value of the Enhancement Layer ID field. [Figure 16a] FIG. 16a shows an example of a portion of an SEI extension syntax structure that has been extended to include an extension layer ID field. [Figure 16b] FIG. 16b shows an example of a portion of the syntax structure of an SPS extension extended to include an extension layer ID field. [Figure 16c] FIG. 16c shows an example of a portion of the syntax structure of a PPS extension extended to include an extension layer ID field. [Figure 17] FIG. 17 shows an example of a VPS syntax that signals information to layers within a video data stream by making multiple passes through the syntax portion that relies on the maximum assumption values ​​described with respect to FIGS. 11 and 12. [Figure 18] 18 shows a schematic example relationship between the Enhancement Layer ID field, the Base Layer ID field, and the Layer ID, in accordance with a particular example, using layer_id_ext to indicate the Enhancement Layer ID field, nuh_layer_id to indicate the Base Layer ID field, and layerID to indicate the Layer ID. The number of representable values ​​of the Enhancement Layer ID field is indicated by max_layer_id_ext(plus 1). Here, for the sake of example, the length of the Enhancement Layer ID field is set to 3 by selecting 2 by setting LengthOfExtension equal to 2. [Figure 19] FIG. 19 shows an example portion of a slice segment header that has been extended to include an enhancement layer ID field. [Figure 20a] FIG. 20a shows an example of the VPS syntax extended to include a syntax element indicating the length of the subfield of the base layer ID field. [Figure 20b] FIG. 20b shows an example of a portion of a modified slice segment header syntax that conditionally and explicitly signals the enhancement layer ID field. [Figure 21] FIG. 21 shows a flow chart illustrating the functionality of a network device deriving a layer ID using a relationship implemented according to the concepts of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0012] Before describing specific embodiments of the present application, an exemplary multi-layer video encoder environment is described with reference to FIG. 2, in which the embodiments and aspects outlined below may be advantageously used. Although this overview of an exemplary multi-layer video encoder environment should not be understood as limiting the embodiments outlined further below, the details and functionality provided illustratively with respect to the following figures are considered to illustrate possible extensions of the embodiments outlined further below to yield more detailed embodiments. Regardless, the multi-layer video encoder environment of the description provided below will facilitate an understanding of the advantages of the embodiments outlined below.

[0013] FIG. 2 illustratively illustrates a multi-layer video encoder 200 that receives video material 202 at an input. For illustrative purposes only, the video material 202 is described as a multi-view video signal that transports video data for each of multiple views, such as texture and depth for each image of the video. Generally, the video material 202 represents a multidimensional sampling of a scene. One dimension is time t. That is, the video material 202 samples the scene temporally, thus producing one image per time instant. Furthermore, the scene is captured from different viewpoints, thus producing one video per view, or one image per view per time instant. Furthermore, the video material 202 provides each view, or a subset of each view, with spatial sampling of the scene's texture (i.e., spatial sampling of the scene's color and depth maps for each view). FIG. 2 illustrates, for example, the distinction between depth maps, which are illustrated using hatching, and texture maps or images, which are not depicted with hatching. However, it should be noted that none of the embodiments outlined further below are limited to such multi-view material, rather the description of Figure 2 is merely provided to ease the discussion set forth below.

[0014] The multi-layer video encoder 200 is configured to encode video material 202 into a multi-layer data stream or video signal 204. In particular, the multi-layer video encoder 200 uses inter-layer prediction to encode the video material into the multi-layer data stream 204 with different layers corresponding to different information levels. This means that the multi-layer video encoder 200 generates a multi-layer data stream to include multiple packets 206, each of which is associated with a different layer. The multi-layer video encoder 200 encodes a certain base level of information content of the video material 202 (e.g., simply the texture of one view or the like) into, for example, a given base layer packet 206. For example, the association of a packet 206 with one of the various layers is derived from a layer identification syntax element structure 208 within each packet 206. For example, the layer identification syntax element structure 208 is included in a header 210 of each packet 206. For example, for purposes of illustration, the numbers 1, 2, and 3 are shown as being written in packet 206. Thus, layers with layer IDs are described by 1, 2, and 3, respectively. For example, among these layers, the layer with layer ID 1 forms the lowest or base layer, providing a base level of information. Other layer packets 206 increase the amount of information available in the base layer packet 206, either for one or more or various information types only. For example, a particular layer packet 206 may have another view coded therein in addition to the view already coded in the base layer or reference layer packet 206. Alternatively, a particular layer packet 206 may have depth information (e.g., view depth information) for a scene of the video material 202, the texture of which is already coded in the base layer or, more generally, in a lower layer, reference layer packet. Similarly, surface reflectance information is coded in a packet of another layer (i.e., spatial sampling of the scene for surface reflectance of objects in the scene, regardless of lighting conditions).Additionally, alpha blending information is coded in packets of another layer (i.e., spatial sampling of transparency of scene points corresponding to individual image samples). A specific layer packet 206 may also add specific color component information or increase spatial resolution (i.e., provide spatial resolution improvement). Similarly, a specific layer packet 206 may simply provide SNR resolution improvement, i.e., increase the signal-to-noise ratio of the coded video material. To avoid redundancy within the multi-layer data stream 204 as far as various layers are concerned, inter-layer prediction is used by the multi-layer video encoder 200. That is, a specific layer packet 206 encodes a prediction residual therein for inter-layer prediction, such that the prediction residual for the former layer is derived from one or more other layers, called reference layers. Inter-layer prediction applies to different entities, such as syntax elements and parameters, describing the video material 202. For example, the prediction residual carried by a specific layer packet 206 may be derived from one or more reference layers in the spatial domain on a pixel-by-pixel basis, correcting inter-layer prediction. Instead, the multi-layer video encoder 200 uses, for example, transform residual coding, where inter-layer prediction correction occurs in a transform domain, such as the DCT domain, on a per-transform coefficient basis. Alternatively or additionally, the multi-layer video encoder 200 is a hybrid video encoder type that uses spatial and / or temporal prediction, e.g., the inter-layer prediction additionally or alternatively involves the refinement of motion vectors. Furthermore, the multi-layer video encoder 200 uses hierarchical multi-tree subdivision to subdivide the images of the video material 202 into coding blocks in units to which different prediction modes (e.g., spatial and temporal prediction) are applied, and / or into transform blocks in units in which the transform is performed on the prediction residual, and / or into other blocks in units in which specific coding parameters are set for coding the video material 202.Then, instead of signaling subdivision information for another layer anew, either the subdivision is entirely adopted from one or more base layers, or refinements to that subdivision information are signaled.

[0015] In the manner of inter-layer dependencies between layers, the layers of the multi-layer video signal 204 are interrelated through the branches of the tree 212. The nodes of the tree 212 are formed by the layers. To transport information about inter-layer dependencies between layers (i.e., to make the information on the inter-dependency tree 212 available to the decoding side), the multi-layer video signal 204 has that information encoded therein.

[0016] 2, for example, illustrates that a multi-layer video encoder 200 intersperses within a video data stream 204 high-level syntax packets 216 that constitute or transport information 214. The information 214 includes interdependent syntax structures to describe inter-layer dependencies within the tree 212.

[0017] However, the information in tree 212 may alternatively be known or reconstructible due to defects (defaults) in the encoder and decoder. Therefore, additionally or alternatively, layer-related information 214 may include information related to some layer ID extension mechanism. In particular, while for many applications, a moderate number of layers is important for forming a multi-layered video signal 204, some other applications may suffer from a very small number of layers signaled by layer identification syntax element structure 208. That is, for those applications requiring a high number of layers, configuring layer identification syntax element structure 208 to accommodate a huge number of layers would incur a penalty in information overhead on the high side when considering the overall diversity of applications. And configuring layer identification syntax element structure 208 to accommodate only a moderate number of layers would exclude other applications from being fully supported by the multi-layered video encoder underlying data stream 204. Therefore, some extension mechanism is used to enable switching between modes of data stream 204. Here, the layer identification syntax element structure 208 only accommodates lower layer numbers, and the mode in which the layer identification syntax element structure accommodates higher layer numbers and information 214 additionally or alternatively involves switching between these modes.

[0018] Although the scope of validity of the high-level syntax packets 216 encompasses the entire multi-layered video signal 204, it is also possible for the packets 216 to belong to smaller predetermined portions, e.g., to chunks into which the multi-layered video signal 204 is temporally divided. Each chunk is a sequence of images. For example, using DASH or another adapted streaming protocol, the multi-layered video encoder 200 may change the number of layers encoded in the bitstream 204 within the just-mentioned unit of predetermined portion, the inter-layer dependencies in the form of the tree structure 212, and / or the switching between expanding and not expanding the layer identification syntax element structure 208. Regardless, periodically transmitting the high-level syntax packets 216 allows the receiver to have multiple random access time instants to begin decoding the multi-layered video signal 214 on a random access basis between them.

[0019] 2 also illustratively illustrates an apparatus for receiving a multi-layer data stream. For example, optional network element 218 receives multi-layer video data stream 204 and processes it for forwarding to a recipient of multi-layer video data stream 204. Accordingly, FIG. 2 also illustrates multi-layer video decoder 220 configured to decode multi-layer video data stream 204. Both entities, network element 218 and multi-layer video decoder 220, form an example of an apparatus for processing multi-layer data stream 204.

[0020] Network element 218 need not be able to decode multi-layer data stream 204 using inter-layer prediction between the various layers. Nevertheless, network element 218 and multi-layer video decoder 220 (i.e., the receiver) need to be informed of packets 206 associated with the various layers and the inter-layer dependencies between them, as defined by tree 212. Network element 218 may, for example, discard packets 206 of data stream 204 associated with layers for which the additional information provided is not necessarily needed in another connection between network element 218 and multi-layer video decoder 220. For example, discarding packets 206 may be necessary due to bitrate limitations within the connection or the receiver's inability to reproduce the additional information or the like. Similarly, multi-layer video decoder 220 may decide to discard some of packets 206 of a particular layer depending on external parameters such as currently available computing power, display capabilities of the playback device, such as spatial resolution, and the maximum number of input views or the like. That is, the device 218 / 220 can read the layer-identifying syntax element structure 208 of the packet 206 and derive inter-layer dependencies between layers, if any, from the information 214, and / or switch between an expanded mode or an unexpanded mode for the layer-identifying syntax element structure 208 corresponding to the information 214, and / or read other characteristics for the layer from the information 214.

[0021] Beyond that, the multi-layer video decoder 220 can reconstruct the video material 202 from the inbound (return) data stream 204 up to a given level by collecting and using information from the packets 206 associated with that layer and all layers to which that layer is interrelated by inter-prediction dependencies (see tree structure 212). That is, the multi-layer video decoder 220 uses the packets 206 of a particular layer as prediction residuals for inter-layer prediction that the multi-layer video decoder 220 derives from the packets 206 of one or more reference layers for that layer, as outlined above. In this regard, the multi-layer video decoder 220 may be a decoder that uses transform residual decoding, hybrid video decoding, hierarchical multi-tree subdivision, and / or another coding concept, as outlined above.

[0022] 2 exemplarily illustrates that multi-layer video encoder 200 and data stream 204 are provided such that packets 206 are included in data stream 204 to form consecutive access units 222 per time instant in such a way that packets 206 belonging to different time instants or pictures are not interleaved with each other, each collecting packets 206 belonging to a particular time instant of that access unit 222. As a result, packets 206 belonging to another time instant either precede or follow this access unit 222 in data stream 204. However, it should be noted that this restriction is chosen only for illustrative purposes, and that a more relaxed arrangement of packets 206 in data stream 204 may alternatively be chosen.

[0023] In the following, the possibility of signaling inter-layer dependencies according to a tree structure 212 is explained in more detail. As mentioned above, this signal includes information 214. According to this possibility, layers are grouped into clusters. In the data stream, inter-layer dependencies between one cluster and layers within another cluster are signaled separately. To that extent, the explanation presented below expresses a possibility of implementing the information 214 of FIG. 2 to signal inter-layer dependencies 212. However, as already noted above, the details explained below with respect to FIG. 3 are not limited to the details presented in FIG. 2. Rather, FIG. 2 should be seen as a possible implementation repository for the explanation with respect to FIG. 3.

[0024] 3 illustrates a network device 18 that is the network element 218 or multi-layer video decoder 220 of FIG. 2, configured to process the multi-layer data stream 10 as data stream 204 of FIG.

[0025] FIG. 3 illustrates a multi-layer data stream 10 in which video material is encoded with different layers identified by layer IDs lID. Each layer corresponds to a different information content level. Although the possibilities in this regard have already been discussed with reference to FIG. 2, for easier understanding, we consider each layer adding a specific view to the data stream 10. However, this is not intended to limit the explanation of FIG. 3. The individual layers may alternatively correspond to a combination of different measures for information content (e.g., number of views and number of spatial resolutions, or the like).

[0026] Thus, multi-layer data stream 10 is comprised of multiple packets 12, illustratively corresponding to packets 206 of FIG. 2. Such packets 12 (or 206) are substreams, allowing, for example, wavefront parallel processing of individual images encoded in video signal 10. Substreams, in turn, are comprised of smaller units, such as NAL units, slices, or the like. However, it is submitted herewith that packets 12 or 206 of FIGS. 2 and 3 may also be NAL units, slices, or other units.

[0027] Each packet 12 is associated with one of the different layers, and to reduce the bit consumption of the multi-layer data stream 10, inter-layer prediction is used such that each packet 12 simply adds "residuals" to packets of a particular subset of lower layers of the data stream 10. As indicated by the reference numeral 14, the "lower layers" are represented by small dots for each individual layer.

[0028] In accordance with the embodiment shown in FIG. 3, inter-layer prediction dependency is limited as described in more detail below and illustrated using solid line 16 in the figure. In particular, the layer ID is formed by the concatenation of two values, as described below. In particular, a network device receiving multi-layer data stream 10 reads, for each packet 12, the layer ID syntax structure shown using reference symbol 208 in FIG. 2. However, this consists of a base layer ID field 20 and an enhancement layer ID field 22 that conditionally depends—e.g., on the base layer ID field or a higher-level syntax element that switches an enhancement mechanism on or off—for example. By way of example, only packets 12 having both fields are shown in the figure. However, as noted above, one or more values ​​of base layer ID field 20 signal the absence of field 22 for an individual packet 12. For example, the entire enhancement mechanism can be turned on or off within the data stream via an enhancement flag in the data stream. Possible conditions for requiring or not requiring the Enhancement Layer ID field are, as mentioned above, that the Base Layer ID field is 0, or that a particular bit is set or not set, or that a particular value is assumed within a subpart of field 20, or that the Base Layer ID field has a value greater or less than a certain value. Further examples are presented below.

[0029] Based on the layer ID syntax structures 20 and 22, the network device 18 derives a layer ID, or lID, identifying the layer to which each packet 12 relates. Different possibilities are described below. The network device 18 also reads from the multi-layer data stream 10 a first interdependence syntax structure, e.g., including the above-identified direct_dependency_flags, and a second interdependence syntax structure, e.g., including the below-identified direct_ext_dependency_flags and / or general_direct_ext_dependency_flags. The first interdependence syntax structure indicates in a binary way the interdependencies between different pairs of values ​​representable by the base layer ID field 20. In contrast, the second interdependence syntax structure indicates in a binary way the interdependencies between different pairs of values ​​representable by the extension layer ID field. Based on both interdependence syntax structures, the network device 18 then forms an inter-layer dependency matrix, which identifies the inter-layer prediction dependencies between different layers (such as those depicted by reference numeral 14 in the figure). The derived method is described below using pseudocode using a fore-next loop relating to the first and second interdependent syntax structures, and also with respect to the following figures. However, it should be noted that the signaled inter-layer prediction dependency does not need to actually be used in the data stream. Rather, the signaling of a possible inter-layer prediction dependency is to instruct a decoder or another network device to take the necessary steps to ensure that the correlated packets, i.e., packets referenced by another packet according to the inter-layer prediction dependency prior to the reference packet, are available in the appropriate order.

[0030] As will become clear from the following description, the construction of the inter-layer dependency matrix 14 is performed in such a way that the second interdependence syntax structure is applied to all cases in which the inter-layer dependency matrix 14 is derived from the same value in the base layer ID field 20 and, consequently, relates to inter-layer prediction dependencies between the layers of the layer ID associated therewith. Also, according to another embodiment described in more detail below, the second interdependence syntax structure is read, for example, for each possible value of the base layer ID field 20 or for a subset of the possible values ​​of the base layer ID field 20, and is transmitted several times in the bitstream 10. Meanwhile, various instantiations of the second interdependence syntax structure are associated with the possible values ​​of the base layer ID field 20 using indexing, such as using included_nuh_layer_id in the example shown below. Even in another embodiment, the structure is generalized in that all possible inter-layer prediction dependencies are allowed and can be represented by the way of the second interdependence syntax structure, i.e., by transmitting the second interdependence syntax structure for each pair of different values ​​of the base layer ID field for which the first interdependence syntax structure indicates the existence of an interdependence.

[0031] In other words, as further explained in FIG. 4, FIG. 3 illustrates an example for signaling inter-layer dependencies according to which separate interdependent syntax structures 24 and 26 are used to describe inter-layer dependencies in a hierarchical manner, with the interface between the ranges of both interdependent syntax structures 24 and 26 corresponding to the interface between the ranges of base layer IDs and extension layer IDs provided by fields 20 and 22 in individual packets of the data stream. The base layer ID and extension layer IDs as provided by fields 20 and 22 uniquely define the layer ID of the individual packets of which fields 20 and 22 are comprised. The set of all representable layer IDs representable by the combination of extension layer IDs and base layer IDs is indicated by dots 30 within circle 28. That is, each dot 30 corresponds to a different couple of base layer IDs and extension layer IDs. For example, a layer ID is a concatenation of base layer IDs and extension layer IDs. Using the base layer IDs provided by field 20 alone, the complete set 28 of layer IDs 30 is subdivided into disjoint sets 32 of layer IDs, hereinafter referred to as clusters 32. All layer IDs belonging to a particular cluster 32 have the same base layer ID. As previously described with respect to Figure 2, the layers associated with layer IDs 30 are interconnected in a tree-like manner for inter-layer prediction, with these inter-layer dependencies illustrated in Figure 4 using dotted lines between dots 30. For ease of understanding, only a subset of the actual inter-layer dependencies are illustrated in Figure 4.

[0032] In any case, the connectivity between two clusters 32 (by way of inter-layer dependencies between the layer of the first cluster 32 and the layer of the second cluster 32 of the pair) is indicated by way of a first interdependent syntactic structure 24. That is, the first interdependent syntactic structure 24 describes inter-layer dependencies coarsely or cluster-wise. In FIG. 4, these interdependencies are illustrated using solid lines between the clusters 32. All pairs of first and second clusters 32 in which at least one layer of the first cluster is connected to one cluster in the second cluster are interconnected and represented in the first interdependent syntactic structure 24. The second interdependent syntactic structure 26 then clarifies which layers of the pair of clusters 32 indicated as interconnected by the first interdependent syntactic structure 24 are actually related to each other by inter-layer prediction. That is, the second interdependent syntactic structure 26 clarifies fine-granular interdependencies. However, the second interdependent syntactic structures 26 also individually define interdependencies between layers within individual clusters 32, i.e., intra-cluster dependencies between layers. In FIG. 4, for example, six clusters 32 are shown, thus resulting in six clusters for which the second interdependent syntactic structures 26 regulate interdependencies internally, in addition to the 15 possible pairs of clusters 32 for which the second interdependent syntactic structures 26 regulate fine-grained interdependencies between layers of other clusters 32. As outlined above and further below, therefore, there are a maximum of 1 to 21 instances 34 of the second interdependent syntactic structures, i.e., one per cluster individually and one per linked pair of clusters 32. Clearly, the second interdependent syntactic structures 26 do not need to be signaled for pairs of clusters 32 signaled as unlinked by the first interdependent syntactic structures 24, thus conserving valuable side information bitrate. When using the second interdependency syntax structure 26 to describe the inter-layer dependencies of different pairs of layers in a cluster 32, it should be noted that the number of layers per cluster 32 is equal for all clusters 32. This is typically the case if the layer ID is described using a concatenation of the base layer ID and the enhancement layer ID. However, it should be noted that the number of layers in a cluster 32 can theoretically vary between clusters 32.In a typical example, a separate instance 34 of the second interdependent syntactic structure 26 is signaled for each interconnected pair of clusters 32, for example in the data stream, with at least one instance 34 being transmitted for each cluster size.

[0033] FIG. 3 illustrates a case where the layer ID is derived from the base layer ID and the enhancement layer ID by using the base layer ID as the most significant digit and the enhancement layer ID as the less significant digit. FIG. 3 also illustrates an exemplary case where one instance 34 is used to describe inter-layer dependencies of layers within one cluster and another instance 34 is used to describe inter-layer dependencies of layers in a different cluster. For completeness, it is noted that the matrix 14 of FIG. 3 has many lines and many columns for layer IDs. Only the lower half of the diagonal is filled in because every layer depends on the previous layer (i.e., a hierarchically lower layer) simply by using inter-layer prediction. In the example of FIG. 3, the column numbers correspond to the layer IDs of layers that depend on another layer (i.e., the base layer) by using inter-layer prediction. These base layers are indicated, for example, by binary numbers. The layer ID corresponds to the current column, while a binary zero indicates a layer that is not involved in the inter-layer prediction of the individual layer. To that extent, in the embodiment of FIG. 3, the second interdependent syntactic structure 26 somewhat describes a sub-matrix of the matrix 14 .

[0034] As already described above, the first interdependent syntactic structure 24 and the second interdependent syntactic structure 26 are constituted by information 214 in high-level packet 216 (compare Figure 2). Figure 5 illustrates an example in which the first interdependent syntactic structure 24 reveals interdependencies between layer clusters depicted at 36. For example, a cluster with base layer ID 2 depends on clusters with base layer IDs 2 and 1.

[0035] Also present in the data stream is a first instance 34 of a second interdependent syntax structure, accommodating the intra-cluster dependencies between layers depicted in Figure 5 in the form of a sub-matrix 38. Furthermore, according to the example of Figure 5, the data stream also includes an instance 34 of a second interdependent syntax structure 26, accommodating the layer-to-layer interdependencies of layers of different clusters. In particular, the second instance describes the dependencies between layers of different clusters via a sub-matrix 40 having one row per enhancement layer ID of the referenced cluster and one column per enhancement layer ID of the referencing cluster.

[0036] In the example of Figure 5, sub-matrix 38 is placed in each position where matrix 36 indicates interdependence between clusters (i.e., each position where a 1 is placed) and the clusters are clusters of the same base layer ID (i.e., clusters on the diagonal of matrix 36). Sub-matrix 40 is then placed in each position where matrix 36 indicates interdependence between clusters of different base layer IDs with a "1". The result is shown at 42.

[0037] It should be noted that the description of inter-layer dependencies via matrices such as matrix 42 is merely one example for describing inter-layer dependencies. Alternative descriptions may be used as well. The manner in which matrices 36-40 are encoded by the first and second interdependent syntax structures is as follows: first interdependent syntax structure 24 signals a binary value for each coefficient of matrix 36 below and including the diagonal; instance 34 of second interdependent syntax structure 26, which denotes matrix 38, signals a binary value for each coefficient of matrix 38 below and including the diagonal; instance 34 of second interdependent syntax structure 26, which denotes matrix 40, signals a binary value for all coefficients of matrix 40.

[0038] After describing the possibilities for how to signal inter-layer dependencies, a more detailed example is exemplarily provided below as an extension to the HEVC standard.

[0039] In particular, layer identification clustering and cluster-dependent signaling are constructed as follows within the existing encoders.

[0040] Two syntax elements, nuh_layer_id and layer_id_ext, are used to group the encoded layers in the video bitstream into clusters that encode dependencies or other things based on characteristics such as their so-called spatial relationship to each other. The structuring of layer identifiers into clusters is allowed for clusters with equally structured encoding dependencies within an individual cluster. That is, the encoding dependencies within all or a subset of the defined clusters are the same. To determine the reference layer of a particular layer, signaling the dependencies within the cluster as an additional set of dependency flags and combining the dependencies with the existing dependency signaling (compare with direct_dependency_flag in the VPS extension of the HEVC extension) is done as described in FIG. 6.

[0041] A direct_ext_dependency_flag[i][j] equal to 0 specifies that the layer with extension index j is not a direct reference layer to the layer with extension index i within the same layer cluster. A direct_dependency_flag[i][j] equal to 1 specifies that the layer with extension index j is a direct reference layer to the layer with extension index i within the same cluster. When direct_ext_dependency_flag[i][j] does not exist for i and j within the range from 0 to (1<<layer_id_ext_len)-1, it is assumed to be equal to 0.

[0042] The alternative syntax restricts the loop of the direct_ext_depencency_flag syntax element by max_layer_id_ext to avoid signaling encoding dependencies on unused layer identifiers.

[0043] More flexible signaling of coding dependencies within and between clusters is allowed as follows. a) By a signal that encodes intra-cluster dependencies on a cluster-by-cluster basis. b) By defining the number of cluster dependency sets and identifying the clusters to which the defined cluster dependency sets apply (e.g., one or the other by nuh_layer_id). c) By hierarchical signaling of dependencies in a way that primary dependencies between clusters are signaled and secondary dependencies at layers within two dependent clusters are signaled. d) By a combination of b) and c). Therefore, the following data is signaled: 1. Dependencies between clusters (e.g., pairs of references and dependent clusters). 2. The number of cluster dependency sets that define the dependencies between layers in the reference (referenced) cluster and the dependent (referencing) cluster. 3. For each cluster dependency set, an indication of which cluster dependencies apply (as signaled in 1). Note that for c) and d) a cluster can depend on itself when there are dependencies between layers within the cluster. a) to d) allow structuring the space of layer identifiers when the feature space used for structuring (e.g., the spatial position of the cameras) is not uniformly used. An example is a two-dimensional camera array with non-constant spatial density, or a non-rectangular two-dimensional camera array. The following gives example syntax for signaling the definition of cluster dependency sets in several ways according to the above.

[0044] a) Figure 7 gives an exemplary embodiment of a video parameter set extension syntax that defines encoding intra-cluster dependencies on a per-cluster basis.

[0045] direct_dependency_flag[i][j] equal to 0 specifies that the layer or cluster with index j is not a direct reference layer or reference cluster for the layer or cluster with index i. direct_dependency_flag[i][j] equal to 1 specifies that the layer or cluster with index j is a direct reference layer or reference cluster for the layer or cluster with index i. When direct_dependency_flag[i][j] is in the range 0 to vps_max_layers_minus1 and is not present for i and j, it is inferred to be equal to 0. general_direct_ext_dependency_flag[i][j] equal to 0 specifies that the layer with extension index j is not a direct reference layer for the layer with extension index i in the same layer cluster. general_direct_ext_dependency_flag[i][j] equal to 1 specifies that the layer with extension index j is a direct reference layer for the layer with extension index i in the same cluster. When general_direct_ext_dependency_flag[i][j] is not present for i and j in the range 0 to max_layer_id_ext, it is inferred to be equal to 0. direct_ext_dependency_flag[i][j][k] equal to 0 specifies that the layer with extension index k is not a direct reference layer for the layer with extension index j in the ith layer cluster. direct_ext_dependency_flag[i][j][k] equal to 1 specifies that the layer with extension index k is a direct reference layer for the layer with extension index j in the ith layer cluster. When direct_ext_dependency_flag[i][j][k] is not present for i, j, and k in the range 0 to max_layer_id_ext, it is inferred to be equal to 0.

[0046] b) Figure 8 gives an exemplary embodiment of syntax in a video parameter set extension that defines encoding intra-cluster dependencies through a signal of the number of cluster dependency sets (number_of_cluster_dependency_sets), encoding dependencies within individual cluster dependency sets (direct_ext_dependency_flag), and applying a cluster (e.g., a cluster identified by a given syntax element included_nuh_layer_id through a positive flag for the cluster's individual nuh_layer_id).

[0047] c) Figure 9 gives an exemplary embodiment for hierarchical signaling of dependencies, in a way that primary dependencies between clusters are signaled and only secondary dependencies between layers within two dependent clusters are signaled.

[0048] direct_dependency_flag[i][j] equal to 0 specifies that the layer or cluster with index j is not a direct reference layer or reference cluster for the layer or cluster with index i. direct_dependency_flag[i][j] equal to 1 specifies that the layer or cluster with index j is a direct reference layer or reference cluster for the layer or reference cluster with index i. When direct_dependency_flag[i][j] is in the range 0 to vps_max_layers_minus1 and is not present for i and j, it is inferred to be equal to 0. Note: If layer_id_ext_len is greater than 0, then direct_dependency_flag[i][j] signals a dependency between clusters. Otherwise (if layer_id_ext_len is equal to 0), direct_dependency_flag[i][j] signals a dependency for layers.

[0049] direct_ext_dependency_flag[i][j][k][l] equal to 0 specifies that the lth layer in the jth cluster is not a direct reference layer for the kth layer in the ith cluster. direct_ext_dependency_flag[i][j][k][l] equal to 1 specifies that the lth layer in the jth cluster is a direct reference layer for the kth layer in the ith cluster. When direct_ext_dependency_flag[i][j][k][l] is not present, it is inferred to be equal to 0.

[0050] d) Figure 10 gives an exemplary embodiment for the combination of b) and c).

[0051] direct_dependency_flag[i][j] equal to 0 specifies that the layer or cluster with index j is not a direct reference layer or reference cluster for the layer or cluster with index i. direct_dependency_flag[i][j] equal to 1 specifies that the layer or cluster with index j is a direct reference layer or reference cluster for the layer or reference cluster with index i. When direct_dependency_flag[i][j] is in the range 0 to vps_max_layers_minus1 and is not present for i and j, it is inferred to be equal to 0. Note: If layer_id_ext_len is greater than 0, then direct_dependency_flag[i][j] signals a dependency between clusters. Otherwise (if layer_id_ext_len is equal to 0), direct_dependency_flag[i][j] signals a dependency for layers.

[0052] direct_ext_dependency_flag[n][k][l] equal to 0 specifies that the lth layer in the jth cluster is not a direct reference layer for the kth layer in the ith cluster when dependency_set_applies_flag[n][i][j] is equal to 1. direct_ext_dependency_flag[n][k][l] equal to 1 specifies that the lth layer in the jth cluster is a direct reference layer for the kth layer in the ith cluster when dependency_set_applies_flag[n][i][j] is equal to 1. dependency_set_applies_flag[n][i][j] equal to 0 specifies that the dependency between layers in the ith cluster and layers in the jth cluster is not specified by direct_ext_dependency_flag[n][k][l]. dependency_set_applies_flag[n][i][j] equal to 1 specifies that the dependency between layers in the ith cluster and layers in the jth cluster is specified by direct_ext_dependency_flag[n][k][l]. When dependency_set_applies_flag[n][i][j] is not present, it is inferred to be equal to 0. When dependency_set_applies_flag[n][i][j] is equal to 0 for a particular combination of i, j, and all n in the range 0 to (number_of_cluster_dependency_sets-1), it implies that layers in the jth cluster are not direct reference layers for any layers in the ith cluster.

[0053] Thus, in the above example of implementing the cluster-based interdependency signaling of Figures 2 to 5 within the HEVC extension, packets 206 in Figure 2 are slices or slice segments, and packets 216 are parameter sets, e.g., video, picture, or continuous parameter sets. According to the syntax example of Figure 7, a first interdependency syntax structure 24 is signaled for each pair of clusters i, j, where i < j, using the syntax element direct_dependency_flag, where vps_max_layers_minus1 is the maximum value of base layer ID - 1. Thus, the syntax element reveals, for example, the coarse matrix 36 of Figure 5. A syntax element 44 called unique_cluster_dependencies_flag distinguishes between two options: either one instance of the second interdependency syntax structure 26 is submitted and applied to all base layer IDs, or another instance is sent for each individual base layer ID i. In the first case, the flag general_direct_ext_dependency_flag is used to construct a sub-matrix according to code 38 for each base layer ID. In the second case, the flag direct_ext_dependency_flag builds a sub-matrix according to code 38 for each base layer ID separately. The unique_cluster_dependencies_flag switches between both options. Thus, in the case of FIG. 7, if unique_cluster_dependencies_flag is equal to 0, the resulting sub-matrix 38 is placed in each position in matrix 36 that corresponds to the referencing and referenced cluster IDs, where 1 is indicated by direct_dependency_flag. The remaining positions, where the first interdependency syntax structure 24 indicates an interdependency between clusters of different cluster IDs, are filled in by default using a default sub-matrix (i.e., a sub-matrix known by all participating devices, such as encoders, network elements, and multi-layer video decoders).In the case of unique_cluster_dependencies_flag equal to 1, for each position where the first interdependency syntax structure 24 indicates a 1 at the position of the equal referenced and referencing cluster ID, a separate sub-matrix for this very cluster ID is signaled and placed at that position in matrix 42. Thus, max_layer_id_ext corresponds to the number of layers within each of the vps_max_layers_minus1 clusters.

[0054] In the case of Figure 8, the first interdependency syntax structure 24 is sent in the same way as in the example of Figure 7: for each cluster ID, one flag per cluster ID is equal to or less than each cluster ID. The flag cluster_dependency_sets_present_flag 46 switches between the first option of Figure 7 using the syntax element general_direct_ext_dependency_flag already outlined above. A different mode according to which the number of instances of the second interdependency syntax structure 26 are submitted in the data stream using the syntax element direct_ext_dependency_flag is explicitly signaled using a syntax element called number_of_cluster_dependency_sets. These instances are then associated with the cluster IDs to which they apply by explicitly listing the cluster IDs to which each instance applies using the syntax element included_nuh_layer_id 50. By this means, instances 34 of the second interdependency syntax structure 26 are not submitted for each cluster ID. Rather, only a smaller number of instances of the second interdependency syntax structure 26 are submitted if cluster_dependency_sets_present_flag is equal to 1. Each instance is associated with a cluster ID associated with the instance, and as far as interdependent pairs of different clusters are concerned, defect rules are applied to fill the matrix 42 at such positions (below the cluster diagonal).

[0055] According to the example of Figure 9, the signaling of the second interdependent syntax structure is illustratively signaled in an interleaved manner with respect to the first interdependent syntax structure. However, this interleaving may be omitted according to alternative embodiments. Regardless, in the embodiment of Figure 9, a sub-matrix 38 or 40 (i.e., an instance 34 of the second interdependent syntax structure 26) is submitted for each pair of cluster IDs (equal or unequal) for which the first interdependent syntax structure 24 indicates inter-cluster dependency. As can be derived from the syntax example, transmitting sub-matrix 40 consumes more bits of direct_ext_dependency_flag than sub-matrix 38. This can be derived from conditional clauses 52 and 54, respectively. That is, in transmitting an instance of the second interdependent syntax structure 26 for an intra-cluster interdependence rule, the referencing cluster ID indicated by counter i is equal to the referenced cluster ID indicated by counter j, and the direct_ext_dependency_flag is transmitted only for the reference and referencing layers (where the enhancement layer ID of the referenced layer (here l) is less than the enhancement layer ID of the referenced layer (here k)) due to the combination of enhancement layer IDs. That is, the direct_ext_dependency_flag of the sub-matrix 38 is transmitted only for positions below the diagonal of the sub-matrix 38. In another case, i.e., for an instance of the second interdependent syntax structure 26 for an interdependent cluster pair i≠j, the complete sub-matrix is ​​transmitted. That is, the flag direct_ext_dependency_flag is transmitted for each position of the sub-matrix 40.

[0056] The embodiment of Figure 10 has a particular property of the embodiments of Figures 8 and 9: it combines complete sub-matrices. That is, instances of the second interdependent syntactic structure 26, i.e., number_of_cluster_dependency_sets in number, are submitted, and then each of these instances is associated with a grid site of matrix 36. Here, for example, individual instances of the second interdependent syntactic structure 26 are applied between them to indicate the existence of cluster independence (compare reference numeral 56).

[0057] Put another way, as described with respect to FIG. 9, the plurality of instances 34 of the second interdependent syntax structure 26 are read from the multi-layer data stream, that is, for pairs (j, k) of values 0 < k ≦ j ≦ n < N where n is set to be equal to N, which are numerical values that can be represented by the base layer ID. However, also as described herein, explicit signaling of the actually used cluster density is used to limit the transmission overhead. The pairs (j, k) are scrutinized (see the loop over i and j at reference numeral 24 in FIG. 9), and reading of the second interdependent syntax structure is performed or suppressed for pairs (j, k) that are conditionally dependent on the first interdependent syntax structure that indicates the presence or absence of interdependence between these pairs (see "if(direct_dependency_flag[i][j])" in FIG. 9, where i and j correspond to j and k respectively). The instance 34 of the second interdependent syntax structure 26 read for pairs (j, k) where j = k indicates that the layer between them with base layer ID j is an inter-layer prediction that depends on another layer between them with base layer ID j. The instance 34 of the second interdependent syntax structure 26 read for different value pairs (j, k) where j > k indicates that the layer between them with base layer ID j is an inter-layer prediction that depends on the layer between them with base layer ID k. However, as in FIG. 7, one instance of the second interdependent syntax structure 26 may also be read from the multi-layer data stream. One instance of the second interdependent syntax structure 26 commonly indicates for individual cluster value pairs (j, j) that the layer between them with base layer ID j is an inter-layer prediction that depends on another layer between them with base layer ID j. In the case of FIG. 7, if the unique_cluster_dependencies_flag is equal to 0, or is common for pairs (j, k) where j ≠ k, the first interdependent syntax structure 24 indicates the presence of interdependence between individual pairs. The layer between them with base layer ID j is an inter-layer prediction that depends on the layer between them with base layer ID k.Let the value that can be represented by the extended layer ID 22 be denoted as M. The second interdependent syntax structure includes one binary value for each individual pair (p, q) where 0 < q < p ≤ m < M, in order to indicate that the layer between those with base layer ID i depends on another layer between those with base layer ID i. And, in order to indicate that the layer with extended value p and base layer ID j depends on the layer with extended value q and base layer ID k, it includes one binary value for each individual pair where 0 < p, q ≤ m < M. Also, as described elsewhere in the specification, m is set to be equal to M. However, also as described here, explicit signaling of the actually used extended density is used to limit the transmission overhead. Further, as shown in FIGS. 8 and 10, the index syntax structure 50 is used to associate an example of the second interdependent syntax structure with individual pairs (j, k) of base layer ID values. Here, FIG. 10 reveals that the associated syntax structure includes a flag for each pair (j, k), and the structural analysis of the associated syntax structure includes skip pairs (j, k) where 0 < k ≤ j ≤ n < N for which the value indicating the absence (independence) of interdependence between pairs (j, k) in the first interdependent syntax structure is included.

[0058] That is, what was described in the above section is presented as "layer identifier clustering and cluster-dependent signaling". In this regard, the apparatus is a video decoder. A video decoder operating in this way can select packets of the bitstream identified by the (extended) layer ID for decoding purposes. However, the apparatus could alternatively be a network element that can discard packets of the bitstream identified by the (extended) layer ID based on external factors such as network utilization (knowledge about decoders, etc.).

[0059] Naturally, the concepts outlined above also apply to existing encoders of coders to be improved by the concepts. An encoder such as the one in FIG. 2 is configured to encode video material into a multi-layer data stream 10 using inter-layer prediction, with different layers corresponding to different information content levels. The multi-layer data stream includes multiple packets 12, each associated with a different layer. Each layer is indexed by a base layer ID 20 or a base layer ID 20 and an enhancement layer ID 22 and inserted into the multi-layer data stream. A first interdependence syntax structure 24 indicates the interdependence between different pairs of values ​​that can be represented by the base layer ID 20. And a second interdependence syntax structure 26 indicates the interdependence between different pairs of values ​​that can be represented by the enhancement layer ID 22. By configuring the first and second interdependence syntax structures in this way, an inter-layer dependency description 14 can be constructed based on the first and second interdependence syntax structures, which reveals possible inter-layer prediction dependencies between different layers.

[0060] The details described next relate to the possibility of signaling layer-related information in an efficient manner. Such layer-related information includes, for example, information regarding inter-layer dependencies as outlined with respect to FIGS. 2 through 10. As a starting point, the situation shown above is often used, according to which the layer identification syntax element structure 208 (compare FIG. 2) formed by each packet 206 consists of a base layer ID field 108 and an enhancement layer ID field 112 depicted in FIG. 11. FIG. 11 shows an apparatus 100 for processing a multi-layer data stream, such as the network element 218 or the multi-layer video decoder 220 shown in FIG. 2. The apparatus 100 is shown to include a receiver 102 configured to receive a multi-layer video signal consisting of a series of packets (each of which includes a layer identification syntax element).

[0061] The multi-layer video signal 104 (corresponding to 204 in FIG. 2) received by the receiver 102 includes a series of packets 106 (corresponding to 206 in FIG. 2). Each packet 106 includes a layer identification syntax element structure (corresponding to 208 in FIG. 2) that includes at least a base layer ID field 108, and is either unconditionally or conditionally dependent on the value of a higher level syntax element or field, such as the base layer ID field 108 or the enhancement layer ID field 112.

[0062] For example, packet 106 may be a NAL unit, slice, tile, picture, or another packet into which multi-layered video signal 104 is subdivided. Also included in device 100, layer identification extender 110 reads a maximum syntax element from the multi-layered video signal, indicating the maximum hypothetical value of an extended layer ID field 112 of packet 108 that includes a predetermined portion 113 of the multi-layered video signal, such as portion 113, for a predetermined portion of the multi-layered video signal that includes multiple packets 106. The predetermined portion 113 may be, for example, a coded video sequence, a chunk, a group of pictures, or the like. The maximum syntax element 114 is included within a particular packet of portion 113, such as a VPS NAL unit. For each packet within the predetermined portion 113, extender 110 determines a layer ID for the individual packet based on a layer ID syntax structure consisting of code 108 and, conditionally, code 112. For example, the concatenation of the values ​​of both syntax elements 108 and 112 yields the layer ID.

[0063] The extender 110 derives the bit length of the Enhancement Layer ID field using the maximum syntax element described above. Alternatively, a special syntax element may be used at this end. Alternatively, the number of bits to represent the Enhancement Layer ID field 112 is fixed by default.

[0064] The extender 110 then determines the maximum number of layers in the portion 112 based on the maximum hypothesis value. For example, for portion 113, the extender 110 also reads a syntax element from the data stream 104 that indicates the maximum hypothesis value for the base layer ID field 108 of the packets 106 in portion 113, and combines both maximum hypothesis values ​​to determine the maximum number of layers.

[0065] The extender 110 determines at least one characteristic for each of the maximum number of layers by parsing the layer characterization syntax portion of the multi-layer video signal a number of times equal to the maximum number of layers. Advantageously, the data stream 104 does not need to signal at least one characteristic for each possible value (i.e., not for the entire cardinality of the enhancement layer ID field 112, but for the actually used subportion outside of this cardinality). Furthermore, the maximum syntax element 114 is even used to derive the number of representation bits of the enhancement layer ID field 112 in the data stream 104, as just described. For example, a "characteristic" could be inter-layer prediction relative to another layer or the like.

[0066] 11, it is explained that when considering all packets 106 / 206 in a data stream 204 / 104 within a predetermined portion of importance 113, it is possible to send explicit signaling of how many out of the expressible states of the Enhancement Layer ID field are actually assumed. For example, if the Enhancement Layer ID field 112 is an N-bit field, then 2 N Enhancement layer IDs 120 can be represented by the enhancement layer ID field 112. The length of field 112 is either fixed or signaled and set in the data stream by way of predetermined high-level syntax elements, shown using dotted lines 122 in FIG. 12. However, for some reasons, not all of the available enhancement layer IDs are used within the predetermined portion of significance 113. For example, in FIG. 12, only seven representable enhancement layer IDs are depicted, i.e., by way of hatching.

[0067] According to FIG. 11, max_layer_id_ext indicates this number of actually used / assumed values ​​of the enhancement layer ID field 112. This syntax element 114 allows for layer-specific information to be inserted into the data stream only for the actually assumed number of enhancement layer IDs, rather than for the complete number of representable values ​​of the enhancement layer ID field 112. Thus, as illustrated by reference numeral 122 in FIG. 12, a device parsing a multi-layer video signal may parse a given layer characterization syntax portion 124 of the multi-layer video signal a number of times equal to the maximum number of layers indicated by syntax element 114, rather than the complete number of actually available / representable values ​​of the enhancement layer ID field. The amount of side information consumed by the data stream is therefore kept lower. For example, in the examples of FIGS. 6 to 10, rather than setting this value (max_layer_id_ext) equal to the maximum number of representable values ​​of the enhancement layer ID field 112, it may be possible to set max_layer_id_ext equal to the assumed maximum number of extension states indicated by syntax element 114. Thus, the "characterizing syntax part" 124 is specified by the previously submitted flag direct_ext_dependency_flag.

[0068] As already outlined above, the presence / signaling of the bit length of field 112 is optional. In any case, the granularity / fidelity with which syntax element 114 indicates the number of actually used / active enhancement layer IDs is increased compared to a more coarse setting of the number of available / representable values ​​of enhancement layer ID field 112 set by syntax element 122. Preferably, the fidelity or granularity with which syntax element 114 indicates the number of actually assumed values ​​of enhancement layer ID field 112 is quite sufficient to indicate the exact number of actually assumed values. Alternatively, a fidelity / granularity somewhere in between and a fidelity / granularity of signaling the maximum number of available / representable values ​​of field 112 provided by syntax element 122 can be achieved as well. That is, the maximum syntax element 114 is (n-1), with n being the bit length of the enhancement layer ID field. 2 Indicates the maximum hypothetical value of the Enhancement Layer ID field 108 in units of less than or equal to.

[0069] 11 and 12, an apparatus such as a network element or a multi-layer video decoder is described that includes a receiver 102 and an extender 110. The receiver 102 is configured to receive a multi-layer video signal 104 consisting of a series of packets 106. Each of the packets 106 includes a layer identification syntax element structure consisting of a base layer ID field and, possibly only conditionally, an enhancement layer ID field 112. The layer identification extender 110 is configured to read a maximum syntax element 114 from the multi-layer video signal 104 for a predetermined portion 113 of the multi-layer video signal that includes multiple packets outside the series of packets 106. The maximum syntax element 114 indicates the maximum hypothetical value of the enhancement layer ID field 112 among the packets 106 composed of the predetermined portion 113. The extender 110 determines a layer ID for each packet 106 based on the layer identification syntax element structure, such as by concatenating the base layer ID and the enhancement layer ID in fields 108 and 112 outlined above for each packet 106 in the predetermined portion 113. The extender 110 also determines a maximum number of layers in the predetermined portion 113 based on a maximum hypothesis value, and for each maximum number of layers, determines at least one characteristic by parsing the layer characterization syntax portion 104 a number of times equal to the maximum number of layers. The "maximum number of layers" indicates the maximum number of layers per cluster when the embodiments of Figures 11 and 12 are combined with the embodiments outlined above with respect to Figures 2-10. In this case, the determination of the "maximum number of layers" directly adopts, for example, the maximum hypothesis value as the "maximum number of layers." The "at least one characteristic" is the number of flags indicating inter-layer dependencies in the second interdependency syntax structure. However, another example of a characteristic is the signaling in the data stream of matching coding parameters set for each layer ID. In that case, the "maximum number of layers" is determined to be equal to the number of used or representable base layer IDs times the maximum assumption value, i.e., the number of enhancement layer IDs actually used / assumed. Other examples are possible as well.In any case, the transmission of the actual assumed values ​​by means of syntax element 114 makes it possible to conserve valuable side information bitrate.

[0070] The concepts of Figures 11 and 12 may be combined with the concepts outlined above for Figures 2-10, as already outlined above, or may be used alone without the previously described concepts.

[0071] However, similar to the above explanations previously presented for Figures 2-10, the following explanations will clarify how the concepts of Figures 11 and 12 can be built into existing video encoders, such as extensions to HEVC.

[0072] In HEVC, a layer identifier variable (called LayerID) is derived from the video and metadata packet headers, e.g., nuh_layer_id signaled in each chunk of video or metadata packets, and additional information explicitly signaled layer identifiers to form a new variable LayerId of the particular data packet to be used for its identification.

[0073] The additional information signaled according to Figures 11 and 12 includes an additional syntax element (compare layer_id_ext within some portion of the header or payload of a video or metadata packet) that indicates that a particular video or metadata packet belongs to an enhancement layer and therefore maintains compatibility with only the base layer for legacy (old and less functionally capable, but prevalent and irreplaceable) devices. When a video or metadata packet belongs to the base layer but is processed by an enhancement layer processing capable device, a default value is assumed for layer_id_ext.

[0074] Figure 13 provides an example for a slice segment header signaling additional information as syntax element layer_id_ext in a slice header contained in a video data packet of an enhancement layer (compare nuh_layer_id>0) of HEVC coded video.

[0075] A mathematical operation is used to combine nuh_layer_id and layer_id_ext into a LayerId that allows for a larger identifier space and is specific to a particular combination of the two values ​​of nuh_layer_id and layer_id_ext. The mathematical operation is, for example, as follows: the bits of nuh_layer_id are used as the MSBs of the LayerId variable and the bits of layer_id_ext are used as the LSBs of the LayerId variable (or vice versa):

[0076] The variable LayerId is determined as follows: where LengthOfExtension is the length of the layer_id_ext syntax element in bits.

[0077] LayerId=(nuh_layer_id< <LengthOfExtension)+layer_id_ext

[0078] The concepts of Figures 11 and 12 involve explicit signaling of the amount of bits used to convey additional information in some portion of the header or payload of a video or metadata packet (compare LengthOfExtension above) in order to efficiently utilize the amount of data sent. Figure 14 gives an example syntax and semantics of LengthOfExtension values ​​signaled in a metadata packet such as the HEVC Video Parameter Set Extension Syntax.

[0079] layer_id_ext_len indicates the number of bits used to extend the LayerId range.

[0080] The concepts of Figures 11 and 12 include explicit signaling of the maximum value of the layer_id_ext syntax element used in the coded video bitstream (e.g., max_layer_id_ext). Based on that, the lowest possible amount of bits required to convey additional information in the header or some part of the payload of a video or metadata packet (compare LengthOfExtension above) is derived to efficiently utilize the amount of data sent. Figure 15 exemplarily gives the syntax and semantics of the maximum value of the layer_id_ext syntax element signaled in a metadata packet such as the HEVC Video Parameter Set Extension Syntax.

[0081] max_layer_id_ext indicates the maximum value of the layer_id_ext syntax element among slice headers in the coded video sequence. Two exemplary variations of the meaning of the layer_id_ext syntax element depending on the above extensions of the invention are given below. layer_id_ext is used in combination with nuh_layer_id to identify a layer. The syntax element layer_id_ext is encoded using layer_id_ext_len bits. When not present, its value is inferred to be 0. layer_id_ext is used in combination with nuh_layer_id to identify a layer. The syntax element layer_id_ext is Ceil(Log2(max_layer_id_ext)) bits coded. When not present, its value is inferred to be 0.

[0082] The layer identifiers of the parameter sets or SEI messages of the HEVC video encoder standard are extended without structural analysis of the dependencies given in Figures 16a to 16c.

[0083] In a layered coding scenario, the signaling related to the layer identifier or its maximum number is adapted to cover the extension layer identifier space (compare the value range of LayerId) instead of the regular explicit layer identifier space (compare the value range of nuh_layer_id), as exemplarily given in the syntax table for video parameter set extensions in Figure 17, where the variable VpsMaxNumLayers is exemplarily given as follows: The variable VpsMaxNumLayers, specifying the maximum number of layers in the range of extension layers, is set equal to (vps_max_layers_minus1)*2^layer_id_ext_len.+1, or The variable VpsMaxNumLayers specifies the maximum number of layers in the range of extended layers (vps_max_layers_minus1). * Set equal to max_layer_id_ext.+1.

[0084] layer_id_in_nalu[i] specifies the value of the LayerId value associated with the VCL NAL unit of the i-th layer, for i in the range from 0 to VpsMaxNumLayers-1, inclusive. When not present, the value of layer_id_in_nalu[i] is inferred to be equal to i.

[0085] layer_id_in_nalu[i] is greater than layer_id_in_nalu[i-1] when i is greater than 0. For i in the range 0 to vps_max_layers_minus1, inclusive, the variable LayerIdInVps[layer_id_in_nalu[i]] is set equal to i.

[0086] dimension_id[i][j] specifies the identifier of the jth current scalability dimension type in the ith layer. The number of bits used to represent dimension_id[i][j] is dimension_id_len_minus1[j]+1 bits. When dimension_id[i][j] does not exist for j in the range 0 to NumScalabilityTypes-1, inclusive, dimension_id[i][j] is ((layer_id_in_nalu[i]&((1<<dimBitOffset[j+1])?1))> >dimBitOffset[j]).

[0087] The variable ScalabilityId[i][smIdx] specifying the smIdx-th scalability dimension type identifier of the i-th layer, the variable ViewId[layer_id_in_nalu[i]] specifying the view identifier of the i-th layer, and the variable ViewScalExtLayerFlag specifying whether the i-th layer is a view scalability extension layer are derived as follows: for(i=1;i<=VpsMaxNumLayers;i++){ lId=layer_id_in_nalu[i] for(smIdx=0, j=0;smIdx<16;smIdx++) if(i!=0)&&scalability_mask[smIdx]) ScalabilityId[i][smIdx]=dimension_id[i][j++] else ScalabilityId[i][smIdx]=0 ViewId[lId]=ScalabilityId[i][0] ViewScalExtLayerFlag[lId]=(ViewId[lId]!=ViewId[0]) }

[0088] Naturally, the concepts outlined above also apply to existing encoders of coders improved by the concepts. An encoder such as that shown in FIG. 2 is configured to encode a video signal into a multi-layered video signal 104 consisting of a series of packets 106. Each of the packets 106 includes a layer identification syntax structure consisting of a base layer ID field 108 and an enhancement layer ID field 112. The encoder is further configured to, for a predetermined portion 113 of the multi-layered video signal including a plurality of packets, insert a maximum syntax element 114 into the multi-layered video signal to indicate a maximum hypothesized value of the enhancement layer ID field of the packet including the predetermined portion 113 of the multi-layered video signal, determine a maximum number of layers in the predetermined portion of the multi-layered video signal based on the maximum hypothesized value, and signal at least one characteristic in the multi-layered video signal 104 by writing, for each of the maximum number of layers, a layer characterization syntax portion 124 of the multi-layered video signal a number of times equal to the maximum number of layers. For example, the encoder postpones outputting the stream 104 until it has finished encoding the portion 113.

[0089] The following figures address the concept of signaling layer IDs of packets included in a multi-layer data stream, and the possibility of achieving efficient signaling of this layer ID using the extension mechanism without resolving backward compatibility, when, for any reason, it is necessary to require that certain values ​​of the base layer ID, such as 0, be excluded from the possibility of adding an enhancement layer ID thereto. The latter requirement results in the following asymmetry: for almost all possible values ​​of the base layer ID, it is possible to add an enhancement layer ID. This results in the following asymmetry: for almost all values ​​of the base layer ID, it is possible to add an enhancement layer ID, thus resulting in the number of almost all values ​​of the base layer ID times the number of representable values ​​of the enhancement layer ID that can represent a layer ID value. However, the limited base layer ID values ​​exclude the possibility of adding any enhancement layer ID. Therefore, either this asymmetry is accepted, or forbidden values ​​of the base layer ID are not used when using the extension mechanism. The solution presented below overcomes this problem in a more efficient way.

[0090] This next conceptual description is now immediately outlined in terms of possible extensions to the HEVC standard where the restrictions just outlined apply exactly, but the concepts outlined below are also transferable to other coders that impose similar restrictions on, for example, the layer identification extension mechanism.

[0091] A limitation of HEVC, as outlined above, is that the base layer cannot be extended. Therefore, to comply with the HEVC standard, the layer id extension value layer_id_ext cannot be signaled for layers with nuh_layer_id equal to 0. Thus, a cluster associated with nuh_layer_id equal to 0 can only contain the base layer itself. An unequal number of layers in such different clusters is inappropriate.

[0092] The problem can be solved by the concept outlined below, which derives clusters and layer id extension values ​​within clusters in a different way than described above. Since it is not possible to have multiple layers for nuh_layer_id equal to 0, the restriction of not having multiple layers is extended to other nuh_layer_id values ​​not equal to 0. Layers belonging to these restricted nuh_layer_id values ​​are then combined with layers from unrestricted nuh_layer_id values ​​to form clusters with an equal number of layers.

[0093] The basic idea is that nuh_layer_id is separated into two values, for example by bit masking. One value is used to determine the cluster; this way different nuh_layer_id values ​​can belong to the same cluster. The other value is used to determine the presence of a layer_id_ext value (thus determining if multiple layers can belong to a nuh_layer_id value) and to determine an additional offset to the layer_id_ext_value in the LayerId derivation.

[0094] An example for an embodiment of the basic idea is given below and illustrated in Figure 18. Figure 18 therefore provides an example for the relationship between layer_id_ext, nuh_layer_id and LayerId for the exemplary case of max_layer_id_ext equal to 3 (LengthOfExtension equal to 2).

[0095] In this example only, the five most significant bits of the six-bit nuh_layer_id value are used to determine the cluster, so two nuh_layer_id values ​​with different least significant bits belong to one cluster.

[0096] If the least significant bit is equal to 0, then the layer_id_ext value is not signaled for only one layer belonging to nuh_layer_id and uh_layer_id. By definition, this layer is the layer with layer_id_ext equal to 0 in the cluster defined by the five most significant bits. Otherwise, if the least significant bit is equal to 1, multiple layers belong to the nuh_layer_id value. For each layer, the layer_id_ext value is explicitly signaled, and each layer belongs to the cluster defined by the five most significant bits.

[0097] The syntax table in Figure 19 is used for the embodiment, which shows the header of the packet 206, i.e., the header of the slice segment, or more precisely, its fragment.

[0098] When the least significant bit of nuh_layer_id is equal to 0, Layer_id_ext is not present. When it is not present, it is inferred to be equal to 0. The variable LayerId is determined as follows: where LengthOfExtension is the length of the layer_id_ext syntax element in bits. LayerId=((nuh_layer_id>>1)< <LengthOfExtension)+layer_id_ext

[0099] In another embodiment, the variable add_layer_id_ext_len is signaled in the bitstream, e.g., VPS. (Note that the previously described embodiment is a special case of this embodiment with add_layer_id_ext_len equal to 1.) The most significant bit (6-add_layer_id_ext_len) of the 6-bit nuh_layer_id syntax element is used to determine the cluster. The least significant bit, add_layer_id_ext_len, specifies whether layer_id_ext is inferred. Additionally, they specify an additional offset to the layer_id_ext value. The syntax tables in Figures 20a and 20b are used to illustrate the embodiment.

[0100] In an embodiment, only layer_id_ext is present when the least significant bit (add_layer_id_ext_len) of nuh_layer_id is equal to 1. When layer_id_ext is not present, it is inferred to be equal to 0. The variable LayerId is determined as follows: where LengthOfExtension is the length of the layer_id_ext syntax element in bits. LayerId= ((nuh_layer_id>>add_layer_id_ext_len)< <LengthOfExtension)+layer_id_ext+layerIdExtOffset

[0101] Thus, in accordance with the concepts outlined above with respect to Figures 18-20b, an apparatus such as a network element of the multi-layer video decoder depicted in Figure 2 is configured to process a multi-layer data stream in which video material is coded in different layers corresponding to different information levels using inter-layer prediction. For each packet comprising the multi-layer data stream, the apparatus does the following: Each packet is associated with one of the different layers, as already described above.

[0102] In particular, as shown in Figure 21, the device reads a base layer ID field 308 from the multi-layer data stream, or more precisely, from the current packet. The base layer ID field includes a first sub-field 308a and a second sub-field 308b. For example, the first sub-field is formed by one or more least significant bits of field 308, while the second sub-field 308b is formed by the other most significant bits of field 308. As shown in the example of Figure 20b, a high-level syntax element 310 called add_layer_id_ext_len defines the length of sub-field 308a.

[0103] After reading the base layer ID field 308 in step 312, the device checks in step 314 whether the subfield 308a satisfies a predetermined criterion. For example, the criterion is met if the subfields are one or the "exact sequence." If the predetermined criterion is met, the device reads the enhancement layer ID field 316 from the data stream, or more precisely, from the current packet, in step 318. Then, in step 320, the device derives an extension value using the enhancement layer ID field such that the extension value is within a first subset of the domain of extension values. The derivation in step 320 corresponds to a simple copy of the bit representation in field 316. It is a requirement of the data stream that the value in the enhancement layer ID field 316 not assume a specific value, such as 0. As further outlined below, if the subfield 308a of the base layer ID field 308 does not meet the criterion, the latter value is reserved.

[0104] After step 320, the device derives a cluster value from the second subfield 308b by mapping from the domain of possible values ​​of the second subfield 308b onto the domain of cluster values. The layer to which the current packet pertains, i.e., the layer comprising fields 308 and 316, is indexed in step 324 using the cluster and extension values ​​derived in steps 320 and 322. For example, the cluster and extension values ​​are simply concatenated with each other, e.g., using the cluster value for the most significant digit. The derivation of the cluster value 322 based on the subfield 308b may also consist, for example, of a direct use of the bits of the subfield 308b as a binary representation of the cluster value.

[0105] However, if the check in step 314 does not meet the predetermined criteria, the device refrains from reading the extension layer ID field from the multi-layer data stream for the current packet. Rather, in step 326, the device sets the extension value to a value that falls within a first subset of the domain of extension values, such as by setting the extension value to 0, and then derives a cluster value from the second subfield 308b by mapping from the domain of possible values ​​of the second subfield 308b onto the domain of cluster values. The derivation in steps 322 and 328 is performed using the same mapping. The layer ID of the current packet is then determined from the cluster and extension values ​​set and derived in steps 326 and 328. Now, if cluster and extension values ​​were derived in steps 320 and 322, the layer ID determination is again performed in the same manner.

[0106] 21 in relation to the embodiment of FIG. 18, the first subfield 308a is one bit long, specifically the LSB of field 308. If this LSB 308a is won and the domain of the extension value across the enhancement layer ID field 316 is {0, 1, 2, 3}, and in the case of FIG. 18, layer_id_ext, i.e., field 316, is two bits long, then the predetermined criterion is met. Therefore, the domain of the extension value, i.e., the first subset of the common domain of the derivation in step 320, is {1, 2, 3}. In particular, the derivation in step 320 is performed by the method of a matrix of units applied to the enhancement layer ID field in the example of FIG. 18. Instead, some additional calculation is used to add the value of the first subfield 308a to the result of the matrix application of units. Therefore, in step 326, the extension value is set to a value that resolves to {1, 2, 3}, i.e., 0. The mappings used in derivations 322 and 328 are the same, but could alternatively be different. The mappings are notably bijective.

[0107] The index determination in step 324 uses the cluster value to set the more significant digits and the extension value to set the less significant digits, as illustrated in Figure 18. Thus, as illustrated in Figure 18, the layer_id_ext field, i.e., the enhancement layer ID field 316, is only present if the LSB of the nuh_layer_id, i.e., the base layer ID field 308, is 1. Otherwise, the layer_id_ext field is inferred to be 0. Thus, the mapping of layer IDs to the combination of layer_id_ext and nuh_layer_id is completely symmetric, and there are no unused bit representations. The inferred value of layer_id_ext is utilized in the data stream by transporting layer_id_ext. As a result, the bits across the data stream for transporting layer_id_ext are reduced to account for the fact that layer_id_ext actually only assumes three different values ​​{1, 2, 3}. For example, a binary unary code is used to map the triplet {1, 2, 3} onto a string of binary numbers that is encoded into and decoded from a data stream using entropy encoding / decoding.

[0108] Therefore, mutual aliasing is described with respect to the following concepts. An apparatus configured to process a multi-layer data stream in which video material is coded using inter-layer prediction with different layers corresponding to different information content levels. A multi-layer data stream including a plurality of packets, each associated with one of the different layers. An apparatus configured to read a base layer ID field from the multi-layer data stream for each packet of the multi-layer data stream. A base layer ID field including a first sub-field and a second sub-field. A check as to whether the first sub-field of the base layer ID field satisfies a predetermined criterion.

[0109] If the first subfield of the base layer ID field meets a predetermined criterion, read the enhancement layer ID field from the multi-layer data stream, derive an enhancement value using the enhancement layer ID field such that the enhancement value is within a first subset of a domain of enhancement values, and derive a cluster value from the second subfield by mapping from the domain of possible values ​​of the second subfield onto the domain of cluster values.

[0110] If the first subfield of the base layer ID field does not meet the predetermined criteria, refrain from reading the enhancement layer ID field from the multi-layer data stream, set the enhancement values ​​to values ​​that disaggregate into a first subset of the domain of enhancement values, and derive cluster values ​​from the second subfield by mapping from the domain of possible values ​​of the second subfield onto the domain of cluster values.

[0111] Here, the device indexes each packet to the associated layer using the cluster and extension value. Note that a default criterion is read from the multi-layer data stream. As explained above, the "first subfield" is 1 bit long, specifically the LSB, if the "default criterion" is one. The "domain of extension values" is, for example, {0, 1, 2, 3}. The "first subset" is {1, 2, 3}. "Using the extension layer ID field" is instead a matrix of units applied to the extension layer ID field, adding the value of the first subfield to the result of matrix application of units. "Set to value that resolves to extension value" is "set to 0." The "mapping from the domain of possible values ​​of the first subfield onto the domain of cluster values" is the same for both cases, but may alternatively be different. The mapping is bijective. "Using cluster and extension values ​​to index the layer to which each packet is associated" includes, for example, using cluster values ​​to set more significant digits and extension values ​​to set less significant digits.

[0112] The latter concept is directly transferred to an encoder configured to encode video material 202 into a multi-layer data stream 204 using inter-layer prediction with different layers corresponding to different information levels. The multi-layer data stream includes multiple packets 206, each associated with a different layer. The layer to which each packet is associated is uniquely determined by a cluster and an extension value. The encoder is configured to insert, for each packet 206 of the multi-layer data stream, a base layer ID field 308 including a first sub-field 308a and a second sub-field 308b into the multi-layer data stream by setting the first sub-field to meet a predetermined criterion using the cluster value to set the second sub-field if the extension value is within a first subset of the extension value domain. The encoder is then configured to insert an enhancement layer ID into the multi-layer data stream using the extension value to set the enhancement layer ID. The encoder is then configured to refrain from inserting the enhancement layer ID field and inserting 312 the base layer ID field 308 into the multi-layer data stream by setting the first subfield using the cluster value for setting the second subfield if the enhancement value is equal to a value that falls within a first subset of the domain of enhancement values, so as not to meet the predetermined criteria. That is, the encoder generates a data stream such that the decoder derives the correct layer IDs in the manner outlined above and uses the inverse mapping compared to these as outlined above for the decoding or receiving side at this end.

[0113] To summarize the above, an extension mechanism is also provided for layer identifiers of enhancement layer data, grouping of layer identifiers into clusters and efficient signaling of coding-dependent information within clusters.

[0114] The details set forth above apply to enhancement layer data in a layered coding scenario, while base layer data is left unchanged. The base layer in a layered coding scenario is identified through a layer identifier (compare nuh_layer_id) in the common header syntax of base layer and enhancement layer data packets.

[0115] In the above example, the individual device at the receiving end is a video decoder. An operational video decoder can therefore select packets of the bitstream identified by the (extended) layerID for decoding. However, the device may alternatively be a network element that can discard packets of the bitstream identified by the (extended) layerID based on external factors such as network utilization, knowledge of the decoder, etc.

[0116] In all of the above embodiments, the multi-layer data stream has video material encoded therein at different layers using inter-layer prediction, such that inter-layer prediction of a layer from another layer adds one or more additional views, depth information, surface reflectance information, color component information, spatial resolution refinement, and SNR resolution refinement.

[0117] The device is a receiver (102) and a layer identification extender (110); the receiver (102) is configured to receive a multi-layer video signal (104) comprising a series of packets (106) each including a layer identification syntax element structure comprised of a base layer ID field (108) and an enhancement layer ID field (112); the layer identification extender (110) reads, for a predetermined portion (113) of the multi-layer video signal including a plurality of packets, a maximum syntax element (114) from the multi-layer video signal indicating a maximum hypothetical value of an enhancement layer ID field of the packets included in the predetermined portion (113) of the multi-layer video signal; for each packet within the predetermined portion of the multi-layer video signal, determining a layer ID for the each packet based on the layer identification syntax element structure; determining a maximum number of layers in the predetermined portion of the multi-layer video signal based on the maximum hypothesis value; and The method is configured to determine at least one characteristic for each of a maximum number of layers by parsing a layer characterization syntax portion (124) of the multi-layer video signal a number of times equal to the maximum number of layers.

[0118] The device according to claim 1 comprises The multi-layer video signal encodes video material in different layers using inter-layer prediction, such that any layer that is inter-layer predicted from another layer adds one or more of additional views, depth information, alpha blending information, color component information, spatial resolution refinement, and SNR resolution refinement.

[0119] The device according to claim 1 or claim 2, wherein the packet (106) is one of the group consisting of a NAL unit, a slice, a tile and a picture, and the predetermined part (113) is one of the group consisting of a coded video sequence, a group of chunks or a group of pictures.

[0120] The apparatus according to any one of claims 1 to 3 is configured to use the maximum hypothesis value to derive a bit length of the enhancement layer ID field.

[0121] The apparatus of any of claims 1 to 4 is configured to derive the bit length of the enhancement layer ID field by rounding up the logarithm to the base 2 of the maximum hypothesis value.

[0122] In the apparatus according to any one of claims 1 to 5, the characteristic is associated with inter-layer prediction dependency of each of the layers on any other layer of the maximum number of layers.

[0123] The apparatus of any of claims 1 to 6 is configured to derive the bit length of the enhancement layer ID field (112) from explicit signaling in a bitstream, and the maximum syntax element (114) indicates the maximum hypothetical value of the enhancement layer ID field in units smaller than the bit length of the enhancement layer ID field-1 (112) by a power of two.

[0124] The apparatus according to any one of claims 1 to 6 is configured to derive the bit length of the enhancement layer ID field (112) from explicit signaling in a bitstream, wherein the maximum syntax element (114) indicates the maximum hypothetical value of the enhancement layer ID field (108) in units of 1.

[0125] The apparatus of any of claims 1 to 6 is configured to determine the layer ID for each packet based on the layer identification syntax element structure by concatenating the base layer ID field and the enhancement layer ID field.

[0126] 10. The apparatus according to any one of claims 1 to 9, comprising a video decoder configured to decode the multi-layer video signal using inter-layer prediction using the characteristics determined for each layer; or A network element configured to discard packets of the multi-layer video signal based on the determined characteristics for each layer.

[0127] The method is: receiving a multi-layer video signal (104) comprising a sequence of packets (106) each including a layer identification syntax element structure comprised of a base layer ID field (108) and an enhancement layer ID field (112); for a predetermined portion (113) of the multi-layered video signal including a plurality of packets, reading from the multi-layered video signal a maximum syntax element (114) indicating a maximum hypothetical value of an enhancement layer ID field of the packets included in the predetermined portion (113) of the multi-layered video signal; for each packet within the predetermined portion of the multi-layer video signal, determining a layer ID for the each packet based on the layer identification syntax element structure; determining a maximum number of layers in the predetermined portion of the multi-layer video signal based on the maximum hypothesis value; At least one characteristic for each of the maximum number of layers is determined by parsing the layer characterization syntax portion (124) of the multi-layer video signal repeatedly a number of times equal to the maximum number of layers.

[0128] The encoder is 1. An encoder for encoding a video signal into a multi-layered video signal consisting of a sequence of packets (106) each including a layer identification syntax element structure consisting of a base layer ID field (108) and an enhancement layer ID field (112), the encoder comprising: for a predetermined portion (113) of the multi-layered video signal comprising a plurality of packets, inserting into the multi-layered video signal a maximum syntax element (114) indicating a maximum hypothetical value of an enhancement layer ID field of the packets included in the predetermined portion (113) of the multi-layered video signal; determining a maximum number of layers in the predetermined portion of the multi-layer video signal based on the maximum hypothesis value; and The layer characterization syntax portion (124) of the multi-layer video signal is configured to signal at least one characteristic for each of a maximum number of layers of the multi-layer video signal (104) by writing the layer characterization syntax portion (124) of the multi-layer video signal a number of times equal to the maximum number of layers.

[0129] The method is: 1. A method for encoding a video signal into a multi-layered video signal (104) consisting of a sequence of packets (106) each including a layer identification syntax element structure comprised of a base layer ID field (108) and an enhancement layer ID field (112), the method comprising: for a predetermined portion (113) of the multi-layered video signal comprising a plurality of packets, inserting into the multi-layered video signal a maximum syntax element (114) indicating a maximum hypothetical value of an enhancement layer ID field of the packets included in the predetermined portion (113) of the multi-layered video signal; determining a maximum number of layers in the predetermined portion of the multi-layer video signal based on the maximum hypothesis value; The layer characterization syntax portion (124) of the multi-layer video signal is written repeatedly a number of times equal to the maximum number of layers to signal at least one characteristic for each of the maximum number of layers in the multi-layer video signal (104).

[0130] A computer program having a program code When the program code is executed on a computer, the computer carries out the method of claim 11 or claim 13.

[0131] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent descriptions of the corresponding method, in which blocks or apparatus correspond to method steps or features of method steps. Analogously, aspects described in the context of a method step also represent descriptions of the corresponding blocks or items or features of the corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware device, such as a microprocessor, a programmable computer, or electronic circuitry. In some embodiments, approximately one or more of the most significant method steps are performed by such an device.

[0132] The inventive coded signals described above may be stored on a digital storage medium or transported over a communication medium such as a wireless communication medium or a wired communication medium such as the Internet.

[0133] Depending on particular implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementation can be performed using a digital storage medium (e.g., floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory) having electronically readable control signals stored thereon in cooperation (or capable of cooperation) with a programmable computer system in which the respective methods are executed. The digital storage medium can thus be computer readable.

[0134] Some embodiments according to the present invention include a data carrier having electronically readable control signals that may cooperate with a programmable computer system to cause one of the methods described herein to be performed.

[0135] In general, embodiments of the present invention may be implemented as a computer program product having program code thereon. When the computer program product is run on a computer, the program code operates to perform one of the methods. For example, the program code may be stored on a machine-readable carrier.

[0136] Another embodiment comprises the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0137] A method embodiment of the present invention is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0138] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recorded medium is generally tangible and / or non-transitory.

[0139] A further embodiment of the inventive method is, therefore, a data stream or sequence of signals representing the computer program for performing one of the methods described herein, the data stream or sequence of signals being, for example, adapted to be transmitted via a data communications connection (e.g. the Internet).

[0140] Another embodiment comprises a processing means (for example a computer, or a programmable logic device) configured to or adapted to perform one of the methods described herein.

[0141] Another embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0142] Another embodiment according to the present invention includes a device or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. For example, the receiver may be a computer, a mobile device, a memory device, or the like. For example, the device or system may include a file server for transferring the computer program to the receiver.

[0143] In some embodiments, a programmable logic device (e.g., a field programmable gate array) is used to perform some or all of the functions of the methods described herein. In some embodiments, the field programmable gate array cooperates with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0144] The above-described embodiments are described solely for the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. We are therefore limited only by the scope of the appended claims, and not by the specific details presented by way of illustration and description of the embodiments herein.

Claims

1. a receiver configured to receive a multi-layer video signal comprising a sequence of packets each including a layer identification syntax element structure comprising a base layer ID field and an enhancement layer ID field; Layer Identification Extender and An apparatus comprising: The layer identification extender reading, for a plurality of packets, a first largest syntax element included in a first Network Abstraction Layer (NAL) unit from the multi-layer video signal, the first largest syntax element indicating a first largest hypothetical value of the enhancement layer ID field for the plurality of packets; reading, for the plurality of packets, a second largest syntax element included in a second NAL unit from the multi-layer video signal, the second largest syntax element indicating a second largest hypothetical value of the base layer ID field for the plurality of packets; determining a maximum number of layers in the predetermined portion of the multi-layer video signal based on the first and second maximum hypotheses; determining at least one characteristic for each of the maximum number of layers by repeatedly parsing a layer characterization syntax portion of the multi-layer video signal a number of times equal to the maximum number of layers; It is configured as follows: the apparatus is configured to obtain a bit length of the Enhancement Layer ID field from explicit signaling in the multi-layer video signal, and the first maximum syntax element indicates the first maximum hypothesis value of the Enhancement Layer ID field in units of less than the square of the bit length of the Enhancement Layer ID field minus one. An apparatus characterized in that

2. 2. The apparatus of claim 1, wherein the multi-layer video signal includes video material coded into different layers using inter-layer prediction, such that any layer inter-layer predicted from another layer adds one or more of additional views, depth information, alpha blending information, color component information, spatial resolution refinement, and SNR resolution refinement.

3. 2. The apparatus of claim 1, wherein the packet is one of a group including a NAL unit, a slice, a tile, and an image, and the predetermined portion is one of a group including a coded video sequence, a chunk, or a group of images.

4. 2. The apparatus of claim 1, further configured to derive a bit length of the enhancement layer ID field using the first maximum hypothesis value.

5. 2. The apparatus of claim 1, further configured to derive the bit length of the enhancement layer ID field by rounding up the base 2 logarithm of the first maximum hypothesis value.

6. The apparatus of claim 1 , wherein the at least one characteristic relates to an inter-layer prediction dependency of each of the layers relative to any other layer of the maximum number of layers.

7. 2. The apparatus of claim 1, wherein the apparatus is configured to obtain a bit length of the enhancement layer ID field from explicit signaling in the multi-layer video signal, and wherein the first maximum syntax element indicates the first maximum hypothesis value of the enhancement layer ID field in units of 1.

8. 2. The apparatus of claim 1, wherein the apparatus is configured to determine a layer ID for the respective packet based on the layer identification syntax element structure by concatenating the base layer ID field and the enhancement layer ID field.

9. The device comprises: a video decoder configured to decode the multi-layer video signal using inter-layer prediction using the characteristics determined for each layer; or a network element configured to discard packets of the multi-layer video signal based on the determined characteristics for each layer.

2. The device according to claim 1, wherein:

10. the layer identification extender is configured to determine, for each of the plurality of packets, a layer ID for the respective packet based on the layer identification syntax element; The at least one characteristic includes a coding parameter set for each of the layer IDs of the maximum number of layers.

2. The device according to claim 1, characterized in that:

11. an encoder configured to encode a multi-layered video signal comprising a sequence of packets each including a layer identification syntax element structure comprising a base layer ID field and an enhancement layer ID field; Layer Identification Extender and An apparatus comprising: The layer identification extender configured to insert, for a plurality of packets, a first largest syntax element into a first Network Abstraction Layer (NAL) unit of the multi-layered video signal, the first largest syntax element indicating a first largest hypothesized value of the enhancement layer ID field of the plurality of packets; configured to insert, for the plurality of packets, a second maximum syntax element into a second NAL unit of the multi-layer video signal, the second maximum syntax element indicating a second maximum hypothesized value of the base layer ID field of the plurality of packets; wherein the maximum number of layers in the predetermined portion of the multi-layer video signal is based on the first and second maximum assumptions; configured to insert into the multi-layer video signal an indication of at least one characteristic for each of the maximum number of layers by iteratively generating a layer characterization syntax portion of the multi-layer video signal using a number of iterations equal to the maximum number of layers; 10. The apparatus of claim 9, wherein the apparatus is configured to insert explicit signaling of a bit length of the Enhancement Layer ID field into the multi-layer video signal, and wherein the first maximum syntax element indicates the first maximum hypothesis value of the Enhancement Layer ID field in units smaller than the square of the bit length of the Enhancement Layer ID field minus one.

12. 12. The apparatus of claim 11, configured to encode video material into different layers using inter-layer prediction, where any layer that is inter-layer predicted from another layer adds one or more of additional views, depth information, alpha blending information, color component information, spatial resolution refinement, and SNR resolution refinement.

13. 12. The apparatus of claim 11, wherein the packet is one of a group including a NAL unit, a slice, a tile, and an image, and the predetermined portion is one of a group including a coded video sequence, a chunk, or a group of images.

14. 12. The apparatus of claim 11, further configured to derive a bit length of the enhancement layer ID field using the first maximum hypothesis value.

15. 12. The apparatus of claim 11, further configured to derive the bit length of the enhancement layer ID field by rounding up the base 2 logarithm of the first maximum hypothesis value.

16. The apparatus of claim 11 , wherein the at least one characteristic relates to an inter-layer prediction dependency of each of the layers relative to any other layer of the maximum number of layers.

17. 12. The apparatus of claim 11, wherein the apparatus is configured to insert explicit signaling of a bit length of the enhancement layer ID field into the multi-layer video signal, and the first maximum syntax element indicates the first maximum hypothesized value of the enhancement layer ID field in units of 1.

18. 12. The apparatus of claim 11, wherein a layer ID for the respective packet can be derived based on the layer identification syntax element structure by concatenating the base layer ID field and the enhancement layer ID field.

19. 12. The apparatus of claim 11, wherein the apparatus is a video encoder configured to encode the multi-layer video signal using inter-layer prediction using the characteristics determined for each layer.

20. configured, for each of the plurality of packets, to determine the layer identification syntax element for the respective packet based on a layer ID; The at least one characteristic includes a coding parameter set for each of the layer IDs of the maximum number of layers.

12. The device according to claim 11 , characterized in that

Images