Network entity for processing a data stream

Abstract

A network entity for processing a data stream into which media content is encoded, the data stream comprising packets, each packet comprising a packet type identifier identifying a packet type associated with the respective packet from a plurality of packet types, wherein each packet having a packet type associated therewith from a first set of packet types from the plurality of packet types comprises an operation point identifier identifying an operation point associated with the respective packet from a plurality of operation points within a scalability space spanned by n scalability axes, wherein each packet having a packet type associated therewith from a second set of packet types from the first set of packet types additionally carries data. The network entity is configured to read a scalability axes descriptor from packets having a predetermined packet type associated therewith that is disjoint from the second set and to interpret the operation point identifier in accordance with the scalability axes descriptor.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 201910876391.6, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to scalable data streams, such as scalable video streams, and network entities that process such scalable data streams, such as decoders or MANEs (Media Aware Network Components). Background Technology

[0003] From a transmission perspective, there is a strong need to adapt video bitstreams in time or other dimensions, as identified and addressed within the H.264 / AVC standard. Encapsulating video data within Network Abstraction Layer (NAL) units and the design decisions reflect this understanding: indicating many important but relatively constant parameters outside the video bitstream (in the so-called parameter set). H.264 / AVC's Scalable Video Coding (SVC) and Multi-View Video Coding (MVC) extensions allow adaptation beyond the time dimension, but the lack of extensibility in the basic H.264 / AVC specification necessitates complex methods to extend H.264 / AVC higher-order syntax in a backtracking-compatible manner. In H.264 / AVC and its extensions, indicating the operational points along the scalability dimension is accomplished through NAL unit header extensions, which include additional bytes for this purpose. The same information is provided by the so-called prefix NAL unit, which contains plain H.264 / AVC video data and constitutes the so-called base layer. The mechanism for the information provided by the enhancement layer extension via the sequence parameter set (SPS) of the base layer (which is encoded by an extension of H.264 / AVC) is established via the so-called subset sequence parameter set (SSPS).

[0004] While the development of the HEVC basic specification is ongoing, efforts are being made to extend it towards 3D video coding, prioritizing the scalability of the basic specification's syntax. The so-called scalable hooks in the basic specification need to be carefully designed to be future-proof. The following paragraphs provide an overview of the current state of HEVC higher-order (HL) syntax and the concepts discussed so far.

[0005] The current state of HEVC standardization is as follows:

[0006] During the current HEVC basic specification and 3D extensions, numerous participants have proposed ways to proceed from the HL syntax specified in H.264 / AVC. The results are reflected in the current working draft of the specification and the numerous contributions of individual participants. The following paragraphs provide an overview of the current discussion.

[0007] As mentioned above, indicating the operational points within the scalability dimension in an SVC or MVC video bitstream requires a (specific) extension to the H.264 / AVCNAL unit header. This is considered a poor solution, as it results in additional effort (e.g., to parse multiple different NAL unit header structures) and requires prefixed NAL units to indicate the base layer. Therefore, efforts are made to ensure that the basic HEVCNAL unit header syntax is sufficiently general to meet the needs of future extensions to the basic specification.

[0008] In the syntax of NAL units, as in the current working draft, the current consensus is to use a two-byte NAL unit header. In the first byte, one bit is used to indicate the nal_ref_flag, instead of the two bits for nal_ref_idc in H.264 / AVC, because this HL feature is not yet widely used in applications. The syntax element nal_unit_type therefore has another bit to indicate the type of the NAL unit, allowing a total of 64 distinguishable types.

[0009] The second byte of the NAL unit header is split into two parts. Bit 1C is used to indicate the temporal_id of the NAL unit, since time scalability is enabled in the basic specification. The remaining 5 bits of the second byte are reserved as one in HEVC-compliant bitstreams. The current understanding of using the remaining 5 bits is that they can be used in future extensions to indicate, for example, a scalability identifier for the layer_id syntax element.

[0010] Although the Picture Parameter Set (PPS) and Sequence Parameter Set as defined in the current HEVC basic specification are relatively similar to the parameter sets previously specified in H.264 / AVC, two new parameter sets (called Adaptation Parameter Set (APS) and Video Parameter Set (VPS)) have been introduced into HEVC, of ​​which only VPS is relevant to the content of this document.

[0011] The video parameter set should indicate parameters such as the number of (e.g., temporal) levels / layers present in the video bitstream, as well as the profiles and levels of all operation points within them. Other parameters to be indicated include dependencies between scalable layers, as if they were indicated in the SVC Scalability Information (SEI) message.

[0012] The following is an additional brief explanation of the semantic presentation of NAL units and video parameter set syntax.

[0013] profile_idc and level_idc indicate the profile and level to which the encoded video sequence conforms.

[0014] `max_temporal_layers_minus1+1` specifies the maximum number of temporal layers in the sequence. The value of `max_temporal_layers_minus1` should be in the range of 0 to 7 (inclusive). The following specifies `more_rbsp_data()`.

[0015] - If there is no more data in the RBSP, then more_rbsp_data() returns a false value.

[0016] - Otherwise, search the RBSP data to find the last (least significant, rightmost) bit in the RBSP that is equal to 1. The following applies, taking into account the position of this bit (which is the first bit of the rbsp_trailing_bits() syntax structure (rbsp_stop_one_bit)).

[0017] - If there is more data in the RBSP before the rbsp_trailing_bits() syntax structure, then more_rbsp_data() returns true.

[0018] Otherwise, more_rbsp_data() will return a false value.

[0019] A nal_ref_flag value of 1 indicates that the contents of the NAL unit contain a sequence parameter set, an image parameter set, an adaptation parameter set, or a slice of a reference image.

[0020] For one or more encoded video sequences that conform to the configuration file specified in Annex 10 (decoded using the decoding process specified in Clauses 2 to 9), a nal_ref_flag of 0 in the NAL unit containing the tile indicates that the tile is a part of the non-reference picture.

[0021] For NAL units of sequence parameter sets, image parameter sets, or adaptation parameter sets, nal_ref_flag should be equal to 1. When nal_ref_flag is equal to 0 for a NAL unit where nal_unit_type is equal to 1 or 4 for a specific image, nal_ref_flag should be equal to 0 for all NAL units where nal_unit_type is equal to 1 or 4 for that image.

[0022] The nal_ref_flag should be equal to 1 for NAL units where nal_unit_type is 5.

[0023] The nal_ref_flag should be equal to 0 for all NAL units where nal_unit_type is equal to 6, 9, 10, 11, or 12.

[0024] The nal_unit_type specifies the type of RBSP data structure contained in the NAL unit, as specified in Table 1.

[0025] The decoder should ignore (remove and discard) the contents of all NAL units that use the reserved value of nal_unit_type.

[0026] Table 1 - Possible NAL unit type codes and NAL unit type categories (a non-exhaustive list)

[0027]

[0028]

[0029] A “profile” is a subset of the entire bitstream syntax. Within the limits imposed by the syntax of a given profile, significant variations in encoder and decoder performance may still be required, depending on the values ​​used for syntax elements in the bitstream, such as the specified size of the decoded image. In many applications, implementing a decoder capable of handling all hypothetical uses of the syntax within a specific profile is currently neither feasible nor cost-effective.

[0030] To address this issue, a "hierarchy" is defined within each configuration file. A hierarchy is a set of constraints imposed on the values ​​of syntax elements in the bitstream. These constraints can be simple restrictions on the values, or they can take the form of constraints on an arithmetic combination of values ​​(e.g., image width multiplied by image height multiplied by the number of images decoded per second).

[0031] Hierarchy: A defined set of constraints governing the values ​​used by syntax elements and variables. The same set of hierarchies is defined across all configuration files, with most aspects of the definition for each hierarchy common to different configuration file systems. Within specified constraints, individual implementations may support different hierarchies for each supported configuration file. The hierarchical system transforms the values ​​of its coefficients before scaling, depending on the context.

[0032] Configuration file: A subset of the syntax rules.

[0033] In the development of HEVC's 3D video coding extension, it has been proposed to shift certain parameters from the tile header to the Access Unit Delimiter (AUD), which is a NAL unit in H.264 / AVC that is optionally located at the beginning of the new Access Unit (AU).

[0034] Another proposal during the specification process of the HEVC 3D video coding extension was to indicate the dependencies between operation points through an indirect method between SPSs. The NAL unit header should carry a reference to the SPS, and within each SPS, a reference to the relevant underlying SPS. This (potentially overlapping) indirect method must be resolved before the lowest (time...) level SPS. This method places a heavy burden on devices such as MANE to delve into the bitstream and retain a large amount of information that can be used to identify operation points.

[0035] In any case, it is advantageous that there is a solution available now that facilitates or makes network entities more efficient in handling scalable data streams.

[0036] This objective is achieved through the subject matter of any of the appended independent claims. Summary of the Invention

[0037] The fundamental finding of this invention is that if packets actually carrying data in a data stream are accompanied by packets of different packet types, the handling of scalable data streams by network entities becomes less complex. These packets of different types have scalability axis descriptors that define the number of scalability axes and their semantic meaning. With this measure, it is sufficient for the data-carrying packet to include an operation point identifier, which identifies the operation point associated with the corresponding packet within the scalability space spanned by the scalability axis. This operation point identifier provides sufficient information to the network entity to associate such a packet with its operation point in the scalability space, assuming that the network entity can access this scalability axis descriptor provided by packets of a predetermined packet type. In practice, this technique allows for a reduction in the amount of data used for most packets, as the operation point identifier only needs to perform the identification task. Scalability axis descriptors can be transmitted less frequently. Simultaneously, the network entity is less frequently burdened with the arduous task of parsing / reading scalability axis descriptors, thus reducing the network entity's workload. On the other hand, according to embodiments of the present invention, network entities can interpret operation point identifiers to at least the extent that they can distinguish packets with the lowest operation point in the sequential operation point order (i.e., basic layer packets) from other packets (i.e., packets belonging to operation points with higher sequential operation point orders, i.e., "enhanced" layer packets).

[0038] According to another aspect of the invention (which may be combined with the first aspect or implemented alone), a fundamental finding of the invention is that network entities can handle scalable data streams more efficiently by using packets different from data-carrying packets to transmit hierarchies and / or profiles, wherein the profiles and / or hierarchical descriptors are segmented into: a first set that explicitly indicates the definition of the available set of encoding options and / or the range of available syntax element values ​​for the corresponding operation point; and a second set of profiles and / or hierarchical descriptors that, by referring to another profile and / or hierarchical descriptor, indicate the definition of the available set of encoding options and / or the range of available syntax element values ​​for its corresponding operation point. Through this measure, the length of the complete sequence of profiles and / or hierarchical descriptors is kept low, thereby saving transmission and parsing overhead. Attached Figure Description

[0039] Preferred embodiments of the invention are described below with reference to the figures. Advantageous embodiments are the subject of the appended claims. In the figures:

[0040] Figure 1A This diagram illustrates the data stream in which media content is encoded.

[0041] Figure 1B The composition of a scalable data stream is illustrated schematically.

[0042] Figure 1C This schematically illustrates the composition of a scalable data stream based on another example;

[0043] Figure 2 A diagram illustrating the state of a network entity receiving a complete data stream;

[0044] Figure 3A This shows the operating points within two-dimensional scalability (e.g., time and space);

[0045] Figure 3B This illustrates the concept of new_profile_level_flag;

[0046] Figure 4 This demonstrates the NAL unit header syntax used in HEVC;

[0047] Figure 5 Show the VPS syntax;

[0048] Figure 6 The vps_extension_data() syntax is displayed;

[0049] Figure 7 A diagram illustrating the data flow based on the instance is provided.

[0050] Figure 8A flowchart showing instances used to segment fixed-length syntax elements or bit sequences respectively;

[0051] Figure 9 A flowchart illustrating the operational modes of an exemplary network entity;

[0052] Figure 10 This demonstrates another operating mode of network entities;

[0053] Figure 11 Demonstrates the NAL unit syntax;

[0054] Figures 12A to 12C Demonstrates SPS syntax;

[0055] Figure 13 Demonstrates VPS syntax; and

[0056] Figure 14 This demonstrates the extended data syntax for VPS. Detailed Implementation

[0057] To facilitate understanding of the embodiments outlined below, regarding Figures 1A to 1C and Figure 2 First, we describe the details of scalable data streams and the problems that arise from these details in handling scalable data streams. Figure 1A Data stream 10 is shown, into which media content (such as audio content, video content, image content, 3D mesh content, or the like) is encoded. The media content may be time-varying, where the data stream represents the media content in a sequence of timestamps, such as images of video, transition blocks or frames of audio signals, or the like.

[0058] like Figure 1A As shown, data stream 10 includes or consists of packets 12. These packets may be equal in size or different in size. Each packet includes a packet type identifier 14. Each packet type identifier 14 identifies the packet type associated with the corresponding packet 12 from a plurality of packet types. The plurality of packet types includes data-carrying packet types. Figure 1A In, for example, in Figure 1A The packets within the data stream 10 shown in the example are illustrated as this data-carrying packet type, exemplarily represented here as "A". In addition to this data-carrying packet type, there may be other packets of other packet types that do not carry the encoded representation of the media content, but rather carry other data, such as supplementary data or information about the origin of the media content, or the like.

[0059] Data-carrying packet type A includes at least one operation point identifier 16, which identifies the operation point associated with the corresponding packet from a plurality of operation points within a scalability space spanned by n scalability axes, where 0 > n, and the operation points have an ordered operation point sequence defined therein. For clarity, refer to Figure 1B The operating point is indicated by circle 18. This operating point is distributed or arranged within the scalability space 20 spanned by the scalability axis, wherein... Figure 1B For illustrative purposes, only two such scalability axes, 22a and 22b, are shown.

[0060] Figure 1B The composition of a scalable data stream, such as scalable data stream 10, is shown. Packet 12 (whose operation point identifier identifies operation point 1) carries data that together form an encoded representation of the media content at operation point 1. Here, as an example, operation point 1 forms a basic representation, i.e., the packet data of operation point 1 provides the minimum amount of information about the media content. For example, in the case of video, such packet data represents the encoded representation of the media content with, for example, low spatial resolution and the lowest SNR. By combining the data carried by the packet belonging to operation point 1 with the data carried by packet 12 of operation point 2 (as illustrated by loop 24), the resulting data produces an encoded representation of the media content, which is reconstructed in some way (e.g., in terms of SNR or spatial resolution) beyond the representation obtained by reconstructing the media content solely from the packet of operation point 1. The encoded representation is extended by adding the data of the packet of operation point 2 to the data of the packet of operation point 1, with the associated characteristics of the media content being... Figure 1B The semantic meaning of axis 2 in the equation, and the scale of axis 22b is an ordinal number table, which sorts operation point 18 according to semantic meaning.

[0061] exist Figure 1B For example, axis 22b relates to spatial resolution, while axis 22a relates to SNR quality. However, other semantic meanings of the scalability axis can also be obtained. For example, additional data from a packet at another operation point can add information to a reconstructable version of the media content that was not available before (i.e., before this data was added), such as another view, another audio channel, a depth map, or the like.

[0062] therefore, Figure 1BAlso shown by another loop 24, the basic encoded representation of the media content can be extended along axis 22b and along 22a. In other words, adding data from packet 12 at one operation point to the data from packet 12 at another operation point or the combination 24 of data from packet 12 at another operation point results in a new combination of data, which, according to one of the scalability axes 22a or 22b, exceeds the representation obtained without this addition, and this sequential effect is shown by arrow 26, which thus forms a sequential operation point order in operation points 18, which traverses the scalability space 20.

[0063] Now, assume that the network entity receives this complete data stream. Figure 2 This situation is illustrated. The network entity is shown as square 28, and for example, a decoder or a MANE. Figure 2 In this process, network entity 28 receives the complete data stream 10, that is, the data stream including all data packets, encompassing all data-carrying packets at all operational points. The task of network entity 28 is to provide media content to the receiving end 30. Figure 2 In this example, network entity 28 uses wireless transmission 32 to forward media content to receiving end 30. Network entity 28 is, for example, an operational management entity. In this situation, network entity 28 must cope with the time-varying transmission capacity to receiving end 30. That is, for example, there are times when the transmission capacity is insufficient to transfer the complete data stream 10 to receiving end 30 for real-time reproduction. Therefore, network entity 28 must "reduce" the data stream 10 to provide a trimmed version or representation of the media content to receiving end 30. Although, for example, the transmission capacity is insufficient to transfer the complete data stream 10, the transmission capacity may be sufficient to transmit, for example, the data stream 10 with packet 12, omitting operation point 5.

[0064] It should be noted that there are other situations where network entity 28 must discard some packets 12 to satisfy some external driving constraints. To this extent, Figure 2 Used as an example only.

[0065] However, in order to perform the aforementioned task of reducing data flow, network entity 28 must inspect at least a portion of the data flow. This portion to be inspected includes the operation point identifier 16. This places a high burden on network entity 28, and therefore it is advantageous to keep this burden low. The embodiments described below achieve this objective.

[0066] Another aspect addressed by the embodiments described below concerns the fact that the decoder's capacity to decode data streams may increase when the order of operation points is extended. For example, a lower-order decoder may have lower capabilities, i.e., a lower profile and / or a lower hierarchy, as explained in the introduction of the specification. Therefore, the addition of some data to a packet at a particular operation point may thus be limited to decoders that meet a minimum profile and / or minimum hierarchy criterion, and thus it is advantageous that network entities (such as decoders or network entities forwarding data streams to decoders) can understand the profile and / or hierarchy requirements associated with operation point 18 in a simple manner, but on the other hand, at a low rate of consumption.

[0067] However, before describing this embodiment, the following should be noted. Specifically, the nomenclature used in the embodiments described below differs from that in the references. Figures 1A to 2 The terminology used in the description is as follows. More specifically, the embodiments described below represent specific implementations of embodiments of this application, tailored for application in the upcoming HEVC standard. Therefore, the aforementioned package will be referred to hereinafter as a NAL unit, whose NAL unit header includes an operation point identifier, which, according to embodiments relating to the first aspect of this application, is further specified by a scalability axis descriptor in a certain type of NAL unit (i.e., a VPS NAL unit). However, the specific embodiments described now are readily transferable to other types of encoders and decoders. To this extent, the following embodiments should be understood as exemplary only.

[0068] Similarly, although Figure 1B The schematic combination 24 forms only appropriate subsets of each other, so each combination is an appropriate subset of all other combinations 24, but the appropriate subsets or appropriate supersets of other combinations 24, or supersets of all other combinations 24, are optional. For example, Figure 1C This presents a case where the sequentiality of operation point 18 (i.e., the property of having the order of operation points as defined therein) should be understood to include cases where a combination also includes the case where a combination is neither a proper subset nor a proper superset of other combinations. Figure 1C For example, combination 24, resulting from adding data from the packet at operation point 4 to the data from the packets at operation points 1 and 2, only partially overlaps with combination 24, resulting from adding data from the packet at operation point 1C to the data from the packets at operation points 1 and 2. The intersection is formed by the latter combination (i.e., the combination of operation points 1 and 2). Therefore, the sequential operation point order includes a branch at operation point 2.

[0069] The following examples present a high-level syntax including the NAL unit header and VPS syntax, which allows scalability information to be indicated in a way that can be adapted for use in video decoders, video bitstream capturers, or network devices. The NAL unit header includes a scalability identifier that allows indication of up to three scalability dimensions.

[0070] Before describing VPS syntax in more detail, regarding Figure 3A and Figure 3B Presenting an overview. For example, this concept can be used in the HEVE standard.

[0071] The new_profile_level_flag presented below is used to indicate that a certain operation point within the scalability dimension of the video (such as time, space, or any other dimension) uses a different profile or level than the operation points described previously. Figure 3A The different operation points within a spatially and temporally scalable video sequence are displayed based on the scalable identifiers (temporal_id and layer_id). Figure 3A It shows the operating points within two-dimensional scalability (e.g., time and space).

[0072] The profiles and levels required to describe the decoder's capabilities, such as supported encoding tools and macroblock throughput, may change at each operation point, depending heavily on the scalability used. Within a loop indicating the profile and level of an operation point, or other parameters, the `new_profile_level_flag` is used to indicate whether a new profile and level exist for the previously described operation point, or to indicate that the profile and level for a specific operation point must be inherited from the previously described operation point through explicit reference.

[0073] The following description relates to an adaptable list of configuration files and hierarchical syntax elements. The configuration files, hierarchies, and optionally other parameters for the operands in the parameter set are provided via a while loop, which allows for direct adaptation—that is, excluding or removing operands and adding operands without changing other syntax elements within the given syntax (i.e., within the VPS). This is achieved using the `more_rbsp_data()` syntax element, which allows a while loop that does not necessarily indicate the absolute number of hierarchies and thus facilitates adaptation of the parameter set. Figure 3B This illustrates the concept of new_profile_level_flag.

[0074] The following description relates to layer status. Layer status indicates how five reserved bits in the second byte of the NAL unit header are interpreted, which can be used as a scalable layer identifier (layer_id) in the HEVC extension for scalable scenarios.

[0075] In the layer status approach, the status identifier is indicated in the NAL unit header. This identifier allows the decoder or MANE to interpret the five reserved bits into independent one-dimensional or multi-dimensional identifiers for video encoding. These identifiers are features of one or more scalability dimensions (such as temporal and spatial, spatial and multi-view, or others). The VPS provides a detailed description of the scalability dimensions and their operational points, while the NAL unit header only provides information on how to interpret the five reserved bits of the NAL unit header syntax to derive one or more scalability identifiers.

[0076] The VPS syntax described below provides a means to visualize the configuration files and hierarchy at each operational point, and is designed to allow for direct, real-time adjustments. Details about this syntax are provided in the following sections.

[0077] First, the proposed NAL unit header syntax is described.

[0078] The second byte of the NAL unit header is used to indicate the so-called layer_scenario and scalability identifier (layer_id), which identifies the operating point in the layer_scenario context.

[0079] Figure 4 The header syntax for the NAL unit of HEVC is given, which has new syntax elements `layer_scenario` and `layer_id`. The `layer_scenario` syntax element indicates the status of the scalability dimension, for example, a scalable dimension to use only temporal scalability (e.g., temporal scalability as in the HEVC basic specification) or a combination of two types of scalability (e.g., temporal and spatial). The `layer_id` syntax element describes the operation point of the specific NAL unit within the indicated scalability dimension and must be interpreted according to the `layer_scenario` syntax element.

[0080] In the case of no scalability or a single scalable dimension (i.e., the value of layer_scenario is zero), the value of layer_id can be interpreted as an unsigned integer, which sequentially lists the operation point according to decoding dependencies and importance. In the case of no scalability, all NAL units should have a layer_id value of zero to indicate that there is no meaningful hierarchical adaptation procedure and that all NAL units have equal importance.

[0081] In the case of two scalability dimensions (i.e., the value of layer_scenario is one), the value of layer_id can be interpreted as two 3-bit unsigned variables. The first 3rd bit gives the operating point of the NAL unit in the first scalability dimension (e.g., time), and the second 3rd bit gives the operating point of the NAL unit in the second scalability dimension (e.g., space).

[0082] When the value of `layer_scenario` is two, the value of `layer_id` can be interpreted as a two-digit unsigned integer variable, which gives the operating point in the first scalability dimension, followed by a four-digit unsigned integer variable describing the operating point in the second scalability dimension. This is advantageous when the number of layers in one dimension is greater than in the other.

[0083] Considering the three scalability dimensions, the corresponding value of layer_scenario is three, and the value of the layer_id syntax element can be interpreted as three 2-digit unsigned integer variables, which give the operation point according to the three scalability dimensions used (e.g., time, space, and multi-view scalability).

[0084] Table 2 provides an overview of the values ​​of layer_scenario and presents the corresponding interpretation of layer_id.

[0085] Table 2: Overview of layer_scenario values.

[0086]

[0087] The corresponding video parameter set syntax will be described next.

[0088] The syntax of the video parameter set must be adjusted to represent the configuration file and hierarchy for each operational point in the possible three-dimensional scalability space, and a means must be provided to adapt it directly to reflect the current adaptation level of the video bitstream. This is achieved by using the `vps_extension_data()` syntax element to represent the configuration file and hierarchy of NAL units using any kind of scalability (e.g., temporal, spatial, or other). Furthermore, it provides detailed information for interpreting the `layer_id` syntax element within the NAL unit header.

[0089] Figure 5 The VPS syntax is given in the document.

[0090] Figure 5 This illustrates the configuration file and hierarchy of NAL units that do not use any kind of scalability; that is, video data encoded according to the HEVC basic specification, which has the lowest possible temporal hierarchy.

[0091] Furthermore, the syntax element `layer_id_type` is provided to allow for a more detailed understanding of the scalability dimension described by the `layer_id` syntax element within the NAL unit header syntax. Table 1C specifies the actual scalability dimension indicated in `layer_id` for the `layer_id_type` syntax element. Corresponding to the basic HEVC specification, only temporal scalability needs to be defined. Additional HEVC scalability extensions will therefore specify different `layer_id_type` values ​​for specific needs, such as 1: temporal scalability and 2: spatial scalability for HEVC-like SVC extensions.

[0092] This allows any device with a VPS to fully interpret the operating points of NAL units within a known scalability dimension, while devices without access to a VPS can still properly adapt the video bitstream, but are unaware of the actual scalability dimension.

[0093] Table 3: layer_id_type and detailed description of the scalability dimension in layer_id

[0094]

[0095] Through such Figure 6 The `vps_extension_data()` function provides a description of an operation point that uses scalability in any dimension (e.g., time level > 0, spatial scalability, etc.). Each operation point is identified by its `layer_id`. `new_profile_level_flag` allows for an explicit indication of the profile and level of the described operation, or an implicit indication by using the `ref_layer` syntax element to reference another operation point description (via its `layer_id`).

[0096] Using the `vps_extension_data()` syntax allows for direct tuning because it explicitly references the operand and can omit it by excluding a fixed number of bytes. As long as other operand descriptions do not reference the omitted `vps_extension_data()` syntax, no bits of the VPS need to be changed (except for the omitted bits), greatly simplifying live tuning.

[0097] Therefore, using about Figures 1A to 1C The nomenclature used, as described in the specific embodiments just outlined, is as follows: Figure 7The data stream 10 comprises the following components. The data stream, generally indicated by reference symbol 10, contains media content encoded therein, such as audio, video, or other media content as indicated above. The data stream 10 includes packets 12 called NAL units. Each packet 12 includes a packet type identifier, i.e., nal_unit_type, which identifies the packet type associated with the corresponding packet 12 from a plurality of packet types. Each packet 12 (such as all NAL units) having a packet type associated with it from a first set of these plurality of packet types includes an operation point identifier 34, i.e., a combination of layer_scenario and layer_id, which identifies the operation point 18 associated with the corresponding packet 12 from a plurality of operation points 18 within the scalability space 20 spanned by n scalability axes 22a, 22b, where 0 > n, and the operation point 18 has an ordered operation point sequence defined therein. In this regard, it should be noted that not every NAL unit or packet 12 necessarily has an operation point identifier 34. Conversely, there may be a package 12 of a certain package type that has general properties in the scope of operation points and therefore does not include arbitrary operation point identifiers.

[0098] However in Figure 7 In one embodiment, each packet 12 (i.e., non-VPS NAL unit) of a second set of packet types associated with that packet type additionally carries data 36. This data 36, ​​along with the packet 12 having an associated arbitrary operation point, facilitates an encoded representation of the media content at the operation point 18 associated with the corresponding packet, which, according to the sequential operation point order, precedes or is equal to the operation point associated with the corresponding packet. In this regard, refer to... Figure 1B and Figure 1C The description is used to obtain details.

[0099] Each package with a predefined package type that is not intersecting with the second set of package types has a scalability axis descriptor 38, which defines the number n of one or more scalability axes and their syntactic meaning, wherein the scalability axis descriptor 38 is represented by layer_id_type appearing in the VPS NAL unit, wherein the rbsp data portion 40 is populated according to VPS syntax 42.

[0100] As described above, each VPS package may also include a sequence of configuration files and / or hierarchical descriptors 44, namely, vps_extension_data, which are associated with at least one subset of a plurality of operation points 18 (e.g., leading operation points in the order of sequential operation points). Each configuration file and / or hierarchical descriptor 44 defines a set of available encoding options (i.e., the minimum configuration file to be supported for rebuild purposes) and / or a range of available syntax element values ​​(i.e., the minimum hierarchical level required for rebuild), and the syntax of the package 12 with the operation point associated with the corresponding configuration file and / or hierarchical descriptor 44 conforms to the set of available encoding options and / or the range of available syntax element values. In other words, the combination of data resulting from adding the data of a package of one operation point 18 to the data of a package associated with one or a combination of other operation points 18 can only be rebuilt for a decoder that supports the corresponding configuration file and / or hierarchical level.

[0101] The first set of profile and / or level descriptors 44 explicitly indicates the definition of the available encoding option set and / or the range of available syntax element values, i.e., descriptors where the syntax element new_profile_level_flag (i.e., flag 46) equals 1. The second, disjoint, non-empty set of profile and / or level descriptors 44 indicates the definition of the available encoding option set and / or the range of available syntax element values ​​by referencing another, previous profile and / or level descriptor 44, i.e., descriptors where the syntax element new_profile_level_flag equals 0. Figure 6 In this context, the explicit representation of the configuration file is accomplished using the syntax element `profile_idc` located at 48, while the syntax element `level_idc` represents the layer indicator 50. The reference frame for another configuration file and / or layer descriptor 44 is accomplished using the syntax element `ref_layer_id` 52 when `new_profile_level_flag = 0`. Clearly, the use of referencing the configuration file and / or layer descriptor 44 reduces the size required to transfer the sequence 54 of configuration files and / or layer descriptors, and as further outlined below, this aspect can be used independently of the combined use of the operation point identifier 34 and the scalability axis descriptor 38.

[0102] While the sequence 54 of profiles and / or hierarchical descriptors 44 may be associated with a sequential operation point order and at least a subset of multiple operation points, this is not necessarily the case. Instead, each profile and / or hierarchical descriptor may have a field 56, namely, layer_id, which defines the operation point 18 associated with the corresponding profile and / or hierarchical descriptor 44. The network entity processing the data stream 10 may determine the sequential operation point order defined in the operation points 18 based on the order of this field of the profiles and / or hierarchical descriptors 44 in the sequence 54. However, the network entity may have the sequential operation point order known by default, depending on, for example, a scalability axis descriptor. For greater precision, the network entity (such as...) Figure 2 Network entity 28) can use the order of operation points identified by field 56 and the sequence of configuration files and / or hierarchical descriptors 44 to deduce the sequential operation point order.

[0103] Field 56 uses a bit sequence to identify the corresponding operation point, which is common to all operation point identifiers 34. In the embodiments outlined below, for example, all operation point identifiers 34 have a fixed-length syntax element (i.e., layer_id) and a scenario syntax element (i.e., layer_scenario), wherein the scenario syntax element illustrates the association of n segments and n scalability axes of the fixed-length syntax element, which is common to the bit sequence of field 56, or constructed equally with the bit sequence of field 56. Figure 8 An example is shown whereby a fixed-length syntax element or field 56 bit sequence 60 or operation point identifier 34 fixed-length syntax element is divided into different numbers of segments according to three different states 62 of the context syntax element. These segments are exemplarily shown using dashed lines for the first state, dotted lines for the second state, and dashed lines for the third state. That is, a network entity that decodes a fixed-length syntax element or bit sequence 60 can identify the correct operation point of an individual fixed-length syntax element or bit sequence 60 as a component in the n-dimensional scalability space 20 by using the bits contained in the corresponding segments.

[0104] In the embodiments described below, the bit sequence used for the syntax element ref_layer_id 52 is as follows: Figure 8The same bit sequence shown in the diagram. Alternatively, related pointers may have been defined using the sequential operation point order in units of rank position. For example, a network entity could simply add ref_layer_id bit by bit to the syntax element layer_id 56 to access or obtain the layer_id of the operation point represented by the current profile and / or level descriptor 44, in order to use the current operation point's profile identifier profile_idc 48 and / or level identifier level_idc 50.

[0105] In the above embodiment, the scalability axis descriptor 38, i.e., layer_id_type, forms a syntax element whose possible states are associated with different combinations of the value of n and the corresponding semantic meanings of the n scalability axes. That is, the possible states of the scalability axis descriptor are mapped to different (n) meanings (1), ..., meanings (n) (refer to Table 3). In other words, using the scalability axis descriptor 38, the semantic meaning of the properties of the network entities in the data stream is read. Figure 8 The values ​​within the partitioned regions are ordinal. However, since there has been no opportunity to successfully read the network entity of the VCL NAL unit, the ordinal characteristics of the values ​​in the partitioned regions of the bit sequence 60 of the operation point identifier 34 (such as those obtained by using the partitions according to the context syntax element 64) can be used to determine the possible reduction of the data stream during data processing.

[0106] Before proceeding to the next specific embodiment, it should be noted that, according to the additional aspects of the invention described above, the scalability axis descriptor 38 may be omitted, and its semantics may be included, for example, in the operation point identifier 34; however, it would therefore require more bits compared to the embodiment now described. Nevertheless, this alternative embodiment would still be advantageous because of the aforementioned sequence 54 of profile and / or hierarchy descriptors 44, which involves explicitly indicating the profile and / or hierarchy descriptors 44 as well as indirectly indicating / referencing the profile and / or hierarchy descriptors 44.

[0107] However, regarding the above description, it should be noted that various modifications can be made to the embodiments outlined above. For example, the number of bits (i.e., 6) selected for the fixed-length syntax element or bit sequence 60 in the above examples can be larger or smaller. A similar consideration is reasonable within the scope of the possible scalability axes n that can be adjusted via the scenario syntax element 64: the set of possible states is (1, 2, 3), but different sets of possible integers for n can also be implemented.

[0108] For the sake of completeness only, Figure 9The operational mode of a network entity is illustrated, for example, by reading a sequence of profile and / or level descriptors 44. First, in step 66, the fields of the first profile and / or level descriptor 44 in Listing 54 are read. Then, in step 68, the network entity checks whether the current profile and / or level descriptor 44 (whose fields were read in step 66) explicitly indicates a profile and / or level or indirectly indicates a profile and / or level. According to the above embodiment, step 68 is designed to read the syntax element `reserved_zero_one_bit`, where the actual decision depends on the value of the syntax element: `new_profile_level_flag` equal to 1 indicates explicit indication, and the syntax element equal to 0 indicates indirect indication.

[0109] In an explicit illustration, the network entity reads the configuration file and / or hierarchy identifier 58 / 50 in step 70 and associates the configuration file and / or hierarchy identifier 58 / 50 with the operation point identified by the field read in step 66. However, in an indirect illustration, the network entity reads a reference field 52 from the data stream in step 72 and, in step 74, uses the configuration file and / or hierarchy identifier associated with the operation point identified by this reference field 52 as the configuration file and / or hierarchy identifier for the operation point identified by the field read in step 66. This process is then repeated until the sequence of configuration files and / or hierarchy descriptors 44 has been scanned.

[0110] Similarly, Figure 10 The operational modes of network entities utilizing the above embodiments are illustrated, wherein the scalability axis descriptor 38 complements the information provided by the operation point identifier 34 in the corresponding packet. For example... Figure 10As shown, the operation point identifier of the current packet is first read in step 76. This reading is performed for all packets 12. Additionally, the network entity checks the VPS NAL unit to read the scalability axis descriptor from it in step 78. Steps 76 and 78 are performed whenever the network entity encounters a packet of the corresponding packet type. Whenever the operation point identifier is read in step 76, the network entity distinguishes in step 80 whether the scalability axis descriptor has not yet been read and is therefore unavailable, or has been read and is therefore available. If available, the network entity decodes the operation point identifier of the current packet in step 82 based on the scalability axis descriptor read in step 78. After decoding, the network entity knows the meaning of the scalability axis and can act accordingly. Specifically, regardless of availability, the network entity selects packets from packets 12 in step 84, such as selecting packets that will be decoded or forwarded to a receiver such as receiver 30. Step 84 may depend on external data, such as data indicating the receiver's ability to reproduce media data, such as the maximum reproducible spatial resolution, the maximum pixel bit depth of the display, etc. In the event that this interpretation is unavailable due to the scalability axis descriptor being unavailable, network entity 28 may be configured to act defensively, for example, by only further processing packets whose follow-up operation point order belongs to the lowest operation point.

[0111] Similarly, network entity 28 can be a decoder, transcoder, media-aware network entity such as an action management entity, or the like.

[0112] Furthermore, entities generating any of the above-described embodiments of the data stream, such as encoders or the like, can be configured to distribute VPS NAL units sufficiently frequently among the data packets to maintain low durations where the network entity faces unavailability issues that occur before scalability axis descriptors are available, as per [reference to...]. Figure 10 As described.

[0113] For network entities that read the sequence of configuration files and / or hierarchical descriptors 44, it should be noted that this network entity can be configured to, in addition to Figure 10 In addition to or as an alternative to the processing procedure, selection is made in package 12 based on the set of available encoding options and / or the range of available syntax element values ​​associated with the corresponding operation point: for example, in the case where the network entity is, for example, a decoder or receiver (with which the network entity communicates), such as in the case where the network entity is an MME, the network entity may exclude operation points whose required profile and / or level exceed its own maximum profile and / or maximum level.

[0114] In other words, regarding Figures 4 to 6The described system is a data stream in which media content is encoded. The data stream includes packets (such as NAL packets), and each packet includes a packet type identifier nal_unit_type that identifies the packet type associated with the corresponding packet from a plurality of packet types. Each packet having a packet type from a first set of the plurality of packet types associated therewith (e.g., all NAL units) includes an operation point identifier layer_scenario and layer_id that identify the operation point associated with the corresponding packet from a plurality of operation points within the scalability space spanned by n scalability axes, where 0 < n. The operation point has an ordered operation point order defined therein. Each packet 12 having an arbitrary packet type from a second set of the first set of the packet types associated therewith (e.g., non-VPS NAL units) additionally carries data that, together with the packet having an arbitrary operation point associated therewith, contributes to the encoded representation of the media content at the operation point associated with the corresponding packet, where the arbitrary operation point is before or equal to the operation point associated with the corresponding packet. Each packet having a predetermined packet type associated therewith that is disjoint from (i.e., outside) the second set (such as a VPS NAL unit) has a scalability axis descriptor (such as layer_id_type) that defines the number of one or more scalability axes and their syntax meaning.

[0115] Another example of the data stream, where each packet having a predetermined packet type associated therewith further includes a sequence vps_extension_data of profile and / or level descriptors, which are respectively associated with at least a subset of the plurality of operation points (e.g., leading operation points in the ordered operation point order). Each profile and / or level descriptor defines a set of available coding options (such as a profile) and / or a range of available syntax element values (such as a level). The syntax of the packet having an operation point associated therewith that is associated with the corresponding profile and / or level descriptor complies with the set of available coding options and / or the range of available syntax element values. The first set of the profile and / or level descriptors clearly indicates the definition of the set of available coding options and / or the range of available syntax element values (such as new_profile_level_flag = 1), and the second, disjoint non-empty set of the profile and / or level descriptors indicates the definition of the set of available coding options and / or the range of available syntax element values by referring to another, previous profile and / or level descriptor (such as new_profile_level_flag = 0).

[0116] In another example of the data stream, the sequence of profile and / or level descriptors is sequentially associated with at least a subset of the plurality of operation points (e.g., leading operation points in the ordered operation point order) in the ordered operation point order.

[0117] In another instance of data flow, each profile and / or layer descriptor has a field (such as layer_id) that identifies the operation point associated with the corresponding profile and / or layer descriptor (i.e., along with 4), and the order of the operation points can be derived from the sequence of profiles and / or layer descriptors.

[0118] In another instance of the data flow, in each profile and / or hierarchy descriptor, the field uses a bit sequence common to the operation point identifiers to identify the operation point associated with the corresponding profile and / or hierarchy descriptor.

[0119] In another instance of data flow, all operation point identifiers have a fixed-length syntax element (such as layer_id) and a context syntax element (such as layer_scenario), which indicates the association (but has no semantic meaning) between n and the n partitions of the fixed-length syntax element and n scalability axes, where the bit sequence is common to the fixed-length syntax element.

[0120] In another instance of the data flow, a second, disjoint, non-empty set of configuration files and / or hierarchical descriptors indicates the definition of the available set of encoding options and / or the range of available syntax element values ​​by referring to another, previous configuration file and / or hierarchical descriptor, using explicit representations of the fields of the referenced configuration file and / or hierarchical descriptor, or by using relevant indicators defined in order of sequential operation points in rank position (where the second alternative is an alternative described above).

[0121] According to another aspect of the data flow, all operation point identifiers have a fixed-length syntax element and a context syntax element, which indicates the association between n and the n segments of the fixed-length syntax element and the n scalability axes. Therefore, the nth part is the ordinal measure of the position of the corresponding operation point along the nth scalability axis.

[0122] According to another aspect of the data flow, the scalability axis descriptor (such as layer_id_type) includes syntax elements whose possible states are associated with different combinations of the values ​​of n and the corresponding semantic meanings of the n scalability axes (i.e., the function maps {n,meaning(1),…,meaning(n)} to {1…max.possible states of layer_id_type}).

[0123] In another data stream, in which media content is encoded, the data stream includes packets, each packet including a packet type identifier that identifies, from a plurality of packet types, the packet type associated with the respective packet, wherein each packet having a packet type from a first set of the plurality of packet types associated therewith includes an operation point identifier that identifies, from a plurality of operation points within a scalability space spanned by n scalability axes, the operation point associated with the respective packet, where 0 < n, the operation point having an ordered operation point order defined therein, wherein each packet having any packet type from a second set of the first set of packet types associated therewith additionally carries data that, together with the packet having any operation point associated therewith, contributes to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before, or equal to, the operation point associated with the respective packet, wherein each packet having a predetermined packet type associated therewith that is disjoint from (i.e., outside of) the second set includes a sequence of profile and / or level descriptors, which are respectively associated with at least a subset of the plurality of operation points (e.g., leading operation points in the ordered operation point order), each profile and / or level descriptor defining a set of available coding options and / or a range of available syntax element values, and the syntax of a packet having an operation point associated therewith that is associated with the respective profile and / or level descriptor complies with the set of available coding options and / or the range of available syntax element values, wherein a first set of the profile and / or level descriptors explicitly dictates the definition of the set of available coding options and / or the range of available syntax element values, and a second, disjoint, non-empty set of the profile and / or level descriptors dictates the definition of the set of available coding options and / or the range of available syntax element values by reference to another, previous profile and / or level descriptor.

[0124] In another instance of the data stream, a sequence of profile and / or level descriptors is sequentially associated with at least a subset of the plurality of operation points (e.g., leading operation points in the ordered operation point order) in the ordered operation point order.

[0125] In another instance of the data stream, each profile and / or level descriptor has a field that identifies the operation point with which the respective profile and / or level descriptor is associated.

[0126] In another instance of the data stream, in each profile and / or level descriptor, the field uses a bit sequence common to the operation point identifiers to identify the operation point with which the respective profile and / or level descriptor is associated, the bit sequence identifying the operation point with which the respective profile and / or level descriptor is associated.

[0127] In another instance of the data stream, all operation point identifiers have a fixed bit-length syntax element and a context syntax element that indicates the association of n and n partitions of the fixed bit-length syntax element with n scalability axes, where the bit sequence is common to the fixed bit-length syntax elements.

[0128] In another instance of the data stream, a second, non-empty, disjoint set of profiles and / or layer descriptors indicates the definition of a set of available coding options and / or a range of available syntax element values by referring to another, previous profile and / or layer descriptor, using an explicit indication of fields of the referred profile and / or layer descriptor, or using a relevant indexer that is defined in terms of ranked positions in sequential operation point order.

[0129] In another instance of the data stream, all operation point identifiers have a flag that indicates whether they belong to a first set of profiles and / or layer descriptors and a second, non-empty, disjoint set of profiles and / or layer descriptors, where the first set explicitly indicates the definition of a set of available coding options and / or a range of available syntax element values, and the second set indicates the definition of a set of available coding options and / or a range of available syntax element values by referring to another, previous profile and / or layer descriptor.

[0130] On the other hand, there is a network entity (e.g., a decoder or MANE) for processing a data stream in which media content is encoded, the data stream including packets, each packet including a packet type identifier that identifies the packet type associated with the respective packet from a plurality of packet types, where each packet having a packet type from a first set of the plurality of packet types associated therewith includes an operation point identifier that identifies the operation point associated with the respective packet from a plurality of operation points within a scalability space spanned by n scalability axes, where 0 < n, the operation point having an ordered operation point order defined therein, where each packet having any packet type from a second set of the first set of packet types associated therewith additionally carries data that, together with the packet having any operation point associated therewith, contributes to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before or equal to the operation point associated with the respective packet, where the network entity is configured to read a scalability axis descriptor from a packet having a predetermined packet type associated therewith that is disjoint from (i.e., outside) the second set and to interpret the operation point identifier depending on the scalability axis descriptor, where the scalability axis descriptor defines the number and semantic meaning of one or more scalability axes.

[0131] In another instance, a network entity is configured to read a sequence of configuration files and / or hierarchical descriptors of a predetermined packet type that is disjoint with (i.e., outside) a second set, each associated with at least a subset of multiple operation points (e.g., leading operation points in sequential operation point order). Each configuration file and / or hierarchical descriptor defines a set of available encoding options and / or a range of available syntax element values. The syntax of a packet with an associated operation point associated with the corresponding configuration file and / or hierarchical descriptor conforms to that set of available encoding options and / or that range of available syntax element values. The scope, wherein the network entity is configured to explicitly read the definition of the available encoding option set and / or available syntax element value range from a first set of configuration files and / or hierarchical descriptors in response to a schematic in the data stream, and to read a reference to another, previous configuration file and / or hierarchical descriptor for a second, disjoint, non-empty set of configuration files and / or hierarchical descriptors, and to derive the definition of the available encoding option set and / or available syntax element value range of the second, disjoint, non-empty set of configuration files and / or hierarchical descriptors from the corresponding referenced previous configuration files and / or hierarchical descriptors.

[0132] In another instance of a network entity, a sequence of configuration files and / or hierarchical descriptors is associated sequentially with at least a subset of multiple operation points (e.g., leading operation points in the order of sequential operation points).

[0133] In another instance of a network entity, each profile and / or hierarchical descriptor has a field that identifies the operation point associated with the corresponding profile and / or hierarchical descriptor.

[0134] In another instance of a network entity, in each profile and / or hierarchical descriptor, the field uses a bit sequence common to the operation point identifiers to identify the operation point associated with the corresponding profile and / or hierarchical descriptor.

[0135] In another instance of a network entity, all operation point identifiers have a fixed-length syntax element and a context syntax element, which illustrates the association between n and the n partitions of the fixed-length syntax element and n scalability axes, wherein the bit sequence is common to the fixed-length syntax element.

[0136] In another instance of a network entity, a second, disjoint, non-empty set of configuration files and / or hierarchical descriptors is defined by referring to another, previous configuration file and / or hierarchical descriptor, using explicit representations of the fields of the referenced configuration file and / or hierarchical descriptor, or by using relevant indicators to represent the definition of the available set of encoding options and / or the range of available syntax element values, which are defined in order of operation point in rank position.

[0137] In another instance of a network entity, all operation point identifiers have a fixed-length syntax element and a context syntax element, wherein the network entity is configured to use the context syntax element to determine n and associate the n segments of the fixed-length syntax element with n scalability axes, so that the nth segment is an ordinal measure of the position of the corresponding operation point along the nth scalability axis.

[0138] In another instance, network entities are configured to select from packets of any packet type that have a first set of associated packet types based on interpretation.

[0139] Another example may have a network entity (such as a decoder or MANE) for processing a data stream in which media content is encoded, the data stream including packets, each packet including a packet type identifier that identifies the packet type associated with the respective packet from a plurality of packet types, wherein each packet having a packet type from a first set of the plurality of packet types associated therewith includes an operation point identifier that identifies the operation point associated with the respective packet from a plurality of operation points within a scalability space spanned by n scalability axes, and 0 < n, the operation point having an ordered operation point order defined therein, wherein each packet having any packet type from a second set of the first set of packet types associated therewith additionally carries data that, together with a packet having any operation point associated therewith, contributes to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before or equal to the operation point associated with the respective packet, wherein the network entity is configured to read a sequence of profiles and / or layer descriptors from packets having a predetermined packet type associated therewith that is disjoint from (i.e., outside) the second set, the sequence being sequentially associated with at least a subset of the plurality of operation points (e.g., leading operation points in ordered operation point order), each profile and / or layer descriptor defining a set of available coding options and / or a range of available syntax element values, the syntax of a packet having an operation point associated therewith that is associated with the respective profile and / or layer descriptor conforming to the set of available coding options and / or the range of available syntax element values, wherein a first set of the profiles and / or layer descriptors explicitly indicates the definition of the set of available coding options and / or the range of available syntax element values, wherein the network entity is configured to explicitly read the definition of the set of available coding options and / or the range of available syntax element values from the first set of the profiles and / or layer descriptors in response to an indication in the data stream, and to read a reference to another, previous profile and / or layer descriptor for a second, disjoint, non-empty set of the profiles and / or layer descriptors, and to derive the definition of the set of available coding options and / or the range of available syntax element values for the second, disjoint, non-empty set of the profiles and / or layer descriptors from the respective referenced previous profile and / or layer descriptor.

[0140] In another embodiment, the network entity is configured to select among packets having any packet type from a first set of a plurality of packet types associated therewith based on the associated set of available coding options and / or the range of available syntax element values.

[0141] The following description of embodiments begins with a further introduction, which may, however, also be transferred to the embodiments described so far.

[0142] Beyond video recording, storage, and playback, its transmission is a primary application in today's communication networks. The inherent instability of specific communication channels necessitates adaptation mechanisms provided by HEVC's higher-order syntax to ensure good performance in diverse environments. Direct adaptation of the video bitstream within the network during transmission (e.g., via a Media Aware Network Component (MANE)) remains a major challenge. Such devices typically have limited resources in terms of processing power and must operate with extremely low latency, hindering in-depth bitstream analysis.

[0143] The need for this configuration has been identified within the H.264 / AVC standard. Encapsulating video data within Network Abstraction Layer (NAL) units and the design decisions reflect this understanding: representing many important but relatively invariant parameters outside the video bitstream (in the so-called parameter set). The lack of extensibility in the basic H.264 / AVC specification led to complex methods for extending the H.264 / AVC high-order syntax in a backtracking-compatible manner. For HEVC, a flexible and extensible high-order syntax should be handled in the basic specification. Therefore, the high-order syntax must provide the necessary information to devices such as MANE in an accessible form (e.g., unentropy decoding) suitable for direct adaptation.

[0144] The concept of a Video Parameter Set (VPS), as defined in the current draft specification, is used to provide the information necessary for adapting the video bitstream, such as operation point descriptions and coding dependencies. This information is combined and, in H.264 and its extensions, is distributed across Layer-Specific Sequence Parameter Sets (SPSs) and sits at the top of the parameter set hierarchy.

[0145] The current definition of VPS in the HEVC draft basic specification defines VPS as a redundant copy of a syntax relating only to a sub-layer requirement for time scalability, which is located in SPS in H.264 and its extensions. Therefore, VPS is not necessary for decoding encoded video sequences. According to the following embodiment, however, the redundancy is omitted and the syntax elements remain only in VPS. Furthermore, according to the following embodiment, however, the illustration of profiles, layers, and sub-layer requirements is changed in such a way that their inherent hierarchy reflects the position of the requirement within the hierarchy of the parameter set (VPS, SPS, and PPS). This means that the most basic and fundamental parameters for bitstream adaptation and capability exchange (i.e., profiles and layer information) are provided at the top of the parameter set hierarchy, i.e., VPS, while less important sub-layer requirements (i.e., sps_max_latency_increase[i], vps_num_reorder_pics[i], vps_max_dec_pic_buffering[i]) are illustrated as in earlier versions of the draft specification. In this way, the most important information will be made available to MAME directly without compiling information from various SPSs. Applications relying on lower-level requirements (such as sub-level requirements for event scalability) can still find the desired information in uncompiled form at lower steps of the parameter set hierarchy. Generally, non-specialized applications will have little interest in this low-level capability information of the terminal device. The goal is to make the configuration files and hierarchical requirements sufficiently granular for most applications, while still allowing corner applications to retrieve this information from lower-level parameter sets.

[0146] This method of defining a VPS allows MANE to be fully aware of the scalability dimension, configuration files, and hierarchy associated with each operand. Operands within the scalability dimension can be represented by references (with redundant configuration files and hierarchy identifiers) to reduce redundancy. Utilizing the VPS extended data syntax to represent the corresponding operand descriptions allows for a directly modular structure that can be easily constructed by the encoder.

[0147] The proposed SEI message enables the MANE and receiver to be aware of potentially adapted video sequences, where not all the operating points described in the VPS and its extensions are present. Allowing the MANE to be aware of the actual (adapted) content of the video sequence is important for facilitating further adaptation (e.g., in overlapping adaptation processes across multiple MANEs).

[0148] To generally indicate the operating points of NAL units within the scalability dimensions appearing at the NAL unit header level, reserved bits in the second byte of the NAL unit header are used to indicate layer identifiers, used to identify enhancement layers or views from base layers or views. The VPS provides the information necessary to interpret the layer identifiers of NAL units in terms of the scalability dimensions of the encoded video sequence. This indication includes the type of scalability and information about how the corresponding bits of the layer identifier correspond to the applicable scalability dimensions. This allows MANE or any other device to correctly construct a scalability operating point matrix that may be multi-dimensional (temporal, quality, spatial, multiple views, depth, etc.) from the information provided in the NAL unit header and the VPS.

[0149] Figure 11 The NAL cell syntax of the embodiment described is shown below, where layer_id_plus1 minus 1 indicates the layer identifier. A layer_id_plus1 equal to 1 indicates that the layer identifier is equal to 0, and the NAL cell belongs to a base layer or a base view. Higher values ​​of layer_id_plus1 indicate that the corresponding NAL cell belongs to an enhancement layer or view associated with the specified layer_id_plus1.

[0150] Figures 12A to 12C The corresponding SPS syntax is displayed in the document, and Figure 13 The corresponding VPS syntax is shown, where: `scalability_type` indicates the type of scalability used in the encoded video sequence, indicated by `layer_id_plus1` in the NAL unit header. When `scalability_type` equals 0, the encoded video sequence conforms to the basic HEVC specification, therefore all NAL units have `layer_id_plus1` equal to 1, and no NAL unit belongs to an enhancement layer or view. (The last sentence appears to be incomplete and possibly refers to further details about VPS syntax.) Figure 4 To interpret higher values ​​of scalability_type.

[0151] profile_space, profile_idc, constraint_flags, and profile_compatibility_flag indicate the information specified in the current WD, and the values ​​indicated are valid for all NAL cells where layer_id_plus1 equals 1.

[0152] level_idc[i] indicates the information specified in the current WD, and the indicated value is valid for all NAL cells where layer_id_plus1 equals 1 and temporal_id equals i.

[0153] layer_id_plus1_dim_len[dim] according to Figure 13 The indicator is used to indicate the length of layer_id_plus1 of the nth scalability dimension. For n=0, it starts at the MSB, or for n>0, it ends at the end of the (n-1)th indicated scalability dimension.

[0154] Table 4: Proposed scalability_type mapping table

[0155] scalability_type max_dim Dimension 0 1 none 1 2 Spatial and mass scalability 2 2 Multi-view and depth scalability 3 4 Multi-view, spatial, quality, and depth scalability 4...15 reserve reserve

[0156] The scalability types exemplified in Table 4 represent a set of scalability that is expected at this point in time, but can be extended by new dimensions and / or combinations as work on HEVC extensions progresses.

[0157] Figure 14 The corresponding VPS extended data syntax is displayed, including:

[0158] `vps_extension_type` indicates the type of VPS extension. The syntax given above applies when the value of `vps_extension_type` is equal to 0.

[0159] layer_id_plus1 indicates the layer_id when the syntax element shown in the VPS extension is valid.

[0160] `max_temporal_layers_minus1 plus 1` specifies the maximum number of temporal sublayers that may exist in the NAL cell of `layer_id_plus1` as intended by the VPS.

[0161] A `new_profile_level_flag` value of 0 indicates that the profile and level associated with the value of `layer_id_plus1` (which is equal to the value specified by `profile_reference_layer_id_plus1`) are valid for NAL units with `layer_id_plus1` as specified in the VPS extension. A `new_profile_level_flag` value of 1 indicates that the profile and level for the specified value of `layer_id_plus1` are explicitly indicated in the VPS extension.

[0162] profile_reference_layer_id_plus1 indicates the value of layer_id_plus1. The relevant configuration file and hierarchy of layer_id_plus1 are valid for the NAL unit where the value of layer_id_plus1 is equal to the value of layer_id_plus1 shown in the VPS extension.

[0163] profile_space, profile_idc, constraint_flags, and profile_compatibility_flag are as specified in the current WD, and the indicated values ​​are valid for all NAL cells where layer_id_plus1 equals the layer_id_plus1 value specified in the VPS extended data.

[0164] level_idc[i] indicates the information specified in the current WD, and the indicated value is valid for all NAL units where layer_id_plus1 is equal to the layer_id_plus1 value specified in the VPS extended data and temporal_id is equal to i.

[0165] num_reference_layers specifies the number of layers identified by layer_id that will be used as references in the decoding process.

[0166] direct_coding_dependency_layer_id_plus1 indicates the layer_id_plus1 of the NAL unit that will be used as a reference in the decoding process.

[0167] The embodiment just described is very similar to the one about Figures 4 to 6 The described embodiments. To this extent, Figures 11 to 14 The same reference symbols as before are used to identify the corresponding syntax elements. To this extent, Figure 7 The description also applies to the new embodiments, however, the differences are highlighted below.

[0168] In detail, according to Figures 11 to 14 In this embodiment, the operation point identifier 34 is fixedly divided into two components: a first component 90, which identifies the level of the time scalability axis in the form of the syntax element temporal_id, and a second component, denoted as 92, which processes the level or level according to another scalability axis. To this extent, in addition to having the second component 92, regarding... Figures 11 to 14 The described embodiments correspond to the previous embodiments, and the second component 92 will be further defined by the scalability axis descriptor.

[0169] The next difference concerns the understanding of the number of scalability axes. Figures 4 to 6 In this embodiment, intermediate information in the form of context syntax element 60 is provided to give network entities the possibility of deriving the location of the operation point of the packet based solely on the packet itself, which, in a general sense, is located within the scalability space. In this embodiment, when VPS NAL unit 42 is unavailable, the network entity can only derive the following facts: 1) the location of the current packet's operation point along the time axis using temporal_id 90; 2) if layer_id_plus1 equals 1, then the operation point is at the lowest level with respect to other scalability axes besides the time scalability axis.

[0170] As described below, all the differences highlighted here apply separately and accordingly. If the syntax element temporal_id 90 is not used, the network entity can be deduced from the fact that layer_id_plus1 equals 1: the operation point of the current package represents the base layer according to all scalability axes.

[0171] In this regard, it should be noted that choosing layer_id_plus1 equal to 1 indicates that the lowest level of any scalability axis other than that time scalability axis is selected as an instance only, and can, for example, be changed to layer_id_plus1 equal to 0.

[0172] and Figures 4 to 6 Another difference in the above embodiments concerns the scalability axis descriptor 38. A scalar value syntax element 94, namely, scalability_type, identifies the number of scalability axes (above the time scalability axis) and the corresponding meaning of such (additional) scalability axes, and for each of these indicating the number of (additional) scalability axes, syntax portion 96 discloses a segmentation of a fixed-length syntax element 60 below component 92, field 56, and reference field 52, which is achieved by assigning the bit order of this fixed-length syntax element 60 to the individual (additional) scalability axis.

[0173] Another difference concerns the VPS package and the fact that the VPS package additionally includes an operation point identifier reference list 98. For each field 56, this operation point identifier reference list 98 identifies the value of that operation point identifier or layer_id_plus1, which forms the necessary basis for the reconstruction of the operation point with respect to field 56. In other words, the operation point identifier reference list 98 identifies combination 24 (see... Figure 1B or Figure 3A and Figure 3B)All the operation points contained therein, the combination being produced by adding data of the operation points identified by field 56. It should be noted that for a certain field 56, the operation point identifier reference list 98 does not necessarily list all the operation points contained in the individual combination 24 of the operation points identified by field 56. On the contrary, it is sufficient for the operation point identifier reference list to list one or more of the reference operation points, the corresponding combinations 24 of which together produce the combination 24 of the operation points identified by the individual field 56.

[0174] In other words, in this regard, it is advantageous for the efficient encoding of the layer and / or profile constraints that confer scalability, the data stream (in which the media content is encoded) includes packets (such as NAL units), each packet including a packet type identifier nal_unit_type that identifies the packet type associated with the corresponding packet from a plurality of packet types, wherein each packet having a packet type from the first set of the plurality of packet types associated therewith (e.g., all NAL units) includes an operation point identifier layer_id_plus1 that identifies the operation point associated with the corresponding packet from a plurality of operation points within the scalability space spanned by n scalability axes, and 0 < n, the operation point having an ordered operation point order defined therein, wherein each packet having an arbitrary packet type from the second set of the first set of the packet types associated therewith (e.g., non-VPS NAL units) additionally carries data that, together with the packet having an arbitrary operation point associated therewith, contributes to the encoded representation of the media content at the operation point associated with the corresponding packet, the arbitrary operation point being before or equal to the operation point associated with the corresponding packet, and wherein each packet having a predetermined packet type associated therewith that is disjoint from (i.e., outside) the second set (such as a VPS NAL unit) has a scalability axis descriptor (such as scalability_type and layer_id_plus1_dim_len) that defines the number of one or more scalability axes and their syntax meaning.

[0175] Another instance of a data stream, wherein each of the packets with the associated packet type also includes a sequence of profile and / or level descriptors, vps_extension_data, which are associated with at least a subset of a plurality of operation points (e.g., preceding operation points in the order of sequential operation points), each profile and / or level descriptor defining a set of available encoding options (e.g., a profile) and / or a range of available syntax element values ​​(e.g., a level), the syntax of a packet with an associated operation point associated with the corresponding profile and / or level descriptor conforms to the set of available encoding options and / or the range of available syntax element values, wherein a first set of profile and / or level descriptors explicitly indicates the definition of the set of available encoding options and / or the range of available syntax element values ​​(e.g., new_profile_level_flag = 1), and a second, disjoint, non-empty set of profile and / or level descriptors indicates the definition of the set of available encoding options and / or the range of available syntax element values ​​by reference to another, previous profile and / or level descriptor (e.g., new_profile_level_flag = 0).

[0176] In another instance of a data flow, a sequence of configuration files and / or hierarchical descriptors is associated sequentially with at least a subset of multiple operation points (e.g., leading operation points in sequential operation point order).

[0177] In another instance of data flow, each profile and / or layer descriptor has a field (such as layer_id_plus1) that identifies the operation point associated with the corresponding profile and / or layer descriptor (i.e., along with 4), and the order of the operation points can be derived from the sequence of profiles and / or layer descriptors.

[0178] In another instance of the data flow, in each profile and / or hierarchy descriptor, the field uses a bit sequence common to the operation point identifiers to identify the operation point associated with the corresponding profile and / or hierarchy descriptor.

[0179] In another instance of the data stream, all operation point identifiers have fixed-bit-length syntax elements (such as layer_id_plus1), and the scalability access descriptor indicates the association of n max_dim and n partitions of the fixed-bit-length syntax element layer_id_plus1_dim_len with n scalability axes, where the bit sequence is common to the fixed-bit-length syntax element.

[0180] In another instance of data flow, a second, disjoint, non-empty set of configuration files and / or hierarchical descriptors indicates the definition of the available set of encoding options and / or the range of available syntax element values ​​by referring to another, previous configuration file and / or hierarchical descriptor, using explicit indications of the fields of the referenced configuration file and / or hierarchical descriptor, or by using relevant indicators defined in order of operation point in rank position (where the second alternative is the alternative described above).

[0181] According to another aspect of the data flow, all operation point identifiers have fixed-bit-length syntax elements, and the scalability axis descriptor indicates the association of n segments of the fixed-bit-length syntax elements with the n scalable axes. Therefore, the nth part is the ordinal measure of the position of the corresponding operation point along the nth scalable axis.

[0182] According to another aspect of the data flow, the scalability axis descriptor (such as scalability_type) includes syntax elements whose possible states are associated with different combinations of the values ​​of n and the corresponding semantic meanings of the n scalability axes (i.e., the function maps {n,meaning(1),…,meaning(n)} to {1…max.scalability_type possible states}).

[0183] In another data stream in which media content is encoded, the data stream includes packets, each packet including a packet type identifier that identifies, from a plurality of packet types, the packet type associated with the respective packet, wherein each packet having a packet type from a first set of the plurality of packet types associated therewith includes an operation point identifier that identifies, from a plurality of operation points within a scalability space spanned by n scalability axes, the operation point associated with the respective packet, and 0 < n, the operation point having an ordered operation point order defined therein, wherein each packet having any packet type from a second set of the first set of packet types associated therewith additionally carries data that, together with the packet having any operation point associated therewith, contributes to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before or equal to the operation point associated with the respective packet, wherein each packet having a predetermined packet type associated therewith that is disjoint from (i.e., outside) the second set includes a sequence of profile and / or level descriptors, each associated with at least a subset of the plurality of operation points (e.g., leading operation points in the ordered operation point order), each profile and / or level descriptor defining a set of available coding options and / or a range of available syntax element values, the syntax of a packet having an operation point associated with the respective profile and / or level descriptor conforming to the set of available coding options and / or the range of available syntax element values, wherein a first set of the profile and / or level descriptors explicitly dictates the definition of the set of available coding options and / or the range of available syntax element values, and a second, disjoint non-empty set of the profile and / or level descriptors dictates the definition of the set of available coding options and / or the range of available syntax element values by reference to another, previous profile and / or level descriptor.

[0184] In another instance of the data stream, a sequence of profile and / or level descriptors is sequentially associated with at least a subset of the plurality of operation points (e.g., leading operation points in the ordered operation point order) in the ordered operation point order.

[0185] In another instance of the data stream, each profile and / or level descriptor has a field that identifies the operation point with which the respective profile and / or level descriptor is associated.

[0186] In another instance of the data stream, in each profile and / or level descriptor, the field uses a bit sequence common to the operation point identifiers to identify the operation point with which the respective profile and / or level descriptor is associated, the bit sequence identifying the operation point with which the respective profile and / or level descriptor is associated.

[0187] In another instance of the data stream, all operation point identifiers have a fixed bit length syntax element, and the scalability access descriptor indicates the association of n max_dim and n partitions layer_id_plus1_dim_len of the fixed bit length syntax element with n scalability axes, where the bit sequence is common to the fixed bit length syntax element.

[0188] In another instance of the data stream, a second, disjoint non-empty set of profiles and / or layer descriptors indicates the definition of a set of available coding options and / or a range of available syntax element values by referring to another, previous profile and / or layer descriptor, using an explicit indication of fields of the referred profile and / or layer descriptor, or using a related indexer, which is defined in ranked positions in the order of operation points.

[0189] In another instance of the data stream, all operation point identifiers have a flag indicating whether they belong to a first set of profiles and / or layer descriptors and a second, disjoint non-empty set of profiles and / or layer descriptors, where the first set explicitly indicates the definition of a set of available coding options and / or a range of available syntax element values, and the second set indicates the definition of a set of available coding options and / or a range of available syntax element values by referring to another, previous profile and / or layer descriptor.

[0190] On the other hand, there is a network entity (e.g., a decoder or a MANE) for processing a data stream into which media content is encoded, the data stream including packets, each packet including a packet type identifier that identifies the packet type associated with the respective packet from a plurality of packet types, where each packet having a packet type from a first set of the plurality of packet types associated therewith includes an operation point identifier that identifies the operation point associated with the respective packet from a plurality of operation points within a scalability space spanned by n scalability axes, and 0 < n, the operation point having an order operation point order defined therein, where each packet having any packet type from a second set of the first set of packet types associated therewith additionally carries data that, together with the packet having any operation point associated therewith, contributes to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before or equal to the operation point associated with the respective packet, where the network entity is configured to read a scalability axis descriptor from a packet having a predetermined packet type associated therewith that is disjoint from (i.e., outside) the second set and to interpret the operation point identifier depending on the scalability axis descriptor, where the scalability axis descriptor defines the number and semantic meaning of one or more scalability axes.

[0191] In another instance, a network entity is configured to read a sequence of configuration files and / or hierarchical descriptors of a predetermined packet type that is disjoint with a second set (i.e., outside the second set), each associated with at least a subset of multiple operation points (e.g., leading operation points in sequential operation point order). Each configuration file and / or hierarchical descriptor defines a set of available encoding options and / or a range of available syntax element values. The syntax of a packet with an associated operation point associated with the corresponding configuration file and / or hierarchical descriptor conforms to that set of available encoding options and / or that range of available syntax element values. The scope, wherein the network entity is configured to explicitly read the definition of the available encoding option set and / or available syntax element value range from a first set of configuration files and / or hierarchical descriptors in response to a schematic in the data stream, and to read a reference to another, previous configuration file and / or hierarchical descriptor for a second, disjoint, non-empty set of configuration files and / or hierarchical descriptors, and to derive the definition of the available encoding option set and / or available syntax element value range of the second, disjoint, non-empty set of configuration files and / or hierarchical descriptors from the individually referenced previous configuration files and / or hierarchical descriptors.

[0192] In another instance of a network entity, a sequence of configuration files and / or hierarchical descriptors is associated sequentially with at least a subset of multiple operation points (e.g., leading operation points in the order of sequential operation points).

[0193] In another instance of a network entity, each profile and / or hierarchical descriptor has a field that identifies the operation point associated with the corresponding profile and / or hierarchical descriptor.

[0194] In another instance of a network entity, in each profile and / or hierarchical descriptor, the field uses a bit sequence common to the operation point identifiers to identify the operation point associated with the corresponding profile and / or hierarchical descriptor.

[0195] In another instance of a network entity, all operation point identifiers have fixed-bit-length syntax elements, and the scalability access descriptor indicates the association of n max_dim and n partitions layer_id_plus1_dim_len of the fixed-bit-length syntax element with n scalable axes, where the bit sequence is common to the fixed-bit-length syntax element.

[0196] In another instance of a network entity, a second, disjoint, non-empty set of configuration files and / or hierarchical descriptors is used to indicate the definition of the available set of encoding options and / or the range of available syntax element values ​​by referring to another, previous configuration file and / or hierarchical descriptor, using explicit indications of the fields of the referenced configuration file and / or hierarchical descriptor, or by using relevant indicators defined in order of operation point in rank position.

[0197] In another instance of the network entity, all operation point identifiers have fixed-bit-length syntax elements, and the scalability access descriptor indicates the association of n max_dim and the n segments layer_id_plus1_dim_len of the fixed-bit-length syntax elements with the n scalability axes. Therefore, the nth part is the ordinal measure of the position of the corresponding operation point along the nth scalability axis.

[0198] In another instance, network entities are configured to select from packets of any packet type that have a first set of associated packet types based on interpretation.

[0199] Another example may have a network entity (such as a decoder or MANE) for processing a data stream in which media content is encoded, the data stream including packets, each packet including a packet type identifier that identifies, from a plurality of packet types, the packet type associated with the respective packet, wherein each packet having a packet type from a first set of the plurality of packet types associated therewith includes an operation point identifier that identifies, from a plurality of operation points within a scalability space spanned by n scalability axes, the operation point associated with the respective packet, and 0 < n, the operation point having an ordered operation point order defined therein, wherein each packet having any packet type from a second set of the first set of packet types associated therewith additionally carries data that, together with the packet having any operation point associated therewith, contributes to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before or equal to the operation point associated with the respective packet, wherein the network entity is configured to read a sequence of profiles and / or level descriptors from packets having a predetermined packet type associated therewith that is disjoint (i.e., outside the second set) from the second set, the sequence being sequentially associated with at least a subset of the plurality of operation points (e.g., leading operation points in the ordered operation point order), each profile and / or level descriptor defining a set of available coding options and / or a range of available syntax element values, the syntax of the packet having an operation point associated therewith corresponding to the respective profile and / or level descriptor complying with the set of available coding options and / or the range of available syntax element values, wherein a first set of the profiles and / or level descriptors explicitly indicates the definition of the set of available coding options and / or the range of available syntax element values, wherein the network entity is configured to explicitly read, in response to an indication in the data stream, the definition of the set of available coding options and / or the range of available syntax element values from the first set of profiles and / or level descriptors, and to read a reference to another, previous profile and / or level descriptor for a second, disjoint non-empty set of the profiles and / or level descriptors, and to derive the definition of the set of available coding options and / or the range of available syntax element values for the second, disjoint non-empty set of the profiles and / or level descriptors from the individually referenced previous profile and / or level descriptor.

[0200] In another embodiment, the network entity is configured to select among packets having any packet type from a first set of a plurality of packet types associated therewith based on the associated set of available coding options and / or the range of available syntax element values.

[0201] The following differences from the above embodiments will be feasible. For example, instead of having a profile / tier descriptor for exactly one operation point or tier ID, the operation point or tier ID can be indicated in the data flow (such as in a VPS) to be grouped into tier sets, where each profile / tier descriptor is associated with an individual in that set. For example, a flag field can be indicated to indicate the grouping: for each set, there can be a flag for each operation point indicating whether the corresponding operation point belongs to the corresponding set. Therefore, instead of field 56 referencing any tier ID, field 56 of the profile / tier descriptor can be indexed to one of the sets indicated by the tier IDs (i.e., the set of operation points). Thus, a reference to another profile / tier descriptor will be accomplished by indicating another tier set in field 52, rather than indicating an individual tier ID therein.

[0202] Furthermore, the scalability axis descriptor 38 can be illustrated in different ways: a scalability type field, consisting of a sequence of markers (each associated with a specific scalability type), can exist in the data stream, such as in a VPS, and indicate the presence of individual scalability types. The order in which the markers are set within this field indicates the number of scalability axes and their semantic meaning. Subdividing the bit sequence of the layer ID field 34 into segments can be accomplished using the length indicator for each scalability type (i.e., each marker set in the scalability type field), just as it is done via layer_id_plus1_dim_len. The scalability type field and layer_id_plus1_dim_len together form the scalability axis descriptor 38.

[0203] In the preceding text, a pair of reference operation point identifier lists 98 and a field 56 identifying the operation point (or set of operation points) associated with the reference operation point identifier list 98 are used to indicate the sequential operation point order, i.e., the dependency between layers. This allows an operation point to depend on more than one of its previous operation points (along the order of its operation point identifiers). In use cases with multiple camera views, for example, it is advantageous to have multiple reference views (i.e., multiple values ​​of nuh_layer_id) indicating each encoded view (i.e., individual values ​​of nuh_layer_id) because depending on the translation of multiple cameras about each other, blocking and exposure of corresponding image areas in adjacent camera views may occur. Therefore, referencing multiple camera views can lead to higher encoding efficiency because the exposed image area (i.e., texture) can be predicted rather than bounded based on any of the referenced camera views. The benefits of encoding are not limited to references within the same scalability dimension; for example, a depth view (regarding the currently encoded camera view or another camera view) can be used to remove motion vectors from the currently encoded dependent camera view. However, allowing an operation point to depend on more than one operation point with an associated lower operation point identifier can also be implemented differently. For example, for each operation point, a marker can indicate which of the operation points with the preceding layer ID (i.e., those with lower operation point identifiers) the corresponding operation point depends on. This field can be represented in a compressed form: the N operation points appearing in the bitstream have an order defined therein, which is defined by their operation point identifiers. Therefore, a sequence of operation points can be scanned from the second but lowest operation point to the highest operation point in the sense of the operation point identifier, where each operation point with a lower operation point identifier inserts a marker into the VPS, thus requiring N(N-1) / 2 bits. That is, for each of the multiple operation points (or at least a subset of those operation points), there may be a syntax in the VPS that identifies other operation points of the multiple operation points identified by the lower operation point identifier, such as a reference layer ID list 98 or a sequence of tags relating to the previous operation point. In other words, the syntax identifies those operation points to which the corresponding operation point to which the syntax belongs is an extension of those operation points, thereby identifying the sequential operation point order.

[0204] Although some aspects have been described in the context of the device, it is clear that such aspects also represent a description of the corresponding method, where blocks or devices correspond to method steps or features of method steps. Similarly, aspects described in the context of method steps also represent a description of corresponding blocks, items, or features of the corresponding device. Some or all of the method steps may be performed by (or using) hardware devices, such as microprocessors, programmable computers, or electronic circuits. In some embodiments, one or more of the most important method steps may be performed by this device.

[0205] The encoded data stream of the present invention can be stored on a digital storage medium or transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium (such as the Internet).

[0206] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software. This implementation can be performed using a digital storage medium (e.g., floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory) that stores electronically readable control signals, which cooperates with (or is capable of cooperating with) a programmable computer system to perform individual methods. Therefore, the digital storage medium may be computer-readable.

[0207] Some embodiments of the invention include a data carrier having electronically readable control signals, which is capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0208] Typically, embodiments of the present invention can be implemented as a computer program product having program code that, when run on a computer, operates to perform one of the methods. The program code may, for example, be stored on a machine-readable medium.

[0209] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0210] In other words, embodiments of the method of the present invention are therefore computer programs having program code that, when run on a computer, performs one of the methods described herein.

[0211] Another embodiment of the method of the present invention is therefore a data carrier (or digital storage medium, or computer-readable medium) having a computer program recorded thereon for performing one of the methods described herein. The data carrier, the digital storage medium, or the recorded medium is generally tangible and / or non-transitory.

[0212] Another embodiment of the method of the present invention is therefore a data stream or signal sequence representing a computer program for performing one of the methods described herein. This data stream or signal sequence may, for example, be configured to be transmitted via a data communication connection (e.g., via the Internet).

[0213] Another embodiment includes a processing component, such as a computer or programmable logic device, configured or adapted to perform one of the methods described herein.

[0214] Another embodiment includes a computer having a computer program installed thereon for performing one of the methods described herein.

[0215] Another embodiment according to the present invention includes a device or a system configured to transmit (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver can be, for example, a computer, a mobile device, a memory device, etc. The device or system can include, for example, an archive server for transmitting the computer program to the receiver.

[0216] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all of the functionality of the methods described herein. In some embodiments, the field programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. Generally, the method is preferably performed by any hardware device.

[0217] The embodiments described above merely illustrate the principles of the present invention. It should be understood that those of ordinary skill in the art will appreciate modifications and variations to the arrangements and details described herein. Therefore, the intention of the present invention is limited only by the scope of the appended claims and not by the specific details presented by way of description and illustration of the embodiments herein.

[0218] The present invention can also be configured in the following manner.

[0219] 1. A data stream into which media content is encoded, the data stream including packets (12), each packet (12) including a packet type identifier (16) that identifies the packet type associated with the corresponding packet from a plurality of packet types,

[0220] wherein each packet having a packet type from a first set of the plurality of packet types associated with each packet (12) includes an operation point identifier (34) that identifies an operation point (18) associated with the corresponding packet from a plurality of operation points within a scalability space (20) spanned by n scalable axes (22a, 22b), and 0 < n, and the operation points have an ordered operation point order defined among the operation points,

[0221] wherein each packet having any packet type from a second set of the first set of packet types associated with each packet (12) additionally carries data (36) that, together with the packet having any operation point associated with the packet, contributes to an encoded representation of the media content at the operation point associated with the corresponding packet, the any operation point being before or equal to the operation point associated with the corresponding packet,

[0222] Each package (44) having a predetermined package type that does not intersect with the second set has a scalability axis descriptor (38) that defines the number n and semantic meaning of the one or more scalability axes.

[0223] 2. The data stream according to 1, wherein each of the packets having a predetermined packet type associated with each of the packets further includes a sequence (54) of one or more configuration files and / or hierarchical descriptors (44) associated with at least a subset of the plurality of operation points, each configuration file and / or hierarchical descriptor (44) defining a set of available encoding options and / or a range of available syntax element values, the syntax of a packet having an operation point associated with it and associated with the corresponding configuration file and / or the corresponding hierarchical descriptor conforming to the set of available encoding options and / or the range of available syntax element values, wherein a first set of configuration files and / or hierarchical descriptors explicitly indicates the definition of the set of available encoding options and / or the range of available syntax element values, and a second, disjoint, non-empty set of configuration files and / or hierarchical descriptors indicates the definition of the set of available encoding options and / or the range of available syntax element values ​​by reference to another configuration file and / or hierarchical descriptor.

[0224] 3. The data flow according to 2, wherein each configuration file and / or hierarchical descriptor (44) has a field (56) that identifies the operation point (18) associated with the corresponding configuration file and / or the corresponding hierarchical descriptor (44).

[0225] 4. According to the data flow described in 3, wherein the second, disjoint, non-empty set of the configuration file and / or the hierarchical descriptor (44) indicates the definition of the available encoding option set and / or the range of available syntax element values ​​by referring to another configuration file and / or hierarchical descriptor and using an explicit indication (52) of the fields of the referenced configuration file and / or hierarchical descriptor.

[0226] 5. The data stream according to 1, wherein for each of at least a subset of the plurality of operation points, each of the packets having a predetermined packet type associated with each of the packets further includes a syntax (98) that identifies other operation points of the plurality of operation points identified by a lower operation point identifier, the corresponding operation point being an extension of the plurality of operation points, thereby indicating the sequential operation point order.

[0227] 6. According to the data stream described in 3, each field (56) uses a bit sequence (60) common to the operation point identifier to identify the corresponding operation point, the bit sequence identifying the corresponding operation point.

[0228] 7. The data stream according to 6, wherein all operation point identifiers have fixed bit length syntax elements (60), and the scalability axis descriptor (38) indicates the association of n and n partitions of the fixed bit length syntax element (60) with the n scalable axes, wherein the bit sequence is common to the fixed bit length syntax elements.

[0229] 8. The data stream according to 6, wherein all operation point identifiers (34) have a fixed bit length syntax element (60) and a context syntax element (64), the context syntax element indicating the association of n and the n segments of the fixed bit length syntax element with the n scalability axes, wherein the bit sequence is common to the fixed bit length syntax element.

[0230] 9. The data stream according to 1, wherein all operation point identifiers (34) have fixed-bit-length syntax elements (60), and the scalability axis descriptor (38) indicates the association of n and the n segments of the fixed-bit-length syntax elements with the n scalable axes, such that the nth segment is an ordinal measure of the position of the corresponding operation point along the nth scalable axis.

[0231] 10. The data stream according to 1, wherein all operation point identifiers (34) have a fixed-length syntax element (60) and a context syntax element (64), the context syntax element indicating the association of n and the n segments of the fixed-length syntax element with the n scalability axes, such that the nth portion is an ordinal measure of the position of the corresponding operation point along the nth scalability axis.

[0232] 11. The data stream according to 1, wherein the scalability axis descriptor (38) includes syntax elements (94), the possible states of which are individually associated with different combinations of the value of n and the semantic meaning of the corresponding semantic meaning of the n scalability axes.

[0233] 12. A data stream in which media content is encoded, the data stream comprising packets (12), each packet comprising a packet type identifier (16) that identifies a packet type associated with a corresponding packet from a plurality of packet types.

[0234] Each packet having a packet type from a first set of the plurality of packet types associated with each packet (12) includes an operation point identifier (34) that identifies an operation point (18) associated with the respective packet from among a plurality of operation points within a scalability space (20) spanned by n scalability axes (22a, 22b), and 0 < n, the operation points having an ordered operation point order defined among the operation points.

[0235] Each packet having any packet type from a second set of the first set of packet types associated with each packet (12) additionally carries data (36) that, together with the packet having any operation point associated with the packet, contributes to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before or equal to the operation point associated with the respective packet.

[0236] Each packet having a predetermined packet type associated with each packet (12) that is disjoint from the second set includes a sequence (54) of profiles and / or level descriptors (44) each associated with one or more of at least a subset of the plurality of operation points, each profile and / or level descriptor (44) defining a set of available coding options and / or a range of available syntax element values, the sequence of packets having an operation point associated with the respective profile and / or the respective level descriptor complying with the set of available coding options and / or the range of available syntax element values, wherein a first set of the profiles and / or the level descriptors explicitly indicates the definition of the set of available coding options and / or the range of available syntax element values, and a second, disjoint non-empty set of the profiles and / or the level descriptors indicates the definition of the set of available coding options and / or the range of available syntax element values by reference to another profile and / or level descriptor.

[0237] 13. The data stream according to 12, wherein each profile and / or level descriptor (44) has a field that identifies the operation point associated with the respective profile and / or the respective level descriptor.

[0238] 14. The data stream according to 13, wherein the second, disjoint non-empty set of the profiles and / or the level descriptors (44) indicates the definition of the set of available coding options and / or the range of available syntax element values by reference to another profile and / or level descriptor and using an explicit indication (52) of the field of the referenced profile and / or level descriptor.

[0239] 15. The data stream according to 12, wherein for each of at least a subset of the plurality of operation points, each of the packets having a predetermined packet type associated with each of the packets further includes a syntax (98) that identifies other operation points of the plurality of operation points identified by a lower operation point identifier, the corresponding operation point being an extension of the plurality of operation points, thereby indicating the sequential operation point order.

[0240] 16. The data stream according to 12, wherein each field (56) uses a bit sequence (60) common to the operation point identifier to identify the corresponding operation point, the bit sequence identifying the corresponding operation point.

[0241] 17. The data stream according to 16, wherein all operation point identifiers have fixed bit length syntax elements (60), and the scalability axis descriptor (38) indicates the association of n and n segments of the fixed bit length syntax element (60) with the n scalable axes, wherein the bit sequence is common to the fixed bit length syntax elements.

[0242] 18. The data stream according to 16, wherein all operation point identifiers (34) have a fixed bit length syntax element (60) and a context syntax element (64), the context syntax element indicating the association of n and the n segments of the fixed bit length syntax element with the n scalability axes, wherein the bit sequence is common to the fixed bit length syntax element.

[0243] 19. The data flow according to 12, wherein all operation point identifiers (34) have a marker indicating whether they belong to the first set of the configuration file and / or the hierarchical descriptor and the second, disjoint, non-empty set of the configuration file and / or the hierarchical descriptor, the first set explicitly indicating the definition of the available encoding option set and / or the range of available syntax element values, and the second, disjoint, non-empty set indicating the definition of the available encoding option set and / or the range of available syntax element values ​​by reference to another, previous configuration file and / or hierarchical descriptor.

[0244] 20. A network entity for processing a data stream, media content being encoded into the data stream, the data stream comprising packets (12), each packet (12) comprising a packet type identifier (16), the packet type identifier identifying a packet type associated with a corresponding packet from a plurality of packet types.

[0245] Each packet having a packet type from a first set of the plurality of packet types associated with each packet (12) includes an operation point identifier (34) that identifies an operation point (18) associated with the respective packet from among a plurality of operation points within a scalability space (20) spanned by n scalability axes (22a, 22b), and 0 < n, and the operation points have an ordered operation point order defined among the operation points.

[0246] Each packet having any packet type from a second set of the first set of packet types associated with each packet (12) additionally carries data (36) that, together with the packet having any operation point associated with the packet, contributes to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before or equal to the operation point associated with the respective packet.

[0247] Wherein the network entity is configured to:

[0248] Read a scalability axis descriptor (38) from a packet having a predetermined packet type that is disjoint from the second set associated with packet (12), the scalability axis descriptor defining the number n and semantic meaning of the one or more scalability axes, and

[0249] Interpret the operation point identifier based on the scalability axis descriptor.

[0250] 21. The network entity according to 20, wherein the network entity is configured to: read a sequence (54) of profiles and / or level descriptors (44) associated individually with at least a subset of the plurality of operation points from a packet having the predetermined packet type that is disjoint from the second set associated with the packet, each profile and / or level descriptor (44) defining a set of available coding options and / or a range of available syntax element values, and the syntax of a packet having an operation point associated with the respective profile and / or the respective level descriptor complies with the set of available coding options and / or the range of available syntax element values, wherein the network entity is configured to: explicitly read, in response to an indication in the data stream, a definition of the set of available coding options and / or the range of available syntax element values from a first set of the profiles and / or the level descriptors, and read a reference to another profile and / or level descriptor for a second, disjoint non-empty set of the profiles and / or level descriptors, and derive a definition of the set of available coding options and / or the range of available syntax element values of the second, disjoint non-empty set of the profiles and / or level descriptors from the respective referenced profile and / or level descriptor.

[0251] 22. The network entity according to 21, wherein each configuration file and / or hierarchical descriptor (44) has a field (56) that identifies the operation point (18) associated with the corresponding configuration file and / or the corresponding hierarchical descriptor (44).

[0252] 23. The network entity according to 22, wherein the second, disjoint, non-empty set of the configuration file and / or the hierarchical descriptor (44) indicates the definition of the available encoding option set and / or the range of available syntax element values ​​by referring to another configuration file and / or hierarchical descriptor and using an explicit indication (52) of the fields of the referenced configuration file and / or hierarchical descriptor.

[0253] 24. The network entity according to 21, wherein the network entity is configured to, for each of at least a subset of the plurality of operation points, also read syntax (98) from the packet having the predetermined packet type associated with the packet, the syntax identifying other operation points of the plurality of operation points identified by a lower operation point identifier, the corresponding operation point being an extension of the plurality of operation points, thereby indicating the sequential operation point order.

[0254] 25. The network entity according to 24, wherein each field (54) uses a bit sequence (60) common to the operation point identifier to identify the corresponding operation point, the bit sequence identifying the corresponding operation point.

[0255] 26. The network entity according to 25, wherein all operation point identifiers have fixed bit length syntax elements (60), and the scalability axis descriptor (38) indicates the association of n and n segments of the fixed bit length syntax element (60) with the n scalability axes, wherein the bit sequence is common to the fixed bit length syntax elements.

[0256] 27. The network entity according to 25, wherein all operation point identifiers (34) have fixed bit length syntax elements (60) and context syntax elements (64), the context syntax elements indicating the association of n and n segments of the fixed bit length syntax elements with the n scalability axes, wherein the bit sequence is common to the fixed bit length syntax elements.

[0257] 28. The network entity according to 20, wherein all operation point identifiers (34) have a fixed bit length syntax element (60), and the scalability axis descriptor (38) indicates n and the association of n partitions of the fixed bit length syntax element with the n scalability axes, such that the nth portion is an ordinal measure of the position of the corresponding operation point along the nth scalability axis.

[0258] 29. The network entity according to 20, wherein all operation point identifiers (34) have a fixed bit length syntax element (60) and a context syntax element (64), wherein the network entity is configured to use the context syntax element to determine n and associate n partitions of the fixed bit length syntax element with the n scalability axes, such that the nth portion is an ordinal measure of the position of the corresponding operation point along the nth scalability axis.

[0259] 30. The network entity according to 20, wherein the network entity is configured to select among the packets having any packet type from the first set of the plurality of packet types associated with the packet based on the interpretation.

[0260] 31. A network entity for processing a data stream, media content being encoded into the data stream, the data stream including packets (12), each packet (12) including a packet type identifier (16) that identifies the packet type associated with the corresponding packet from a plurality of packet types,

[0261] wherein each packet having a packet type from the first set of the plurality of packet types associated with each packet (12) includes an operation point identifier (34) that identifies an operation point (18) associated with the corresponding packet from a plurality of operation points within a scalability space (20) spanned by n scalability axes (22a, 22b), and 0 < n, the operation points having an ordered operation point order defined among the operation points,

[0262] wherein each packet having any packet type from the second set of the packet types associated with each packet (12) additionally carries data (36) that, together with the packet having any operation point associated with the packet, contributes to an encoded representation of the media content at the operation point associated with the corresponding packet, the any operation point being before or equal to the operation point associated with the corresponding packet,

[0263] wherein the network entity is configured to:

[0264] Read a sequence of configuration files and / or level descriptors associated with at least one or more of a subset of the plurality of operating points from packets having a predetermined packet type associated with the packet that is disjoint from the second set, each configuration file and / or level descriptor defining a set of available coding options and / or a range of available syntax element values, and the syntax of the packet having an operating point associated with the corresponding configuration file and / or the corresponding level descriptor conforms to the set of available coding options and / or the range of available syntax element values, wherein the network entity is configured to: explicitly read, in response to a cue in the data stream, the definition of the set of available coding options and / or the range of available syntax element values from a first set of the configuration files and / or the level descriptors, and read, for a second, disjoint non-empty set of the configuration files and / or level descriptors, a reference to another configuration file and / or level descriptor, and derive, from the corresponding referenced configuration file and / or level descriptor, the definition of the set of available coding options and / or the range of available syntax element values for the second, disjoint non-empty set of the configuration files and / or level descriptors.

[0265] 32. The network entity according to 31, wherein the network entity is configured to select, among the packets having any packet type from the first set of the plurality of packet types associated with the packet, based on the associated set of available coding options and / or the associated range of available syntax element values.

[0266] 33. A method for processing a data stream into which media content is encoded, the data stream including packets (12), each packet (12) including a packet type identifier (16) that identifies the packet type associated with the corresponding packet from a plurality of packet types,

[0267] wherein each packet having a packet type from the first set of the plurality of packet types associated with each packet (12) includes an operating point identifier (34) that identifies an operating point (18) associated with the corresponding packet from a plurality of operating points within a scalability space (20) spanned by n scalability axes (22a, 22b), and 0 < n, and the operating points have an ordered operating point order defined among the operating points.

[0268] Each packet of any packet type of the second set of the first set of the packet types associated with each packet (12) additionally carries data (36), the data together with the packet having any operation point associated with the packet contributing to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before or equal to the operation point associated with the respective packet.

[0269] Wherein, the method comprises:

[0270] Reading a scalability axis descriptor (38) from a packet having a predetermined packet type associated with packet (12) that is disjoint from the second set, the scalability axis descriptor defining the number n and semantic meaning of the one or more scalability axes, and

[0271] Interpreting the operation point identifier according to the scalability axis descriptor.

[0272] 34. A method for processing a data stream, media content being encoded into the data stream, the data stream including packets (12), each packet (12) including a packet type identifier (16), the packet type identifier identifying the packet type associated with the respective packet from a plurality of packet types.

[0273] Wherein, each packet having a packet type of the first set of the plurality of packet types associated with each packet (12) includes an operation point identifier (34), the operation point identifier identifying an operation point (18) associated with the respective packet from a plurality of operation points within a scalability space (20) spanned by n scalability axes (22a, 22b), and 0 < n, the operation points having an ordered operation point order defined among the operation points.

[0274] Each packet of any packet type of the second set of the first set of the packet types associated with each packet (12) additionally carries data (36), the data together with the packet having any operation point associated with the packet contributing to an encoded representation of the media content at the operation point associated with the respective packet, the any operation point being before or equal to the operation point associated with the respective packet.

[0275] Wherein, the method comprises:

[0276] A sequence of one or more associated profiles and / or hierarchical descriptors, each associated with a predetermined packet type that is disjoint from the second set, is read from packets having associated packets. Each profile and / or hierarchical descriptor defines a set of available encoding options and / or a range of available syntax element values. The syntax of a packet having an associated operation point associated with the corresponding profile and / or hierarchical descriptor conforms to the set of available encoding options and / or the range of available syntax element values. The network entity is configured to: explicitly read the definition of the set of available encoding options and / or the range of available syntax element values ​​from a first set of profiles and / or hierarchical descriptors in response to a hint in the data stream; and read a reference to another profile and / or hierarchical descriptor for a second, disjoint, non-empty set of profiles and / or hierarchical descriptors; and derive the definition of the set of available encoding options and / or the range of available syntax element values ​​for the second, disjoint, non-empty set of profiles and / or hierarchical descriptors from the corresponding referenced profile and / or hierarchical descriptor.

[0277] 35. A computer program, when run on a computer or processor, for performing the processing method according to 33 and 34.

[0278] References

[0279] [1] Thomas Wiegand, Gary J. Sullivan, Gisle Bjontegaard, Ajay Luthra, “Overview of the H.264 / AVC Video Coding Standard”, IEEE Transactions on Circuits, Systems and Video Technology, Vol. 13, No. N7, July 2003.

[0280] [2] JCT-VC, “High-Efficiency VideoCoding (HEVC) text specification Working Draft 6”, JCTVC-H1003, February 2012.

[0281] [3] ISO / IEC 13818-1: MPEG-2 System Specification.

Claims

1. A method of storing a media signal, comprising storing a data stream on a digital storage medium, the data stream including packets, the data stream having the media signal encoded therein, and generating the media signal by performing an encoding step, the encoding step including: Each package is provided with a package type identifier, which identifies the package type associated with the corresponding package from a plurality of package types. For each package having package types from a first set of package types associated with each package, an operation point identifier is provided. This operation point identifier identifies the operation point associated with the corresponding package from multiple operation points within a scalability space spanned by n scalability axes, where 0 < n, and the operation points have a defined order of operation points. Data is provided for each packet having any packet type from a first set of packet types in a second set, which is associated with each packet. This data, together with the packet having any operation point associated with that packet, facilitates an encoded representation of media content at the operation point associated with the corresponding packet, which is prior to or equal to the operation point associated with the corresponding packet. Insert scalability axis descriptors into the data stream, the scalability axis descriptors defining the number n of the one or more scalable axes and their semantic meaning.

2. The method of claim 1, wherein, Each of the packages having a predetermined package type associated with each of the packages also includes a sequence of one or more associated profiles and / or hierarchical descriptors from at least a subset of the plurality of operation points, each profile and / or hierarchical descriptor defining a set of available encoding options and / or a range of available syntax element values. The syntax of a package having an operation point associated with that profile and / or hierarchical descriptor conforms to the set of available encoding options and / or the range of available syntax element values. A first set of profiles and / or hierarchical descriptors explicitly indicates the definition of the set of available encoding options and / or the range of available syntax element values, and a second, disjoint, non-empty set of profiles and / or hierarchical descriptors indicates the definition of the set of available encoding options and / or the range of available syntax element values ​​by reference to another profile and / or hierarchical descriptor.

3. The method of claim 2, wherein, Each configuration file and / or hierarchical descriptor has a field that identifies the operation point associated with the corresponding configuration file and / or the corresponding hierarchical descriptor.

4. The method of claim 3, wherein, The second, disjoint, non-empty set of the configuration file and / or the hierarchy descriptor indicates the definition of the available encoding option set and / or the range of available syntax element values ​​by referencing another configuration file and / or hierarchy descriptor and using an explicit indication of the fields of the referenced configuration file and / or hierarchy descriptor.

5. The method of any one of claims 1 to 4, wherein, For each of at least a subset of the plurality of operation points, each of the packages having a predetermined package type associated with each of the packages also includes a syntax that identifies other operation points of the plurality of operation points identified by a lower operation point identifier, the corresponding operation point being an extension of the plurality of operation points, thereby indicating the sequential operation point order.

6. The method of claim 3, wherein, Each field uses a common bit sequence of the operation point identifier to identify the corresponding operation point, the bit sequence identifying the corresponding operation point.

7. The method of claim 6, wherein, All operation point identifiers have fixed-bit-length syntax elements, and the scalability axis descriptor indicates the association of n and the n segments of the fixed-bit-length syntax elements with the n scalable axes, wherein the bit sequence is common to the fixed-bit-length syntax elements.

8. The method of claim 6, wherein, All operation point identifiers have a fixed-length syntax element and a context syntax element, wherein the context syntax element indicates the association of n and the n segments of the fixed-length syntax element with the n scalability axes, and the bit sequence is common to the fixed-length syntax element.

9. The method of claim 1, wherein, All operation point identifiers have fixed-bit-length syntax elements, and the scalability axis descriptor indicates the association of n and the n segments of the fixed-bit-length syntax elements with the n scalable axes, such that the nth segment is an ordinal measure of the position of the corresponding operation point along the nth scalable axis.

10. The method of claim 1, wherein, All operation point identifiers have a fixed-length syntax element and a context syntax element, the context syntax element indicating the association of n and the n segments of the fixed-length syntax element with the n scalability axes, such that the nth part is an ordinal measure of the position of the corresponding operation point along the nth scalability axis.

11. The method of claim 1, wherein, The scalability axis descriptor includes syntax elements, the possible states of which are individually associated with different combinations of the value of n and the semantic meaning of the corresponding semantic meaning of the n scalability axes.

12. A method of storing a media signal, comprising storing a data stream on a digital storage medium, the data stream including packets, the data stream having the media signal encoded therein, and generating the media signal by performing an encoding step, the encoding step including: Each package is provided with a package type identifier, which identifies the package type associated with the corresponding package from a plurality of package types. For each package having package types from a first set of package types associated with each package, an operation point identifier is provided. This operation point identifier identifies the operation point associated with the corresponding package from multiple operation points within a scalability space spanned by n scalability axes, where 0 < n, and the operation points have a defined order of operation points. Data is provided for each packet of any packet type from a first set and a second set of packet types, each packet having data associated with it. This data, together with the packet having any operation point associated with it, facilitates an encoded representation of the media signal at the operation point associated with the corresponding packet, which is prior to or equal to the operation point associated with the corresponding packet. A sequence of configuration files and / or hierarchical descriptors, each associated with at least a subset of the plurality of operation points, is inserted into the data stream. Each configuration file and / or hierarchical descriptor defines a set of available encoding options and / or a range of available syntax element values. A sequence of packets with associated operation points that are associated with the corresponding configuration file and / or hierarchical descriptor conforms to the set of available encoding options and / or the range of available syntax element values. A first set of configuration files and / or hierarchical descriptors explicitly indicates the definition of the set of available encoding options and / or the range of available syntax element values, and a second, disjoint, non-empty set of configuration files and / or hierarchical descriptors indicates the definition of the set of available encoding options and / or the range of available syntax element values ​​by reference to another configuration file and / or hierarchical descriptor.

13. The method of claim 12, wherein, Each configuration file and / or hierarchical descriptor has a field that identifies the operation point associated with the corresponding configuration file and / or the corresponding hierarchical descriptor.

14. The method of claim 13, wherein, The second, disjoint, non-empty set of the configuration file and / or the hierarchy descriptor indicates the definition of the available encoding option set and / or the range of available syntax element values ​​by referencing another configuration file and / or hierarchy descriptor and using an explicit indication of the fields of the referenced configuration file and / or hierarchy descriptor.

15. The method of any one of claims 12 to 14, wherein, For each of at least a subset of the plurality of operation points, each of the packages having a predetermined package type associated with each of the packages also includes a syntax that identifies other operation points of the plurality of operation points identified by a lower operation point identifier, the corresponding operation point being an extension of the plurality of operation points, thereby indicating the sequential operation point order.

16. The method of claim 12, wherein, Each field uses a common bit sequence of the operation point identifier to identify the corresponding operation point, the bit sequence identifying the corresponding operation point.

17. The method of claim 16, wherein, All operation point identifiers have fixed-bit-length syntax elements, and the scalability axis descriptor indicates the association of the n segments of the fixed-bit-length syntax elements with the n scalability axes, wherein the bit sequence is common to the fixed-bit-length syntax elements.

18. The method of claim 16, wherein, All operation point identifiers have a fixed-length syntax element and a context syntax element, wherein the context syntax element indicates the association of n and the n segments of the fixed-length syntax element with the n scalability axes, and the bit sequence is common to the fixed-length syntax element.

19. The method of claim 12, wherein, All operation point identifiers have a marker indicating whether they belong to the first set of the configuration file and / or the hierarchical descriptor and the second, disjoint, non-empty set of the configuration file and / or the hierarchical descriptor. The first set explicitly indicates the definition of the available encoding option set and / or the range of available syntax element values, and the second, disjoint, non-empty set indicates the definition of the available encoding option set and / or the range of available syntax element values ​​by referring to another, previous configuration file and / or hierarchical descriptor.

20. A decoder for decoding a data stream, media content being encoded into the data stream, the data stream comprising packets, each packet comprising a packet type identifier, the packet type identifier identifying a packet type associated with a corresponding packet from a plurality of packet types. wherein Each package, having package types from a first set of said multiple package types associated with each package, includes an operation point identifier that identifies the operation point associated with the corresponding package from a plurality of operation points within a scalability space spanned by n scalability axes, where 0 < n, and the operation points have an ordered operation point sequence defined among said operation points. The decoder is configured as follows: Read scalability axis descriptors from the data stream, wherein the scalability axis descriptors define the number n and semantic meaning of the one or more scalable axes, and For each package of the package types in the first set of the plurality of package types, Read the operation point identifier of the corresponding packet, and The appropriate packet to be decoded is selected based on the corresponding packet's operation point identifier and the scalability axis descriptor.

21. The decoder of claim 20, wherein, All operation point identifiers have fixed-bit-length syntax elements, and the scalability axis descriptor indicates the association of n and the n segments of the fixed-bit-length syntax elements with the n scalable axes, such that the nth segment is an ordinal measure of the position of the corresponding operation point along the nth scalable axis.

22. An encoder for encoding media signals into a data stream, the data stream comprising packets, the encoder being configured to: Each package is provided with a package type identifier, which identifies the package type associated with the corresponding package from a plurality of package types. Provide a package operation point identifier for each package type from a first set of package types, each operation point identifier identifying the operation point associated with the corresponding package from a plurality of operation points spanned by n scalability axes, where 0 < n, and the operation points having a defined order of operation points. Scalability axis descriptors are inserted into the data stream, wherein the scalability axis descriptors define the number n and semantic meaning of the one or more scalable axes, and For each package of the package types in the first set of the plurality of package types, The operation point identifier of the corresponding package is written in a manner that selects the corresponding package for decoding based on the operation point identifier of the corresponding package and based on the scalability axis descriptor.

23. The encoder of claim 22, wherein, All operation point identifiers have fixed-bit-length syntax elements, and the scalability axis descriptor indicates the association of n and the n segments of the fixed-bit-length syntax elements with the n scalable axes, such that the nth segment is an ordinal measure of the position of the corresponding operation point along the nth scalable axis.

24. A decoding method for decoding a data stream, media content being encoded into the data stream, the data stream comprising packets, each packet comprising a packet type identifier, the packet type identifier identifying a packet type associated with a corresponding packet from a plurality of packet types. wherein Each package, having package types from a first set of said multiple package types associated with each package, includes an operation point identifier that identifies the operation point associated with the corresponding package from a plurality of operation points within a scalability space spanned by n scalability axes, where 0 < n, and the operation points have an ordered operation point sequence defined among said operation points. The method includes: Scalability axis descriptors are read from the data stream, the scalability axis descriptors defining the number n of the one or more scalable axes and their semantic meaning, and For each package of the package types in the first set of the plurality of package types, Read the operation point identifier of the corresponding packet, and The appropriate packet to be decoded is selected based on the corresponding packet's operation point identifier and the scalability axis descriptor.

25. The method of claim 24, wherein, All operation point identifiers have fixed-bit-length syntax elements, and the scalability axis descriptor indicates the association of n and the n segments of the fixed-bit-length syntax elements with the n scalable axes, such that the nth segment is an ordinal measure of the position of the corresponding operation point along the nth scalable axis.

26. An encoding method for encoding media signals into a data stream, the data stream comprising packets, the method comprising: Each package is provided with a package type identifier, which identifies the package type associated with the corresponding package from a plurality of package types. Provide a package operation point identifier for each package type from a first set of package types, each operation point identifier identifying the operation point associated with the corresponding package from a plurality of operation points spanned by n scalability axes, where 0 < n, and the operation points having a defined order of operation points. Scalability axis descriptors are inserted into the data stream, wherein the scalability axis descriptors define the number n and semantic meaning of the one or more scalable axes, and For each package of the package types in the first set of the plurality of package types, The operation point identifier of the corresponding package is written in a manner that selects the corresponding package for decoding based on the operation point identifier of the corresponding package and based on the scalability axis descriptor.

27. The method of claim 26, wherein, All operation point identifiers have fixed-bit-length syntax elements, and the scalability axis descriptor indicates the association of n and the n segments of the fixed-bit-length syntax elements with the n scalable axes, such that the nth segment is an ordinal measure of the position of the corresponding operation point along the nth scalable axis.

28. A method of storing a media signal, comprising storing a data stream on a digital storage medium, the data stream including packets, the data stream having the media signal encoded therein, and generating the media signal by performing an encoding step, the encoding step including: Each package is provided with a package type identifier, which identifies the package type associated with the corresponding package from a plurality of package types. Provide a package operation point identifier for each package type from a first set of package types, each operation point identifier identifying the operation point associated with the corresponding package from a plurality of operation points spanned by n scalability axes, where 0 < n, and the operation points having a defined order of operation points. Scalability axis descriptors are inserted into the data stream, wherein the scalability axis descriptors define the number n and semantic meaning of the one or more scalable axes, and For each package of the package types in the first set of the plurality of package types, The operation point identifier of the corresponding package is written in a manner that selects the corresponding package for decoding based on the operation point identifier of the corresponding package and based on the scalability axis descriptor.

29. The method of claim 28, wherein, All operation point identifiers have fixed-bit-length syntax elements, and the scalability axis descriptor indicates the association of n and the n segments of the fixed-bit-length syntax elements with the n scalable axes, such that the nth segment is an ordinal measure of the position of the corresponding operation point along the nth scalable axis.

30. A computer-readable storage medium for storing a computer program that, when run on a computer or processor, performs the method according to claim 26.

31. A decoder for processing a data stream, video content being encoded into the data stream, the data stream comprising packets, each packet comprising a packet type identifier, the packet type identifier identifying a packet type associated with a corresponding packet from a plurality of packet types. wherein Each package, having a first set of package types associated with each package, includes an operation point identifier that identifies an operation point associated with the corresponding package from a plurality of operation points within a scalability space spanned by n scalability axes, where 0 < n, and the operation points have an ordered operation point sequence defined among the operation points. Each packet, having any packet type from the first set of packet types associated with each packet, additionally carries data, which, together with the packet having any operation point associated with the packet, contributes to an encoded representation of the video content at the operation point associated with the corresponding packet, the operation point being either prior to or equal to the operation point associated with the corresponding packet. The decoder is configured as follows: A sequence of unentropy-decoded profiles and / or hierarchical descriptors is read from packets of a predetermined packet type that is disjoint with the second set and is associated with each of at least one or more of the plurality of operation points. Each profile and / or hierarchical descriptor defines a set of available encoding options and / or a range of available syntax element values. The syntax of a packet having an operation point associated with that profile and / or hierarchical descriptor conforms to the set of available encoding options and / or the range of available syntax element values. The decoder is configured... This involves: explicitly reading the definition of the available encoding option set and / or the available syntax element value range from the first set of the configuration files and / or the hierarchical descriptors in response to a schematic in the data stream; reading a reference to another configuration file and / or hierarchical descriptor for the second, disjoint, non-empty set of the configuration files and / or the hierarchical descriptors; and deriving the definition of the available encoding option set and / or the available syntax element value range of the second, disjoint, non-empty set of the configuration files and / or the hierarchical descriptors from the corresponding referenced configuration files and / or hierarchical descriptors.

32. The decoder according to claim 31, wherein, The decoder is configured to select from the packages of any package type from the first set of the plurality of package types associated with the package, based on the associated set of available encoding options and / or the associated range of available syntax element values.

33. A method for processing a data stream, wherein media content is encoded into the data stream, the data stream comprising packets, each packet comprising a packet type identifier, the packet type identifier identifying a packet type associated with a corresponding packet from a plurality of packet types. in, Each package, having a first set of package types associated with each package, includes an operation point identifier that identifies an operation point associated with the corresponding package from a plurality of operation points within a scalability space spanned by n scalability axes, where 0 < n, and the operation points have an ordered operation point sequence defined among the operation points. Each packet, having any packet type from the first set of packet types associated with each packet, additionally carries data, which, together with the packet having any operation point associated with the packet, contributes to an encoded representation of the media content at the operation point associated with the corresponding packet, the operation point being either prior to or equal to the operation point associated with the corresponding packet. The method includes: Read scalability axis descriptors from packages of a predetermined package type that is disjoint with the second set, which are associated with packages. These scalability axis descriptors define the number n and semantic meaning of the one or more scalability axes. The operation point identifier is interpreted based on the scalability axis descriptor.

34. A method for decoding a data stream, wherein video content is encoded into the data stream, the data stream comprising packets, each packet comprising a packet type identifier, the packet type identifier identifying a packet type associated with a corresponding packet from a plurality of packet types. in, Each package, having a first set of package types associated with each package, includes an operation point identifier that identifies an operation point associated with the corresponding package from a plurality of operation points within a scalability space spanned by n scalability axes, where 0 < n, and the operation points have an ordered operation point sequence defined among the operation points. Each packet, having any packet type from the first set of the packet types associated with each packet, additionally carries data, which, together with the packet having any operation point associated with the packet, contributes to an encoded representation of the video content at the operation point associated with the corresponding packet, the operation point being either prior to or equal to the operation point associated with the corresponding packet. The method includes: A sequence of unentropy-decoded profiles and / or hierarchical descriptors is read from packets having a predetermined packet type that is disjoint with the second set and is associated with each packet. Each profile and / or hierarchical descriptor is associated with one or more of at least a subset of the plurality of operation points. Each profile and / or hierarchical descriptor defines a set of available encoding options and / or a range of available syntax element values. The syntax of a packet having an associated operation point associated with the corresponding profile and / or the corresponding hierarchical descriptor conforms to the set of available encoding options and / or the range of available syntax element values. The network entity is configured as follows: In response to a schematic in the data stream, the definition of the available encoding option set and / or the available syntax element value range is explicitly read from the first set of the configuration files and / or the hierarchical descriptors, and a reference to another configuration file and / or hierarchical descriptor is read for the second, disjoint, non-empty set of the configuration files and / or the hierarchical descriptors, and the definition of the available encoding option set and / or the available syntax element value range of the second, disjoint, non-empty set of the configuration files and / or the hierarchical descriptors is derived from the corresponding referenced configuration files and / or hierarchical descriptors.

35. A computer-readable storage medium for storing a computer program that, when run on a computer or processor, performs the method according to claim 34.

Images