Determining an SEI message processing list in media coding

Abstract

Embodiments of the invention are related to decoder for decoding a data stream which is associated with supplemental enhancement information (SEI) messages, to encoder for encoding generating a data stream which is associated with SEI messages and are related to corresponding methods and bitstreams.

Description

[0001] Determining an SEI Message Processing List in Media Coding

[0002] Description

[0003] Technical Field

[0004] Embodiments of the invention are related to media decoder for decoding a data stream which is associated with supplemental enhancement information (SEI) messages, to media encoder for generating a data stream which is associated with SEI messages and are related to corresponding methods and bitstreams.

[0005] Background of the Invention

[0006] In the field of digital video coding, supplemental information may be conveyed within a bitstream to signal processing operations that are to be applied during decoding and / or post-processing. Such information may be conveyed, for example, by means of Supplemental Enhancement Information (SEI) messages as defined in video coding standards such as High Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC). SEI messages may indicate processing stages forming part of a processing chain, wherein individual stages may be indicated as mandatory or optional. Such chain processing can be important for a coding quality and coding accuracy / consistency and improvements in this regard are always desired.

[0007] Therefore, it is desired to get a concept which improves the signaling and derivation of SEI processing chains.

[0008] This is achieved by the subject matter of the independent claims of the present application.

[0009] Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application.

[0010] Summary of the Invention

[0011] Embodiments relate to a Media decoder (e.g. a video decoder) for decoding a data stream. The embodiments are configured to decode a data stream (e.g. video data stream) that is associated

[0012] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXwith (e.g. the SEI messages might be conveyed within the data stream, or separately, i.e. out of band) supplemental enhancement information (SEI) messages. The media decoder being configured to decode SEI processing order information from the data stream, determine a list of zero, one or more SEI messages and processing the list of zero, one or more SEI messages (e.g. post-processing steps / stages applied by the media decoder after the decoding a media signal such as a video from the data stream onto the decoded media signal, namely in the processing order and merely those in the list so that a post-processed processed media signal results). The SEI processing order information (e.g. sei_processing_order( )) defines a processing order among, and an optionality with respect to, SEI messages (SEI messages may define parameters for certain post-processing steps / stages to be performed by the media decoder after the decoding a media signal such as a video from the data stream on the decoded media signal) associated with the data stream. The list (e.g. PoSeiList; in other words, this list is a (e.g., not necessarily proper) subset out of the SEI messages ordered along the processing order) of zero, one or more SEI messages is determined based on the SEI processing order information and which of the SEI messages the media decoder supports (e.g. the supported set may include all SEI messages which the media decoder supports, or merely ones supported by the media decoder and fulfilling one or more other characteristics such as based on display characteristics of a display ought to output media decoded from the bitstream by the media decoder; e.g. whether an SEI message is supported by the media decoder may depend on the SEI message’s type, such as whether the media decoder is able to parse the SEI message; for instance, some SEI message might be attributed to media decoders of extended capabilities or a higher profile, exceeding the media decoder’s capabilities or profile). Media decoder according to embodiments of the invention are further configured to, based on the SEI processing order information, attribute the SEI messages to one or more SEI sub-chains comprising a base sub-chain and zero or more subordinate subchains such that each subordinate sub-chain has a, in processing order, contiguous subsequence of one or more SEI messages attributed thereto, wherein the SEI processing order information classifies each SEI message as one of forming - along the processing order - a start of a subchain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain. Media decoder according to embodiments of the invention are further configured to determine the list of zero, one or more SEI messages by determining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information as non-optional, populating the list of zero, one or more SEI messages with the SEI messages along the processing order. Wherein the list of zero, one or more SEI messages

[0013] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXwith the SEI messages along the processing order is populated with the following steps: leaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information as optional (in other words, leaving out SEI messages not supported by the media decoder that are indicated by the SEI processing order information as optional), leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain. Depending on which aspect of the invention the embodiments relate to, the embodiments comprise different further features and functionalities.

[0014] Also, embodiments relate to a media encoder (e.g. a video encoder) configured to generate a data stream (e.g. video data stream) which is associated with (e.g. the SEI messages might be conveyed within the data stream, or separately, i.e. out of band) supplemental enhancement information (SEI) messages. The media encoder being configured to encode SEI processing order information (e.g. sei_processing_order( )) into the data stream which defines a processing order among, and an optionality with respect to, SEI messages (SEI messages may define parameters for certain post-processing steps / stages to be performed by the media decoder after the decoding a media signal such as a video from the data stream on the decoded media signal) associated with the data stream; so that a media decoder, is capable of determining a list (e.g. PoSeiList; in other words, this list is a (not necessarily proper) subset out of the SEI messages ordered along the processing order) of zero, one or more SEI messages based on the SEI processing order information and by, based on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains comprising a base sub-chain and zero or more subordinate sub-chains such that each subordinate sub-chain has a, in processing order, contiguous subsequence of one or more SEI messages attributed thereto, wherein the SEI processing order information classifies each SEI message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a subchain, or continuing - along the processing order - a sub-chain; and determining the list of zero, one or more SEI messages by determining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information as non-optional, populating the list of zero, one or more SEI messages with the SEI messages along the processing order with leaving out SEI messages not supported by the media decoder and is

[0015] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXindicated by the SEI processing order information as optional, leaving out each sub-ordinate subchain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain. Depending on which aspect of the invention the embodiments relate to, the embodiments comprise different further features and functionalities.

[0016] In all embodiments the feature of determining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information as non-optional is an optional step and can be left out.

[0017] In all embodiments the feature of leaving out SEI messages not supported by the media decoder and is indicated by the SEI processing order information as optional, is an optional step and can be left out.

[0018] In all embodiments the feature of leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective sub-ordinate subchain and an SEI message attributed thereto classified as forming an end of the respective subordinate sub-chain is an optional step and can be left out.

[0019] Embodiments related to a first aspect of the invention describing a media decoder are further configured to populate the list of zero, one or more SEI messages with the SEI messages along the processing order with leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective sub-ordinate sub-chain in the processing order, wherein, by bitstream constraint, there does not exist any SEI message attributed to the base sub-chain, which is indicated by the SEI processing order information as being non-optional and which follows, in the processing order, a sub-ordinate sub-chain having an SEI message attributed thereto classified as forming a start of the said sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the said sub-ordinate sub-

[0020] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXchain (e.g. an open ended sub-chain). Embodiments also comprise a corresponding method for decoding.

[0021] Also, embodiments related to the first aspect of the invention describing a media encoder are further configured to encode SEI processing order information into the data stream so that a media decoder is capable of populating the list of zero, one or more SEI messages with the SEI messages along the processing order with leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate subchain and no SEI message attributed thereto classified as forming an end of the respective subordinate sub-chain, and all SEI messages following the respective sub-ordinate sub-chain in the processing order, wherein the media encoder is configured to encode the SEI processing order information attributed to the base sub-chain which is such that there does not exist any SEI message indicated by the SEI processing order information (116) as being non-optional and which follows, in the processing order, a sub-ordinate sub-chain having an SEI message attributed thereto classified as forming a start of the said sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the said sub-ordinate sub-chain.

[0022] Embodiments related to the first aspect of the invention can also be described as having a bitstream constraint on positioning of non-optional base-sub-chain SEI messages; constraining their position to occur only upstream to first (in processing order) open-ended sub-chain in order to prevent such non-optional base-sub-chain SEI messages to change importance, thereby improving the list construction leading to a more efficient processing list signaling.

[0023] Embodiments according to the first aspect of the invention provide an improved mechanism for deriving a processing list of Supplemental Enhancement Information (SEI) messages when SEI processing chains comprise subordinate sub-chains. In particular, by defining a rule according to which subordinate sub-chains that are open-ended and contain unsupported SEI messages are omitted together with all SEI messages following the respective sub-chain in the processing order, the decoder is prevented from continuing the derivation of the processing list in a context where the semantic interpretation of the processing chain would otherwise become ambiguous. Consequently, inconsistent interpretations that may arise when a root process is located downstream of an unsupported sub-chain are avoided, thereby increasing robustness and determinism of the decoding and post-processing behavior.

[0024] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXFurther advantages arise from the bitstream constraint according to which non-optional SEI messages attributed to the base sub-chain are restricted to positions preceding any open-ended subordinate sub-chain in the processing order. By preventing mandatory base-sub-chain SEI messages from appearing downstream of such open-ended structures, the invention avoids situations in which the importance or applicability of mandatory processing stages would implicitly depend on decoder support of preceding subordinate sub-chains. This constraint therefore preserves the intended priority semantics of mandatory SEI messages while enabling efficient construction of the SEI processing list and ensuring consistent signaling and interpretation across decoders with different capability sets.

[0025] Embodiments related to a second aspect of the invention describing a media decoder are further configured to populate the list of zero, one or more SEI messages with the SEI messages along the processing order with leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all sub-ordinate sub-chains following the respective sub-ordinate sub-chain in the processing order, with including (e.g. including, into the list 122, or populating or, appending to, the list with), if present, any SEI message supported by the media decoder, attributed to the base sub-chain and following the respective sub-ordinate sub-chain in the processing order, or any SEI message supported by the media decoder, attributed to the base sub-chain and following the respective sub-ordinate sub-chain in the processing order, which is indicated by the SEI processing order information as non-optional. Embodiments also comprise a corresponding method for decoding.

[0026] Also, embodiments related to the second aspect of the invention describing a media encoder are further configured to encode SEI processing order information into the data stream so that a media decoder is capable of populating the list of zero, one or more SEI messages with the SEI messages along the processing order with leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate subchain and no SEI message attributed thereto classified as forming an end of the respective subordinate sub-chain, and all sub-ordinate sub-chains following the respective sub-ordinate sub-

[0027] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXchain in the processing order, with including (e.g. including it to the list 122), if present, any SEI message supported by the media decoder, attributed to the base sub-chain and following the respective sub-ordinate sub-chain in the processing order, or any SEI message supported by the media decoder, attributed to the base sub-chain and following the respective sub-ordinate subchain in the processing order, which is indicated by the SEI processing order information (116) as non-optional.

[0028] Embodiments relating to the second aspect of the invention can be described as having a variation of list construction by inevitably populating the list with supported non-optional base-sub-chain SEI messages, for example, even when occurring downstream an open-ended sub-chain affected by, including, a not supported non-optional SEI message, thereby proving all non-optional base-sub-chain SEI messages with equal importance irrespective of their positions and improving the list construction leading to a more efficient processing list signaling.

[0029] Embodiments according to the second aspect of the invention provide an improved mechanism for deriving the list of zero, one or more SEI messages when SEI processing chains comprise sub-ordinate sub-chains. In particular, by defining a rule according to which all sub-ordinate subchains following an open-ended sub-ordinate sub-chain containing at least one unsupported SEI message are left out, the media decoder can be prevented from attempting to process subordinate sub-chains whose context has been rendered incomplete by a preceding unsupported sub-ordinate sub-chain. This increases the robustness and determinism of the list construction behavior across conforming decoder implementations.

[0030] Embodiments related to a third aspect of the invention describing a media decoder are further configured to populate the list of zero, one or more SEI messages with the SEI messages along the processing order with leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective sub-ordinate sub-chain in the processing order, wherein, by bitstream constraint, there does not exist any SEI message indicated by the SEI processing order information as forming an end of a subordinate sub-chain which follows, in the processing order, an SEI message classified as forming a start of a further sub-ordinate sub-chain (e.g. may be the respective sub-ordinate sub-chain) which has no SEI

[0031] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXmessage attributed thereto which is classified as forming an end of the further sub-ordinate subchain. Embodiments also comprise a corresponding method for decoding.

[0032] Also, embodiments related to the third aspect of the invention describing a media encoder are further configured to encode SEI processing order information into the data stream so that a media decoder is capable of populating the list of zero, one or more SEI messages with the SEI messages along the processing order with leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate subchain and no SEI message attributed thereto classified as forming an end of the respective subordinate sub-chain, and all SEI messages following the respective sub-ordinate sub-chain in the processing order, wherein, by bitstream constraint, there does not exist any SEI message indicated by the SEI processing order information (116) as forming an end of a subordinate subchain which follows, in the processing order, an SEI message classified as forming a start of a further sub-ordinate sub-chain (e.g. may be the respective sub-ordinate sub-chain) which has no SEI message attributed thereto which is classified as forming an end of the further sub-ordinate sub-chain.

[0033] Embodiments related to the third aspect of the invention can be described as coder having a bitstream constraint on positioning of “closed” sub-chain SEI messages; constraining their position to occur only upstream of the start of a first (in processing order) open-ended sub-chain in order to prevent non-optional base-sub-chain SEI messages after the start of an open subordinate sub-chain, thereby improving the list construction leading to a more efficient processing list signaling.

[0034] Embodiments according to the third aspect of the invention provide an improved mechanism for deriving the list of zero, one or more SEI messages when SEI processing chains comprise subordinate sub-chains. In particular, by defining the bitstream constraint according to which there does not exist any SEI message classified as forming an end of a sub-ordinate sub-chain which follows, in the processing order, an SEI message classified as forming a start of a further subordinate sub-chain having no SEI message attributed thereto classified as forming an end, the third aspect prevents the occurrence of closed sub-ordinate sub-chains downstream of open-ended sub-ordinate sub-chains. This ensures that the structure of the processing chain remains unambiguous and consistently interpretable across all conforming decoder implementations.

[0035] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXThe inventors have found, that when processing chains are structured using sub-chains, situations can arise in which the derivation of the list of SEI messages is terminated without the list being set to empty, even though a root process exists further in the processing chain. This may lead to inconsistent or inaccurate interpretation of the intended processing semantics. Embodiments related to the first second or third aspect of the invention can help mitigate such processing errors, thereby achieving a better coding of the list of SEI messages.

[0036] The following described features and functionalities can be a feature or functionality of any of the de / encoders according to any of the first second or third aspect of the invention.

[0037] In certain embodiments the coding can be performed by coding from or into the data stream, for each of the one or more SEI messages, one or more sub-chain-related (e.g. optionality-related) syntax elements (e.g. po_sei_importance_flag[i] and / or po_sei_processing_degree_flag[i]) comprised by the SEI processing order information, wherein, according to embodiments, a corresponding media decoder can be able to determine the list of zero, one or more SEI messages by determining for each of the one or more SEI messages whether same is a non -optional SEI message (e.g. a mandatory processing stage; e.g. non-optional SEI message (e.g. SEI message Group; e.g. sub-chain), i.e. an SEI message or sub-chain for which po_sei_importance_flag[i] is equal to 1 and po_sei_processing_degree_flag[i] is equal to 1), based on the one or more sub-chain -related syntax elements of the SEI messages of the respective SEI message.

[0038] In certain embodiments the coding can be performed by coding from or into the data stream, for each of the one or more SEI messages, one or more sub-chain-related syntax elements (e.g. po_sei_importance_flag[i] and / or po_sei_processing_degree_flag[i]) comprised by the SEI processing order information, wherein, according to embodiments, a corresponding media decoder can be able to attribute the SEI messages by determining for each of the SEI messages whether same is a start of a sub-chain (e.g. S in Fig. 1; e.g. an SEI message or sub-chain for which po_sei_importance_flag[i] is equal to 1 and po_sei_processing_degree_flag[i] is equal to 0; e.g. the i-th SEI message type starts a sub-chain), an end of a sub-chain (e.g. E in Fig. 1; e.g. an SEI message or sub-chain for which po_sei_importance_flag[i] is equal to 0 and po_sei_processing_degree_flag[i] is equal to 1; e.g. the i-th SEI message type ends a sub-chain), a continuation of a sub chain (e.g. C in Fig. 1; e.g. an SEI message or sub-chain for which

[0039] FH250304PEP-Vorschlag Nachanmeldung v2 - CO - ANMELDEFASSUNG.DOCXpo_sei_importance_flag[i] is equal to 1 and po_sei_processing_degree_flag[i] is equal to 1, wherein no previous, in the processing order, first SEI message indicating a start of a sub-chain exists for which no second SEI message exists that is in between, in the processing order, the first SEI message and the respective SEI message that is indicating an end of a sub-chain; e.g. an SEI message or sub-chain for which po_sei_importance_flag[i] is equal to 1 and po_sei_processing_degree_flag[i] is equal to 0, wherein If a sub-chain start is indicated with index startldx less than i and there is no sub-chain end with index greater than startldx and less than i, the i-th SEI message type belongs to a sub-chain (e.g. is a continuation of a sub-chain).), and a non-optional [e.g. a non-optional SEI message].

[0040] Embodiments according to which the SEI processing order information comprises, for each SEI message, one or more sub-chain-related syntax elements, such as po_sei_importance_flag[i] and po_sei_processing_degree_flag[i], can provide a compact and efficient signaling for the sub-chain structure of the processing chain. By encoding only one or two flags per SEI message, the attribution of each SEI message to one of four possibilities, namely forming a start of a subordinate sub-chain, forming an end of a sub-ordinate sub-chain, continuing a sub-ordinate subchain, or constituting a non-optional SEI message attributed to the base sub-chain, can be unambiguously derived by the media decoder without requiring additional syntax elements, thereby contributing to a more efficient processing list signaling.

[0041] In certain embodiments the coding can be performed by coding from or into the data stream, for each of the one or more SEI messages, one or more sub-chain-related (e.g. optionality-related) syntax elements (e.g. po_sei_importance_flag[i] and / or po_sei_processing_degree_flag[i]) comprised by the SEI processing order information, wherein, according to embodiments, a corresponding media decoder can be able to, when the respective sub-chain-related syntax element indicates the continuation of a sub-chain (e.g. an SEI message or sub-chain for which po_sei_importance_flag[i] is equal to 1 and po_sei_processing_degree_flag[i] is equal to 1), differentiate between the continuation of a sub-chain and a non-optional SEI message of the base sub-chain by analyzing attributions of one or more sub-chain-related syntax elements prior (e.g. downstream), along the processing order, to the respective sub-chain-related syntax element (e.g. If a sub-chain start is indicated (e.g. prior) with index startldx less than i and there is no subchain end with index greater than startldx and less than i, the i-th SEI message type belongs to a sub-chain (e.g. indicates a continuation), otherwise, the i-th SEI message type is a mandatory processing stage (e.g. non-optional SEI message; e.g. non-optional)).

[0042] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXEmbodiments according to which the media decoder differentiates between a continuation of a sub-ordinate sub-chain and a non-optional SEI message attributed to the base sub-chain by analyzing the attributions of one or more sub-chain-related syntax elements preceding the respective SEI message in the processing order can provide a particularly efficient signaling mechanism. By reusing the same syntax element value combination for both roles, for example po_sei_importance_flag[i] equal to 1 and po_sei_processing_degree_flag[i] equal to 1, the number of distinct flag combinations required to fully describe the processing chain structure can be minimized. Such embodiments can avoid additional syntax elements to distinguish between the two roles, thereby reducing the signaling overhead of the SEI processing order information.

[0043] In certain embodiments the coder are video coder configured for reconstructing or encoding a video (e.g. as a media signal) from or into the data stream by block based predictive decoding and transform based residual decoding.

[0044] In certain embodiments the coding can be performed by coding from or into the data stream, for each of the one or more SEI messages, one or more processing-order related syntax elements (e.g. po_sei_processing_order[i]), wherein, according to embodiments, a corresponding media decoder can be able to determine the processing order using the one or more processing-order related syntax elements.

[0045] Embodiments also comprise methods performing any of the functionalities of the previously described media encoder or media decoder.

[0046] Further embodiments are related to a Bitstream generated by any of the media encoders described herein.

[0047] In accordance with embodiments, one or more SEI messages can, for example, be film grain characteristics SEI message. Additionally or alternatively, one or more SEI messages can, for example, be film grain region SEI messages.

[0048] Brief Description of the Drawings

[0049] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXThe drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which:

[0050] Fig. 1 shows an exemplary schematic view of a media decoder related to the first, second or third aspect of the invention;

[0051] Fig. 2 shows an exemplary schematic view of a media decoder related to the first aspect of the invention;

[0052] Fig. 3 shows an exemplary schematic view of a media decoder related to the second aspect of the invention;

[0053] Fig. 4 shows an exemplary schematic view of a media decoder related to the third aspect of the invention;

[0054] Fig. 5 shows an exemplary schematic view of a media encoder related to the first aspect of the invention;

[0055] Fig. 6 shows an exemplary schematic view of a media encoder related to the first aspect of the invention;

[0056] Fig. 7 shows an exemplary schematic view of a media encoder related to the first aspect of the invention;

[0057] Fig. 8a / b Fig. 8a schematically show exemplary sets of SEI messages and possible list than can result from then when being processed by other approaches.

[0058] Detailed Description of the Embodiments

[0059] Equal or equivalent elements or elements with equal or equivalent functionality are denoted in the following description by equal or equivalent reference numerals even if occurring in different figures.

[0060] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXIn the following description, a plurality of details is set forth to provide a more throughout explanation of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail in order to avoid obscuring embodiments of the present invention. In addition, features of the different embodiments described herein after may be combined with each other, unless specifically noted otherwise.

[0061] The description now resumes with a general description of SEI processing and obtaining of a list of zero or more SEI messages for processing. Features and functionalities described herein can be attributed to embodiments related to the first second or third aspect of the invention.

[0062] Embodiments may also be described as performing a SPO Signaling - root processing.

[0063] In accordance with embodiments, the SEI (supplemental enhancement information messages) processing order (SPO) SEI message can be designed to communicate the preferred order in which a decoder should process various types of SEI messages that may be included in a Coded Layer Video Sequence (CLVS). This processing order can, for example, be determined by the encoder, which can also specify which SEI messages are essential (e.g. which are non-optional SEI messages) and which are optional. In scenarios where a corresponding decoder may lack support for certain messages, the SPO message guides the decoder in managing the processing chain effectively, ensuring that critical messages are prioritized while optional messages can be bypassed if necessary.

[0064] The list of SEI messages that may be applied to a picture can be stored, in order, in PoSeiList (e.g. list 122), parts of the syntax and semantics relevant for the derivation of this list are reproduced below. For example, the following table 1 depicts a non-exhaustive, exemplary overview of SEI processing order information. For example, po_sei_importance_flag[i] and po_sei_processing_degree_flag[i] may together, or individually, be viewed as sub-chain related syntax elements (e.g. 115).

[0065] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXTable 1 : Exemplary overview of SEI processing order information

[0066] sei_processing_order( payloadSize ) { Descriptor

[0067] for( i = 0, i < po_num_sei_messages_minus2 + 2; i++ ) {

[0068] po_sei_wrapping_flag[ i ] u(1) po_sei_importance_flag[ i ] u(1) po_sei_processing_degree_flag[ i ] u(1) po_sei_payload_type[ i ] u(12) po_sei_prefix_flag[ i ] u(1) po_sei_processing_order[ i ] u(8) }

[0069] }

[0070]

[0071] po_sei_importance_flag[ i ] may affect the derivation of PoSeiList, which is the list 122 of SEI messages that a decoding system should process fora particular picture picA, as specified herein.

[0072] po_sei_processing_degree_flag[ i ] may affect the derivation of PoSeiList as specified herein.

[0073] A processing chain may contain zero or more sub-chains. A sub-chain may include such SEI message types that either all these SEI message types should be processed by a decoding system or, if the decoding system cannot interpret or does not support one or more of these SEI message types, none of the SEI message types of the sub-chain should be processed. The SEI message types that belong to a sub-chain may be determined by the values of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ] as specified herein.

[0074] For example, po_sei_importance_flag and po_sei_processing_degree_flag may, inter alia, represent the one or more sub-chain-related syntax elements 115, wherein the assignment / attribution to sub-chains may be defined according to the table 2. Specific flag values are to be understood merely as representative and may be changed arbitrary.

[0075] Table 2 is exemplary specifying the interpretation of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ] in accordance with embodiments.

[0076] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXTable 2: Exemplary interpretation of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ]

[0077] po_sei_- po_sei_processing_- Interpretation importance_flag[ i ] degree_flag[ i ]

[0078] When the decoding system cannot interpret or does not support the i-th SEI message type, it shall ignore all data

[0079] 0 0 associated with the loop variable value of i and exclude the i-th SEI message type from the processing chain performed by the decoding system.

[0080] The i-th SEI message type ends a sub0 1

[0081] chain.

[0082] The i-th SEI message type starts a sub1 0

[0083] chain.

[0084] If a sub-chain start is indicated with index startldx less than i and there is no subchain end with index greater than startldx and less than i, the i-th SEI message type belongs to a sub-chain.

[0085] 1 1 Otherwise, the i-th SEI message type is a mandatory processing stage. In this case, when the decoding system cannot interpret or does not support the i-th SEI message type, it should not apply the processing chain.

[0086]

[0087] A sub-chain may be defined as being open-ended when no end of sub-chain is indicated prior to the start of the next sub-chain (if any) or the end of the processing chain, whichever is earlier in the processing order. When the decoding system cannot interpret or does not support the i-th SEI message type for which po_sei_importance_flag[PoSeiTypeldx[ i ]] + po_sei_processing_degree_flag[PoSeiTypeldx[ i ]] is greater than or equal to 1 and the i-th SEI message type belongs to a sub-chain subChainB, the following can apply:

[0088] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX- If subChainB is open-ended, the decoding system 110 may not process subChainB or any of the processing stages following subChainB.

[0089] - Otherwise, the decoding system 110 may be configured to not process subChainB.

[0090] The list PoProcStgldx indicates the processing stage indices of the SEI message types in the processing chain (in other words, PoProcStgldx may indicate the processing order 118), and the list PoSeiTypeldx indicates the SEI message type indices of the processing stages in the processing chain (e.g. PoSeiTypeldx may be used by a decoder 110 to determine if a decoder supports 123 the SEI message type), The lists PoProcStgldx and PoSeiTypeldx is derived as follows:

[0091] - For each of the SEI message types of in the processing chain, the following may apply in a non-decreasing order of the corresponding po_sei_processing_order[ i ] values, with j being set equal to 0 initially:

[0092] PoProcStgldx[ i ] = j

[0093] PoSeiTypeldx[ j ] = i (xx)

[0094] Where PoProcStgldx[ i ] indicates the processing stage index (e.g. that indicates the processing order) of the i-th SEI message type in the processing chain, and PoSeiTypeldx[ j ] indicates the SEI message (114) type index of the j-th processing stage in the processing chain.

[0095] The list poSubChainldx[ j ] for j ranging from 0 to PoNumProcStgs - 1, inclusive, specifying the sub-chain index of the j-th processing stage of the processing chain, is be derived as follows: poSubChainFlag = 0

[0096] poSubChainPrevIdx = 0

[0097] for(j = 0; j < PoNumProcStgs; j++ ) { (xx) idx = PoSeiTypeldx[ j ]

[0098] if( po_sei_importance_flag[ idx ] = = 1 && po_sei_processing_degree_flag[ idx ] = = 1 )

[0099] if( poSubChainFlag = = 0 )

[0100] poSubChainldx[ j ] = 0

[0101] else

[0102] poSubChainldx[ j ] = poSubChainPrevIdx

[0103] else if( po_sei_importance_flag[ idx ] = = 0 && po_sei_processing_degree_flag[ idx ]

[0104] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX= = 1 ) {

[0105] poSubChainldx[ j ] = poSubChainFlag * poSubChainPrevIdx

[0106] poSubChainFlag = 0

[0107] } else if( po_sei_importance_flag[ idx ] = = 1 && po_sei_processing_degree_flag[ idx ] = = 0 ) {

[0108] poSubChainPrevldx++

[0109] poSubChainldx[ j ] = poSubChainPrevIdx

[0110] poSubChainFlag = 1

[0111] } else

[0112] poSubChainldx[ j ] = poSubChainFlag * poSubChainPrevIdx

[0113] }

[0114] It is a requirement of bitstream conformance that when po_sei_importance_flag[ P°SeiTypeldx[ j ] ] is equal to 0, po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equal to 1 for any value of j in the range of 0 to PoNumProcStgs - 1, inclusive, poSubChainldx[ j ] shall be greater than 0.

[0115] NOTE 7 - The requirement above disallows a value combination of po_sei_importance_flag[ PoSeiTypeldx[ j ] ] and po_sei_processing_degree_flag[ P°SeiTypeldx[ j ] ] that indicates an end of a sub-chain to appear outside of a sub-chain.

[0116] For a picture, the list PoSeiList, indicting the list of SEI messages that may be applied to the picture, the list PoSeiTypeList, indicating the SEI message type indices of the SEI messages that may be applied to the picture, and the variable PoNumSeiMsgs, indicating the number of SEI messages that may be applied to the picture, is derived as follows:

[0117] - PoSeiList can be set to be initially empty, and seiListldx and PoNumSeiMsgs can be both initially set equal to 0.

[0118] - The following applies in increasing order of j for all values of j in the range of 0 to PoNumProcStgs - 1, inclusive, unless terminated earlier as specified below:

[0119] - When an SEI message seiA associated with the PoSeiTypeldx[ j ]-th SEI message type persists for picA, the following can apply:

[0120] - If all of the following conditions are true, seiA is added at the end of PoSeiList, PoSeiTypeList[ seiListldx ] is set equal to PoSeiTypeldx[ j ], PoNumSeiMsgs is set equal to PoNumSeiMsgs + 1 , and seiListldx is set equal to seiListldx + 1 :

[0121] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX- The decoding system 110 can interpret and supports the functionality indicated by seiA.

[0122] - Either of the following conditions is true:

[0123] - poSubChainldx[ j ] is equal to 0.

[0124] - The decoding system can interpret and supports the functionality indicated by all SEI message types for all values of k such that poSubChainldx[ k ] is equal to poSubChainldx[ j ] and po_sei_importance_flag[ PoSeiTypeldx[ k ] ] + po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is greater than or equal to 1.

[0125] - Otherwise, if poSubChainldx[ j ] is greater than 0, po_sei_importance_flag[ P°SeiTypeldx[ j ] ] is equal to 1, po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equal to 0, and there is no value of k such that poSubChainldx[ k ] is equal to poSubChainldx[ j ], po_sei_importance_flag[ PoSeiTypeldx[ k ] ] is equal to 0 and po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is equal to 1, the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.

[0126] - Otherwise, if poSubChainldx[ j ] is equal to 0, po_sei_importance_flag[ P°SeiTypeldx[ j ] ] is equal to 1 and po_sei_processing_degree_flag[ P°SeiTypeldx[ j ] ] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, PoNumSeiMsgs is set equal to 0, and the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.

[0127] The description resumes with discussing of a signaling issue for "root" stages. In the above transcription, one can observe that when an SEI message has poSubChainldx[ j ] equal to 0, and both po_sei_importance_flag[ PoSeiTypeldx[j]] and po_sei_processing_degree_flag[ PoSeiTypeldx[j]] equal to 1, it has a greater importance, often referred as a root process. Meaning, it is a process important to the whole processing chain, and in case it's not supported PoSeiList is set to be empty and the derivation of PoSeiList is terminated.

[0128] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXHowever, with the creation of sub-chains it is also possible to terminate the derivation of PoSeiList without setting it to be empty even if there is a root process further in the processing chain. The examples 1 and 2 below illustrates such case:

[0129] Example 1 :

[0130] Example 1 is described with respect to Fig. 8a. Fig. 8a schematically shows an exemplary set of SEI messages 114 along a processing order(from left to right). The SEI messages 114 have been attributed to sub-chains 117, whereas SEI messages 0, 1 and 2 are assigned to sub-ordinate subchains and SEI message 3 is assigned to a base sub-chain. Fig. 8a further shows possible lists 124 (e.g. depending on which SEI messages a decoder supports) that can be obtained by other approaches when adopting the semantics of the present application.

[0131] po_sei_importa poSubC 1 = open-ended po_sei_processing

[0132] nce_flag[poSeiT hainldx[

[0133] j _degree_flag[ poS

[0134] ypeldx[j ]] j ] (e.g.

[0135] eiTypeldx[ j ]]

[0136] 117)

[0137] 0 1 0 1 1

[0138] 1 1 0 2 0

[0139] 2 0 1 2 0

[0140] 3 1 1 0 0

[0141]

[0142] In this example, it is currently possible to have (e.g. possible lists 124):

[0143] • 0, 3

[0144] • 0, 1, 2, 3

[0145] • None (in case either 0 or 3 are not supported)

[0146] Notice that in this case the lack of support from process 0, disallows process 3 to be performed.

[0147] Example 2:

[0148] Example 2 is described with respect to Fig. 8b. Fig. 8b schematically shows an exemplary set of SEI messages 114 along a processing order(from left to right). The SEI messages 114 have been attributed to sub-chains 117, whereas SEI messages 0 to 6 are assigned to sub-ordinate subchains and SEI message 7 is assigned to a base sub-chain. Fig. 8b further shows possible lists 124 that can be obtained by other approaches when adopting the semantics of the present application.

[0149] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXpo_sei_importa po_sei_processing poSubC 1 = open-ended

[0150] j nce_flag[poSeiT _degree_flag[ poS hainldx[

[0151] ypeldx[ j ]] eiTypeldx[ j ]] j ]

[0152] 0 1 0 1 1

[0153] 1 1 0 2 1

[0154] 2 1 1 2 1

[0155] 3 1 0 3 1

[0156] 4 1 1 3 1

[0157] 5 1 0 4 0

[0158] 6 0 1 4 0

[0159] 7 1 1 0 0

[0160]

[0161] In a further example with more open-ended sub-chains, the current specification allows PoSeiList (e.g. list 122) to be terminated without containing root process 7. The following combinations (e.g. possible lists 124) are possible:

[0162] • 0

[0163] • 0, 1, 2

[0164] • 0, 1, 2, 3, 4, 7

[0165] • 0, 1, 2, 3, 4, 5, 6, 7

[0166] • None (if 0 is not supported or all processes - at least 0 up to 4 - but 7 are supported) It is undesirable to have a critical process at the root that can be skipped. In addition, it is undesirable that the derivation of the list of processes described is as follows:

[0167] 1) User 1 supports only 0, 1 and 2 => The list PoSeiList consist of processes 0, 1 and 2 2) User 2 supports only 0, 1 , 2, 3 and 4 => The list PoSeiList is empty.

[0168] This creates a sort of disjoint sub-chain that binds together sub-chain 3 (only) with the 7thprocess and changes the importance of the critical root process 7. It is argued herein, that this is not the intended behavior for this process since it was designed to be critical for the whole processing chain.

[0169] The issue outlined above can be addressed by embodiments of the invention.

[0170] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXThe description resumes with a description of embodiments related to the first aspect of the Invention.

[0171] In accordance with embodiments related to the first aspect of the invention, a constraint can be added that prohibits the assignment of any critical root process if there are existing open-ended sub-chains defined prior to it. This can ensure that the processing hierarchy is maintained, preventing conflicts between critical processes (e.g. defined in non-optional SEI messages) and previously established sub-chains, thereby enhancing the clarity and efficiency of the processing order.

[0172] In accordance with embodiments the above outlined syntax can be modified. Since the semantics in the table for the interpretation of po_sei_importance_flag[ j ] = 1 and po_sei_processing_degree_flag[ j ] = 1 seems not to consider such a case in which such a process belongs to a root after an open-ended sub-chain, in accordance with embodiments, such issue can be addressed by disallowing to define such root processes after open-ended subchains. This can ensure that the processing hierarchy is maintained, preventing conflicts between critical processes and previously established sub-chains, thereby enhancing the clarity and efficiency of the processing order. In accordance with embodiments, the following exemplary constraint is proposed:

[0173] Constraint: It is a requirement of bitstream conformance that when poSubChainldx[ j ] equal to 0 and po_sei_importance_flag[ PoSeiTypeldx[ j ] ] is equal to 1 and po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equal to 1 for any value of j in the range of 0 to PoNumProcStgs - 1, inclusive, there shall not exist any index k, k < j, such that poSubChainldx[ k ] is open-ended.

[0174] This requirement disallows the combination of po_sei_importance_flag[ PoSeiTypeldx[ j ] ] and po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] with poSubChainldx[ j ] equal to 0, indicating a processing stage to be marked as critical to the whole processing chain, when it follows any prior open-ended sub-chain.

[0175] In accordance with embodiments related to the first aspect of the invention, when, during traversal of the processing stages in increasing order of j, the decoder encounters an SEI message type that it does not support, and the corresponding processing stage belongs to a subordinate sub-

[0176] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXchain for which no end-of-sub-chain indicator has been signaled prior to the start of the next subordinate sub-chain or the end of the processing chain, the decoder is configured to terminate the derivation of PoSeiList at that point, retaining all previously appended entries. This reflects the bitstream constraint as in embodiments related to the first aspect, according to which no non-optional base-sub-chain SEI message may follow such an open-ended subordinate sub-chain, thereby ensuring that no mandatory root process is inadvertently omitted as a consequence of the early termination. The encoder, correspondingly, is constrained to generate SEI processing order information that satisfies this structural requirement, guaranteeing that the termination semantics of the decoder lead to a consistent and deterministic outcome across all conforming decoder implementations.

[0177] In accordance with embodiments related to the first aspect of the invention, the media encoder may be configured to validate, prior to finalizing the SEI processing order information, whether any non-optional SEI message attributed to the base sub-chain would be positioned, in processing order, downstream of any subordinate sub-chain that qualifies as open-ended. If such a configuration is detected during the construction of the processing chain, the encoder may be configured to either reorder the processing stages such that all non-optional base-sub-chain SEI messages precede the first open-ended subordinate sub-chain, or to ensure that each subordinate sub-chain that precedes a non-optional base-sub-chain SEI message is closed by an end-of-sub-chain indicator. Optionally, the media encoder can be configured to, if neither reordering nor closure is feasible, omit the non-optional base-sub-chain SEI message from the processing chain and signal it by alternative means. This encoder-side enforcement can, for example achieve that the resulting bitstream is conformant with the bitstream constraint of the first aspect, thereby ensuring that decoders can unambiguously construct the list PoSeiList.

[0178] The description resumes with a description of embodiments related to the second aspect of the Invention.

[0179] In accordance with embodiments related to the second aspect of the invention, embodiments can also be configured to ensure that all existing critical root processes within the processing chain are supported. This implies that all possible combinations of PoSeiList must contain all critical root process.

[0180] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXIn the Example 1, that can mean the following outcomes would now be possible (e.g. possible lists 124):

[0181] • 0, 3

[0182] • 0, 1, 2, 3

[0183] • 3

[0184] • None (if 3 is not supported)

[0185] For Example 2 (e.g. possible lists 124 for Ex. 2 include the following),

[0186] • 0, 7

[0187] • 0, 1, 2, 7

[0188] • 0, 1, 2, 3, 4, 7

[0189] • 0, 1, 2, 3, 4, 5, 6, 7

[0190] • 7

[0191] • None (if 7 is not supported)

[0192] Such embodiments can allow receivers (e.g. decoder 110) with the capabilities to support only the critical root processes to still apply the critical part of the processing chain.

[0193] In accordance with embodiments the above outlined syntax can be modified. For a picture, the list PoSeiList, indicting the list of SEI messages that may be applied to the picture, the list PoSeiTypeList, indicating the SEI message type indices of the SEI messages that may be applied to the picture, and the variable PoNumSeiMsgs, indicating the number of SEI messages that may be applied to the picture, can be derived as follows:

[0194] - PoSeiList is initially empty, and seiListldx and PoNumSeiMsgs are both initially set equal to 0, and restrict_to_root_processes_flag is equal to 0.

[0195] - The following can apply in increasing order of j for all values of j in the range of 0 to PoNumProcStgs - 1, inclusive, unless terminated earlier as specified below:

[0196] - When an SEI message seiA associated with the PoSeiTypeldx[ j ]-th SEI message type persists for picA, the following applies:

[0197] - If all of the following conditions are true, seiA is added at the end of PoSeiList, PoSeiTypeList[ seiListldx ] is set equal to PoSeiTypeldx[ j ], PoNumSeiMsgs is set equal to PoNumSeiMsgs + 1 , and seiListldx is set equal to seiListldx + 1 : - The decoding system can interpret and supports the functionality indicated by seiA.

[0198] - Either of the following conditions is true:

[0199] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX- poSubChainldx[ j ] is equal to 0.

[0200] - restrict_to_root_processes_flag is equal to 0 and the decoding system can interpret and supports the functionality indicated by all SEI message types for all values of k such that poSubChainldx[ k ] is equal to poSubChainldx[ j ] and po_sei_importance_flag[ PoSeiTypeldx[ k ] ] + po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is greater than or equal to 1.

[0201] - Otherwise, if poSubChainldx[ j ] is greater than 0, po_sei_importance_flag[ P°SeiTypeldx[ j ] ] is equal to 1, po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equal to 0, and there is no value of k such that poSubChainldx[ k ] is equal to poSubChainldx[ j ], po_sei_importance_flag[ PoSeiTypeldx[ k ] ] is equal to 0 and po_sei_processing_degree_flag[ P°SeiTypeldx[ k ] ] is equal to 1, restrict_to_root_processes_flag is set to 1.

[0202] - Otherwise, if poSubChainldx[ j ] is equal to 0, po_sei_importance_flag[ P°SeiTypeldx[ j ] ] is equal to 1 and po_sei_processing_degree_flag[ P°SeiTypeldx[ j ] ] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, PoNumSeiMsgs is set equal to 0, and the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.

[0203] In accordance with embodiments related to the second aspect of the invention, once the variable restrict_to_root_processes_flag has been set to 1 during the traversal of the processing stages, it shall remain equal to 1 for all subsequent processing stages regardless of the sub-chain attribution of those stages. In particular, encountering a further subordinate sub-chain, whether open-ended or closed, after restrict_to_root_processes_flag has been set to 1 shall not cause the flag to be reset to 0. This can ensure that once the processing chain has been identified as containing an unsupported open-ended sub-chain, the decoder restricts the further construction of PoSeiList exclusively to SEI messages attributed to the base sub-chain, irrespective of any intervening sub-chain structures. Correspondingly, the encoder, when generating the SEI processing order information for a processing chain intended to be processed under the second aspect, may be aware that any subordinate sub-chains positioned after the first open-ended subordinate sub-chain containing an unsupported SEI message will be entirely excluded from the list PoSeiList of decoders that do not support said open-ended sub-chain.

[0204] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXIn other words, the media decoder can be configured to populate the list of zero, one or more SEI messages with the SEI messages along the processing order such that, once a sub-ordinate subchain has been left out which contains at least one SEI message which is not supported by the media decoder and has an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, all sub-ordinate sub-chains following the respective sub-ordinate sub-chain in the processing order are left out irrespective of whether such further sub-ordinate sub-chains have an SEI message attributed thereto classified as forming an end of the respective further sub-ordinate sub-chain or not.

[0205] The description resumes with a description of embodiments related to the third aspect of the Invention.

[0206] In accordance with embodiments related to the third aspect of the invention, embodiments can also be configured to prevent the early termination of the PoSeiList before reaching a critical root process by disallowing closed sub-chains following open-ended sub-chains. In such embodiments, signaling a critical root (or any process with poSubChainldx[j] = 0) becomes impossible (e.g. is forbidden). Thus, once an open-ended sub-chain is initiated, no other type of sub-chain should be permitted to follow.

[0207] The description resumes with a description of embodiments related to the first second and / or third aspect of the Invention.

[0208] Fig. 1 shows an exemplary schematic view of a decoder 110 according to an embodiment related to the first, second or third aspect of the invention. The decoder 110 can be configured to decode from a data stream 112 SEI processing order information 116 that relate to SEI messages 114 and define a processing order 118 of the SEI messages 114 as well as an optionality of the SEI messages 114. The decoder 110 can be configured to determine a list 122 of zero, one or more SEI messages 114 using the SEI process order information 116. The SEI processing order information 116 may, for example, comprise of one or more sub-chain related syntax elements 115, that, for example, can define, for a respective SEI message 114, a start of a sub-chain 117 (indicated in Fig. 1 with ‘S’), an End of a sub-chain 117 (indicated in Fig. 1 with ‘E’) or a continuation of a sub-chain (indicated in Fig. 1 with ‘E’).

[0209] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXIn accordance with embodiments, the SEI processing order information 116 may comprise exactly one sub-chain related syntax elements 115 for each SEI message 114.

[0210] The sub-chain related syntax elements 115 may, for example, further be used to determine the optionality of the respective SEI message 114, wherein start and end of a sub-chain are deemed non-optional and a fourth state (e.g. 0, 0 as in table 2) of the sub-chain related syntax elements 115 may be used to define the respective SEI message as optional.

[0211] The decoder can be configured to classify 121 the SEI messages in starting, ending or continuing a sub-chain.

[0212] The decoder 110 may, for example, be configured to determine the processing order 118 for the SEI messages 114 based on one or more processing-order related syntax elements [e.g. po_sei_processing_order[i]] that are comprised by the SEI processing order information 116.

[0213] The decoder 110 can, for example, be configured to assign / attributing the respective SEI messages 114 to a base sub-chain 119 or to zero, one or more sub-ordinate sub-chains 120I.4. The assignment / attribution to sub-chains 117 can be made in a manner aware to the processing order. The SEI processing order information 116 signals, for instance, for each SEI message whether a sub-ordinate sub-chain is newly opened (e.g. representing a start) at the respective SEI message 114 in processing order 118, whether a currently open sub-ordinate sub-chain is ended at the respective SEI message, or whether a currently open sub-ordinate SEI message is continued at the respective SEI message.

[0214] If, for example, for a certain sub-chain 117 no end is specified one can call this sub-chain an open-ended sub-chain 120I_3. If, for example, an end of a sub-chain 117 was indicated directly prior (e.g. downstream) to an SEI message for which an indication of a continuation is coded by the SEI processing order information 116, or if the respective SEI Message is the first element in the processing order (i.e. if no sub-chain is currently open at the SEI message in processing order), this SEI message can, for example, be assigned to the base sub-chain 119.

[0215] The classification 121 of SEI messages 114 as optional or non-optional by the SEI processing order information 116 could be done in addition to the indication of start / end / continuation of the

[0216] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXsubordinate sub-chains, i.e. could be done by extra syntax elements, or could be done by common syntax elements whose states indicate both optionality as well as sub-chain extension.

[0217] As a consequence, a respective SEI message 114 that is coded to form a continuation of a subchain and that is attributed to the base sub-chain 119 due to lack of any subordinate sub-chain which would be open, might be signaled to be optional or be non-optional (e.g. non optional). Non-optional SEI messages of the base sub-chain 119 can, for example, be called critical root processes or mandatory processing stages. SEI massages attributed to a start or end of a subordinate sub-chain may be also determined as non-optional. The decoder 110 can, for example, be configured to determine the sub-chain 117 as well as the optionality assignment of SEI messages using two bits, encoded from the data stream 112, e.g. po_sei_importance_flag[i] and po_sei_processing_degree_flag[i], wherein the decoder 110 is configured to determine the assignment according to the table 2.

[0218] The decoder 110 may, for example, be configured to utilize a preliminary list (e.g. poSubChainldx) for the determining of the list 122. The preliminary list may, for example, be determined by the decoder 110 using the above described pseudocode. The preliminary list represents, for instance, the processing list finally to be processed by the decoder for the case that all SEI messages were supported by the decoder 110. The preliminary list may for example determine the processing order 118 of the SEI messages.

[0219] The decoder 110 can be configured to determine, for each respective SEI message 114 an information or at least for each of one or more leading in processing order up to finishing the list 122 construction described in the following, if the decoder 110 supports 123 the respective SEI message 114. In other words, the decoder 110 may be configured to determine a set of supported SEI messages 114. The decoder 110 can, for example, be configured to determine the list 122 of one or more SEI messages 114 using the information derived for sub-chain 117 affiliation of the SEI messages 114 and the supporting 123 information of the SEI messages 114.

[0220] The decoder 110 can, for example, be configured to, starting with an empty processing list 122, traverse the SEI messages 114 in the preliminary list and successively append SEI messages 114 traversed in the preliminary list to the processing list 122, controlled according to the SEI messages 114 being supported 123 or not, and the SEI messages’ attribution to sub-chains, (e.g. append, in the processing order 118, to the list 122). For instance, a prerequisite for appending a

[0221] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXcurrently (in the preliminary list) traversed SEI message 114 to the processing list 122, might be that this SEI message 114 is supported 123 by the decoder (e.g. the SEI message type of this SEI message). If the currently (in the preliminary list) traversed SEI message 114 is member of a sub-ordinate sub-chain 117 a further prerequisite might be, that all SEI messages 114 that are attributed to this sub-ordinate sub-chain and are deemed non-optional by the decoder 110 might have to be supported 123 by the decoder (e.g. all SEI message types of the SEI messages 114 of the sub-ordinate sub-chain 117 have to be supported 123). If the prerequisite and the further prerequisite are met for the currently traversed SEI message, the decoder 110 can be configured to append / include the currently traversed SEI message to the list 122. Once the decoder 110 has encountered (e.g. while traversing the preliminary list) a non-supported SEI message type, for example in a in a sub-ordinate sub-chain 1201-4, the decoder can be configured to further traverse the SEI messages 114 in the preliminary list and only append / include SEI messages 114 that are attributed to the base sub-chain 119 or SEI messages that are attributed to the base subchain 119 and that were determined by the decoder 110 as non-optional.

[0222] In other words, the decoder 110 can, for example, be configured to, along the processing order 118, include a (e.g. all) respective SEI message 114 of a sub-chain 117 in the list 122 as long as all SEI message types of the SEI messages 114, that are deemed non-optional, associated with the respective (e.g. e.g. sub-ordinate) sub-chain 117 of the respective SEI message are supported 123 by the decoder 110. If, along the processing order 118, an SEI message type of an SEI message is not supported 123, the corresponding (e.g. sub-ordinate) sub-chain (e.g. and all SEI messages related to this sub-chain) is not included / appended to the list 122. Going forward, (e.g. along the processing order 118) the decoder 110 can, for example, be configured to only include SEI messages of the base sub-chain 119 or can, for example be configured to only include SEI messages 114 of the base sub-chain 119, that were deemed non-optional.

[0223] For example, in Fig. 1 a third SEI message along the processing order 118 is not supported 123 by the decoder 110. Due to the affiliation of the third SEI message with the second sub-ordinate sub-chain, a second SEI message along the processing order is not included in the list 122. The following fourth to seventh SEI message along the process order 118 are also not included in the list 122 since they do not relate to the base sub-chain 119.

[0224] The decoder 110 can, for example, be configured to determine an empty list 122 [e.g. empty set; e.g. list 122 comprising zero SEI messages 114] if any SEI Message 114 of the base sub-chain

[0225] FH250304PEP-Vorschlag Nachanmeldung v2 - CO - ANMELDEFASSUNG.DOCX119 that was deemed non-optional by the decoder 110, according to the data stream 112, is not supported 123 by the decoder 110.

[0226] Fig. 1 further shows the possible set of lists 124 that can, for example, be possible for the exemplary setting of the shown SEI messages 114 in their corresponding processing order 118. Namely, the possible lists 124 are exemplary (starting from 0 and going from left to right) {[0,7];

[0227] [0,1, 2, 7]; [0,1, 2, 3, 4, 7]; [0,1 ,2, 3, 4, 5, 6, 7]; [7]; [0]}. The possible lists 124 are merely illustrative and can optionally be determined by the decoder 110. Embodiments of the invention can be configured to, but are not limited / restricted to, determining the set of possible lists 124.

[0228] With respect to Fig. 8b it can be seen, that the decoder 110 is capable of accurately determining the list 122 for all possible cases.

[0229] An encoder according to embodiments can be configured to determine the list 122, such that a decoder is capable of processing the list 122.

[0230] In accordance with embodiments, a media encoder may, for example, be called Video encoder and a Media decoder may, for example, be called Video decoder.

[0231] In accordance with embodiments applicable to all three aspects of the invention, the media decoder may be configured to determine whether it supports a given SEI message type prior to initiating the derivation of the list PoSeiList. In particular, the decoder may maintain an internal capability set or support table that maps SEI message payload types, as indicated by po_sei_payload_type[i], to a binary support indication. Such determination of support may, for example, be performed statically based on the decoder profile or level, or dynamically based on display characteristics, operational mode, or runtime conditions. By performing this capability assessment prior to list construction, the decoder is enabled to efficiently prune the processing chain during traversal without requiring repeated or redundant capability lookups. The encoder, correspondingly, may be configured to signal the SEI processing order information with knowledge of the capability distribution of the intended decoder population, thereby enabling optimally structured processing chains for a given deployment scenario.

[0232] In accordance with embodiments applicable to all three aspects of the invention, the determination of the list PoSeiList may be performed on a per-picture basis, taking into account the persistence

[0233] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXscope of each SEI message associated with a given picture. SEI messages may persist for a single picture, for a defined number of pictures, for an entire coded layer video sequence (CLVS), or until a subsequent SEI message of the same type is encountered.

[0234] Fig. 2, 3 and 4 show exemplary schematic views of media decoder 210, 310 and 410, respectively, that are related to the first, second and third aspect of the invention, respectively. The media decoder 210, 310 and / or 410 may be combined individually or in combination with any feature or functionality described with respect to media decoder 110.

[0235] The description will first resume with a general description applicable to media decoder 210, 310 and 410 before describing them individually.

[0236] The media decoder 210, 310 and 410 may comprise an SPO decoder 231 configured to decode SEI processing order information 116 from a data stream 112. The SEI processing order information 116 may define a processing order 118 among SEI messages 114 associated with the data stream 112, as well as an optionality with respect to those SEI messages 114.

[0237] The media decoder 210, 310 and 410may further comprise an attributer 236 configured to attribute the SEI messages 114, based on the SEI processing order information 116, to one or more SEI sub-chains 117. The sub-chains 117 may comprise a base sub-chain 119 and zero or more subordinate sub-chains 120, wherein each subordinate sub-chain 120 may have a contiguous subsequence of one or more SEI messages 114 attributed thereto in processing order 118. The SEI processing order information 116 may classify each SEI message 114 as forming a start of a sub-chain, forming an end of a sub-chain, or continuing a sub-chain, each along the processing order 118.

[0238] The media decoder 210, 310 and 410may further comprise a list determinator 232 configured to determine a list 122 of zero, one or more SEI messages 114 based on the SEI processing order information 116 and on which SEI messages 114 the media decoder 210, 310 and 410 supports. The list determinator 232 may be configured to determine the list 122 to be empty where the base sub-chain 119 comprises at least one SEI message 114 that is not supported by the media decoder 210, 310 and 410and that is indicated as non-optional by the SEI processing order information 116. Otherwise, the list determinator 232 may populate the list 122 with SEI messages 114 along the processing order 118, wherein the list determinator 232 may leave out SEI

[0239] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXmessages 114 that are not supported by the media decoder 210, 310 and 410 and are indicated as optional by the SEI processing order information 116. The list determinator 232 may further leave out each subordinate sub-chain 120 that contains at least one unsupported SEI message 114 and that has both a start-classified and an end-classified SEI message 114 attributed thereto, i.e. a closed subordinate sub-chain.

[0240] The media decoder 210, 310 and 410 may further comprise a list processor 233 configured to process the list 122 of zero, one or more SEI messages 114 determined by the list determinator 232, for example by applying post-processing steps to a media signal decoded from the data stream 112 in accordance with the SEI messages 114 included in the list 122.

[0241] In particular, the list processor 233 may be configured to apply post-processing steps or stages to a media signal decoded from the data stream 112, in accordance with the SEI messages 114 included in the list 122 and in the processing order 118 defined by the SEI processing order information 116. For example, the list processor 233 may be configured to interpret the parameters defined by each SEI message 114 in the list 122 and to apply a corresponding postprocessing operation to the decoded media signal, such that a post-processed media signal results. The list processor 233 may be configured to process only those SEI messages 114 that are included in the list 122, thereby skipping any SEI messages 114 that were left out during list determination by the list determinator 232. The post-processing operations applied by the list processor 233 may include, for example, color volume transformation, display adaptation, film grain synthesis, or other enhancement operations as defined by the respective SEI messages 114 in the list 122.

[0242] The description now resumes with a description of the individual media decoder 210, 310 and 410 with respect to Fig. 2, 3 and 4.

[0243] Since the media encoders 550, 650 and 750 are each configured to encode the SEI processing order information 116 in a manner that enables a corresponding media decoder210, 310 and 410 to determine the a 122 as described above, the structure and functionality of the media encoders 550, 650 and 750 can be described as being closely related to that of their respective media decoders 210, 310 and 410. Accordingly, the media encoders 550, 650 and 750 are depicted only schematically in Figs. 5, 6 and 7 respectively, each comprising an SPO encoder 551 and

[0244] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXreference is made to the description of the corresponding media decoders 210, 310 and 410 with respect to Figs. 1, 2, 3 and 4 for further details.

[0245] In the media decoder 210 according to the first aspect, the list determinator 232 may additionally leave out each subordinate sub-chain 120 that contains at least one unsupported SEI message 114 and that has a start-classified SEI message 114 attributed thereto but no end-classified SEI message 114, i.e. an open-ended subordinate sub-chain, together with all SEI messages 114 following that subordinate sub-chain 120 in the processing order 118. A bitstream constraint 240 may provide that no SEI message 114 attributed to the base sub-chain 119 and indicated as non-optional by the SEI processing order information 116 follows, in processing order 118, any such open-ended subordinate sub-chain 120.

[0246] In the media decoder 310 according to the second aspect, the list determinator 232 may additionally leave out each subordinate sub-chain 120 that contains at least one unsupported SEI message 114 and that has a start-classified SEI message 114 attributed thereto but no end-classified SEI message 114, i.e. an open-ended subordinate sub-chain, together with all subordinate sub-chains 120 following that open-ended subordinate sub-chain 120 in the processing order 118. However, the list determinator 232 may include 341, if present, any SEI message 114 that is supported by the media decoder 310, attributed to the base sub-chain 119, and follows the respective open-ended subordinate sub-chain 120 in the processing order 118, or any such SEI message 114 that is additionally indicated as non-optional by the SEI processing order information 116.

[0247] In the media decoder 410 according to the third aspect, the list determinator 232 may additionally leave out each subordinate sub-chain 120 that contains at least one unsupported SEI message 114 and that has a start-classified SEI message 114 attributed thereto but no end-classified SEI message 114, i.e. an open-ended subordinate sub-chain, together with all SEI messages 114 following that subordinate sub-chain 120 in the processing order 118. A bitstream constraint 442 may provide that there does not exist any SEI message 114 classified as forming an end of a subordinate sub-chain 120 which follows, in the processing order 118, an SEI message 114 classified as forming a start of a further subordinate sub-chain 120 having no end-classified SEI message 114 attributed thereto, thereby preventing closed subordinate sub-chains 120 from appearing downstream of any open-ended subordinate sub-chain 120.

[0248] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXWhile potentially differing in how they handle open-ended subordinate sub-chains 120, the media decoders 210, 310 and 410 according to the first, second and third aspects share a common underlying architecture comprising the SPO decoder 231, the attributed 36, the list determinator 232 and the list processor 233 as well as common base rules for constructing the list 122. The aspects differ in the specific rules for inclusion and bitstream constraints 240, 442 applied by the list determinator 232 once an open-ended subordinate sub-chain 120 containing an unsupported SEI message 114 is encountered, as described above.

[0249] In some embodiments, the features and functionalities of the media decoders 210, 310 and 410 may also be mixed. For example, the list determinator 232 may apply rules and bitstream constraints from the first and second aspects in combination, from the first and third aspects in combination, from the second and third aspects in combination, or from all three aspects in combination.

[0250] Fig. 5 shows an exemplary schematic view of a media encoder 550 related to the first aspect of the invention. The media encoder 550 is configured to generate a data stream 112, for example, by encoding pictures. The media encoder 550 comprises an SPO encoder 551 configured to encode SEI processing order information 116 into the data stream 112. The media encoder 550 may be designed to encode the SEI processing order information 116 such that the media decoder 110 and / or 210 is capable of determining a list of zero, one or more SEI messages based om the SEI processing order information 116.

[0251] In particular, the SPO encoder 551 of media encoder 550 is configured to encode the SEI processing order information 116 satisfying a bitstream constraint 240, such that there does not exist any SEI message 114 attributed to the base sub-chain 119 that is indicated as non-optional and that follows, in processing order 118, an open-ended subordinate sub-chain 120, i.e. a subordinate sub-chain 120 having a start-classified SEI message 114 attributed thereto but no end-classified SEI message 114 attributed thereto.

[0252] Fig. 6 shows an exemplary schematic view of a media encoder 650 related to the second aspect of the invention. The media encoder 650 is configured to generate a data stream 112, for example, by encoding pictures. The media encoder 650 comprises an SPO encoder 551 configured to encode SEI processing order information 116 into the data stream 112. The media encoder 650 may be designed to encode the SEI processing order information 116 such that the media decoder

[0253] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX110 and / or 310 is capable of determining a list 122 of zero, one or more SEI messages 114 based on the SEI processing order information 116.

[0254] In particular, the SPO encoder 551 of media encoder 650 may be configured to encode the SEI processing order information 116 such that a media decoder 110, 310 is capable of leaving out all subordinate sub-chains 120 following an open-ended subordinate sub-chain 120 containing at least one unsupported SEI message 114, while including any supported SEI message 114 attributed to the base sub-chain 119 and following such an open-ended subordinate sub-chain 120 in the processing order 118.

[0255] Fig. 7 shows an exemplary schematic view of a media encoder 750 related to the third aspect of the invention. The media encoder 750 is configured to generate a data stream 112, for example, by encoding pictures. The media encoder 750 comprises an SPO encoder 551 configured to encode SEI processing order information 116 into the data stream 112. The media encoder 750 may be designed to encode the SEI processing order information 116 such that the media decoder 110 and / or 410 is capable of determining a list 122 of zero, one or more SEI messages 114 based on the SEI processing order information 116.

[0256] In particular, the SPO encoder 551 of media encoder 750 may be configured to encode the SEI processing order information 116 satisfying a bitstream constraint 442, such that there does not exist any SEI message 114 classified as forming an end of a subordinate sub-chain 120 which follows, in processing order 118, an SEI message 114 classified as forming a start of a further subordinate sub-chain 120 having no end-classified SEI message 114 attributed thereto, thereby preventing closed subordinate sub-chains 120 from appearing downstream of any open-ended subordinate sub-chain 120.

[0257] In embodiments, optionally, the subject-matter of each of the previous embodiments or the subject-matter claimed below can be individually combined with any of the individual embodiments of the three groups of embodiments defined in the following:

[0258] A first group of embodiments is now described:

[0259] According to an embodiment, the apparatus (e.g. decoder 110) may further comprise a decoded picture buffer (DPB) and be configured to decode a current AU using inter-picture prediction from

[0260] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXa referenced reference picture stored in the DPB to obtain a decoded picture, and insert the decoded picture into the DPB, assign to each reference picture stored in the DPB a classification as one of a short-term reference picture, a long-term reference picture and an unused-for-reference picture, read DPB mode information from the current AU, if the DPB mode information indicates a first mode, remove one or more reference pictures classified as a short-term picture, according to a FIFO strategy, from the DPB, if the DPB mode information indicates a second mode, read memory management control information comprising at least one command in the current AU and execute the at least one command so as to change the classification assigned to at least one of the reference pictures stored in the DPB, and use the classification of the reference pictures in the DPB, for managing reference picture removal from the DPB.

[0261] In an embodiment, the apparatus may be configured to read from the current AU an indication whether the decoded picture is not used for inter-picture prediction; perform the insertion of the decoded picture into the DPB, if the decoded picture is not indicated to be not used for interpicture prediction or not directly to be output, and directly output the decoded picture without buffering same in the DPB, if the decoded picture is indicated to be not used for inter-picture prediction and directly to be output.

[0262] According to an embodiment, the apparatus may be configured to assign a frame index to each reference picture in the DPB, classified to be a long-term picture, and use a predetermined reference picture in the DPB, classified to be a long-term picture, as the referenced reference picture in the DPB if the frame index assigned to the predetermined reference picture is referred to in the current AU.

[0263] In an embodiment, the apparatus may be configured to one or more of: if the at least one command in the current AU is a first command, re-classify a reference picture in the DPB, classified to be a short-term reference picture, as an unused-for-reference picture, if the at least one command in the current AU is a second command, re-classify a reference picture in the DPB, classified to be a long-term reference picture, as an unused-for-reference picture, if the at least one command in the current AU is a third command, re-classify a reference picture in the DPB, classified to be a short-term picture, as a long-term reference picture, and assign a frame index to the re-classified reference picture, if the at least one command in the current AU is a fourth command, set an upper frame index limit according to the fourth command, and re-classify all reference picture in the DPB, classified to be a long-term picture, and having assigned thereto a

[0264] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXframe index exceeding the upper frame index limit, as an unused-for-reference picture, if the at least one command in the current AU is a sixth command, classify the current picture as a longterm picture, as an unused-for-reference picture, and assign a frame index to the re-classified reference picture.

[0265] According to an embodiment, the apparatus may be configured to remove any reference picture from the DPB, which is classified as an unused-for-reference picture, and which is no longer to be output.

[0266] In an embodiment the apparatus may be configured to read an entropy coding mode indicator from the data stream, and decode prediction residual data from the current AU using a context adaptive variable length coding mode if the entropy coding mode indicator indicates the context adaptive variable length coding mode, and using a context adaptive binary arithmetic coding mode if the entropy coding mode indicator indicates the context adaptive binary arithmetic coding mode.

[0267] According to an embodiment, the apparatus may be configured to derive quarter pel values in the referenced reference picture based on a motion vector in the current AU and using 6-tap FIR filter so as to derive half-pel values and averaging neighboring half-pel values.

[0268] In an embodiment, the apparatus may be configured to:

[0269] decode a current AU using inter-picture prediction from a referenced reference picture stored in the DPB to acquire a decoded picture, and to insert the decoded picture into the DPB,

[0270] assign to each reference picture stored in the DPB a classification as one of a short-term reference picture, a long-term reference picture and an unused-for-reference picture,

[0271] read DPB mode information from the current AU,

[0272] if the DPB mode information indicates a first mode, remove one or more reference pictures classified as a short-term picture, according to a first-in-first-out (FIFO) strategy, from the DPB,

[0273] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXif the DPB mode information indicates a second mode, read memory management control information comprising at least one command in the current AU and execute the at least one command so as to change the classification assigned to at least one of the reference pictures stored in the DPB, and use the classification of the reference pictures in the DPB, for managing reference picture removal from the DPB.

[0274] According to an embodiment, the apparatus may be configured to:

[0275] read from the current AU an indication whether the decoded picture is not used for inter-picture prediction;

[0276] perform the insertion of the decoded picture into the DPB, if the decoded picture is not indicated to be not used for inter-picture prediction or not directly to be output, and directly output the decoded picture without buffering same in the DPB, if the decoded picture is indicated to be not used for inter-picture prediction and directly to be output.

[0277] In an embodiment, the apparatus may be configured to:

[0278] assign a frame index to each reference picture in the DPB, classified to be a long-term picture, and

[0279] use a predetermined reference picture in the DPB, classified to be a long-term picture, as the referenced reference picture in the DPB if the frame index assigned to the predetermined reference picture is referred to in the current AU.

[0280] According to an embodiment, the apparatus may be configured to one or more of:

[0281] if the at least one command in the current AU is a first command,

[0282] re-classify a reference picture in the DPB, classified to be a short-term reference picture, as an unused-for-reference picture,

[0283] if the at least one command in the current AU is a second command,

[0284] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXre-classify a reference picture in the DPB, classified to be a long-term reference picture, as an unused-for-reference picture,

[0285] if the at least one command in the current AU is a third command,

[0286] re-classify a reference picture in the DPB, classified to be a short-term picture, as a longterm reference picture, and assign a frame index to the re-classified reference picture,

[0287] if the at least one command in the current AU is a fourth command,

[0288] set an upper frame index limit according to the fourth command, and re-classify all reference picture in the DPB, classified to be a long-term picture, and having assigned thereto a frame index exceeding the upper frame index limit, as an unused-for-reference picture,

[0289] if the at least one command in the current AU is a sixth command,

[0290] classify the current picture as a long-term picture, as an unused-for-reference picture, and assign a frame index to the re-classified reference picture.

[0291] In an embodiment, the apparatus may be configured to:

[0292] remove any reference picture from the DPB, which is classified as an unused-for-reference picture, and which is no longer to be output.

[0293] The video encoder is an encoder configured for encoding the video into the video data stream by block-based predictive coding and transform-based residual coding by

[0294] encoding prediction residual data of the residual block into the video data stream

[0295] by use of context adaptive variable length coding by using

[0296] a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing the residual block, and a trailing-one number, indicating a

[0297] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXnumber of non-zero transform coefficients having an absolute value of one when traversing the coefficients along a scan order,

[0298] one or more second syntax elements indicating a sign of the non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0299] one or more third syntax elements indicating a value of the non-zero transform coefficients except for the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0300] a fourth syntax element indicating a total number of zero-valued transform coefficient levels in the transform block from a firstly-encountered non-zero transform coefficient in the scan order onwards, and

[0301] one or more fifth syntax elements indicting positions of the non-zero transform coefficients along the scan order by indicating a number of consecutive zero-valued transform coefficients in the scan order between in the scan order consecutively encountered nonzero transform coefficients, or

[0302] by use of context-adaptive binary arithmetic coding by

[0303] encoding a significance map which indicates positions of non-zero transform coefficients in a transform block representing the residual block by, in a forward scan traversing transform coefficients of the transform block, encoding a significance flag which indicates whether a non-zero transform coefficient is positioned at a current position, and, if so, and if the current position is not the last in the forward scan, encoding a last-significance flag which indicates whether the non-zero transform coefficient positioned at the current position is the last non-zero transform coefficient in the forward scan order, and encoding the non-zero transform coefficients’ values sequentially in a reverse scan order, reversing the forward scan order.

[0304] The video decoder of an embodiment may be configured for decoding the video from the video data stream by block based predictive and transform based residual decoding by

[0305] decoding prediction residual data of a residual block from the video data stream

[0306] by use of context-adaptive variable length decoding by using

[0307] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXa first syntax element indicating a total number of non-zero transform coefficients in a transform block representing the residual block, and a trailing-one number, indicating a number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along a scan order,

[0308] one or more second syntax elements indicating a sign of the non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0309] one or more third syntax elements indicating a value of the non-zero transform coefficients except for the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0310] a fourth syntax element indicating a total number of zero-valued transform coefficient levels in the transform block from a firstly-encountered non-zero transform coefficient in the scan order onwards, and

[0311] one or more fifth syntax elements indicting positions of the non-zero transform coefficients along the scan order by indicating a number of consecutive zero-valued transform coefficients in the scan order between in the scan order consecutively encountered nonzero transform coefficients, or

[0312] by use of context-adaptive binary arithmetic decoding by

[0313] decoding a significance map which indicates positions of non-zero transform coefficients in a transform block representing the residual block by, in a forward scan traversing transform coefficients of the transform block, decoding a significance flag which indicates whether a non-zero transform coefficient is positioned at a current position, and, if so, and if the current position is not the last in the forward scan, decoding a last-significance flag which indicates whether the non-zero transform coefficient positioned at the current position is the last non-zero transform coefficient in the forward scan order, and decoding the non-zero transform coefficients’ values sequentially in a reverse scan order, reversing the forward scan order.

[0314] The video is encoded into the video data stream of an embodiment by block-based predictive coding and transform-based residual coding by

[0315] encoding prediction residual data of the residual block into the video data stream

[0316] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXby use of context adaptive variable length coding by using

[0317] a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing the residual block, and a trailing-one number, indicating a number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along a scan order,

[0318] one or more second syntax elements indicating a sign of the non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0319] one or more third syntax elements indicating a value of the non-zero transform coefficients except for the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0320] a fourth syntax element indicating a total number of zero-valued transform coefficient levels in the transform block from a firstly-encountered non-zero transform coefficient in the scan order onwards, and

[0321] one or more fifth syntax elements indicting positions of the non-zero transform coefficients along the scan order by indicating a number of consecutive zero-valued transform coefficients in the scan order between in the scan order consecutively encountered nonzero transform coefficients, or

[0322] by use of context-adaptive binary arithmetic coding by

[0323] encoding a significance map which indicates positions of non-zero transform coefficients in a transform block representing the residual block by, in a forward scan traversing transform coefficients of the transform block, encoding a significance flag which indicates whether a non-zero transform coefficient is positioned at a current position, and, if so, and if the current position is not the last in the forward scan, encoding a last-significance flag which indicates whether the non-zero transform coefficient positioned at the current position is the last non-zero transform coefficient in the forward scan order, and encoding the non-zero transform coefficients’ values sequentially in a reverse scan order, reversing the forward scan order.

[0324] According to a method of an embodiment,

[0325] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXencoding the video into the video data stream is conducted by block-based predictive coding and transform-based residual coding by

[0326] encoding prediction residual data of the residual block into the video data stream

[0327] by use of context adaptive variable length coding by using

[0328] a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing the residual block, and a trailing-one number, indicating a number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along a scan order,

[0329] one or more second syntax elements indicating a sign of the non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0330] one or more third syntax elements indicating a value of the non-zero transform coefficients except for the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0331] a fourth syntax element indicating a total number of zero-valued transform coefficient levels in the transform block from a firstly-encountered non-zero transform coefficient in the scan order onwards, and

[0332] one or more fifth syntax elements indicting positions of the non-zero transform coefficients along the scan order by indicating a number of consecutive zero-valued transform coefficients in the scan order between in the scan order consecutively encountered nonzero transform coefficients, or

[0333] by use of context-adaptive binary arithmetic coding by

[0334] encoding a significance map which indicates positions of non-zero transform coefficients in a transform block representing the residual block by, in a forward scan traversing transform coefficients of the transform block, encoding a significance flag which indicates whether a non-zero transform coefficient is positioned at a current position, and, if so, and if the current position is not the last in the forward scan, encoding a last-significance flag which indicates whether the non-zero transform coefficient positioned at the current position is the last non-zero transform coefficient in the forward scan order, and

[0335] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXencoding the non-zero transform coefficients’ values sequentially in a reverse scan order, reversing the forward scan order.

[0336] In another method of an embodiment,

[0337] decoding the video from the video data stream is conducted by block based predictive and transform based residual decoding by

[0338] decoding prediction residual data of a residual block from the video data stream

[0339] by use of context-adaptive variable length decoding by using

[0340] a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing the residual block, and a trailing-one number, indicating a number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along a scan order,

[0341] one or more second syntax elements indicating a sign of the non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0342] one or more third syntax elements indicating a value of the non-zero transform coefficients except for the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order,

[0343] a fourth syntax element indicating a total number of zero-valued transform coefficient levels in the transform block from a firstly-encountered non-zero transform coefficient in the scan order onwards, and

[0344] one or more fifth syntax elements indicting positions of the non-zero transform coefficients along the scan order by indicating a number of consecutive zero-valued transform coefficients in the scan order between in the scan order consecutively encountered nonzero transform coefficients, or

[0345] by use of context-adaptive binary arithmetic decoding by

[0346] decoding a significance map which indicates positions of non-zero transform coefficients in a transform block representing the residual block by, in a forward scan traversing transform coefficients of the transform block, decoding a significance flag which indicates whether a non-zero transform coefficient is positioned at a current position, and, if so, and

[0347] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXif the current position is not the last in the forward scan, decoding a last-significance flag which indicates whether the non-zero transform coefficient positioned at the current position is the last non-zero transform coefficient in the forward scan order, and decoding the non-zero transform coefficients’ values sequentially in a reverse scan order, reversing the forward scan order.

[0348] In the following, a second group of embodiments is described:

[0349] A particular embodiment relates to PCM. The prediction, transform, quantization and entropy coding are bypassed, and the samples are directly represented by a pre-defined number of bits. When PCM is enabled the number of bits (minus'!) used for representing the luma and chroma samples is indicated respectively. Besides, the minimum block sizes and maximum block sizes (as a difference to the minimum) for 2Nx2N blocks for which PCM can best used are indicated. When a 2Nx2N block is being parsed, if the sizes of that block are in between the described minimum and maximum values a flag indicates whether the PCM mode is used or not. When the PCM mode is used the PCM samples are byte aligned and luma samples of the block are directly represented / parsed in raster scan with the indicated PCM luma bit length and afterwards the chroma samples of the PCM block are represented / parsed in raster scan with the indicated PCM chroma bit length, being the first half of the chroma samples Cb and the remaining Cr samples. Furthermore, the CABAC decoding engine is terminated every time PCM flag is parsed and initialized after the decoding PCM syntax

[0350] Another embodiment relates to Intra Mode Dep. Coeff. Scan Order. The scan order of the coefficients in an intra coded block is dynamically changed based on the transform block sizes and intra modes. The scanning is carried out for each 4x4 subblocks / regions of the transform block (e.g., using only one coefficient region for the 4x4 transform block size, 4 coefficient regions for 8x8 transform blocks, 16 regions for 16x16 transform block size). The selection of the scanning method for intra modes is dependent on the transform block sizes. For transform block sizes of 16x16 and 32x32 the scanning is performed diagonally (starting at 0,0 -> 1,0 -> 0, 1-> 2,0-> 1,1 -> 0,2 -> 3,0 and so on - e.g. decreasing the y value and increasing the x value with step 1 at each step and when y equals 0 restarting with y having a value of x+1 and setting x to 0 while starting at value 0,0). For transform block sizes of 4x4 (luma or chroma) and 8x8 (luma) the coefficient scanning order depends on the intra mode associated with a intra direction of the prediction of neighboring blocks. The vertical scan is used when the prediction direction is close to horizontal

[0351] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXand the horizontal scan is used when the prediction direction is close to vertical. For other prediction directions, the diagonal up-right scan is used.

[0352] A further embodiment relates to Transform Quantization Bypass Mode. For each coding unit, or coding block it can be indicated whether both transform and quantization are bypassed indicating that the residual signal from inter- or intra-picture prediction is directly entropy coded for that block. In that mode the in-loop filter is skipped.

[0353] The video encoder of an embodiment may, e.g., be configured for encoding the video into the video data stream by block-based predictive coding and transform-based residual coding by

[0354] encoding prediction residual data of an intra predicted block into the video data stream

[0355] by use of context-adaptive binary arithmetic coding by

[0356] encoding a coordinate of a position in a transform block representing the prediction residual data at which a last non-zero transform coefficient is encountered when traversing transform coefficients of the transform block along a predetermined scan order, and

[0357] sequentially encoding values of transform coefficients including and ranked, along the predetermined scan order, between the last non-zero transform coefficient and a firstly scanned transform coefficient, and

[0358] selecting the predetermined scan order among a diagonal scan order, a horizontal scan order, and a vertical scan order depending on an intra prediction mode of the intra predicted block by use of a mapping which maps each of a plurality of intra prediction modes onto a corresponding one of the diagonal scan order, the horizontal scan order, and the vertical scan order.

[0359] The video decoder of an embodiment may, e.g., be configured for decoding the video from the video data stream by block based predictive decoding and transform based residual decoding by

[0360] decoding prediction residual data of an intra predicted block from the video data stream by use of context-adaptive binary arithmetic decoding by

[0361] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXdecoding a coordinate of a position in a transform block representing the prediction residual data at which position a last non-zero transform coefficient is encountered when traversing transform coefficients of the transform block along a predetermined scan order, and

[0362] sequentially decoding values of transform coefficients including and ranked, along the predetermined scan order, between the last non-zero transform coefficient and a firstly scanned transform coefficient, and

[0363] selecting the predetermined scan order among a diagonal scan order, a horizontal scan order, and a vertical scan order depending on an intra prediction mode of the intra predicted block by use of a mapping which maps each of a plurality of intra prediction modes onto a corresponding one of the diagonal scan order, the horizontal scan order, and the vertical scan order.

[0364] The video is encoded into the video data stream of an embodiment by block-based predictive coding and transform-based residual coding by

[0365] encoding prediction residual data of an intra predicted block into the video data stream

[0366] by use of context-adaptive binary arithmetic coding by

[0367] encoding a coordinate of a position in a transform block representing the prediction residual data at which a last non-zero transform coefficient is encountered when traversing transform coefficients of the transform block along a predetermined scan order, and

[0368] sequentially encoding values of transform coefficients including and ranked, along the predetermined scan order, between the last non-zero transform coefficient and a firstly scanned transform coefficient, and

[0369] selecting the predetermined scan order among a diagonal scan order, a horizontal scan order, and a vertical scan order depending on an intra prediction mode of the intra predicted block by use of a mapping which maps each of a plurality of intra prediction modes onto a corresponding one of the diagonal scan order, the horizontal scan order, and the vertical scan order.

[0370] According to a method of an embodiment, encoding the video into the video data stream is conducted by block-based predictive coding and transform-based residual coding by

[0371] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXencoding prediction residual data of an intra predicted block into the video data stream

[0372] by use of context-adaptive binary arithmetic coding by

[0373] encoding a coordinate of a position in a transform block representing the prediction residual data at which a last non-zero transform coefficient is encountered when traversing transform coefficients of the transform block along a predetermined scan order, and

[0374] sequentially encoding values of transform coefficients including and ranked, along the predetermined scan order, between the last non-zero transform coefficient and a firstly scanned transform coefficient, and

[0375] selecting the predetermined scan order among a diagonal scan order, a horizontal scan order, and a vertical scan order depending on an intra prediction mode of the intra predicted block by use of a mapping which maps each of a plurality of intra prediction modes onto a corresponding one of the diagonal scan order, the horizontal scan order, and the vertical scan order.

[0376] In a method of another embodiment, decoding the video from the video data stream is conducted by block based predictive decoding and transform based residual decoding by

[0377] decoding prediction residual data of an intra predicted block from the video data stream by use of context-adaptive binary arithmetic decoding by

[0378] decoding a coordinate of a position in a transform block representing the prediction residual data at which position a last non-zero transform coefficient is encountered when traversing transform coefficients of the transform block along a predetermined scan order, and

[0379] sequentially decoding values of transform coefficients including and ranked, along the predetermined scan order, between the last non-zero transform coefficient and a firstly scanned transform coefficient, and

[0380] selecting the predetermined scan order among a diagonal scan order, a horizontal scan order, and a vertical scan order depending on an intra prediction mode of the intra predicted block by use of a mapping which maps each of a plurality of intra prediction modes onto a corresponding one of the diagonal scan order, the horizontal scan order, and the vertical scan order.

[0381] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXIn the following, a third group of embodiments is described:

[0382] Multiple Reference Lines (MRL): For intra prediction, not only the adjacent line of neighboring samples can be used but also one of the two non-adjacent reference lines as the reference line for intra-picture prediction of luma samples, corresponding to the two or three lines away from the current block.

[0383] Adaptive MV Resolution (AMVR): A selection of the MV resolution at the CU level is performed. For inter predicted CUs, the selected MV resolution is indicated and can be one quarter, one half, whole integer, or four, in units of luma samples. If half luma sample resolution is selected, an alternative luma interpolation filter is used for the half-sample position in this block, i.e. different interpolation filter are used when the resolution is quarter-per or half-pel for the MV.

[0384] History-Based MV Prediction (HMVP): In addition to spatial and temporal neighbor MV predictions, a new candidate type is added for MV prediction in the merge mode and AMVP candidate list. The HMVP candidates are established using a five-entry table that is maintained and updated using a first-in-first-out (FIFO) rule. The motion vector candidates list are generated by using the spatial and temporal neighbors and HMVP candidates.

[0385] Affine Motion: An affine motion model with CU-level signaling is used for luma. The CU-level affine motion can be either a 4-parameter model or a 6-parameter model. The 4-parameter model uses two MVs, which correspond to two control points located at the top-left and top-right corners of the CU and the 6-parameter model uses three MVs, which corresponds to three control points located at the top-left, bottom-left and top-right corners. When a CU is coded in affine motion mode, the luma block of the CU is spilt into 4 * 4 subblocks and the MV at the central sample position of each subblock is calculated according to the affine motion model and set as the subblock MV based on the control points. The subblock MV is rounded to 1 / 16 luma sample precision during the calculation and a set of 6-tap interpolation filters is applied to generate the prediction of each sub- bloc. As for the case of non-affine, a merge mode and AMVP mode are used for prediction and coding of affine motion parameters.

[0386] Coefficient coding: When encoding the coefficient level, first a flag (SigFlag) is indicated, that specifies that whether coefficient level is not 0. When the flag is equal to 1 (the coefficient level is

[0387] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXnot 0) a further flag is indicated (Gt1-Flag), which indicates whether the absolute level is greater than 1, in which case (absolute level is greater than 1) two further flags are present, a parity level flag (Parity-Flag) that specifies the parity of the transform coefficient level and a further flag (Gt3-Flag) that specifies whether the absolute value of the transform coefficient level is greater than a 3.

[0388] The video encoder of an embodiment may, e.g., be configured for encoding the video into the video data stream by block based predictive encoding and transform based residual encoding by

[0389] encoding prediction residual data of a residual block into the video data stream by use of context-adaptive binary arithmetic coding of quantization indices of transform coefficients of a transform block representing the residual block and sequential quantization of the transform coefficients to obtain the quantization indices, according to which a quantizer for quantizing a current transform coefficient depends on a parity of quantization indices of previous quantization indices.

[0390] The video decoder of an embodiment may, e.g., be configured for decoding the video from the video data stream by block-based predictive decoding and transform-based residual decoding by

[0391] decoding prediction residual data of a residual block from the video data stream by use of context-adaptive binary arithmetic decoding of quantization indices of transform coefficients of a transform block representing the residual block and sequential dequantization of the quantization indices according to which a value of a current transform coefficient depends on a parity of quantization indices of previous quantization indices.

[0392] The video is encoded into the video data stream by block based predictive encoding and transform based residual encoding by

[0393] encoding prediction residual data of a residual block into the video data stream by use of context-adaptive binary arithmetic coding of quantization indices of transform coefficients of a transform block representing the residual block and sequential quantization of the transform coefficients to obtain the quantization indices, according to which a quantizer

[0394] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXfor quantizing a current transform coefficient depends on a parity of quantization indices of previous quantization indices.

[0395] According to a method of an embodiment, encoding the video into the video data stream is conducted by block based predictive encoding and transform based residual encoding by

[0396] encoding prediction residual data of a residual block into the video data stream by use of context-adaptive binary arithmetic coding of quantization indices of transform coefficients of a transform block representing the residual block and sequential quantization of the transform coefficients to obtain the quantization indices, according to which a quantizer for quantizing a current transform coefficient depends on a parity of quantization indices of previous quantization indices.

[0397] In a method of an embodiment, decoding the video from the video data stream is conducted by block-based predictive decoding and transform-based residual decoding by

[0398] decoding prediction residual data of a residual block from the video data stream by use of context-adaptive binary arithmetic decoding of quantization indices of transform coefficients of a transform block representing the residual block and sequential dequantization of the quantization indices according to which a value of a current transform coefficient depends on a parity of quantization indices of previous quantization indices.

[0399] As a supplement to the above description, another embodiment is follows:

[0400] The list poSubChainldx[j ] forj rangingfrom 0 to PoNumProcStgs - 1, inclusive, specifying the sub-chain index of the j-th processing stage of the processing chain, is derived as follows:

[0401] poSubChainFlag = 0

[0402] poSubChainPrevIdx = 0

[0403] for( j = 0; j < PoNumProcStgs; j++ ) {

[0404] idx = PoSeiTypeldx[ j ]

[0405] if( po_sei_importance_flag[ idx] = = 1 && po_sei_processing_degree_flag[ idx] = = 1 )

[0406] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXif( poSubChainFlag = = 0 )

[0407] poSubChainldx[ j ] = 0 (1) else

[0408] poSubChainldx[ j ] = poSubChainPrevIdx

[0409] else

[0410] if( po_sei_importance_flag[ idx ] = = 0 && po_sei_processing_degree_flag[ idx] = = 1 ){

[0411] poSubChainldx[j ] = poSubChainFlag* poSubChainPrevIdx poSubChainFlag = 0

[0412] } else

[0413] if( po_sei_importance_flag[ idx ] = = 1 && po_sei_processing_degree_flag[ idx] = = 0 ){

[0414] poSubChainPrevldx++

[0415] poSubChainldx[j ] = poSubChainPrevIdx

[0416] poSubChainFlag = 1

[0417] } else

[0418] poSubChainldx[j ] = poSubChainFlag* poSubChainPrevIdx

[0419] }

[0420] For a picture, the list PoSeiList, indictingthe list of SEI messages that may be applied to the picture, the list PoSeiTypeList, indicatingthe SEI message type indices of the SEI messages that may be applied to the picture, and the variable PoNumSeiMsgs, indicatingthe number of SEI messages that may be applied to the picture, are derived as follows:

[0421] PoSeiList is initially empty, and seiListldx and PoNumSeiMsgs are both initially set equalto 0.

[0422] - The following applies in increasing order of j for all values of j in the range of Oto PoNumProcStgs - 1 , inclusive, unless terminated earlier as specified below:

[0423] - When an SEI message seiA associated with the PoSeiTypeldx[ j ]-th SEI message type persists for picA, the following applies:

[0424] If both of the following conditions are true, seiA is added at the end of PoSeiList, PoSeiTypeList[ seiListldx] is set equal to PoSeiTypeldx[ j ], PoNumSeiMsgs is set equalto PoNumSeiMsgs + 1 , and seiListldx is set equalto seiListldx + 1 :

[0425] - The decoding system can interpret and supports the functionality indicated by seiA.

[0426] Either of the following conditions is true:

[0427] poSubChainldx[ j ] is equalto 0.

[0428] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX- The decoding system can interpret and supports the functionality indicated by all SEI message types for all values of k such that poSubChainldx[ k] is equalto poSubChainldx[ j ] and po_sei_importance_flag[ PoSeiTypeldx[ k ] ] + po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is greater than or equalto 1.

[0429] Otherwise, if poSubChainldx[ j ] is greater than 0,

[0430] po_sei_importance_flag[ PoSeiTypeldx[ j ] ] is equal to 1 , po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equalto 0, and there is no value of k such that poSubChainldx[ k] is equalto poSubChainldx[ j ], po_sei_importance_flag[ PoSeiTypeldx[ k ] ] is equal to 0 and po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is equal to 1 , the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.

[0431] Otherwise, if poSubChainldx[ j ] is equalto 0,

[0432] po_sei_importance_flag[ PoSeiTypeldx[ j ] ] is equal to 1 and po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equalto 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, PoNumSeiMsgs is set equal to 0, and the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.

[0433] It is a requirement of bitstream conformance that, for any value of j in the range of Oto PoNumProcStgs - 1, inclusive, when poSubChainldx[ j ] is equal to 0, po_sei_importance_flag[ PoSeiTypeldx[ j ] ] is equalto 1, and po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equalto 1, there shall not exist any index k less than j, such that poSubChainldx[ k] is open-ended.

[0434] NOTE 1 -The requirement above disallows a combination of po_sei_importance_flag[ PoSeiTypeldx[ j ] ] and po_sei_processing_degree_flag[ PoSeiTypeldx[j ] ] that indicates an end of a sub-chain to appear outside of a subchain.

[0435] Here, according to an embodiment, a media decoder decoding the data stream which is associated with supplemental enhancement information, SEI, messages, decodes SEI processing order information from the data stream, namely in form of po_sei_importance_flag[ ] and po_sei_processing_degree_flag[ ], which defines a processing order among, and an optionality with respect to, SEI messages associated with the data stream. It determines a list PoSeiList of zero, one or more SEI messages based on the SEI processing order information and based on which of the SEI messages the media decoder supports. It processes the list of zero, one or more SEI messages. The media decoder, based on the SEI processing order information, attributes the SEI messages to one or more SEI sub-chains comprising a base sub-chain for which poSubChainldx[ j ] = 0 and zero or more subordinate subchains forwhich poSubChainldx[ j ] > 0 such that each subordinate sub-chain has a, in processing order, contiguous subsequence of one or more SEI messages attributed thereto. That is,

[0436] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXpoSubChainldx[ ], along processing order, develops so that equal values of poSubChainldx[ ] at different positions along processing order are not interrupted by a different value for poSubChainldx[ ] an a position between these positions along processing order. The SEI processing order information classifies each SEI message as one of forming - along the processing order- a start of a sub-chain (e.g. po_sei_importance_flag[ idx ] = = 1 && po_sei_processing_degree_flag[ idx ] = = 0), forming - along the processing order- an end of a sub-chain (e.g. po_sei_importance_flag[ idx ] = = 0 && po_sei_processing_degree_flag[ idx ] = = 1), or continuing - along the processing order- a sub-chain (e.g. po_sei_importance_flag[ idx ] = = 0 && po_sei_processing_degree_flag[ idx ] = = 0); the media decoder determines the list of zero, one or more SEI messages by determining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information as non-optional (e.g. as done in “Otherwise, if poSubChainldxf j ] is equal to 0, po_sei_importance_flag[ PoSeiTypeldxf j ] ] is equal to 1 and po_sei_processing_degree_flag[ PoSeiTypeldxf j ] ] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, PoNumSeiMsgs is set equal to 0, and the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated”), and populating the list of zero, one or more SEI messages with the SEI messages along the processing order with leaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information as optional (e.g. non of the two “Otherwise, if poSubChainldx[ j ] is...” applies so that such SEI is simply left out), leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain (e.g. no SEI in a sub-chain is added unless all non-optional therein are supported due to “The decoding system can interpret and supports the functionality indicated by all SEI message types for all values of k such that poSubChainldx[ k ] is equal to poSubChainldx[ j ] and po_sei_importance_flag[ PoSeiTypeldx[ k ] ] + po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is greater than or equal to 1”, and the first “Otherwise, if poSubChainldx[ j ] is...” merely applies to open-ended sub-chains”), and leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain

[0437] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXand no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective sub-ordinate sub-chain in the processing order (this is due to the first “Otherwise, if poSubChainldx[ j ] is...”), wherein, by bitstream constraint, there does not exist any SEI message attributed to the base sub-chain, which is indicated by the SEI processing order information as being non-optional and which follows, in the processing order, a sub-ordinate sub-chain having an SEI message attributed thereto classified as forming a start of the said sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the said sub-ordinate sub-chain (see “It is a requirement of bitstream conformance that, for any value of j in the range of 0 to PoNumProcStgs - 1, inclusive, when poSubChainldx[ j ] is equal to 0, po_sei_importance_flag[ PoSeiTypeldx[ j ] ] is equal to 1 , and po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equal to 1 , there shall not exist any index k less than j, such that poSubChainldx[ k ] is open-ended”).

[0438] Note that, according to an alternative wording, according to all described embodiments, populating the list 122 of zero, one or more SEI messages with the SEI messages along the processing order involves leaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information 116 as optional, leaving out each subordinate sub-chain containing at least one SEI message which is not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and leaving out each sub-ordinate sub-chain containing at least one SEI message which is not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate subchain and no SEI message attributed thereto classified as forming an end of the respective subordinate sub-chain, and all SEI messages following the respective sub-ordinate sub-chain in the processing order. The at least one SEI message is non-optional by not being mentioned as being'indicated as being optional.

[0439] Implementation alternatives:

[0440] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or

[0441] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXfeature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be executed by such an apparatus.

[0442] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.

[0443] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0444] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0445] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0446] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0447] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, ora computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and / or non-transitionary.

[0448] FH250304PEP-Vorschlag Nachanmeldung v2 - CO - ANMELDEFASSUNG.DOCXA further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.

[0449] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0450] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0451] A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0452] In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware apparatus.

[0453] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0454] The apparatus described herein, or any components of the apparatus described herein, may be implemented at least partially in hardware and / or in software.

[0455] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0456] FH250304PEP-Vorschlag Nachanmeldung v2 - CO - ANMELDEFASSUNG.DOCXThe methods described herein, or any components of the apparatus described herein, may be performed at least partially by hardware and / or by software.

[0457] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the arrangements and the details described herein will be apparent to others skilled in the art. It is the intent, therefore, to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

[0458] FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX

Claims

58Claims1. Media decoder (110, 210) configured to decode a data stream (112) which is associated with supplemental enhancement information, SEI, messages (114), configured todecode SEI processing order information (116) from the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112);determining a list (122) of zero, one or more SEI messages (114) based onthe SEI processing order information (116) andwhich of the SEI messages (114) the media decoder (110, 210) supports;processing the list (122) of zero, one or more SEI messages (114);wherein the media decoder (110, 210) is configured tobased on the SEI processing order information (116), attribute the SEI messages (114) to one or more SEI sub-chains (117) comprising a base sub-chain (119) and zero or more subordinate sub-chains (120I.4) such that each subordinate sub-chain (120-i.4) has a, in processing order (118), contiguous subsequence of one or more SEI messages (114) attributed thereto, wherein the SEI processing order information (116) classifies each SEI message (114) as one of forming - along the processing order (118) - a start of a subchain, forming - along the processing order (118) - an end of a sub-chain, or continuing - along the processing order (118) - a sub-chain;determine the list (122) of zero, one or more SEI messages (114) bydetermining the list (122) of zero, one or more SEI messages (114) to be empty if the base sub-chain (119) comprises at least one SEI message (114) which is not supported by the media decoder (110) and is indicated by the SEI processing order information (116) as non-optional, andFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX59populating the list (122) of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective subordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andleaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective subordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective subordinate sub-chain in the processing order, wherein, by bitstream constraint (240), there does not exist any SEI message attributed to the base sub-chain, which is indicated by the SEI processing order information (116) as being non-optional and which follows, in the processing order, a sub-ordinate sub-chain having an SEI message attributed thereto classified as forming a start of the said sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the said sub-ordinate sub-chain.

2. Media decoder (110) configured to decode a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), configured todecode SEI processing order information (116) from the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112);determining a list (122) of zero, one or more SEI messages (114) based onFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX60the SEI processing order information (116) andwhich of the SEI messages (114) the media decoder (110) supports (123);processing the list (122) of zero, one or more SEI messages;wherein the media decoder (110) is configured tobased on the SEI processing order information (116), attribute the SEI messages to one or more SEI sub-chains (117) comprising a base sub-chain 119 and zero or more subordinate sub-chains such that each subordinate sub-chain (120I.4) has a, in processing order (118), contiguous subsequence of one or more SEI messages (114) attributed thereto, wherein the SEI processing order information (116) classifies (121) each SEI message (114) as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain;determine the list (122) of zero, one or more SEI messages bydetermining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list (122) of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and is indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain (120I.4) containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain (120I.4) and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain (120I.4), andFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX61leaving out each sub-ordinate sub-chain (120I.4) containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain (120I.4) and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain (120I.4), and all subordinate sub-chains (1204-4) following the respective sub-ordinate sub-chain in the processing order, with including (341), if present, any SEI message supported by the media decoder, attributed to the base sub-chain (119) and following the respective subordinate sub-chain (1204-4) in the processing order, or any SEI message supported by the media decoder, attributed to the base sub-chain (119) and following the respective subordinate sub-chain in the processing order, which is indicated by the SEI processing order information (116) as non-optional.

3. Media decoder (110) configured to decode a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), configured todecode SEI processing order information (116) from the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112);determining a list (122) of zero, one or more SEI messages based onthe SEI processing order information (116) andwhich of the SEI messages (114) the media decoder supports;processing the list (122) of zero, one or more SEI messages;wherein the media decoder is configured tobased on the SEI processing order information (116), attribute the SEI messages to one or more SEI sub-chains (117) comprising a base sub-chain (119) and zero or more subordinate sub-chains (120I.4) such that each subordinate sub-chain (1204-4) has a, in processing order (118), contiguous subsequence of one or more SEI messages (114) attributed thereto, wherein the SEI processing order information (116) classifies each SEIFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX62message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain;determine the list (122) of zero, one or more SEI messages (114) bydetermining the list (122) of zero, one or more SEI messages (114) to be empty if the base sub-chain (119) comprises at least one SEI message (114) which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list (122) of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andleaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective sub-ordinate sub-chain in the processing order, wherein, by bitstream constraint (442), there does not exist any SEI message indicated by the SEI processing order information (116) as forming an end of a subordinate sub-chain which follows, in the processing order, an SEI message classified as forming a start of a further sub-ordinate sub-chain which has no SEI message attributed thereto which is classified as forming an end of the further sub-ordinate sub-chain.FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX4. Media decoder of any previous claims, configured to,decode from the data stream, for each of the one or more SEI messages, one or more sub-chain-related syntax elements (115) comprised by the SEI processing order information (116),wherein the media decoder is configured to determine the list (122) of zero, one or more SEI messages bydetermining for each of the one or more SEI messages whether same is a non - optional SEI message, based on the one or more sub-chain -related syntax elements of the SEI messages of the respective SEI message.

5. Media decoder (110) of any previous claims, configured to,decode from the data stream (112), for each of the one or more SEI messages (114), one or more sub-chain-related syntax elements (115) comprised by the SEI processing order information (116),wherein the media decoder (110) is configured to attributing the SEI messages bydetermining for each of the SEI messages whether same is a start of a sub-chain, an end of a sub-chain, a continuation of a sub chain a non-optional SEI message.

6. Media decoder (110) of claim 5, configured to,decode from the data stream (112), for each of the one or more SEI messages (114), one or more sub-chain-related syntax elements (115) comprised by the SEI processing order information (116),FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCXwherein the media decoder (110) is configured to, when the respective sub-chain-related syntax element (115) indicates the continuation of a sub-chain, differentiate between the continuation of a sub-chain and a non-optional SEI message of the base sub-chain byanalyzing attributions of one or more sub-chain-related syntax elements prior, along the processing order, to the respective sub-chain-related syntax element.

7. Media decoder of claim 5 or 6, configured todecode, from the data stream (112), the one or more sub-chain-related syntax elements (115) by decoding an importance flag and a processing flag for each SEI message (114).

8. Media decoder of claim 7,wherein the media decoder (110) is configured to classify each of the SEI messages as forming the start of the sub-chain (120I.4) if the importance flag of the respective SEI message is equal to a first value and the processing degree flag of the respective SEI message is equal to a second value.

9. Media decoder of claim 7 or 8,wherein the media decoder (110) is configured to classify each of the SEI messages (114) as forming the end of the sub-chain (120I.4) if the importance flag of the respective SEI message is equal to a or the second value and the processing degree flag of the respective SEI message is equal to a or the first value.

10. Media decoder of any previous claims,wherein the media decoder is a video decoder (110) configured for reconstructing a video from the data stream by block based predictive decoding and transform based residual decoding.

11. Media decoder of any previous claims, configured toFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX65decode from the data stream (112), for each of the one or more SEI messages (114), one or more processing-order related syntax elements (141),wherein the media decoder (110) is configured to determine the processing order (118) using the one or more processing-order related syntax elements (141).

12. Media encoder (550) configured to generate a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), configured toencode SEI processing order information (116) into the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112); so that a media decoder (110, 210), is capable of determining a list (122) of zero, one or more SEI messages based onthe SEI processing order information (116), andbybased on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains comprising a base sub-chain and zero or more subordinate sub-chains such that each subordinate sub-chain has a, in processing order, contiguous subsequence of one or more SEI messages attributed thereto, wherein the SEI processing order information (116) classifies each SEI message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain; anddetermining the list (122) of zero, one or more SEI messages (114) bydetermining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list of zero, one or more SEI messages with the SEI messages along the processing order withFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX66leaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective subordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andleaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective subordinate sub-chain in the processing order, wherein the media encoder is configured to encode the SEI processing order information (116) attributed to the base sub-chain which is such that there does not exist any SEI message indicated by the SEI processing order information (116) as being non-optional and which follows, in the processing order, a subordinate sub-chain having an SEI message attributed thereto classified as forming a start of the said sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the said sub-ordinate sub-chain.

13. Media encoder (650) configured to generate a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), configured toencode SEI processing order information (116) into the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112); so that a media decoder(110, 310) is capable ofdetermining a list (122) of zero, one or more SEI messages based onthe SEI processing order information (116), andFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX67bybased on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains comprising a base sub-chain and zero or more subordinate sub-chains such that each subordinate sub-chain has a, in processing order, contiguous subsequence of one or more SEI messages attributed thereto, wherein the SEI processing order information (116) classifies each SEI message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain; anddetermining the list of zero, one or more SEI messages bydetermining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and is indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective subordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andleaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all sub-ordinate sub-chains following the respective sub-ordinate sub-chain in the processing order, with including (341), if present,FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX68any SEI message supported by the media decoder, attributed to the base sub-chain and following the respective sub-ordinate sub-chain in the processing order, or any SEI message supported by the media decoder, attributed to the base sub-chain and following the respective sub-ordinate sub-chain in the processing order, which is indicated by the SEI processing order information (116) as non-optional.

14. Media encoder (750) configured to generate a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), configured toencode SEI processing order information (116) into the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112); so that a media decoder (110, 410), is capable ofdetermining a list (122) of zero, one or more SEI messages based onthe SEI processing order information (116) and bybased on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains (117) comprising a base sub-chain (119) and zero or more subordinate sub-chains (120I.4) such that each subordinate sub-chain (120-i.4) has a, in processing order (118), contiguous subsequence of one or more SEI messages (114) attributed thereto, wherein the SEI processing order information (116) classifies each SEI message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain; anddetermining the list (122) of zero, one or more SEI messages (114) bydetermining the list (122) of zero, one or more SEI messages (114) to be empty if the base sub-chain (119) comprises at least one SEI message (114) which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX69populating the list (122) of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective subordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andleaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective subordinate sub-chain in the processing order, wherein, by bitstream constraint (442), there does not exist any SEI message indicated by the SEI processing order information (116) as forming an end of a subordinate sub-chain which follows, in the processing order, an SEI message classified as forming a start of a further sub-ordinate sub-chain which has no SEI message attributed thereto which is classified as forming an end of the further subordinate sub-chain.

15. Media encoder of any of the claims 12 to 14, configured to,encode into the data stream (112), for each of the one or more SEI messages, one or more sub-chain-related syntax elements (115) comprised by the SEI processing order information (116); so that a media decoder is capable ofdetermining the list (122) of zero, one or more SEI messages (114) by determining for each of the one or more SEI messages whether same is a non - optional SEI message, based on the one or more sub-chain related syntax elements (115) of the SEI messages of the respective SEI message.FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX7016. Media decoder of any of the claims 12 to 15, configured to,encode into the data stream, for each of the one or more SEI messages, one or more sub- chain-related syntax elements (115) comprised by the SEI processing order information (116); so that a media decoder is capable ofattributing the SEI messages bydetermining for each of the SEI messages whether same is a start of a sub-chain, an end of a sub-chain, a continuation of a sub chain and a non-optional sub-chain.

17. Media encoder of claim 16, configured to,encode into the data stream, for each of the one or more SEI messages, one or more sub- chain-related syntax elements (115) comprised by the SEI processing order information (116); so that a media decoder is capable ofwhen the respective sub-chain-related syntax element indicates the continuation of a subchain, differentiating between the continuation of a sub-chain and a non-optional SEI message of the base sub-chain byanalyzing attributions of one or more sub-chain-related syntax elements prior, along the processing order, to the respective sub-chain-related syntax element.

18. Media encoder of claim 16 or 17, configured toencode, into the data stream (112), the one or more sub-chain-related syntax elements (115) by encoding an importance flag and a processing flag for each SEI message (114).

19. Media encoder of claim 18,wherein the media encoder is configured to indicate, for each SEI message (114) forming the start of the sub-chain (120 1-4), said start by encoding the importance flag of theFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX71respective SEI message with a first value and the processing degree flag of the respective SEI message with a second value.

20. Media encoder of claim 18 or 19,wherein the media encoder is configured to indicate, for each SEI message (114) forming the end of the sub-chain (120I.4) start by encoding the importance flag of the respective SEI message with a or the second value and the processing degree flag of the respective SEI message with a or the first value.

21. Media encoder of any of the previous claims 12 to 20wherein the media encoder is a video encoder configured for encoding into the data stream by block-based predictive coding and transform-based residual coding.

22. Media encoder of any of the previous claims 12 to 21 , configured toencode into the data stream (112), for each of the one or more SEI messages (114), one or more processing-order related syntax elements (141), such that a media decoder is capable ofdetermining the processing order (118) using the one or more processing-order related syntax elements (141).

23. Method for decoding a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), comprisingdecoding SEI processing order information (116) from the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112);determining a list (122) of zero, one or more SEI messages based onthe SEI processing order information (116) andFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX72which of the SEI messages (114) the method supports ;processing the list (122) of zero, one or more SEI messages ,wherein the method comprisesbased on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains (117) comprising a base sub-chain (119) and zero or more subordinate sub-chains (120I.4) such that each subordinate sub-chain (120-i.4) has a, in processing order (118), contiguous subsequence of one or more SEI messages (114) attributed thereto, wherein the SEI processing order information (116) classifies each SEI message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain;determining the list (122) of zero, one or more SEI messages (114) bydetermining the list (122) of zero, one or more SEI messages (114) to be empty if the base sub-chain (119) comprises at least one SEI message (114) which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list (122) of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective subordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX73leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective subordinate sub-chain in the processing order, wherein, by bitstream constraint (240), there does not exist any SEI message indicated by the SEI processing order information (116) comprised by the base sub-chain which is as being non-optional and which follows, in the processing order, a sub-ordinate sub-chain having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain.

24. Method for generating a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), comprisingencoding SEI processing order information (116) into the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112); so that a media decoder, is capable of determining a list (122) of zero, one or more SEI messages based onthe SEI processing order information (116) andbybased on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains comprising a base sub-chain and zero or more subordinate sub-chains such that each subordinate sub-chain has a, in processing order, contiguous subsequence of one or more SEI messages attributed thereto, wherein the SEI processing order information (116) classifies each SEI message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain;andFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX74determining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective sub-ordinate subchain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andleaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective subordinate sub-chain in the processing order,wherein the method comprisesencoding the SEI processing order information (116) such that there does not exist any SEI message indicated by the SEI processing order information (116) comprised by the base sub-chain which is as being non-optional and which follows, in the processing order, a sub-ordinate sub-chain having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain.

25. Method for decoding a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), comprisingFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX75decoding SEI processing order information (116) from the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112);determining a list (122) of zero, one or more SEI messages (114) based onthe SEI processing order information (116) andwhich of the SEI messages (114) the media decoder (110) supports (123) ;processing the list (122) of zero, one or more SEI messages ,wherein the method (110) comprisesbased on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains (117) comprising a base sub-chain 119 and zero or more subordinate sub-chains such that each subordinate sub-chain (120I.4) has a, in processing order (118), contiguous subsequence of one or more SEI messages (114) attributed thereto, wherein the SEI processing order information (116) classifies (121) each SEI message (114) as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain;determining the list (122) of zero, one or more SEI messages bydetermining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list (122) of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and is indicated by the SEI processing order information (116) as optional,FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX76leaving out each sub-ordinate sub-chain (120I.4) containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain (120I.4) and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain (120I.4), andleaving out each sub-ordinate sub-chain (1204-4) containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain (120I.4) and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain (120I.4), and all sub-ordinate subchains (120I-4) following the respective sub-ordinate sub-chain in the processing order, with including (341), if present, any SEI message supported by the media decoder, attributed to the base sub-chain (119) and following the respective sub-ordinate sub-chain (120I-4) in the processing order, or any SEI message supported by the media decoder, attributed to the base sub-chain (119) and following the respective sub-ordinate sub-chain in the processing order, which is indicated by the SEI processing order information (116) as non-optional.

26. Method for generating a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), comprisingencoding SEI processing order information (116) into the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112); so that a media decoder is capable ofdetermining a list (122) of zero, one or more SEI messages based onthe SEI processing order information (116)byFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX77based on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains comprising a base sub-chain and zero or more subordinate sub-chains such that each subordinate sub-chain has a, in processing order, contiguous subsequence of one or more SEI messages attributed thereto, wherein the SEI processing order information (116) classifies each SEI message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain; anddetermining the list of zero, one or more SEI messages bydetermining the list of zero, one or more SEI messages to be empty if the base sub-chain comprises at least one SEI message which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and is indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective subordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andleaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all sub-ordinate sub-chains following the respective sub-ordinate sub-chain in the processing order, with including (341), if present, any SEI message supported by the media decoder, attributed to the base sub-chain and following the respective sub-ordinate sub-chain in the processing order, or any SEIFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX78message supported by the media decoder, attributed to the base sub-chain and following the respective sub-ordinate sub-chain in the processing order, which is indicated by the SEI processing order information (116) as non-optional.

27. Method for decoding a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), comprisingdecoding SEI processing order information (116) from the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112);determining a list (122) of zero, one or more SEI messages based onthe SEI processing order information (116) andwhich of the SEI messages (114) the media decoder supports ;processing the list (122) of zero, one or more SEI messages ,wherein the method comprisesbased on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains (117) comprising a base sub-chain (119) and zero or more subordinate sub-chains (120I.4) such that each subordinate sub-chain (120-i.4) has a, in processing order (118), contiguous subsequence of one or more SEI messages (114) attributed thereto, wherein the SEI processing order information (116) classifies each SEI message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain;determining the list (122) of zero, one or more SEI messages (114) bydetermining the list (122) of zero, one or more SEI messages (114) to be empty if the base sub-chain (119) comprises at least one SEI message (114) which is notFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX79supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list (122) of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective subordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andleaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having an SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective subordinate sub-chain in the processing order, wherein, by bitstream constraint (240), there does not exist any SEI message indicated by the SEI processing order information (116) as forming an end of a subordinate sub-chain which follows, in the processing order, an SEI message classified as forming a start of a further sub-ordinate sub-chain which has no SEI message attributed thereto which is classified as forming an end of the further subordinate sub-chain.

28. Method for generating a data stream (112) which is associated with supplemental enhancement information (SEI) messages (114), comprisingencoding SEI processing order information (116) into the data stream (112) which defines a processing order (118) among, and an optionality with respect to, SEI messages (114) associated with the data stream (112); so that a media decoder, is capable ofFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX80determining a list (122) of zero, one or more SEI messages based onthe SEI processing order information (116) and bybased on the SEI processing order information (116), attributing the SEI messages to one or more SEI sub-chains (117) comprising a base sub-chain (119) and zero or more subordinate sub-chains (120I.4) such that each subordinate sub-chain (120-i.4) has a, in processing order (118), contiguous subsequence of one or more SEI messages (114) attributed thereto, wherein the SEI processing order information (116) classifies each SEI message as one of forming - along the processing order - a start of a sub-chain, forming - along the processing order - an end of a sub-chain, or continuing - along the processing order - a sub-chain; anddetermining the list (122) of zero, one or more SEI messages (114) bydetermining the list (122) of zero, one or more SEI messages (114) to be empty if the base sub-chain (119) comprises at least one SEI message (114) which is not supported by the media decoder and is indicated by the SEI processing order information (116) as non-optional,populating the list (122) of zero, one or more SEI messages with the SEI messages along the processing order withleaving out SEI messages not supported by the media decoder and being indicated by the SEI processing order information (116) as optional,leaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having both an SEI message attributed thereto classified as forming a start of the respective subordinate sub-chain and an SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, andleaving out each sub-ordinate sub-chain containing at least one SEI message which is non-optional and not supported by the media decoder and having anFH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX81SEI message attributed thereto classified as forming a start of the respective sub-ordinate sub-chain and no SEI message attributed thereto classified as forming an end of the respective sub-ordinate sub-chain, and all SEI messages following the respective subordinate sub-chain in the processing order (118), wherein, by bitstream constraint (442), there does not exist any SEI message indicated by the SEI processing order information (116) as forming an end of a subordinate sub-chain which follows, in the processing order, an SEI message classified as forming a start of a further sub-ordinate sub-chain which has no SEI message attributed thereto which is classified as forming an end of the further sub-ordinate sub-chain.

29. Bitstream generated by the media encoder of any of the claims 12 to 22.FH250304PEP-Vorschlag Nachanmeldung v2 - CC - ANMELDEFASSUNG.DOCX

Images