Video coding SEI messages indicating crucial processing stages

Abstract

An example method includes: writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a sub-chain; wherein the sub-chain comprises plurality of processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the plurality of processing stages of the sub-chain are processed; defining value combinations of first and second flags to indicate the sub-chain; wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value; wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

Description

VIDEO CODINGTECHNICAL FIELD

[0001] The examples and non-limiting embodiments relate generally to video processing and, more particularly to, using sub-chains for supplemental enhancement information processing order.BACKGROUND

[0002] It is known to provide standardized formats for encoding, signaling, or decoding of media data.SUMMARY

[0003] Example 1: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a subchain; wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed; defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain; wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the subchain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

[0004] Example 2: The apparatus of example 1 , wherein a processing stage is described by an information message type in the processing order information message.

[0005] Example 3: The apparatus of example 2, wherein the first flag equal to the second value and the second flag equal to the second value indicates an optional information message type.

[0006] Example 4: The apparatus of any of the previous examples, wherein the first flag equal to the first value and the second flag equal to the first value indicates a crucial information message type. In some embodiments, crucial information message types may be different from crucial processing stages. For example, a start and end of sub-chains are crucial processing stages, whereas crucial information message types are crucial stages that neither start nor end of a sub-chain. .

[0007] Example 5: The apparatus of example 4, wherein the crucial information message type is crucial to the sub-chain when the crucial information message type belongs to the sub-chain or the entire processing chain when the crucial information message type does not belong to the sub-chain.

[0008] Example 6: The apparatus of example 3, wherein when the first flag equal to the second value and the second flag equal to the second value indicates the optional information message type and when a functionality indicated by the optional information message type cannot be interpreted or is not supported by the decoding system, data associated with a loop entry is ignored by the decoding system and the optional information messagetype is excluded from the processing chain performed by the decoding system, and wherein the processing chain is described with a loop, and wherein the loop entry describes a processing stage.

[0009] Example / : The apparatus of example 1, wherein when the first flag equal to the first value and the second flag equal to the second value indicates the start of the sub-chain and when a functionality indicated by the information message type cannot be interpreted or is not supported by the decoding system, one of the following applies: when a next value combination of the first flag and the second flag, in processing order, for either the start or end of the sub-chain indicates the start of the sub-chain, data associated with a loop entry and following loop entries in processing order are ignored by the decoding system and the information message type and a following information message types in processing order are excluded from the processing chain performed by the decoding system; or when the next value combination of the first flag and the second flag, in processing order, for either the start or end of the sub-chain indicates the end of a sub-chain: data associated with the loop entry and following loop entries in processing order until the end of the sub-chain is ignored by the decoding system; and the information message type and following information message types in processing order until the end of the subchain are excluded from the processing chain performed by the decoding system; and wherein the processing chain is described with a loop, and wherein the loop entry describes a processing stage.

[0010] Example 8: The apparatus of example 1 , wherein when the first flag equal to the second value and the second flag equal to the first value indicates the end of the sub-chain and when a functionality indicated by the information message type cannot be interpreted or is not supported by the decoding system, data associated with a loop entry and preceding loop entries in processing order since the start of the sub-chain is ignored by the decoding system and the information message type and preceding information message types in processing order since the start of the sub-chain are excluded from the processing chain performed by the decoding system, and wherein the processing chain is described with a loop, and wherein the loop entry describes a processing stage.

[0011] Example 9: The apparatus of example 4, wherein when the first flag equal to the first value and the second flag equal to the first value indicates the crucial information message type and when a functionality indicated by the information message type cannot be interpreted or is not supported by the decoding system, one of the following applies: when the crucial information message type describes a processing stage of the sub-chain, the sub-chain is excluded from the processing chain performed by the decoding system; or when the crucial information message type describes a processing stage not in any sub-chain, the processing chain is not performed by the decoding system.

[0012] Example 10: The apparatus of example 1 , wherein a sub-chain at a nesting level comprises zero or more sub-chains at a next nesting level and zero or more information message types not included in the zero or more sub-chains at next nesting level.

[0013] Example 11: The apparatus of example 10, wherein the apparatus is further caused to perform: indicating, in the processing order information message, whether an information message type belongs to the subchain and nesting levels for the information message type when the information message type belongs to the subchain.

[0014] Example 12: The apparatus of example 10 or 11, wherein when a functionality indicated by thecrucial information message of the sub-chain at a nesting level cannot be interpreted or is not supported by the decoding system, data associated with the sub-chain are excluded from the processing chain performed by the decoding system.

[0015] Example 13: The apparatus of any of the examples 10 to 12, wherein a sub-chain start depth, and a sub-chain end depth is indicated and / or encoded in the information message type.

[0016] Example 14: The apparatus of example 13, wherein when a current sub chain level is initialized to zero, value combinations of the sub-chain start depth, and the sub-chain end depth are specified as following: the sub-chain start depth equal to the second value and the sub-chain end depth equal to the second value indicates an optional information message type; the sub-chain start depth equal to a start depth that is greater than the second value indicates that the information message is a start of the zero or more sub-chains at the start depth level and a current sub chain level is incremented by a value of the start depth; the sub-chain end depth equal to an end depth that is greater than the second value indicates that the information message is an end of the zero or more sub-chains at the end depth level and the current sub chain level is decremented by a value of the end depth; when the current sub-chain depth level is equal to the second value, the value of the sub-chain start depth is equal to the first value, the value of the sub-chain end depth is equal to the first value, and the information message type is crucial; and / or when the functionality indicated by the crucial information message cannot be interpreted or is not supported by the decoding system, the processing chain specified by the crucial information message is not be performed by the decoding system.

[0017] Example 15: The apparatus of any of the examples 13 or 14, wherein values of the start sub-chain depth and the end sub-chain depth at current sub chain level is greater than or equal to the second value.

[0018] Example 16: The apparatus of any of the previous examples, wherein when a start of the sub-chain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain in processing order that is followed by a start of the next sub-chain prior to the end of the sub-chain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

[0019] Example 17: The apparatus of example 16, wherein when an i-th information message type cannot be interpreted or not supported by the decoding system and the i-th information message type belongs to a second sub-chain, the following applies: when a start of the second sub-chain comprises a processing stage index greater than or equal to open ended start index, the second sub-chain and any of the processing stages following the second sub-chain are not processed by the decoding system; and when the start of the second sub-chain comprises the processing stage index less than open ended start index, the second sub-chain is not processed by the decoding system.

[0020] Example 18: The apparatus of example 17, wherein the i-th information message type is interpreted as following: when value of i is less than the open ended start index and there is no preceding start of a sub-chain without a matching end of the sub-chain, the i-th information message type is a mandatory processing stage, and wherein when the i-th information message type cannot be interpreted or supported by the decoding system, the processing chain specified by the i-th information message is not applied or processed by the decoding system;when a sub-chain start is indicated with a start index less than the value of i and there is no sub-chain end with index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and when none of the above apples, the i-th information message type forms a sub-chain; the i-th information message type is a mandatory processing stage, wherein when the i-th information message type cannot be interpreted or supported by the decoding system, the processing chain specified by the i-th information message is not applied or processed by the decoding system; or the apparatus is disallowed from creating a processing order information message.

[0021] Example 19: The apparatus of example 16, wherein the first flag and the second flag are interpreted as following: when the first flag and the second flag comprise the second value, and wherein when the i-th information message type cannot be interpreted or supported by the decoding system, data associated with a loop variable value of i is ignored by the decoding system and the i-th information message type is excluded from the processing chain performed by the decoding system; when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain; when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts a sub-chain, and wherein the start of the first sub-chain in processing order that is followed by the start the next sub-chain prior to the end of first sub-chain is the start of open-ended sub-chains comprising the processing stage index as the open ended start index; and when the first flag comprises the first value and the second flag comprises the first value, and wherein when value of i is less than the open ended start index and there is no preceding start of a sub-chain without a matching end of the sub-chain, the i-th information message type is a mandatory processing stage, wherein when the i-th information message type cannot be interpreted or supported by the decoding system, the processing chain specified by the i-th information message is not applied or processed by the decoding system; and wherein when a sub-chain start is indicated with a start index less than the value of i and there is no sub-chain end with index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and otherwise the i-th information message type forms a sub-chain.

[0022] Example 20: The apparatus of example 16, wherein the first flag and the second flag are interpreted as following: when the first flag and the second flag comprise the second value, the i-th information message is optional; when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain; when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts the sub-chain; and when the first flag comprises the first value and the second flag comprises the first value, and wherein when the i-th information message type precedes open-ended sub-chains, the processing chain specified by the i-th information message is a mandatory processing stage; and otherwise the i-th information message belongs to the sub-chain.

[0023] Example 21 : The apparatus of example 16, wherein the first flag and the second flag are interpreted as following: when the first flag and the second flag comprise the second value, and wherein when the i-th information message type cannot be interpreted or supported by the decoding system, data associated with a loop variable value of i is ignored by the decoding system and the i-th information message type is excluded from the processing chain performed by the decoding system; when the first flag comprises the second value and the second flagcomprises the first value, the i-th information message ends a sub-chain; when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts a sub-chain; and when the first flag comprises the first value and the second flag comprises the first value, and wherein when a sub-chain start is indicated with start index less than value of i and there is no sub-chain with end index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and wherein when the i-th information message type precedes open-ended sub-chains of a processing chain, the the processing chain specified by the i-th information message is a mandatory processing stage, and wherein when the i-th information message type cannot be interpreted or supported by the decoding system the processing chain specified by the i-th information message is not applied or processed by the decoding system; and otherwise, the i-th information message type forms a sub-chain.

[0024] Example 22: The apparatus of any of the previous examples, wherein: the first value comprises one and the second value comprises zero; or the first value comprises zero and the second value comprises one.

[0025] Example 23: The apparatus of any of the previous examples, wherein the first flag comprises a processing order supplemental enhancement information importance flag and the second flag comprises a processing order processing degree flag.

[0026] Example 24: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when the apparatus cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the sub-chain are processed; decoding for a processing stage a first flag and a second flag; wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

[0027] Example 25: The apparatus of example 24, wherein a processing stage is described by an information message type in the processing order information message.

[0028] Example 26: The apparatus of example 1925 wherein the first flag equal to the second value and the second flag equal to the second value indicates an optional information message type.

[0029] Example 27: The apparatus of any of the examples 24 to 26, wherein the first flag equal to the first value and the second flag equal to the first value indicates a crucial information message type. In some embodiments, crucial information message types may be different from crucial processing stages. For example, a start and end of sub-chains are crucial processing stages, whereas crucial information message types are crucial stages that neither start nor end of a sub-chain.

[0030] Example 28: The apparatus of example 27, wherein the crucial information message type is crucial to the sub-chain when the crucial information message type belongs to the sub-chain or the entire processing chain when the crucial information message type does not belong to the sub-chain.

[0031] Example 29: The apparatus of example 26, wherein when the first flag equal to the second value and the second flag equal to the second value indicates the optional information message type and when a functionality indicated by the optional information message type cannot be interpreted or is not supported by the apparatus, the apparatus is further caused to perform: ignoring data associated with a loop entry ; excluding the optional information message type from the processing chain performed; wherein the processing chain is described with a loop; and wherein the loop entry describes a processing stage.

[0032] Example 30: The apparatus of example 24, wherein when values of the first flag and the second flag indicate a crucial processing stage, when the crucial processing stage belongs to the sub-chain, and when a functionality indicated by the information message type cannot be interpreted or is not supported by the apparatus, the apparatus is further caused to perform: excluding the sub-chain from the processing chain.

[0033] Example 31 : The apparatus of example 24, wherein when values of the first flag and the second flag indicate a crucial processing stage, when the crucial processing stage does not belong to any sub-chain, and when a functionality indicated by the information message type cannot be interpreted or is not supported by the apparatus, the apparatus is further caused to perform: omitting the processing chain.

[0034] Example 32: The apparatus of example 24, wherein a sub-chain at a nesting level comprises zero or more sub-chains at a next nesting level and zero or more information message types not included in the zero or more sub-chains at next nesting level.

[0035] Example 33: The apparatus of example 32, wherein the apparatus is further caused to perform: decoding, from the processing order information message, whether an information message type belongs to the sub-chain and nesting levels for the information message type when the information message type belongs to the sub-chain.

[0036] Example 34: The apparatus of example 32 or 33, wherein when a functionality indicated by the crucial information message of the sub-chain at a nesting level cannot be interpreted or is not supported by the apparatus, the apparatus is further caused to perform: excluding data associated with the sub-chain from the processing chain.

[0037] Example 35: The apparatus of any of the examples 32 to 34, wherein a sub-chain start depth, and a sub-chain end depth is determined and / or decoded from the information message type.

[0038] Example 36: The apparatus of example 35, wherein when a current sub chain level is initialized to zero, value combinations of the sub-chain start depth, and the sub-chain end depth are specified as following: the subchain start depth equal to the second value and the sub-chain end depth equal to the second value indicates an optional information message type; the sub-chain start depth equal to a start depth that is greater than the second value indicates that the information message is a start of the zero or more sub-chains at the start depth level and a current sub chain level is incremented by a value of the start depth; the sub-chain end depth equal to an end depth that is greater than the second value indicates that the information message is an end of the zero or more sub-chains at the end depth level and the current sub chain level is decremented by a value of the end depth;when the current sub-chain depth level is equal to the second value, the value of the sub-chain start depth is equal to the first value, the value of the sub-chain end depth is equal to the first value, and the information message type is crucial; and / or when the functionality indicated by the crucial information message cannot be interpreted or is not supported by the apparatus is further caused to perform: omitting the processing chain specified by the crucial information message.

[0039] Example 37: The apparatus of any of the examples 35 or 36, wherein values of the start sub-chain depth and the end sub-chain depth at current sub chain level is greater than or equal to the second value.

[0040] Example 38: The apparatus of any of the examples 24 to 37, wherein when a start of the sub-chain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain in processing order that is followed by a start of the next sub-chain prior to the end of the subchain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

[0041] Example 39: The apparatus of example 38, wherein when the apparatus cannot interpret or support an i-th information message type and the i-th information message type belongs to a second sub-chain, the following applies: when a start of the second sub-chain comprises a processing stage index greater than or equal to open ended start index, the apparatus is further caused to perform: omitting the second sub-chain and any of the processing stages following the second sub-chain; and when the start of the second sub-chain comprises the processing stage index less than open ended start index, the apparatus is further caused to perform: omitting the second sub-chain.

[0042] Example 40: The apparatus of example 39, wherein the i-th information message type is interpreted as following: when value of i is less than the open ended start index and there is no preceding start of a sub-chain without a matching end of the sub-chain, the i-th information message type is a mandatory processing stage, and wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting the processing chain specified by the i-th information message; when a sub-chain start is indicated with a start index less than the value of i and there is no sub-chain end with index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and when none of the above apples, the i-th information message type forms a sub-chain; the i-th information message type is a mandatory processing stage, wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting the processing chain specified by the i-th information message; or an encoder is disallowed from creating a processing order information message.

[0043] Example 41 : The apparatus of example 38, wherein the first flag and the second flag are interpreted as following: when the first flag and the second flag comprise the second value, and wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting or ignoring the data associated with a loop variable value of i and excluding the i-th information message type from the processing chain; when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain; when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts a sub-chain, and wherein the start of the firstsub-chain in processing order that is followed by the start the next sub-chain prior to the end of first sub-chain is the start of open-ended sub-chains comprising the processing stage index as the open ended start index; and when the first flag comprises the first value and the second flag comprises the first value, and wherein when value of i is less than the open ended start index and there is no preceding start of a sub-chain without a matching end of the sub-chain, the i-th information message type is a mandatory processing stage, wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting the processing chain specified by the i-th information message; and wherein when a sub-chain start is indicated with a start index less than the value of i and there is no sub-chain end with index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and otherwise the i-th information message type forms a sub-chain.

[0044] Example 42: The apparatus of example 38, wherein the first flag and the second flag are interpreted as following: when the first flag and the second flag comprise the second value, the i-th information message is optional; when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain; when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts the sub-chain; and when the first flag comprises the first value and the second flag comprises the first value, and wherein when the i-th information message type precedes open-ended sub-chains, the processing chain specified by the i-th information message is a mandatory processing stage; and otherwise the i-th information message belongs to the sub-chain.

[0045] Example 43: The apparatus of example 38, wherein the first flag and the second flag are interpreted as following: when the first flag and the second flag comprise the second value, and wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting or ignoring data associated with a loop variable value of i and excluding the i'-th information message type from the processing chain; when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain; when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts a sub-chain; and when the first flag comprises the first value and the second flag comprises the first value, and wherein when a sub-chain start is indicated with start index less than value of i and there is no sub-chain with end index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and wherein when the i-th information message type precedes open-ended sub-chains of a processing chain, the the processing chain specified by the i-th information message is a mandatory processing stage, and wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting the the processing chain specified by the i-th information message; and otherwise, the i-th information message type forms a sub-chain.

[0046] Example 44: The apparatus of any of the examples 24 to 43, wherein: the first value comprises one and the second value comprises zero; or the first value comprises zero and the second value comprises one.

[0047] Example 45: The apparatus of any of the examples 24 to 44, wherein the first flag comprises a processing order supplemental enhancement information importance flag and the second flag comprises a processing order processing degree flag.

[0048] Example 46: The apparatus of example 24, wherein when the apparatus cannot interpret the first crucial processing stage or the second crucial processing stage, the apparatus is further caused to perform: omitting the sub-chain.

[0049] Example 47: A method comprising: writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed; defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain; wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

[0050] Example 48: A method comprising: decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when the method cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the sub-chain are processed; decoding for a processing stage a first flag and a second flag; wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

[0051] Example 49: A method comprising operations that implement operations as indicated as being performed by any of the apparatuses as claimed in any of examples 1 to 23.

[0052] Example 50: A method comprising operations that implement operations as indicated as being performed by any of the apparatuses as claimed in any of examples 24 to 46.

[0053] Example 51: An apparatus comprising: means for writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed; means for defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain; wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

[0054] Example 52: The apparatus of example 51, wherein the apparatus further comprises means for performing operations as indicated as being performed by any of the apparatuses as claimed in any of examples 2 to 23.

[0055] Example 53: An apparatus comprising: means for decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when the apparatus cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the sub-chain are processed; means for decoding for a processing stage a first flag and a second flag; wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

[0056] Example 54: The apparatus of example 53, wherein the apparatus further comprises means for performing operations as indicated as being performed by any of the apparatuses as claimed in any of examples 25 to 46.

[0057] Example 55: A computer readable medium comprising program instructions that, when executed by an apparatus, cause the apparatus to perform: writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed; defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain; wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the sub-chain ends with a second crucial processing stage indicatedby the first flag being equal to the second value and the second flag being equal to the first value.

[0058] Example 56: The apparatus of example 55, wherein the apparatus is further caused to perform operations as indicated as being performed by any of the apparatuses as claimed in any of examples 2 to 23.

[0059] Example 57: The computer readable medium of any of the examples 55 or 56, wherein the computer readable medium comprises a non-transitory computer readable medium.

[0060] Example 58: A computer readable medium comprising program instructions that, when executed by an apparatus, cause the apparatus to perform: decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when the apparatus cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the sub-chain are processed; decoding for a processing stage a first flag and a second flag; wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

[0061] Example 59: The apparatus of example 58, wherein the apparatus is further caused to perform operations as indicated as being performed by any of the apparatuses as claimed in any of examples 25 to 46.

[0062] Example 60: The computer readable medium of any of the examples 58 or 59, wherein the computer readable medium comprises a non-transitory computer readable medium.

[0063] Example 61: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a subchain; wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed; defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain; wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value; and wherein when a start of the sub-chain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain inprocessing order that is followed by a start of the next sub-chain prior to the end of the sub-chain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

[0064] Example 62: The apparatus of example 61, wherein the apparatus is further caused to perform operations as indicated as being performed by any of the apparatuses as claimed in any of examples 2 to 23.

[0065] Example 63: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform: decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when the apparatus cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the sub-chain are processed; decoding for a processing stage a first flag and a second flag; wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value; and wherein when a start of the subchain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain in processing order that is followed by a start of the next sub-chain prior to the end of the sub-chain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

[0066] Example 64: The apparatus of example 63, wherein the apparatus is further caused to perform operations as indicated as being performed by any of the apparatuses as claimed in any of examples 25 to 46.

[0067] Example 65: A method comprising: writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed; defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain; wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value; and wherein when a start of the sub-chain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain in processing order that is followed by a start of the next sub-chain prior to the end of the sub-chain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

[0068] Example 66: The method of example 65 comprising operations that implement operations as indicated as being performed by any of the apparatuses as claimed in any of examples 2 to 23.

[0069] Example 67: A method comprising: decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain; wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when the apparatus cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the sub-chain are processed; decoding for a processing stage a first flag and a second flag; wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value; wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; and wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value; and wherein when a start of the sub-chain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain in processing order that is followed by a start of the next sub-chain prior to the end of the sub-chain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

[0070] Example 68: The method of example 67 comprising operations that implement operations as indicated as being performed by any of the apparatuses as claimed in any of examples 25 to 46.BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The foregoing embodiments and other features are explained in the following description, taken in connection with the accompanying drawings, wherein:

[0072] FIG. 1 shows schematically an apparatus employing embodiments of the examples described herein.

[0073] FIG. 2 shows schematically a user equipment suitable for employing embodiments of the examples described herein.

[0074] FIG. 3 further shows schematically electronic devices employing embodiments of the examples described herein connected using wireless and wired network connections.

[0075] FIG. 4 is a block diagram illustrating a system in accordance with an example.

[0076] FIG. 5 represents an example of a supplemental enhancement information processing order (SPO) supplemental enhancement information (SEI) messages message that specifies a cascade of three NNPFs.

[0077] FIG. 6 illustrates an example of a processing chain.

[0078] FIG. 7 illustrates an example implementation for realizing the example of FIG. 6, in accordance with an embodiment.

[0079] FIG. 8 illustrates an example of a processing chain that includes nested sub-chains, in accordance with an embodiment.

[0080] FIG. 9 is an example apparatus, which may be implemented in hardware, and is caused to, implement examples described herein.

[0081] FIG. 10 shows a representation of an example of non-volatile memory media used to store instructions that implement the examples described herein.

[0082] FIG. 11 is an example method performed with an encoder, based on the examples described herein.

[0083] FIG. 12 is an example method performed with an decoder, based on the examples described herein.

[0084] FIG. 13 is another example method performed with an encoder, based on the examples described herein.

[0085] FIG. 14 is another example method performed with an decoder, based on the examples described herein.DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0086] The following acronyms and abbreviations that may be found in the specification and / or the drawing figures are defined as follows (the abbreviations may be appended with each other or with other characters using e.g. a hyphen or dash (-), and may be case insensitive):4CC four character code5G fifth generation cellular network technology5GC 5G core networka.k.a. also known asAVC advanced video codingCU coding unitDSP digital signal processorDU distributed uniteNB (oreNodeB) evolved Node B (for example, an LTE base station)EN-DC E-UTRA-NR dual connectivityen-gNB or En-gNB node providing NR user plane and control plane protocol terminations towards the UE, and acting as secondary node in EN-DCE-UTRA evolved universal terrestrial radio access, for example, the LTE radio access technologyF1 or F1-C interface between CU and DU control interfacegNB (orgNodeB) base station for 5G / NR, for example, a node providing NR user plane and control plane protocol terminations towards the UE, and connected via the NG interface to the 5GCIEC International Electrotechnical CommissionloT internet of thingsISO International Organization for StandardizationISOBMFF ISO base media file formatJPEG joint photographic experts groupLTE long-term evolutionmdat MediaDataBoxMIME Multipurpose Internet Mail ExtensionMME mobility management entitymoov MovieBoxMP4 file format for MPEG-4 Part 14 filesMPEG moving picture experts groupMPEG-2 H.222 / H.262 as defined by the ITUMPEG-4 audio and video coding standard for ISO / IEC 14496 ng or NG new generationng-eNB or NG-eNB new generation eNBNR new radio (5G radio)N / Wor NW networkPDCP packet data convergence protocolPHY physical layerPNG portable network graphicsRAN radio access networkRFC request for commentsRLC radio link controlRRC radio resource controlRRH remote radio headRU radio unitRx receiverSDAP service data adaptation protocolSGW serving gatewaySMF session management functionSPS sequence parameter setSVC scalable video codingS1 interface between eNodeBs and the EPC trak Track BoxTx transmitterUE user equipmentUICC Universal Integrated Circuit CardUPF user plane functionURL uniform resource locatorX2 interconnecting interface between two eNodeBs in LTE networkXn interface between two NG-RAN nodes

[0087] Some embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all, embodiments may be shown. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. As used herein, the terms ‘data,’ ‘content,’ ‘information,’ and similar terms may be used interchangeably to refer to data capable of being transmitted, received and / or stored in accordance with embodiments of the present invention. Thus, use of any such terms should not be taken to limit the spirit and scope of embodiments.

[0088] Described herein is a method and apparatus for using sub-chains for supplemental enhancement information processing order.

[0089] The following describes in detail a suitable apparatus and possible method for using sub-chains for supplemental enhancement information processing order according to embodiments. In this regard reference is first made to FIG. 1 and FIG. 2, where FIG. 1 shows an example block diagram of an electronic device or apparatus 100. The apparatus 100 may be an Internet of Things (loT) apparatus configured to perform various functions, such as for example, gathering information by one or more sensors, receiving or transmitting information, analyzing information gathered or received by the apparatus, or the like. The apparatus may comprise a video coding system, which may incorporate a codec. FIG. 2 shows a layout of an apparatus according to an example embodiment. The elements of FIG. 1 and FIG. 2 are explained next.

[0090] The apparatus 100 may for example be a mobile terminal or user equipment of a wireless communication system, a sensor device, a tag, or other lower power device. However, it would be appreciated that embodiments of the examples described herein may be implemented within any electronic device or apparatus which may process data by neural networks.

[0091] The apparatus 100 may comprise a housing 101 for incorporating and protecting the device. The apparatus 100 further may comprise a display 102 in the form of a liquid crystal display. In other embodiments of the examples described herein the display may be any suitable display technology suitable to display an image or video. The apparatus 100 may further comprise a keypad 104. In other embodiments of the examples described herein any suitable data or user interface mechanism may be employed. For example the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.

[0092] The apparatus may comprise a microphone 106 or any suitable audio input which may be a digital or analog signal input. The apparatus 100 may further comprise an audio output device which in embodiments of the examples described herein may be any one of: an earpiece 108, speaker, or an analog audio or digital audio output connection. The apparatus 100 may also comprise a battery (or in other embodiments of the examples describedherein the device may be powered by any suitable mobile energy device such as solar cell, fuel cell or clockwork generator). The apparatus 100 may further comprise a camera 109 capable of recording or capturing images and / or video. The apparatus 100 may further comprise an infrared port for short range line of sight communication to other devices. In other embodiments the apparatus 100 may further comprise any suitable short range communication solution such as for example a Bluetooth wireless connection or a USB / firewire wired connection.

[0093] The apparatus 100 may comprise a controller 110, processor or processor circuitry for controlling the apparatus 100. The controller 110 may be connected to memory 112 which in embodiments of the examples described herein may store both data in the form of image and audio data and / or may also store instructions for implementation on the controller 110. The controller 110 may further be connected to codec circuitry 114 suitable for carrying out coding and / or decoding of audio and / or video data or assisting in coding and / or decoding carried out by the controller.

[0094] The apparatus 100 may further comprise a card reader 118 and a smart card 116, for example a UICC and UICC reader for providing user information and being suitable for providing authentication information for authentication and authorization of the user at a network.

[0095] The apparatus 100 may comprise radio interface circuitry 120 connected to the controller and suitable for generating wireless communication signals for example for communication with a cellular communications network, a wireless communications system or a wireless local area network. The apparatus 100 may further comprise an antenna 122 connected to the radio interface circuitry 120 for transmitting radio frequency signals generated at the radio interface circuitry 120 to other apparatus(es) and / or for receiving radio frequency signals from other apparatus(es).

[0096] The apparatus 100 may comprise a camera capable of recording or detecting individual frames which are then passed to the codec circuitry 114 or the controller for processing. The apparatus may receive the video image data for processing from another device prior to transmission and / or storage. The apparatus 100 may also receive either wirelessly or by a wired connection the image for coding / decoding. The structural elements of apparatus 100 described above represent examples of means for performing a corresponding function.

[0097] With respect to FIG. 3, an example of a system within which embodiments of the examples described herein can be utilized is shown. The system 300 comprises multiple communication devices which can communicate through one or more networks. The system 300 may comprise any combination of wired or wireless networks including, but not limited to a wireless cellular telephone network (such as a GSM, UMTS, CDMA, LTE, 4G, 5G network, etc.), a wireless local area network (WLAN) such as defined by any of the IEEE 802.x standards, a Bluetooth personal area network, an Ethernet local area network, a token ring local area network, a wide area network, and the Internet.

[0098] The system 300 may include both wired and wireless communication devices and / or apparatus 100 suitable for implementing embodiments of the examples described herein.

[0099] For example, the system shown in FIG. 3 shows a mobile telephone network 301 and a representation of the internet 302. Connectivity to the internet 302 may include, but is not limited to, long range wireless connections, short range wireless connections, and various wired connections including, but not limited to,telephone lines, cable lines, power lines, and similar communication pathways.

[0100] The example communication devices shown in the system 300 may include, but are not limited to, an electronic device or apparatus 100, a combination of a personal digital assistant (PDA) and a mobile telephone 304, a PDA 306, an integrated messaging device (IMD) 308, a desktop computer 310, a notebook computer 312, or a head-mounted apparatus. The head-mounted apparatus may be a head-mounted display (HMD), or glasses having a device such as a camera configured to encode and / or decode images and / or video. The apparatus 100 may be stationary or mobile when carried by an individual who is moving. The apparatus 100 may also be located in a mode of transport including, but not limited to, a car, a truck, a taxi, a bus, a train, a boat, an airplane, a bicycle, a motorcycle or any similar suitable mode of transport.

[0101] The embodiments may also be implemented in a set-top box; e.g., a digital TV receiver, which may / may not have a display or wireless capabilities, in tablets or (laptop) personal computers (PC), which have hardware and / or software to process neural network data, in various operating systems, and in chipsets, processors, DSPs and / or embedded systems offering hardware / software based coding.

[0102] Some or further apparatus may send and receive calls and messages and communicate with service providers through a wireless connection 314 to a base station 316. The base station 316 may be connected to a network server 318 that allows communication between the mobile telephone network 301 and the internet 302. The system may include additional communication devices and communication devices of various types.

[0103] The communication devices may communicate using various transmission technologies including, but not limited to, code division multiple access (CDMA), global systems for mobile communications (GSM), universal mobile telecommunications system (UMTS), time divisional multiple access (TDMA), frequency division multiple access (FDMA), transmission control protocol-internet protocol (TCP-IP), short messaging service (SMS), multimedia messaging service (MMS), email, instant messaging service (IMS), Bluetooth, IEEE 802.11, 3GPP Narrowband loT and any similar wireless communication technology. A communications device involved in implementing various embodiments of the examples described herein may communicate using various media including, but not limited to, radio, infrared, laser, cable connections, and any suitable connection.

[0104] In telecommunications and data networks, a channel may refer either to a physical channel or to a logical channel. A physical channel may refer to a physical transmission medium such as a wire, whereas a logical channel may refer to a logical connection over a multiplexed medium, capable of conveying several logical channels. A channel may be used for conveying an information signal, for example a bitstream, from one or several senders (or transmitters) to one or several receivers.

[0105] The embodiments may also be implemented in so-called loT devices. The Internet of Things (loT) may be defined, for example, as an interconnection of uniquely identifiable embedded computing devices within the existing Internet infrastructure. The convergence of various technologies has and may enable many fields of embedded systems, such as wireless sensor networks, control systems, home / building automation, etc. to be included in the Internet of Things (loT). In order to utilize the Internet loT devices are provided with an IP address as a unique identifier. loT devices may be provided with a radio transmitter, such as a WLAN or Bluetooth transmitter or a RFID tag. Alternatively, loT devices may have access to an IP-based network via a wired network,such as an Ethernet-based network or a power-line connection (PLC).

[0106] FIG. 4 is a block diagram illustrating a system or apparatus 400 in accordance with several examples. In an example, the encoder 402 is used to encode an image or video, and the encoder 402 may be implemented in a transmitting apparatus 404. The encoder 402 produces a bitstream 406 comprising signaling that is received by the receiving apparatus 408, which implements a decoder 410. The encoder 402 sends the bitstream 406 that comprises the herein described signaling. The decoder 410 forms the image or video, and the receiving apparatus 408 may present this to the user, e.g., via a smartphone, television, or projector among many other options.

[0107] In some examples, the encoder 402 may reside in a separate apparatus from the transmitting apparatus 404. In some examples, the apparatus comprising the encoder 402 may be connected to the transmitting apparatus 404, e.g., through a memory bus. In some examples, the encoder 402 may produce the bitstream 406 that is stored, e.g. in a mass memory.

[0108] In some examples, the decoder 410 may reside in a separate apparatus from the receiving apparatus 408. In some examples, the apparatus comprising the decoder 410 may be operationally connected to the receiving apparatus 408, e.g., through a memory bus. In some examples, the decoder 410 may obtain the bitstream 406 from a mass memory.

[0109] In some examples, the transmitting apparatus 404 and the receiving apparatus 408 are at least partially within a common apparatus, and for example, are located within a common housing 412. For example, the common apparatus comprising the encoder 402 and decoder 410 implements a codec. In other examples, the encoder 402 and the decoder 410 are at least partially not within a common apparatus and have at least partially different housings, but when together, may still implement a codec.

[0110] As indicated at 414, the decoder 410 performs an operation(s) or action(s) based on the received signaling.

[0111] In some examples, encoding 416 performs encoding of supplemental enhancement information processing order (SPO) supplemental enhancement information (SEI) messages, based on the examples described herein. In some examples, decoding 418 performs decoding of SPO SEI messages, based on the examples described herein.

[0112] Having thus introduced a suitable but non-limiting technical context for the practice of the example embodiments of the present disclosure, example embodiments will now be described in detail.

[0113] Fundamentals of video / image coding

[0114] Video codec includes an encoder that transforms the input video into a compressed representation suited for storage / transmission and a decoder that can decompress the compressed video representation back into a viewable form. Typically, an encoder discards some information in the original video sequence in order to represent the video in a more compact form (that is, at lower bitrate).

[0115] Typical hybrid video codecs, for example ITU-T H.263 and H.264, encode the video information in two phases. Firstly pixel values in a certain picture area (or “block”) are predicted for example by motion compensation means (finding and indicating an area in one of the previously coded video frames that corresponds closely to the block being coded) or by spatial means (using the pixel values around the block to be coded in a specified manner).Secondly the prediction error, e.g., the difference between the predicted block of pixels and the original block of pixels, is coded. This is typically done by transforming the difference in pixel values using a specified transform (e.g., Discrete Cosine Transform (DCT) or a variant of it), quantizing the coefficients and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the accuracy of the pixel representation (picture quality) and size of the resulting coded video representation (file size or transmission bitrate).

[0116] Inter prediction, which may also be referred to as temporal prediction, motion compensation, or motion-compensated prediction, exploits temporal redundancy. In inter prediction the sources of prediction are previously decoded pictures (a.k.a. reference pictures).

[0117] In temporal inter prediction, the sources of prediction are previously decoded pictures in the same scalable layer. In intra block copy (IBC; a.k.a. intra-block-copy prediction), prediction may be applied similarly to temporal inter prediction but the reference picture is the current picture and only previously decoded samples may be referred in the prediction process. Inter-layer or inter-view prediction may be applied similarly to temporal inter prediction, but the reference picture is a decoded picture from another scalable layer or from another view, respectively. In some cases, inter prediction may refer to temporal inter prediction only, while in other cases inter prediction may refer collectively to temporal inter prediction and any of intra block copy, inter-layer prediction, and inter-view prediction provided that they are performed with the same or similar process than temporal prediction. Inter prediction, temporal inter prediction, or temporal prediction may sometimes be referred to as motion compensation or motion-compensated prediction.

[0118] Intra prediction utilizes the fact that adjacent pixels within the same picture are likely to be correlated. Intra prediction can be performed in spatial or transform domain, e.g., either sample values or transform coefficients can be predicted. Intra prediction is typically exploited in intra coding, where no inter prediction is applied.

[0119] One outcome of the coding procedure is a set of coding parameters, such as motion vectors and quantized transform coefficients. Many parameters can be entropy-coded more efficiently if they are predicted first from spatially or temporally neighboring parameters. For example, a motion vector may be predicted from spatially adjacent motion vectors and only the difference relative to the motion vector predictor may be coded. Prediction of coding parameters and intra prediction may be collectively referred to as in-picture prediction.

[0120] The decoder reconstructs the output video by applying prediction means similar to the encoder to form a predicted representation of the pixel blocks (using the motion or spatial information created by the encoder and stored in the compressed representation) and prediction error decoding (inverse operation of the prediction error coding recovering the quantized prediction error signal in spatial pixel domain). After applying prediction and prediction error decoding means the decoder sums up the prediction and prediction error signals (pixel values) to form the output video frame. The decoder (and encoder) can also apply additional filtering means to improve the quality of the output video before passing it for display and / or storing it as prediction reference for the forthcoming frames in the video sequence.

[0121] In typical video codecs the motion information is indicated with motion vectors associated with each motion compensated image block. Each of these motion vectors represents the displacement of the image blockin the picture to be coded (in the encoder side) or decoded (in the decoder side) and the prediction source block in one of the previously coded or decoded pictures. In order to represent motion vectors efficiently those are typically coded differentially with respect to block specific predicted motion vectors. In typical video codecs the predicted motion vectors are created in a predefined way, for example calculating the median of the encoded or decoded motion vectors of the adjacent blocks. Another way to create motion vector predictions is to generate a list of candidate predictions from adjacent blocks and / or co-located blocks in temporal reference pictures and signaling the chosen candidate as the motion vector predictor. In addition to predicting the motion vector values, the reference index of previously coded / decoded picture can be predicted. The reference index is typically predicted from adjacent blocks and / or or co-located blocks in temporal reference picture. Moreover, typical high efficiency video codecs employ an additional motion information coding / decoding mechanism, often called merging / merge mode, where all the motion field information, which includes motion vector and corresponding reference picture index for each available reference picture list, is predicted and used without any modification / correction. Similarly, predicting the motion field information is carried out using the motion field information of adjacent blocks and / or co-located blocks in temporal reference pictures and the used motion field information is signaled among a list of motion field candidate list filled with motion field information of available adjacent / co-located blocks.

[0122] In typical video codecs the prediction residual after motion compensation is first transformed with a transform kernel (like DCT) and then coded. The reason for this is that often there still exists some correlation among the residual and transform can in many cases help reduce this correlation and provide more efficient coding.

[0123] Typical video encoders utilize Lagrangian cost functions to find optimal coding modes, e.g., the desired Macroblock mode and associated motion vectors. This kind of cost function uses a weighting factor A to tie together the (exact or estimated) image distortion due to lossy coding methods and the (exact or estimated) amount of information that is required to represent the pixel values in an image area:C = D + AR

[0124] where C is the Lagrangian cost to be minimized, D is the image distortion (e.g. Mean Squared Error) with the mode and motion vectors considered, and R the number of bits needed to represent the required data to reconstruct the image block in the decoder (including the amount of data to represent the candidate motion vectors).

[0125] The High Efficiency Video Coding (H.265 / HEVC a.k.a. HEVC) standard was originally developed by the Joint Collaborative Team - Video Coding (JCT-VC) of VCEG and MPEG. The standard was published by both parent standardization organizations, and it is referred to as ITU-T Recommendation H.265 and ISO / IEC International Standard 23008-2, also known as MPEG-H Part 2 High Efficiency Video Coding (HEVC). Version 2 of the H.265 / HEVC standard included scalable, multiview, fidelity range, three-dimensional, and screen content coding extensions which may be abbreviated SHVC, MV-HEVC, REXT, 3D-HEVC, and SCC, respectively.

[0126] Versatile Video Coding (WC) (MPEG-I Part 3), a.k.a. ITU-T H.266, is a video compression standard developed by the Joint Video Experts Team (JVET) of the Moving Picture Experts Group (MPEG), (formally ISO / IEC JTC1 SC29 WG11) and Video Coding Experts Group (VCEG) of the International TelecommunicationUnion (ITU) to be the successor to HEVC / H.265.

[0127] A specification of the AV1 bitstream format and decoding process were developed by the Alliance for Open Media (AOM). The AV1 specification was published in 2018. AOM is reportedly working on the AV2 specification.

[0128] Some key definitions, bitstream and coding structures, and concepts of some video coding standards and specifications are described in this section for providing background for a video encoder, decoder, encoding method, decoding method, and a bitstream structure, wherein the embodiments may be implemented. It is to be understood that embodiments are not limited to the referenced video coding standards or specifications.

[0129] Video coding standards may specify the bitstream syntax and semantics as well as the decoding process for error-free bitstreams, whereas the encoding process might not be specified, but encoders may just be required to generate conforming bitstreams. Bitstream and decoder conformance can be verified with the Hypothetical Reference Decoder (HRD). The standards may contain coding tools that help in coping with transmission errors and losses, but the use of the tools in encoding may be optional and decoding process for erroneous bitstreams might not have been specified.

[0130] An elementary unit for the input to an encoder and the output of a decoder, respectively, in many cases is a picture. A picture given as an input to an encoder may also be referred to as a source picture, and a picture decoded by a decoded may be referred to as a decoded picture or a reconstructed picture.

[0131] The source and decoded pictures are each comprised of one or more sample arrays. The sample arrays of a picture may be referred to as luma (or L or Y) and chroma, where the two chroma arrays may be referred to as Cb and Cr; regardless of the actual color representation method in use. The actual color representation method in use can be indicated e.g., in a coded bitstream e.g., using the Video Usability Information (VUI) syntax of HEVC or alike. A component may be defined as an array or single sample from one of the three sample arrays (luma and two chroma) or the array or a single sample of the array that compose a picture in monochrome format.

[0132] Samples of a sample array have a certain bit depth, such as 8 bits per sample or 10 bits per sample. A bit depth implicitly specifies a value range, which may be referred to as the full range. For example, the full range is from 0 to 255, inclusive, for 8 bits per sample, or from 0 to 1023, inclusive, for 10 bits per sample. The source video may use allocate a narrower sample value range than the full range. A specific value range, sometimes referred to as the studio range, has been specified in the ITU-T H.273 standard specifying coding-independent code points for video. A source value range may interchangeably be referred to as a source sample value range, and may be defined as the sample value range of the video that is given as input to a video encoder to be encoded.

[0133] A picture may be defined to be either a frame or a field. A frame comprises a matrix of luma samples and possibly the corresponding chroma samples. A field is a set of alternate sample rows of a frame and may be used as encoder input, when the source signal is interlaced. Chroma sample arrays may be absent (and hence monochrome sampling may be in use) or chroma sample arrays may be subsampled when compared to luma sample arrays.

[0134] A bitstream may be defined as a sequence of bits or a sequence of syntax structures. A bitstream format may constrain the order of syntax structures in the bitstream.

[0135] A syntax element may be defined as an element of data represented in a bitstream. A syntax structure may be defined as zero or more syntax elements present together in a bitstream in a specified order.

[0136] Syntax structures may be specified, for example, using arithmetic, logical, relational, bit-wise, and assignment operators similar to those available in many programming languages. For example, & may indicate a bit-wise ‘AND’ operation. Furthermore, syntax structures may be specified with reference to mathematical functions.

[0137] Bit-wise operations may be defined as follows:& bit-wise "and"

[0138] When operating on integer arguments, operates on a two's complement representation of the integer value. When operating on a binary argument that contains fewer bits than another argument, the shorter argument is extended by adding more significant bits equal to 0.| bit-wise "or"

[0139] When operating on integer arguments, operates on a two's complement representation of the integer value. When operating on a binary argument that contains fewer bits than another argument, the shorter argument is extended by adding more significant bits equal to 0.Abit-wise "exclusive or"

[0140] When operating on integer arguments, operates on a two's complement representation of the integer value. When operating on a binary argument that contains fewer bits than another argument, the shorter argument is extended by adding more significant bits equal to 0.x » y arithmetic right shift of a two's complement integer representation of x by y binary digits

[0141] This function is defined only for non-negative integer values of y. Bits shifted into the most significant bits (MSBs) as a result of the right shift have a value equal to the MSB of x prior to the shift operation.x « y arithmetic left shift of a two's complement integer representation of x by y binary digits

[0142] This function is defined only for non-negative integer values of y. Bits shifted into the least significant bits (LSBs) as a result of the left shift have a value equal to 0.

[0143] Syntax structures and semantics may use the values of variables derived from the values of syntax elements. Naming conventions may be defined for variables. For example, variables may be named by a mixture of lower case and upper case letter and without any underscore characters. Variables starting with an upper case letter may be derived for the decoding of the current syntax structure and all depending syntax structures. Variables starting with an upper case letter may, in some cases, be used in the decoding process for later syntax structures without mentioning the originating syntax structure of the variable. Variables starting with a lower case letter may only be used in relation to the syntax structure or function they have been defined for.

[0144] Video coding specifications may define an elementary unit that for the output an of an encoder and / or for the input to a decoder. For example, such an elementary unit may be an open bitstream unit (OBU), as specified e.g. in AV1, ora Network Abstraction Layer (NAL) unit, as specified e.g. in HEVC orVVC.

[0145] In some video codecs, an elementary unit for the output of an encoder and the input of a decoder, respectively, may be a Network Abstraction Layer (NAL) unit. For transport over packet-oriented networks orstorage into structured files, NAL units may be encapsulated into packets or similar structures. A bytestream format has been specified in some video coding standards for transmission or storage environments that do not provide framing structures. The bytestream format separates NAL units from each other by attaching a start code in front of each NAL unit. To avoid false detection of NAL unit boundaries, encoders run a byte-oriented start code emulation prevention algorithm, which adds an emulation prevention byte to the NAL unit payload if a start code would have occurred otherwise. In order to enable straightforward gateway operation between packet- and stream-oriented systems, start code emulation prevention may always be performed regardless of whether the bytestream format is in use or not. A NAL unit may be defined as a syntax structure containing an indication of the type of data to follow and bytes containing that data in the form of an RBSP interspersed as necessary with emulation prevention bytes. A raw byte sequence payload (RBSP) may be defined as a syntax structure containing an integer number of bytes that is encapsulated in a NAL unit. An RBSP is either empty or has the form of a string of data bits containing syntax elements followed by an RBSP stop bit and followed by zero or more subsequent bits equal to 0.

[0146] A bitstream may be defined to logically include a syntax structure, such as a NAL unit, when the syntax structure is transmitted along the bitstream but may be included in the bitstream according to the bitstream format. A bitstream may be defined to natively comprise a syntax structure, when the bitstream includes the syntax structure.

[0147] In some coding formats or standards, a bitstream may be in the form of a network abstraction layer (NAL) unit stream or a byte stream, that forms the representation of coded pictures and associated data forming one or more coded video sequences.

[0148] In some coding formats, such as AV1, a bitstream may comprise a sequence of open bitstream units (OBUs). An OBU comprises a header and a payload, wherein the header identifies a type of the OBU. Furthermore, the header may comprise a size of the payload in bytes.

[0149] In some coding standards, NAL units include a header and payload. In some coding standards, the NAL unit header indicates the type of the NAL unit. In some coding standards, the NAL unit header indicates a scalability layer identifier (e.g., called nu h_layer_id) , which may be used, e.g., for indicating spatial or quality layers, views of a multiview video, or auxiliary layers (such as depth maps or alpha planes). In some coding standards, the NAL unit header includes a temporal sublayer identifier, which may be used for indicating temporal subsets of the bitstream, such as a 30-frames-per-second subset of a 60-frames-per-second bitstream.

[0150] Bitstreams or coded video sequences may be encoded to be temporally scalable as follows. Each picture may be assigned to a particular temporal sub-layer. A temporal sub-layer may be equivalently called a sublayer, temporal sublayer, sublayer, or temporal level. Temporal sub-layers may be enumerated, e.g., from 0 upwards. The lowest temporal sub-layer, sub-layer 0, may be decoded independently. Pictures at temporal sublayer 1 may be predicted from reconstructed pictures at temporal sub-layers 0 and 1. Pictures at temporal sublayer may be predicted from reconstructed pictures at temporal sub-layers 0, 1, and 2, and so on. In other words, a picture at temporal sub-layer N does not use any picture at temporal sub-layer greater than N as a reference for inter prediction. The bitstream created by excluding all pictures greater than or equal to a selected sub-layer valueand including pictures remains conforming.

[0151] Each picture of a temporally scalable bitstream may be assigned with a temporal identifier (also known as TID, temporal layer identifier, sub-layer identifier, sublayer identifier, temporal sub-layer identifier, temporal sublayer identifier, or temporal layer ID), which may be, for example, assigned to a variable Temporalld. The temporal identifier may, for example, be indicated in a NAL unit header or in an OBU extension header. Temporalld equal to 0 corresponds to the lowest temporal level. The bitstream created by excluding all coded pictures having a Temporalld greater than or equal to a selected value and including all other coded pictures remains conforming. Consequently, a picture having Temporalld equal to tid_value does not use any picture having a Temporalld greater than tid_value as a prediction reference. In some video coding standards, a sub-layer or a temporal sublayer may be defined to be a temporal scalable layer (or a temporal layer, TL) of a temporal scalable bitstream, consisting of VCL NAL units with a particular value of the Temporalld variable and the associated non-VCL NAL units.

[0152] NAL units can be categorized into Video Coding Layer (VCL) NAL units and non-VCL NAL units. VCL NAL units are typically coded slice NAL units.

[0153] A non-VCL NAL unit may be for example one of the following types: a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a supplemental enhancement information (SEI) NAL unit, an access unit delimiter, an end of sequence (EOS) NAL unit, an end of bitstream (EOB) NAL unit, or a filler data NAL unit. Parameter sets may be needed for the reconstruction of decoded pictures, whereas many of the other non-VCL NAL units may not be necessary for the reconstruction of decoded sample values.

[0154] Some coding formats specify parameter sets that may carry parameter values needed for the decoding or reconstruction of decoded pictures. A parameter may be defined as a syntax element of a parameter set. A parameter set may be defined as a syntax structure that contains parameters and that can be referred to from or activated by another syntax structure, for example, using an identifier.

[0155] Instead of or in addition to parameter sets at different hierarchy levels (e.g., sequence and picture), video coding formats may include header syntax structures, such as a sequence header or a picture header.

[0156] A sequence header may precede any other data of the coded video sequence in the bitstream order. It may be allowed to repeat a sequence header in the bitstream, e.g., to provide a sequence header at a random access point.

[0157] A picture header may precede any coded video data for the picture in the bitstream order. A picture header may be interchangeably referred to as a frame header. Some video coding specifications may enable carriage of a picture header in a dedicated picture header NAL unit or a frame header OBU or alike. Some video coding specifications may enable carriage of a picture header in a NAL unit, OBU, or alike syntax structure that also contains coded picture data.

[0158] A coded picture may be defined as a coded representation of a picture.

[0159] A random access point may be defined as a location within a bitstream where decoding can be started.

[0160] A random access picture or a Random Access Point (RAP) picture may be defined as a picture thatserves as a random access point, i.e., as a picture where decoding can be started. In some contexts, the term random-access picture may be used interchangeably with the term RAP picture.

[0161] An intra random access point (IRAP) picture, when contained in a single-layer bitstream or an independent layer, may comprise only intra-coded image segments. Furthermore, an IRAP picture may constrain subsequence pictures in output order to be such that they can be correctly decoded without performing the decoding process of any pictures that precede the IRAP picture in decoding order.

[0162] Some coding standards or specifications, such as H.265 / HEVC, may use the NAL unit type of VCL NAL unit(s) of a picture to indicate a picture type. In H.266AA / C, the NAL unit type indicates a picture type when mixed VCL NAL unit types within a coded picture are disabled (pps_mixed_nalu_types_in_pic_flag is equal to 0 in the referenced PPS), while otherwise it indicates a subpicture type.

[0163] Some coding standards or specifications may indicate a picture type in a picture header or a frame header or alike.

[0164] In some coding formats, picture unit (PU) may be defined as a set of data units, such as NAL units, that are associated with each other, are consecutive in decoding order, and contain exactly one coded picture. For example, certain non-video-coding data units, such as non-VCL NAL units, may be next to coded video data units in decoding order and the respective picture unit may comprise both these non-video-coding data units and the video coding data units of a coded picture.

[0165] In some coding formats, an access unit (AU) may be defined as a set of NAL units that are associated with each other according to a specified classification rule, are consecutive in decoding order, and include at most one coded picture at any scalability layer (e.g., with any specific value of nuhjayerjd in some coding formats, such as HEVC or WC). In some coding formats, an access unit comprises one or more complete picture units. In some coding formats, in addition to including the VCL NAL units of a coded picture, an access unit may also include non-VCL NAL units associated with the coded picture. Said specified classification rule may, for example, associate pictures with the same output time or picture order count value into the same access unit.

[0166] In some coding formats, a coded video sequence (CVS) may be defined as a sequence of coded pictures in decoding order that is independently decodable and is followed by another coded video sequence or the end of the bitstream.

[0167] In some coding formats, such as AV1 , a coded video sequence comprises one or more temporal units. A temporal unit consists of a series of OBUs starting from a temporal delimiter, optional sequence headers, optional metadata OBUs, a sequence of one or more frame headers, each followed by zero or more tile group OBUs as well as optional padding OBUs. A temporal unit may be defined to comprise all the OBUs that are associated with a specific, distinct time instant. A temporal unit may comprise a temporal delimiter OBU, and all the OBUs that follow, up to but not including the next temporal delimiter. A temporal delimiter OBU may be defined as an indication that the following OBUs will have a different presentation / decoding time stamp from the one of the last frame prior to the temporal delimiter.

[0168] A coded layer video sequence (CLVS) may be defined as a sequence of pictures and associated other data within the same scalable layer (e.g., with the same value of nuhjayerjd) that is decodable independently ofother pictures in the same layer.

[0169] Output order may be defined as the order in which the decoded pictures are output by a decoder.

[0170] Some coding formats use a concept of picture order count (POC). A value of POC is derived for each picture and is non-decreasing with increasing picture position in output order. In some coding formats, an increasing value of POC indicates the output order of pictures within a single scalability layer and a single CVS. POC may be used in the decoding process for example for implicit scaling of motion vectors and for reference picture list initialization. Furthermore, POC may be used in the verification of output order conformance.

[0171] In video coding standards, a compliant bit stream must be able to be decoded by a hypothetical reference decoder that may be conceptually connected to the output of an encoder and consists of at least a predecoder buffer, a decoder and an output / display unit. This virtual decoder may be known as the hypothetical reference decoder (HRD) or the video buffering verifier (VBV). A stream is compliant if it can be decoded by the HRD without buffer overflow or, in some cases, underflow. Buffer overflow happens if more bits are to be placed into the buffer when it is full. Buffer underflow happens if some bits are not in the buffer when said bits are to be fetched from the buffer for decoding / playback. One of the motivations for the HRD is to avoid so-called evil bitstreams, which would consume such a large quantity of resources that practical decoder implementations would not be able to handle.

[0172] HRD models may include instantaneous decoding, while the input bitrate to the coded picture buffer (CPB) of HRD may be regarded as a constraint for the encoder and the bitstream on decoding rate of coded data and a requirement for decoders for the processing rate. An encoder may include a CPB as specified in the HRD for verifying and controlling that buffering constraints are obeyed in the encoding. A decoder implementation may also have a CPB that may but does not necessarily operate similarly or identically to the CPB specified for HRD.

[0173] A Decoded Picture Buffer (DPB) may be used in the encoder and / or in the decoder. There may be two reasons to buffer decoded pictures, for references in inter prediction and for reordering decoded pictures into output order. Some coding formats, such as HEVC, provide a great deal of flexibility for both reference picture marking and output reordering, separate buffers for reference picture buffering and output picture buffering may waste memory resources. Hence, the DPB may include a unified decoded picture buffering process for reference pictures and output reordering. A decoded picture may be removed from the DPB when it is no longer used as a reference and is not needed for output. An HRD may also include a DPB. DPBs of an HRD and a decoder implementation may but do not need to operate identically.

[0174] A decoder and / or an HRD may comprise a picture output process. The output process may be considered to be a process in which the decoder provides decoded and cropped pictures as the output of the decoding process. The output process may be a part of video coding standards, e.g., as a part of the hypothetical reference decoder specification. In output cropping, lines and / or columns of samples may be removed from decoded pictures according to a cropping rectangle to form output pictures. A cropped decoded picture may be defined as the result of cropping a decoded picture based on the conformance cropping window specified e.g., in the sequence parameter set that is referred to by the corresponding coded picture. Hence, it may be considered that the conformance cropping window specifies the cropping rectangle to form output pictures from decodedpictures.

[0175] Some video coding specifications enable metadata OBUs. A metadata OBU comprises a type field, which specifies the type of metadata. A metadata OBU may be understood to be similar to an SEI NAL unit or an SEI message.

[0176] Video coding specifications may enable the use of supplemental enhancement information (SEI) messages or alike. Some video coding specifications include SEI NAL units, and some video coding specifications contain both prefix SEI NAL units and suffix SEI NAL units, where the former type can start a picture unit or alike and the latter type can end a picture unit or alike. An SEI NAL unit contains one or more SEI messages, which are not required for the decoding of output pictures but may assist in related processes, such as picture output timing, post-processing of decoded pictures, rendering, error detection, error concealment, and resource reservation.

[0177] ITU-T Recommendation H.274, which is equivalent to ISO / IEC 23002-7, may be called "versatile supplemental enhancement information messages for coded video bitstreams" and be referred to as "versatile supplemental enhancement information" or SEI. The VSEI standard specifies the syntax and semantics of video usability information (VUI) parameters and supplemental enhancement information (SEI) messages. The VUI parameters and SEI messages defined in the VSEI standard are designed to be conveyed within coded video bitstreams in a manner specified in a video coding specification or to be conveyed by other means determined by the specifications for systems that make use of such coded video bitstreams. The VSEI standard is intended for use with WC coded video bitstreams, although it is drafted in a manner intended to be sufficiently generic that it may also be used with other types of coded video bitstreams.

[0178] Several SEI messages are specified in H.264 / AVC, H.265 / HEVC, H.266AA / C, and H.274A / SEI standards, and the user data SEI messages enable organizations and companies to specify SEI messages for their own use. The standards may contain the syntax and semantics for the specified SEI messages but a process for handling the messages in the recipient might not be defined. Consequently, encoders may be required to follow the standard specifying a SEI message when they create SEI message(s), and decoders might not be required to process SEI messages for output order conformance. One of the reasons to include the syntax and semantics of SEI messages in standards is to allow different system specifications to interpret the supplemental information identically and hence interoperate. It is intended that system specifications can require the use of particular SEI messages both in the encoding end and in the decoding end, and additionally the process for handling particular SEI messages in the recipient can be specified.

[0179] Neural-network post-filters

[0180] Version 3 of the VSEI standard includes the specification of the neural-network post-filter characteristics (NNPFC) and neural-network post-filter activation (NNPFA) supplemental enhancement information (SEI) messages. Extensions to NNPFC SEI message are being specified for version 4 of the VSEI standard.

[0181] The neural-network post-filter characteristics (NNPFC) SEI message specifies a neural network that may be used as a post-processing filter. The use of specified neural-network post-processing filters (NNPFs) for specific pictures is indicated with neural-network post-filter activation (NNPFA) SEI messages.

[0182] SEI processing order SEI message

[0183] A decoding system may be defined as an apparatus that performs decoding and may also perform post-processing of decoded video. A decoding system may comprise a video decoder. The post-processing may be comprised in the video decoder or may receive input from the video decoder.

[0184] An encoding system may be defined as an apparatus that performs encoding and may also perform pre-processing of video prior to encoding. An encoding system may comprise a video encoder. The pre-processing may be comprised in the video encoder or may produce output to be encoded by the video encoder.

[0185] As per JVET-AI2006, the SEI processing order SEI message may have the following syntax: <<<

[0186] Some parts of the semantics of the SEI processing order SEI message are described as follows:

[0187] The SEI processing order (SPO) SEI message carries information indicating the preferred processing order, as determined by the encoder (e.g., the content producer), for a group of types of SEI messages that may be present in a CVS.

[0188] Use of this SEI message requires the definition of the following:

[0189] - Two lists of payloadType values, SeiProcessingOrderSeiList and SpoProcessSeiList.

[0190] SeiProcessingOrderSeiList comprises the payloadType values of the SEI message types that are allowed to be indicated in the SPO SEI message.

[0191] SpoProcessList comprises the payloadType values of the SEI message types that specify processes. The payload type values in SpoProcessList are among those of SeiProcessingOrderSeiList.

[0192] The semantics of the SPO SEI message uses the concept of types of SEI messages. SEI messages that have different payloadType values are considered different types of SEI messages. Additionally, different SEI messages that have the same payloadType value but are differentiated by values of syntax elements in the SEI payload are considered different types of SEI messages. Such differentiation by values of syntax elements in the SEI payload is to be performed by comparing values sent using po_sei_prefix_data_bit[ i ][j ] syntax elements, when present, or values sent as SEI messages within a processing order nesting SEI message, when present. For example, neural-network post-filter characteristics (NNPFC) SEI messages can be differentiated by having different nnpfcjd values. The terms "SEI message type" and "type of an SEI message" may be used interchangeably.

[0193] When the i-th SEI message seiA in any SPO SEI message has po_sei_wrapping_flag[ i ] and po_sei_prefix_flag[ i ] both equal to 0, there shall be no other SEI message seiB included in the same SPO SEI message or in a different SPO SEI message in the current CVS for which all of the following are true:

[0194] - The value of po_sei_payload_type[ i ] of sei B is the same as that for seiA;

[0195] - The value of po_sei_wrapping_flag[ i ] of seiB is equal to O; and

[0196] - The value of po_sei_prefix_flag[ i ] of seiB is equal to 1;

[0197] When an SPO SEI message with a particular value of po_id is present in any access unit of a CVS, an SPO SEI message with that particular value of pojd shall be present in the first access unit of the CVS in decoding order. The number of SEI messages and the payloadType codes of the SEI messages indicated within each SPO SEI message with the same value of pojd persist in decoding order from the current access unit until the end of the CVS in output order.

[0198] The SPO SEI message can carry one or more SEI prefix indications of a particular payloadType. When present, each SEI prefix indication is a bit string that follows the SEI payload syntax of that value of payloadType and contains a number of complete syntax elements starting from the first syntax element in the SEI payload. These SEI prefix indications should provide sufficient information to determine the specific processing order for types of SEI messages having the same value of payloadType but a different preferred processing order.

[0199] po_id contains an identifying number to identify the SPO SEI message.

[0200] A processing chain consists of a list of types of SEI messages identified by an SPO SEI message in the preferred processing order indicated in the SPO SEI message.

[0201] Each type of SEI message in the processing chain indicated by an SPO SEI message is identified by the syntax elements po_sei_payload_type[ i ], po_sei_wrapping_flag[ i ], po_sei_processing_order[ i ] and, when present, po_num_bits_in_prefix_indication_minus1[ i ] and po_prefix_data_bit[ i ][j ].

[0202] An SEI message type is not required to belong to any processing chain and may belong to any number of processing chains identified by SPO SEI messages with different po_id values.

[0203] Each SEI message of an SEI message type identified within the SPO SEI message has the same persistence scope as if the SEI message was carried outside of the SPO SEI message and not identified within an SPO SEI message.

[0204] When an SEI message specifies a process and is not associated with a processing chain specified by any SPO SEI message, it is implicitly a processing chain by itself. Some standards, such as Rec. ITU-T H.266 | ISO / IEC 23090-3, have specified an implicit processing chain of a super-resolution NNPF followed by another NNPF. Implicitly specified processing chains are treated like processing chains specified by SPO SEI messages when selecting SEI messages to be applied.

[0205] Processing chains can be alternatives to each other, e.g., such that at most processing chain is chosen to be applied, or they can be complementary, e.g., such that more than one processing chain is chosen and applied separately, with each processing chain generating one output.

[0206] po_num_sei_messages_minus2 plus 2 indicates the number of types of SEI messages for which the preferred order of processing is indicated in the SPO SEI message.

[0207] po_sei_wrapping_flag[ i ] equal to 1 specifies that an SEI message that applies as the i-th SEI message type in the processing chain specified in this SPO SEI message, when present, is an SEI message that is included in a PON SEI message for which both of the following conditions are true:

[0208] - pon_target_po_id[j ] with any value of j is equal to pojd.

[0209] - There is a k-th loop entry in the processing order nesting SEI message such that the payloadType of the k-th nested SEI message is equal to po_sei_payload_type[ i ] and pon_processing_order[ k ] is equal to po_sei_processing_order[ i ].

[0210] po_sei_wrapping_flag[ i ] equal to 0 specifies that an SEI message that applies as the i-th SEI message type in the processing chain specified in this SPO SEI message, when present, is an SEI message that is not included in a PON SEI message and for which both of the following conditions are true:

[0211] - The payloadType of the SEI message is equal po_sei_payload_type[ i ]; and

[0212] - po_sei_prefix_flag[ i ] is equal to 0, or when po_sei_prefix_flag[ i ] is equal to 1, the payload of the SEI message starts with the values of po_sei_prefix_data_bit[ i ][ j ].

[0213] po_sei_wrapping_flag[ i ] equal to 1 enables SEI messages to be carried within the processing order nesting SEI message to prevent such SEI messages from being incorrectly interpreted by decoders that do not process the SPO SEI message. Thus, po_sei_wrapping_flag[ i ] equal to 1 is intended to be used when po_sei_wrapping_flag[ i ] equal to 0 can lead to unintended results being produced by such decoders.

[0214] po_sei_importance_flag[ i ] equal to 1 affects the derivation of PoSeiList, which is the list of SEI messages that a decoding system should process for a particular picture picA, as specified below.

[0215] po_sei_importance_flag[ i ] equal to 0 specifies that when the decoding system cannot interpret or does not support the functionality indicated by the i-th SEI message type, it shall ignore all data associated with the loop variable value of i and exclude the i-th SEI message type from the processing chain performed by the decodingsystem.

[0216] po_sei_processing_degree_flag[ i ] affects the derivation of PoSeiList as specified below.

[0217] PoSeiList is derived as follows:

[0218] - PoSeiList is initially empty.

[0219] - The following applies in a non-decreasing order of po_sei_processing_order[ i ] values for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1 , inclusive, unless terminated earlier as specified below:- When an SEI message seiA associated with the i-th SEI message type persists for picA, the following applies:- If the decoding system can interpret and supports the functionality indicated by seiA, seiA is added at the end of PoSeiList.- Otherwise, if po_sei_importance_flag[ i ] is equal to 1 and po_sei_processing_degree_flag[ i ] is equal to 0, the derivation of PoSeiList is terminated.- Otherwise, if po_sei_importance_flag[ i ] is equal to 1 and po_sei_processing_degree_flag[ i ] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, and the derivation of PoSeiList is terminated.

[0220] po_sei_payload_type[ i ] specifies the payloadType value of the i-th type of SEI message.

[0221] po_sei_prefix_flag [ i ] equal to 1 specifies that po_num_bits_in_prefix_indication_minus1 [ i ] and some po_sei_prefix_data_bit[ i ][j ] syntax elements are present. po_sei_prefix_flag[ i ] equal to 0 specifies that these syntax elements are not present.

[0222] The value of po_sei_payload_type[ i ] for each i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive, shall be equal to a value in SeiProcessingOrderSeiList.

[0223] When po_sei_payload_type[ i ] is equal to any value in SpoProcessSeiList, the i-th type of SEI message indicates a process.

[0224] spoPropertySeiList is set to consist of the payloadType values included in SeiProcessingOrderSeiList excluding the paylaodType values included in SpoProcessSeiList. When po_sei_payload_type[ i ] is equal to any value in spoPropertySeiList, the i-th type of SEI message indicates a property.

[0225] po_sei_processing_order[ i ] indicates the preferred order of processing of the i-th type of SEI message for which preferred processing order information is provided in the SPO SEI message. For any two different integer values of m and n, po_sei_processing_order[ m ] less than po_sei_processing_order[ n ] indicates that the type of SEI message associated with index m should be processed before the type of SEI message associated with index n, and po_sei_processing_order[ m ] equal to po_sei_processing_order[ n ] indicates that there is no preferred order of processing between the types of SEI messages associated with indexes m and n (e.g., they can indicate different properties that are both applicable at that stage, or one can indicate a property and the other can indicate a process).

[0226] For i greater than 0, po_sei_processing_order[ i ] shall be greater than or equal to po_sei_processing_order[ i - 1 ].

[0227] Let seiMsgA be an SEI message that applies as the i-th SEI message type in the processing chain specified in this SPO SEI message, persists for a particular picture picA, and is associated with po_sei_processing_order[ i ] equal to poValA.

[0228] Let seiMsgSet be a set of SEI messages that includes SEI messages for which all of the following conditions are true:- The SEI message applies as the k-th SEI message type in the processing chain specified in this SPO SEI message with any value of k less than i;- The SEI message persists for picA;- po_sei_processing_order[ k ] is less than poValA; and- The payloadType value of the SEI message is among the values included in SpoProcessSeiList.

[0229] In an example, let seiMsgSet be further constrained to consist of each SEI message for which all of the following conditions are true:- The SEI message applies as the k-th SEI message type in the processing chain specified in this SPO SEI message with any value of k less than i;- The SEI message persists for picA;- po_sei_processing_order[ k ] is less than poValA; and- The payloadType value of the SEI message is among the values included in SpoProcessSeiList.

[0230] The pictures to which the semantics of seiMsgA apply are specified as follows:- When seiMsgSet is non-empty, the semantics of seiMsgA apply to all the pictures generated by the process implied by the SEI message that has the greatest value of po_sei_processing_order[ k ] among the SEI messages in seiMsgSet;- Otherwise, the semantics of seiMsgA apply to picA.

[0231] When an NNPF process outputs more than one NNPF-generated picture, the semantics of an SEI message that follows the NNPF in the processing order apply to all these NNPF-generated pictures.

[0232] po_num_bits_in_prefix_indication_minus1 [ i ] and po_sei_prefix_data_bit[ i ][j ], when present, have the same semantics as the num_bits_in_prefix_indication_minus1 [ i ] and sei_prefix_data_bit[ i ][j ] syntax elements of the SEI prefix indication SEI message, with prefix_sei_payload_type replaced by po_sei_payload_type[ i ].

[0233] When more than one SPO SEI message with a particular value of pojd is present in a CVS, the values of po_num_sei_messages_minus2 and, for each value of i, the values of po_sei_wrapping_flag[ i ], po_sei_prefix_flag[ i ], po_sei_importance_flag[ i ], po_sei_payload_type[ i ], po_sei_processing_order[ i ] shall be the same as in the other SPO SEI messages in the CVS with the same value of pojd.

[0234] SpoProcessSeiList may comprise the payloadType values for any of the following example SEI messages (but may not be limited to them):- Film grain characteristics SEI message, which may be used for indicating film grain synthesis(FGS) in a processing chain;- Region-wise packing SEI message, which may be used for indicating unpacking of rectangular regions from a cropped decoded picture (potentially processed by earlier processing stages) to form an omnidirectional projected picture (e.g., of equirectangular projection or cubemap projection) in a processing chain;- Colour transform information SEI message, which may be used for colour transformation in a processing chain;- Neural-network post-filter activation (NNPFA) SEI message and / or neural-network post-filter characteristics (NNPFC) SEI message, which may be used for neural-network-based filtering in a processing chain; and / or- Display orientation SEI message, which may be used for rotating the picture by 90, 180, or 270 degrees anticlockwise and / or horizontal mirroring. If both rotating and mirroring are indicated, the mirroring precedes the rotation.

[0235] Processing order nesting SEI message

[0236] As per JVET-AI2006, the processing order nesting SEI message may have the following syntax (): <<

[0237] Some parts of the semantics of the processing order nesting SEI message are described as follows.

[0238] The processing order nesting (PON) SEI message includes one or more SEI messages that should be applied only as parts of the processing chain identified by an associated SEI processing order SEI message and should not be applied in a manner that would contradict with the processing chain identified by the associated SEI processing order SEI message.

[0239] Use of this SEI message requires the definition of the following:- The syntax structure of the container of SEI messages, sei_pon_nested_message( )

[0240] The SEI messages contained in a PON SEI message are referred to as PON-nested SEI messages.

[0241] An encoder can include multiple PON SEI messages in the same access unit. For example, a first PON SEI message in an access unit can contain a PON-nested SEI message that applies to multiple processing chainsand one or more other PON SEI messages in the same access unit that apply to a single processing chain.

[0242] It is a requirement of bitstream conformance that the semantics and effect of an SEI message that is not a PON-nested SEI message shall not depend on any PON-nested SEI message.

[0243] pon_num_po_ids_minus1 plus 1 specifies the number of the SEI processing order SEI messages SEI associated with this PON SEI message.

[0244] pon _target_po_id[ i ] indicates the po_id of the i-th SEI processing order SEI message associated with this PON SEI message.

[0245] pon_num_seis_minus1 plus 1 specifies the number of the PON-nested SEI messages that are included in this PON SEI message.

[0246] pon_processing_order[ i ] specifies the position of the i-th PON-nested SEI message within the processing order defined by the associated SEI processing order SEI message. When i is greater than 0, pon_processing_order[ i ] shall be greater than or equal to pon_processing_order[ i - 1 ].

[0247] An associated SEI processing order SEI message for the i-th PON-nested SEI message is an SEI processing order SEI message that has an entry k for which all of the following conditions are true:- po_sei_processing_order[ k ] is equal to pon_processing_order[ i ];- po_sei_payload_type[ k ] is equal to the payloadType value of the i-th PON-nested SEI message;and- When po_sei_prefix_flag[ k ] is equal to 1, po_sei_prefix_data_bit[ k ][ j ] for j in the range of 0 to po_num_bits_in_prefix_indication_minus1[ k ], inclusive, contain the same content as the po_num_bits_in_prefix_indication_minus1[ k ] plus 1 initial bits of the SEI message payload of the i-th PON-nested SEI message.

[0248] The i-th PON-nested SEI message may have any number of associated SEI processing order SEI messages in the range of 0 to pon_num_pojds_minus1 + 1, inclusive.

[0249] When the i-th PON-nested SEI message has an associated SEI processing order SEI message, the i-th PON-nested SEI message should be applied as the k-th loop entry of the associated SEI processing order SEI message.

[0250] The semantics of the i-th PON-nested SEI message applied as the k-th loop entry of the associated SEI processing order SEI message with a particular pojd value apply without considering any of the PON-nested SEI messages not associated with any SEI processing order SEI message with that particular pojd value.

[0251] For each SEI processing order SEI message that is present in the CVS and has pojd equal to pon_target_po Jd[ m ] for any value of m in the range of 0 to pon_num_pojds_minus1 , inclusive, there shall be at least one value n in the range of 0 to pon_num_seis_minus1, inclusive, for which the SEI processing order SEI message is the associated SEI processing order SEI message for the n-th PON-nested SEI message.

[0252] po processing degree flag] i ] of the SPO SEI message

[0253] The following combinations of po_seijmportance_flag[ i ] and po_sei_processing_degree_flag[ i ] are specified in JVET-AI2006 (e.g., with an informal summary of the semantics):

[0254] po_seijmportance_flag[ i ] equal to 0 and po_sei_processing_degree_flag[ i ] equal to 0 or 1. Optionalprocessing stage, e.g., the i-th processing stage can be skipped when the decoding system cannot interpret the i-th processing stage or does not support it;

[0255] po_sei_importance_flag[ i ] equal to 1 and po_sei_processing_degree_flag[ i ] equal to 0. When the decoding system cannot interpret the i-th processing stage or does not support it, the decoding system should apply the processing chain up to and excluding the i-th processing stage; and / or

[0256] po_sei_importance_flag[ i ] equal to 1 and po_sei_processing_degree_flag[ i ] equal to 1. When the decoding system cannot interpret the i-th processing stage or does not support it, the decoding system should not apply the processing chain.

[0257] FIGs. 5 to 8 present examples of processing chains specified by SPO SEI messages. In these figures, the output order of pictures is, for example, depicted using an arrow 501 and the processing order of SEI messages is, for example, depicted using an arrow 503. Dotted arrows and curly brackets indicate inputs that are used by a process of the processing chain to produce an output picture of that process. Rectangles indicate pictures and the numbers within the rectangles indicate the output order. Vertical alignment of rectangles and the same number within the rectangles both indicate that the corresponding pictures are versions of the same picture, wherein the versions may comprise the cropped decoded picture and / or picture(s) resulting from processes of the processing chain.

[0258] FIG. 5 represents an example of an SPO SEI message that specifies a cascade of three NNPFs. A picture rate upsampling NNPF 502, which interpolates one picture between a pair of cropped decoded pictures 504 (e.g., cropped decoded pictures 504a and 504b; 504c and 504d). A spatial upsampling NNPF 506 increases spatial resolution of the cropped decoded pictures (e.g., the cropped decoded pictures 504a - 504d) and the interpolated pictures (e.g., interpolated pictures 505a, 505b, and 505c). A quality enhancement NNPF 508, which takes three pictures (e.g., pictures 510a, 510b, and 510c; or pictures 510c, 51 Od, and 510e; or pictures 51 Oe, 510f, and 510g) as an input and enhances the quality of the midmost picture (e.g., the picture 510b, 510d, or 510f) to generate enhanced pictures (e.g., 512a, 512b, or 512c).

[0259] The example assumes that the spatial upsampling may cause the visual artefacts caused by the picture rate upsampling to be more visible and hence the content provider may want to allow spatial upsampling only in combination with quality enhancement. By setting the values of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ] as presented in FIG. 5, the content provider can indicate that either of the following processing chains is allowed:

[0260] Picture rate upsampling NNPF only; or

[0261] The entire processing chain with all the three NNPFs.

[0262] It is envisioned that it may be common to have a mandatory process at the end of a processing chain. For example, film grain synthesis may be the last processing step for professional video targeted for human viewing and color transform may be the last processing step for video targeted for machine analysis at RGB domain.

[0263] It is asserted that the current SPO SEI design does not enable to describe that two or more consecutive processes in a processing chain are to be processed or none of them are to be processed when there is any mandatory process following these consecutive processes. When a decoding system does not understand one ormore of these of these consecutive processes or does not support all of the functionality these consecutive processes, it should skip and ignore these processes but still process further processes in the processing chain.

[0264] FIG. 6 illustrates an example of a processing chain 600. In this example, cascaded NNPFs are either both to be processed or both to be ignored, followed by a mandatory film grain synthesis. The processing chain 600 includes following processes:- A picture rate upsampling NNPF 602, which interpolates one picture between a pair of cropped decoded pictures 604 (e.g., cropped decoded pictures 604a and 604b; 604c and 604d); and - Quality enhancement NNPF 606, which takes three pictures (e.g., pictures 604a, 605a, and 604b) as input and enhances the quality of the midmost picture (e.g., the picture 605a is enhanced to become 612a, the picture 605b is enhanced to become 612b, the picture 605c is enhanced to become 612c);- Film grain synthesis 608 adds film grain to pictures (e.g., resulting to pictures 610a, 610b, 610c, 610d, 610e, 610f, and 610g).

[0265] The example assumes that the picture rate upsampling causes visual artefacts and hence the content provider wants to allow picture rate upsampling only in combination with quality enhancement. The content provider also wants to allow skipping both NNPFs. Furthermore, film grain synthesis is always required. In other words, the content provider would like to express that either of the following processing chains is allowed:

[0266] Film grain synthesis only; or

[0267] The entire processing chain with both NNPFs and film grain synthesis.

[0268] It is asserted that the content provider cannot express the above-described allowed processing chains in a single SPO SEI message.

[0269] Various embodiments propose to specify a combination of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ] values to indicate a sub-chain. A sub-chain may be defined to include two or more processing stages of a processing chain such that either all processing stages marked as crucial should be processed or, when the decoding system cannot interpret or does not support a processing stage marked as crucial, none of the processing stages of the sub-chain should be processed.

[0270] In an embodiment, a sub-chain is defined to include the following processing stages in a non-decreasing order of po_sei_processing_order[ i ]:- The sub-chain starts with a crucial processing stage that has po_sei_importance_flag[ i ] is equal to 1 and po_sei_processing_degree_flag[ i ] is equal to 0;- The sub-chain may have zero or more optional processing stages (po_sei_importance_flag[ i ] is equal to 0 and po_sei_processing_degree_flag[ i ] is equal to 0) and zero or more crucial processing stages (po_sei_importance_flag[ i ] is equal to 1 and po_sei_processing_degree_flag[ i ] is equal to 1); and - The sub-chain ends with a crucial processing stage that has po_sei_importance_flag[ i ] is equal to 0 and po_sei_processing_degree_flag[ i ] is equal to 1.

[0271] In some examples or embodiments, terms crucial and essential may be used interchangeably For exmaple, a crucial processing stage can be interchangeably referred to as an essential processing stage.

[0272] As discussed above, SEI message types included in a processing chain specified by an SPO SEI message may correspond to processes or may describe properties. A processing stage may be defined as an SEI message type or the associated process or property included in a processing chain specified by an SPO SEI message. A processing stage may be interchangeably a processing step. In some cases, a processing stage or a processing step may be defined more restrictively to mean a process of a processing chain specified by an SPO SEI message.

[0273] FIG. 7 illustrates an example implementation for realizing the example of Error! Reference source not found., in accordance with an embodiment. The proposed usage of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ] for the example of FIG. 7 is explained as follows:The picture rate upsampling NNPF 702 starts a sub-chain (po_sei_importance_flag[ i ] is equal to 1 and po_sei_processing_degree_flag[ i ] is equal to 0).The quality enhancement NNPF 706 ends the sub-chain (po_sei_importance_flag[ i ] is equal to 0 and po_sei_processing_degree_flag[ i ] is equal to 1).When the decoding system cannot interpret or does not support at least one NNPF in the sub-chain, it should not apply the sub-chain.The film grain synthesis 708 should be processed (po_sei_importance_flag[ i ] is equal to 1 and po_sei_processing_degree_flag[ i ] is equal to 1); when the decoding system cannot interpret or does not support film grain synthesis, it should not apply this processing chain.

[0274] The embodiments presented in following sections may be used interchangeably and achieve the same outcomes. It is to be understood that embodiments may be formed by combining features from embodiments presented in these sections.

[0275] Set of example embodiments

[0276] In an embodiment, a decoding system derives a list of SEI messages, PoSeiList, to be applied to a picture picA as a processing chain with the following:- The decoding system includes all SEI message types indicated by an SPO SEI message into poSeiOder, which is a list of SEI message types, in a non-decreasing order of po_sei_processing_order[ i ] values.- The decoding system derives the values of poldx[ i ] such that poSeiOrder[ i ] is the poldx[ i ]-th SEI message type indicated by the SPO SEI message.- poSubChainldx[ i ], which is equal to 0 when poSeiOrder[ i ] is not in a sub-chain and equal to a non-zero sub-chain index otherwise, is derived by traversing poSeiOrder[ i ] in decreasing order of i by identifying the end and start of sub-chains from the respective value combinations of po_sei_importance_flag[ i ] and po_processing_degree_flag[ i ].- polgnoreFlag[ i ] is initialized to 0 for all values of i. Subsequently, poIgnoreFlag [ i ] is set equal to 1 for all SEI message types of a sub-chain when the decoding system cannot interpret or does not support the functionality of any crucial SEI message type of the sub-chain.- PoSeiList is initially set to be empty.- The following applies in increasing order of i for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive, unless terminated earlier as specified below:- Only those SEI messages that have polgnoreFlag[ i ] equal to 0 are considered for inclusion into PoSei List. When an SEI message seiA associated with the poldx[ i ]-th SEI message type persists for picA and poIgnoreFlag [ i ] is equal to 0, the following applies:- When the decoding system can interpret and supports the functionality indicated by seiA, seiA is added at the end of PoSeiList.- Otherwise, when the value combination of po_sei_importance_flag[ poldx[ i ] ] and po_sei_processing_degree_flag[ poldx[ i ] ] indicates the start of a subchain that lasts until the end of the processing chain, the derivation of PoSeiList is terminated.- Otherwise, when the value combination of po_sei_importance_flag[ poldx[ i ] ] and po_sei_processing_degree_flag[poldx[ i ]] indicates an essential SEI message type, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, and the derivation of PoSeiList is terminated.

[0277] In an embodiment, a decoding system derives a list of SEI messages, PoSeiList, to be applied to a picture picA as a processing chain with the following:

[0278] Let poSeiOrder be a list of SEI message types that includes SEI message types indicated by this SPO SEI message in a non-decreasing order of po_sei_processing_order[ i ] values for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1 , inclusive. In an embodiment, let poSeiOrder be further constrained so that it consists of all the SEI message types indicated by this SPO SEI message in a non-decreasing order of po_sei_processing_order[ i ] values for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive.

[0279] Let the values of poldx[ i ] be such that poSeiOrder] i ] is the poldx[ i ]-th SEI message type indicated by this SPO SEI message for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive.

[0280] The list poSubChainldx[ i ] for i ranging from 0 to po_num_sei_messages_minus2 + 1, inclusive, specifying the sub-chain index of poSeiOrder] i ], is derived as follows:poSubChainFlag = 0poSubChainPrevIdx = 0for( i = po_num_sei_messages_minus2 + 1; i >= 0; i- - ) {if( poSubChainFlag = = 0 && po_sei_importance_flag[ poldx[ i ] ] = = 0 && po_sei_processing_degree_flag[ poldx[ i ] ] = = 1 ) {poSubChainFlag = 1poSubChainPrevldx++poSubChainldx[ i ] = poSubChainFlag * poSubChainPrevIdxif( po_sei_importance_flag[ poldx[ i ] ] = = 1 &&po_sei_processing_degree_flag[ poldx[ i ] ] = = 0 )poSubChainFlag = 0

[0281] The list polgnoreFlag[ i ] for i ranging from 0 to po_num_sei_messages_minus2 + 1, inclusive, specifying whether poSeiOrder[ i ] is ignored in derivation of PoSei List, is derived by the following:- polgnoreFlag[ i ] is set equal to 0 for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive.- The following applies for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1 , inclusive:- When poSubChainldx[ i ] is greater than 0, po_sei_importance_flag[ poldx[ i ] ] is equal to 1 or po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 1, and the decoding system cannot interpret or does not support the functionality indicated by the poldx[ i ]-th SEI message type, polgnoreFlag[ j ] is set equal to 1 for all values of j such that poSubChainldx[ j ] is equal to poSubChainldx[ i ].

[0282] PoSeiList is derived as follows:- PoSeiList is initially empty.- The following applies in increasing order of i for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive, unless terminated earlier as specified below:- When an SEI message seiA associated with the poldx[ i ]-th SEI message type persists forpicA and polgnoreFlag[ i ] is equal to 0, the following applies:- When the decoding system can interpret and supports the functionality indicated by seiA, seiA is added at the end of PoSeiList.- Otherwise, when po_sei_importance_flag[ poldx[ i ] ] is equal to 1, po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 0, the derivation of PoSeiList is terminated.- Otherwise, when po_sei_importance_flag[ poldx[ i ] ] is equal to 1 and po_sei_processing_degree_flag[poldx[ i ]] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, and the derivation of PoSeiList is terminated.

[0283] In an alternative expression of the embodiment above, the list poSubChainldx[ i ] is derived as follows:

[0284] The list poSubChainldx[ i ] for i ranging from 0 to po_num_sei_messages_minus2 + 1, inclusive, specifying the sub-chain index of poSeiOrder[ i ], is derived as follows:- poSubChainFlag is set equal to 0.- poSubChainPrevIdx is set equal to 0.- The following ordered steps apply in decreasing order of i in the range of 0 topo_num_sei_messages_minus2 + 1, inclusive:- It is a requirement of bitstream conformance that when poSubChainFlag is equal to 1, po_sei_importance_flag[ poldx[ i ] ] shall not be equal to 0 and po_sei_processing_degree_flag[ poldx[ i ] ] shall not be equal to 1.- When poSubChainFlag is equal to 0, po_sei_importance_flag[ poldx[ i ] ] is equal to 0, and po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 1, poSubChainFlag is set equal to 1 and poSubChainPrevIdx is incremented by 1.- poSubChainldx[ i ] is set equal to poSubChainFlag * poSubChainPrevIdx.- When po_sei_importance_flag[ poldx[ i ] ] is equal to 1, and po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 0, poSubChainFlag is set equal to 0.

[0285] Another set of example embodiments

[0286] In an embodiment, a decoding system derives a list of SEI messages, PoSeiList, to be applied to a picture picA as a processing chain with the following:- The decoding system includes all SEI message types indicated by an SPO SEI message into poSeiOder, which is a list of SEI message types, in a non-decreasing order of po_sei_processing_order[ i ] values.- The decoding system derives the values of poldx[ i ] such that poSeiOrder[ i ] is the poldx[ i ]-th SEI message type indicated by the SEI processing order (SPO) SEI message.- PoSeiList is initially set to be empty.- poSubChainFlag, which indicates if the poSeiOrder[ i ] belongs to a sub-chain, is set equal to 0, poSubChainSeiList, which is a list of SEI messages of a current sub-chain, is set to be empty, and poSubChainSupportedFlag, which indicates if the decoding system supports all crucial SEI messages of the current sub-chain, is set equal to 1.- The following applies in increasing order of i for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive, unless terminated earlier as specified below:- When the value combination of po_sei_importance_flag[ poldx[ i ] ] and po_sei_processing_degree_flag[ poldx[ i ] ] indicates the start of a sub-chain, poSubChainFlag is set equal to 1.- When an SEI message seiA associated with the poldx[ i ]-th SEI message type persists for picA, the following applies:- When the decoding system can interpret and supports the functionality indicated by seiA, seiA is added at the end of PoSeiList (when poSubChainFlag is equal to 0) or at the end of PoSubChainSeiList (when poSubChainFlag is equal to 1).- Otherwise, when poSubChainFlag is equal to 1 and the values of po_sei_importance_flag[ poldx[ i ] ] and / or po_sei_processing_degree_flag[ poldx[ i ] ] indicate a crucial SEI message type, poSubChainSupportedFlag isset equal to 0.- Otherwise, when the value combination of po_sei_importance_flag[ poldx[ i ] ] and po_sei_processing_degree_flag[poldx[ i ]] indicates an essential SEI message type, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, and the derivation of PoSeiList is terminated.- When poSubChainFlag is equal to 1 , the following applies:- poSubChainEndFlag, which indicates if the i-th SEI message type in poSeiOrder is the end of a sub-chain, is derived as follows:- When i is equal to po_num_sei_messages_minus2 + 1, poSubChainEndFlag is set equal to 1.- Otherwise, when the value combination of po_sei_importance_flag[ poldx[ i ] ] and po_sei_processing_degree_flag[ poldx[ i ] ] indicates the end of a sub-chain, poSubChainEndFlag is set equal to 1. - Otherwise, when the value combination of po_sei_importance_flag[ poldx[ i + 1 ] ] and po_sei_processing_degree_flag[ poldx[ i + 1 ] ] indicates the start of a sub-chain and poSubChainSupportedFlag is equal to 1, poSubChainEndFlag is set equal to 1.- Otherwise, when the value combination of po_sei_importance_flag[ poldx[ i + 1 ] ] and po_sei_processing_degree_flag[ poldx[ i + 1 ] ] indicates the start of a sub-chain and poSubChainSupportedFlag is equal to 0, the derivation of PoSeiList is terminated.- Otherwise, poSubChainEndFlag is set equal to 0.- When poSubChainEndFlag is equal to 1 , the following ordered steps apply:- When poSubChainSupportedFlag is equal to 1 , the SEI messages in poSubChainSeiList are added at the end of PoSeiList in the same order as they appear in poSubChainSeiList.- poSubChainFlag is set equal to 0, poSubChainSeiList is set to be empty, and poSubChainSupportedFlag is set equal to 1.

[0287] In an embodiment, a decoding system derives a list of SEI messages, PoSeiList, to be applied to a picture picA as a processing chain with the following:

[0288] Let poSeiOrder be a list of SEI message types that includes SEI message types indicated by this SPO SEI message in a non-decreasing order of po_sei_processing_order[ i ] values for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive. In an example, let poSeiOrder be further constrained to consist of each SEI message type indicated by this SPO SEI message in a non-decreasing order of po_sei_processing_order[ i ] values for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive.

[0289] Let the values of poldx[ i ] be such that poSeiOrder[ i ] is the poldx[ i ]-th SEI message type indicated by this SPO SEI message for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive.

[0290] PoSeiList is derived as follows:- PoSeiList is initially empty.- poSubChainFlag is set equal to 0, poSubChainSeiList is set to be empty, and poSubChainSupportedFlag is set equal to 1.- The following applies in increasing order of i for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive, unless terminated earlier as specified below:- When po_sei_importance_flag[ i ] is equal to 1 and po_sei_processing_degree_flag[ i ] is equal to 0, poSubChainFlag is set equal to 1.- When an SEI message seiA associated with the poldx[ i ]-th SEI message type persists for picA, the following applies:- When the decoding system can interpret and supports the functionality indicated by seiA, seiA is added at the end of PoSeiList (if poSubChainFlag is equal to 0) or at the end of PoSubChainSeiList (otherwise).- Otherwise, when both of the following conditions are true, poSubChainSupportedFlag is set equal to 0.- poSubChainFlag is equal to 1.- po_sei_importance_flag[ poldx[ i ] ] is equal to 1 or po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 1.- Otherwise, when po_sei_importance_flag[ poldx[ i ] ] is equal to 1 and po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, and the derivation of PoSeiList is terminated.- When poSubChainFlag is equal to 1, the following applies:- poSubChainEndFlag is derived as follows:- When i is equal to po_num_sei_messages_minus2 + 1, poSubChainEndFlag is set equal to 1.- Otherwise, when po_sei_importance_flag[ poldx[ i ] ] is equal to 0 and po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 1, poSubChainEndFlag is set equal to 1.- Otherwise, when po_sei_importance_flag[ poldx[ i + 1 ] ] is equal to 1, po_sei_processing_degree_flag[ poldx[ i + 1 ] ] is equal to 0, and poSubChainSupportedFlag is equal to 1, poSubChainEndFlag is set equal to 1.- Otherwise, when po_sei_importance_flag[ poldx[ i + 1 ] ] is equal to 1, po_sei_processing_degree_flag[ poldx[ i + 1 ] ] is equal to 0, andpoSubChainSupportedFlag is equal to 0, the derivation of PoSeiList is terminated.- Otherwise, poSubChainEndFlag is set equal to 0.- When poSubChainEndFlag is equal to 1 , the following ordered steps apply:- When poSubChainSupportedFlag is equal to 1, the SEI messages in poSubChainSeiList are added at the end of PoSeiList in the same order as they appear in poSubChainSeiList.- poSubChainFlag is set equal to 0, poSubChainSeiList is set to be empty, and poSubChainSupportedFlag is set equal to 1.

[0291] Yet another set of embodiments

[0292] In an embodiment, a decoding system derives a list of SEI messages, PoSeiList, to be applied to a picture picA as a processing chain with the following steps:- The decoding system includes all SEI message types indicated by an SPO SEI message into poSeiOder, which is a list of SEI message types, in a non-decreasing order of po_sei_processing_order[ i ] values.- The decoding system derives the values of poldx[ i ] such that poSeiOrder[ i ] is the poldx[ i ]-th SEI message type indicated by the SPO SEI message.- poSubChainldx[ i ], which is equal to 0 if poSeiOrder[ i ] is not in a sub-chain and equal to a nonzero sub-chain index otherwise, is derived by traversing poSeiOrder[ i ] in decreasing order of i by identifying the end and start of sub-chains from the respective value combinations of po_sei_importance_flag[ i ] and po_processing_degree_flag[ i ].- PoSeiList is initially set to be empty.- poSubChainFlag, which indicates when the poSeiOrder[ i ] belongs to a sub-chain, is set equal to 0, poSubChainSeiList, which is a list of SEI messages of a current sub-chain, is set to be empty, and poSubChainPrevIdx, which indicates the index of the previous SEI message type in poSeiOrder, is set equal to 0.- The following applies in increasing order of i for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive, unless terminated earlier as specified below:- When an SEI message seiA associated with the poldx[ i ]-th SEI message type persists for picA, the following applies:- When the previous sub-chain has ended (e.g., when poSubChainPrevIdx is greater than 0 and poSubChainldx[ i ] is not equal to poSubChainPrevIdx), the following ordered steps apply:- When poSubChainSupportedFlag is equal to 1 , the SEI messages in poSubChainSeiList are added at the end of PoSeiList in the same order as they appear in poSubChainSeiList.- poSubChainSeiList is set to be empty.- When a sub-chain starts (i.e., when poSubChainldx[ i ] is greater than 0 and not equal to poSubChainPrevIdx), poSubChainSupportedFlag is set equal to 1.- poSubChainPrevIdx is set equal to poSubChainldx[ i ].- If seiA belongs to a sub-chain (i.e, poSubChainldx[ i ] is greater than 0), the following applies:- When the decoding system can interpret and supports the functionality indicated by seiA, seiA is added at the end of poSubChainSeiList.- Otherwise, when po_sei_importance_flag[ i ] is equal to 1 or po_sei_processing_degree_flag[ i ] is equal to 1, poSubChainSupportedFlag is set equal to 0.- Otherwise, when the decoding system can interpret and supports the functionality indicated by seiA, seiA is added at the end of PoSeiList.- Otherwise, when the value combination of po_sei_importance_flag[ poldx[ i ] ] and po_sei_processing_degree_flag[ poldx[ i ] ] indicates the start of a subchain that lasts until the end of the processing chain, the derivation of PoSeiList is terminated.- Otherwise, when the value combination of po_sei_importance_flag[ poldx[ i ] ] and po_sei_processing_degree_flag[poldx[ i ]] indicates an essential SEI message type, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, and the derivation of PoSeiList is terminated.- When poSubChainSeiList is non-empty and poSubChainSupportedFlag is equal to 1, the SEI messages in poSubChainSeiList are added at the end of PoSeiList in the same order as they appear in poSubChainSeiList.

[0293] In an embodiment, a decoding system derives a list of SEI messages, PoSeiList, to be applied to a picture picA as a processing chain with the following:

[0294] Let poSeiOrder be a list of SEI message types that includes the SEI message types indicated by this SPO SEI message in a non-decreasing order of po_sei_processing_order[ i ] values for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1 , inclusive. Let the values of poldx[ i ] be such that poSeiOrder[ i ] is the poldx[ i ]-th SEI message type indicated by this SPO SEI message for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive.

[0295] poSubChainldx[ i ] is derived as follows:- poSubChainFlag is set equal to 0.- poSubChainPrevIdx is set equal to 0.- The following ordered steps apply in decreasing order of i in the range of 0 topo_num_sei_messages_minus2 + 1, inclusive:- It is a requirement of bitstream conformance that when poSubChainFlag is equal to 1, po_sei_importance_flag[ poldx[ i ] ] shall not be equal to 0 and po_sei_processing_degree_flag[ poldx[ i ] ] shall not be equal to 1.- When poSubChainFlag is equal to 0, po_sei_importance_flag[ poldx[ i ] ] is equal to 0, and po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 1, poSubChainFlag is set equal to 1 and poSubChainPrevIdx is incremented by 1.- poSubChainldx[ i ] is set equal to poSubChainFlag * poSubChainPrevIdx.- When po_sei_importance_flag[ poldx[ i ] ] is equal to 1, and po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 0, poSubChainFlag is set equal to 0.

[0296] PoSeiList is derived as follows:- PoSeiList is initially empty.- poSubChainSeiList is initially empty.- poSubChainFlag is set equal to 0.- poSubChainPrevIdx is set equal to 0.- The following applies in increasing order of i for all values of i in the range of 0 to po_num_sei_messages_minus2 + 1, inclusive, unless terminated earlier as specified below:- When an SEI message seiA associated with the poldx[ i ]-th SEI message type persists for picA, the following applies:- When poSubChainPrevIdx is greater than 0 and poSubChainldx[ i ] is not equal to poSubChainPrevIdx, the following ordered steps apply:- When poSubChainSupportedFlag is equal to 1 , the SEI messages in poSubChainSeiList are added at the end of PoSeiList in the same order as they appear in poSubChainSeiList.- poSubChainSeiList is set to be empty.- When poSubChainldx[ i ] is greater than 0 and not equal to poSubChainPrevIdx, poSubChainSupportedFlag is set equal to 1.- poSubChainPrevIdx is set equal to poSubChainldx[ i ].- If poSubChainldx[ i ] is greater than 0, the following applies:- When the decoding system can interpret and supports the functionality indicated by seiA, seiA is added at the end of poSubChainSeiList.- Otherwise, when po_sei_importance_flag[ i ] is equal to 1 or po_sei_processing_degree_flag[ i ] is equal to 1, poSubChainSupportedFlag is set equal to 0.- Otherwise, when the decoding system can interpret and supports the functionality indicated by seiA, seiA is added at the end of PoSeiList.- Otherwise, when po_sei_importance_flag[ poldx[ i ] ] is equal to 1, po_sei_processing_degree_flag[ poldx[ i ] ] is equal to 0, the derivation of PoSeiList is terminated.- Otherwise, when po_sei_importance_flag[ poldx[ i ] ] is equal to 1 and po_sei_processing_degree_flag[poldx[ i ]] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, and the derivation of PoSeiList is terminated. - When poSubChainSeiList is non-empty and poSubChainSupportedFlag is equal to 1, the SEI messages in poSubChainSeiList are added at the end of PoSeiList in the same order as they appear in poSubChainSeiList.

[0297] Semantics

[0298] In an embodiment, the value combinations of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ] may, for example, be specified as follows:- po_sei_importance_flag[ i ] equal to 0 and po_sei_processing_degree_flag[ i ] equal to 0 indicates an optional SEI message type.- po_sei_importance_flag[ i ] equal to 0 and po_sei_processing_degree_flag[ i ] equal to 1 indicates the end of a sub-chain.- po_sei_importance_flag[ i ] equal to 1 and po_sei_processing_degree_flag[ i ] equal to 0 indicates the start of a sub-chain.- po_sei_importance_flag[ i ] equal to 1 and po_sei_processing_degree_flag[ i ] equal to 1 indicates an essential SEI message type (to process the sub-chain where it belongs (if any) or the entire processing chain (if it does not belong to any sub-chain).

[0299] In an embodiment, the semantics of the value combinations of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ] may, for example, be specified as follows:- When the value combination of po_sei Jmportance_flag[ i ] and po_sei_processing_degree_flag[ i ] indicates an optional SEI message type, and the decoding system cannot interpret or does not support the functionality indicated by the i-th SEI message type, it shall ignore all data associated with the loop variable value of i and exclude the i-th SEI message type from the processing chain performed by the decoding system.- When the value combination of po_sei Jmportance_flag[ i ] and po_sei_processing_degree_flag[ i ] indicates the start of a sub-chain, and the decoding system cannot interpret or does not support the functionality indicated by the i-th SEI message type:- When the next value combination of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ], in processing order, for either the start or end of a sub-chain indicates the start of a sub-chain, the decoding system shall ignore all data associated with the loop variable value of i and all following loop variable values in processing order and exclude the i-th SEI message type and all following SEImessage types in processing order from the processing chain performed by the decoding system.- Otherwise (the next value combination of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ], in processing order, for either the start or end of a sub-chain indicates the end of a sub-chain), the decoding system shall ignore all data associated with the loop variable value of i and all following loop variable values in processing order until the end of the sub-chain, inclusive, and exclude the i-th SEI message type and all following SEI message types in processing order until the end of the sub-chain, inclusive, from the processing chain performed by the decoding system.- When the value combination of po_sei Jmportance_flag[ i ] and po_sei_processing_degree_flag[ i ] indicates the end of a sub-chain, and the decoding system cannot interpret or does not support the functionality indicated by the i-th SEI message type:- The decoding system shall ignore all data associated with the loop variable value of i and all preceding loop variable values in processing order since the start of the subchain, inclusive, and exclude the i-th SEI message type and all preceding SEI message types in processing order since the start of the sub-chain, inclusive, from the processing chain performed by the decoding system.- When the value combination of po_sei Jmportance_flag[ i ] and po_sei_processing_degree_flag[ i ] indicates an essential SEI message type, and the decoding system cannot interpret or does not support the functionality indicated by the i-th SEI message type:- When the next value combination of po_sei_importance_flag[ i ] and po_sei_processing_degree_flag[ i ], in processing order, for either the start or end of a sub-chain indicates the end of a sub-chain, the decoding system shall ignore all data associated with all loop variable values in processing order since the start of the sub-chain until the end of the sub-chain, inclusive, and exclude all the SEI message types in processing order since the start of the sub-chain until the end of the subchain from the processing chain performed by the decoding system.- Otherwise, the processing chain specified by this SPO SEI message should not be performed.

[0300] Nested sub-chains

[0301] In an embodiment, sub-chains may be nested as follows: A sub-chain at the (i+1 )-th nesting level may be nested or included in a sub-chain at the i-th nesting level. A sub-chain at the i-th nesting level is associated with a sub-chain at the (i+1 )-th nesting level, when the start of the sub-chain at the (i+1 )-th nesting level coincides or follows the start of the sub-chain at the i-th nesting level and the end of the sub-chain at the (i+1 )-th nesting level coincides or precedes the end of the sub-chain at the i-th nesting level. A sub-chain at the i-th nesting level may comprise zero or more sub-chains at the (i+1 )-th nesting level and zero or more SEI message types not includedin the nested sub-chains at the (i+1)-th nesting level.

[0302] In an embodiment, an encoder indicates, in an SPO SEI message, if the i-th SEI message type belongs to a sub-chain and the nesting levels for the i-th SEI message type (when it belongs to a sub-chain). In an embodiment, a decoding system decodes, from an SPO SEI message, if the i-th SEI message type belongs to a sub-chain and the nesting levels for the i-th SEI message type (when it belongs to a sub-chain).

[0303] In an embodiment, when a decoding system cannot interpret or does not support the functionality indicated by any crucial SEI message of a sub-chain at i-the nesting level (hereafter, subChainA), the decoding system ignores all data associated with the sub-chain subChainA and exclude the sub-chain subChainA from the processing chain performed by the decoding system. If the sub-chain subChainA is nested within a sub-chain subChainB at the nesting level (i - 1), the decoding system should include the sub-chain subChainB without the sub-chain subChainA in the processing chain performed by the decoding system.

[0304] In an embodiment, when a decoding system cannot interpret or does not support the functionality indicated by any crucial SEI message of a sub-chain at i-the nesting level (hereafter, subChainA), the decoding system ignores all data associated with the sub-chain subChainA and all sub-chains that are levels less than i and comprise the sub-chain subChainA, and exclude all these sub-chains from the processing chain performed by the decoding system.

[0305] In an embodiment, a sub-chain start depth and a sub-chain end depth is indicated and / or decoded for the i-th SEI message type and may be referred to as po_sei_sub_chain_start_depth[ i ] and po_sei_sub_chain_end_depth[ i ], respectively, without loss of generality.

[0306] In an embodiment, the value combinations of po_sei_sub_chain_start_depth[ i ] and po_sei_sub_chain_end_depth[ i ] may, for example, be specified as follows, when currSubChainLevel is initialized to 0:- po_sei_sub_chain_start_depth[ i ] equal to 0 and po_sei_sub_chain_end_depth[ i ] equal to 0 indicates an optional SEI message type.- po_sei_sub_chain_start_depth[ i ] equal to sDepth that is greater than 0 indicates that the i-th SEI message is the start of sub-chain(s) at sDepth levels and currSubChainLevel is incremented by sDepth.- po_sei_sub_chain_end_depth[ i ] equal to eDepth that is greater than 0 indicates that the i-th SEI message is the end of sub-chain(s) at eDepth levels and currSubChainLevel is decremented by eDepth.- It may be required for bitstream conformance that the values of po_sei_sub_chain_start_depth[ i ] and po_sei_sub_chain_end_depth[ i ] shall be such that currSubChainLevel is greater than or equal to 0 for all values of i.- When currSubChainLevel is equal to 0, po_sei_sub_chain_start_depth[ i ] is equal to 1, and po_sei_sub_chain_end_depth[ i ] is equal to 1, the i-th SEI message type is essential. When the decoding system cannot interpret or does not support the functionality indicated by an essential SEI message, the processing chain specified by this SPO SEI message should not beperformed.

[0307] In an embodiment, the following syntax may be used within the SPO SEI message syntax to include po_sei_sub_chain_start_depth[ i ] and po_sei_sub_chain_end_depth[ i ]:<

[0308] FIG. 8 illustrates an example of a processing chain that includes nested sub-chains, in accordance with an embodiment. The processing chain includes following processes:- Picture rate upsampling NNPF 802, which interpolates one picture (e.g., picture 805a, 805b, and 805c) between a pair of cropped decoded pictures 804 (e.g., pair of cropped pictures 804a and 804b; and 804b and 804c; and 804c and 804d);- Quality enhancement NNPF 806, which takes one picture (e.g., 804a, 805a, 804b, 805b 804c , 805c, or 804d) as input and enhances it (e.g., to become 808a, 808b, 808c, 808d, 808e, 808f, or 808g); and- Film grain synthesis 810 adds film grain to pictures (e.g., resulting to pictures 812a, 812b, 812c, 8128d, 812e, 812f, or 812g).

[0309] The content provider expresses that any of the following processing chains is allowed:- Film grain synthesis only;- Quality enhancement NNPF, followed by film grain synthesis; or- Picture rate upsampling NNPF, followed by quality enhancement NNPF, followed by film grain synthesis.

[0310] The content provider may express that other combinations, such as picture rate upsampling, followed by film grain synthesis are disallowed or discouraged.

[0311] FIG. 8 also illustrates how to the content provider sets the values of po_sei_sub_chain_start_depth[ i ] and po_sei_sub_chain_end_depth[ i ], which are referred to as start_depth and endjdepth respectively in FIG. 8.

[0312] Open-ended sub-chains

[0313] In an embodiment, when a start of a sub-chain and an end of sub-chain are indicated by particular valuecombinations of po_importance_flag[ i ] and po_processing_degree_flag[ i ], the start of the first sub-chain subChainA in processing order that is followed by the start the next sub-chain prior to an end of a sub-chain is the start of open-ended sub-chains, which may be denoted to have the processing stage index poOpenEndedStartldx.

[0314] In an embodiment, when the decoding system cannot interpret or does not support the i-th SEI message type and the i-th SEI message type belongs to a sub-chain subChainB, the following applies: If the start of subChainB has processing stage index greater than or equal to poOpenEndedStartldx, the decoding system should not process subChainB or any of the processing stages following subChainB. Otherwise, the decoding system should not process subChainB.

[0315] In an embodiment, the i-th SEI message type is interpreted as follows: If i is less than poOpenEndedStartldx and there is no preceding start of a sub-chain without a matching end of a sub-chain, the i-th SEI message type is a mandatory processing stage. In this case, when the decoding system cannot interpret or does not support the i-th SEI message type, it should not apply the processing chain. Otherwise, if a sub-chain start is indicated with index startldx less than i and there is no sub-chain end with index greater than startldx and less than i, the i-th SEI message type belongs to a sub-chain. Otherwise, the embodiment may be realized with any of the following options: the i-th SEI message type forms a sub-chain of its own; the i-th SEI message type is a mandatory processing stage (i.e., when the decoding system cannot interpret or does not support the i-th SEI message type, it should not apply the processing chain); or it is disallowed for an encoder to create an SPO SEI message that falls under this otherwise condition.

[0316] In an embodiment, po_importance_flag[ i ] and po_processing_degree_flag[ i ] may be interpreted as follows:

[0317] In an embodiment, po_importance_flag[ i ] and po_processing_degree_flag[ i ] may be interpreted as follows:

[0318] In an embodiment, po_importance_flag[ i ] and po_processing_degree_flag[ i ] may be interpreted as follows:

[0319] In an embodiment, po_importance_flag[ i ] and po_processing_degree_flag[ i ] may be interpreted as follows:

[0320] The lists PoProcStgldx, indicating the processing stage indices of the SEI message types in theprocessing chain, and PoSeiTypeldx, indicating the SEI message type indices of the processing stages in the processing chain, are derived as follows:

[0321] - For each of the SEI message types of in the processing chain, the following applies in a non-decreasing order of the corresponding po_sei_processing_order[ i ] values, with j being set equal to 0 initially:PoProcStgldx[ i ] =jPoSeiTypeldx[ j ] = ij++Where PoProcStgldx[ i ] indicates the processing stage index of the i-th SEI message type in the processing chain, and PoSeiTypeldx[ j ] indicates the SEI message type index of the j-th processing stage in the processing chain.

[0322] In an embodiment, The list poSubChainldx[ j ] for j ranging from 0 to PoNumProcStgs - 1, inclusive, specifying the sub-chain index of the j-th processing stage of the processing chain, is derived as follows:poSubChainFlag = 0poSubChainPrevIdx = 0for( j = 0; j < PoNumProcStgs; j++ ) {idx = PoSeiTypeldx[ j ]if( po_sei Jmportance_flag[ idx ] = = 1 && po_sei_processing_degree_flag[ idx ] = = 1 ) { if( poSubChainFlag = = 0 )poSubChainldx[ j ] = 0elsepoSubChainldx[ j ] = poSubChainPrevIdx} else if( po_sei Jmportance_flag[ idx ] = = 0 && po_sei_processing_degree_flag[ idx ] = = 1 ) { poSubChainldx[ j ] = poSubChainPrevIdxpoSubChainFlag = 0} else if( po_sei Jmportance_flag[ idx ] = = 1 && po_sei_processing_degree_flag[ idx ] = = 0 ) { poSubChainPrevldx++poSubChainldx[ j ] = poSubChainPrevIdxpoSubChainFlag = 1} elsepoSubChainldx[ j ] = poSubChainFlag * poSubChainPrevIdx

[0323] In an embodiment, for a picture, the list PoSeiList, indicting the list of SEI messages that may be applied to the picture, the list PoSeiTypeList, indicating the SEI message type indices of the SEI messages that may be applied to the picture, and the variable PoNumSeiMsgs, indicating the number of SEI messages that may be applied to the picture, are derived as follows:PoSeiList is initially empty, and seiListldx and PoNumSeiMsgs are both initially set equal to 0.The following applies in increasing order of j for all values of j in the range of 0 to PoNumProcStgs - 1, inclusive, unless terminated earlier as specified below:When an SEI message seiA associated with the PoSeiTypeldx[ j ]-th SEI message type persists for picA, the following applies:If all of the following conditions are true, seiA is added at the end of PoSeiList, PoSeiTypel_ist[ seiListldx ] is set equal to PoSeiTypeldx[ j ], PoNumSeiMsgs is set equal to PoNumSeiMsgs + 1 , and seiListldx is set equal to seiListldx + 1 :The decoding system can interpret and supports the functionality indicated by seiA.Any of the following conditions is true:poSubChainldx[ j ] is equal to 0.The decoding system can interpret and supports the functionality indicated by all SEI message types for all values of k such that poSubChainldx[ k ] is equal to poSubChainldx[j ] and po_sei_importance_flag[ PoSeiTypeldx[ k ] ] + po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is greater than or equal to 1. Otherwise, if poSubChainldx[j ] is greater than 0 and there is no value of k such that poSubChainldx[ k ] is equal to poSubChainldx[ j ], po_sei_importance_flag[ PoSeiTypeldx[ k ] ] is equal to 0 and po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is equal to 1, the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.Otherwise, if poSubChainldx[ j ] is equal to 0, po_sei_importance_flag[ PoSeiTypeldx[ j ] ] is equal to 1 and po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, PoNumSeiMsgs is set equal to 0, and the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.

[0324] In an embodiment, the list poSubChainldx[ j ] for j ranging from 0 to PoNumProcStgs - 1, inclusive, specifying the sub-chain index of the j-th processing stage of the processing chain, and poSubChainOpenEndedFlag[ scldx], specifying whether the sub-chain with index scldx is open-ended, are derived as follows:poSubChainFlag = 0poSubChainPrevIdx = 0poOpenEndedFlag = 0 / * set to 1 at the first open-ended sub-chain 7for( j = 0; j < PoNumProcStgs; j++ ) {idx = PoSeiTypeldx[ j ]if( po_sei_importance_flag[ idx ] = = 1 && po_sei_processing_degree_flag[ idx ] = = 1 ) { if( poSubChainFlag = = 0 && poOpenEndedFlag = = 0 )poSubChainldx[ j ] = 0else if( poSubChainFlag = = 0 && poOpenEndedFlag = = 1 ) {poSubChainPrevldx++poSubChainldx[ j ] = poSubChainPrevIdx} elsepoSubChainldx[ j ] = poSubChainPrevIdx} else if( po_sei Jmportance_flag[ idx ] = = 0 && po_sei_processing_degree_flag[ idx ] = = 1 ) { poSubChainldx[ j ] = poSubChainPrevIdxpoSubChainFlag = 0} else if( po_sei Jmportance_flag[ idx ] = = 1 && po_sei_processing_degree_flag[ idx ] = = 0 ) { poSubChainPrevldx++poSubChainldx[ j ] = poSubChainPrevIdxpoSubChainFlag = 1poSubChainOpenEndedFlag[ poSubChainPrevIdx ] = 1for( k = j + 1, scEnd = 0; k < PoNumProcStgs && IscEnd; k++ ) {kldx = PoSeiTypeldx[ k ]if( po_sei_importance_flag[ kldx ] = = 1 && po_sei_processing_degree_flag[ kldx ] = = 0 ) scEnd = 1else if ( po_sei -importance Jlag[ kldx ] = = 0 &&po_sei_processing_degree_flag[ kldx ] = = 1 ) {scEnd = 1poSubChainOpenEndedFlag[ poSubChainPrevIdx ] = 0poOpenEndedFlag = poOpenEndedFlag | poSubChainOpenEndedFlag[ poSubChainPrevIdx ] } elsepoSubChainldx[ j ] = poSubChainFlag * poSubChainPrevIdx

[0325] In an embodiment, for a picture, the list PoSeiList, indicting the list of SEI messages that may be applied to the picture, the list PoSeiTypeList, indicating the SEI message type indices of the SEI messages that may be applied to the picture, and the variable PoNumSeiMsgs, indicating the number of SEI messages that may be applied to the picture, are derived as follows:PoSeiList is initially empty, and seiListldx and PoNumSeiMsgs are both initially set equal to 0.The following applies in increasing order of j for all values of j in the range of 0 to PoNumProcStgs - 1, inclusive, unless terminated earlier as specified below:When an SEI message seiA associated with the PoSeiTypeldx[ j ]-th SEI message type persists for picA, the following applies:If all of the following conditions are true, seiA is added at the end of PoSeiList, PoSeiTypeList[ seiListldx ] is set equal to PoSeiTypeldx[ j ], PoNumSeiMsgs is set equal to PoNumSeiMsgs + 1 , and seiListldx is set equal to seiListldx + 1 :The decoding system can interpret and supports the functionality indicated by seiA.Any of the following conditions is true:poSubChainldx[ j ] is equal to 0.The decoding system can interpret and supports the functionality indicated by all SEI message types for all values of k such that poSubChainldx[ k ] is equal to poSubChainldx[j ] and po_sei_importance_flag[ PoSeiTypeldx[ k ] ] + po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is greater than or equal to 1. Otherwise, if poSubChainldx[j ] is greater than 0 and poSubChainOpenEndedFlag[poSubChainldx[j ] ] is equal to 1, the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.Otherwise, if poSubChainldx[ j ] is equal to 0, po_sei_importance_flag[ PoSeiTypeldx[ j ] ] is equal to 1 and po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, PoNumSeiMsgs is set equal to 0, and the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.

[0326] In an embodiment, the list poSubChainldx[ j ] for j ranging from 0 to PoNumProcStgs - 1, inclusive, specifying the sub-chain index of the j-th processing stage of the processing chain, and poOpenEndedStartldx, specifying the processing stage index of the start of open-ended sub-chains, are derived as follows:poSubChainFlag = 0poSubChainPrevIdx = 0poOpenEndedStartldx = 255poPrevSubChainStartldx = 255for( j = 0; j < PoNumProcStgs; j++ ) {idx = PoSeiTypeldx[ j ]if( po_sei Jmportance_flag[ idx ] = = 1 && po_sei_processing_degree_flag[ idx ] = = 1 ) { if( poSubChainFlag = = 0 && j < poOpenEndedStartldx )poSubChainldx[ j ] = 0else if( poSubChainFlag = = 0 ) {poSubChainPrevldx++poSubChainldx[ j ] = poSubChainPrevIdx} elsepoSubChainldx[ j ] = poSubChainPrevIdx} else if( po_sei Jmportance_flag[ idx ] = = 0 && po_sei_processing_degree_flag[ idx ] = = 1 ) { poSubChainldx[ j ] = poSubChainPrevIdxpoSubChainFlag = 0} else if( po_sei Jmportance_flag[ idx ] = = 1&& po_sei_processing_degree_flag [ idx ] = = 0 ) {if( poSubChainFlag && poOpenEndedStartldx = = 255 )poOpenEndedStartldx = poPrevSubChainStartldxpoSubChainPrevldx++poSubChainldx[ j ] = poSubChainPrevIdxpoSubChainFlag = 1poPrevSubChainStartldx =j} elsepoSubChainldx[ j ] = poSubChainFlag * poSubChainPrevIdx

[0327] In an embodiment, for a picture, the list PoSei List, indicting the list of SEI messages that may be applied to the picture, the list PoSeiTypeList, indicating the SEI message type indices of the SEI messages that may be applied to the picture, and the variable PoNumSeiMsgs, indicating the number of SEI messages that may be applied to the picture, are derived as follows:PoSeiList is initially empty, and seiListldx and PoNumSeiMsgs are both initially set equal to 0.The following applies in increasing order of j for all values of j in the range of 0 to PoNumProcStgs - 1, inclusive, unless terminated earlier as specified below:When an SEI message seiA associated with the PoSeiTypeldx[ j ]-th SEI message type persists for picA, the following applies:If all of the following conditions are true, seiA is added at the end of PoSeiList, PoSeiTypeList[ seiListldx ] is set equal to PoSeiTypeldx[ j ], PoNumSeiMsgs is set equal to PoNumSeiMsgs + 1 , and seiListldx is set equal to seiListldx + 1 :The decoding system can interpret and supports the functionality indicated by seiA.Any of the following conditions is true:poSubChainldx[ j ] is equal to 0.The decoding system can interpret and supports the functionality indicated by all SEI message types for all values of k such that poSubChainldx[ k ] is equal to poSubChainldx[ j ] and po_sei_importance_flag[ PoSeiTypeldx[ k ] ] + po_sei_processing_degree_flag[ PoSeiTypeldx[ k ] ] is greater than or equal to 1. Otherwise, if poSubChainldx[ j ] is greater than 0 and j is greater than or equal to poOpenEndedStartldx, the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.Otherwise, if poSubChainldx[ j ] is equal to 0, po_sei_importance_flag[ PoSeiTypeldx[ j ] ] is equal to 1 and po_sei_processing_degree_flag[ PoSeiTypeldx[ j ] ] is equal to 1, the processing chain specified by this SPO SEI message should not be performed for picA, PoSeiList is set to be empty, PoNumSeiMsgs is set equal to 0, and the derivation of PoSeiList, PoSeiTypeList, and PoNumSeiMsgs is terminated.

[0328] Sub-chain nesting level

[0329] In an embodiment, sub-chains are associated with nesting levels determined from values of po_sei_importance_flag[ i ] and po_processing_degree_flag[ i ].

[0330] In an embodiment, when a start of a sub-chain is indicated by a particular value combination of po_sei Jmportance_flag[ i ] and po_processing_degree_flag[ i ], the nesting level is increased by one compared to the nesting level of the (i - 1 )-th processing stage.

[0331] In an embodiment, when an end of a sub-chain is indicated by a particular value combination of po_sei_importance_flag[ i ] and po_processing_degree_flag[ i ], the nesting level is decreased by one compared to the nesting level of the (i - 1)-th processing stage.

[0332] In an embodiment, a particular value combination of po_sei_importance_flag[ i ] and po_processing_degree_flag[ i ] is interpreted as follows: If the number of sub-chains started prior to the i-th processing stage is greater than the number of sub-chains ended prior to the i-th processing stage, the i-th SEI message type is a member of a sub-chain. Otherwise, the-i-th SEI message type does not belong to a sub-chain, is mandatory in the processing chain, and when the decoding system cannot interpret or does not support the i-th SEI message type, it should not apply the processing chain.

[0333] In an embodiment, when a decoding system cannot interpret or does not support one or more of the SEI message types in a sub-chain at a particular nesting level MA, the SEI message types of the sub-chain and all subsequent sub-chains in the processing order should not be processed up to and excluding the end of the processing chain, the next sub-chain with a nesting level less than or equal to MA, or a mandatory processing stage, whichever is the earliest in the processing order of the processing chain.

[0334] In an embodiment, po_importance_flag[ i ] and po_processing_degree_flag[ i ] may be interpreted as follows:

[0335] Sub-chain impact coverage

[0336] In an embodiment, sub-chains are associated with impact coverages determined from values of po_sei_importance_flag[ i ] and po_processing_degree_flag[ i ].

[0337] In an embodiment, when a decoding system cannot interpret or does not support one or more of the SEI message types in a sub-chain, the decoding should not process any processing stages within the impact coverage of the sub-chain.

[0338] In an embodiment, when a start of a sub-chain, as indicated by a particular value combination of po_sei_importance_flag[ i ] and po_processing_degree_flag[ i ], is followed by an end of a sub-chain prior to another start of a sub-chain in a processing order, the sub-chain impact coverage is determined to comprise the sub-chain.

[0339] In an embodiment, when a start of a sub-chain, as indicated by a particular value combination of po_sei Jmportance_flag[ i ] and po_processing_degree_flag[ i ], is not followed by an end of a sub-chain prior to another start of a sub-chain in a processing order, the sub-chain impact coverage is determined to comprise the processing stages from the start of the sub-chain until the end of the processing chain.

[0340] Association of optional processing stages

[0341] In an embodiment, an optional processing stage, as indicated by a particular value combination of po_sei_importance_flag[ i ] and po_processing_degree_flag[ i ], is associated with the same sub-chain as the previous non-optional processing stage. For example, if the (i-1 )-th SEI message type is the previous non-optional processing stage and the i-th SEI message type is an optional processing stage, the i-th SEI message type is associated with the sub-chain where the (i-1 )-th SEI message type belongs to (if any).

[0342] Other embodiments and generalizations

[0343] Instead of or in addition to the above-described embodiments for the semantics of po_sei Jmportance_flag[ i ] equal to 0 and po_processing_degree_flag[ i ] equal to 1 , any of the following semantics could be used:- In an embodiment, when po_sei_importance_flag[ i ] is equal to 0, po_processing_degree_flag[ i ] indicates a priority for processing. For example, the following semantics may be used:- po_sei_importance_flag[ i ] equal to 0 specifies that when the decoding system cannot interpret or does not support the functionality indicated by the i-th SEI message type, it shall ignore all data associated with the loop variable value of i and exclude the i-th SEI message type from the processing chain performed by the decoding system. When po_sei_importance_flag[ i ] is equal to 0,po_processing_degree_flag[ i ] is equal to 0, po_sei_importance_flag[ j ] is equal to 0, po_processing_degree_flag[ j ] is equal to 1, the decoding system can interpret or supports the functionality indicated by both the i-th and j-th SEI message types individually, and the decoding system does not support the functionality indicated by the i-th and j-th SEI message types together, e.g., due to computational complexity limitations, the decoding system should process the j-th SEI message type rather than the i-th SEI message type.- In an embodiment, when po_sei_importance_flag[ i ] is equal to 0, i is greater than 0 and po_sei_processing_order[ i ] is equal to po_sei_processing_order[ i - 1], po_processing_degree_flag[ i ] indicates whether the i-th processing stage is an alternative to ( i - 1 )-th processing stage or the i-th and ( i - 1 )-th processing stage can be performed in any order.

[0344] It is to be understood that the embodiments may be similarly realized with any other mapping of syntax element value combinations to the semantics and / or decoding system operation.

[0345] It is to be understood that instead of using two 1 -bit syntax elements (po_sei_importance_flag[ i ] and po_processing_degree_flag[ i ]), a single 2-bit syntax element may be used.

[0346] Embodiments and examples have been presented with reference to particular processing chains. It is to be understood that embodiments and examples are not limited to these particular processing chains. In an example embodiment, an encoding system determines a processing chain and encodes a description of the processing chain in an SPO SEI message, wherein the processing chain comprises the following in the processing order:- Zero or more crucial processing stages, which may be referred to as initial crucial processing stages - A sub-chainOne or more crucial processing stages, which may be referred to as trailing crucial processing stages

[0347] In an embodiment, an encoding system determines a sub-chain to include processing stages that are not expected to be supported by all decoding devices, whereas the initial and trailing crucial processing stages may be expected to be supported by most or all decoding devices.

[0348] In an embodiment, an encoding system determines SEI message types for initial crucial processing stages to include, but not necessarily limited to, one or more of the following:- Any SEI message(s) defining mapping of the spatial regions of a picture to another spatial arrangement.For example, the region-wise packing SEI message.- A colour transform SEI message, when all subsequent processing stages are applied in a different colour space than that used for the coded video. For example, subsequent processing may be performed in the RGB colour space, whereas the YCbCr colour space was used for the coded video.

[0349] In an embodiment, an encoding system determines SEI message types for a sub-chain to include, but not necessarily limited to, one or more of the following:SEI message(s), such as NNPFC and / or NNPFA SEI messages, controlling two or more processing stages on neural-network-based post-processing that are intended to be performed in cascade; or - A colour transform SEI message followed by at least one processing stage, such as neural-network post-filtering, that expects a different colour space as input than that used for the coded video, whereas the trailing crucial processing stages do not depend on the used colour space

[0350] In an embodiment, an encoding system determines SEI message types for trailing crucial processing stages to include, but not necessarily limited to, one or more of the following:- A colour transform SEI message, when the post-processed video is displayed or consumed by a machine task that expects a particular colour space different than that used in preceding; processing stages and / or for the coded video;- A display orientation SEI message; or- A film grain characteristics SEI message.

[0351] FIG. 9 is an example apparatus 900, which may be implemented in hardware, configured to implement the examples described herein. The apparatus 900 comprises at least one processor 902 (e.g., an FPGA and / or CPU), at least one memory 904 including computer program code 905, the computer program code 905 having instructions to carry out the methods described herein, wherein the at least one memory 904 and the computer program code 905 are configured to, with the at least one processor 902, cause the apparatus 900 to implement circuitry, a process, component, module, or function (implemented with control module 906) to implement the examples described herein, including using sub-chains for supplemental enhancement information processing order. Optionally included encoder 908 of the control module 906 implements encoding based on the examples described herein, and optionally included decoder 910 implements decoding based on the examples described herein. The at least one memory 904 may be a non-transitory memory, a transitory memory, a volatile memory (e.g. RAM), or a non-volatile memory (e.g., ROM).

[0352] The apparatus 900 includes a display and / or I / O interface 912, which includes user interface (Ul) circuitry and elements, that may be used to display features or a status of the methods described herein (e.g., as one of the methods is being performed or at a subsequent time), or to receive input from a user such as with using a keypad, camera, touchscreen, touch area, microphone, biometric recognition, one or more sensors, etc. The apparatus 900 includes one or more communication e.g. network (N / W) interfaces (l / F(s)) 914. The communication l / F(s) 914 may be wired and / or wireless and communicate over the Internet / other network(s) via any communication technique including via one or more links 916. The communication l / F(s) 914 may comprise one or more transmitters or one or more receivers.

[0353] The transceiver 918 comprises one or more transmitters 920 and one or more receivers 922. The transceiver 918 and / or communication l / F(s) 914 may comprise standard well-known components such as an amplifier, filter, frequency-converter, (de)modulator, and encoder / decoder circuitries and one or more antennas, such as antennas 924 used for communication over wireless link 926.

[0354] The control module 906 of the apparatus 900 comprises one of or both parts 906-1 and / or 906-2, whichmay be implemented in a number of ways. The control module 906 may be implemented in hardware as control module 906-1, such as being implemented as part of the at least one processor 902. The control module 906-1 may be implemented also as an integrated circuit or through other hardware such as a programmable gate array. In another example, the control module 906 may be implemented as control module 906-2, which is implemented as computer program code (having corresponding instructions) 905 and is executed by the at least one processor 902. For instance, the at least one memory 904 store instructions that, when executed by the at least one processor 902, cause the apparatus 900 to perform one or more of the operations as described herein. Furthermore, the at least one processor 902, the at least one memory 904, and example algorithms (e.g., as flowcharts and / or signaling diagrams), encoded as instructions, programs, or code, are means for causing performance of the operations described herein.

[0355] The apparatus 900 to implement the functionality of control module 906 may correspond to any of the apparatuses depicted herein. Alternatively, apparatus 900 and its elements may not correspond to any of the other apparatuses depicted herein, as apparatus 900 may be part of a self-organizing / optimizing network (SON) node or other node, such as a node in a cloud.

[0356] The apparatus 900 may also be distributed throughout the network including within and between apparatus 900 and any network element (such as a base station and / or terminal device and / or user equipment).

[0357] Interface 928 enables data communication and signaling between the various items of apparatus 900, as shown in FIG. 9. For example, the interface 928 may be one or more buses such as address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like. Computer program code (e.g. instructions) 905, including control module 906 may comprise object-oriented software configured to pass data or messages between objects within computer program code 905. The apparatus 900 need not comprise each of the features mentioned, or may comprise other features as well. The various components of apparatus 900 may at least partially reside in a housing 930, or a subset of the various components of apparatus 900 may at least partially be located in different housings, which different housings may include housing 930.

[0358] FIG. 10 shows a schematic representation of non-volatile memory media 1000a (e.g. computer / compact disc (CD) or digital versatile disc (DVD)) and 1000b (e.g. universal serial bus (USB) memory stick) and 1000c (e.g. cloud storage for downloading instructions and / or parameters 1002 or receiving emailed instructions and / or parameters 1002) storing instructions and / or parameters 1002 which when executed by a processor allows the processor to perform one or more of the operations of the methods described herein. Instructions and / or parameters 1002 may represent or correspond to a non-transitory computer readable medium.

[0359] FIG. 11 is an example method 1100 performed with an encoder, based on the examples described herein. At 1102, the method 1100 includes writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a sub-chain. At 1104, the method 1100 includes, wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed. At 1106, themethod 1100 includes defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain. At 1108, the method 1100 includes, wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value. At 1110, the method 1100 includes, wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value. At 1112, the method 1100 includes, wherein the subchain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

[0360] In an embodiment, a processing stage is described by an information message type in the processing order information message.

[0361] In an embodiment, the first flag equal to the second value and the second flag equal to the second value indicates an optional information message type.

[0362] In an embodiment, the first flag equal to the first value and the second flag equal to the first value indicates a crucial information message type. In some embodiments, crucial information message types may be different from crucial processing stages. For example, a start and end of sub-chains are crucial processing stages, whereas crucial information message types are crucial stages that neither start nor end of a sub-chain.

[0363] In an embodiment, the crucial information message type is crucial to the sub-chain when the crucial information message type belongs to the sub-chain or the entire processing chain when the crucial information message type does not belong to the sub-chain.

[0364] The method 1100 may be performed with an encoding apparatus, such as the apparatus 100, 900, or apparatuses depicted in FIG. 3 and FIG. 4, for example, the transmitting apparatus 404 with the encoder 402, or the apparatus 400 with the encoder 402.

[0365] FIG. 12 is an example method 1200 performed with a decoder, based on the example embodiments described herein. At 1202, the method 1200 includes decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain. At 1204, the method 1200 includes, wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when a decoding system cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the sub-chain are processed. At 1206, the method 1200 includes, decoding for a processing stage a first flag and a second flag. At 1208, the method 1200 includes, wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value. At 1210, the method 1200 includes, wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value. At 1212 the method 1200 includes, wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

[0366] In an embodiment, a processing stage is described by an information message type in the processing order information message.

[0367] In an embodiment, the first flag equal to the second value and the second flag equal to the second value indicates an optional information message type.

[0368] In an embodiment, the first flag equal to the first value and the second flag equal to the first value indicates a crucial information message type. In some embodiments, crucial information message types may be different from crucial processing stages. For example, a start and end of sub-chains are crucial processing stages, whereas crucial information message types are crucial stages that neither start nor end of a sub-chain.

[0369] In an embodiment, the crucial information message type is crucial to the sub-chain when the crucial information message type belongs to the sub-chain or the entire processing chain when the crucial information message type does not belong to the sub-chain.

[0370] The method 1200 may be performed with a decoding apparatus, such as the apparatus 100, 900, or apparatuses depicted in FIG. 3 and FIG. 4, for example, the receiving apparatus 408 with the decoder 410, or the apparatus 400 with the decoder 410.

[0371] FIG. 13 is another example method 1300 performed with an encoder, based on the examples described herein. At 1302, the method 1300 includes writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a sub-chain. At 1304, the method 1300 includes, wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed. At 1306, the method 1300 includes defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain. At 1308, the method 1300 includes, wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value. At 1310, the method 1300 includes, wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value. At 1312, the method 1300 includes, wherein the subchain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value. At 1314, the method 1300 includes, wherein when a start of the subchain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain in processing order that is followed by a start of the next sub-chain prior to the end of the sub-chain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

[0372] In an embodiment, a processing stage is described by an information message type in the processing order information message.

[0373] In an embodiment, the first flag equal to the second value and the second flag equal to the second value indicates an optional information message type.

[0374] In an embodiment, the first flag equal to the first value and the second flag equal to the first value indicates a crucial information message type. In some embodiments, crucial information message types may be different from crucial processing stages. For example, a start and end of sub-chains are crucial processing stages, whereas crucial information message types are crucial stages that neither start nor end of a sub-chain.

[0375] In an embodiment, the crucial information message type is crucial to the sub-chain when the crucial information message type belongs to the sub-chain or the entire processing chain when the crucial information message type does not belong to the sub-chain.

[0376] The method 1300 may be performed with an encoding apparatus, such as the apparatus 100, 900, or apparatuses depicted in FIG. 3 and FIG. 4, for example, the transmitting apparatus 404 with the encoder 402, or the apparatus 400 with the encoder 402.

[0377] FIG. 14 is another example method 1400 performed with a decoder, based on the example embodiments described herein. At 1402, the method 1400 includes decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain. At 1404, the method 1400 includes, wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when a decoding system cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the subchain are processed. At 1406, the method 1400 includes, decoding for a processing stage a first flag and a second flag. At 1408, the method 1400 includes, wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value. At 1410, the method 1400 includes, wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value. At 1412 the method 1400 includes, wherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value. At 1414 the method 1400 includes, wherein when a start of the sub-chain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain in processing order that is followed by a start of the next sub-chain prior to the end of the sub-chain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

[0378] In an embodiment, a processing stage is described by an information message type in the processing order information message.

[0379] In an embodiment, the first flag equal to the second value and the second flag equal to the second value indicates an optional information message type.

[0380] In an embodiment, the first flag equal to the first value and the second flag equal to the first value indicates a crucial information message type. In some embodiments, crucial information message types may be different from crucial processing stages. For example, a start and end of sub-chains are crucial processing stages, whereas crucial information message types are crucial stages that neither start nor end of a sub-chain.

[0381] In an embodiment, the crucial information message type is crucial to the sub-chain when the crucialinformation message type belongs to the sub-chain or the entire processing chain when the crucial information message type does not belong to the sub-chain.

[0382] The method 1400 may be performed with a decoding apparatus, such as the apparatus 100, 900, or apparatuses depicted in FIG. 3 and FIG. 4, for example, the receiving apparatus 408 with the decoder 410, or the apparatus 400 with the decoder 410.

[0383] As described above, FIG. 11 to 14 include flowcharts of an apparatus (e.g. 100, 400, 900, or any other apparatuses described herein), method, and computer program product according to certain example embodiments. It will be understood that each block of the flowcharts, and combinations of blocks in the flowcharts, may be implemented by various means, such as hardware, firmware, processor, circuitry, and / or other devices associated with execution of software including one or more computer program instructions. For example, one or more of the procedures described above may be embodied by computer program instructions. In this regard, the computer program instructions which embody the procedures described above may be stored by a memory (e.g.112 or 904) of an apparatus employing an embodiment of the present invention and executed by processing circuitry (e.g., 110 or 902) of the apparatus. As will be appreciated, any such computer program instructions may be loaded onto a computer or other programmable apparatus (e.g., hardware) to produce a machine, such that the resulting computer or other programmable apparatus implements the functions specified in the flowchart blocks. These computer program instructions may also be stored in a computer-readable memory that may direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture, the execution of which implements the function specified in the flowchart blocks. The computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operations to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide operations for implementing the functions specified in the flowchart blocks.

[0384] A computer program product is therefore defined in those instances in which the computer program instructions, such as computer-readable program code portions, are stored by at least one non-transitory computer-readable storage medium with the computer program instructions, such as the computer-readable program code portions, being configured, upon execution, to perform the functions described above, such as in conjunction with the flowchart(s) of FIGs. 11 to 14. In other embodiments, the computer program instructions, such as the computer-readable program code portions, need not be stored or otherwise embodied by a non-transitory computer-readable storage medium, but may, instead, be embodied by a transitory medium with the computer program instructions, such as the computer-readable program code portions, still being configured, upon execution, to perform the functions described above.

[0385] Accordingly, blocks of the flowcharts support combinations of means for performing the specified functions and combinations of operations for performing the specified functions for performing the specified functions. It will also be understood that one or more blocks of the flowcharts, and combinations of blocks in the flowcharts, may be implemented by special purpose hardware-based computer systems which perform thespecified functions, or combinations of special purpose hardware and computer instructions.

[0386] In some embodiments, certain ones of the operations above may be modified or further amplified. Furthermore, in some embodiments, additional optional operations may be included. Modifications, additions, or amplifications to the operations above may be performed in any order and in any combination.

[0387] In the above, some example embodiments have been described with the help of syntax of the bitstream. It needs to be understood, however, that the corresponding structure and / or computer program may reside at the encoder for generating the bitstream and / or at the decoder for decoding the bitstream.

[0388] In the above, where example embodiments have been described with reference to an encoder, it needs to be understood that the resulting bitstream and the decoder have corresponding elements in them. Likewise, where example embodiments have been described with reference to a decoder, it needs to be understood that the encoder has structure and / or computer program for generating the bitstream to be decoded by the decoder.

[0389] Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, although the foregoing descriptions and the associated drawings describe example embodiments in the context of certain example combinations of elements and / or functions, it should be appreciated that different combinations of elements and / or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and / or functions than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Accordingly, the description is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0390] It should be understood that the foregoing description is only illustrative. Various alternatives and modifications may be devised by those skilled in the art. For example, features recited in the various dependent claims could be combined with each other in any suitable combination(s). In addition, features from different embodiments described above could be selectively combined into a new embodiment. Accordingly, the description is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.

[0391] References to a ‘computer’, 'processor1, etc. should be understood to encompass not only computers having different architectures such as single / multi-processor architectures and sequential (Von Neumann) / parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGA), application specific circuits (ASIC), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device such as instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device, and the like.

[0392] As used herein, the term 'circuitry1may refer to any of the following: (a) hardware circuitimplementations, such as implementations in analog and / or digital circuitry, and (b) combinations of circuits and software (and / or firmware), such as (as applicable): (i) a combination of processor(s) or (ii) portions of processor(s) / software including digital signal processor(s), software, and memory(ies) that work together to cause an apparatus to perform various functions, and (c) circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even when the software or firmware is not physically present. This description of ‘circuitry’ applies to uses of this term in this application. As a further example, as used herein, the term ‘circuitry’ would also cover an implementation of merely a processor (or multiple processors) or a portion of a processor and its (or their) accompanying software and / or firmware. The term ‘circuitry’ would also cover, for example and when applicable to the particular element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or another network device.

[0393] Circuitry or Circuit: As used in this application, the term ‘circuitry’ or ‘circuit’ may refer to one or more or all of the following:(a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry); and(b) combinations of hardware circuits and software, such as (as applicable):(i) a combination of analog and / or digital hardware circuit(s) with software / firmware; and(ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions); and(c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0394] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example, and when applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

Claims

CLAIMSWhat is claimed is:

1. An apparatus comprising :at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform:writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a sub-chain;wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed;defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain;wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value;wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; andwherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

2. The apparatus of claim 1, wherein a processing stage is described by an information message type in the processing order information message.

3. The apparatus of claim 2, wherein the first flag equal to the second value and the second flag equal to the second value indicates an optional information message type.

4. The apparatus of any of the previous claims, wherein the first flag equal to the first value and the second flag equal to the first value indicates a crucial information message type.

5. The apparatus of claim 4, wherein the crucial information message type is crucial to the subchain when the crucial information message type belongs to the sub-chain or the entire processing chain when the crucial information message type does not belong to the sub-chain.

6. The apparatus of claim 3, wherein when the first flag equal to the second value and the second flag equal to the second value indicates the optional information message type and when a functionality indicated by the optional information message type cannot be interpreted or is not supported by the decoding system, data associated with a loop entry is ignored by the decoding system and the optional information message type is excluded from the processing chain performed by the decoding system, andwherein the processing chain is described with a loop, and wherein the loop entry describes a processing stage.

7. The apparatus of claim 1 , wherein when the first flag equal to the first value and the second flag equal to the second value indicates the start of the sub-chain and when a functionality indicated by the information message type cannot be interpreted or is not supported by the decoding system, one of the following applies:when a next value combination of the first flag and the second flag, in processing order, for either the start or end of the sub-chain indicates the start of the sub-chain, data associated with a loop entry and following loop entries in processing order are ignored by the decoding system and the information message type and a following information message types in processing order are excluded from the processing chain performed by the decoding system; orwhen the next value combination of the first flag and the second flag, in processing order, for either the start or end of the sub-chain indicates the end of a sub-chain:data associated with the loop entry and following loop entries in processing order until the end of the sub-chain is ignored by the decoding system; andthe information message type and following information message types in processing order until the end of the sub-chain are excluded from the processing chain performed by the decoding system; andwherein the processing chain is described with a loop, and wherein the loop entry describes a processing stage.

8. The apparatus of claim 1 , wherein when the first flag equal to the second value and the second flag equal to the first value indicates the end of the sub-chain and when a functionality indicated by the information message type cannot be interpreted or is not supported by the decoding system, data associated with a loop entry and preceding loop entries in processing order since the start of the subchain is ignored by the decoding system and the information message type and preceding information message types in processing order since the start of the sub-chain are excluded from the processing chain performed by the decoding system, and wherein the processing chain is described with a loop, and wherein the loop entry describes a processing stage.

9. The apparatus of claim 4, wherein when the first flag equal to the first value and the second flag equal to the first value indicates the crucial information message type and when a functionality indicated by the information message type cannot be interpreted or is not supported by the decoding system, one of the following applies:when the crucial information message type describes a processing stage of the sub-chain, the sub-chain is excluded from the processing chain performed by the decoding system; orwhen the crucial information message type describes a processing stage not in any sub-chain, the processing chain is not performed by the decoding system.

10. The apparatus of claim 1, wherein a sub-chain at a nesting level comprises zero or more subchains at a next nesting level and zero or more information message types not included in the zero or more sub-chains at next nesting level.

11. The apparatus of claim 10, wherein the apparatus is further caused to perform: indicating, in the processing order information message, whether an information message type belongs to the sub-chain and nesting levels for the information message type when the information message type belongs to the sub-chain.

12. The apparatus of claim 10 or 11 , wherein when a functionality indicated by the crucial information message of the sub-chain at a nesting level cannot be interpreted or is not supported by the decoding system, data associated with the sub-chain are excluded from the processing chain performed by the decoding system.

13. The apparatus of any of the claims 10 to 12, wherein a sub-chain start depth, and a sub-chain end depth is indicated and / or encoded in the information message type.

14. The apparatus of claim 13, wherein when a current sub chain level is initialized to zero, value combinations of the sub-chain start depth, and the sub-chain end depth are specified as following: the sub-chain start depth equal to the second value and the sub-chain end depth equal to the second value indicates an optional information message type;the sub-chain start depth equal to a start depth that is greater than the second value indicates that the information message is a start of the zero or more sub-chains at the start depth level and a current sub chain level is incremented by a value of the start depth;the sub-chain end depth equal to an end depth that is greater than the second value indicates that the information message is an end of the zero or more sub-chains at the end depth level and the current sub chain level is decremented by a value of the end depth;when the current sub-chain depth level is equal to the second value, the value of the sub-chain start depth is equal to the first value, the value of the sub-chain end depth is equal to the first value, and the information message type is crucial; and / orwhen the functionality indicated by the crucial information message cannot be interpreted or is not supported by the decoding system, the processing chain specified by the crucial information message is not be performed by the decoding system.

15. The apparatus of any of the claims 13 or 14, wherein values of the start sub-chain depth and the end sub-chain depth at current sub chain level is greater than or equal to the second value.

16. The apparatus of any of the previous claims, wherein when a start of the sub-chain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain in processing order that is followed by a start of the next sub-chain prior to the end of the sub-chain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

17. The apparatus of claim 16, wherein when an i-th information message type cannot be interpreted or not supported by the decoding system and the i-th information message type belongs to a second subchain, the following applies:when a start of the second sub-chain comprises a processing stage index greater than or equal to open ended start index, the second sub-chain and any of the processing stages following the second sub-chain are not processed by the decoding system; andwhen the start of the second sub-chain comprises the processing stage index less than open ended start index, the second sub-chain is not processed by the decoding system.

18. The apparatus of claim 17, wherein the i-th information message type is interpreted as following:when value of i is less than the open ended start index and there is no preceding start of a subchain without a matching end of the sub-chain, the i-th information message type is a mandatory processing stage, and wherein when the i-th information message type cannot be interpreted or supported by the decoding system, the processing chain specified by the i-th information message is not applied or processed by the decoding system;when a sub-chain start is indicated with a start index less than the value of i and there is no subchain end with index greater than start index and less than i, the i-th information message type belongs to a sub-chain; andwhen none of the above apples, the i-th information message type forms a sub-chain; the i-th information message type is a mandatory processing stage, wherein when the i-th information message type cannot be interpreted or supported by the decoding system, the processing chain specified by the i-th information message is not applied or processed by the decoding system; or the apparatus is disallowed from creating a processing order information message.

19. The apparatus of claim 16, wherein the first flag and the second flag are interpreted as following:when the first flag and the second flag comprise the second value, and wherein when the i-th information message type cannot be interpreted or supported by the decoding system, data associated with a loop variable value of i is ignored by the decoding system and the i-th information message type is excluded from the processing chain performed by the decoding system;when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain;when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts a sub-chain, and wherein the start of the first sub-chain in processing order that is followed by the start the next sub-chain prior to the end of first sub-chain is the start of open-ended sub-chains comprising the processing stage index as the open ended start index; and when the first flag comprises the first value and the second flag comprises the first value, and wherein when value of i is less than the open ended start index and there is no preceding start of a subchain without a matching end of the sub-chain, the i-th information message type is a mandatory processing stage, wherein when the i-th information message type cannot be interpreted or supported bythe decoding system, the processing chain specified by the i-th information message is not applied or processed by the decoding system; and wherein when a sub-chain start is indicated with a start index less than the value of i and there is no sub-chain end with index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and otherwise the i-th information message type forms a sub-chain.

20. The apparatus of claim 16, wherein the first flag and the second flag are interpreted as following:when the first flag and the second flag comprise the second value, the i-th information message is optional;when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain;when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts the sub-chain; andwhen the first flag comprises the first value and the second flag comprises the first value, and wherein when the i-th information message type precedes open-ended sub-chains, the processing chain specified by the i-th information message is a mandatory processing stage; and otherwise the i-th information message belongs to the sub-chain.

21. The apparatus of claim 16, wherein the first flag and the second flag are interpreted as following:when the first flag and the second flag comprise the second value, and wherein when the i-th information message type cannot be interpreted or supported by the decoding system, data associated with a loop variable value of i is ignored by the decoding system and the i-th information message type is excluded from the processing chain performed by the decoding system;when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain;when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts a sub-chain; andwhen the first flag comprises the first value and the second flag comprises the first value, and wherein when a sub-chain start is indicated with start index less than value of i and there is no sub-chain with end index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and wherein when the i-th information message type precedes open-ended sub-chains of a processing chain, the the processing chain specified by the i-th information message is a mandatory processing stage, and wherein when the i-th information message type cannot be interpreted or supported by the decoding system the processing chain specified by the i-th information message is not applied or processed by the decoding system; and otherwise, the i-th information message type forms a sub-chain.

22. The apparatus of any of the previous claims, wherein:the first value comprises one and the second value comprises zero; orthe first value comprises zero and the second value comprises one.

23. The apparatus of any of the previous claims, wherein the first flag comprises a processing ordersupplemental enhancement information importance flag and the second flag comprises a processing order processing degree flag.

24. An apparatus comprising:at least one processor; andat least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to perform:decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain;wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when the apparatus cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the sub-chain are processed;decoding for a processing stage a first flag and a second flag;wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value;wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; andwherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

25. The apparatus of claim 24, wherein a processing stage is described by an information message type in the processing order information message.

26. The apparatus of claim 25, wherein the first flag equal to the second value and the second flag equal to the second value indicates an optional information message type.

27. The apparatus of any of the claims 24 to 26, wherein the first flag equal to the first value and the second flag equal to the first value indicates a crucial information message type.

28. The apparatus of claim 27, wherein the crucial information message type is crucial to the subchain when the crucial information message type belongs to the sub-chain or the entire processing chain when the crucial information message type does not belong to the sub-chain.

29. The apparatus of claim 26, wherein when the first flag equal to the second value and the second flag equal to the second value indicates the optional information message type and when a functionality indicated by the optional information message type cannot be interpreted or is not supported by the apparatus, the apparatus is further caused to perform:ignoring data associated with a loop entry ;excluding the optional information message type from the processing chain performed; wherein the processing chain is described with a loop; andwherein the loop entry describes a processing stage.

30. The apparatus of claim 24, wherein when values of the first flag and the second flag indicate a crucial processing stage, when the crucial processing stage belongs to the sub-chain, and when a functionality indicated by the information message type cannot be interpreted or is not supported by the apparatus, the apparatus is further caused to perform: excluding the sub-chain from the processing chain.

31. The apparatus of claim 24, wherein when values of the first flag and the second flag indicate a crucial processing stage, when the crucial processing stage does not belong to any sub-chain, and when a functionality indicated by the information message type cannot be interpreted or is not supported by the apparatus, the apparatus is further caused to perform: omitting the processing chain.

32. The apparatus of claim 24, wherein a sub-chain at a nesting level comprises zero or more subchains at a next nesting level and zero or more information message types not included in the zero or more sub-chains at next nesting level.

33. The apparatus of claim 32, wherein the apparatus is further caused to perform: decoding, from the processing order information message, whether an information message type belongs to the subchain and nesting levels for the information message type when the information message type belongs to the sub-chain.

34. The apparatus of claim 32 or 33, wherein when a functionality indicated by the crucial information message of the sub-chain at a nesting level cannot be interpreted or is not supported by the apparatus, the apparatus is further caused to perform: excluding data associated with the sub-chain from the processing chain.

35. The apparatus of any of the claims 32 to 34, wherein a sub-chain start depth, and a sub-chain end depth is determined and / or decoded from the information message type.

36. The apparatus of claim 35, wherein when a current sub chain level is initialized to zero, value combinations of the sub-chain start depth, and the sub-chain end depth are specified as following: the sub-chain start depth equal to the second value and the sub-chain end depth equal to the second value indicates an optional information message type;the sub-chain start depth equal to a start depth that is greater than the second value indicates that the information message is a start of the zero or more sub-chains at the start depth level and a current sub chain level is incremented by a value of the start depth;the sub-chain end depth equal to an end depth that is greater than the second value indicates that the information message is an end of the zero or more sub-chains at the end depth level and the current sub chain level is decremented by a value of the end depth;when the current sub-chain depth level is equal to the second value, the value of the sub-chain start depth is equal to the first value, the value of the sub-chain end depth is equal to the first value, and the information message type is crucial; and / orwhen the functionality indicated by the crucial information message cannot be interpreted or is not supported by the apparatus is further caused to perform: omitting the processing chain specified bythe crucial information message.

37. The apparatus of any of the claims 35 or 36, wherein values of the start sub-chain depth and the end sub-chain depth at current sub chain level is greater than or equal to the second value.

38. The apparatus of any of the claims 24 to 37, wherein when a start of the sub-chain and an end of the sub-chain are indicated by particular value combinations of the first flag and the second flag, the start of a first sub-chain in processing order that is followed by a start of the next sub-chain prior to the end of the sub-chain is a start of open-ended sub-chains comprising a processing stage index of the start of the sub-chain as an open ended start index.

39. The apparatus of claim 38, wherein when the apparatus cannot interpret or support an i-th information message type and the i-th information message type belongs to a second sub-chain, the following applies:when a start of the second sub-chain comprises a processing stage index greater than or equal to open ended start index, the apparatus is further caused to perform: omitting the second sub-chain and any of the processing stages following the second sub-chain; andwhen the start of the second sub-chain comprises the processing stage index less than open ended start index, the apparatus is further caused to perform: omitting the second sub-chain.

40. The apparatus of claim 39, wherein the i-th information message type is interpreted as following:when value of i is less than the open ended start index and there is no preceding start of a subchain without a matching end of the sub-chain, the i-th information message type is a mandatory processing stage, and wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting the processing chain specified by the i-th information message;when a sub-chain start is indicated with a start index less than the value of i and there is no subchain end with index greater than start index and less than i, the i-th information message type belongs to a sub-chain; andwhen none of the above apples, the i-th information message type forms a sub-chain; the i-th information message type is a mandatory processing stage, wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting the processing chain specified by the i-th information message; or an encoder is disallowed from creating a processing order information message.

41. The apparatus of claim 38, wherein the first flag and the second flag are interpreted as following:when the first flag and the second flag comprise the second value, and wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting or ignoring the data associated with a loop variable value of i and excluding the i-th information message type from the processing chain;when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain;when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts a sub-chain, and wherein the start of the first sub-chain in processing order that is followed by the start the next sub-chain prior to the end of first sub-chain is the start of open-ended sub-chains comprising the processing stage index as the open ended start index; and when the first flag comprises the first value and the second flag comprises the first value, and wherein when value of i is less than the open ended start index and there is no preceding start of a subchain without a matching end of the sub-chain, the i-th information message type is a mandatory processing stage, wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting the processing chain specified by the i-th information message; and wherein when a sub-chain start is indicated with a start index less than the value of i and there is no sub-chain end with index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and otherwise the i-th information message type forms a sub-chain.

42. The apparatus of claim 38, wherein the first flag and the second flag are interpreted as following:when the first flag and the second flag comprise the second value, the i-th information message is optional;when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain;when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts the sub-chain; andwhen the first flag comprises the first value and the second flag comprises the first value, and wherein when the i-th information message type precedes open-ended sub-chains, the processing chain specified by the i-th information message is a mandatory processing stage; and otherwise the i-th information message belongs to the sub-chain.

43. The apparatus of claim 38, wherein the first flag and the second flag are interpreted as following:when the first flag and the second flag comprise the second value, and wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting or ignoring data associated with a loop variable value of i and excluding the i-th information message type from the processing chain;when the first flag comprises the second value and the second flag comprises the first value, the i-th information message ends a sub-chain;when the first flag comprises first value and the second flag comprises second value, the i-th information message type starts a sub-chain; andwhen the first flag comprises the first value and the second flag comprises the first value, and wherein when a sub-chain start is indicated with start index less than value of i and there is no sub-chain with end index greater than start index and less than i, the i-th information message type belongs to a sub-chain; and wherein when the i-th information message type precedes open-ended sub-chains of aprocessing chain, the the processing chain specified by the i-th information message is a mandatory processing stage, and wherein when the apparatus cannot interpret or support the i-th information message type, the apparatus is further caused to perform: omitting the the processing chain specified by the i-th information message; and otherwise, the i-th information message type forms a sub-chain.

44. The apparatus of any of the claims 24 to 43, wherein:the first value comprises one and the second value comprises zero; orthe first value comprises zero and the second value comprises one.

45. The apparatus of any of the claims 24 to 44, wherein the first flag comprises a processing order supplemental enhancement information importance flag and the second flag comprises a processing order processing degree flag.

46. The apparatus of claim 24, wherein when the apparatus cannot interpret the first crucial processing stage or the second crucial processing stage, the apparatus is further caused to perform: omitting the sub-chain.

47. A method comprising:writing, in or along a bitstream, a processing order information message to indicate a processing chain comprising a sub-chain;wherein the sub-chain comprises two or more processing stages of the processing chain such that either processing stages marked as crucial are processed, or when a processing stage marked as crucial cannot be interpreted or is not supported by a decoding system, none of the two or more processing stages of the sub-chain are processed;defining value combinations of a first flag and a second flag, in the processing order information message, to indicate the sub-chain;wherein the sub-chain starts with a first crucial processing stage indicated by the first flag being equal to a first value and the second flag being equal to a second value;wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; andwherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

48. A method comprising:decoding, from or along a bitstream, a processing order information message to determine a processing chain comprising a sub-chain;wherein the sub-chain comprises two or more processing stages of a processing chain such that either processing stages marked as crucial are processed, or when a decoding system cannot interpret or does not support a processing stage marked as crucial, none of the two or more processing stages of the sub-chain are processed;decoding for a processing stage a first flag and a second flag;wherein the sub-chain starts with the first crucial processing stage indicated by the first flag being equal to a first value and a second flag being equal to a second value;wherein the sub-chain further comprises zero or more crucial processing stages indicated by the first flag being equal to the first value and the second flag being equal to the first value and zero or more optional processing stages indicated by the first flag being equal to a second value and the second flag being equal to the second value; andwherein the sub-chain ends with a second crucial processing stage indicated by the first flag being equal to the second value and the second flag being equal to the first value.

49. A method comprising operations that implement operations as indicated as being performed by any of the apparatuses as claimed in any of claims 1 to 23.

50. A method comprising operations that implement operations as indicated as being performed by any of the apparatuses as claimed in any of claims 24 to 46.

Images