Authentication of data streams

Abstract

The present invention provides an apparatus and method for decoding data streams that should be checked for reliability. [Solution] An apparatus and method for checking the reliability of a data stream are described. According to one embodiment, one or more supplemental information messages are considered in the reliability of the data stream. According to another embodiment, the data stream indicates whether a segment is the first segment or whether a segment is the last segment in a sequence to which segments are assigned. According to a further embodiment, repetition of a particular supplemental information message within a particular part of the data stream is either not included in the reliability check or is prohibited within the data stream.

Description

Technical Field

[0001] Embodiments of the present invention relate to an apparatus and method for decoding a data stream to be checked for reliability. Further embodiments relate to an apparatus and method for making a data stream checkable for reliability. Further embodiments relate to a data stream checkable for reliability. In particular, embodiments relate to media data streams such as video data streams or audio data streams, and decoders and encoders for decoding or encoding such data streams respectively.

Background Art

[0002] Content authentication is important to avoid media tampering. The rapid progress of AI has produced sophisticated deepfakes, blurring the boundary between genuine and fake content, causing significant concerns regarding cybersecurity and copyright. Therefore, in recent years, it has become important to be able to verify the authenticity of media. An example of a method of performing such authentication consists of, or includes, providing a digital signature to the media by first hashing the media asset and then signing it with the secret key of the content generator, so that, on the client side, when the public key of the content generator is provided, the client can compare the provided signature with the value of a hash calculated independently based on the received media asset. If the values match, the client can safely assume that the media has not been tampered with.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The complex structure of data streams can complicate their authentication. This is particularly true in scenarios where a data stream is modified or generated by an entity, such as a scenario where a data stream consists of multiple substreams, or a scenario where an entity combines multiple data streams, or a scenario where a portion of a data stream is extracted. If this entity is not the entity that encoded the original data stream, and if the recalculation of the digital signature that would enable data stream verification is to be avoided, then a new solution is needed to enable reliable verification of the data stream. [Means for solving the problem]

[0004] An object of embodiments of the present invention is to provide a concept for checking data streams with respect to reliability, which offers an improved trade-off between high adaptability to complex streaming scenarios, low computational effort, and a low bitrate of signaling overhead required to provide the information necessary for authentication. This objective is achieved by the subject matter of the independent claim.

[0005] Embodiments of a first aspect of the present invention relate to the verification of supplemental information messages for a data stream, wherein the supplemental information messages provide supplemental information for the payload data carried in the payload packets of the data stream. The terms data stream verification and checking the reliability of a data stream are used interchangeably below and relate, for example, to the verification that the content of a data stream corresponds to the content provided by a content provider that has signed the content using a certificate. Including supplemental information messages in the data stream verification process may have the disadvantage that, in some scenarios, intermediate entities in the data stream transmission chain may want to drop some parts of the data stream. For example, an entity may drop supplemental information messages that are not required for the use case in which the intermediate entity prepares the data stream. On the other hand, the information carried by the supplemental information messages may affect the content of the decoded data, and as a result, an unauthorized modification of the supplemental information messages may go undetected if the supplemental information messages are excluded from the verification process.

[0006] Embodiments according to a first aspect of the present invention rely on the idea of ​​including an instruction in the data stream that indicates whether one or more supplemental information messages should be considered for verifying the data stream. Using this instruction, the receiver knows whether one or more supplemental information messages should be included in the verification process, and can therefore determine the portion of the data stream that underlies the verification according to the instruction. Thus, this instruction allows an encoder that makes the data stream checkable for reliability to choose whether one or more supplemental information messages should be included in the reliability check. Thus, a choice can be made between an improved level of verification by including supplemental information messages in the reliability check without having to recalculate the digital signature for the verification process, and greater flexibility in verifying the supplemental information messages at a later stage, for example, by an intermediate network entity.

[0007] An embodiment according to the first aspect provides a device for decoding a data stream, the data stream comprising a plurality of payload packets carrying payload data, further comprising supplemental information packets, the supplemental information packets carrying supplemental information messages. The device is configured to derive instructions from the data stream indicating whether one or more supplemental information messages should be considered to validate the data stream, to determine a given portion of the data stream by including one or more supplemental information messages in the given portion if the instructions indicate that one or more supplemental information messages should be considered to validate the data stream, and to obtain a digital signature from the instructions in the data stream to validate the given portion.

[0008] An embodiment according to the first aspect provides a device for decoding a data stream. The device is configured to check the data stream for reliability, and the data stream includes a plurality of payload packets carrying payload data, and further includes supplemental information packets, the supplemental information packets carrying supplemental information messages. The device is configured to derive an instruction from the data stream indicating whether supplemental information messages should be considered for validating the data stream; to determine a predetermined portion of the data stream that is to be validated, by including one or more supplemental information messages within the predetermined portion if the instruction indicates that supplemental information messages should be considered for validating the data stream; to obtain a digital signature based on the data stream; and to check whether the predetermined portion of the data stream conforms to the digital signature.

[0009] An embodiment according to the first aspect provides a device for encoding a data stream. The device is configured to make the data stream checkable for reliability, and to encode within the data stream a plurality of payload packets that carry payload data, further encoding supplemental information packets within the data stream, the supplemental information packets that carry supplemental information messages. The device is configured to insert instructions into the data stream indicating whether one or more supplemental information messages should be considered for verifying the data stream, to determine that a given portion of the data stream should be checkable for reliability by including one or more supplemental information messages within the given portion if the instructions indicate that one or more supplemental information messages should be considered for verifying the data stream, and to determine that the given portion should be checkable for reliability, and to obtain a digital signature based on the given portion.

[0010] Embodiments according to the first aspect provide a method for decoding a data stream, the data stream comprising a plurality of payload packets carrying payload data, further comprising supplemental information packets, the supplemental information packets carrying supplemental information messages, and the method is configured to derive an instruction from the data stream indicating whether one or more supplemental information messages should be considered for validating the data stream, determine a predetermined portion of the data stream by including one or more supplemental information messages in the predetermined portion if the instruction indicates that one or more supplemental information messages should be considered for validating the data stream, and obtain a digital signature from the instruction in the data stream for validating the predetermined portion.

[0011] Embodiments according to the first aspect provide a method for decoding a data stream, the method comprising checking the data stream for authenticity, the data stream comprising a plurality of payload packets carrying payload data, further comprising supplemental information packets, the supplemental information packets carrying supplemental information messages, the method comprising deriving an instruction from the data stream indicating whether the supplemental information messages should be considered for validating the data stream, determining a predetermined portion of the data stream which is a predetermined portion to be validated by including one or more of the supplemental information messages in the predetermined portion if the instruction indicates that the supplemental information messages should be considered for validating the data stream, obtaining a digital signature based on the data stream, and checking whether the predetermined portion of the data stream conforms to the digital signature.

[0012] Embodiments of the first aspect provide a method for encoding a data stream, the method comprising: making the data stream checkable for reliability; encoding a plurality of payload packets carrying payload data within the data stream, further encoding supplemental information packets within the data stream, the supplemental information packets carrying supplemental information messages, the method comprising: inserting instructions into the data stream indicating whether one or more supplemental information messages should be considered for verifying the data stream; determining a given portion of the data stream by including one or more supplemental information messages within the given portion if the instructions indicate that one or more supplemental information messages should be considered for verifying the data stream, and determining that the given portion should be checkable for reliability; and obtaining a digital signature based on the given portion (e.g., and inserting the digital signature into the data stream).

[0013] An embodiment according to the first aspect provides a data stream having internally encoded data signals, the data stream being checkable for reliability, the data stream comprising a plurality of payload packets carrying payload data, further comprising supplemental information packets, the supplemental information packets carrying supplemental information messages, and one or more supplemental information messages including instructions indicating whether the reliability of the data stream should be considered.

[0014] Embodiments according to a second aspect of the present invention rely on the idea of ​​including an indication in the data stream that indicates whether a segment of the temporal sequence of segments in a data stream is the first segment of an assigned subsequence of the temporal sequence, or, in a second alternative example of the second aspect, whether this segment is the last segment of an assigned subsequence of the temporal sequence. The indication that a segment is the first / last segment of an assigned subsequence is used to form a verification string that is verified using a digital signature. By signaling the start of a signed subsequence of a sequence of temporal segments, the receiver can recognize the start of a signed subsequence, and as a result, for example, in a scenario where two bitstreams are concatenated, the receiver can handle the first segment appropriately in a reliability check, i.e., whether the current segment is the first segment of a signed subsequence or not. For example, to verify the temporal consistency of a sequence of temporal segments, the verification of the current segment may include hash values ​​obtained from preceding segments so that a reliability check can verify whether the sequence of temporal segments in the sequence is genuine. If the receiver is unaware that the current segment is the first segment of a signed subsequence, it may include the hash value of a preceding segment in its reliability check, resulting in a negative outcome. In response to an indication that the current segment is the first segment of a signed subsequence, the receiver may perform a reliability check independently of the preceding segment. Thus, the indication that a segment is the first segment of a signed subsequence enables reliable verification of a concatenated data stream in which independently signed subsequences are concatenated. Note that the above example of a method for verifying the continuity of time segments in a data stream is merely illustrative, and alternative methods may be used to verify a sequence of signed segments in which the underlying concepts of embodiments of the second aspect are applicable in an equivalent manner.In a data stream, indicating that the current segment is the last segment of a signed subsequence ensures that the sequence cannot be shortened unless the shortening is recognized by the receiver. For example, if the data stream ends without the receiver receiving a time segment that is indicated as the last segment of a signed subsequence, the receiver will recognize that subsequent segments are missing.

[0015] An embodiment according to a second aspect provides a device for decoding a data stream, the data stream comprising a temporal sequence of segments. The device is configured to determine a predetermined portion of a segment in a sequence of segments, derive an instruction from the data stream indicating whether a segment is the first segment of a signed subsequence of the temporal sequence, or whether a segment is the last segment of a signed subsequence of the temporal sequence, and derive a digital signature from the data stream to be matched against a verification string obtained based on the predetermined portion and the instruction.

[0016] Embodiments of a second aspect provide a device for decoding a data stream, the device being configured to check the data stream for reliability, the data stream comprising a temporal sequence of segments. The device is configured to verify a segment of a sequence of segments by: applying a hash function to a predetermined portion of the segments to obtain a hash value; deriving an instruction from the data stream indicating whether a segment is the first segment of a signed subsequence of the temporal sequence, or whether a segment is the last segment of a signed subsequence of the temporal sequence; forming a verification string based on the hash value and the instruction; deriving a digital signature from the data stream and checking whether the verification string conforms to the digital signature.

[0017] An embodiment according to a second aspect provides a device for encoding a data stream. The device is configured to encode a temporal sequence of segments into the data stream, making the data stream trustworthy. The device is configured to make a segment of a sequence of segments trustworthy by having a hash function applied to a predetermined portion of the segment to obtain a hash value; inserting an instruction into the data stream indicating whether the segment is the first segment of a signed subsequence of the temporal sequence, or whether the segment is the last segment of a signed subsequence of the temporal sequence; forming a verification string based on the hash value and the instruction; and obtaining a digital signature based on the verification string.

[0018] Embodiments of a second aspect provide a method for decoding a data stream, wherein the data stream comprises a temporal sequence of segments, and the method includes: determining a predetermined portion of a segment in a sequence of segments; deriving an instruction from the data stream indicating whether a segment is the first segment of a signed subsequence of the temporal sequence, or whether a segment is the last segment of a signed subsequence of the temporal sequence; and deriving a digital signature from the data stream to be matched with a verification string obtained based on the predetermined portion and the instruction.

[0019] Embodiments of a second aspect provide a method for decoding a data stream, the method comprising checking the data stream for reliability, the data stream comprising a temporal sequence of segments, the method comprising: applying a hash function to a given portion of the segments to obtain a hash value; deriving an instruction from the data stream indicating whether a segment is the first segment of a signed subsequence of the temporal sequence, or whether a segment is the last segment of a signed subsequence of the temporal sequence; forming a verification string based on the hash value and the instruction; verifying a segment of the sequence of segments by deriving a digital signature from the data stream and checking whether the verification string conforms to the digital signature.

[0020] Embodiments of a second aspect provide a method for encoding a data stream, the method comprising making the data stream trustworthy and encoding a temporal sequence of segments within the data stream, the method comprising: obtaining a hash value by hashing a predetermined portion of the segments with a hash function; inserting an instruction into the data stream indicating whether the segment is the first segment of a signed subsequence of the temporal sequence or whether the segment is the last segment of a signed subsequence of the temporal sequence; forming a verification string based on the hash value and the instruction; and obtaining a digital signature based on the verification string (e.g., and inserting the digital signature into the data stream), thereby making the segments of the sequence of segments trustworthy.

[0021] Embodiments of a second aspect provide a data stream which is checkable for reliability and includes instructions indicating the temporal sequence of segments, whether a segment is the first segment of a signed subsequence of the temporal sequence, or whether a segment is the last segment of a signed subsequence of the temporal sequence.

[0022] A third embodiment of the present invention relies on the idea that, if the verification chain in the temporal sequence of segments is interrupted at a segment, a verification string is formed for a reliability check of the segment in the sequence of segments in the data stream, independent of the preceding segment immediately preceding the segment in the temporal sequence. For this purpose, a receiver of the data stream, such as a decoder, can derive from the data stream whether the verification chain has been interrupted at a segment. For example, interruption detection may be performed based on explicit signaling in the data stream, or it may be performed, for example, based on implicit signaling, by comparing signaled verification parameters for each segment in the temporal sequence. By detecting interruptions and forming verification strings independently of preceding segments, reliability checks become possible in scenarios where bitstream splicing occurs, i.e., when a sequence of jointly signed segments is interrupted by inserting an unsigned segment between two segments in a verification chain, or a segment belonging to a further verification chain, resulting in a segment in the sequence of segments in the data stream that does not belong to the same verification chain.

[0023] An embodiment according to a third aspect provides a device for decoding a data stream, the data stream comprising a temporal sequence of segments. The device is configured to derive from the data stream, for each segment in the sequence of segments, whether the verification chain within the temporal sequence of segments was interrupted at the segment, and, if the data stream indicates that the verification chain was interrupted and the data stream comprises a digital signature for checking the segment for trustworthiness, to derive from the data stream a digital signature to be checked against a verification string independent of the preceding segment immediately preceding the segment in the temporal sequence.

[0024] An embodiment according to a third aspect provides a device for decoding a data stream. The device is configured to check the data stream for reliability, and the data stream includes a temporal sequence of segments. The device is configured to derive from the data stream whether the verification chain in the temporal sequence of segments is interrupted at the segment, and if the data stream indicates that the verification chain is interrupted and the data stream includes a digital signature for the segment, to derive a digital signature from the data stream, to form a verification string independent of the preceding segment immediately preceding the segment in the temporal sequence, and to check whether the verification string conforms to the digital signature.

[0025] Embodiments according to the third aspect provide an apparatus for encoding a data stream. The apparatus is configured to make the data stream checkable for reliability and to encode a temporal sequence of segments in the data stream. The apparatus is configured to form a verification string independent of a preceding segment immediately before the segment in the temporal sequence when a verification chain in the temporal sequence of segments is interrupted by a segment and when the segment should be made checkable for reliability, and to obtain a digital signature based on the verification string, and to insert the digital signature into the data stream.

[0026] Embodiments according to the third aspect provide a method for decoding a data stream, the data stream including a temporal sequence of segments, the method including deriving, for segments of the sequence of segments from the data stream, whether a verification chain in the temporal sequence of segments is interrupted at the segment, and deriving, from the data stream, a digital signature to be checked against a verification string independent of a preceding segment immediately before the segment in the temporal sequence when the data stream indicates that the verification chain is interrupted and when the data stream includes a digital signature for checking the segment for reliability.

[0027] An embodiment according to the third aspect provides a method for decoding a data stream, the method including checking the data stream for reliability, the data stream including a temporal sequence of segments, the method comprising deriving from the data stream whether a verification chain within the temporal sequence of segments is interrupted by a segment, and, when the data stream indicates that the verification chain is interrupted and when the data stream includes a digital signature for a segment, deriving the digital signature from the data stream, forming a verification string independent of a preceding segment immediately before the segment in the temporal sequence, and checking whether the verification string conforms to the digital signature.

[0028] An embodiment according to the third aspect provides a method for encoding a data stream, the method including making the data stream checkable for reliability and encoding a temporal sequence of segments within the data stream, the method comprising forming a verification string independent of a preceding segment immediately before the segment in the temporal sequence when a verification chain within the temporal sequence of segments is interrupted by a segment and when the segment is to be made checkable for reliability, obtaining a digital signature based on the verification string, and inserting the digital signature into the data stream.

[0029] A fourth embodiment of the present invention relates to the handling of supplemental information messages, for example, the supplemental information messages described in relation to the first embodiment. The fourth embodiment relies on the finding that repetition of identical instances of a supplemental information message, which may be present in the data stream to enhance error resilience, can complicate the verification process because such multiple instances may be dropped when the data stream is received. In such cases, the data stream provided to the decoder may be different from the data stream used by the encoder to generate the digital signature, and as a result, the verification process may yield negative results. The fourth embodiment addresses this problem by including only one instance of multiple identical instances of a supplemental information message in a portion of the data stream, and a reliability check is performed on that portion. This process can be applied to all supplemental information messages to be considered for reliability checks, or only to a subset of these messages, for example, to a certain type of supplemental information message. By including only one instance of multiple instances, the result of forming a portion for reliability checks is the same regardless of whether one or more of the multiple instances were lost during transmission or dropped at the receiver, and as a result, the reliability check yields reliable results regardless of the redundant information present in the data stream.

[0030] An embodiment according to a fourth aspect provides a device for decoding a data stream. The device is configured to check the data stream for reliability, and the data stream includes a plurality of payload packets carrying payload data, and further includes supplemental information packets, the supplemental information packets carrying a plurality of supplemental information messages. The device is configured to determine a given portion of the data stream by checking whether a set of supplemental information messages among a plurality of supplemental information messages contains multiple instances of the same supplemental information message, and if a set of supplemental information messages contains a set of multiple instances of the same supplemental information message, then including only one instance of the same supplemental information message from the set of multiple instances. The device is further configured to obtain a digital signature based on the data stream and to check whether a given portion of the data stream conforms to the digital signature.

[0031] A fourth embodiment provides a device for decoding a data stream. The device is configured to make the data stream trustworthy, and to encode within the data stream a plurality of payload packets that carry payload data, further encoding within the data stream supplemental information packets that carry a plurality of supplemental information messages. The device is further configured to determine a predetermined portion of the data stream by checking whether a set of supplemental information messages among a plurality of supplemental information messages contains multiple instances of the same supplemental information message, and if the set of supplemental information messages contains a set of multiple instances of the same supplemental information message, to include only one instance of the same supplemental information message from the set of multiple instances. The device is further configured to obtain a digital signature based on the predetermined portion.

[0032] Embodiments according to a fourth aspect provide a method for decoding a data stream, the method comprising checking the data stream for authenticity, wherein the data stream comprises a plurality of payload packets carrying payload data, and further comprises supplemental information packets, the supplemental information packets carrying a plurality of supplemental information messages, and the method comprises determining a given portion of the data stream by checking whether a set of supplemental information messages among the plurality of supplemental information messages contains multiple instances of the same supplemental information message, and if the set of supplemental information messages contains a set of multiple instances of the same supplemental information message, then containing only one instance of the same supplemental information message from the set of multiple instances. The method further comprises obtaining a digital signature based on the data stream, and checking whether a given portion of the data stream conforms to the digital signature.

[0033] Embodiments according to a fourth aspect provide a method for encoding a data stream, the method comprising: making the data stream checkable for reliability; encoding a plurality of payload packets carrying payload data into the data stream; and further encoding a plurality of supplemental information packets carrying a plurality of supplemental information messages into the data stream. The method further comprises determining a given portion of the data stream by checking whether a set of supplemental information messages among a plurality of supplemental information messages contains multiple instances of the same supplemental information message; and, if the set of supplemental information messages contains a set of multiple instances of the same supplemental information message, including only one instance of the set of multiple instances of the same supplemental information message. The method further comprises obtaining a digital signature based on the given portion (e.g., further inserting a digital signature into the data stream).

[0034] Embodiments according to a fifth aspect of the present invention relate to the problem of supplemental information messages and the repetition of these messages, as described, for example, with respect to the fourth aspect. According to embodiments of the fifth aspect, the problem is addressed by providing a data stream such that the repetition of a given type of supplemental information message is prohibited within a given unit of the data stream. In other words, according to embodiments of the fifth aspect, a given unit of the data stream contains at most one identical instance of a given message type of supplemental information message. Thus, embodiments of the fifth aspect provide a simple solution that enables a reliable verification process without the need to check the data stream for repetition of supplemental information messages.

[0035] A fifth embodiment provides a device for encoding a data stream. The device is configured to encode a plurality of payload packets carrying payload data within a data stream, further encoding supplemental information packets within the data stream, so that the supplemental information packets carry supplemental information messages, and to make the data sequence checkable with respect to reliability. The device is further configured to provide a data stream such that each unit, which is a predetermined unit type of the data stream, contains at most one identical instance of a supplemental information message of a predetermined message type.

[0036] A fifth embodiment provides a device for decoding a data stream. The device is configured to decode a data sequence from a data stream, which includes a plurality of payload packets carrying payload data and supplemental information packets, wherein the supplemental information packets carry supplemental information messages, and to perform decoding and reliability checks on the data sequence. The device is configured to check whether each unit of a predetermined unit type in the data stream contains at most one identical instance of a supplemental information message of a predetermined message type.

[0037] Embodiments of a fifth aspect provide a method for decoding a data stream, the method comprising decoding a data sequence from a data stream, the sequence comprising a plurality of payload packets carrying payload data and supplemental information packets, wherein the supplemental information packets carry supplemental information messages, the method comprising decoding and checking the data sequence for reliability, the method comprising checking whether each unit of a predetermined unit type in the data stream contains at most one identical instance of a supplemental information message of a predetermined message type.

[0038] Embodiments of a fifth aspect provide a method for encoding a data stream, the method comprising encoding a data sequence in which a plurality of payload packets carrying payload data and further supplemental information packets are included, wherein the supplemental information packets carry supplemental information messages, and the method makes the data sequence checkable with respect to reliability. The method further comprises providing a data stream such that each unit, which is a predetermined unit type of the data stream, contains at most one identical instance of a supplemental information message of a predetermined message type.

[0039] A favorable embodiment is defined by the subject matter of the dependent claim. Embodiments of the present disclosure are described in more detail below with reference to the drawings. [Brief explanation of the drawing]

[0040] [Figure 1] This shows a device for decoding a data stream according to one embodiment. [Figure 2] This shows a verification module according to one embodiment. [Figure 3] This shows a device for encoding a data stream according to one embodiment. [Figure 4]An apparatus for decoding a data stream according to one embodiment of the first aspect is shown. [Figure 5] An embodiment of the first aspect is shown, which is a device for decoding a data stream having instructions for each type of supplemental information message to be verified. [Figure 6] An apparatus for decoding a data stream having individual instructions for supplemental information messages to be verified, according to one embodiment of the first aspect, is shown. [Figure 7] This demonstrates the verification of each substream of a data stream segment according to one embodiment. [Figure 8] An apparatus for decoding a data stream according to one embodiment of the second aspect is shown. [Figure 9] A second aspect of the present invention is shown, which is an apparatus for encoding a data stream. [Figure 10] A third embodiment of a device for decoding a data stream is shown. [Figure 11] A third embodiment of a device for encoding a data stream is shown. [Figure 12] A fourth aspect of the apparatus for decoding a data stream is shown. [Figure 13A] This indicates a repetition of the supplementary information message. [Figure 13B] This indicates a repetition of the supplementary information message. [Figure 14A] The verification string according to the second embodiment is shown. [Figure 14B] The verification string according to the second embodiment is shown. [Figure 15] A sequence of data stream segments according to one embodiment of the third aspect is shown. [Figure 16] A video encoder according to one embodiment is shown. [Figure 17] This shows a video decoder according to one embodiment. [Figure 18] This shows the subdivision of a picture according to one embodiment. [Modes for carrying out the invention]

[0041] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings, where the same or similar elements or elements having the same or similar functionality are assigned the same reference numeral or identified by the same name. Several details are provided in the following description to provide a complete description of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that other embodiments may be carried out without these specific details. In addition, features of the different embodiments described herein may be combined with each other unless otherwise specified.

[0042] In the following description, when referring to a data stream, embodiments of the present invention include data streams in which a media data stream in which a media signal is encoded, for example, a video data stream in which video is encoded, or an audio data stream in which an audio signal is encoded. Generally, embodiments of the present invention may be applied to any sequentially signaled data stream, for example, a sampled signal, for example, a sampled measurement signal in which a data stream is encoded. According to one embodiment, the data stream has a waveform signal in which a waveform signal is encoded, for example, a biomedical waveform signal. Some aspects of the present invention are described below with respect to video data streams, but these aspects may be applied to other types of data streams in an equivalent manner unless otherwise specified.

[0043] Figure 1 shows a device 20 for decoding a data stream 14. The device 20 may be referred to as a decoder 20. The data stream 14 may be a media data stream, such as a video data stream or an audio data stream, or any of the data stream types listed above. For example, the decoder 20 may decode a media signal, such as a video signal or an audio signal, from the data stream 14. The decoder 20 comprises an extractor 21. The extractor 21 obtains a digital signature 43 based on the data stream 14. According to one embodiment, the extractor 21 derives the digital signature 43 from the data stream 14. In other words, the digital signature 43 may be included in the data stream 14. According to an alternative embodiment, the data stream 14 includes a resource instruction containing the digital signature 43, and the extractor 21 uses the resource instruction to obtain the digital signature 43 from the resource indicated in the data stream 14. For example, digital signature 43 is for checking the authenticity of the data stream. In other words, digital signature 43 may be for verifying the data stream 14, for example, for verifying the authenticity of the data stream 14.

[0044] The decoder 20 further comprises a sub-determiner 30 configured to determine a predetermined portion 13 of the data stream 14 on which a reliability check, also hereafter referred to as a reliability check, should be performed. A digital signature 43 may be associated with the predetermined portion. In other words, the digital signature 43 may be for verifying the predetermined portion 13. For example, verification or reliability checks of the data stream 14 may be performed part by part, i.e., in units of portions of the data stream 14. Therefore, the digital signature 43 may be for checking the reliability of portions of the data stream, in other words, for verifying portions of the data stream 14. A reliability check, i.e., a reliability check of the data stream 14 or a predetermined portion 13, includes checking whether the predetermined portion of the data stream 14 conforms to or matches the digital signature 43.

[0045] According to one embodiment, the decoder 20 includes a verification module 41 configured to perform a reliability check. As shown in Figure 1, the verification module 41 receives a digital signature 43 and a predetermined portion 13 and performs a reliability check. The verification module 41 is an optional feature of the device 20, as shown by the dashed line in Figure 1. In other words, according to one embodiment, the decoder 20 is configured to check a predetermined portion 13 of the data stream 14 for reliability. In other words, according to one embodiment, the decoder 20 is configured to check whether a predetermined portion 13 of the data stream 14 conforms to or matches a digital signature.

[0046] As already stated, the verification module 41 is optional in the device 20. According to alternative embodiments, the reliability check may be performed externally to the decoder 20. In other words, the verification module 41 does not necessarily have to be part of the decoder 20, but may be part of a separate entity, such as a device for checking the reliability of the data stream 14. For example, in this case, the decoder 20 may provide or transfer the digital signature 43 and a predetermined portion 13 for the reliability check. For example, the decoder 20 may collect information for the reliability check, such as the digital signature 43 and a predetermined portion 13, and provide the collected information for the reliability check. For example, the decoder 20 may form a concatenation of information for the reliability check. These alternative embodiments in which the verification module 41 is not part of the device 20 may be embodied in combination with all embodiments of all aspects of the present invention.

[0047] Further optional features of the apparatus 20 of Figure 1, which may be applicable to all embodiments described herein, are described below. For example, the data stream 14 may contain multiple packets. The partial decision-maker 30 may determine a predetermined portion 13 for each packet, for example, by including one or more of the multiple packets. For example, optionally, a data stream 14 may contain multiple payload packets 16. A payload packet 16 may carry payload data, for example, data to be transmitted by the data stream 14. For example, if the data stream 14 is a media data stream, the payload packet 16 may carry coded media data, such as video data if the data stream 14 is a video data stream. In other words, a payload packet 16 may be a video coding layer (VCL) or network abstraction layer (NAL) unit carrying coded video payload packets, for example, video data, for example, encoded video data. For example, the video data may refer to information from which sample values ​​of video pictures are reconstructed.

[0048] The data stream 16 may optionally further include supplemental information packets 18, which may be alternatively referred to as supplemental information payload packets, in contrast to coded data payload packets that carry coded data. For example, in the case of a video data stream, particularly if H.264, H.265, and H.266 can format the video data stream, the supplemental information packets may be Supplemental Extended Information (SEI)NAL units. For example, the supplemental information packets 18 are interspersed among the payload packets 16. The supplemental information packets 18 carry supplemental information messages 19. For example, the supplemental information messages 19 assist processes related to decoding, displaying, or other purposes, but are not required by the decoding process to determine the values ​​of samples in the decoded picture of the video. In other words, for example, supplemental information packets may carry information about encoding options and information for the decoding process, but they do not contain encoded samples of the signal encoded within the data stream.

[0049] For example, each supplementary information message may be associated with one of the payload packets. For example, each supplemental information message is associated with one of the payload packets, which is, for example, part of the same sample of the sampled signal encoded within the data stream, for example, the same picture unit or the same access unit as the supplemental information packet.

[0050] As described below with respect to embodiments of the first, fourth, and fifth aspects, a predetermined portion 13 may optionally include one or more supplemental information messages that may be contained in one or more supplemental information packages. In other words, a predetermined portion 13 may optionally include one or more supplemental information packets.

[0051] For example, a predetermined portion 13 includes all or a subset of the payload packets 16 of a segment of the data stream 14. For example, a predetermined portion 13 may include portions of the payload packets 16, for example, each portion of the payload packets that should be included in the predetermined portion, or it may include the entire payload packet. The segment of the data stream 14 from which a predetermined portion 13 can be determined may be, for example, an independently coded sequence of the data stream 14, for example, a coded video sequence (CVS) in the case of a video data stream. In other examples, the segment from which a predetermined portion 13 can be determined may be indicated in the data stream 14, for example, by one or more instructions that associate packets of the data stream with the predetermined portion 13, and / or by instructions such as supplemental information messages that indicate the start and end of the segment from which the predetermined portion 13 is selected. For example, the start and end may be indicated by the respective supplemental information messages.

[0052] According to the embodiment, the data stream 14 may include a set of verification parameters for verifying a predetermined portion 13, or a segment of the data stream 14 containing the predetermined portion, for example, a segment 17 as described in relation to Figure 7. For example, the set of verification parameters may include one or more of the following: a hash function for performing an authenticity check of the predetermined portion 13 (for example, a hash function 31 as described in relation to Figure 2), a certificate for decrypting a digital signature 43 for performing an authenticity check of the predetermined portion 13, an identifier for associating the predetermined portion with a media asset, and instructions on how to derive the predetermined portion 13. The set of verification parameters may be signaled, for example, with a supplemental information message that may correspond to a DSC ISCI message, as described later. For example, instruction 51 may be included in the set of verification parameters.

[0053] Figure 2 shows one embodiment of a verification module 41, which may be optional as part of the decoder 20. Alternatively, the reliability check described in relation to the verification module 41 may be performed outside of the decoder 20. According to the embodiment of Figure 2, the reliability check, i.e., checking whether a given portion 13 of the data stream 14 conforms to the digital signature 43, includes subjecting the given portion 13 to a hash function 31 to obtain a hash value 33. According to this embodiment, the verification module 41 further includes a verification block 39 that checks whether the hash value 33 conforms to the digital signature 43.

[0054] The following describes the further optional features of verification module 41. According to one embodiment, the verification block 39 includes a verification string generator 49 configured to form a verification string, for example, an IdString of the sample syntax described later, based on a hash value 33. For example, the verification string generator 49 may form a concatenation including the hash value 33 and, optionally, a media asset identifier that identifies the media asset to which a given portion 13 of the media belongs, an identifier of the algorithm of the hash function 31, and further information such as one or more further hash values ​​obtained from earlier portions of the data stream 14. According to this embodiment, the verification block 39 includes a decryption block 59 that decrypts the digital signature 43 to obtain a check value 47, and the verification block 39 checks whether the verification string 48 matches the check value 47.

[0055] For example, the decryption module 59 may use an asymmetric cryptographic public key 45 to decrypt the digital signature 43. For example, the extractor 21 may derive the public key 45 based on an instruction in the data stream 14, for example, a resource identifier that indicates a resource from which the public key 45 can be derived, and then derive the public key 45 from the indicated resource in the data stream 14. In other words, according to one embodiment, the verification block 39 checks whether the hash value 33 conforms to the digital signature 43 by forming a verification string based on the hash value and optionally based on further information, and by comparing the verification string with the digital signature 43 using the public key (comparing the verification string with the digital signature may include decryption, which is performed by the decryption block 59).

[0056] For example, the generation of the digital signature 43 can be performed on the encoder side by forming a verification string and signing the verification string using the secret key of an asymmetric encryption scheme. For example, signing may involve further hashing, i.e., hashing the verification string using a further hash function to obtain a further hash value, and signing the further hash value. In this example, it may be impossible for the decoder to reconstruct the verification string from the digital signature 43, but instead, it can check whether the check value formed using the hash value 33 conforms to the digital signature, for example, by forming the verification string and deriving the check value by hashing the verification string using a further hash function. In other words, in this case, verification by verification block 39 may involve hashing the verification string using a further hash function to obtain a further hash value, and checking whether the further hash value conforms to the digital signature, for example, by decrypting the digital signature using a public key and checking whether the resulting check value is equal to the further hash value.

[0057] In other words, according to one embodiment, checking whether the hash value 33 conforms to or matches the check value 47 may include, for example, forming a verification string using the hash value 33 by concatenating it with further information such as a further hash value or hash function identifier, as described below, and hashing the verification string using a further hash function. The verification block 39 may then check whether the hashed verification string is equal to the check value 47 decoded from the digital signature. On the encoder side, according to this embodiment, the digital signature may be generated by forming a verification string, similar to the decoder side, hashing it using a further hash function, and signing the hashed verification string to obtain the digital signature 43.

[0058] According to an alternative embodiment, the check value 47 may correspond to a verification string, for example, a hash value 33, or the concatenation of hash value 33 with further information such as a further hash value or hash function identifier. In other words, decrypting the digital signature in this case may result in hash value 33 as part of (or the entirety of) check value 47. In this case, the digital signature may become larger due to the omission of further hashes. For example, which of the above alternatives is used may depend on the selected hash function.

[0059] Figure 3 shows a device 10 for encoding a data stream 14. The device 10 may be referred to as an encoder 10. The device 10 is configured to make the data stream 14 verifiable in terms of reliability. The encoder 10 provides the data stream 14 by inserting, for example, encoding, multiple payload packets 60 carrying payload data and supplemental information packets 18 carrying supplemental information messages 90 into the data stream 14. In other words, the encoder 10 may provide the data stream 14 described in relation to Figure 1. For example, the encoder 10 includes an inserter 23 for inserting the payload packets and supplemental information packets into the data stream 14. The encoder 10 includes a verification module 41' that obtains a digital signature 43 based on a predetermined portion of the data stream 14. The encoder 10 further includes a portion decision-maker 30' that determines a predetermined portion 13. For example, the portion decision-maker 30' determines a predetermined portion 13 by including, for example, a portion of the data 14' in a predetermined portion 13 that corresponds to a predetermined portion 13 determined by the portion decision-maker 30, based on the data 14' to be inserted into the data stream 14.

[0060] Any description of device 20 can be optionally applied to encoder 10 in the sense that information derived from data stream 14 by device 20 can be inserted into data stream 14 by device 10. Furthermore, any hash function used by verification module 41, such as hash function 31, may be equivalent to the corresponding hash function used by verification module 41'. The same applies to the input of the corresponding hash function, such as hash function 31, used to derive the hash value 33.

[0061] The interaction between the verification module of the decoder 20 and the verification module 41' of the encoder 10 has already been briefly described above. The verification module 41' may target a verification string containing a hash value derived by applying a hash function to a predetermined portion 13. The verification module 41' may further sign the verification string using, for example, the secret key of the asymmetric cryptographic scheme described above, in order to generate a digital signature 43. For example, the digital signature 43 may be inserted into the data stream 14 by the inserter 23, or the inserter 23 may insert into the data stream 14 a resource instruction from which the digital signature 43 can be derived.

[0062] Any description relating to device 20 may be optionally applied equivalently to device 10, in the sense that information derived from data stream 14 by device 20 may be inserted into data stream 14 by device 10. Furthermore, any hash function, such as hash function 31 used by device 10, may be equivalent to the corresponding hash function used by device 20. The same applies to the inputs to the corresponding hash function, such as hash function 31, used to derive the hash value 33. The generation of the digital signature 43 and the verification performed by device 20 using the digital signature 43 may each be part of an asymmetric cryptographic scheme, and these steps may each be performed by a pair of private and public keys, where at least the private key is used to sign in order to generate the digital signature 43, and the public key is used to decrypt in order to verify the verification string formed on the receiver side against the digital signature 43.

[0063] The following describes an embodiment of the first aspect of the present invention. Figure 4 shows a device 20 for decoding a data stream 14 according to one embodiment of the first aspect of the present invention. The device 20 in Figure 4 may optionally correspond to the decoder 20 in Figure 1, that is, the decoder 20 in Figure 4 may be based on any of the embodiments described in relation to Figure 1. Furthermore, the embodiments described below may optionally be combined with any of the embodiments described in relation to Figures 1 to 3.

[0064] According to an embodiment of the first aspect of the present invention, the extractor 21 is configured to derive an instruction 51 from the data stream 14 indicating whether one or more supplemental information messages should be considered, used, or included in order to validate the data stream 14, for example, to check the data stream for reliability. According to an embodiment of the first aspect, the partial decision-maker 30 determines a predetermined portion 13 by including one or more supplemental information messages in the predetermined portion 13 if the instruction 51 indicates one or more supplemental information messages that should be considered to validate the data stream 14. The one or more supplemental information messages that the instruction 51 refers to, i.e., the instruction 51 indicates whether to include the supplemental information messages in the predetermined portion 13, may be referred to as a validation set of supplemental information messages.

[0065] It should be noted that the final decision on whether to include or exclude a supplemental information message or packet from a given portion may depend on one or more further criteria, as described, for example, with respect to the fourth aspect. In other words, instruction 51 may be one of several criteria for whether or not to include a supplemental information message from a given portion 13. In other words, one embodiment of the present invention is a device 20 for decoding a data stream 14 (for example, a device for decoding media signals from a media data stream), the data stream comprising a plurality of payload packets 16 (for example, VCL NAL units) carrying payload data (for example, media data, for example, video data) (for example, encoded video data, for example, video data is information from which sample values ​​of picture in video are reconstructed), and supplementary information packets 18 (for example, supplementary information payload packets) (for example, SEIThe Supplemental Information Packet further includes NAL units (e.g., scattered between payload packets), and the Supplemental Information Packet carries Supplemental Information Messages 19 (e.g., information that assists processes related to decoding, displaying, or other purposes, but is not required by the decoding process to determine the values ​​of samples in the decoded picture), and the device takes from the data stream instructions 51 (e.g., syntactic elements, e.g., dsci_sei_mess) indicating whether one or more Supplemental Information Messages should be considered (or used or included) for validating the data stream 40 (e.g., validating the reliability of the data stream, in other words, checking the data stream for reliability) (or, e.g., whether the Supplemental Information should not be considered to authenticate a given portion 13). The system is configured to: derive ages_digitally_signed_flag) 21; determine a given portion 13 of the data stream by including one or more supplemental information messages in the given portion if instruction 51 indicates that one or more supplemental information messages should be considered in order to verify the data stream (e.g., which given portion should be verified or checked for trustworthiness); and obtain a digital signature 43 from the instructions in the data stream to verify the given portion 13 21 (e.g., using the information derived from the data stream, e.g., deriving a digital signature from the data stream or deriving a digital signature from an indicated source in the data stream) (e.g., the digital signature is derived based on the given portion).

[0066] According to one embodiment, the device is configured to verify a given portion of a data stream by checking whether that portion conforms to (or matches) a digital signature (for example, to check the reliability of the data stream or a given portion thereof).

[0067] According to one embodiment, the device is configured to provide (or transfer) a predetermined portion and a digital signature for verification of that portion (for example, performed by a further device or entity) (for example, verification includes checking whether a predetermined portion of a data stream conforms to (or matches) a digital signature). In other words, one embodiment of the present invention described in relation to Figure 4 is a device 20 for decoding a data stream 14 (e.g., a device for decoding media signals from a media data stream), the device is configured to check the data stream for reliability (e.g., a device for verifying the data stream), and the data stream includes a plurality of payload packets 16 (e.g., VCL NAL units) that carry payload data (e.g., encoded video data, where the data is information from which sample values ​​of a picture are reconstructed), and supplementary information packets 18 (e.g., supplementary information payload packets) (e.g., SEIThe Supplemental Information Packet further includes NAL units (e.g., scattered between payload packets), and the Supplemental Information Packet carries Supplemental Information Messages 19 (e.g., information that assists processes related to decoding, displaying, or other purposes, but is not required by the decoding process to determine the values ​​of samples in the decoded picture), and the device takes from the data stream instructions 51 (e.g., syntax elements, e.g., dsci_sei_messages_digitally_signed) indicating whether (or should be used or included) (or whether the Supplemental Information should not be considered to authenticate a given portion) the Supplemental Information Messages (e.g., one or more Supplemental Information Messages) should be considered to validate the data stream (e.g., to validate the reliability of the data stream) (e.g., one or more Supplemental Information Messages) (e.g., one or more syntactic elements, e.g., dsci_sei_messages_digitally_signed) The system is configured to derive a flag 21, and, if instruction 51 indicates that supplemental information messages should be considered in order to verify the data stream, to determine which part of the data stream should be verified (for example, which part should be checked for trustworthiness) by including one or more supplemental information messages 19 in the part 13 30, and to obtain a digital signature 43 based on the data stream 14 (for example, by using information derived from the data stream, for example, by deriving a digital signature from the data stream or by deriving a digital signature from an indicated source in the data stream) and to check whether the part of the data stream 13 conforms to (or matches) the digital signature 43 41.

[0068] Further optional features of the embodiment of the first aspect are described below. According to one embodiment, the verification set of supplemental information messages includes a subset of all supplemental information messages in the data stream 14. In other words, instruction 51 may indicate whether supplemental information messages belonging to the verification set should be included in a predetermined portion 13 or excluded from a predetermined portion 13. In other words, according to one embodiment, instruction 51 indicates whether supplemental information messages belonging to the verification set of supplemental information messages should be included in a predetermined portion 13 or excluded from the predetermined portion 13. For example, optionally, the data stream may include further supplemental information messages that do not belong to the verification set, which should always be included in the verification, i.e., in the predetermined portion.

[0069] In one embodiment, instruction 51 indicates whether one or more supplemental information messages should be considered in a predetermined portion 13, or should be included within the predetermined portion 13, or whether there are any supplemental information messages that should be considered for verifying the data stream, for example, whether there are any supplemental information messages that should be included within the predetermined portion 13. In other words, instruction 51 can distinguish between considering that a verification set should be included within a predetermined portion and not including any supplemental information messages, for example, any supplemental information messages of a segment referenced by a digital signature 43, in the predetermined portion. In other words, instruction 51 indicates whether supplemental information messages, i.e., supplemental information messages of a verification set, should be included within a predetermined portion 13 or excluded from the predetermined portion 13, that is, whether supplemental information messages should be considered for verifying the data stream, or whether there are any supplemental information messages that should be considered for verifying the data stream or the predetermined portion, for example, whether there are no supplemental information messages to be considered at all.

[0070] The verification set may include all supplemental information messages, or parts thereof, of the segment of the data stream referenced by the digital signature 43. In the latter case, the verification set may be predefined, for example, by message type, or signaled in the data stream 14, for example, by indicating which message types should be included or excluded, or by individually identifying messages or packets containing each message. Further details of optional choices are described in more detail below.

[0071] The validation set for supplemental information messages can be defined by the supplemental information message type, by identifying the supplemental information messages that should be included in the validation set individually or on a packet-by-packet basis, or by any combination of these criteria. According to packet-by-packet selection, for example, by individually identifying which packets should be included in or excluded from a given portion, or by packet type, the entire supplemental information packet may be included or excluded from a given portion. In other words, with packet-by-packet selection, all supplemental information messages contained within the selected supplemental information packet are included in the validation set for supplemental information messages.

[0072] According to one embodiment, if instruction 51 indicates that one or more supplemental information messages should be considered in order to verify a predetermined portion 13, the partial decision-maker 30 determines the predetermined portion 13 by including one or more supplemental information packets in the predetermined portion 13. In other words, including supplemental information messages in a predetermined portion 13 may be done on a per-packet basis; that is, one or more supplemental information messages to be considered for modifying a predetermined portion 13 may be identified or indicated by identifying or indicating one or more supplemental information packets of the data stream 14 that should be included within the predetermined portion 13.

[0073] For example, according to one embodiment, the partial decision-maker 30 may include in the predetermined portion 13 any supplemental information messages that should be included within the predetermined portion 13, for example, all of the supplemental information packets 18 that carry any one of the one or more supplemental information messages of the verification set.

[0074] According to one embodiment, the partial decision-maker 30 conditions the decision of whether or not to include one of a plurality of supplemental information packets in a predetermined portion 13 on the result of checking whether the supplemental information packet contains a supplemental information message of one of the supplemental message types in a set of supplemental message types. In other words, the identification of supplemental information messages belonging to a verification set can be performed based on the supplemental information message type by including a specific type of supplemental information message. In addition, the inclusion or exclusion of supplemental information messages can be performed on a packet-by-packet basis by including each supplemental information packet that contains at least one supplemental information message from the set of supplemental information message types.

[0075] According to one embodiment, instruction 51 is a flag having a first state and a second state. According to this embodiment, the partial decision-maker determines a predetermined portion 13 of the data stream 14 by determining one or more supplemental information messages to be included in a predetermined portion 13 based on a predefined set of supplemental information messages when the flag has the first state, and by ensuring that none of the supplemental information messages from the predefined set are included in the predetermined portion when the flag has the second state. For example, when the flag has the first state, the partial decision-maker 30 may include all of the supplemental information messages from the predefined set, or at least one instance of all of the supplemental information messages from the predefined set, as will be described in more detail below. For example, instruction 51 may be a flag that has only a first state and a second state. In other words, instruction 51 may be binary.

[0076] According to one embodiment, a predefined set of supplemental information messages includes all supplemental information messages of supplemental information packets that refer to multiple payload packets. According to another embodiment, a predefined set of supplemental information messages includes all supplemental information messages of any type from a set of supplemental information message types. For example, the set of supplemental information message types is a subset of multiple possible types of supplemental information messages that may be defined depending on the type of data stream 14. For example, the set of supplemental information message types may be predefined; that is, for example, the set of supplemental information message types may be recognized by the decoder 20 without needing to be transmitted in the data stream 14.

[0077] According to one embodiment, the identification of one or more supplemental information messages to be included within a predetermined portion is signaled in the data stream 14 when instruction 51 indicates that another supplemental information message should be considered in order to validate the data stream.

[0078] Below, an embodiment of how to identify the validation set of supplemental messages in data stream 14 will be described with reference to Figures 5 and 6. Figure 5 shows one embodiment of a decoder 20 according to a first aspect of the present invention. For example, the decoder 20 in Figure 5 may be an embodiment of the decoder 20 in Figure 1, that is, any of the details described above can be optionally applied to the decoder 20 in Figure 5. According to Figure 5, the data stream 14 includes a syntactic element 53 that identifies a set of types of supplemental information messages to be contained within a predetermined portion 13. In other words, according to this embodiment, the verification set can be identified in terms of supplemental information message types. For example, in the exemplary example of Figure 5, the supplemental information packet 18' contains a supplemental information message 19' of the type indicated by the syntactic element 53. According to this embodiment, the extractor 21 derives the syntactic element 53 from the data stream 14. The information of the syntactic element 53 may be used by the partial decision-maker 30 to determine, for a given part 13, whether the given part 13 contains a supplemental information message of a type included in the set of supplemental information message types indicated by the syntactic element 53, or whether the given part 13 contains a supplemental information packet containing a supplemental information message of a type included in the set of supplemental information message types indicated by the syntactic element 53, as shown in Figure 5.

[0079] In embodiments where a supplemental information packet contains only one supplemental information message, it should be noted that the type of the supplemental information message may be considered equivalent to the type of the supplemental information packet. In other words, there is not necessarily a distinction between the supplemental information message type and the supplemental information packet type, but the type of the supplemental information packet may be considered equivalent to the type of the supplemental information message carried in each supplemental information packet. According to one embodiment, the syntactic element 53 is signaled by a supplemental information message, for example, a supplemental information message referred to below as a verification parameter message. For example, the verification parameter message carries the verification parameter set described above. For example, the supplemental information message carrying the syntactic element 53 may be contained in a supplemental information packet.

[0080] According to one embodiment, instruction 51 and syntactic element 53 are included in the same supplemental information message, for example, a validation parameter message. According to one embodiment, instruction 51 is signaled by syntactic element 53, for example, by a predetermined state of syntactic element 53. For example, according to one embodiment, the syntactic element 53 has a plurality of first and second states. According to each of the first states, one or more supplemental information messages should be included in a given part 13, and the first state distinguishes different sets of verification. In other words, the first state identifies the supplemental information messages that should be included in the given part. In other words, each of the first states indicates one or more supplemental information messages, which are not necessarily a true subset, that should be included in the given part 13. According to the second state, none of the one or more supplemental information messages are included in the given part 13. Therefore, according to one embodiment, if the syntactic element 53 has one of the first states, the partial decision-maker 30 determines one or more supplementary information messages to be included in a predetermined part 13, depending on the state of the syntactic element 53. If the syntactic element has a second state, the partial decision-maker 30 does not include any of the one or more supplementary information messages in the predetermined part 13.

[0081] According to one embodiment, if instruction 51 indicates that one or more supplemental information messages should be included in a predetermined portion 13, the extractor 21 derives from the data stream 14 a first syntactic element (e.g., dsci_num_sei_message_types_digitally_signed_minus1, described later) that indicates the number of supplemental information message types to be considered for the predetermined portion 13, and further derives from the data stream 14 the number of each second syntactic element that identifies each supplemental information message type that should be included in the predetermined portion 13. For example, the number of each second syntactic element (e.g., dsci_digitally_signed_sei_type[i]) corresponds to the number indicated by the first syntactic element. In other words, for example, a validation set may be indicated in terms of supplemental information message types, the size of the validation set is variable, and it is signaled in the data stream 14. Thus, the number of supplemental information messages that should be included in a predetermined portion can be adaptively set by the encoder.

[0082] For example, supplemental information message types that may be dropped during processing of a data stream by a file parser or an entity that extracts substreams from a data stream may be excluded from the authenticity check to avoid the need to recalculate the digital signature. On the other hand, supplemental information messages may contain information that affects the output of the decoded data stream and therefore verification may be desirable. By making the size of the verification set variable and determining whether each individual type should be included in or excluded from the verification set, the decoder can individually adapt the trade-offs between these aspects for each data stream. At the same time, instructions in terms of supplemental information message types provide group identification of supplemental information messages that should be included in a given portion, thereby avoiding individual identification of supplemental information messages in the verification set. According to one embodiment, the first syntactic element and the second syntactic element may be signaled, for example, with instruction 51 in the verification parameter message described above.

[0083] Figure 6 shows a further embodiment of the decoder 20. The decoder 20 in Figure 6 may optionally be an example of the decoder 20 in Figure 1 or Figure 4. According to one embodiment of Figure 6, the partial decision-maker 30 conditions the decision of whether to include a supplemental information payload packet of supplemental information payload packet 18 in a predetermined portion 13 on the result of checking whether the supplemental information payload packet contains a supplemental information message (e.g., as a prefix of further supplemental information messages, e.g., further supplemental information messages within the same packet) that instructs the predetermined portion 13 to include the supplemental information payload packet. For example, in Figure 6, each of the payload packets 18' contains a supplemental information message 55 that instructs the predetermined portion 13 to include its respective supplemental information packet 18', while supplemental information packet 18'' does not contain such an instruction.

[0084] It should be noted that, optionally, the partial decision-maker 30 may condition its decision on whether or not to include a supplemental information packet 18' in a predetermined portion 13 on one or more further conditions. In other words, according to one embodiment, a supplemental information packet 18' that instruction 55 indicates should be included in a predetermined portion 13 is not necessarily included in the predetermined portion 13, but the decision may depend on further conditions. For example, as will be described later, if the supplemental information message is repeated, only one instance of each message may be included in the predetermined portion. Alternatively, according to one embodiment, the partial decision-maker 30 includes a supplemental information packet 18' which includes a supplemental information message 55 that instructs to include a supplemental information packet in a predetermined portion.

[0085] A further condition for the partial decision-maker 30 to determine whether to include or exclude a supplemental information packet from a predetermined portion 30 may be whether each supplemental information packet belongs to a predetermined set of supplemental information packets that are always included in a predetermined portion 13 or never included in a predetermined portion 13. Therefore, according to one embodiment, if a supplemental information packet does not contain a supplemental information message 55 instructing that the supplemental information packet be included in a predetermined portion, the partial decision-maker 30 excludes each supplemental information packet if each supplemental information packet belongs to a predetermined set of supplemental information messages, and excludes each supplemental information packet if each supplemental information packet does not belong to a predetermined set of supplemental information packets. It should be noted that the decision may optionally rely on further conditions, such as the reliance on repetition already described. For example, supplemental information message 55 may be a prefix supplemental information message preceding all supplemental information messages within the supplemental information packet, or it may be a further supplemental information message.

[0086] The following section continues the explanation of the decoder 20 in Figure 4 and describes further alternative examples for instructing the verification set. According to one embodiment, the decision of whether or not to include a supplemental information packet in a predetermined portion 13 is conditional on checking whether there is a prefix supplemental information packet preceding the supplemental information packet that instructs the inclusion of a subsequent, for example, immediately following supplemental information packet in the predetermined portion 13. In other words, compared to the embodiment described with respect to Figure 6, a prefix supplemental information packet may be used for instruction instead of a supplemental information message 55. All further details described with respect to Figure 6 may optionally apply to this embodiment as well. According to another embodiment, the decision of whether or not to include a supplemental information message in a given portion 13 is conditional on checking whether there is a prefix supplemental information message preceding the supplemental information message that instructs the subsequent, for example, immediately following supplemental information message to be included in the given portion 13. In other words, according to this embodiment, inclusion or exclusion may be on a message-by-message basis rather than on a packet-by-packet basis. The details described with respect to Figure 6 may optionally apply to this embodiment as well.

[0087] According to a further embodiment, the decision of whether or not to include a supplemental information packet in a predetermined portion 13 is conditional on the result of checking whether the supplemental information packet includes a nesting supplemental information message that instructs the predetermined portion to include the supplemental information packet. For example, a nesting supplemental information message may include a set of one or more supplemental information messages. From the fact that a set of supplemental information messages is signaled within the nesting supplemental information message, the partial decision-maker 30 may conclude that a supplemental information packet containing the nesting supplemental information message should be considered to be included in a predetermined portion 13. In other words, according to this embodiment, inclusion in or exclusion from a predetermined portion may be on a packet-by-packet basis, and the indication of whether a packet is included or excluded may be signaled by the nesting supplemental information message.

[0088] According to another embodiment, nesting supplemental information messages are used to signal whether to include or exclude supplemental information messages from a given part for each message. According to this embodiment, the decision of whether or not to include supplemental information messages is conditional on the result of checking whether or not a nesting supplemental information message that instructs to include supplemental information messages in a given part 13 contains supplemental information messages.

[0089] As already mentioned above, the inclusion or exclusion of supplemental information packets may be further conditional on whether they contain supplemental information messages of a type belonging to a set of types of supplemental information messages that should be included in a given portion. In other words, for example, this set of types of supplemental information messages includes supplemental information message types that should be included in a given portion without being explicitly signaled in the data stream 14. In other words, the set of types may be predefined. A predefined set of supplemental information messages may be added to supplemental information message types that are adaptively signaled to be included in a given portion.

[0090] Whether a supplemental information message belongs to a predetermined set of supplemental information message types that should be included in a predetermined portion may be independent of or dependent on instruction 51. In other words, supplemental information messages belonging to a predetermined set of types may always be included in a predetermined portion. Independent of instruction 51, or if instruction 51 indicates that supplemental information messages should be considered for validating the data stream, the supplemental information messages of the predetermined set of types are considered to be included in the predetermined portion 13, and optionally, any additional supplemental information messages indicated to the data stream 14 are considered to be included in the predetermined portion. If instruction 51 indicates that one or more supplemental information messages should not be considered for validating the data stream, according to one embodiment, the supplemental information messages of the predetermined set of types are considered to be included in the predetermined portion 13 without considering that any further supplemental information messages are included in the predetermined portion 13. According to another embodiment, if instruction 51 indicates that one or more supplemental information messages should not be considered for validating the data stream, neither the supplemental information messages of the predetermined set of types nor any further supplemental information messages are considered to be included in the predetermined portion 13. The inclusion and exclusion of supplementary information messages by identification using a predetermined set of types can be performed on a per-packet basis.

[0091] According to one embodiment, a supplemental information packet is deemed to be included in a predetermined portion if it contains any supplemental information message from a predetermined set of types. According to another embodiment, a supplemental information packet is deemed to be included in a predetermined portion if the first supplemental information message belongs to a predetermined set of supplemental information message types. For example, according to one embodiment, the partial decision-maker 30 conditions the decision of whether or not to include one supplemental information packet among a plurality of supplemental information packets in a predetermined part on the result of checking whether the supplemental information packet contains one of the types of supplemental information messages from a set of types of supplemental information messages that should be included in the predetermined part.

[0092] For example, if instruction 51 indicates that supplemental information messages should be considered in order to validate the data stream, the partial decision-maker 30 may check for one of the supplemental information packets among the multiple supplemental information packets (for example, for each of the supplemental information packets) whether the supplemental information packet contains one of the types of supplemental information message that should be included in a given portion, and if the supplemental information packet contains one of the types of supplemental information message that should be included in a given portion, it may include the supplemental information packet in the given portion.

[0093] According to one embodiment, the partial decision-maker 30 conditions the decision of whether or not to include one supplemental information packet among a plurality of supplemental information packets in a predetermined part on the result of checking whether the first supplemental information message among the one or more supplemental information messages carried in that supplemental information packet is one of the set of types of supplemental information messages that should be included in the predetermined part. For example, if the instruction indicates that supplemental information should be considered in order to authenticate a given part, then for one of several supplemental information packets (for example, for each supplemental information packet), check whether the first supplemental information message among the one or more supplemental information messages carried in the supplemental information packet is one of the set of types of supplemental information messages that should be included in the given part, and if the first supplemental information message among the one or more supplemental information messages carried in the supplemental information packet is one of the set of types of supplemental information messages that should be included in the given part, then include the supplemental information packet in the given part.

[0094] The following describes further alternatives to identifying verification sets, utilizing the part-by-part verification system implemented in the reliability check. In other words, the verification substreams, described later with respect to Figure 7, can be used to identify supplemental informational messages that should be included in the reliability check. Figure 7 shows a further embodiment of the decoder 20. The features described in relation to Figure 7 can be optionally combined with any of the aforementioned embodiments of the decoder 20. According to the embodiment of Figure 7, the reliability check is performed in units of parts 13', 13''. In other words, according to this embodiment, the data stream 14 includes several parts in which the data stream 14 can be verified in units thereof. For example, the data stream 14 includes a digital signature for each of the parts. Optionally, the data stream 14 may include the aforementioned verification parameter set for each of the parts. Alternatively, the verification parameters for all or a subset of the parts of a segment may be included in a common verification parameter set. The predetermined part 13 described above may be one of several parts. Each part may be defined by assigning one or more packets or data contained in packets to it (each part).

[0095] For example, in Figure 7, payload packet 16' is assigned to part 13', and payload packet 16'' is assigned to part 13''. Performing verification of the data stream 14 on a part-by-part basis may include determining each part, for example, parts 13' and 13'' in Figure 7, and subjecting each part to a reliability check 41. For this purpose, the data stream 14 may indicate the respective digital signatures for each part, for example, the digital signature 43' for part 13' and the digital signature 43'' for part 13'' in Figure 7. For example, parts as described in Figure 7 may be referred to as substreams or verification substreams of the data stream 14.

[0096] For example, in the case of a video data stream, different layers of a layered video data stream may be assigned to different substreams. For example, different layers may carry different representations of the video encoded within the video data stream 14, for example, having different spatial resolutions, carrying different types of data such as texture and depth, or carrying different views of a scene. As a further example, different substreams may be associated with different time layers of a media data stream, each time layer carrying samples to form a representation of a media signal encoded at a different time resolution, for example, resulting in a higher time resolution when different time layers are combined. However, it should be noted that these are merely examples of organizing data in a media data stream within different substreams, and the association between packets and substreams may depend on an encoder or entity that makes the data stream 14 checkable in terms of reliability. In other words, the data stream 14 may contain, or may contain, instructions for each of the several parts of the data stream 14 that are verifiable on a unit basis.

[0097] In some embodiments, in addition to the data stream being verifiable in units of the aforementioned parts in the sense of substreams, reliability checks may be performed in units of segments 17 of the temporal sequence of segments of the data stream 14. A segment may be referred to as a verification period. In other words, within one segment of the data stream 14, multiple substreams, e.g., parts 13', 13'' described above, may be defined, and the segments are verifiable in units thereof, and packets or data belonging to each of the substreams do not necessarily form a contiguous part of the data stream 14, but rather, within one segment, packets or data fragments may be individually assigned to one of the substreams. Hereinafter, only one segment 17 is considered, i.e., when referring to a part, for example, the aforementioned substreams within one segment 17 may be referred to. For example, in the embodiments described with respect to the first to fifth embodiments, a given part 13 may be a part of one segment 17 of the data stream 14. According to one embodiment, the data stream 14 includes, for example, an indication of the number of substreams of the data stream within a single segment, such that, for example, only one segment of the data stream is considered as a data stream.

[0098] According to one embodiment, the extractor 21 derives several parts from the data stream 14, for example, the number of substreams of the data stream, by deriving instructions for several parts from the data stream 14. According to this embodiment, the part decisioner 30 may assign each payload packet 16 to one of the parts, for example, one or more of the parts, that is, it does not necessarily have to be assigned to each of the parts.

[0099] The following describes an embodiment according to the first aspect of the present invention, which utilizes the concept of substreams to verify supplemental information messages. However, it should be noted that the substream concept described above with respect to Figure 7 may be applicable independently of the first aspect of the present invention within the scope of further aspects of the present invention described later. According to one embodiment, several parts are given an order, such as a hierarchical order, among them, and each part has a rank within that order. According to one embodiment, the verification of the parts is performed from the lowest rank to the highest rank among the parts. According to one embodiment, the verification of a part relies on a lower-ranking neighboring part of the part in its order. For example, a hash value derived from a lower-ranking part may be included in the verification string used to verify that part.

[0100] According to one embodiment, the reliability check of a portion may optionally rely on one or more lower-ranked portions. For example, one or more hash values ​​derived from one or more of the lower-ranked portions on which the reliability check of that portion relies may be included in the verification string for verifying that portion. In other words, each part can provide verification for itself, and a hash value already derived for a part can be reused to verify a higher-ranked part.

[0101] According to one embodiment, the set of supplemental information messages to be considered for reliability checks is verified as part of one predetermined substream. For example, the supplemental information messages to be verified may be collected in one substream of a predetermined rank within an order defined between the substreams. According to one embodiment in which the verification subset is identified in terms of supplemental information message types, the partial decision-maker 30 may include supplemental information messages among supplemental information messages having any type from the set of supplemental information message types that should be included in the reliability check into a predetermined part 13 having a predetermined rank in an order defined among several parts according to this embodiment.

[0102] For example, a given rank is the highest rank in an order defined among several parts. Alternatively, supplementary information messages to be included in the reliability check may be included in the portion to which the payload packet to which each supplementary information message is associated is allocated. As mentioned above, each supplementary information message may be associated with one of the payload packets. Supplementary information messages to be included in the reliability check may be allocated in the portion to which the payload packet to which the supplementary information message is associated is allocated.

[0103] According to one embodiment, the data stream 14 includes instructions to distinguish between including all supplemental information messages of a certain type that should be included in a reliability check in a substream of a predetermined rank, or including all supplemental information messages of a certain type that should be included in a reliability check in the portion associated with their respective assigned payload packets. According to another embodiment, some parts include a first part, each of which has a payload packet associated with each of the first parts, and each of the first parts has a second part of some parts associated with each of the first parts, the second part of which should contain supplemental information messages to be considered for reliability checks and to be assigned to the payload packet associated with each of the first parts. In other words, some parts include a first part and a second part, each of which has a payload packet associated with each of the second parts, the first part being for verifying the payload packet, and the second part being for verifying the supplemental information messages assigned to the payload packet of the respective associated first part.

[0104] More generally, according to one embodiment, each supplemental information message is associated with one of the payload packets, and the device is configured to assign each supplemental information message of a data stream having any type from a set of types of supplemental information messages to one of the parts, that one part is associated with another of the parts, and that other of the parts is associated with the payload packet associated with each supplemental information message.

[0105] According to one embodiment, each supplemental information packet is associated with one of the payload packets, and the device is configured to assign each supplemental information packet of a data stream carrying any type of supplemental information message to one of the parts, the one part of which is associated with another part, and the other part of which is associated with the payload packet associated with each supplemental information packet. According to one embodiment, the device 20 is configured to derive a syntactic element from a data stream that indicates the number of parts, the syntactic element having a value, and the device is configured to infer that the number of parts corresponds to the value plus 1, multiplied by 2. According to one embodiment, the device 20 is configured to derive a syntactic element from a data stream that indicates the number of parts, the syntactic element has a value, and the device is configured to infer that the number of parts corresponds to the value plus 1.

[0106] According to one embodiment, one part and one further part follow each other in a hierarchical order defined among several parts, for example, one part follows one further part in a hierarchical order. According to one embodiment, in the apparatus, the position index of one part in a hierarchical order defined between parts corresponds to the position index of a further part in the hierarchical order plus half the number of parts.

[0107] The following describes an embodiment of a second aspect of the present invention. Figure 8 shows a device 20 for decoding a data stream 14 according to one embodiment of a second aspect of the present invention. The device 20 in Figure 8 may optionally be an embodiment of the device 20 in Figure 1, that is, the device 20 in Figure 8 may be based on any of the embodiments described with respect to Figures 1 to 4. Furthermore, embodiments of the second aspect may optionally be combined with any of the embodiments described with respect to the first aspect. Features described with respect to Figure 7 may optionally be applied to embodiments of the second aspect.

[0108] According to an embodiment of the second aspect, the data stream includes a temporal sequence of segments 17, 17*, as described with respect to Figure 7, for example. For example, a segment forms a temporal sequence with respect to the signals encoded in the data stream 14, with respect to the temporal order of the samples of the signal, such as the presentation order or presentation time assigned to the samples. For example, a segment may be a video segment that forms a temporal sequence with respect to the presentation order or presentation time assigned to the video segment. For example, each of the segments 17 may be an encoded video sequence (CVS). According to the embodiment of Figure 8, the partial decision-maker 30 determines a predetermined portion 13 of one of the segments 17* to be checked for reliability, as described above, for example. As already stated, the predetermined portion 13 does not necessarily have to be a contiguous portion of segment 17*, and may be assigned to individual packets or data fragments, as shown by the hatched portion of segment 17* in Figure 8.

[0109] According to an embodiment of the first alternative example of the second aspect, for a reliability check of segment 17*, the extractor 21 derives an instruction 61 from the data stream 14 indicating whether segment 17* is the first segment of a signed subsequence of a temporal sequence. For example, instruction 61 may be a syntactic element signaled in the data stream 14, such as a flag, such as DSCI_first_signed_segment_flag described later. According to the second alternative embodiment of the second aspect, instruction 61 indicates whether the segment is the last segment of a signed subsequence of a temporal sequence. For example, instruction 61 may be a syntactic element signaled in the data stream 14, such as a flag, such as DSCI_last_signed_segment_flag described later.

[0110] According to an embodiment of the second aspect, instruction 61 is included in the verification of a predetermined portion 13. In other words, instruction 61 may be included in the determination of the verification string 48 performed by the verification string generator 39 in Figure 8. In the reliability check 41, as shown in Figure 8, in block 39, the verification string 48 is formed based on the predetermined portion 13 and instruction 61. For example, block 39 may include blocks 31 and 49 in Figure 2. Block 63 of the reliability check 41, as shown in Figure 10, checks whether the verification string 48 conforms to the digital signature 43. For example, block 63 may include block 59 in Figure 2 and a comparison between the check value 47 and the verification string 48. According to one embodiment, the verification string 48 is formed by combining, for example, concatenating, hash values ​​derived by applying a hash function to the predetermined portion 13. According to another embodiment, the combination of instruction 61 and the predetermined portion 13, for example concatenation, is applied to a hash function to obtain a hash value, which is used to form the verification string 48. In other words, the instructions may be included in the designated part 13.

[0111] In other words, for example, a verification string may reflect or rely on both the hash value 31 and the instruction 61. For example, the verification within it may include the hash value 31 and the instruction 61, for example, a value or flag representing the instruction 61. In other words, one embodiment of the present invention is a device 20 for decoding a data stream, the data stream comprising a temporal sequence of segments 17 (e.g., coded video sequence (CVS)) (e.g., video segments form a temporal sequence with respect to the presentation order or presentation time assigned to the video segments of the temporal sequence), the device 30 determines, for a segment of the sequence of segments, a predetermined portion 13 of the segment, and from the data stream, i) whether the segment is the first segment of a signed (or verifiable) subsequence of the temporal sequence, or ii) The system is configured to derive an instruction 61 (e.g., a syntactic element, e.g., a flag, e.g., dsci_last_signed_segment_flag) indicating whether the segment is the last segment of a signed (or verifiable) subsequence of a temporal sequence 21, and to derive a digital signature 43 from a data stream to be checked against a verification string 48 obtained based on a given portion 13 and instruction 61 (e.g., the verification string reflects or relies on a hash value and an instruction, e.g., the verification string includes a hash value and an instruction, e.g., the value of a flag representing the instruction).

[0112] According to one embodiment, the device is configured to verify a predetermined portion 13 by checking whether a verification string 48 conforms to (or matches) a digital signature 43 (for example, to check the reliability of a data stream or a predetermined portion 13 thereof). According to one embodiment, the device is configured to provide (or transmit) a predetermined portion 13, an instruction 61, and a digital signature 43 for verification of the predetermined portion 13 (for example, performed by a further device or entity) (for example, verification includes checking whether the predetermined portion 13 of the data stream conforms to (or matches) the digital signature 43). According to one embodiment, a predetermined portion 13 is subjected to a hash function to obtain a hash value, and the verification string 48 is obtained based on the hash value.

[0113] According to one embodiment, a predetermined portion 13 and instructions (for example, their concatenation) are subjected to a hash function to obtain a hash value, and the verification string 48 is obtained based on the hash value. In other words, one embodiment of the present invention is a device 20 for decoding a data stream 14, the device is configured to check the data stream for reliability (e.g., a device for verifying the data stream), the data stream includes a temporal sequence of segments 17 (e.g., segments of encoded payload data, e.g., coded video sequence (CVS)) (e.g., segments form a temporal sequence with respect to the presentation order or presentation time assigned to the segments of the temporal sequence), the device obtains a hash value 33 by applying a hash function 31 to a predetermined portion 13 of the segments, i) whether the segment is the first segment of a signed (or verifiable) subsequence of the temporal sequence The system is configured to verify a segment (e.g., the current or a given) of a sequence of segments by 2) deriving an instruction 61 (e.g., a syntactic element, e.g., a flag, e.g., dsci_last_signed_segment_flag) indicating whether the segment is the last segment of a signed (or verifiable) subsequence of a temporal sequence 21, forming a verification string 48 based on the hash value 33 and the instruction 61 (e.g., the verification string reflects or relies on the hash value and the instruction, e.g., the verification string includes the hash value and the instruction, e.g., the value of a flag representing the instruction), deriving a digital signature 43 from the data stream, and checking whether the verification string 48 conforms to the digital signature 43.

[0114] Further optional selections of embodiments of the second aspect are described below. According to one embodiment, instruction 61 is included in the data stream 14 within a supplemental information message associated with segment 17*, for example, within a DSCI SEI message described later. For example, instruction 61 may be part of the verification parameter set described above. Alternatively, instruction 61 may be part of a supplemental information message that carries the digital signature 43, such as the DSCV SEI message described later. In other words, according to one embodiment of the first alternative of the second aspect, the supplemental information message containing instruction 61 precedes a predetermined portion 13 in the data stream 14. According to one embodiment of the second alternative of the second aspect, the supplemental information message containing instruction 61 follows a predetermined portion 13 in the data stream 14 or precedes the final payload packet of a predetermined portion 13 in the data stream 14.

[0115] According to one embodiment of the first alternative example of the second aspect, the reliability check 14 of a predetermined portion 13 of segment 17* is performed in such a manner that, when segment 17* is the first segment of a signed subsequence, a verification string 48 is formed independently of any segment of the data stream 14 that precedes segment 17* in the temporal sequence.

[0116] For example, if a segment is not the first segment in a signed subsequence, the verification string 48 may be formed using the previous hash value obtained for the preceding segment that precedes segment 17* in the sequence of segments in data stream 14. For example, when referring to a “preceding segment”, it means the preceding segment immediately before segment 17*, i.e., the segment whose authenticity is currently being checked. In other words, the instruction indicating whether segment 17* is the first segment of a signed subsequence provides the decoder, or device for checking the authenticity of the data stream 14, with information on whether the current segment is the first segment of a signed subsequence, thereby enabling the entity performing the authenticity check to infer whether the hash value from the previous segment should be included in the authenticity check in order to successfully verify the digital signature for the current segment 17*. For example, to provide verification of the temporal consistency of a sequence of temporal segments, rather than just verifying individual segments, segment verification may include the hash value of the previous segment within the verification string of the segment currently being checked, for example, to verify that no segments have been canceled or no additional segments have been added.

[0117] A receiver switching to a stream does not necessarily know whether the segment it is currently receiving is the first segment of a signed subsequence of the media data stream. An instruction indicating whether a segment is the first segment allows the receiver to infer that if the segment is not the first segment and the receiver does not have the preceding segment, it cannot perform a reliability check. If the segment is the first segment, the receiver knows from instruction 61 that a reliability check should be performed independently of the previous segment and that it can perform a reliability check. Therefore, if the reliability check fails, the receiver knows that the segment is unreliable. Furthermore, if two data streams are concatenated and there is no instruction in the data stream indicating a switch from the first data stream to the second data stream, the receiver may assume that the first segment of the second data stream should be verified using the hash value of the last segment of the first data stream. However, if these two data streams are checkable for reliability independently of each other, such a check may fail, and the receiver may identify a verification problem for the first segment of the second data stream. By having information that the segment is the first segment of the second data stream, a receiver according to one embodiment of the present invention may perform a reliability check independently of the first data stream, and as a result the check will succeed. The presence of an instruction that a segment is the last segment of a signed subsequence prevents the data stream from being unintentionally shortened.

[0118] As already explained with respect to Figure 7, the data stream 14 or its segments may optionally be verifiable in units of parts also referred to as substreams. According to an embodiment of the second aspect, the segment 17* is verifiable in units of several parts having a hierarchical order. According to one embodiment, if a predetermined portion 13 is not the first portion in a hierarchical order of parts, the reliability check 41 generates a verification string 48 based on the previous hash value obtained for the preceding portion of the segment 17* that precedes the predetermined portion 13 in the hierarchical order. For example, the preceding portion may be the portion on which the reliability check for the predetermined portion relies, in other words, the reliability check for the predetermined portion 13 refers to the preceding portion.

[0119] According to one embodiment, if a predetermined portion 13 is the first portion in a hierarchical order of parts, for example, the lowest-ranked portion as described above, and the subsequent portion of the predetermined portion 13 is verified, and the subsequent portion follows the predetermined portion 13 in a hierarchical order, for example directly or indirectly, the reliability check 41 forms a verification string 48 for verifying the subsequent portion based on a hash value obtained by applying a hash function to the predetermined portion 13.

[0120] According to one embodiment, instruction 61 indicating whether segment 17* is the first segment in a signed subsequence is selectively signaled, for example, only if a given part 13 to be verified is the first part in a hierarchical order of several parts. The same may apply if instruction 61 indicates whether segment 17* is the last segment in a signed subsequence. This conditional signaling is particularly useful when instruction 61 is signaled with a syntactic structure specific to individual parts within segment 17*, such as a syntactic structure that signals each of several parts of segment 17*, such as the DSCV SEI message described later. Conditionally signaling instruction 61 only to the first part, for example the lowest-ranked substream, prevents signaling information that is the same for all parts of the segment multiple times. In other words, when verifying a part of several parts that is not the first part in a hierarchical sequence, the device 20 may refrain from deriving instruction 61 from the data stream 14.

[0121] Figure 9 shows a device 10 for encoding a data stream according to one embodiment of the second aspect. The device 10 in Figure 9 may optionally correspond to the device 10 in Figure 3, that is, the device 10 in Figure 9 may be based on any of the embodiments described with respect to Figures 1 to 3. According to the embodiment in Figure 9, the encoder 10 makes the data stream 14 verifiable for reliability (see verification module 41'). The inserter 23 of the encoder 10 encodes the temporal sequence of segments 17, 17* into the data stream 14. The device 10 makes the current segment 17* of the sequence of segments verifiable in terms of reliability by applying a hash function 31 to a predetermined portion 13 of segment 17* to obtain a hash value 33. The inserter 23 of the device 10 inserts an instruction 61 into the data stream 14, which indicates, in a first alternative example, whether the current segment 17* is the first segment of a signed subsequence of the temporal sequence, or in a second alternative example, whether the current segment 17* is the last segment of a signed subsequence of the temporal sequence. According to this embodiment, a verification string 48 is formed based on the predetermined portion 13 and the instruction 61, for example, with respect to the verification module 41 in Figure 8. The verification string is used to obtain a digital signature 43 by signing the verification string 43, for example, using a secret key of an asymmetric cryptographic scheme. The inserter 23 inserts the digital signature 43 or its instruction into the data stream 14.

[0122] By relying on instruction 61 for the validation string, it is prevented that instruction 61 may be corrupted, for example, modified in an improper way that could weaken the reliability check, and in particular, that the benefit of including instruction 61 within the data stream 14, as described above, may be lost.

[0123] The following describes an embodiment of a third aspect of the present invention. Figure 10 shows a device 20 for decoding a data stream 14 according to one embodiment of a third aspect of the present invention. The device 20 in Figure 10 may optionally be an embodiment of the device 20 in Figure 1, that is, any of the details described with respect to Figures 1 to 3 may optionally be implemented in the device 20 in Figure 10. Furthermore, the features described with respect to Figures 7 and 8 may optionally be implemented in the device in Figure 10.

[0124] According to the embodiment of Figure 10, the data stream 14 includes a temporal sequence of segments 17, 17*, as described, for example, with respect to Figure 7 or Figure 8. According to Figure 10, the device 20 derives from the data stream 14 of segment 17* of the sequence of segments that are currently to be checked for reliability whether the verification chain in the temporal sequence of segments was interrupted at a segment. If the device 20 arrives at the result that the verification chain was interrupted at a segment, and the data stream 14 includes a digital signature 43 for checking segment 17* for reliability, the extractor 21 extracts the digital signature 43 from the data stream 14 to provide the digital signature 43 for the reliability check 41. The reliability check 41 may be performed as described above, but according to an embodiment of the third aspect, the formation of the verification string 48 is performed independently of the preceding video segment immediately preceding the segment currently being checked in the temporal sequence if the verification chain was interrupted at the current segment.

[0125] For example, a validation chain is a signed subsequence of a temporal sequence of segments. For example, a break may occur if a data stream is spliced ​​in the current segment, i.e., if the current segment does not belong to the same signal as the signal carried in the preceding segment of data stream 14. For example, in the case of media signals, a data stream may be spliced ​​to insert advertisements between segments of a media data stream, resulting in a data stream where the validation chain is interrupted at the splicing point. In general, splicing can refer to components of data stream segments that are not checkable together in terms of reliability.

[0126] In other words, one embodiment of the present invention is a device 20 for decoding a data stream 14, the data stream comprising a temporal sequence of segments 17 (e.g., coded video sequence (CVS)) (e.g., the segments form a temporal sequence with respect to the presentation order or presentation time assigned to the segments of the temporal sequence), and the device is configured to derive from the data stream, for a segment 17* of the sequence of segments, whether the verification chain (or signed subsequence of the temporal sequence) in the temporal sequence of the segments was interrupted in the segment 21, and if the data stream indicates that the verification chain was interrupted and the data stream includes a digital signature 43 for checking the segment for trustworthiness (e.g., a digital signature 43 for checking the segment or part thereof for trustworthiness) (e.g., the device is configured to check whether the data stream includes a digital signature for the segment, and if the data stream does not include a digital signature for the segment, the device does not check the segment for trustworthiness), derive from the data stream a digital signature 43 for checking against a verification string 48 independent of the preceding segment immediately preceding the segment 17* in the temporal sequence 21. According to one embodiment, the device is configured to verify a segment by checking whether a verification string 48 conforms to (or matches) a digital signature 43 (for example, to check the reliability of a data stream or a predetermined portion 13 thereof).

[0127] According to one embodiment, the device is configured to provide (or transfer) a predetermined portion 13 of a segment and a digital signature 43 for segment verification (for example, performed by a further device or entity) (e.g., verification includes checking whether a predetermined portion 13 of a data stream conforms to (or matches) the digital signature 43). In other words, one embodiment of the present invention is a device 20 for decoding a data stream 14, the device is configured to check the data stream for reliability (e.g., a device for validating a data stream), the data stream includes a temporal sequence of segments (e.g., coded video sequence (CVS)) (e.g., segments form a temporal sequence with respect to the presentation order or presentation time assigned to the segments of the temporal sequence), the device derives from the data stream whether a verification chain (or signed subsequence of the temporal sequence) in the temporal sequence of segments has been interrupted in the segment 21, and whether the data stream indicates that the verification chain has been interrupted If the data stream contains a digital signature 43 for a segment (for example, a digital signature 43 for checking the segment or a portion thereof for authenticity) (for example, the device is configured to check whether the data stream contains a digital signature 43 for a segment, and for example, if the data stream does not contain a digital signature for a segment, the device does not check the segment for authenticity), the device is configured to derive the digital signature 43 from the data stream 21, to form a verification string 48 independent of the preceding segment that is immediately preceding the segment in the temporal sequence 39, and to check whether the verification string 48 conforms to the digital signature 43 63.

[0128] Further optional selections of embodiments are described below. According to one embodiment, the device 20 may check whether the data stream 14 contains a digital signature for the current segment. For example, if it does not, the current segment will not be checked for trustworthiness. In other words, the current segment, which is inserted into a verification chain and thereby causes an interruption in the verification chain, may not be verifiable, i.e., it may not contain a digital signature. In these cases, the trustworthiness check may be omitted. According to one embodiment, the data stream 14 includes an instruction for the current segment indicating whether the validation chain was interrupted at the current segment. For example, the instruction may be a flag, such as a syntactic element like DSCI_splicing_flag, as described later. For example, the instruction may be included in the validation parameter set described above.

[0129] In Figure 10, the current segment is indicated using reference numeral 17*. The temporal sequence of the data stream 14 in Figure 10 includes segments 17 and 17* for illustrative purposes. Segment 17 belongs to a verification chain, such as a signed subsequence of the temporal sequence. Segment 17* is inserted between two segments 17, thereby causing a break in the verification chain at the current segment 17*. According to one embodiment, the device 20 can check whether segment 17* is checkable in terms of reliability, and if segment 17* is not checkable in terms of reliability, the device 20 can assume that the verification chain was interrupted at segment 17*.

[0130] In the reliability check 41, as shown in Figure 10, the verification string 48 is formed in block 39 based on a predetermined portion 13. For example, block 39 may include blocks 31 and 49 in Figure 2. Block 63 of the reliability check 41, as shown in Figure 10, checks whether the verification string 48 conforms to the digital signature 43. For example, block 63 may include block 59 in Figure 2 and a comparison between the check value 47 and the verification string 48.

[0131] According to one embodiment, the formation of the verification string 48 depends on whether the device 20 detects a break in the verification chain at segment 17*. If the verification chain is broken (i.e., the device 20 detects a break), and the data stream 14 contains a digital signature for segment 17*, the verification string 48 is formed by using a preceding hash value obtained by applying the hash function 31 to the portion of the preceding segment. If the device 20 detects a break, and the data stream 14 contains a digital signature for segment 17*, the verification string 48 is formed independently of any of the segments 17 that precede segment 17* in the temporal sequence.

[0132] According to one embodiment, the formation of the verification string 48 may further depend on whether segment 17* is the first segment of a further verification chain. If segment 17* is the first segment of a further verification chain, the verification string 48 is formed independently of any segments preceding segment 17* in the temporal sequence. In the example, if the segment is not the first segment of a further verification chain, the verification string 48 may be formed using a previous hash value, which is obtained by applying a hash function to a portion of a further preceding segment in the temporal sequence. For example, the further preceding segment is a segment belonging to a further verification chain. In other words, the sequence of the temporal sequence of the data stream 14 may include multiple verification chains that can be interleaved. The device 20 may check whether segment 17* is the first segment of a further verification chain at a splicing point, i.e., the point where the device 20 detects a break in the verification chain, i.e., segment 17*, so that the temporal consistency between segment 17* and its preceding segment in the further verification chain can be checked.

[0133] In more general terms, the device 20 may check whether segment 17* is the first segment in a further verification chain. For example, this check may be performed using instruction 61 as described with respect to Figure 8. Thus, the third embodiment may be combined with the second embodiment of the present invention. Alternatively, the device 20 may determine whether segment 17* is the first segment in a further verification chain based on the verification information signaled for segment 17* in the data stream 14. In other words, the device 20 may derive verification information for segment 17* from the data stream 14 and compare that verification information with preceding verifications for another further preceding segment or a further preceding segment in a temporal sequence. For example, the verification information may be information from the verification parameter set described above, such as a DSCI SEI message, or a portion of the information signal therein. If the verification information matches the preceding verification information, the device may infer that segment 17* is not the first segment in a further verification chain; otherwise, i.e., if the verification information does not match, for example, if it is not identical to the preceding verification information, the device 20 may infer that segment 17* is not the first segment in a further verification chain.

[0134] For example, a match between verification information and preceding verification information may mean that they are identical, or that an evaluation of the verification information results in the conclusion that the verification information and the preceding verification information are part of the same content or verification chain, for example, that they are subsequent segments in the same verification chain. For example, the content identifier of the verification information may be changed for each segment, which may allow checking whether a segment is a subsequent segment of the same content. For example, when determining whether segment 17* is the first segment in a further verification chain, the device 20 may compare the verification information of segment 17* with the verification information of multiple preceding segments in the temporal sequence, for example, all segments, or all segments except the immediately preceding segment. For example, the verification information may include one or more of the following: a certificate instruction containing a public key for decrypting the digital signature 43, a hash function instruction, and a segment content identifier that associates the segment with a media asset.

[0135] According to one embodiment, if the data stream 14 indicates that the verification chain has been interrupted at segment 17*, the device 20 may store the hash value of the preceding segment for verification of the segment following segment 17*. In other words, the device 20 may store the hash value obtained by applying a hash function to the portion of the preceding segment and use this hash value for subsequent verification of the subsequent segment belonging to the verification chain that was interrupted at segment 17*. In other words, the device 20 may continue the verification chain with the subsequent segment of the verification chain.

[0136] Figure 11 shows a device 10 for encoding a data stream according to one embodiment of a third aspect of the present invention. The device 10 in Figure 11 may be one embodiment of the device 10 in Figure 3. The device 10 in Figure 11 may provide a data stream 14 as described in the context of the device 20 in Figure 10. The device 10 according to Figure 11 is configured to make the data stream 14 trustworthy. The device 10 inserts a temporal sequence of segments 17, including, for example, segments 17, 17*, into the data stream 14. If the verification chain in the temporal sequence of segments is interrupted at segment 17* in the temporal sequence of segments, and segment 17* should be trustworthy, the device 20 makes segment 17* trustworthy by forming a verification string 48 independent of the preceding segment immediately preceding segment 17* in the temporal sequence. The device 10 obtains a digital signature 43 based on the verification string 48 and inserts the digital signature 43 into the data stream 14.

[0137] The following describes an embodiment of a fourth aspect of the present invention.

[0138] Figure 12 shows an apparatus 20 according to one embodiment of a fourth aspect of the present invention. The apparatus 20 in Figure 12 may optionally be an embodiment of the apparatus 20 in Figure 1, that is, any of the details described with respect to Figures 1 to 3 may be optionally combined with the apparatus 20 in Figure 12. As already stated above, embodiments of the fourth aspect may be combined with embodiments of the first aspect described with respect to Figures 4 to 7. According to the embodiment in Figure 12, the data stream 14 includes a plurality of payload packets 16, and further includes a plurality of supplementary information messages 19, for example, as described with respect to Figure 4. According to the embodiment in Figure 12, the apparatus 20 determines a predetermined portion 13 by checking whether a set of supplementary information messages among the plurality of supplementary information messages includes multiple instances of the same supplementary information message. If a set of supplementary information messages includes a set of multiple instances of the same supplementary information message, the portion determination 30 in Figure 12 includes only one instance of the same supplementary information message from the set of multiple instances in the predetermined portion 13. For example, further instances of the multiple instances of the same supplementary information message may be excluded from the predetermined portion 13. Reliability check 41 can be performed as described with respect to Figures 1 and 2.

[0139] For example, multiple identical instances of a supplemental information message may exist within the data stream 14 for the purpose of error correction or to make the data stream 14 more resilient to packet loss. However, if one of the multiple instances of a packet is lost and therefore cannot be included in a given portion 13, the reliability check 41 may fail. Therefore, in order to ensure not only that the data stream 14 is resilient to packet loss but also that the reliability check 41 is successful, the device 20 in Figure 12 contains only one instance of multiple identical supplemental information messages, and therefore, it is sufficient for one of the identical instances to be received in order for the verification check to be performed successfully.

[0140] In other words, one embodiment of the present invention is a device 20 for decoding a data stream 14, the device is configured to check the data stream for reliability (e.g., a device for validating the data stream), and the data stream includes a plurality of payload packets 16 (e.g., VCL NAL units) carrying payload data (e.g., encoded video data, where the data is information from which sample values ​​of a picture are reconstructed), and supplementary information packets 18 (e.g., SEI The device further includes NAL units) (for example, scattered between payload packets), and the supplemental information packet carries multiple supplemental information messages (for example, information that assists processes related to decoding, displaying, or other purposes, but is not required by the decoding process to determine the values ​​of samples in a decoded picture), and the device is configured to determine a given portion 13 of the data stream by checking whether a set of supplemental information messages among the multiple supplemental information messages contains multiple instances of the same supplemental information message (for example, repeated supplemental information messages), and if the set of supplemental information messages contains a set of multiple instances of the same supplemental information message, then to include only one instance of the same supplemental information message from the set of multiple instances (for example, excluding several further instances of the same supplemental information message at the same time), obtain a digital signature 43 based on the data stream (for example, using information derived from the data stream, for example, deriving a digital signature from the data stream or deriving a digital signature from an indicated source in the data stream), and check whether a given portion 13 of the data stream conforms to (or matches) the digital signature 43.

[0141] The decision of whether or not to include supplemental information messages in a given portion may be made in accordance with any of the criteria already described with respect to the first embodiment, for example with respect to Figures 4 to 7. For example, the set of supplemental information messages among multiple supplemental information messages that are checked for multiple instances of the same supplemental information message may be the verification set of supplemental information messages described with respect to Figure 4, i.e., the set of supplemental information messages that are considered to be included in the reliability check 41, or it may be a further set, such as a subset of the verification set. According to one embodiment, the positioning device positions one instance of a plurality of identical supplemental information messages at a predetermined position within a predetermined portion 13.

[0142] For example, a predetermined position is defined relative to a sample of a data stream, for example, the start or end position of a picture unit of a video data stream contained in a predetermined portion 13. Alternatively, a predetermined position may be defined relative to a predetermined payload packet, for example, by the type of payload packet contained in the predetermined portion 13, for example, in the case of a video data stream, it may be a predetermined coded video payload packet or a predetermined supplemental information packet.

[0143] According to one embodiment, the partial decision-maker 30 positions one or more supplemental information messages to be included in a given part 13 according to a predefined order among the one or more supplemental information messages within that part 13. In other words, for example, the partial decision-maker 30 can rearrange packets or supplemental information messages by comparing them to their order in the data stream 14. Therefore, the order within a given part 13 can be the same regardless of which of multiple instances of the same supplemental information message is included in the given part. According to one embodiment, the device 20 includes a set of supplemental information messages for a further predetermined portion of the data stream [e.g., the entire data stream, or one or more validation substreams of the data stream], wherein the supplemental information message is one [e.g., any one] of a set of supplemental information message types [e.g., a set of types that are predefined or signaled in the data stream, for example, to be included in a predetermined portion]. According to another embodiment, the device 20 includes in a set all supplemental information messages for a further predetermined portion of the data stream [e.g., the entire data stream, or one or more validation substreams of the data stream], where the supplemental information messages are one [e.g., any one] of a set of supplemental information message types [e.g., a predefined set of types that should be included in a predetermined portion, for example, a set that is signaled in the data stream].

[0144] According to one embodiment, a further predetermined portion includes or consists of data from one access unit of a data stream [e.g., all coded video data or all data from the video data stream relating to one time frame of a video], or data from one picture unit [e.g., all coded video data or all data from the video data stream relating to one picture of a video]. According to one embodiment, a further predetermined portion includes or consists of data from one validation substream of the data stream. For example, in Figure 12, the data stream 14 includes supplemental information messages 19, 19', 19'', and 19''' for illustrative purposes, where these supplemental information messages 19 and 19' may be identical, and supplemental information messages 19'' and 19''' may be identical. For example, if, according to further selection criteria, these supplemental information messages should be included in a predetermined portion 13, the portion decision-maker 30 includes one instance of each identical message, for example, messages 19 and 19'' in Figure 12, in the predetermined portion 13.

[0145] Referring to Figure 3, a device 10 for encoding a data stream according to a fourth aspect of the present invention will be described. According to one embodiment of the fourth aspect of the present invention, the device 10 determines a predetermined portion 13, as described with respect to Figure 11, that is, by including only one instance of the same supplemental information message. Then, as described with respect to Figure 3, a determination of a digital signature 43 is performed based on the predetermined portion 13.

[0146] The following describes an embodiment of a fifth aspect of the present invention. Referring to Figure 3, an apparatus 10 for encoding a data stream 14 according to a fifth aspect of the present invention will be described. According to one embodiment of the apparatus 10 according to the fifth aspect, the apparatus 10 encodes one or more data sequences within the data stream 14. For example, the data sequences may be temporal subsequences of the data stream 14. For example, a data sequence can be a coded media sequence, such as a section or segment of a media signal. For instance, if the data stream is a video data stream, the data sequence is a coded video sequence (CVS) or coded layer video sequence (CLVS).

[0147] For example, in the case of a video data stream, the data sequence may be a portion of the video data stream, which includes a sequence of video pictures extending from, for example, an access point from which the video data stream can be decoded (for example, independently decoded without requiring information from a previous coded video sequence of the video data stream) to a subsequent access point of the video data stream. According to this embodiment of the fifth aspect, the data sequence includes a plurality of payload packets and a plurality of supplemental information packets carrying supplemental information messages. According to this embodiment, the device 10 makes the data sequence trustworthy by determining a predetermined portion of the data sequence and generating a digital signature 43 based on the previously determined portion, as described with respect to Figure 3.

[0148] According to an embodiment of the fifth aspect, the device 10 is configured to provide a data stream such that each unit of the data stream, which is of a predetermined unit type, contains at most one identical instance of a supplemental information message of a predetermined message type. For example, identical instances may mean that two messages are identical in terms of content, e.g., they have the same content or carry the same payload. In other words, not only are the types of supplemental messages identical, but the information they convey is also identical.

[0149] For example, providing a data stream as described above means that the device 10 ensures that the data stream is constructed such that each unit of a predetermined unit type carries at most one identical instance of a supplemental information message of a predetermined message type. In other words, the data stream 14 may be provided or constructed such that none of the units of a given unit type contain two or more instances of a given message type of supplemental information message whose payload data is identical. In other words, one embodiment of the present invention is a device 10 that encodes a data stream 14 (e.g., the data stream is divided into units), the device including a plurality of payload packets 16 (e.g., VCL NAL units) that carry payload data (e.g., encoded video data, e.g., video data is information from which sample values ​​of pictures of a video are reconstructed) (e.g., each part includes one or more payload packets 16), and supplementary information packets 18 (e.g., SEI Encoding a data sequence (e.g., coded media sequence, e.g., unit of media signal, e.g., coded video sequence (CVS), or coded layer video sequence (CLVS)) (e.g., a portion of a video data stream, the portion of which includes a sequence of video pictures extending from an access point from which the video data stream can be decoded (e.g., independently decoded without requiring information from a preceding coded video sequence of the video data stream) to a subsequent access point of the video data stream) (e.g., CVS or CLVS), wherein the supplemental information packet is The device is configured to encode 23 and carry supplemental information messages (e.g., information that assists processes related to decoding, displaying, or other purposes, but is not required by the decoding process to determine the values ​​of samples in a decoded picture) (e.g., each supplemental information packet is part of one of several parts, each part containing one or more supplemental information packets), and to make the data sequence checkable for reliability (or digitally sign the data sequence) 41', and the device further includes providing a data stream such that each unit of a predetermined message type (e.g.,The data stream is configured to provide such a data stream that it contains at most one identical instance of a supplemental information message of a given type (for example, identical in terms of content, for example, having identical content or carrying the same payload) (for example, it is configured to ensure that the data stream is constructed in such a way that two instances of a supplemental information message are considered identical if they carry the same payload, for example, if the entire payload data contained in the two messages is identical) (for example, in other words, the data stream is provided such that no unit of a given unit type contains two or more instances of a supplemental information message of a given message type whose payload data is identical).

[0150] For example, a given unit type could be one of the following: 1. An encoded video sequence containing an encoded sequence of video pictures within a data stream. 2. An encoded layered video sequence containing an encoded sequence of pictures of video layers encoded within a data stream. For example, a layer includes one of several representations of a video signal within a video data stream, such as the layer described above. 3. A time frame, for example, an access unit (AU), contains video data belonging to one time instance of the video. 4. Pictures. For example, one time frame may contain multiple pictures belonging to different layers, for example, belonging to different representations of a video. 5. A decoding unit that represents the picture portion.

[0151] According to one embodiment, a supplemental information message of a predetermined message type carries information about one of the following: - Picture timing, for example, providing timing information about the picture level to the virtual reference decoder (HRD). - Noise characteristics in film grain synthesis, - Arrangement of packed frames, for example, arranging two stereo views side by side within an encoded picture, - A projection type that uses equirectangular projection to create an omnidirectional camera view of an encoded picture.

[0152] For example, in HEVC, a given message type can be one of the following:

[0153] 0 buffer period 1. Picture Timing Give HRD timing information regarding picture levels. 19 film_grain_characteristics Describe the noise characteristics in film grain synthesis. 45 Frame Packing Arrangement This describes the arrangement of packed frames, for example, the arrangement of two stereo views side by side within an encoded picture.

[0154] 47 Display orientation 56 Green Metadata 129 Parameter set inclusion 132 decoded_picture_hash 133 scalable_nesting 137 mastering_display_colour_volume 142 colour_transform_info 144 content_light_level_info 145 dependent_rap_indication 147 alternative_transfer_characteristics 148 ambient_viewing_environment 149 content_colour_volume 150 equirectangular_projection This describes a projection type that uses equirectangular projection to create an omnidirectional camera view of a coded picture. 153 generalized_cubemap_projection 154 sphere_rotation( 155 regionwise_packing 156 omni_viewport( 165 alpha_channel_info 168 frame_field_info 177 research_representation_info 179 multiview_acquisition_info 180 multiview_view_position 200 sei_manifest 201 sei_prefix_indication 202 annotated_regions 203 subpic_level_info 204 sample_aspect_ratio_info 205 shutter_interval_info 206 extended_drap_indication 207 constrained_rasl_encoding_indication 208 scalability_dimension_info 209 vdi_sei_envelope 210 nn_post_filter_characteristics 211 nn_post_filter_activation 212 phase_indication

[0155] According to one embodiment, there exists a set of message types relating to the device 20 providing the data stream 14, as described for a predetermined message type. That is, the device 20 according to this embodiment provides the data stream 14 such that, for each message type in the set of message types, each unit of the video data stream, which is a predetermined unit type, contains at most one identical instance of the supplemental information message of each message type. A predetermined message type is one of the set of message types. For example, this set of message types may be predetermined, for example, as defined by a codec. For example, the set of message types is defined by the syntax element PicUnitRepConSeiAuthList, which will be described later.

[0156] According to one embodiment, a set of message types is signaled in the data stream 14, for example, by syntactic elements. According to one embodiment, the device 10 inserts a supplemental information packet into the data sequence containing information for verifying the data sequence, such as the verification parameter set described above. In other words, the information for verifying the data sequence may include a hash function instruction, a certificate for decrypting a digital signature, and one or more of several verification substreams.

[0157] The apparatus 10 according to the fifth embodiment can make the data sequence verifiable in terms of reliability by, as described with reference to Figure 3, by determining a predetermined portion of the data sequence and obtaining a digital signature 43 based on the predetermined portion 13. Further details that make the data sequence verifiable in terms of reliability can optionally be applied equivalently to the explanations of blocks 30' and 41' in Figure 3.

[0158] According to one embodiment, the device 10 determines a set from the payload packets of a data sequence. For example, the set includes all payload packets belonging to a substream of the data sequence, for example, a substream of video encoded within the data sequence, for example, encoded video payload packets. The partial determiner 30' may include the set of payload packets in a predetermined part 13, and may further include in the predetermined part 13 all supplementary information messages that are of a predetermined message type or any of the message types of a set of message types, associated with the payload packets 16 of the set of payload packets, as described above.

[0159] In other words, supplementary information messages associated with payload packets that should be included in a predetermined portion 13 can be included in that predetermined portion regardless of whether or not multiple instances exist. In a further embodiment, supplemental information messages associated with payload packets are generally not included in a predetermined portion, and only certain types of supplemental information messages are always included in a predetermined portion. That is, the inserter 23 may insert instructions into the data stream 14 that indicate a further set of message types of supplemental information message types. The further set of message types indicates the message types that should be included in a predetermined portion 13. In this embodiment, the portion determiner 30' includes all supplemental information messages that are any of the message types from the further set of message types associated with the payload packets of the set of payload packets in the predetermined portion 13. For example, certain supplemental information message types may be excluded from consideration for further sets of message types, such as supplemental information messages for filler data types, or messages signaling filler data. In other words, it may be prohibited to include messages of filler data type in further sets of message types.

[0160] According to one embodiment, the data sequence is verifiable in units of several verifiable parts, each containing a predetermined portion 13. For example, the verifiable portion, i.e., the portion and data sequence described with respect to Figure 7, may be the segment 17 described with respect to Figure 7. According to one embodiment, a supplementary information message, which is a further predetermined message type, is assigned to a predetermined verifiable part among several verifiable parts. For example, a further predetermined message type is a message type in which a predetermined unit includes multiple identical instances. For example, a further predetermined message type is one of payloadType=4 (user data registered) and payloadType=5 (user data not registered).

[0161] According to one embodiment, the partial decision-maker 30' assigns all instances of supplemental information messages to a predetermined type of unit, which is a unit containing multiple identical instances of supplemental information messages of a further predetermined message type, into a predetermined verifiable portion. All of several verifiable portions can be checked with respect to reliability, for example, as described with reference to Figure 7. For example, a given verifiable part may be the highest-ranked verifiable part according to an order defined among several verifiable parts.

[0162] Making certain verifiable parts checkable in terms of reliability can be done sequentially according to a defined order among the verifiable parts, as explained with respect to Figure 7, for example. According to one embodiment, each unit is one of a set of unit types. According to one embodiment, the unit types are ordered according to a hierarchical order, and the first unit type, which is a unit of a first unit type having a lower level in the hierarchical order, is included in the unit of a second unit type, which is a unit of a second unit type having a higher level in the hierarchical order.

[0163] According to one embodiment, the constraint of inserting only one instance of a particular supplemental information message may apply only to data sequences that should be verifiable in terms of reliability. According to one embodiment, a set of unit types includes one or more of the following: -An encoded video sequence, each containing an encoded sequence of video pictures, or -An encoded layered video sequence, each containing an encoded sequence of pictures from a video layer [for example, a layer containing one of several representations of a video signaled by a data stream, e.g., multiview video quality (e.g., resolution) or viewpoint, or data type (e.g., depth data vs. luminance / chromaticity data)], or - Time frame (e.g., Access Unit (AU)) (e.g., each time frame contains video data belonging to one time instance of the video), or - Pictures (for example, one time frame may contain multiple pictures belonging to different layers, for example, belonging to different representations of a video), or - A decoding unit (for example, a decoding unit representing a portion of a picture).

[0164] According to one embodiment, the constraint of inserting only one instance of a particular supplemental information message may apply only to data sequences that should be verifiable in terms of reliability. In other words, according to one embodiment, in an operating mode in which a predetermined type of supplemental extended information message should be repeated,

[0165] [For example, depending on the operational settings] if the data sequence should be checkable for reliability, the data sequence is made checkable for reliability and the data stream is provided such that each unit, which is a unit of a predetermined unit type of the data stream [e.g., CVS, CLVS, AU, PU, ​​DU], contains at most one identical instance of a supplemental information message of a predetermined message type [e.g., a predetermined type of supplemental information message] [for example, configured to ensure that the data stream is constructed correctly], [For example, depending on the operating settings] if the data sequence should not be checkable for reliability, insert multiple identical instances of a supplemental information message of a given message type [for example, a given type of supplemental information message] into the data stream. It is structured in this way.

[0166] An apparatus for decoding a data stream according to one embodiment of a fifth aspect of the present invention will be described with reference to Figure 1. According to the embodiment of the fifth aspect, the apparatus 20 checks whether each unit of the data stream, which is of a predetermined unit type, contains at most one identical instance of a supplemental information message of a predetermined message type. For example, if, in response to a check, each unit of a data stream of a given unit type contains more than one identical instance of a given message type of supplemental information message, the decoder may reject the data stream as non-conforming or suppress validation of the data stream if it finds that a unit contains more than one identical instance of such supplemental information message. In other words, one embodiment of the present invention is a device 20 for decoding a data stream 14, the device being configured to decode a data sequence from the data stream, which includes a plurality of payload packets 16 carrying payload data and further supplemental information packets 18, wherein the supplemental information packets carry supplemental information messages, and to decode and make the data sequence checkable for reliability, the device being configured to check whether each unit of the data stream, which is of a predetermined unit type, contains at most one identical instance of a supplemental information message of a predetermined message type (for example, if, in response to the check, a unit of the data stream, which is of a predetermined unit type, contains more than one identical instance of a supplemental information message of a predetermined message type, the device rejects the data stream as non-compliant (or suppresses the validation of the data stream)).

[0167] All further embodiments and features of the apparatus 10 described in relation to the fifth aspect can also be optionally applied to the apparatus 20, in particular, in a method for determining a predetermined portion 13. Although the descriptions in Figures 1 to 12 relate to the apparatus, the block diagrams in these figures can be considered alternatively to flow diagrams of each method, where each block represents a step in that method. Therefore, Figures 1 to 12 further provide examples of each method. The embodiments of the present invention will be described again below, in other words, specific and further embodiments of the present invention will be described. The embodiments described with respect to Figures 1 to 12 may be considered generalizations of the embodiments described below, but the following description may further include additional embodiments of the present invention that can be carried out independently of the embodiments described above. Any of the features and details described with respect to the embodiments described below may be optionally incorporated into the embodiments described above.

[0168] Although the following explanation refers to video data streams, it is clear that the same concepts can be applied equivalently to any type of media data stream, or any type of data stream that includes sampled data. A further example of sampled data may be waveform signals. In addition to audio signals, another example of waveform signals is biomedical waveform signals. To authenticate a video, the client must be provided with information about the certificate carrying the public key that may be used to verify and provide one or more signatures of the video, including how the hashing was performed (including the hashing method used, how the data is organized, and what was actually hashed and how).

[0169] The following describes existing solutions for content validation. Subsequent embodiments of the present invention may be described in terms of modifications to these existing solutions, and therefore, details of these existing solutions may form features of embodiments of the present invention. For this purpose, existing solutions involve including three SEI messages within each coded video sequence (CVS) of the video bitstream. 1. Digitally Signed Content Initialization SEI Message: Carries a C2PA manifest containing information such as the hashing method used, a URL to obtain a certificate, or the public key and UUID of the content used when calculating the signature, so that different bitstreams (e.g., different audio and video streams) that are individually signed and belong to the same content can be identified as belonging to the same content (thus preventing the use of audio from different content in a particular video, and avoiding the use of something someone said at a different time in a new video), and also carries the number of substreams used to generate the signature. The latter refers to splitting the content into different substreams, each with its own signature, so that if less important parts of a video are dropped (e.g., to layer-encode a high-resolution enhancement layer in case of network congestion), the received parts can still be authenticated. 2. Digitally signed content selection SEI message: Identifies the substream ID to which the picture slice (i.e., VCL NAL unit) belongs. 3. Digitally signed content verification SEI message: Provides a signature corresponding to each substream.

[0170] The existing solutions for these three SEI messages and verification processes are reproduced below. Details of these solutions may be optionally implemented in embodiments of the present invention, for example, in terms of modifications to these solutions, unless otherwise specified. Digitally signed content initialization SEI message: JPEG2026086353000002.jpg78168 The dsci_hash_method_type indicates the secure hash algorithm used to compute message digests for a subset of SPS, PPS, APS, PH, and VCL NAL units of an encoded video sequence. Based on these message digests and the digital signature present in the digitally signed content verification SEI message, the decoder can verify that the encoded video was generated by the content originator indicated by dsci_key_register_idx if the flag in dsci_use_key_register_idx_flag is equal to 1. The supported values ​​for the syntactic element dsci_hash_method_type, the block size used to compute the message digest, and the size of the computed message digest are specified. Values ​​for dsci_hash_method_type not listed in Table 1 are reserved for future use by ITU-T|ISO / IEC and should not be present in payload data conforming to this version of this specification. The decoder shall ignore trusted initialization SEI messages containing reserved values ​​for dsci_hash_method_type. The secure hash algorithms listed in Table 1 are specified in the Secure Hash Standard FIPS PUB 180-4.

[0171] Table 1 - Support values ​​for dsci_hash_method_type JPEG2026086353000003.jpg51170 dsci_key_source_uri contains a URI having the syntax and semantics specified in IETF Internet Standard 66. If dsci_key_retrieval_mode_idc is equal to 0, dsci_key_source_uri specifies a C2PA manifest store as defined in the C2PA Technical Specification. If dsci_key_retrieval_mode_idc is equal to 1, the following applies: -If dsci_use_key_register_idx_flag is equal to 0, the URI identifies the content provider's certificate which may be used to verify the signature present in the following digitally signed content verification SEI message. - If not (dsci_use_key_register_idx_flag is equal to 1), the URI identifies the certificate register and the content provider's certificate that can be used to verify the signature present in the following digitally signed content verification SEI message, as indicated by dsci_key_register_idx.

[0172] Adding 1 to dsci_num_verification_substreams_minus1 indicates the number of substreams in which a signature may exist within the following digitally signed content verification SEI message, calculated from the message digest. The variable NumVerificationSubstream is derived as follows: NumVerificationSubstream=dsci_num_verification_substreams_minus1+1. A dsci_key_retrieval_mode_idc equal to 0 indicates that the URI contained in dsci_key_source_uri specifies a C2PA manifest store as defined in the C2PA Technical Specification. A dsci_key_retrieval_mode_idc equal to 1 indicates that the URI contained in dsci_key_source_uri, and dsci_key_register_idx, if present, specify a certificate. In this version of the Specification, dsci_key_retrieval_mode_idc shall be in the range of 0 to 1. The decoder shall also accept other values ​​for dsci_key_retrieval_mode_idc, but shall ignore the content of the digitally signed content initialization SEI message, associated digitally signed content selection SEI message, and associated digitally signed content verification SEI message.

[0173] A dsci_use_key_register_idx_flag equal to 1 indicates that the URI contained in dsci_key_source_uri specifies a certificate register and that the syntax element dsci_key_register_idx is present in the SEI message. A dsci_use_key_register_idx_flag equal to 0 indicates that the URI contained in dsci_key_source_uri specifies a certificate and that the syntax element dsci_key_register_idx is not present in the SEI message. When dsci_key_retrieval_mode_idc is equal to 0, the media asset for which the active manifest provides content binding is a digitally signed content initialization SEI message, as specified in the C2PA technical specification. The following constraints apply to the C2PA manifest store identified by dsci_key_source_uri.

[0174] - The active manifest shall contain exactly one c2pa.hash.data, which has strong binding force on content assertions as specified in the C2PA technical specifications. The exclusion range specified in -c2pa.hash.data shall match the dsci_key_source_uri byte in the digitally signed content initialization SEI message. dsci_key_register_idx contains an index that specifies the content provider's certificate in the certificate register indicated by dsci_key_source_uri, which, if present, can be used to verify the signature present in the following digitally signed content verification SEI message.

[0175] The certificate indicated by dsci_key_register_idx when the syntax elements dsci_key_retrieval_mode_idc, dsci_use_key_register_idx_flag, dsci_key_source_uri, and dsci_use_key_register_idx_flag are equal to 1 shall specify the digital signing method using the relevant parameters (if applicable) and the public key of the content provider. When dsci_key_retrieval_mode_idc is equal to 1, the format in which this information is provided is outside the scope of this specification. It is proposed to use a digital signature algorithm compliant with FIPS 186-5, “Digital Signature Standard”. A dsci_content_uuid_present_flag equal to 1 indicates that the syntactic element dsci_content_uuid exists. A dsci_content_uuid_present_flag equal to 0 indicates that the syntactic element dsci_content_uuid does not exist. When dsci_key_retrieval_mode_idc is equal to 0, dsci_content_uuid_present_flag is considered equal to 1.

[0176] dsci_content_uuid, if present, indicates the identifier of the video content and shall have a value specified as a UUID in accordance with ISO / IEC 11578:1996, Annex A.

[0177] If a digitally signed content initialization SEI message exists within the AU, the calculation of the NumVerificationSubstream message digest is initialized according to the FIPS PUB 180-4 specification for the specified dsci_hash_method_type. Each SPS, PPS, APS, PH, and VCL NAL unit following the digitally signed content initialization SEI message is associated with one of the NumVerificationSubstream message digests, where the verification substream ID is either indicated by the digitally signed content selection SEI message or inferred to be equal to 0 if no digitally signed content selection SEI message exists for the PU. The message used to calculate the k-th message digest is obtained by concatenating all SPS, PPS, APS, and VCL NAL units associated with the k-th verification substream, where k is in the range of 0 or greater and dsci_num_verification_substreams_minus1 or less. The message digest calculation is block-based, and the block size is specified in Table 1 depending on the value of dsci_hash_method_type. For each SPS, PPS, APS, PH, and VCL NAL unit, the associated message digest is updated according to the algorithm specified in FIPS PUB 180-4 for the given dsci_hash_method_type. Note that since the message digest is computed for the concatenation of all SPS, PPS, APS, PH, and VCL NAL units in the verification substream, some of the processing blocks will typically span two or more consecutive NAL units.

[0178] Digitally signed content selection SEI message: JPEG2026086353000004.jpg18166 dscs_verification_substream_id indicates the verification substream to which the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture are assigned. If a digitally signed content initialization SEI message is present in the current coded video sequence, but a digitally signed content selection SEI message is not present in the coded picture, the value of dscs_verification_substream_id is inferred to be equal to 0. The value of dscs_verification_substream_id must be greater than or equal to 0 and less than or equal to dsci_num_verification_substream_minus1.

[0179] As specified below, the message digest of the verification substream having an ID equal to dscs_verification_substream_id is updated with the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture, according to the dsci_hash_method_type specified in the preceding digitally signed content initialization SEI message.

[0180] Digitally signed content verification SEI message: JPEG2026086353000005.jpg29168 dscv_verification_substream_id indicates the verification substream to which the SEI message applies.

[0181] Adding 1 to dscv_signature_length_in_octets_minus1 specifies the length of the syntax element dscv_signature in octets (one octet consists of 8 bits). dscv_signature contains the digital signature for the verification substream indicated by dscv_verification_substream_id.

[0182] The verification of a bitstream signature consists of the following ordered steps: 1. The calculation of the message digest, known as CurrDigest, is ultimately determined as follows: -The concatenation of SPS, PPS, APS, PH, and VCL NAL units for the verification substream having an ID equal to dscv_verification_substream_id is padded according to the FIPS PUB 180-4 specification. Note that it is sufficient to pad only the last NAL unit of the verification substream. - The calculation of the message digest, CurrDigest, is finalized according to the FIPS PUB 180-4 specification. The length of the message digest (in bits) is shown in Table 1.

[0183] 2. The reference message digest RefDigest is determined as follows: -If dscv_verification_substream_id is greater than 0, the reference message digest RefDigest is the last calculated message digest of the verification substream having an ID equal to dscv_verification_substream_id-1. A bitstream conformance requirement is that any digitally signed content verification SEI associated with a verification substream ID equal to dscv_verification_substream_id-1 exists before the digitally signed content verification SEI message having a verification substream ID equal to dscv_verification_substream_id. -Instead, if the current digitally signed content verification SEI message is the first digitally signed content verification SEI in the coded video sequence with a verification ID equal to 0, and the preceding coded video sequence did not contain any digitally signed content initialization SEI messages (including the case where the current coded video sequence is the first coded video sequence in the bitstream), then RefDigest is set to equal a bit string consisting of DigestSize bits equal to 1, where DigestSize is the size of the message digest as specified in Table 1. - Otherwise, the reference message digest RefDigest is the last calculated message digest of the validation substream that has an ID equal to 0.

[0184] 3. The identifier string IdString is constructed by concatenating the binary representation of the reference message digest RefDigest, the current message digest, and dsci_hash_method_type, as well as dsci_content_uuid2 if present. The number of bits in RefDigest is determined by the value of dsci_hash_method_type that was valid when calculating the value of RefDigest, and the number of bits in CurrDigest is determined by the current value of dsci_hash_method_type, the value of dsci_hash_method_type is represented by 8 bits, and if present, the value of dsci_content_uuid is represented by 128 bits.

[0185] 4. The IdString represents the message used to verify the signature. The signature verification algorithm and the public key used to verify the signature are indicated by the syntax elements dsci_use_key_register_idx_flag, dsci_key_source_uri, and, if dsci_use_key_register_idx_flag is equal to 1, dsci_key_register_idx.

[0186] Note 1 - Since the bit stream used for signature verification includes RefDigest, it can be verified that the SPS, PPS, APS, PH, and VCL NAL units used to calculate the current message digest are correct, and it can also be additionally verified that no additional SPS, PPS, APS, PH, and VCL NAL units have been added to the bitstream and no SPS, PPS, APS, PH, and VCL NAL units have been removed from the bitstream. Note 2 - When the decoder tunes to the bitstream, it cannot correctly calculate the value of RefDigest, and therefore cannot verify the configured IdString for the first digitally signed Content Verification SEI message. However, it is possible to verify the signature starting from the second digitally signed Content Verification SEI message. After verification, the message digest of the verification substream with an ID equal to dscv_verification_substream_id is reinitialized according to the FIPS PUB 180-4 specification for the specified dsci_hash_method_type.

[0187] Embodiments of the present invention will be described below. It is not sufficient to simply authenticate data that directly affects the picture decoded by the decoder, namely VCL NAL units. Several processes can be linked to the video data by using metadata that describes post-processing steps that can significantly alter the content. One example is a neural network post-filter SEI message, which can output a picture very different from the one output by the decoder using only VCL NAL units. Furthermore, some descriptive metadata, such as copyright or content labeling, may not affect the output sample itself, but could potentially alter important information that has other implications.

[0188] Another angle of attack could involve face generation techniques that could be manipulated and exploited to illegally alter decoded pictures. Furthermore, masking / concealment attacks, i.e., concealing decoded samples within the output, could potentially be carried out by introducing malicious film grain or manipulated display overlays. Additionally, all frame-packing-related SEI messages could be manipulated through malicious means in their respective applications to mask or conceal decoded samples. Furthermore, SEI messages with registered user data and those without should be signed to ensure the authenticity of application-specific solutions.

[0189] It may not always be important to still sign such metadata with VCL NAL units, because it prevents some metadata from being irrelevant and some systems that do not need such metadata from dropping it, which goes against the design of video encoding standards, which would allow such data to be placed in a less important bitstream and dropped without affecting the decoding process.

[0190] The same applies to the repetition of SEI messages, which is a robustness measure in video encoding standards that allows for the repeated placement of several SEI messages with the same content within a particular portion of the bitstream. Any such repetition makes it possible to be robust against packet loss affecting redundant data, and any solution for media authenticity must be robust to the same extent, i.e., the ability to test the authenticity of received data should not be hindered by the loss of redundant data.

[0191] Further embodiments according to the first aspect of the present invention will be described below. In the first embodiment, information is added to the bitstream that allows for the identification of whether or not such metadata (i.e., SEI message) should be considered for hashing / signature calculation. Its instantiation is shown below. Digitally signed content initialization SEI message A dsci_sei_messages_digitally_signed_flag equal to JPEG2026086353000006.jpg821651 specifies that non-VCL NAL units corresponding to CVS SEI messages are used to compute the message digest. A dsci_sei_messages_digitally_signed_flag equal to 0 specifies that non-VCL NAL units corresponding to CVS SEI messages are not used to compute the message digest. In this regard, there are various options for how to handle the numerous SEI message types.

[0192] In one embodiment, if the signaling indicates that SEI messages should be considered in order to sign the video bitstream by dsci_sei_messages_digitally_signed_flag, then all SEI messages are included in the message digest. Alternatively, this process may apply only to a predefined set of SEI messages identified as relevant to signing. For example, in one embodiment, the video coding standard may define a list RelevantForSigningSEIMessages containing one or more SEI message types, such as those listed below.

[0193] • NN Post-Filter related: NNPFC, NNPFA SEI messages, Film grain synthesis: FGS • SEI processing sequence that instructs the order of post-processing of SEI messages: SEI message: SPO • Masking related: FGS SEI messages, display overlay SEI messages, • Face generation related: GFV SEI message, GFVE SEI message • Packing related: Frame packing arrangement SEI message, packed area information SEI message, packing SEI message per area, configuration rectangle SEI message • SEI messages indicating registered and unregistered user data • Law, regulation, and annotation-related SEI messages: Copyright, content labeling, or text description SEI messages, AI usage restriction SEI messages In an alternative embodiment, the SEI messages to be included in the authenticity processing can also be identified through a range of values ​​for an SEI message type that can also host reserved space for future SEI messages.

[0194] As a further alternative embodiment, signaling can indicate different possible combinations of SEI messages to be signed. For example, this can be done by adding syntactic elements that point to different combinations, as follows: A dsci_sei_messages_digitally_signed_flag equal to JPEG2026086353000007.jpg921641 specifies that non-VCL NAL units corresponding to CVS SEI messages are used to compute the message digest. A dsci_sei_messages_digitally_signed_flag equal to 0 specifies that non-VCL NAL units corresponding to CVS SEI messages are not used to compute the message digest.

[0195] dsci_sei_messages_digitally_signed_idc identifies the SEI message used to calculate the message digest as specified in Table X. The value of dsci_sei_messages_digitally_signed_idc must be in the range from 0 to Y. JPEG2026086353000008.jpg59143 Instead of using two syntactic elements, you can use only `idc` if the value 0 means that no SEI messages are used in the message digest.

[0196] As a further alternative, the SEI type used to compute the message digest can be signaled in the bitstream. An example is shown below. JPEG2026086353000009.jpg117165 The value of dsci_num_sei_message_types_digitally_signed_minus1 plus 1 specifies the number of SEI message types used to calculate the message digest. dsci_digitally_signed_sei_type[i] identifies the i-th SEI type used to compute the message digest.

[0197] Note that if the type value is greater than 255, the SEI message type can use more than 8 bits. Therefore, the above syntax can be extended to have 16 bits instead of 8 bits. Alternatively, a further syntax element (e.g., dsci_digitally_signed_sei_type_length_inbytes_minus1) can be added to indicate the length of the syntax element dsci_digitally_signed_sei_type[i] (e.g., 8*(dsci_digitally_signed_sei_type_length_inbytes_minus1+1) bits). Alternatively, instead of pre-identifying which SEI messages will be used for message digests, each SEI message used for message digests can be preceded by a prefix SEI message, or nesting SEI messages containing any SEI message can be used to indicate that such SEI messages will be used for message digests.

[0198] It should be noted that SEI messages are encapsulated within a single SEI NAL unit, which may contain more than one SEI message. Currently, authentication is performed based on the NAL unit (the entire NAL unit). In the case of SEI messages, in one embodiment only the SEI message may be considered, but as an alternative, the entire SEI NAL unit may also be considered. For this purpose, it may be useful to determine whether or not an SEI NAL unit is used based on a first SEI message. For this purpose, a constraint is applied that SEI messages with SEI message types that should not be validated should not be included in the same NAL unit as SEI messages with SEI message types that should be validated.

[0199] A further aspect to consider is the assignment of SEI messages to specific substreams. Different options exist. 1. SEI messages belong to the highest-level substream ID specified. 2. SEI messages belong to the same substream ID as the PU containing such SEI messages. 3. When an SEI message is instructed to be used for message digest / signature generation, the number of substreams is twice the number signaled, and the substreams of the SEI message are instructed by a digitally signed content selection SEI message.

[0200] Regarding option 1, the following changes may be necessary. 1.2 Digitally Signed Content Selection SEI Message 1.2.1 Digitally Signed Content Selection SEI Message Syntax JPEG2026086353000010.jpg18159 1.2.2 Digitally Signed Content Selection SEI Message Semantics A digitally signed content selection SEI message provides a mechanism for associating an encoded picture with one of the verification substreams specified within a digitally signed content initialization SEI message.

[0201] [...] dscs_verification_substream_id indicates the verification substream to which the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture are assigned. If a digitally signed content initialization SEI message is present in the current coded video sequence, but a digitally signed content selection SEI message is not present in the coded picture, the value of dscs_verification_substream_id is inferred to be equal to 0 for the SPS, PPS, APS, PH, and VCL NAL units. When dsci_sei_messages_digital_signed_flag is equal to 1, the value of dscs_verification_substream_id is inferred to be equal to dsci_num_verification_substream_minus1 of the SEI messages specified by the DSCI SEI message (where the subset of SEI messages used for message digesting may be specified by the DSCI SEI message). The value of dscs_verification_substream_id must be between 0 and dsci_num_verification_substream_minus1.

[0202] As an alternative, the syntax of the DSCI SEI message can be extended as follows to enable signaling of whether option 1 (the top-level substream of the SEI message) or option 2 (the same SEI message substream as the VCL NAL unit of the coded picture) is being used. JPEG2026086353000011.jpg87170 The dsci_sei_messages_digital_signed_highest_substream_flag equal to 1 specifies that the SEI message used to calculate the message digest is assigned to the highest verification substream. The dsci_sei_messages_digital_signed_highest_substream_flag equal to 0 specifies that the SEI message used to calculate the message digest is assigned to the same verification substream as the VCL NAL unit of the associated coded picture.

[0203] 1.2.2 Digital Signed Content Selection SEI Message Semantics The digital signed content selection SEI message provides a mechanism for associating a coded picture with one of the verification substreams indicated in the digital signed content initialization SEI message. [...] The dscs_verification_substream_id indicates the verification substream to which the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture are assigned. If the digital signed content initialization SEI message exists within the current coded video sequence but the digital signed content selection SEI message does not exist in the coded picture, the value of the dscs_verification_substream_id is inferred to be equal to 0 for the SPS, PPS, APS, PH, and VCL NAL units. The value of the dscs_verification_substream_id shall be greater than or equal to 0 and less than or equal to dsci_num_verification_substream_minus1.

[0204] A further alternative could be to do as described in Option 3 above. This can be instantiated in the standard as follows. JPEG2026086353000012.jpg94170 The dsci_sei_messages_digital_signed_separate_substream_flag equal to 1 specifies that the SEI message used to calculate the message digest is assigned to a separate verification substream specified by the semantics of the DSCS SEI message. The dsci_sei_messages_digital_signed_separate_substream_flag equal to 0 specifies that the SEI message used to calculate the message digest is assigned to the same verification substream as the VCL NAL unit of the associated coded picture.

[0205] 1.2.2 Digital Signature of Selected Content SEI Message Semantics The digital signature of selected content SEI message provides a mechanism for associating a coded picture with one of the verification substreams indicated in the digital signature of content initialization SEI message. [...] dscs_verification_substream_id indicates the verification substream to which the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture are assigned. If dsci_sei_messages_digital_signed_flag is equal to 1 and dsci_sei_messages_digital_signed_highest_substream_flag is equal to 0, the SEI message used for message digest belongs to the same verification substream. If dsci_sei_messages_digital_signed_flag is equal to 1 and dsci_sei_messages_digital_signed_highest_substream_flag is equal to 1, the SEI message used for message digest belongs to a verification substream with an ID equal to dscs_verification_substream_id plus dsci_num_verification_substream_minus1+1. If a digitally signed content initialization SEI message is present in the current coded video sequence, but a digitally signed content selection SEI message is not present in the coded picture, the value of dscs_verification_substream_id is inferred to be equal to 0 for SPS, PPS, APS, PH, and VCL NAL units. If dsci_sei_messages_digital_signed_flag is equal to 1, dsci_sei_messages_digital_signed_highest_substream_flag is equal to 1, and no digitally signed content selection SEI message is present for the coded picture, the SEI message used for message digest belongs to a verification substream with an ID equal to dsci_num_verification_substreams_minus1 plus 1. The value of dscs_verification_substream_id is greater than or equal to 0 and less than or equal to dsci_num_verification_substream_minus1. Alternatively, as mentioned above, syntactic elements can be introduced to distinguish between options 1, 2, and 3.

[0206] Further embodiments of a fourth aspect of the present invention will be described below. A further aspect to consider is the repetition of SEI messages, i.e., a mechanism for repeating SEI message NAL units of the same content within a specific portion of the bitstream to be resilient to packet loss, as shown, for example, in Figure 13A. Deletion should not affect the outcome of the decoding process and any SEI message-related post-processing, as long as one of the SEI messages remains in the bitstream.

[0207] When all SEI messages, including the original SEI message and N repeating SEI messages, are used to compute the message digest, deleting the associated data can disrupt the authentication of the remaining bitstream. The following solution for computed message digests makes the authenticity check robust against the deletion of such redundant data.

[0208] In one embodiment, the calculation of the message digest takes into account only one of N+1 SEI messages (hereinafter referred to as the selected SEI message), ignoring the remaining SEI messages. It must be ensured that the selected SEI message is in the correct position (i.e., the same as on the encoder side or the transmitter side) in the data used to calculate the message digest. Thus, in this embodiment, the selected SEI message is repositioned near a predefined position in the bitstream portion, for example, at the end or beginning of the PU, before or after certain other NAL units such as VCL or non-VCL NAL units, or to a certain NAL unit type, or to yet another criterion.

[0209] Additional issues arise when different types of SEI messages are considered. If one SEI message is placed between different repetitions, assume that SEI A and SEI B reside within the same bitstream portion. Depending on which bitstream portion (occurrence of the repetitive SEI message) is dropped, the order of the remaining SEI messages may change as shown in Figure 13B. For example, depending on which occurrence of SEI B is dropped, SEI A may come before or after SEI B.

[0210] Therefore, in the alternative embodiment, in order to ensure the same order on both the sender and receiver sides, an additional sorting procedure should be applied to the selected SEI messages before repositioning and message digest calculation, for example, sorting by SEI message type, size, or ID, or any combination thereof, if applicable.

[0211] In one embodiment, if the repetition of an SEI message is to be used for multiple different SEI messages within the same bitstream portion, the different SEI messages must be placed in a joint SEI message NAL unit as separate SEI message payloads and repeated together in this form. That is, SEI messages can be interleaved, but when repeated, they (all) must be in the same SEI NAL for each repetition.

[0212] As a further alternative, or in addition to the above, bitstream constraints may prohibit the occurrence of other non-repeating and / or repetitive SEI messages from occurring between any N+1 occurrences of any repetitive SEI message in bitstream order. That is, unless repeated as described above within the same SEI NAL unit, it is either not always permitted to allow any other SEI messages between repetitions, or it is not permitted for another SEI message to exist between repetitions.

[0213] Further embodiments of a fifth aspect of the present invention are described below, which may be considered alternatives to the fourth aspect. A further alternative is to have bitstream constraints in the codec specification that do not allow the specific problematic use of SEI message repetition, such as interleaving related to SEI message repetition. For example, the following bitstream constraint addresses this problem by not allowing SEI message repetition when CVS carries messages for authentication.

[0214] If a digitally signed content initialization SEI message exists within the CVS, the following restrictions apply to the bitstream conformance requirements (for repetition of SEI messages): -For each payloadType value included in PicUnitRepConSeiAuthList, it is assumed that no identical sei_payload() syntax structure exists within the PU. - Assume that no identical sei_payload() syntax structure with payloadType equal to 130 exists within the DU.

[0215] The PicUnitRepConSeiAuthList is defined as a set of SEI message types that are designated as digitally signed and whose repetition within a PU is not permitted. Such a list may be subject to further constraints, such as excluding certain SEI message types. It may also be the case that only a subset of all digitally signed SEI message types are not permitted to be repeated. As described above, SEI message types are distinguished depending on whether they are applied to a PU (Picture Unit) or a DU (Decode Unit - i.e., something smaller than a picture). Further distinctions can be made based on whether these also apply to CVS, CLVS, or AU.

[0216] Therefore, depending on the association with one of these CVS, CLVS, AU, PU, ​​or DU, repetition (exact copying) within the CVS, CLVS, AU, PU, ​​or DU can be prohibited for each type of SEI message. Furthermore, there are SEI message types that have no restrictions on repetition, such as the following: -3 (Filler data) (For example, this information does not affect the decoding result and therefore does not need to be signed), -4 (User data registered) and / or 5 (User data not registered) (for example, the content is variable and does not affect the validation of other data).

[0217] Further options include including a semantic constraint on a syntactic element that indicates which SEI message payload types are digitally signed (e.g., dsci_digital_signed_sei_type[i]) in order to prohibit the digital signing of SEI messages with a payloadType equal to 3 (filler data), as follows:

[0218] dsci_digital_signed_sei_type[i] identifies the i-th SEI type used to calculate the message digest. The value of dsci_digital_signed_sei_type[i] is not equal to 3. If a digitally signed content initialization SEI message exists within the CVS, the bitstream conformance requirement is that an SEI message with a payloadType equal to 4 (user data registered) and 5 (user data not registered) be associated with each individual substream.

[0219] Further embodiments of a second aspect of the present invention will be described below. The currently defined SEI messages enable signaling the signature of substreams within a segment and enable verifying temporal consistency by signing across the hashes of consecutive segments. However, so far, there is no signaling for the start or end point of the signed bitstream (section). In particular, the lack of end signaling can allow the bitstream to be shortened (or cut) at the end without being noticed by the decoder. Explicit signaling of the start point enables, for example, instructing the decoder to perform special processing in order to combine the hash of the previous segment with the signature. Also, explicit signaling of the start point enables concatenating independently encoded and signed bitstreams using the same signature key (e.g., two segments of a newscast). Unless an EOB NAL unit is used, the decoder cannot recognize that the second bitstream is permitted to break the link and may incorrectly indicate a verification problem.

[0220] Solution 1: Signal a flag indicating the end of the signed bitstream section. Variant of signaling a) The flag is added to the digitally signed content initialization SEI message as follows. JPEG2026086353000013.jpg82163 This is preferred signaling, for the following reasons. - Only one initialization SEI is signaled for all substreams - The information is available to the decoder at the start of the already signed segment - Since the type of hash method is already included in the signature, no additional data dependency is introduced between the initialization SEI and the signature

[0221] Variant of signaling b) Alternatively, the flag is added to the digitally signed content verification SEI message as follows: JPEG2026086353000014.jpg34166 Regarding variations of both signaling methods: To ensure that a malicious user cannot alter the instructions for the last signed bitstream segment, a flag value is added to the signature, and for example, the input data for the signature may consist of the elements shown in Figure 14A.

[0222] Solution 2: Signals a flag indicating the start of a signed bitstream section. Variations of signaling a) The following flag is added to the digitally signed content initialization SEI message: JPEG2026086353000015.jpg81164 This is a desirable signaling behavior for the following reasons: - Only one initialization SEI is signaled for each substream. - The information is available to the decoder at the beginning of the already signed segment. - No additional data dependencies are introduced between the initialization SEI and the signature, as the hashing method type is already included in the signature. Variations of signaling (b) Alternatively, the flag is added to the digitally signed content verification SEI message as follows: JPEG2026086353000016.jpg34166 Regarding variations of both signaling methods: To ensure that a malicious user cannot alter the instructions for the last signed bitstream segment, a flag value is added to the signature, and for example, the input data for the signature may consist of the elements shown in Figure 14A.

[0223] If the signed bitstream segment is the first, a temporary reference to the previous segment is not used. In this case, refDigest must be as follows: a) As shown in Figure 14B, the signature is not included at all, or b) Set all bytes of the hash to 0x00 (length depends on the hash type used) Conditional signaling in digitally signed content verification SEI Since time-dependent signatures are used only in base substream 0, the flags indicating the start and end of the signed bitstream section are only relevant within this substream.

[0224] If one or both flags are signaled in a digitally signed content verification SEI, they should only be present if the SEI message applies to substream 0. JPEG2026086353000017.jpg51166 This saves signaling bits by avoiding redundancy and potential contention.

[0225] Further embodiments of a third aspect of the present invention will be described below. Further extensions are described below. The solutions presented above make it possible to splice entire authenticated sections in a way that allows for content verification. In a typical scenario, splicing occurs as a result of an unexpected event, such as an ad insertion, in which some content is introduced in the middle of a video bitstream (i.e., a section). Such spliced ​​content may or may not be authenticated. A diagram illustrating such a splicing case is shown in Figure 15.

[0226] The illustrated use case presents a problem because A (start) can be validated accurately. If B is validated, it may not be validated correctly when the last segment of A is considered. In principle, there may be no problem even if B is not validated. However, when returning to A (continuation), A (continuation) is not validated against A (start), meaning something can be deleted, or A (continuation) is validated against the last segment of B and results in an error.

[0227] In a further embodiment, a solution is provided in which the splicing is signaled (e.g., by a flag called dsci_splicing_flag) so that a decoder verifying an authenticated bitstream can identify such splicing points. This can also be used for parts of an unauthenticated bitstream, but can be used to indicate that some content is being spliced ​​together, and the receiver interprets that if the content preceding the splicing point has not ended, it is likely to continue at the point where the spliced ​​content ends (e.g., after B ends). At a splicing point where the following content is authenticated (e.g., A (continuation)), if the content is new and not simply a continuation of the previous content (e.g., it is certified from a new content provider, or has a new UUID, or in some way an indication has been added that it starts with a new segment - e.g., dsci_first_signed_segment_flag), then random access may be performed and it may be treated as if that segment were the first segment. Otherwise, if the content is not new but a continuation of previously decoded content (A continuation), it needs to be checked for temporal consistency with the last segment of the same content received some time ago. To do this, the hash value of the last segment before content B was spliced ​​should be used. In the previous example, the first segment of A (continuation) might be checked against the last segment of A (start).

[0228] Therefore, splicing instructions within a bitstream provide decoders validating more than one video sequence with information that the segment immediately following the splicing point and the segment preceding the splicing point should not be validated together. In some circumstances, if the segment following the splicing point should be validated (i.e., containing authentication information), it may be that it should be validated by a preceding segment sent some time earlier. Therefore, at the splicing point, the decoder needs to identify the following:

[0229] a) Whether the subsequent segment of the new content is authenticated. a. If not authenticated, is it possible that a previously authenticated segment may be needed in the future to validate further segments? b. If authenticated, is it the first segment of the content? b) When a subsequent segment of new content needs to be authenticated, is it not the first segment of the new content where the previous segment of the same content should be found?

[0230] The choice to derive the aforementioned information can be made using a single flag, dsci_splicing_flag, as follows: When authenticated content is received and the splicing flag is found at some point, the decoder interprets that the content may continue (follow) at some point in the future and stores a digest message of the last segment of the lowest substream (the substream used for temporal consistency). Note that this can happen with B in the example, whether B is authenticated or not, if such a splicing flag is indicated. As a further option if B is not authenticated, the fact that B was not authenticated can be directly interpreted as meaning that splicing occurred.

[0231] After some time, when the decoder encounters further splicing flags, it determines whether the content is the first segment of new content or not, or not the first segment of previously sent content. For example, the decoder compares a digitally signed content SEI message to a digitally signed content SEI message received previously. If a digitally signed content initialization SEI message is exactly the same as a previously received digitally signed content initialization SEI message (i.e., it points to the same certificate, has the same UUID value, and the same hash_method_type), the decoder can determine that the subsequent segment is not the first segment of new content, and therefore validates that segment against the previously stored digest message (the last one) to which such an SEI message was applied, and validates it against that last segment. This comparison of digitally signed content initialization SEI messages can be done in various ways.

[0232] 1) Exact match as described 2) Exact match only if a UUID exists. 3) Consistency check: Allow some values ​​to be changed. a. For example, hash_method_type b. UUIDs can change, but their generation follows specific rules. i. The previous UUID + 1 ii. Calculations are performed based on time information, and there is no gap between the previous time used and the current time. A further option might be to provide some additional signaling to facilitate the decision of whether a digest message needs to be remembered and whether a segment needs to be validated with the previous segment (not the immediately preceding segment). For example, if the last segment of the content is indicated within the bitstream, and the last segment flag is not indicated at the splicing point, then the digest message should be remembered. Similarly, if the first segment flag is not indicated within the subsequent segment at the splicing point, a previously remembered digest message is retrieved and validated together. If the last flag is indicated, the digest message is not remembered, and if the first message is indicated, the segment is not validated with any previous segment.

[0233] The instantiation of such syntax is shown within a digitally signed content initialization SEI message as follows: JPEG2026086353000018.jpg83167 However, splicing information may be provided in a separate SEI message.

[0234] [Video coding method] The following describes a video encoding scheme in which embodiments of the present invention can be optionally implemented. In other words, the device 20 according to any of the embodiments described above may be a video decoder implemented according to any of the embodiments of the decoder 20 described below. Similarly, the device 10 may be a video encoder according to any of the embodiments of the encoder 10 described below. The following description of the figures begins with a presentation of an encoder and decoder for a block-based predictive codec for encoding video pictures, in order to form an example of a coding framework in which embodiments of the present invention may be incorporated. Each encoder and decoder will be described with reference to Figures 16, 17, and 18. Hereafter, embodiments of the concept of the present invention will be presented, along with a description of how such concepts may be incorporated into the respective encoders and decoders of Figures 16 and 17. However, subsequent figures and embodiments described below may also be used to form encoders and decoders that do not operate according to the underlying coding framework of the encoders and decoders of Figures 16, 17, and 18.

[0235] Figure 16 illustrates an apparatus for predictively encoding picture 12 into a data stream 14 using transform-based residual coding as an example. The apparatus or encoder is indicated by reference numeral 10. Figure 17 shows the corresponding decoder 20, i.e., an apparatus 20 configured to predictively decode picture 12' from the data stream 14 using transform-based residual decoding, where an apostrophe is used to indicate that picture 12' reconstructed by decoder 20 deviates from picture 12 originally encoded by apparatus 10 in terms of coding loss introduced by quantization of the predictive residual signal. While Figures 16 and 17 use transform-based predictive residual coding as an example, embodiments of the present application are not limited to this type of predictive residual coding. This also applies to other details described with respect to Figures 16 and 17, as outlined below.

[0236] Encoder 10 is configured to apply a spatial-spectral transform to the predicted residual signal and encode the thus obtained predicted residual signal into the data stream 14. Similarly, decoder 20 is configured to decode the predicted residual signal from the data stream 14 and apply a spectral-spatial transform to the thus obtained predicted residual signal. The encoder 10 may internally include a predictive residual signal generator 22, which generates a predictive residual 24 to measure the deviation of the predictive signal 26 from the original signal, i.e., from the picture 12. The predictive residual signal generator 22 may, for example, be a subtractor that subtracts the predictive signal from the original signal, i.e., from the picture 12. The encoder 10 then further includes a converter 28 that applies a spatial-spectral transform to the predictive residual signal 24 to obtain a spectral domain predictive residual signal 24', which is then quantized by a quantizer 32 also provided in the encoder 10. The thus quantized predictive residual signal 24'' is encoded in the bitstream 14. For this purpose, the encoder 10 may optionally include an entropy coder 34 that entropy codes the predictive residual signal to be transformed and quantized in the datastream 14. The predicted signal 26 is generated by the prediction stage 36 of the encoder 10 based on the predicted residual signal 24'' encoded in the data stream 14 and decodeable therefrom. For this purpose, the prediction stage 36 may internally include an inverse quantizer 38 that inversely quantizes the predicted residual signal 24'' to obtain a spectral domain predicted residual signal 24'''' corresponding to the signal 24' excluding quantization losses, as shown in Figure 16, and an inverse converter 40 that subsequently applies an inverse transform, i.e., spectral-spatial transform, to the latter predicted residual signal 24'''' to obtain a predicted residual signal 24'''' corresponding to the original predicted residual signal 24 excluding quantization losses. The coupler 42 of the prediction stage 36 then recombines the predicted signal 26 and the predicted residual signal 24'''' by addition or other means to obtain a reconstructed signal 46, i.e., a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to signal 12'. Next, the prediction module 44 in the prediction stage 36 generates a prediction signal 26 based on the signal 46, for example, by using spatial prediction, i.e., intra-picture prediction, and / or temporal prediction, i.e., inter-picture prediction.

[0237] Similarly, the decoder 20 may be internally composed of components corresponding to the prediction stage 36, as shown in Figure 17, and may be interconnected in a manner corresponding to the prediction stage 36. In particular, the entropy decoder 50 of the decoder 20 entropy decodes the spectral domain prediction residual signal 24'' quantized from the data stream, and then the inverse quantizer 52, inverse converter 54, coupler 56, and prediction module 58, which are interconnected and work together in the manner described above with respect to the module of the prediction stage 36, recover a signal reconstructed based on the prediction residual signal 24'', and as a result, as shown in Figure 17, the output of the coupler 56 yields the reconstructed signal, i.e., picture 12'.

[0238] Although not specifically described above, it is readily apparent that encoder 10 can set several coding parameters, including prediction mode and motion parameters, according to several optimization methods, such as several rate and distortion-related criteria, i.e., methods for optimizing coding cost. For example, encoder 10 and decoder 20, and their corresponding modules 44 and 58, may support different prediction modes, such as intra-coding mode and inter-coding mode. The granularity at which the encoder and decoder switch between these prediction mode types may correspond to the subdivision of picture 12 and 12' into coding segments or coding blocks, respectively. At the level of these coding segments, for example, a picture may be subdivided into intra-coded blocks and inter-coded blocks. The intra-coded blocks are predicted based on the spatial, already coded / decoded neighborhood of each block, as outlined in more detail below. Several intracoding modes may exist, and each intracoding segment may be selected, including directional or angular intracoding modes, to satisfy each segment by extrapolating neighboring sample values ​​along a specific direction specific to each directional intracoding mode into each intracoding segment. The intracoding modes may also include one or more further modes, such as a DC coding mode in which the prediction for each intracoding block assigns DC values ​​to all samples within each intracoding segment, and / or a planar intracoding mode in which the prediction for each block is approximated or determined to be a spatial distribution of sample values ​​described by a two-dimensional linear function over the sample positions of each intracoding block, having a plane driving slope and offset defined by a two-dimensional linear function based on neighboring samples. In comparison, the intercoding blocks may be predicted, for example, in time.In the case of an intercoded block, a motion vector may be signaled within the data stream, the motion vector indicating the spatial displacement of the portion of the previously coded picture to which picture 12 belongs, and the previously coded / decoded picture in that portion is sampled to obtain the predictive signal for each intercoded block. In addition to the residual signal coding contained in the data stream 14, such as the entropy coded transformation coefficient level representing the quantized spectral domain predictive residual signal 24'', the data stream 14 may have further selective parameters encoded within the data stream 14, such as coding mode parameters for assigning coding modes to various blocks, several predictive parameters of the blocks such as motion parameters for the intercoded segments, and parameters for controlling and signaling the subdivision of pictures 12 and 12' into segments, respectively. The decoder 20 uses these parameters to subdivision the picture in the same way that the encoder did, thereby assigning the same predictive mode to the segments, performing the same predictive actions, and yielding the same predictive signals.

[0239] Figure 18 shows the relationship between, on the one hand, the reconstructed signal, i.e., the reconstructed picture 12', and on the other hand, the combination of the predicted residual signal 24'''' and the predicted signal 26, which are signaled in the data stream 14. As already mentioned above, the combination can be additive. In Figure 18, the predicted signal 26 is shown as a subdivision of the picture area into intracoded blocks shown exemplarily with hatching and interconnected blocks shown exemplarily without hatching. The subdivision can be a regular subdivision of the picture area into rows and columns of square or non-square blocks, or any subdivision such as a quadtree subdivision of the picture 12 into multiple leaf blocks of varying sizes from a tree root block, a mixture of which is shown in Figure 18, where the picture area is first subdivided into rows and columns of a tree root block, and then further subdivided into one or more leaf blocks according to a recursive multitree subdivision.

[0240] Here too, the data stream 14 may have an intra-coding mode encoded within the data stream 14 for the intra-coded blocks 80, so that one of several supported intra-coding modes is assigned to each intra-coded block 80. For the intercoded blocks 82, the data stream 14 may have one or more motion parameters encoded within the data stream 14. Generally speaking, the intercoded block 82 is not limited to being encoded in time. Alternatively, the intercoded block 82 may be any block predicted from parts encoded before the current picture 12 itself, such as a picture encoded before the video to which the picture 12 belongs, or a picture in another view or a hierarchically lower layer if the encoder and decoder are scalable encoders and decoders, respectively.

[0241] The predicted residual signal 24'''' in Figure 18 is also shown as a subdivision of the picture area into block 84. These blocks are sometimes called transformation blocks to distinguish them from coded blocks 80 and 82. In practice, Figure 18 shows that encoder 10 and decoder 20 may use two different subdivisions of picture 12 and picture 12' into blocks, namely, one subdivision into coded blocks 80 and 82 and the other subdivision into transformation block 84. Both subdivisions may be the same, i.e., each coded block 80 and 82 may simultaneously form a transformation block 84, but Figure 18 shows, for example, that the subdivision into transformation block 84 forms an extension of the subdivision into coded blocks 80 and 82, thereby such that any boundary between the two blocks 80 and 82 covers the boundary between the two blocks 84, or in other words, each block 80 and 82 coincides with one of the transformation blocks 84 or coincides with a cluster of transformation blocks 84. However, the subdivisions may also be determined or selected independently of each other, so that the transformation block 84 can alternatively cross the block boundary between blocks 80 and 82. Therefore, as far as subdivision to transformation block 84 is concerned, the same description as that presented for subdivision to blocks 80 and 82 applies: namely, block 84 may be the result of regular subdivision of the picture area into blocks (with or without row and column arrangement), the result of recursive multi-tree subdivision of the picture area, or a combination thereof, or any other type of block formation. Incidentally, it should be noted that blocks 80, 82, and 84 are not limited to squares, rectangles, or any other shape.

[0242] Figure 18 further shows that the combination of the prediction signal 26 and the prediction residual signal 24'''' directly yields the reconstructed signal 12'. However, it should be noted that, according to an alternative embodiment, more than one prediction signal 26 may be combined with the prediction residual signal 24'''' to yield the picture 12'.

[0243] In Figure 18, the transformation blocks 84 are assumed to have the following significance: The converters 28 and inverse converters 54 perform these transformations in units of these transformation blocks 84. For example, many codecs use some kind of DST or DCT for all transformation blocks 84. Some codecs allow skipping transformations, and as a result, for some of the transformation blocks 84, the predicted residual signal is encoded directly in the spatial domain. However, according to embodiments described later, the encoder 10 and decoder 20 are configured to support several transformations. For example, the transformations supported by the encoder 10 and decoder 20 may include: • DCT-II (or DCT-III), where DCT represents the Discrete Cosine Transform. • DST-IV, where DST represents the discrete sine transform. · DCT-IV · DST-VII • Identity transformation (IT) Naturally, the converter 28 may support all forward versions of these conversions, while the decoder 20 or the inverse converter 54 may support their corresponding reverse or inverse versions. • Inverse DCT-II (or Inverse DCT-III) · Reverse DST-IV ·Inverse DCT-IV · Reverse DST-VII • Identity transformation (IT) The following explanation provides further details on which transformations may be supported by the encoder 10 and decoder 20. Note that in any case, the set of supported transformations may include only one transformation, such as a single spectral-to-spatial or spatial-to-spectral transformation.

[0244] As outlined above, Figures 16, 17, and 18 are presented as examples of how the concepts of the present invention, further described below, may be implemented to form specific examples of encoders and decoders according to this application. To that extent, the encoders and decoders in Figures 16 and 17 may represent possible embodiments of encoders and decoders described herein. However, Figures 16 and 17 are merely examples. Nevertheless, an encoder according to an embodiment of this application may perform block-based encoding of picture 12 using concepts outlined in more detail below, which may differ from the encoder in Figure 16, for example, being a still picture encoder rather than a video encoder, not supporting interpretation, or performing subdivision into blocks 80 in a manner different from that illustrated in Figure 18. Similarly, a decoder according to an embodiment of the present application may perform block-based decoding of picture 12' from data stream 14 using the coding concept further outlined below, but may differ from decoder 20 in Figure 17, for example, in that it is a still picture decoder rather than a video decoder, does not support intra-prediction, subdivides picture 12' into blocks in a different manner than described with respect to Figure 18, and / or derives prediction residuals from data stream 14 in the spatial domain rather than the transformation domain.

[0245] The embodiments provide an H.264 / AVC decoder, an H.264 / AVC video data stream, a video encoder for providing the H.264 / AVC video data stream, or an H.265 / HEVC decoder, an H.265 / HEVC video data stream, a video encoder for providing the H.265 / HVC video data stream, or an H.266 / VVC decoder, an H.266 / VVC video data stream, a video encoder for providing the H.266 / VVC video data stream.

[0246] The following describes embodiments of the present invention that can be optionally implemented within or combined with the framework described above with respect to Figures 16 to 18. All embodiments of the present invention as disclosed herein may be optionally implemented in combination with any of the four video encoding modifications described below. That is, the data stream 14 may be a data stream according to any of these modifications, the device 20 may be configured to decode video from the data stream 14 according to any of the three modifications, and the device 14 may be configured to provide the data stream according to any of the four modifications. The first and second modifications may be optimally combined with each other.

[0247] According to the first of two modifications of the present invention, the video is decoded from the video data stream by block-based prediction and transformation-based residual decoding, by decoding the predicted residual data of the residual blocks into and from the video data stream. According to the first modification, the predicted residual data of the residual block is decoded using context-adaptive variable-length decoding, and the decoding is performed as follows: - A first syntactic element indicating the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a trailing-one numerical value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, -One or more second syntactic elements indicating the sign of a non-zero transformation coefficient having an absolute value of 1 when traversing the coefficients along the scan order, -One or more third syntactic elements indicating the values ​​of non-zero conversion coefficients, excluding the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, -A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, -One or more fifth syntactic elements that indicate the position of a non-zero conversion coefficient along the scan order by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients, This is done using, or As an alternative to using context-adaptive variable-length decoding, decoding the predicted residual data of the residual block is performed using context-adaptive binary arithmetic decoding, and context-adaptive binary arithmetic decoding is performed. - Decoding an importance map indicating the location of non-zero conversion coefficients within a conversion block representing a residual block by decoding an importance flag indicating whether a non-zero conversion coefficient is located at the current position in a forward scan traversing the conversion coefficients of the conversion block, and if a non-zero conversion coefficient is located at the current position, and the current position is not the end of the forward scan, by decoding a final importance flag indicating whether the non-zero conversion coefficient located at the current position is the last non-zero conversion coefficient in the forward scan sequence, and - Decode the values ​​of the non-zero conversion coefficients sequentially using the reverse scan order, which is the reverse of the forward scan order. It is done by [the specified method].

[0248] According to the first of two modifications of the present invention, the video is encoded in the video data stream by block-based predictive and transform-based residual encoding, by encoding the predicted residual data of the residual blocks into the video data stream. According to the first modification, the predicted residual data of the residual block is encoded using context-adaptive variable-length encoding, and the encoding is performed as follows: - A first syntactic element indicating the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a trailing-one numerical value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, -One or more second syntactic elements indicating the sign of a non-zero transformation coefficient having an absolute value of 1 when traversing the coefficients along the scan order, -One or more third syntactic elements indicating the values ​​of non-zero conversion coefficients, excluding the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, -A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, -One or more fifth syntactic elements that indicate the position of a non-zero conversion coefficient along the scan order by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients, This is done using, or As an alternative to using context-adaptive variable-length encoding, encoding the predicted residual data of the residual block is performed using context-adaptive binary arithmetic encoding, and context-adaptive binary arithmetic encoding is, - Encoding an importance map indicating the location of non-zero conversion coefficients within a conversion block representing a residual block by encoding an importance flag indicating whether a non-zero conversion coefficient is located at the current position in a forward scan traversing the conversion coefficients of the conversion block, and if a non-zero conversion coefficient is located at the current position, and the current position is not the end of the forward scan, by encoding a final importance flag indicating whether the non-zero conversion coefficient located at the current position is the last non-zero conversion coefficient in the forward scan order, and - Encode the values ​​of the non-zero conversion coefficients sequentially using the reverse scan order, which is the reverse of the forward scan order. It is done by [the specified method].

[0249] According to the second modification, the video decoder (or device for decoding video from a video data stream) comprises an encoded picture buffer (CPB) and a decoded picture buffer (DPB). The video decoder, Receiving a data stream containing video pictures encoded in a data stream in a coded order as a sequence of access units (AUs) (for example, the term "access unit" refers to a portion of a video data stream containing coded video data or information relating to one time frame of the video), The current AU, which has been deleted from the CPB, is decoded using picture-to-picture prediction from the referenced picture stored in the DPB to obtain the decoded picture, and Inserting the decoded picture into the DPB, Assigning a classification to each reference picture stored in the DPB as one of the following: short-term reference picture, long-term reference picture, or picture not used for reference. Read DPB mode information from the current AU. If the DPB mode information indicates a first mode, remove one or more reference pictures classified as short-term pictures from the DPB in accordance with the First-In, First-Out (FIFO) strategy. If the DPB mode information indicates a second mode, read the memory management control information, which includes at least one command in the current AU, and execute at least one command to change the classification assigned to at least one of the reference pictures stored in the DPB, and to use the classification of the reference picture in the DPB to manage the removal of the reference picture from the DPB. It is configured to perform the following actions.

[0250] According to the second modification, the video data stream is to be decoded by being fed into a decoder having an encoded picture buffer (CPB). According to the second modification, the video encoder (or device for encoding video into a video data stream) is configured to encode the pictures of the video encoded in the coded order into the data stream as a sequence of access units (AUs), When the device encodes AU, Encoding the current picture into the current AU using picture-to-picture predictions from the referenced picture stored in the decoded picture buffer (DPB), Inserting a decoded version of the current picture in the DPB into the DPB itself. Assigning a classification to each reference picture stored in the DPB as one of the following: short-term reference picture, long-term reference picture, or picture not used for reference. To write DPB mode information into the current AU. If the DPB mode information indicates a first mode, then, according to the FIFO strategy, remove one or more reference pictures classified as short-term pictures from the DPB. If the DPB mode information indicates a second mode, write memory management control information including at least one command into the current AU, the command instructs to change the classification assigned to at least one of the reference pictures stored in the DPB, and the classification of the reference picture in the DPB is used to manage the removal of the reference picture from the DPB. It is configured to perform the following actions.

[0251] According to a third modification, the video is decoded from the video data stream by block-based predictive decoding and transformation-based residual decoding, which involves decoding the predicted residual data of the residual blocks from the video data stream using context-adaptive binary arithmetic decoding of the quantization index of the transformation coefficients of the transformation blocks representing the residual blocks, and sequential inverse quantization of the quantization index, where the value of the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. According to a third modification, the video is encoded within the video data stream by block-based predictive encoding and transform-based residual encoding, which is done by encoding the predicted residual data of the residual blocks into the data stream using context-adaptive binary arithmetic encoding of the quantization index of the transformation coefficients of the transformation block representing the residual block, and sequential quantization of the transformation coefficients to obtain the quantization index, where the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index.

[0252] All three variations relate to video encoders, video decoders, methods for decoding video, methods for encoding video, and the video data streams obtained by each encoding method.

[0253] The following describes further optional details and characteristics of the first and second variations. Embodiments of the first and second modified versions may conform to H.264 / AVC. Embodiments of the third modified version may conform to H.266 / VVC. One embodiment of the first and second modifications relates to the handling of decoded pictures and their buffering in a DPB, which is a decoded picture buffer.

[0254] According to one embodiment, two types of reference pictures can be distinguished: short-term and long-term. The encoder does the same when emulating the decoder's DPB-filled state at each point in time during decoding. A reference picture can be marked as "not used for reference" when it is no longer needed for predictive referencing. The conversion between these three states (short-term, long-term, not used for reference) is controlled by the decoded reference picture marking process. Two alternative decoded reference picture marking mechanisms exist: an implicit sliding window process and an explicit memory management control operation (MMCO) process. For each currently decoded picture or each currently decoded AU, the data stream signals which process should be used for DPB management. The sliding window process marks short-term reference pictures as "not used for reference" when the number of reference frames equals a given maximum number (max-num-ref-frames in SPS). Short-term reference pictures are stored in a first-in, first-out (FIFO) manner, so the most recently decoded short-term picture is retained in the DPB. The explicit MMCO process is controlled by several MMCO commands. If this mode is selected for the current AU or the picture currently being decoded, the bitstream will contain one or more of these commands for it, or in this case for the AU. An MMCO command may be any of the following: 1) mark one or more short-term or long-term reference pictures as "not used for reference", 2) mark all pictures as "not used for reference", or 3) mark the current reference picture or an existing short-term reference picture as long-term and assign a long-term picture index to that long-term picture. The reference picture marking operation, as well as any output for presentation and removal of the picture from the DPB, may be performed after the picture has been decoded.

[0255] Details of several possible but optional reference picture marking mechanisms are described below. 1) The first aspect concerns frame number gaps and non-existent pictures. Although not described above, each reference picture in a DPB may be associated with a frame number. Normally, this number increments by 1 for each reference picture, but frame number gaps may be allowed by setting a corresponding high-level (such as sequence-level) flag, sometimes called a parameter-gaps-in-frame-num-allowed-flag, to 1, to allow, for example, an encoder or MANE (Media Aware Network Element) to deliver a bitstream where the frame number increments by 2 or more for each reference picture preceding in the decoding order. This can be advantageous for supporting temporal scalability. A sequence of AUs with gaps in frame numbers may be received, and non-existent pictures may be created to fill the gaps. The non-existent pictures are assigned a frame number value in the gap and are considered reference pictures during decoded reference picture marking, but are not used for output (and therefore not displayed). The absence of a picture ensures that the state of the DPB for the frame numbers of pictures present in the DPB is the same for decoders that received the picture and decoders that did not receive the picture.

[0256] Another possible aspect of the first and second modifications concerns the loss of reference pictures when using sliding windows. If a reference picture is lost, assuming the loss has been detected, it may be possible to hide the picture and, if possible, report the loss to the encoder, provided a feedback channel is available. If gaps in frame numbers are not allowed, discontinuities in frame number values ​​indicate an unintended loss of a reference picture. If gaps in frame numbers are allowed, discontinuities in frame number values ​​may occur due to either the intentional deletion of a time layer or subsequence, or accidental picture loss, and the decoder should infer picture loss only if a non-existent picture was referenced in the interprediction process. The picture order number of a hidden picture may be unknown, in which case the decoder may use an incorrect reference picture without detecting any error when decoding the B picture.

[0257] Further possible embodiments of the first and second modifications relate to the loss of reference pictures due to MMCO. If a reference picture containing an MMCO command that marks a short-term reference picture as "not used for reference" is lost, the state of the reference picture in the DPB may become incorrect, and as a result, the reference picture list for several pictures following the lost picture may become incorrect. If a picture containing an MMCO command related to a long-term reference picture is lost, there is a risk that the number of long-term reference pictures in the DPB will differ from the number if that picture had been received, resulting in an "incorrect" sliding window process for all subsequent pictures. That is, the encoder and decoder may have different numbers of short-term reference pictures, and as a result, the operation of the sliding window process may not be synchronized. Further exacerbating the situation is that the decoder may not necessarily recognize that the sliding window process is out of sync.

[0258] The MMCO commands are shown below. In different embodiments, one or more or all of these commands may apply. Further options for the implementation of decoders and encoders by the first and second modifications are described below, optionally in combination with the options relating to DPB management described above, relating to entropy decoding of several syntactic elements, such as residual data in the form of conversion coefficients into the bitstream. Reversible entropy coding of irreversibly quantized conversion coefficients is an important part of efficient video codecs. One such method is called context-adaptive variable-length coding (CAVLC), in which the encoder switches between different variable-length code (VLC) tables for different syntactic elements in a context-adaptive manner, depending on the values ​​of syntactic elements previously sent within the same slice. Encoders and decoders may use CAVLC. Each CAVLC coded bit in the bitstream can be associated with a single syntactic element, since each syntactic element is coded into the bitstream by writing the corresponding codeword selected for that syntactic element from a context-adaptively selected code table into the bitstream. Therefore, relevant information regarding the scan-order transformation coefficient levels present in the bitstream is available in a form that can be directly accessed as syntactic elements when CAVLC is used. Encoders and decoders can signal transformation coefficients in the bitstream using CAVLC. The following syntactic elements, for example, syntactic elements with the following semantics, may be used.

[0259] - A single syntactic element (indicated by CoeffToken) that specifies the total number of non-zero conversion coefficient levels within the conversion block. - One or more syntactic elements (indicated by CoeffToken) that indicate the number of subsequent transformation coefficient levels with a value of 1, for example, the sequence of syntactic elements that occur last when scanning syntactic elements in scan order up to the last non-zero syntactic element, all of which are 1, and their signs (indicated by trailing_ones_sign_flag) - One or more syntactic elements for each non-zero conversion coefficient, excluding the subsequent conversion coefficient that indicates the conversion coefficient level value. - A single syntactic element indicating the total number of zero-value conversion coefficient levels. - A syntactic element that indicates the number of consecutive zero-value conversion coefficient levels in the scan sequence from the current scan position until a non-zero value conversion coefficient level is encountered.

[0260] Alternatively or additionally, the encoder may choose to use CABAC, and therefore between context-adaptive binary arithmetic coding and CAVLC, signaling the choice in the bitstream, and the decoder may read this signal and decode the residual data in a specified manner.

[0261] Further options for implementing decoders and encoders by the first and second modifications are described below, which may optionally be combined with either the DPB management or the aforementioned CAVLC, and relate to a quarter-pixel interpolation filter. To enable interpretation with finer granularity than a normal full-pixel sample grid, a sample interpolation process is used to derive sample values ​​at sub-pixel sample positions that may range from half-pixel to quarter-pixel positions. One method of performing quarter-pixel interpolation may be used by encoders and decoders and is as follows: First, a 6-tap FIR filter is used to generate sample values ​​at half-pixel positions, and then the half-pixel position sample values ​​generated by interpolation are averaged to generate sample values ​​at quarter-pixel positions for the luminance component.

[0262] Further embodiments by modifications of the first and second are described below. According to one embodiment, the device may further comprise a decoded picture buffer (DPB), and may be configured to decode the current AU using picture predictions from referenced reference pictures stored in the DPB to obtain the decoded picture, insert the decoded picture into the DPB, assign each reference picture stored in the DPB a classification as one of short-term reference picture, long-term reference picture, and picture not used for reference, read DPB mode information from the current AU, if the DPB mode information indicates a first mode, remove one or more reference pictures classified as short-term pictures from the DPB according to a FIFO strategy, and if the DPB mode information indicates a second mode, read memory management control information including at least one command in the current AU, change the classification assigned to at least one of the reference pictures stored in the DPB, and execute at least one command to use the classification of reference pictures in the DPB to manage the removal of reference pictures from the DPB.

[0263] In one embodiment, the device may be configured to read an instruction from the current AU indicating whether the decoded picture should not be used for inter-picture prediction, to insert the decoded picture into the DPB if it is not indicated that the decoded picture should not be used for inter-picture prediction or should not be output directly, and to output the decoded picture directly in the DPB without buffering it if it is indicated that the decoded picture should not be used for inter-picture prediction and should be output directly.

[0264] According to one embodiment, the device may be further configured to assign a frame index to each reference picture in a DPB classified as a long-running picture, and to use a given reference picture in a DPB classified as a long-running picture as a referenced reference picture in the DPB when the frame index assigned to the given reference picture is referenced in the current AU.

[0265] In one embodiment, the device may be configured to perform one or more of the following: if at least one command in the current AU is a first command, reclassify reference pictures in a DPB classified as short-term reference pictures as pictures not used for reference; if at least one command in the current AU is a second command, reclassify reference pictures in a DPB classified as long-term reference pictures as pictures not used for reference; if at least one command in the current AU is a third command, reclassify reference pictures in a DPB classified as short-term pictures as long-term reference pictures and assign a frame index to the reclassified reference pictures; if at least one command in the current AU is a fourth command, set an upper frame index limit according to the fourth command and reclassify all reference pictures in a DPB classified as long-term pictures that have been assigned a frame index exceeding the upper frame index limit as pictures not used for reference; if at least one command in the current AU is a sixth command, classify the current picture as a long-term picture, a picture not used for reference, and assign a frame index to the reclassified reference pictures.

[0266] In one embodiment, the device may be configured to remove from the DPB any reference pictures that are classified as not being used for reference and therefore should no longer be output. In one embodiment, the device may be configured to read an entropy coding mode indicator from a data stream and decode the predicted residual data from the current AU using the context-adaptive variable-length coding mode if the entropy coding mode indicator indicates the context-adaptive variable-length coding mode, and using the context-adaptive binary arithmetic coding mode if the entropy coding mode indicator indicates the context-adaptive binary arithmetic coding mode. In one embodiment, the device may be configured to derive a quarter-pixel value in a referenced picture based on the motion vector in the current AU and using a 6-tap FIR filter, such that it derives a half-pixel value and averages the half-pixel values ​​of neighboring pixels.

[0267] In one embodiment, the apparatus is The current AU is decoded using picture-to-picture predictions from the referenced picture stored in the DPB to obtain the decoded picture, and the decoded picture is inserted into the DPB. Assigning a classification to each reference picture stored in the DPB as one of the following: short-term reference picture, long-term reference picture, or picture not used for reference. Read DPB mode information from the current AU. If the DPB mode information indicates a first mode, remove one or more reference pictures classified as short-term pictures from the DPB in accordance with the First-In, First-Out (FIFO) strategy. If the DPB mode information indicates a second mode, read the memory management control information, which includes at least one command in the current AU, and execute at least one command to change the classification assigned to at least one of the reference pictures stored in the DPB, and to use the classification of the reference picture in the DPB to manage the removal of the reference picture from the DPB. It can be configured to perform the following actions.

[0268] According to one embodiment, the apparatus is Read instructions from the current AU regarding whether the decoded picture will be used for picture-to-picture prediction. If it is not indicated that the decoded picture should not be used for inter-picture prediction or should not be output directly, insert the decoded picture into the DPB; if it is indicated that the decoded picture should not be used for inter-picture prediction and should be output directly, output the decoded picture directly in the DPB without buffering it. It can be configured to perform the following actions.

[0269] In one embodiment, the apparatus is Assigning a frame index to each referenced picture in the DPB classified as a long-term picture, and If a specified reference picture in a DPB classified as a long-running picture is referenced within the DPB when the frame index assigned to that specified reference picture is referenced within the current AU, then it should be used as the referenced reference picture within the DPB. It can be further configured to perform the following: According to one embodiment, the apparatus is If at least one command in the current AU is the first command, Reclassifying reference pictures within a DPB that were previously categorized as short-term reference pictures as pictures not used for reference. If at least one command in the current AU is the second command, Reclassifying reference pictures within a DPB that were previously categorized as long-term reference pictures as pictures not used for reference. If at least one command in the current AU is a third command, Reclassifying reference pictures within a DPB that were previously categorized as short-term pictures as long-term reference pictures, and assigning frame indices to the reclassified reference pictures. If at least one command in the current AU is the fourth command, The fourth command sets an upper frame index limit, and all referenced pictures in the DPB that are classified as long-term pictures and have frame indexes exceeding the upper frame index limit are reclassified as pictures not used for reference. If at least one command in the current AU is the sixth command, The current picture, which is a long-term picture, is reclassified as a picture that is not used for reference, and a frame index is assigned to the reclassified reference picture. It can be configured to perform one or more of the following: In one embodiment, the apparatus is It can be configured to remove any reference pictures from the DPB that are classified as pictures not used for reference and therefore should no longer be output.

[0270] The following describes the further optional details and characteristics of the third variant. Multiple Reference Lines (MRLs): For intra-prediction, in addition to adjacent lines of neighboring samples, one of two non-adjacent reference lines, corresponding to a position two or three lines away from the current block, can also be used as a reference line for in-picture prediction of luminance samples. Adaptive MV Resolution (AMVR): Selection of MV resolution is performed at the CU level. For interpreted CUs, the selected MV resolution is indicated and can be quarter, half, an integer, or 4 in units of luminance samples. If a half-luminance sample resolution is selected, an alternative luminance interpolation filter is used for the half-sample positions within this block, i.e., a different interpolation filter is used when the resolution is a quarter or half pixel relative to the MV.

[0271] History-Based MV Prediction (HMVP): In addition to spatially and temporally neighborly MV prediction, a new candidate type is added for merge mode and MV prediction in the AMVP candidate list. HMVP candidates are established using a 5-entry table that is maintained and updated using a first-in, first-out (FIFO) rule. The motion vector candidate list is generated using spatially and temporally neighborly and HMVP candidates. Affine Motion: An affine motion model with CU-level signaling is used for luminance. The CU-level affine motion can be either a 4-parameter or 6-parameter model. The 4-parameter model uses two MVs corresponding to two control points located at the upper-left and upper-right corners of the CU, while the 6-parameter model uses three MVs corresponding to three control points located at the upper-left, lower-left, and upper-right corners. When the CU is coded in affine motion mode, the luminance block of the CU is divided into 4x4 subblocks, and the MV at the central sample position of each subblock is calculated according to the affine motion model and set as the subblock MV based on the control points. The subblock MV is rounded to 1 / 16 luminance sample accuracy during computation, and a set of 6-tap interpolation filters is applied to generate the prediction for each subblock. For non-affine cases, merge mode and AMVP mode are used for predicting and coding the affine motion parameters.

[0272] Coefficient encoding: When encoding a coefficient level, first a flag (SigFlag) is indicated to specify whether the coefficient level is non-zero. If the flag is equal to 1 (the coefficient level is non-zero), then a further flag (Gt1-Flag) is indicated to specify whether the absolute level is greater than 1. In that case (when the absolute level is greater than 1), there are two further flags: a parity level flag (Parity-Flag) that specifies the parity of the converted coefficient level, and another flag (Gt3-Flag) that specifies whether the absolute value of the converted coefficient level is greater than 3.

[0273] The fourth modification of video decoding is performed by decoding the video from the video data stream using block-based predictive decoding and transformation-based residual decoding, the block-based predictive decoding and transformation-based residual decoding are performed by decoding the predicted residual data of the intra-predicted blocks from the video data stream using context-adaptive binary arithmetic decoding, the context-adaptive binary arithmetic decoding is performed Decoding the coordinates of the position within the transformation block representing the predicted residual data encountered when traversing the transformation coefficients of the transformation block according to a predetermined scan order, and Sequentially decoding the values ​​of the included and ranked transformation coefficients in a predetermined scan order between the last non-zero transformation coefficient and the first transformation coefficient scanned, and The predetermined scan order is selected from diagonal scan order, horizontal scan order, and vertical scan order by using a mapping that maps each of the multiple intra-prediction modes to the corresponding one among the diagonal scan order, horizontal scan order, and vertical scan order, according to the intra-prediction mode of the intra-predicted block. It is done by [the specified method].

[0274] A fourth modification relating to the present invention performs video encoding by encoding video into a video data stream using block-based predictive encoding and transform-based residual encoding, the block-based predictive encoding and transform-based residual encoding are performed by encoding intra-predicted block predictive residual data into the video data stream using context-adaptive binary arithmetic encoding, Encoding the coordinates of the position within the transformation block representing the predicted residual data encountered when traversing the transformation coefficients of the transformation block according to a predetermined scan order, and Sequentially encoding the values ​​of the included and ranked transformation coefficients in a predetermined scan order between the last non-zero transformation coefficient and the first transformation coefficient scanned, as well as The predetermined scan order is selected from diagonal scan order, horizontal scan order, and vertical scan order by using a mapping that maps each of the multiple intra-prediction modes to the corresponding one among the diagonal scan order, horizontal scan order, and vertical scan order, according to the intra-prediction mode of the intra-predicted block. It is done by [the specified method].

[0275] The present invention includes a video encoder, a video decoder, and a method for decoding video according to the above-described video decoding modification, as well as a method for encoding video according to a fourth modification of the above-described video, and a video data stream obtained by each encoding method.

[0276] The following describes further optional details and features of specific video encoding and video decoding variations of embodiments of the present invention. A fourth modified embodiment may conform to H.265 / HEVC. [PCM] Prediction, transformation, quantization, and entropy coding are bypassed, and samples are represented directly by a predefined number of bits. When PCM is enabled, the number of bits (minus 1) used to represent luminance samples and chroma samples is indicated, respectively. Furthermore, the minimum and maximum block sizes (difference from the minimum) for a 2N×2N block in which PCM can be optimally used are indicated. When a 2N×2N block is being parsed, if the size of that block is between the described minimum and maximum values, the flag indicates whether PCM mode is being used. When PCM mode is used, PCM samples are byte-aligned, the luminance samples of the block are represented / parsed directly in a raster scan with the indicated PCM luminance bit length, and then the chroma samples of the PCM block are represented / parsed in a raster scan with the indicated PCM chroma bit length, with the first half of the chroma samples being Cb samples and the remainder being Cr samples. Furthermore, the CABAC decoding engine terminates each time the PCM flag is parsed and initialized after decoding the PCM syntax.

[0277] [Intramode Dependency Coefficient Scan Order] The scan order of coefficients within an intracoded block is dynamically changed based on the transformation block size and intra mode. The scan is performed for every 4x4 subblock / region of the transformation block (e.g., a 4x4 transformation block size uses only one coefficient region, an 8x8 transformation block uses four coefficient regions, and a 16x16 transformation block size uses sixteen regions). The choice of scan method in intra mode depends on the transformation block size. For 16x16 and 32x32 transformation block sizes, the scan is performed diagonally (starting from 0,0 -> 1,0 -> 0,1 -> 2,0 -> 1,1 -> 0,2 -> 3,0, etc. - for example, decreasing the y value by step 1 and increasing the x value by step 1 at each step, restarting the y value from x+1 when y is equal to 0, and setting x to 0 as it starts from value 0,0). For 4x4 (luminance or chroma) and 8x8 (luminance) conversion block sizes, the coefficient scan order depends on the intra-mode associated with the intra-direction of the prediction of neighboring blocks. Vertical scanning is used when the prediction direction is nearly horizontal, and horizontal scanning is used when the prediction direction is nearly vertical. For other prediction directions, diagonal upper-right scanning is used.

[0278] [Conversion Quantization Bypass Mode] For each coding unit or coding block, it may be indicated whether both transformation and quantization are bypassed, which indicates that the residual signals from inter-picture or intra-picture predictions are directly entropy coded for that block. In that mode, the in-loop filter is skipped.

[0279] [Further embodiments] The embodiments of the present invention will be described again below in more general terms. The following embodiments may be optionally combined with any of the features described above. In the following further embodiments, the following options may apply to all embodiments. The data stream may be a video data stream or an audio data stream. The device for decoding the data stream may be a device for decoding video from a video data stream, or a device for decoding an audio signal from an audio data stream. The device for encoding the data stream may be a device for encoding video into a video data stream, or an audio signal into an audio data stream. The payload packet may be a coded video payload packet carrying video data, for example, encoded video data. The term “supplementary information packet” may be used interchangeably with the term “supplementary information payload packet.” The expressions “check the data stream for reliability” and “verify the data stream” may be used interchangeably.

[0280] Different embodiments can be combined; that is, any feature defined with respect to any of the embodiments may be combined with any of the further embodiments. 1. A device 20 for decoding a data stream 14, wherein the data stream includes a plurality of payload packets 16 carrying payload data, and further includes supplemental information packets 18, the supplemental information packets carrying supplemental information messages 19, and the device, From the data stream, derive an instruction 51 indicating whether one or more supplemental information messages should be considered to validate the data stream 40. A predetermined portion 13 of the data stream is determined by including one or more supplemental information messages in the predetermined portion, when instruction 51 indicates that one or more supplemental information messages should be considered in order to validate the data stream, 30. Obtaining a digital signature 43 to verify a predetermined portion 13 from the instructions in the data stream 21 Apparatus 20 is configured to perform the following actions.

[0281] 2. The apparatus according to Embodiment 1, wherein the apparatus is configured to verify a predetermined portion of a data stream by checking whether that portion conforms to a digital signature. 3. The apparatus according to Embodiment 1 or 2, wherein the apparatus is configured to provide a predetermined part and a digital signature for verifying the predetermined part. 4. A device 20 for decoding a data stream 14, wherein the device is configured to check the data stream for reliability, and the data stream includes a plurality of payload packets 16 carrying payload data, and further includes supplemental information packets 18, the supplemental information packets carrying supplemental information messages 19, and the device, From the data stream, derive an instruction 51 indicating whether supplemental information messages should be considered for validating the data stream 21. A predetermined portion 13 of the data stream that is to be verified is determined by including one or more of the supplemental information messages 19 within the predetermined portion 13 when instruction 51 indicates that supplemental information messages should be considered in order to verify the data stream 30. To obtain a digital signature 43 based on data stream 14, Check whether a given portion 13 of the data stream conforms to the digital signature 43 41 Apparatus 20 is configured to perform the following actions.

[0282] 5. Check whether a given portion 13 of the data stream conforms to the digital signature 43. The process involves applying a hash function to a predetermined portion 13 to obtain a hash value, Check whether the hash value conforms to digital signature 43, The apparatus according to any one of Embodiments 1 to 4, including the apparatus described above. 6. Check whether the hash value conforms to the digital signature 43. Forming a verification string 48 based on the hash value, Decrypting digital signature 43 to obtain the check value, The verification string 48 is checked to see if it matches the check value, The apparatus according to Embodiment 5, including the apparatus described in Embodiment 5.

[0283] 7. The predetermined portion 13 is determined by including one or more supplemental information packets in the predetermined portion 13 when the instruction indicates that one or more supplemental information messages should be considered in order to verify the predetermined portion 13. The apparatus according to any one of Embodiments 1 to 6.

[0284] 8. The apparatus according to any one of embodiments 1 to 7, configured to include in the predetermined portion 13 all supplemental information packets that carry any supplemental information messages to be included in the predetermined portion 13. 9. The apparatus according to any one of embodiments 1 to 8, wherein the decision of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a predetermined portion 13 is conditional on the result of checking whether or not the supplemental information payload packet contains a supplemental information message of any supplemental information message type from a set of supplemental information message types. 10. The instruction is a flag having a first state and a second state, and the device controls a predetermined portion 13 of the data stream. If the flag has the first state, one or more supplemental information messages to be included in a predetermined part 13 are determined based on a predefined set of supplemental information messages among the supplemental information messages. If the flag has the second state, none of the supplementary information messages should be included in the predetermined part 13. The apparatus according to any one of embodiments 1 to 9, configured to be determined by

[0285] 11. The apparatus according to Embodiment 10, wherein a predefined set of supplemental information messages includes all supplemental information messages of supplemental information packets that reference multiple payload packets 16. 12. The apparatus according to Embodiment 10, wherein a predefined set of supplemental information messages includes all supplemental information messages of any type from a set of types of supplemental information messages. 13. When the instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the system is further configured to derive identification information of one or more supplemental information messages that should be included in a given portion 13 from the data stream. The apparatus according to any one of Embodiments 1 to 12.

[0286] 14. The apparatus according to Embodiment 13, wherein, when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the apparatus is configured to derive from the data stream a syntactic element that identifies a set of types of supplemental information messages that should be included in a predetermined portion 13. 15. The apparatus according to any one of embodiments 1 to 14, configured to derive from the data stream syntactic elements that indicate whether one or more supplemental information messages should be considered for validation of the data stream and that signal identification information of one or more supplemental information messages to be included in a predetermined portion 13.

[0287] 16. A syntactic element has multiple first and second states, and the device, If a syntactic element has one of the first states, one or more supplementary information messages to be included in a predetermined part 13 are determined according to the state of the syntactic element, If the syntactic element has a second state, none of the one or more supplemental information messages shall be included in the predetermined part 13. The apparatus according to embodiment 15, configured to perform the following:

[0288] 17. The apparatus according to Embodiment 13, wherein, when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the apparatus is configured to derive from the data stream a first syntactic element indicating the number of supplemental information message types to be considered for a given portion 13, and a number of second syntactic elements, each identifying each of the supplemental information message types that should be included in the given portion 13. 18. The determination of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a predetermined portion 13 is further configured to be conditional on the result of checking whether the supplemental information packet is preceded by a prefix supplemental information packet that instructs the predetermined portion 13 to include subsequent supplemental information packets. The apparatus described in Embodiment 7. 19. The decision of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a predetermined portion 13 is further configured to be conditional on the result of checking whether the supplemental information packet contains a supplemental information message that instructs the predetermined portion 13 to include the supplemental information packet. The apparatus described in Embodiment 7.

[0289] 20. The decision of whether or not to include one of several supplemental information messages in a given portion is further configured to be conditional on the result of checking whether the supplemental information message is preceded by a prefix supplemental information message that instructs the given portion 13 to include a subsequent supplemental information message. The apparatus described in Embodiment 7. 21. The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a given portion is further configured to be conditional on the result of checking whether the supplemental information packet contains a nesting supplemental information message that instructs the supplemental information packet to be included in the given portion 13. The apparatus described in Embodiment 7.

[0290] 22. The decision of whether or not to include one of several supplemental information messages in a given portion is further configured to be conditional on the result of checking whether the supplemental information message is included in a nesting supplemental information message that instructs the given portion 13 to include the supplemental information message. The apparatus described in Embodiment 7. 23. When determining the predetermined portion 13, The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a predetermined portion is configured to be conditional on the result of checking whether the supplemental information packet contains any supplemental information message of any type from the set of types of supplemental information messages that should be included in the predetermined portion 13. The apparatus according to any one of Embodiments 1 to 22.

[0291] 24. When determining the designated portion, The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a predetermined portion is configured to be conditional on the result of checking whether the first supplemental information message among one or more supplemental information messages carried within the supplemental information packet is one of a set of types of supplemental information messages that should be included in the predetermined portion 13. The apparatus according to any one of Embodiments 1 to 23.

[0292] 25. Deriving several parts from a data stream, wherein the data stream is verifiable in units of several parts, and several parts include a predetermined part 13. Assigning each of the payload packets 16 to one of several parts, An apparatus according to any one of embodiments 1 to 24, configured to perform the following: 26. When determining the predetermined portion 13, When the instruction indicates that one or more supplemental information messages should be considered to validate the data stream, the predetermined part 13 is configured to include one supplemental information message from among multiple supplemental information messages having any type from a set of types of supplemental information messages, and the predetermined part 13 has a predetermined rank in an order defined among several parts. The apparatus described in Embodiment 25.

[0293] 27. The apparatus according to embodiment 26, wherein a predetermined rank is the highest rank in an order defined among several parts. 28. When determining the predetermined portion 13, The payload packet 16 assigned to a predetermined portion 13 is included in the predetermined portion 13, When the instruction indicates that one or more supplemental information messages should be considered to validate the data stream, a predetermined portion 13 includes supplemental information messages for the data stream associated with the payload packet 16 assigned to the predetermined portion 13, having any type from the set of supplemental information message types. The apparatus according to embodiment 25, configured to perform the following. 29. The apparatus according to Embodiment 25 or 28, wherein each supplemental information packet is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information packet carrying any type of supplemental information from a set of types of supplemental information to one or more portions to which the payload packet associated with each supplemental information packet is allocated.

[0294] 30. When determining the predetermined portion 13, When the instructions indicate that supplementary information should be considered to validate the data stream, based on the instructions in the data stream, The predetermined portion 13 includes all supplemental information of a data stream having any type from a set of supplemental information types, wherein the predetermined portion 13 has a predetermined rank, for example, the highest rank in an order defined among several portions. A predetermined portion 13 includes all supplemental information of the data stream associated with the payload packet 16 assigned to the predetermined portion 13, having any type from the set of supplemental information types. The apparatus according to embodiment 25, 28, or 29, which is configured to be determined from among the following.

[0295] 31. The apparatus according to Embodiment 25, wherein each supplemental information message is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information message of a data stream having any type from a set of types of supplemental information messages to one of the parts, the one part of which is associated with one of the further parts, and the further part is associated with the payload packet 16 associated with each supplemental information message. 32. The apparatus according to Embodiment 25, wherein each supplemental information packet is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information packet of a data stream carrying any type of a set of types of supplemental information messages to one of the parts, the one part of which is associated with another of the parts, and the other of the parts is associated with the payload packet 16 associated with each supplemental information packet. 33. The system is configured to derive syntactic elements that indicate the number of parts from a data stream, where the syntactic elements have values, and the device, It is configured to infer that some parts of the number correspond to the value plus 1, multiplied by 2. The apparatus according to Embodiment 31 or 32.

[0296] 34. The system is configured to derive syntactic elements that indicate the number of parts from a data stream, where the syntactic elements have values, and the device, It is configured to infer that the number of some parts corresponds to the value plus 1. The apparatus according to Embodiment 31 or 32. 35. The apparatus according to embodiment 33 or 34, wherein one part and one further part succeed to one another in a hierarchical order defined among several parts, for example, one part succeeds to one further part in a hierarchical order. 36. The apparatus according to embodiment 33 or 34, wherein the position index of one part in a hierarchical order defined between parts corresponds to the position index of one further part in the hierarchical order plus half the number of parts.

[0297] 37. The apparatus according to any one of embodiments 1 to 36, wherein each supplemental information message is associated with one of the payload packets 16. 38. The apparatus according to any one of embodiments 1 to 37, wherein each supplemental information message is associated with one of the associated payload packets 16, and the associated payload packet is part of the same picture unit or the same access unit as the supplemental information packet. 39. When determining a predetermined part 13, check whether one or more supplemental information messages that should be included in the predetermined part 13 contain multiple instances of the same supplemental information message, When one or more supplementary information messages to be included in a predetermined part 13 include multiple instances of the same supplementary information message, it is necessary to include only one instance of the multiple instances of the same supplementary information message. An apparatus according to any one of embodiments 1 to 38, configured to perform the following:

[0298] 40. The apparatus according to Embodiment 39, configured to perform a check on a single picture unit or access unit when checking whether one or more supplemental information messages that should be included in a predetermined portion 13 include multiple instances of the same supplemental information message. 41. The apparatus according to embodiment 39, configured to position one instance of a plurality of identical supplemental information messages at a predetermined location within a predetermined portion 13. 42. The designated position is It is defined for the start or end position of the picture unit included in the predetermined part 13, or A predetermined payload packet included in a predetermined portion 13, for example, a predetermined payload packet or a predetermined supplementary information packet, is defined as follows: The apparatus described in Embodiment 41.

[0299] 43. The apparatus according to any one of embodiments 39 to 42, configured to position one or more supplemental information messages to be included in a predetermined portion 13 according to a predefined order of one or more supplemental information messages within a predetermined portion 13. 44. A device 10 for encoding a data stream, wherein the device is configured to make the data stream verifiable in terms of reliability, and to encode a plurality of payload packets 16 that carry payload data within the data stream, further encoding supplemental information packets 18 within the data stream, and encoding the supplemental information packets that carry supplemental information messages, and the device is configured to do the following: Insert an instruction 51 into the data stream indicating whether one or more supplemental information messages should be considered to validate the data stream. Determine a predetermined portion 13 of a data stream by including one or more supplemental information messages within the predetermined portion 13 when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, and determine that the predetermined portion 13 should be checkable with respect to reliability, Obtaining a digital signature 43 based on a predetermined portion 13 41' Apparatus 10 is configured to perform the following actions.

[0300] 45. When obtaining a digital signature 43, The process involves applying a hash function to a predetermined portion 13 to obtain a hash value, Obtaining a digital signature 43 based on the hash value, The apparatus according to embodiment 44, configured to perform the following: 46. ​​When obtaining a digital signature 43, Form a verification string 48 based on the hash value. Sign the verification string 48 to obtain the digital signature 43. The apparatus according to embodiment 45, configured to perform the following. 47. A predetermined portion 13 is determined by including one or more supplemental information packets in the predetermined portion 13 when the instruction indicates that one or more supplemental information messages should be considered in order to verify the predetermined portion 13. The apparatus according to any one of embodiments 44 to 46.

[0301] 48. The apparatus according to any one of embodiments 44 to 47, configured to include in the predetermined portion 13 all supplemental information packets that carry any supplemental information messages to be included in the predetermined portion 13. 49. The apparatus according to any one of embodiments 44 to 48, wherein the decision of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a predetermined portion 13 is conditional on the result of checking whether the supplemental information packet includes a supplemental information message of any supplemental information message type from a set of supplemental information message types. 50. The instruction is a flag having a first state and a second state, and the device, The flag is set to the first state, and one or more supplementary information messages that should be included in a given part 13 are determined based on a predefined set of supplementary information messages, The flag is set to the second state, and none of the supplementary information messages are included in the designated part 13, The apparatus according to any one of embodiments 44 to 49, configured to be determined from among the following.

[0302] 51. The apparatus according to Embodiment 50, wherein a predefined set of supplemental information messages includes all supplemental information messages of supplemental information packets that reference multiple payload packets 16. 52. The apparatus according to Embodiment 50, wherein a predefined set of supplemental information messages includes all supplemental information messages of any type from a set of types of supplemental information messages.

[0303] 53. The apparatus according to any one of embodiments 44 to 52, further configured to insert into the data stream identification information of one or more supplemental information messages to be included in a predetermined portion 13 when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream.

[0304] 54. The apparatus according to Embodiment 53, wherein, when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, a syntactic element is configured to insert into the data stream a set of types of supplemental information messages that should be included in a predetermined portion 13 from the data stream. 55. The apparatus according to any one of embodiments 44 to 54, configured to insert into the data stream a syntactic element that indicates whether one or more supplemental information messages should be considered for validation of the data stream and signals identification information of one or more supplemental information messages to be included in a predetermined portion 13. 56. A syntactic element has multiple first states and second states, and the device, Setting a syntactic element to one of the first states in accordance with one or more supplementary information messages that should be included in a predetermined part 13, The syntactic element is set to the second state, so that none of the one or more supplementary information messages are included in the predetermined part 13, The apparatus according to embodiment 55, configured to distinguish between the following.

[0305] 57. The apparatus according to Embodiment 53, wherein, when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the apparatus is configured to insert into the data stream a first syntactic element indicating the number of supplemental information message types to be considered for a given portion 13, and a number of second syntactic elements, each identifying each of the supplemental information message types that should be included in the given portion 13. 58. The determination of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a given portion is further configured to be conditional on the result of checking whether the supplemental information packet is preceded by a prefix supplemental information packet that instructs the given portion 13 to include subsequent supplemental information packets. The apparatus described in Embodiment 53. 59. The decision of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a predetermined portion 13 is further configured to be conditional on the result of checking whether the supplemental information packet contains a supplemental information message that instructs the predetermined portion 13 to include the supplemental information packet. The apparatus described in Embodiment 47.

[0306] 60. The decision of whether or not to include one of several supplemental information messages in a given portion is further configured to be conditional on the result of checking whether the supplemental information message is preceded by a prefix supplemental information message that instructs the given portion 13 to include a subsequent supplemental information message. The apparatus described in Embodiment 47. 61. The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a given portion is further configured to be conditional on the result of checking whether the supplemental information packet contains a nesting supplemental information message that instructs the supplemental information packet to be included in the given portion 13. The apparatus described in Embodiment 47. 62. The decision of whether or not to include one of several supplemental information messages in a given portion is further configured to be conditional on the result of checking whether the supplemental information message is included in a nesting supplemental information message that instructs the given portion 13 to include a supplemental information packet. The apparatus described in Embodiment 47.

[0307] 63. When determining the predetermined portion 13, The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a predetermined portion is configured to be conditional on the result of checking whether the supplemental information packet contains any supplemental information message of any type from the set of types of supplemental information messages that should be included in the predetermined portion 13. The apparatus according to any one of embodiments 44 to 62. 64. When determining the predetermined portion 13, The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a predetermined portion is configured to be conditional on the result of checking whether the first supplemental information message among one or more supplemental information messages carried within the supplemental information packet is one of a set of types of supplemental information messages that should be included in the predetermined portion 13. The apparatus according to any one of embodiments 44 to 63.

[0308] 65. The apparatus according to any one of embodiments 44 to 64, wherein each supplemental information packet is configured to provide supplemental information packets such that it does not contain both a supplemental information message of one type from a set of types of supplemental information packets that should be included in a predetermined portion 13, and a supplemental information message of a type that is not included in the set of types. 66. In a data stream, to indicate several parts, wherein the data stream is verifiable in units of several parts, and several parts include a predetermined part 13. Assigning each of the payload packets 16 to one of several parts, An apparatus according to any one of embodiments 44 to 65, configured to perform the following: 67. When determining the predetermined portion 13, When the instruction indicates that one or more supplemental information messages should be considered to validate the data stream, the given portion 13 includes one supplemental information message from among multiple supplemental information messages having any type from a set of supplemental information message types, and the given portion 13 has a predetermined rank in an order defined among several portions. The apparatus described in Embodiment 66.

[0309] 68. The apparatus according to embodiment 67, wherein a predetermined rank is the highest rank in an order defined among several parts. 69. When determining the predetermined portion 13, The payload packet 16 assigned to a predetermined portion 13 is included in the predetermined portion 13, When the instruction indicates that one or more supplemental information messages should be considered to validate the data stream, a predetermined portion 13 includes supplemental information messages for the data stream associated with the payload packet 16 assigned to the predetermined portion 13, having any type from the set of supplemental information message types. The apparatus according to embodiment 66, configured to perform the following: 70. The apparatus according to embodiment 66 or 67, wherein each supplemental information packet is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information packet carrying any type of supplemental information from a set of types of supplemental information to one or more portions to which the payload packet associated with each supplemental information packet is assigned.

[0310] 71. When determining the predetermined portion 13, When the instructions indicate that supplementary information should be considered to validate the data stream, The predetermined portion 13 includes all supplemental information of a data stream having any type from a set of supplemental information types, wherein the predetermined portion 13 has a predetermined rank, for example, the highest rank in an order defined among several portions. A predetermined portion 13 includes all supplemental information of the data stream associated with the payload packet 16 assigned to the predetermined portion 13, having any type from the set of supplemental information types. The apparatus according to embodiment 66, 67, or 68, which is configured to insert instructions into the data stream that are instructions for distinguishing between the two.

[0311] 72. The apparatus according to Embodiment 66, wherein each supplemental information message is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information message of a data stream having any type from a set of types of supplemental information messages to one of the parts, the one part of which is associated with another of the parts, and the other of the parts is associated with the payload packets 16 associated with each supplemental information message. 73. The apparatus according to Embodiment 33, wherein each supplemental information packet is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information packet of a data stream carrying any type of a set of types of supplemental information messages to one of the parts, the one part of which is associated with another of the parts, and the other of the parts is associated with the payload packet 16 associated with each supplemental information packet. 74. The data stream is configured to insert syntactic elements that indicate the number of parts, and the syntactic elements have a value, and the device, The value is configured to be set to the value obtained by dividing the number of some parts by 2 and subtracting 1. The apparatus according to embodiment 72 or 73.

[0312] 75. It is configured to derive syntactic elements that indicate the number of parts from a data stream, where the syntactic elements have values, and the device, The value is configured to be set to a value obtained by subtracting 1 from the number of some parts. The apparatus according to embodiment 72 or 73. 76. The apparatus according to embodiment 74 or 75, wherein one part and one further part succeed to one another in a hierarchical order defined among several parts, for example, one part succeeds to one further part in a hierarchical order. 77. The apparatus according to embodiment 74 or 75, wherein the position index of one part in a hierarchical order defined between parts corresponds to the position index of a further part in the hierarchical order plus half the number of parts.

[0313] 78. The apparatus according to any one of embodiments 44 to 77, wherein each supplemental information message is associated with one of the payload packets 16. 79. The apparatus according to any one of embodiments 44 to 78, wherein each supplemental information message is associated with one of the associated payload packets 16, and the associated payload packet is part of the same picture unit or the same access unit as the supplemental information packet. 80. When determining a predetermined part 13, check whether one or more supplemental information messages that should be included in the predetermined part 13 contain multiple instances of the same supplemental information message, When one or more supplementary information messages to be included in a predetermined part 13 include multiple instances of the same supplementary information message, it is necessary to include only one instance of the multiple instances of the same supplementary information message. An apparatus according to any one of embodiments 44 to 79, configured to perform the following:

[0314] 81. The apparatus according to embodiment 80, configured to perform a check on a single picture unit or access unit when checking whether one or more supplemental information messages that should be included in a predetermined portion 13 include multiple instances of the same supplemental information message. 82. The apparatus according to embodiment 80, configured to position one instance of a plurality of identical supplemental information messages at a predetermined location within a predetermined portion 13. 83. The designated position is It is defined for the start or end position of the picture unit included in the predetermined part 13, or A predetermined payload packet included in a predetermined portion 13, for example, a predetermined payload packet or a predetermined supplementary information packet, is defined as follows: The apparatus described in Embodiment 82.

[0315] 84. The apparatus according to any one of embodiments 80 to 83, configured to position one or more supplemental information messages to be included in a predetermined portion 13 according to a predefined order of one or more supplemental information messages within a predetermined portion 13. 85. The apparatus according to any one of embodiments 44 to 84, wherein when a supplemental information packet contains multiple instances of the same supplemental information packet, the apparatus is configured to insert multiple instances into a data stream without being interleaved by another of the supplemental information packets. 86. The apparatus according to any one of embodiments 44 to 85, configured to provide a data stream such that when the supplemental information messages of a data stream include multiple instances of a first supplemental information message and multiple instances of a second supplemental information message, one instance each of the first supplemental information message and the second supplemental information message is included in one of the supplemental information packets.

[0316] 87. A method for decoding a data stream 14, wherein the data stream includes a plurality of payload packets 16 carrying payload data, and further includes supplemental information packets 18, the supplemental information packets carrying supplemental information messages 19, and the method From the data stream, derive an instruction 51 indicating whether one or more supplemental information messages should be considered to validate the data stream 40. A predetermined portion 13 of the data stream is determined by including one or more supplemental information messages in the predetermined portion, when instruction 51 indicates that one or more supplemental information messages should be considered in order to validate the data stream, 30. Obtaining a digital signature 43 to verify a predetermined portion 13 from the instructions in the data stream 21 Methods that include...

[0317] 88. A method for decoding a data stream 14, the method comprising checking the data stream for reliability, wherein the data stream comprises a plurality of payload packets 16 carrying payload data, and further comprises supplem...

Claims

1. A device (20) for decoding a data stream (14), wherein the device is configured to check the data stream for reliability, the data stream includes a plurality of payload packets (16) carrying payload data, and further includes supplemental information packets (18), the supplemental information packets carrying supplemental information messages (19), and the device, From the data stream, an instruction (51) is derived (21) indicating whether a supplementary information message should be considered for verifying the data stream. A predetermined portion (13) of the data stream that is to be verified is determined by including one or more of the supplemental information messages (19) within the predetermined portion (13) when an instruction (51) indicates that supplemental information messages should be considered in order to verify the data stream (30). To obtain a digital signature (43) based on the aforementioned data stream (14), Checking whether a predetermined portion (13) of the data stream conforms to the digital signature (43) (41) It is configured to do the following: The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by The device (20) is shown.

2. Checking whether the predetermined portion (13) of the data stream conforms to the digital signature (43) is The predetermined portion (13) is subjected to the hash function to obtain a hash value, Checking whether the hash value conforms to the digital signature (43), The apparatus according to claim 1, including the following:

3. Checking whether the hash value conforms to the digital signature (43) is A verification string (48) is formed based on the aforementioned hash value, Deciphering the aforementioned digital signature (43) to obtain a check value, The verification string (48) is checked to see if it matches the check value, The apparatus according to claim 2, including the following:

4. The predetermined portion (13) is determined by including one or more of the supplemental information packets in the predetermined portion (13) when the instruction indicates that one or more of the supplemental information messages should be considered in order to verify the predetermined portion (13). The apparatus according to any one of claims 1 to 3.

5. The apparatus according to any one of claims 1 to 4, wherein all supplemental information packets that carry any supplemental information messages to be included in the predetermined portion (13) are configured to be included in the predetermined portion (13).

6. The apparatus according to any one of claims 1 to 5, wherein the decision of whether or not to include one of the plurality of supplemental information packets in the predetermined portion (13) is conditional on the result of checking whether or not the supplemental information payload packet contains a supplemental information message of any supplemental information message type from a set of supplemental information message types.

7. The instruction is a flag having a first state and a second state, and the device determines the predetermined portion (13) of the data stream, If the flag has the first state, one or more supplemental information messages to be included in the predetermined portion (13) are determined based on a predefined set of supplemental information messages among the supplemental information messages. If the flag has the second state, none of the supplementary information messages shall be included in the predetermined portion (13). The apparatus according to any one of claims 1 to 6, configured to be determined by

8. The apparatus according to claim 7, wherein the predefined set of supplemental information messages includes all of the supplemental information messages of the supplemental information packets that refer to the plurality of payload packets (16).

9. The apparatus according to claim 7, wherein the predefined set of supplemental information messages includes all supplemental information messages of any type from the set of types of supplemental information messages.

10. If the instruction indicates that one or more supplemental information messages should be considered in order to verify the data stream, the system is further configured to derive identification information of one or more supplemental information messages that should be included in the predetermined portion (13) from the data stream. The apparatus according to any one of claims 1 to 9.

11. The apparatus according to claim 10, wherein, when the instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the apparatus is configured to derive from the data stream syntactic elements that identify a set of types of the one or more supplemental information messages to be included in the predetermined portion (13).

12. The apparatus according to any one of claims 1 to 11, configured to derive from the data stream syntactic elements that indicate whether the one or more supplemental information messages should be considered for validating the data stream and that signal identification information of the one or more supplemental information messages to be included in the predetermined portion (13).

13. The syntactic element has a plurality of first and second states, and the device, If the syntactic element has one of the first states, one or more supplementary information messages to be included in the predetermined part (13) are determined according to the state of the syntactic element. If the syntactic element has the second state, none of the one or more supplementary information messages are included in the predetermined part (13), The apparatus according to claim 12, configured to perform the following:

14. The apparatus according to claim 13, wherein, when the instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the apparatus is configured to derive from the data stream a first syntactic element indicating the number of supplemental information message types to be considered for the predetermined portion (13), and a number of second syntactic elements, each identifying a respective supplemental information message type to be included in the predetermined portion (13).

15. The determination of whether or not to include one of the plurality of supplemental information packets in the predetermined portion (13) is further configured to be conditional on the result of checking whether the supplemental information packet is preceded by a prefix supplemental information packet that instructs the predetermined portion (13) to include a subsequent supplemental information packet. The apparatus according to claim 4.

16. The determination of whether or not to include one of the plurality of supplemental information packets in the predetermined portion (13) is further configured to be conditional on the result of checking whether the supplemental information packet contains a supplemental information message instructing that the predetermined portion (13) be included in the supplemental information packet. The apparatus according to claim 4.

17. The determination of whether or not to include one of the multiple supplemental information messages in the predetermined portion is further configured to be conditional on the result of checking whether the supplemental information message is preceded by a prefix supplemental information message that instructs the predetermined portion (13) to include a subsequent supplemental information message. The apparatus according to claim 4.

18. The determination of whether or not to include one of the plurality of supplemental information packets in the predetermined portion is further configured to be conditional on the result of checking whether the supplemental information packet contains a nesting supplemental information message instructing the predetermined portion (13) to include the supplemental information packet. The apparatus according to claim 4.

19. The determination of whether or not to include one of the multiple supplemental information messages in the predetermined portion is further configured to be conditional on the result of checking whether the supplemental information message is included in a nesting supplemental information message that instructs the predetermined portion (13) to include the supplemental information message. The apparatus according to claim 4.

20. When determining the predetermined portion (13), The decision of whether or not to include one of the plurality of supplemental information packets in the predetermined portion is configured to be conditional on the result of checking whether the supplemental information packet contains any supplemental information message of any type from the set of types of supplemental information messages that should be included in the predetermined portion (13). The apparatus according to any one of claims 1 to 19.

21. When determining the predetermined portion, The decision of whether or not to include one of the plurality of supplemental information packets in the predetermined portion is conditional on the result of checking whether the first supplemental information message among the one or more supplemental information messages carried in the supplemental information packet is one of the set of types of supplemental information messages that should be included in the predetermined portion (13). The apparatus according to any one of claims 1 to 20.

22. The derivation involves deriving several parts from the data stream such that the data stream is verifiable in units of several parts, and the several parts include the predetermined part (13). Each of the payload packets (16) is assigned to one of the aforementioned parts, The apparatus according to any one of claims 1 to 21, configured to perform the following:

23. The apparatus according to any one of claims 1 to 22, wherein each of the supplementary information messages is associated with one of the payload packets (16).

24. The apparatus according to any one of claims 1 to 23, wherein each of the supplemental information messages is associated with one of the associated payload packets (16), and the associated payload packet is part of the same picture unit or the same access unit as the supplemental information packet.

25. A device (10) for encoding a data stream, wherein the device is configured to make the data stream verifiable in terms of reliability, and to encode a plurality of payload packets (16) that carry payload data within the data stream, further encoding supplemental information packets (18) within the data stream, and the supplemental information packets that carry supplemental information messages, and the device is configured to do the following: Insert an instruction (51) into the data stream indicating whether one or more supplemental information messages should be considered in order to verify the data stream (23), The predetermined portion (13) of the data stream is determined by including one or more of the supplemental information messages within the predetermined portion (13) when the instruction indicates that one or more supplemental information messages should be considered in order to verify the data stream (30'), and the predetermined portion (13) is determined to be checkable with respect to reliability (30'), Obtaining a digital signature (43) based on the predetermined portion (13) (41') It is configured to do the following: The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by Apparatus (10).

26. A method for decoding a data stream (14), the method comprising checking the data stream for reliability, wherein the data stream comprises a plurality of payload packets (16) carrying payload data, and further comprises supplemental information packets (18), the supplemental information packets carrying supplemental information messages (19), The method described above is From the data stream, an instruction (51) is derived (21) indicating whether a supplementary information message should be considered for verifying the data stream. The predetermined portion (13) of the data stream that is to be verified is determined by including one or more of the supplemental information messages (19) within the predetermined portion (13) when the instruction (51) indicates that supplemental information messages should be considered in order to verify the data stream (30). To obtain a digital signature (43) based on the data stream (14), Checking (41) whether a given portion (13) of the data stream conforms to the digital signature (43). Includes, The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by method.

27. A method for encoding a data stream, the method comprising: making the data stream verifiable in terms of reliability; encoding a plurality of payload packets (16) carrying payload data within the data stream, further encoding supplemental information packets within the data stream, the supplemental information packets carrying supplemental information messages, and the method Insert an instruction (51) into the data stream indicating whether one or more supplemental information messages should be considered in order to verify the data stream. Determine (30') a predetermined portion (13) of the data stream by including one or more of the supplemental information messages within the predetermined portion (13) when the instruction indicates that one or more of the supplemental information messages should be considered in order to verify the data stream, and determine (30') that the predetermined portion (13) should be checkable with respect to reliability. Obtaining a digital signature (43) based on the predetermined portion (13) (41') Includes, The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by method.

28. A data stream having data signals encoded within the data stream, the data stream being checkable with respect to reliability, and the data stream being A plurality of payload packets (16) that carry payload data, wherein the data stream further includes supplemental information packets, and the supplemental information packets carry supplemental information messages, and the plurality of payload packets (16) Instructions indicating whether one or more supplemental information messages should be considered regarding the reliability of the data stream, Includes, The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by Data stream.

29. A device (20) for decoding a data stream (14), wherein the device is configured to check the data stream for reliability, and the data stream includes a temporal sequence of segments (17), and the device decodes the segments (17*) of the temporal sequence of segments, A predetermined portion (13) of the segment is subjected to a hash function (31) to obtain a hash value (33). (21) deriving an instruction (61) from the data stream, wherein the instruction (61) is Whether the segment is the first segment of the signed subsequence of the temporal sequence, Whether the segment is the last segment of the signed subsequence of the temporal sequence, To instruct, to derive (21), Form a verification string (48) based on the hash value (33) and the instruction (61), The process involves deriving a digital signature (43) from the data stream and checking whether the verification string (48) conforms to the digital signature (43). It is configured to be verified by, The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by Apparatus (20).

30. Checking whether the verification string (48) conforms to the digital signature (43) is equivalent to deciphering the digital signature (43) and obtaining the check value. The verification string (48) is checked to see if it matches the check value, The apparatus according to claim 29, including the apparatus described in claim 29.

31. The apparatus according to claim 29 or 30, configured to derive the instruction from a supplemental information message associated with the segment.

32. The apparatus according to claim 31, wherein the supplemental information message indicates a certificate for decrypting the hash function and / or the digital signature (43).

33. The apparatus according to claim 29, configured to derive the instruction from a supplemental information message indicating the digital signature (43).

34. The device is configured to derive an instruction from the data stream indicating whether the segment is the first segment of a signed subsequence of the temporal sequence, and the device is configured to derive an instruction indicating whether the segment is the first segment of a signed subsequence of the temporal sequence. When the segment is the first segment of the signed subsequence, the verification string (48) is configured to form independently of any portion of the data stream that precedes the segment in the temporal sequence. The apparatus according to any one of claims 29 to 33.

35. If the segment is not the first segment of the signed subsequence, the verification string (48) is formed based on the previous hash value obtained for the preceding segment of the segment. The apparatus according to claim 34.

36. The apparatus according to any one of claims 29 to 35, wherein the apparatus is configured to derive several parts, wherein the segment (17) is verifiable in units of several parts, and the several parts include the predetermined part (13), and the parts of the several parts have a hierarchical order.

37. The aforementioned device If the predetermined portion (13) is not the first portion in the hierarchical order of the portions, the verification string (48) is formed based on the previous hash value obtained for the preceding portion that precedes the predetermined portion (13) in the hierarchical order. The apparatus according to claim 36.

38. The apparatus according to claim 36 or 37, wherein the predetermined portion (13) is the first portion in the hierarchical order of the portions, and the apparatus is configured to, when verifying subsequent portions of the portions that follow the predetermined portion (13) in the hierarchical order, to form a subsequent verification string (48) for verifying the subsequent portion based on the hash value obtained by applying the predetermined portion (13) to the hash function.

39. The apparatus according to any one of claims 36 to 38, configured to selectively derive the instruction from the data stream when the predetermined portion (13) is the first portion in the hierarchical order of several portions.

40. A device (10) for encoding the data stream, wherein the device is configured to make the data stream verifiable in terms of reliability and to encode a temporal sequence of segments within the data stream, and the device encodes the segments of the temporal sequence of segments, A hash value is obtained by applying a hash function to a predetermined portion (13) of the aforementioned segment. Within the aforementioned data stream, Whether the segment is the first segment of the signed subsequence of the temporal sequence, Whether the segment is the last segment of the signed subsequence of the temporal sequence Insert an instruction (61) that instructs (23), Form a verification string (48) based on the hash value and the instruction, Obtaining a digital signature (43) based on the verification string (48) It is configured to allow checking regarding reliability, The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by Apparatus (10).

41. A method for decoding a data stream (14), the method comprising checking the data stream for reliability, wherein the data stream comprises a temporal sequence of segments (17), and the method comprises a segment (17*) of the temporal sequence of segments, A predetermined portion (13) of the segment is subjected to a hash function (31) to obtain a hash value (33). (21) Deriving the instruction (61) from the data stream, Whether the segment is the first segment of the signed subsequence of the temporal sequence, Whether the segment is the last segment of the signed subsequence of the temporal sequence, To instruct, to derive (21), Form a verification string (48) based on the hash value (33) and the instruction (61), The process involves deriving a digital signature (43) from the data stream and checking whether the verification string (48) conforms to the previous digital signature (43). This includes verifying by The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by method.

42. A method for encoding a data stream, the method comprising making the data stream reliable and encoding a temporal sequence of segments within the data stream, wherein the method comprises: A hash value is obtained by applying a hash function to a predetermined portion (13) of the aforementioned segment. Within the data stream, Whether the segment is the first segment of the signed subsequence of the temporal sequence, Whether the segment is the last segment of the signed subsequence of the temporal sequence Insert instructions that indicate, Forming a verification string (48) based on the hash value and according to the instructions, Obtaining a digital signature (43) based on a verification string (48) This includes making it possible to check reliability, The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by method.

43. A data stream, wherein the data stream is checkable in terms of reliability, and the data stream is Temporal sequence of segments Whether the segment is the first segment of the signed subsequence of the temporal sequence, Whether the segment is the last segment of the signed subsequence of the temporal sequence Instructions to give instructions Includes, The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by Data stream.

44. A device (20) for decoding a data stream (14), wherein the device is configured to check the data stream for reliability, and the data stream includes a temporal sequence of segments, and the device From the data stream, derive whether the verification chain within the temporal sequence of the segment was interrupted in the segment (21), If the data stream indicates that the verification chain has been interrupted, and the data stream includes a digital signature (43) of the segment, Deriving the digital signature (43) from the data stream (21), In the aforementioned temporal sequence, a verification string (48) independent of the preceding segment immediately preceding the segment (39) is formed. The verification string (48) is checked (63) to see if it conforms to the digital signature (43), It is configured to do the following: The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by Apparatus (20).

45. The apparatus according to claim 44, configured to derive an instruction from the data stream indicating whether the verification chain was interrupted at the segment.

46. The apparatus according to claim 44 or 45, configured to check whether the segment is checkable in terms of reliability, and, if the segment is not checkable in terms of reliability, to assume that the verification chain is interrupted at the segment.

47. Checking whether the verification string (48) conforms to the digital signature (43) is Deciphering the aforementioned digital signature (43) to obtain a check value, The verification string (48) is checked to see if it matches the check value, The apparatus according to claim 44, including the apparatus described in claim 44.

48. The apparatus according to any one of claims 44 to 47, wherein forming the verification string (48) includes applying a hash function to a predetermined portion (13) of the segment to obtain a hash value, and forming the verification string (48) based on the hash value.

49. Forming the verification string (48) includes forming the verification string (48) using a previous hash value if the data stream does not indicate an interruption of the verification chain and the data stream includes a digital signature (43) of the segment, the previous hash value is obtained by applying a hash function to the portion of the preceding segment. The apparatus according to any one of claims 44 to 48.

50. The apparatus according to any one of claims 44 to 49, wherein forming the verification string (48) is independent of any of the segments preceding the segment in the temporal sequence when the data stream indicates that the verification chain has been interrupted and the data stream includes a digital signature (43) of the segment.

51. Checking the segment for reliability is done if the data stream indicates that the verification chain has been interrupted, and if the data stream includes a digital signature (43) of the segment. Check whether the aforementioned segment is the first segment of a further verification chain, When the preceding odd segment is the first segment of a further verification chain, the verification string (48) is formed which is independent of any of the segments preceding the segment in the temporal sequence, The apparatus according to any one of claims 44 to 50, including the following:

52. Checking the reliability of the aforementioned segment is necessary if the segment is not the first segment in a further verification chain. This includes forming the verification string (48) using a further earlier hash value, the further earlier hash value being obtained by applying a hash function to a portion of a further preceding segment of the temporal sequence. The apparatus according to claim 51.

53. Checking the segment for reliability is done if the data stream indicates that the verification chain has been interrupted, and if the data stream includes a digital signature (43) of the segment. Check whether the aforementioned segment is the first segment of a further verification chain, If the segment is not the first segment of a further verification chain, a verification string (48) is formed using a further earlier hash value, wherein the further earlier hash value is obtained by applying a hash function to a portion of an even earlier segment of the temporal sequence. The apparatus according to any one of claims 44 to 52, including

54. When checking whether the aforementioned segment is the first segment of a further verification chain, The system is configured to derive an instruction from the data stream indicating whether the segment is the first segment in the validation chain. The apparatus according to claim 52 or 53.

55. When checking whether the aforementioned segment is the first segment of a further verification chain, Deriving validation information for the segment from the data stream, and comparing the validation information with prior validation information for further preceding segments of the temporal sequence. If the verification information matches the preceding verification information, infer that the segment is not the first segment of a further verification chain. If the verification information does not match the preceding verification information, infer that the segment is the first segment of a further verification chain. The apparatus according to claim 52 or 53, configured to perform the following:

56. The aforementioned verification information, Instructions for a certificate to decrypt the aforementioned digital signature (43), Instructions for the hash function, Identifier of the content of the aforementioned segment The apparatus according to claim 54, comprising one or more of the following.

57. Checking the aforementioned segment in terms of reliability is If the instruction indicates that the verification chain has been interrupted, and the data stream does not contain a digital signature (43) for the segment, or if the segment is the first segment in a further verification chain, The storage of a previous hash value, wherein the previous hash value is obtained by applying a hash function to a portion of the preceding segment to a subsequent segment of the temporal sequence. The apparatus according to any one of claims 44 to 56.

58. A device (10) for encoding a data stream (14), wherein the device is configured to make the data stream verifiable in terms of reliability and to encode a temporal sequence of segments within the data stream, and the device is configured to If the verification chain within the temporal sequence of the segment is interrupted at the segment, and the segment should be checkable for reliability, In the aforementioned temporal sequence, a verification string (48) is formed that is independent of the preceding segment immediately preceding the segment, and Obtaining a digital signature (43) based on the verification string (48), and Inserting the aforementioned digital signature (43) into the data stream It is configured to do the following: The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by The device (10).

59. A method for decoding a data stream (14), wherein the method includes checking the data stream for reliability, wherein the data stream includes a temporal sequence of segments, and the method From the data stream, derive whether the validation chain within the temporal sequence of the segment was interrupted in the segment (21), and If the data stream indicates that the verification chain has been interrupted, and if the data stream includes a digital signature (43) of the segment, Deriving the digital signature (43) from the data stream (21), In the aforementioned temporal sequence, a verification string (48) independent of the preceding segment immediately preceding the segment is formed (39), and Checking whether the verification string (48) conforms to the digital signature (43) (63) Includes, The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by method.

60. A method for encoding a data stream (14), the method comprising making the data stream reliable and encoding a temporal sequence of segments within the data stream, the method If the verification chain within the temporal sequence of the segment is interrupted by the segment, and the segment should be checkable for reliability, In the aforementioned temporal sequence, a verification string (48) is formed that is independent of the preceding segment immediately preceding the segment, and Obtaining a digital signature (43) based on the verification string (48), and Inserting the aforementioned digital signature (43) into the data stream Includes, The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding. The aforementioned block-based predictive coding and good-faith transformation-based residual coding This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding. The aforementioned context-adaptive variable-length coding is, A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a tracing-one value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of the non-zero conversion coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of the non-zero conversion coefficients, excluding the number of non-zero conversion coefficients having an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements indicating the position of the non-zero conversion coefficients along the scan order, by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients in the scan order, It is done using The block-based predictive encoding and the transformation-based residual encoding are, Encoding the predicted residual data of the residual block into the data stream using context-adaptive binary arithmetic coding of the quantization index of the transformation coefficient of the transformation block representing the residual block, and sequential quantization of the transformation coefficient to obtain a quantization index, wherein the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by method.

61. A data stream (14) generated using the method of claim 60.

62. A computer program for carrying out the method of claim 26, or claim 27, or claim 41, or claim 42, or claim 59, or claim 60, when executed on a computer or signal processor.

Images