Authentication of supplementary information in data streams

Abstract

The present invention provides an apparatus and method for decoding data streams that should be checked for reliability. [Solution] An apparatus and method for checking the reliability of a data stream are described. According to one embodiment, the data stream includes instructions indicating whether one or more supplemental informational messages should be included in a portion of the data stream, and that portion should be used to perform a reliability check of the data stream.

Description

Technical Field

[0001] Embodiments of the present invention relate to an apparatus and method for decoding a data stream to be checked for reliability. Further embodiments relate to an apparatus and method for making a data stream checkable for reliability. Further embodiments relate to a data stream checkable for reliability. In particular, embodiments relate to media data streams such as video data streams or audio data streams, and decoders and encoders for decoding or encoding such data streams respectively.

Background Art

[0002] Content authentication is important to avoid media tampering. With the rapid progress of AI, sophisticated deepfakes have been created, blurring the boundary between real and fake content, causing significant concerns regarding cybersecurity and copyright. Therefore, in recent years, it has become important to be able to verify the authenticity of media. An example of a method for performing such authentication consists of, or includes, first hashing a media asset and then providing a digital signature to the media by signing it with the secret key of the content generator, so that on the client side, when the public key of the content generator is given, the client can compare the provided signature with the value of the hash calculated independently based on the received media asset. If the values match, the client can safely assume that the media has not been tampered with.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The complex structure of data streams can complicate their authentication. This is particularly true in scenarios where a data stream is modified or generated by an entity, such as a scenario where a data stream consists of multiple substreams, or a scenario where an entity combines multiple data streams, or a scenario where a portion of a data stream is extracted. If this entity is not the entity that encoded the original data stream, and if the recalculation of the digital signature that would enable data stream verification is to be avoided, then a new solution is needed to enable reliable verification of the data stream. [Means for solving the problem]

[0004] An object of embodiments of the present invention is to provide a concept for checking data streams with respect to reliability, which offers an improved trade-off between high adaptability to complex streaming scenarios, low computational effort, and a low bitrate of signaling overhead required to provide the information necessary for authentication. This objective is achieved by the subject matter of the independent claim.

[0005] Embodiments of the present invention relate to the verification of supplemental information messages for a data stream, where the supplemental information messages provide supplemental information for the payload data carried in the payload packets of the data stream. The terms data stream verification and checking the reliability of a data stream are used interchangeably below and relate, for example, to the verification that the content of a data stream corresponds to the content provided by a content provider that has signed the content using a certificate. Including supplemental information messages in the data stream verification process may have the disadvantage that, in some scenarios, intermediate entities in the data stream transmission chain may want to drop some parts of the data stream. For example, an entity may drop supplemental information messages that are not required for the use case in which the intermediate entity prepares the data stream. On the other hand, the information carried by the supplemental information messages may affect the content of the decoded data, and as a result, an unauthorized modification of the supplemental information message may go undetected if the supplemental information message is excluded from the verification process.

[0006] Embodiments of the present invention rely on the idea of ​​including instructions in the data stream that indicate whether one or more supplemental information messages should be considered for verifying the data stream. Using these instructions, the receiver knows whether one or more supplemental information messages should be included in the verification process, and can then determine, according to the instructions, the portion of the data stream that underlies the verification. Thus, these instructions enable an encoder that makes the data stream checkable for reliability to choose whether one or more supplemental information messages should be included in the reliability check. This allows for a choice between an improved level of verification by including supplemental information messages in the reliability check without the need to recalculate the digital signature for the verification process, and greater flexibility in verifying the supplemental information messages at a later stage, for example, by an intermediate network entity.

[0007] The embodiment provides a device for decoding a data stream, the data stream comprising a plurality of payload packets carrying payload data, further comprising supplemental information packets, the supplemental information packets carrying supplemental information messages. The device is configured to derive instructions from the data stream indicating whether one or more supplemental information messages should be considered to validate the data stream, to determine a given portion of the data stream by including one or more supplemental information messages in the given portion if the instructions indicate that one or more supplemental information messages should be considered to validate the data stream, and to obtain a digital signature from the instructions in the data stream to validate the given portion.

[0008] The embodiment provides a device for decoding a data stream. The device is configured to check the data stream for reliability, and the data stream includes a plurality of payload packets carrying payload data, and further includes supplemental information packets, the supplemental information packets carrying supplemental information messages. The device is configured to derive an instruction from the data stream indicating whether supplemental information messages should be considered for validating the data stream; to determine a predetermined portion of the data stream that is to be validated, by including one or more of the supplemental information messages within the predetermined portion if the instruction indicates that supplemental information messages should be considered for validating the data stream; to obtain a digital signature based on the data stream; and to check whether the predetermined portion of the data stream conforms to the digital signature.

[0009] The embodiment provides a device for encoding a data stream. The device is configured to make the data stream checkable for reliability, encode a plurality of payload packets carrying payload data within the data stream, further encode supplemental information packets within the data stream, and encode the supplemental information packets carrying supplemental information messages. The device is configured to insert instructions into the data stream indicating whether one or more supplemental information messages should be considered for verifying the data stream, determine that a given portion of the data stream should be checkable for reliability by including one or more supplemental information messages within the given portion if the instructions indicate that one or more supplemental information messages should be considered for verifying the data stream, and determine that the given portion should be checkable for reliability, and obtain a digital signature based on the given portion.

[0010] The embodiment provides a method for decoding a data stream, the data stream comprising a plurality of payload packets carrying payload data, further comprising supplemental information packets, the supplemental information packets carrying supplemental information messages, and the method is configured to derive an instruction from the data stream indicating whether one or more supplemental information messages should be considered for validating the data stream, determine a given portion of the data stream by including one or more supplemental information messages in that portion if the instruction indicates that one or more supplemental information messages should be considered for validating the data stream, and obtain a digital signature from the instruction in the data stream for validating the given portion.

[0011] Embodiments provide a method for decoding a data stream, the method comprising checking the data stream for authenticity, the data stream comprising a plurality of payload packets carrying payload data, further comprising supplemental information packets, the supplemental information packets carrying supplemental information messages, the method comprising deriving an instruction from the data stream indicating whether the supplemental information messages should be considered for validating the data stream, determining a predetermined portion of the data stream which is a predetermined portion to be validated by including one or more of the supplemental information messages in the predetermined portion if the instruction indicates that the supplemental information messages should be considered for validating the data stream, obtaining a digital signature based on the data stream, and checking whether the predetermined portion of the data stream conforms to the digital signature.

[0012] Embodiments provide a method for encoding a data stream, the method comprising: making the data stream checkable for reliability; encoding a plurality of payload packets carrying payload data within the data stream, further encoding supplemental information packets within the data stream, the supplemental information packets carrying supplemental information messages, the method comprising: inserting instructions within the data stream indicating whether one or more supplemental information messages should be considered for validating the data stream; determining a given portion of the data stream by including one or more supplemental information messages within the given portion if the instructions indicate that one or more supplemental information messages should be considered for validating the data stream, and determining that the given portion should be checkable for reliability; and obtaining a digital signature based on the given portion (e.g., and inserting the digital signature into the data stream).

[0013] The embodiment provides a data stream having internally encoded data signals, the data stream being checkable for reliability, the data stream comprising a plurality of payload packets carrying payload data, further comprising supplemental information packets, the supplemental information packets carrying supplemental information messages, and one or more supplemental information messages including instructions indicating whether the reliability of the data stream should be considered.

[0014] A favorable embodiment is defined by the subject matter of the dependent claim. Embodiments of the present disclosure are described in more detail below with reference to the drawings. [Brief explanation of the drawing]

[0015] [Figure 1] This shows a device for decoding a data stream according to one embodiment. [Figure 2] This shows a verification module according to one embodiment. [Figure 3] This shows a device for encoding a data stream according to one embodiment. [Figure 4] An apparatus for decoding a data stream according to one embodiment of the first aspect is shown. [Figure 5] An embodiment of the first aspect is shown, which is a device for decoding a data stream having instructions for each type of supplemental information message to be verified. [Figure 6] An apparatus for decoding a data stream having individual instructions for supplemental information messages to be verified, according to one embodiment of the first aspect, is shown. [Figure 7] This demonstrates the verification of each substream of a data stream segment according to one embodiment. [Figure 8] A video encoder according to one embodiment is shown. [Figure 9] This shows a video decoder according to one embodiment. [Figure 10] This shows the subdivision of a picture according to one embodiment.

Best Mode for Carrying Out the Invention

[0016] Next, embodiments of the present invention will be described in more detail with reference to the accompanying drawings, and elements having the same or similar elements or the same or similar functionality are assigned the same reference numerals or are identified by the same names. In the following description, a plurality of details are set forth in order to provide a complete description of embodiments of the present disclosure. However, it will be apparent to those skilled in the art that other embodiments may be practiced without these specific details. In addition, the features of the different embodiments described herein may be combined with each other unless specifically stated otherwise.

[0017] In the following description, when referring to a data stream, embodiments of the present invention include a media data stream in which a media signal is encoded, for example, a video data stream in which video is encoded, or an audio data stream in which an audio signal is encoded. Generally, embodiments of the present invention may be used for any sequentially signaled data stream, for example, a sampled signal, for example, a data stream in which a sampled measurement signal is encoded. According to one embodiment, the data stream has a waveform signal encoded therein, for example, a biomedical waveform signal. Although some aspects of the present invention are described below with respect to video data streams, these aspects may be applied to other types of data streams in an equivalent manner unless specifically stated otherwise.

[0018] Figure 1 shows a device 20 for decoding a data stream 14. The device 20 may be referred to as a decoder 20. The data stream 14 may be a media data stream, such as a video data stream or an audio data stream, or any of the data stream types listed above. For example, the decoder 20 may decode a media signal, such as a video signal or an audio signal, from the data stream 14. The decoder 20 comprises an extractor 21. The extractor 21 obtains a digital signature 43 based on the data stream 14. According to one embodiment, the extractor 21 derives the digital signature 43 from the data stream 14. In other words, the digital signature 43 may be included in the data stream 14. According to an alternative embodiment, the data stream 14 includes a resource instruction containing the digital signature 43, and the extractor 21 uses the resource instruction to obtain the digital signature 43 from the resource indicated in the data stream 14. For example, digital signature 43 is for checking the authenticity of the data stream. In other words, digital signature 43 may be for verifying the data stream 14, for example, for verifying the authenticity of the data stream 14.

[0019] The decoder 20 further comprises a sub-determiner 30 configured to determine a predetermined portion 13 of the data stream 14 on which a reliability check, also hereafter referred to as a reliability check, should be performed. A digital signature 43 may be associated with the predetermined portion. In other words, the digital signature 43 may be for verifying the predetermined portion 13. For example, verification or reliability checks of the data stream 14 may be performed part by part, i.e., in units of portions of the data stream 14. Therefore, the digital signature 43 may be for checking the reliability of portions of the data stream, in other words, for verifying portions of the data stream 14. The reliability check, i.e., the check of the data stream 14 or the predetermined part 13 regarding reliability, includes checking whether a predetermined part of the data stream 14 conforms to or matches the digital signature 43.

[0020] According to one embodiment, the decoder 20 includes a verification module 41 configured to perform a reliability check. As shown in FIG. 1, the verification module 41 receives the digital signature 43 and the predetermined part 13 and performs a reliability check. The verification module 41 is an optional feature of the device 20, as shown by the dashed line in FIG. 1. In other words, according to one embodiment, the decoder 20 is configured to check a predetermined part 13 of the data stream 14 regarding reliability. Further in other words, according to one embodiment, the decoder 20 is configured to check whether a predetermined part 13 of the data stream 14 conforms to or matches a digital signature.

[0021] As already described, the verification module 41 is optional in the device 20. According to an alternative embodiment, the reliability check may be performed externally to the decoder 20. In other words, the verification module 41 does not necessarily have to be part of the decoder 20 and may be part of a separate entity such as a device for checking the reliability of the data stream 14. For example, in this case, the decoder 20 may provide or transfer the digital signature 43 and the predetermined part 13 for the reliability check. For example, the decoder 20 may collect information for the reliability check such as the digital signature 43 and the predetermined part 13 and provide the collected information for the reliability check. For example, the decoder 20 may form a concatenation of information for the reliability check. These alternative embodiments where the verification module 41 is not part of the device 20 may be implemented in combination with all embodiments of all aspects of the present invention.

[0022] Further optional features of the device 20 of FIG. 1, applicable to all embodiments described herein, are described below. For example, the data stream 14 may contain multiple packets. The partial decision-maker 30 may determine a predetermined portion 13 for each packet, for example, by including one or more of the multiple packets. For example, optionally, a data stream 14 may contain multiple payload packets 16. A payload packet 16 may carry payload data, for example, data to be transmitted by the data stream 14. For example, if the data stream 14 is a media data stream, the payload packet 16 may carry coded media data, such as video data if the data stream 14 is a video data stream. In other words, a payload packet 16 may be a video coding layer (VCL) or network abstraction layer (NAL) unit carrying coded video payload packets, for example, video data, for example, encoded video data. For example, the video data may refer to information from which sample values ​​of video pictures are reconstructed.

[0023] The data stream 16 may optionally further include supplemental information packets 18, which may be alternatively referred to as supplemental information payload packets, in contrast to coded data payload packets that carry coded data. For example, in the case of a video data stream, particularly if H.264, H.265, and H.266 can format the video data stream, the supplemental information packets may be Supplemental Extended Information (SEI)NAL units. For example, the supplemental information packets 18 are interspersed among the payload packets 16. The supplemental information packets 18 carry supplemental information messages 19. For example, the supplemental information messages 19 assist processes related to decoding, displaying, or other purposes, but are not required by the decoding process to determine the values ​​of samples in the decoded picture of the video. In other words, for example, supplemental information packets may carry information about encoding options and information for the decoding process, but they do not contain encoded samples of the signal encoded within the data stream.

[0024] For example, each supplementary information message may be associated with one of the payload packets. For example, each supplemental information message is associated with one of the payload packets, which is, for example, part of the same sample of the sampled signal encoded within the data stream, for example, the same picture unit or the same access unit as the supplemental information packet.

[0025] As described below with respect to embodiments of the first, fourth, and fifth aspects, a predetermined portion 13 may optionally include one or more supplemental information messages that may be contained in one or more supplemental information packages. In other words, a predetermined portion 13 may optionally include one or more supplemental information packets.

[0026] For example, a predetermined portion 13 includes all or a subset of the payload packets 16 of a segment of the data stream 14. For example, a predetermined portion 13 may include portions of the payload packets 16, for example, each portion of the payload packets that should be included in the predetermined portion, or it may include the entire payload packet. The segment of the data stream 14 from which a predetermined portion 13 can be determined may be, for example, an independently coded sequence of the data stream 14, for example, a coded video sequence (CVS) in the case of a video data stream. In other examples, the segment from which a predetermined portion 13 can be determined may be indicated in the data stream 14, for example, by one or more instructions that associate packets of the data stream with the predetermined portion 13, and / or by instructions such as supplemental information messages that indicate the start and end of the segment from which the predetermined portion 13 is selected. For example, the start and end may be indicated by the respective supplemental information messages.

[0027] According to the embodiment, the data stream 14 may include a set of verification parameters for verifying a predetermined portion 13, or a segment of the data stream 14 containing the predetermined portion, for example, a segment 17 as described in relation to Figure 7. For example, the set of verification parameters may include one or more of the following: a hash function for performing an authenticity check of the predetermined portion 13 (for example, a hash function 31 as described in relation to Figure 2), a certificate for decrypting a digital signature 43 for performing an authenticity check of the predetermined portion 13, an identifier for associating the predetermined portion with a media asset, and instructions on how to derive the predetermined portion 13. The set of verification parameters may be signaled, for example, with a supplemental information message that may correspond to a DSC ISCI message, as described later. For example, instruction 51 may be included in the set of verification parameters.

[0028] Figure 2 shows one embodiment of a verification module 41, which may be optional as part of the decoder 20. Alternatively, the reliability check described in relation to the verification module 41 may be performed outside of the decoder 20. According to the embodiment of Figure 2, the reliability check, i.e., checking whether a given portion 13 of the data stream 14 conforms to the digital signature 43, includes subjecting the given portion 13 to a hash function 31 to obtain a hash value 33. According to this embodiment, the verification module 41 further includes a verification block 39 that checks whether the hash value 33 conforms to the digital signature 43.

[0029] The following describes the further optional features of verification module 41. According to one embodiment, the verification block 39 includes a verification string generator 49 configured to form a verification string, for example, an IdString of the sample syntax described later, based on a hash value 33. For example, the verification string generator 49 may form a concatenation including the hash value 33 and, optionally, a media asset identifier that identifies the media asset to which a given portion 13 of the media belongs, an identifier of the algorithm of the hash function 31, and further information such as one or more further hash values ​​obtained from earlier portions of the data stream 14. According to this embodiment, the verification block 39 includes a decryption block 59 that decrypts the digital signature 43 to obtain a check value 47, and the verification block 39 checks whether the verification string 48 matches the check value 47.

[0030] For example, the decryption module 59 may use an asymmetric cryptographic public key 45 to decrypt the digital signature 43. For example, the extractor 21 may derive the public key 45 based on an instruction in the data stream 14, for example, a resource identifier that indicates a resource from which the public key 45 can be derived, and then derive the public key 45 from the indicated resource in the data stream 14. In other words, according to one embodiment, the verification block 39 checks whether the hash value 33 conforms to the digital signature 43 by forming a verification string based on the hash value and optionally based on further information, and by comparing the verification string with the digital signature 43 using the public key (comparing the verification string with the digital signature may include decryption, which is performed by the decryption block 59).

[0031] According to the alternative embodiment, all or part of the above further information for deriving the verification string 48 is concatenated with a predetermined portion 13 and subjected to a hash function 31 to obtain a hash value. In other words, according to these alternative embodiments, all or part of the above further information, such as a hash function identifier and / or further hash values ​​or digital signatures of the previous segment or portion, is used to derive the verification string, and this information is reflected in the verification string by hashing this information together with the predetermined portion 13, and therefore by deriving a hash value 33 based on this information.

[0032] For example, the generation of the digital signature 43 can be performed on the encoder side by forming a verification string and signing the verification string using the secret key of an asymmetric encryption scheme. For example, signing may involve further hashing, i.e., hashing the verification string using a further hash function to obtain a further hash value, and signing the further hash value. In this example, it may be impossible for the decoder to reconstruct the verification string from the digital signature 43, but instead, it can check whether the check value formed using the hash value 33 conforms to the digital signature, for example, by forming the verification string and deriving the check value by hashing the verification string using a further hash function. In other words, in this case, verification by verification block 39 may involve hashing the verification string using a further hash function to obtain a further hash value, and checking whether the further hash value conforms to the digital signature, for example, by decrypting the digital signature using a public key and checking whether the resulting check value is equal to the further hash value.

[0033] In other words, according to one embodiment, checking whether the hash value 33 conforms to or matches the check value 47 may include, for example, forming a verification string using the hash value 33 by concatenating it with further information such as a further hash value or hash function identifier, as described below, and hashing the verification string using a further hash function. The verification block 39 may then check whether the hashed verification string is equal to the check value 47 decoded from the digital signature. On the encoder side, according to this embodiment, the digital signature may be generated by forming a verification string, similar to the decoder side, hashing it using a further hash function, and signing the hashed verification string to obtain the digital signature 43.

[0034] According to an alternative embodiment, the check value 47 may correspond to a verification string, for example, a hash value 33, or the concatenation of hash value 33 with further information such as a further hash value or hash function identifier. In other words, decrypting the digital signature in this case may result in hash value 33 as part of (or the entirety of) check value 47. In this case, the digital signature may become larger due to the omission of further hashes. For example, which of the above alternatives is used may depend on the selected hash function.

[0035] Figure 3 shows a device 10 for encoding a data stream 14. The device 10 may be referred to as an encoder 10. The device 10 is configured to make the data stream 14 verifiable in terms of reliability. The encoder 10 provides the data stream 14 by inserting, for example, encoding, multiple payload packets 60 carrying payload data and supplemental information packets 18 carrying supplemental information messages 90 into the data stream 14. In other words, the encoder 10 may provide the data stream 14 described in relation to Figure 1. For example, the encoder 10 includes an inserter 23 for inserting the payload packets and supplemental information packets into the data stream 14. The encoder 10 includes a verification module 41' that obtains a digital signature 43 based on a predetermined portion of the data stream 14. The encoder 10 further includes a portion decision-maker 30' that determines a predetermined portion 13. For example, the portion decision-maker 30' determines a predetermined portion 13 by including, for example, a portion of the data 14' in a predetermined portion 13 that corresponds to a predetermined portion 13 determined by the portion decision-maker 30, based on the data 14' to be inserted into the data stream 14.

[0036] Any description of device 20 can be optionally applied to encoder 10 in the sense that information derived from data stream 14 by device 20 can be inserted into data stream 14 by device 10. Furthermore, any hash function used by verification module 41, such as hash function 31, may be equivalent to the corresponding hash function used by verification module 41'. The same applies to the input of the corresponding hash function, such as hash function 31, used to derive the hash value 33.

[0037] The interaction between the verification module of the decoder 20 and the verification module 41' of the encoder 10 has already been briefly described above. The verification module 41' may target a verification string containing a hash value derived by applying a hash function to a predetermined portion 13. The verification module 41' may further sign the verification string using, for example, the secret key of the asymmetric cryptographic scheme described above, in order to generate a digital signature 43. For example, the digital signature 43 may be inserted into the data stream 14 by the inserter 23, or the inserter 23 may insert into the data stream 14 a resource instruction from which the digital signature 43 can be derived.

[0038] Any description relating to device 20 may be optionally applied equivalently to device 10, in the sense that information derived from data stream 14 by device 20 may be inserted into data stream 14 by device 10. Furthermore, any hash function, such as hash function 31 used by device 10, may be equivalent to the corresponding hash function used by device 20. The same applies to the inputs to the corresponding hash function, such as hash function 31, used to derive the hash value 33. The generation of the digital signature 43 and the verification performed by device 20 using the digital signature 43 may each be part of an asymmetric cryptographic scheme, and these steps may each be performed by a pair of private and public keys, where at least the private key is used to sign in order to generate the digital signature 43, and the public key is used to decrypt in order to verify the verification string formed on the receiver side against the digital signature 43.

[0039] Embodiments of the present invention will be described below. Figure 4 shows a device 20 for decoding a data stream 14 according to one embodiment of the present invention. The device 20 in Figure 4 may optionally correspond to the decoder 20 in Figure 1, that is, the decoder 20 in Figure 4 may be based on any of the embodiments described in relation to Figure 1. Furthermore, the embodiments described below may optionally be combined with any of the embodiments described in relation to Figures 1 to 3.

[0040] According to embodiments of the present invention, the extractor 21 is configured to derive an instruction 51 from the data stream 14 indicating whether one or more supplemental information messages should be considered, used, or included in order to validate the data stream 14, for example, to check the data stream for reliability. According to embodiments of the first aspect, the partial decision-maker 30 determines a predetermined portion 13 by including one or more supplemental information messages in the predetermined portion 13 if the instruction 51 indicates one or more supplemental information messages to be considered in order to validate the data stream 14. The one or more supplemental information messages that the instruction 51 refers to, i.e., the instruction 51 indicates whether to include the supplemental information messages in the predetermined portion 13, may be referred to as a validation set of supplemental information messages.

[0041] It should be noted that the final decision on whether to include or exclude a supplemental information message or packet from a given portion may depend on one or more further criteria, as described, for example, with respect to the fourth aspect. In other words, instruction 51 may be one of several criteria for whether or not to include a supplemental information message from a given portion 13. In other words, one embodiment of the present invention is a device 20 for decoding a data stream 14 (for example, a device for decoding media signals from a media data stream), the data stream comprising a plurality of payload packets 16 (for example, VCL NAL units) carrying payload data (for example, media data, for example, video data) (for example, encoded video data, for example, video data is information from which sample values ​​of picture in video are reconstructed), and supplementary information packets 18 (for example, supplementary information payload packets) (for example, SEIThe Supplemental Information Packet further includes NAL units (e.g., scattered between payload packets), and the Supplemental Information Packet carries Supplemental Information Messages 19 (e.g., information that assists processes related to decoding, displaying, or other purposes, but is not required by the decoding process to determine the values ​​of samples in the decoded picture), and the device takes from the data stream instructions 51 (e.g., syntactic elements, e.g., dsci_sei_mess) indicating whether one or more Supplemental Information Messages should be considered (or used or included) for validating the data stream 40 (e.g., validating the reliability of the data stream, in other words, checking the data stream for reliability) (or, e.g., whether the Supplemental Information should not be considered to authenticate a given portion 13). The system is configured to: derive ages_digitally_signed_flag) 21; determine a given portion 13 of the data stream by including one or more supplemental information messages in the given portion if instruction 51 indicates that one or more supplemental information messages should be considered in order to verify the data stream (e.g., which given portion should be verified or checked for trustworthiness); and obtain a digital signature 43 from the instructions in the data stream to verify the given portion 13 21 (e.g., using the information derived from the data stream, e.g., deriving a digital signature from the data stream or deriving a digital signature from an indicated source in the data stream) (e.g., the digital signature is derived based on the given portion).

[0042] According to one embodiment, the device is configured to verify a given portion of a data stream by checking whether that portion conforms to (or matches) a digital signature (for example, to check the reliability of the data stream or a given portion thereof).

[0043] According to one embodiment, the device is configured to provide (or transfer) a predetermined portion and a digital signature for verification of that portion (for example, performed by a further device or entity) (for example, verification includes checking whether a predetermined portion of a data stream conforms to (or matches) a digital signature). In other words, one embodiment of the present invention described in relation to Figure 4 is a device 20 for decoding a data stream 14 (e.g., a device for decoding media signals from a media data stream), the device is configured to check the data stream for reliability (e.g., a device for verifying the data stream), and the data stream includes a plurality of payload packets 16 (e.g., VCL NAL units) that carry payload data (e.g., encoded video data, where the data is information from which sample values ​​of a picture are reconstructed), and supplementary information packets 18 (e.g., supplementary information payload packets) (e.g., SEIThe Supplemental Information Packet further includes NAL units (e.g., scattered between payload packets), and the Supplemental Information Packet carries Supplemental Information Messages 19 (e.g., information that assists processes related to decoding, displaying, or other purposes, but is not required by the decoding process to determine the values ​​of samples in the decoded picture), and the device takes from the data stream instructions 51 (e.g., syntax elements, e.g., dsci_sei_messages_digitally_signed) indicating whether (or should be used or included) (or whether the Supplemental Information should not be considered to authenticate a given portion) the Supplemental Information Messages (e.g., one or more Supplemental Information Messages) should be considered to validate the data stream (e.g., to validate the reliability of the data stream) (e.g., one or more Supplemental Information Messages) (e.g., one or more syntactic elements, e.g., dsci_sei_messages_digitally_signed) The system is configured to derive a flag 21, and, if instruction 51 indicates that supplemental information messages should be considered in order to verify the data stream, to determine which part of the data stream should be verified (for example, which part should be checked for trustworthiness) by including one or more supplemental information messages 19 in the part 13 30, and to obtain a digital signature 43 based on the data stream 14 (for example, by using information derived from the data stream, for example, by deriving a digital signature from the data stream or by deriving a digital signature from an indicated source in the data stream) and to check whether the part of the data stream 13 conforms to (or matches) the digital signature 43 41.

[0044] Further optional features of the embodiments are described below. According to one embodiment, the verification set of supplemental information messages includes a subset of all supplemental information messages in the data stream 14. In other words, instruction 51 may indicate whether supplemental information messages belonging to the verification set should be included in a predetermined portion 13 or excluded from a predetermined portion 13. In other words, according to one embodiment, instruction 51 indicates whether supplemental information messages belonging to the verification set of supplemental information messages should be included in a predetermined portion 13 or excluded from the predetermined portion 13. For example, optionally, the data stream may include further supplemental information messages that do not belong to the verification set, which should always be included in the verification, i.e., in the predetermined portion.

[0045] In one embodiment, instruction 51 indicates whether one or more supplemental information messages should be considered in a predetermined portion 13, or should be included within the predetermined portion 13, or whether there are any supplemental information messages that should be considered for verifying the data stream, for example, whether there are any supplemental information messages that should be included within the predetermined portion 13. In other words, instruction 51 can distinguish between considering that a verification set should be included within a predetermined portion and not including any supplemental information messages, for example, any supplemental information messages of a segment referenced by a digital signature 43, in the predetermined portion. In other words, instruction 51 indicates whether supplemental information messages, i.e., supplemental information messages of a verification set, should be included within a predetermined portion 13 or excluded from the predetermined portion 13, that is, whether supplemental information messages should be considered for verifying the data stream, or whether there are any supplemental information messages that should be considered for verifying the data stream or the predetermined portion, for example, whether there are no supplemental information messages to be considered at all.

[0046] The verification set may include all supplemental information messages, or parts thereof, of the segment of the data stream referenced by the digital signature 43. In the latter case, the verification set may be predefined, for example, by message type, or signaled in the data stream 14, for example, by indicating which message types should be included or excluded, or by individually identifying messages or packets containing each message. Further details of optional choices are described in more detail below.

[0047] The validation set for supplemental information messages can be defined by the supplemental information message type, by identifying the supplemental information messages that should be included in the validation set individually or on a packet-by-packet basis, or by any combination of these criteria. According to packet-by-packet selection, for example, by individually identifying which packets should be included in or excluded from a given portion, or by packet type, the entire supplemental information packet may be included or excluded from a given portion. In other words, with packet-by-packet selection, all supplemental information messages contained within the selected supplemental information packet are included in the validation set for supplemental information messages.

[0048] According to one embodiment, if instruction 51 indicates that one or more supplemental information messages should be considered in order to verify a predetermined portion 13, the partial decision-maker 30 determines the predetermined portion 13 by including one or more supplemental information packets in the predetermined portion 13. In other words, including supplemental information messages in a predetermined portion 13 may be done on a per-packet basis; that is, one or more supplemental information messages to be considered for modifying a predetermined portion 13 may be identified or indicated by identifying or indicating one or more supplemental information packets of the data stream 14 that should be included within the predetermined portion 13.

[0049] For example, according to one embodiment, the partial decision-maker 30 may include in the predetermined portion 13 any supplemental information messages that should be included within the predetermined portion 13, for example, all of the supplemental information packets 18 that carry any one of the one or more supplemental information messages of the verification set.

[0050] According to one embodiment, the partial decision-maker 30 conditions the decision of whether or not to include one of a plurality of supplemental information packets in a predetermined portion 13 on the result of checking whether the supplemental information packet contains a supplemental information message of one of the supplemental message types in a set of supplemental message types. In other words, the identification of supplemental information messages belonging to a verification set can be performed based on the supplemental information message type by including a specific type of supplemental information message. In addition, the inclusion or exclusion of supplemental information messages can be performed on a packet-by-packet basis by including each supplemental information packet that contains at least one supplemental information message from the set of supplemental information message types.

[0051] According to one embodiment, instruction 51 is a flag having a first state and a second state. According to this embodiment, the partial decision-maker determines a predetermined portion 13 of the data stream 14 by determining one or more supplemental information messages to be included in a predetermined portion 13 based on a predefined set of supplemental information messages when the flag has the first state, and by ensuring that none of the supplemental information messages from the predefined set are included in the predetermined portion when the flag has the second state. For example, when the flag has the first state, the partial decision-maker 30 may include all of the supplemental information messages from the predefined set, or at least one instance of all of the supplemental information messages from the predefined set, as will be described in more detail below. For example, instruction 51 may be a flag that has only a first state and a second state. In other words, instruction 51 may be binary.

[0052] According to one embodiment, a predefined set of supplemental information messages includes all supplemental information messages of supplemental information packets that refer to multiple payload packets. According to another embodiment, a predefined set of supplemental information messages includes all supplemental information messages of any type from a set of supplemental information message types. For example, the set of supplemental information message types is a subset of multiple possible types of supplemental information messages that may be defined depending on the type of data stream 14. For example, the set of supplemental information message types may be predefined; that is, for example, the set of supplemental information message types may be recognized by the decoder 20 without needing to be transmitted in the data stream 14.

[0053] According to one embodiment, the identification of one or more supplemental information messages to be included within a predetermined portion is signaled in the data stream 14 when instruction 51 indicates that another supplemental information message should be considered in order to validate the data stream.

[0054] Below, an embodiment of how to identify the validation set of supplemental messages in data stream 14 will be described with reference to Figures 5 and 6. Figure 5 shows one embodiment of the decoder 20 of the present invention. For example, the decoder 20 in Figure 5 may be an embodiment of the decoder 20 in Figure 1, that is, any of the details described above can be optionally applied to the decoder 20 in Figure 5. According to Figure 5, the data stream 14 includes a syntactic element 53 that identifies a set of types of supplemental information messages that should be included in a given portion 13. In other words, according to this embodiment, the verification set can be identified in terms of supplemental information message types. For example, in the exemplary example of Figure 5, the supplemental information packet 18' includes a supplemental information message 19' of the type indicated by the syntactic element 53. According to this embodiment, the extractor 21 derives the syntactic element 53 from the data stream 14. The information of the syntactic element 53 can be used by the portion decision-maker 30 to determine, for a given portion 13, whether the given portion 13 contains a supplemental information message of a type included in the set of supplemental information message types indicated by the syntactic element 53, or whether the given portion 13 contains a supplemental information packet containing a supplemental information message of a type included in the set of supplemental information message types indicated by the syntactic element 53, as shown in Figure 5.

[0055] In embodiments where a supplemental information packet contains only one supplemental information message, it should be noted that the type of the supplemental information message may be considered equivalent to the type of the supplemental information packet. In other words, there is not necessarily a distinction between the supplemental information message type and the supplemental information packet type, but the type of the supplemental information packet may be considered equivalent to the type of the supplemental information message carried in each supplemental information packet. According to one embodiment, the syntactic element 53 is signaled by a supplemental information message, for example, a supplemental information message referred to below as a verification parameter message. For example, the verification parameter message carries the verification parameter set described above. For example, the supplemental information message carrying the syntactic element 53 may be contained in a supplemental information packet.

[0056] According to one embodiment, instruction 51 and syntactic element 53 are included in the same supplemental information message, for example, a validation parameter message. According to one embodiment, instruction 51 is signaled by syntactic element 53, for example, by a predetermined state of syntactic element 53. For example, according to one embodiment, the syntactic element 53 has a plurality of first and second states. According to each of the first states, one or more supplemental information messages should be included in a given part 13, and the first state distinguishes different sets of verification. In other words, the first state identifies the supplemental information messages that should be included in the given part. In other words, each of the first states indicates one or more supplemental information messages, which are not necessarily a true subset, that should be included in the given part 13. According to the second state, none of the one or more supplemental information messages are included in the given part 13. Therefore, according to one embodiment, if the syntactic element 53 has one of the first states, the partial decision-maker 30 determines one or more supplementary information messages to be included in a predetermined part 13, depending on the state of the syntactic element 53. If the syntactic element has a second state, the partial decision-maker 30 does not include any of the one or more supplementary information messages in the predetermined part 13.

[0057] According to one embodiment, if instruction 51 indicates that one or more supplemental information messages should be included in a predetermined portion 13, the extractor 21 derives from the data stream 14 a first syntactic element (e.g., dsci_num_sei_message_types_digitally_signed_minus1, described later) that indicates the number of supplemental information message types to be considered for the predetermined portion 13, and further derives from the data stream 14 the number of each second syntactic element that identifies each supplemental information message type that should be included in the predetermined portion 13. For example, the number of each second syntactic element (e.g., dsci_digitally_signed_sei_type[i]) corresponds to the number indicated by the first syntactic element. In other words, for example, a validation set may be indicated in terms of supplemental information message types, the size of the validation set is variable, and it is signaled in the data stream 14. Thus, the number of supplemental information messages that should be included in a predetermined portion can be adaptively set by the encoder.

[0058] For example, supplemental information message types that may be dropped during processing of a data stream by a file parser or an entity that extracts substreams from a data stream may be excluded from the authenticity check to avoid the need to recalculate the digital signature. On the other hand, supplemental information messages may contain information that affects the output of the decoded data stream and therefore verification may be desirable. By making the size of the verification set variable and determining whether each individual type should be included in or excluded from the verification set, the decoder can individually adapt the trade-offs between these aspects for each data stream. At the same time, instructions in terms of supplemental information message types provide group identification of supplemental information messages that should be included in a given portion, thereby avoiding individual identification of supplemental information messages in the verification set. According to one embodiment, the first syntactic element and the second syntactic element may be signaled, for example, with instruction 51 in the verification parameter message described above.

[0059] Figure 6 shows a further embodiment of the decoder 20. The decoder 20 in Figure 6 may optionally be an example of the decoder 20 in Figure 1 or Figure 4. According to one embodiment of Figure 6, the partial decision-maker 30 conditions the decision of whether to include a supplemental information payload packet of supplemental information payload packet 18 in a predetermined portion 13 on the result of checking whether the supplemental information payload packet contains a supplemental information message (e.g., as a prefix of further supplemental information messages, e.g., further supplemental information messages within the same packet) that instructs the predetermined portion 13 to include the supplemental information payload packet. For example, in Figure 6, each of the payload packets 18' contains a supplemental information message 55 that instructs the predetermined portion 13 to include its respective supplemental information packet 18', while supplemental information packet 18'' does not contain such an instruction.

[0060] It should be noted that, optionally, the partial decision-maker 30 may condition its decision on whether or not to include a supplemental information packet 18' in a predetermined portion 13 on one or more further conditions. In other words, according to one embodiment, a supplemental information packet 18' that instruction 55 indicates should be included in a predetermined portion 13 is not necessarily included in the predetermined portion 13, but the decision may depend on further conditions. For example, if the supplemental information message is repeated, only one instance of each message may be included in the predetermined portion. Alternatively, according to one embodiment, the partial decision-maker 30 includes a supplemental information packet 18' which includes a supplemental information message 55 that instructs to include a supplemental information packet in a predetermined portion.

[0061] A further condition for the partial decision-maker 30 to determine whether to include or exclude a supplemental information packet from a predetermined portion 30 may be whether each supplemental information packet belongs to a predetermined set of supplemental information packets that are always included in a predetermined portion 13 or never included in a predetermined portion 13. Therefore, according to one embodiment, if a supplemental information packet does not contain a supplemental information message 55 instructing that the supplemental information packet be included in a predetermined portion, the partial decision-maker 30 excludes each supplemental information packet if each supplemental information packet belongs to a predetermined set of supplemental information messages, and excludes each supplemental information packet if each supplemental information packet does not belong to a predetermined set of supplemental information packets. It should be noted that the decision may optionally rely on further conditions, such as the reliance on repetition already described. For example, supplemental information message 55 may be a prefix supplemental information message preceding all supplemental information messages within the supplemental information packet, or it may be a further supplemental information message.

[0062] The following section continues the explanation of the decoder 20 in Figure 4 and describes further alternative examples for instructing the verification set. According to one embodiment, the decision of whether or not to include a supplemental information packet in a predetermined portion 13 is conditional on checking whether there is a prefix supplemental information packet preceding the supplemental information packet that instructs the inclusion of a subsequent, for example, immediately following supplemental information packet in the predetermined portion 13. In other words, compared to the embodiment described with respect to Figure 6, a prefix supplemental information packet may be used for instruction instead of a supplemental information message 55. All further details described with respect to Figure 6 may optionally apply to this embodiment as well. According to another embodiment, the decision of whether or not to include a supplemental information message in a given portion 13 is conditional on checking whether there is a prefix supplemental information message preceding the supplemental information message that instructs the subsequent, for example, immediately following supplemental information message to be included in the given portion 13. In other words, according to this embodiment, inclusion or exclusion may be on a message-by-message basis rather than on a packet-by-packet basis. The details described with respect to Figure 6 may optionally apply to this embodiment as well.

[0063] According to a further embodiment, the decision of whether or not to include a supplemental information packet in a predetermined portion 13 is conditional on the result of checking whether the supplemental information packet includes a nesting supplemental information message that instructs the predetermined portion to include the supplemental information packet. For example, a nesting supplemental information message may include a set of one or more supplemental information messages. From the fact that a set of supplemental information messages is signaled within the nesting supplemental information message, the partial decision-maker 30 may conclude that a supplemental information packet containing the nesting supplemental information message should be considered to be included in a predetermined portion 13. In other words, according to this embodiment, inclusion in or exclusion from a predetermined portion may be on a packet-by-packet basis, and the indication of whether a packet is included or excluded may be signaled by the nesting supplemental information message.

[0064] According to another embodiment, nesting supplemental information messages are used to signal whether to include or exclude supplemental information messages from a given part for each message. According to this embodiment, the decision of whether or not to include supplemental information messages is conditional on the result of checking whether or not a nesting supplemental information message that instructs to include supplemental information messages in a given part 13 contains supplemental information messages.

[0065] As already mentioned above, the inclusion or exclusion of supplemental information packets may be further conditional on whether they contain supplemental information messages of a type belonging to a set of types of supplemental information messages that should be included in a given portion. In other words, for example, this set of types of supplemental information messages includes supplemental information message types that should be included in a given portion without being explicitly signaled in the data stream 14. In other words, the set of types may be predefined. A predefined set of supplemental information messages may be added to supplemental information message types that are adaptively signaled to be included in a given portion.

[0066] Whether a supplemental information message belongs to a predetermined set of supplemental information message types that should be included in a predetermined portion may be independent of or dependent on instruction 51. In other words, supplemental information messages belonging to a predetermined set of types may always be included in a predetermined portion. Independent of instruction 51, or if instruction 51 indicates that supplemental information messages should be considered for validating the data stream, the supplemental information messages of the predetermined set of types are considered to be included in the predetermined portion 13, and optionally, any additional supplemental information messages indicated to the data stream 14 are considered to be included in the predetermined portion. If instruction 51 indicates that one or more supplemental information messages should not be considered for validating the data stream, according to one embodiment, the supplemental information messages of the predetermined set of types are considered to be included in the predetermined portion 13 without considering that any further supplemental information messages are included in the predetermined portion 13. According to another embodiment, if instruction 51 indicates that one or more supplemental information messages should not be considered for validating the data stream, neither the supplemental information messages of the predetermined set of types nor any further supplemental information messages are considered to be included in the predetermined portion 13. The inclusion and exclusion of supplementary information messages by identification using a predetermined set of types can be performed on a per-packet basis.

[0067] According to one embodiment, a supplemental information packet is deemed to be included in a predetermined portion if it contains any supplemental information message from a predetermined set of types. According to another embodiment, a supplemental information packet is deemed to be included in a predetermined portion if the first supplemental information message belongs to a predetermined set of supplemental information message types. For example, according to one embodiment, the partial decision-maker 30 conditions the decision of whether or not to include one supplemental information packet among a plurality of supplemental information packets in a predetermined part on the result of checking whether the supplemental information packet contains one of the types of supplemental information messages from a set of types of supplemental information messages that should be included in the predetermined part.

[0068] For example, if instruction 51 indicates that supplemental information messages should be considered in order to validate the data stream, the partial decision-maker 30 may check for one of the supplemental information packets among the multiple supplemental information packets (for example, for each of the supplemental information packets) whether the supplemental information packet contains one of the types of supplemental information message that should be included in a given portion, and if the supplemental information packet contains one of the types of supplemental information message that should be included in a given portion, it may include the supplemental information packet in the given portion.

[0069] According to one embodiment, the partial decision-maker 30 conditions the decision of whether or not to include one supplemental information packet among a plurality of supplemental information packets in a predetermined part on the result of checking whether the first supplemental information message among the one or more supplemental information messages carried in that supplemental information packet is one of the set of types of supplemental information messages that should be included in the predetermined part. For example, if the instruction indicates that supplemental information should be considered in order to authenticate a given part, then for one of several supplemental information packets (for example, for each supplemental information packet), check whether the first supplemental information message among the one or more supplemental information messages carried in the supplemental information packet is one of the set of types of supplemental information messages that should be included in the given part, and if the first supplemental information message among the one or more supplemental information messages carried in the supplemental information packet is one of the set of types of supplemental information messages that should be included in the given part, then include the supplemental information packet in the given part.

[0070] The following describes further alternatives to identifying verification sets, utilizing the part-by-part verification system implemented in the reliability check. In other words, the verification substreams, described later with respect to Figure 7, can be used to identify supplemental informational messages that should be included in the reliability check. Figure 7 shows a further embodiment of the decoder 20. The features described in relation to Figure 7 can be optionally combined with any of the aforementioned embodiments of the decoder 20. According to the embodiment of Figure 7, the reliability check is performed in units of parts 13', 13''. In other words, according to this embodiment, the data stream 14 includes several parts in which the data stream 14 can be verified in units thereof. For example, the data stream 14 includes a digital signature for each of the parts. Optionally, the data stream 14 may include the aforementioned verification parameter set for each of the parts. Alternatively, the verification parameters for all or a subset of the parts of a segment may be included in a common verification parameter set. The predetermined part 13 described above may be one of several parts. Each part may be defined by assigning one or more packets or data contained in packets to it (each part).

[0071] For example, in Figure 7, payload packet 16' is assigned to part 13', and payload packet 16'' is assigned to part 13''. Performing verification of the data stream 14 on a part-by-part basis may include determining each part, for example, parts 13' and 13'' in Figure 7, and subjecting each part to a reliability check 41. For this purpose, the data stream 14 may indicate the respective digital signatures for each part, for example, the digital signature 43' for part 13' and the digital signature 43'' for part 13'' in Figure 7. For example, parts as described in Figure 7 may be referred to as substreams or verification substreams of the data stream 14.

[0072] For example, in the case of a video data stream, different layers of a layered video data stream may be assigned to different substreams. For example, different layers may carry different representations of the video encoded within the video data stream 14, for example, having different spatial resolutions, carrying different types of data such as texture and depth, or carrying different views of a scene. As a further example, different substreams may be associated with different time layers of a media data stream, each time layer carrying samples to form a representation of a media signal encoded at a different time resolution, for example, resulting in a higher time resolution when different time layers are combined. However, it should be noted that these are merely examples of organizing data in a media data stream within different substreams, and the association between packets and substreams may depend on an encoder or entity that makes the data stream 14 checkable in terms of reliability. In other words, the data stream 14 may contain, or may contain, instructions for each of the several parts of the data stream 14 that can be verified on a unit basis.

[0073] In one embodiment, in addition to the data stream being verifiable in units of the aforementioned parts in the sense of substreams, reliability checks may be performed in units of segments 17 of the temporal sequence of segments of the data stream 14. A segment may be referred to as a verification period. In other words, within one segment of the data stream 14, multiple substreams, for example, the aforementioned parts 13', 13'', may be defined, and the segments are verifiable in units thereof, and packets or data belonging to each of the substreams do not necessarily form a contiguous part of the data stream 14, but rather, within one segment, packets or data fragments may be individually assigned to one of the substreams. Hereinafter, only one segment 17 is considered, that is, when referring to a part, for example, the aforementioned substreams within one segment 17 may be referred to. According to one embodiment, the data stream 14 includes, for example, an indication of the number of substreams of the data stream within a single segment, such that, for example, only one segment of the data stream is considered as a data stream.

[0074] According to one embodiment, the extractor 21 derives several parts from the data stream 14, for example, the number of substreams of the data stream, by deriving instructions for several parts from the data stream 14. According to this embodiment, the part decisioner 30 may assign each payload packet 16 to one of the parts, for example, one or more of the parts, that is, it does not necessarily have to be assigned to each of the parts.

[0075] The following describes an embodiment of the present invention that utilizes the concept of substreams to verify supplemental information messages. According to one embodiment, several parts are given an order, such as a hierarchical order, among them, and each part has a rank within that order. According to one embodiment, the verification of the parts is performed from the lowest rank to the highest rank among the parts. According to one embodiment, the verification of a part relies on a lower-ranking neighboring part of the part in its order. For example, a hash value derived from a lower-ranking part may be included in the verification string used to verify that part.

[0076] According to one embodiment, the reliability check of a portion may optionally rely on one or more lower-ranked portions. For example, one or more hash values ​​derived from one or more of the lower-ranked portions on which the reliability check of that portion relies may be included in the verification string for verifying that portion. In other words, each part can provide verification for itself, and a hash value already derived for a part can be reused to verify a higher-ranked part.

[0077] According to one embodiment, the set of supplemental information messages to be considered for reliability checks is verified as part of one predetermined substream. For example, the supplemental information messages to be verified may be collected in one substream of a predetermined rank within an order defined between the substreams. According to one embodiment in which the verification subset is identified in terms of supplemental information message types, the partial decision-maker 30 may include supplemental information messages among supplemental information messages having any type from the set of supplemental information message types that should be included in the reliability check into a predetermined part 13 having a predetermined rank in an order defined among several parts according to this embodiment.

[0078] For example, a given rank is the highest rank in an order defined among several parts. Alternatively, supplementary information messages to be included in the reliability check may be included in the portion to which the payload packet to which each supplementary information message is associated is allocated. As mentioned above, each supplementary information message may be associated with one of the payload packets. Supplementary information messages to be included in the reliability check may be allocated in the portion to which the payload packet to which the supplementary information message is associated is allocated.

[0079] According to one embodiment, the data stream 14 includes instructions to distinguish between including all supplemental information messages of a certain type that should be included in a reliability check in a substream of a predetermined rank, or including all supplemental information messages of a certain type that should be included in a reliability check in the portion associated with their respective assigned payload packets. According to another embodiment, some parts include a first part, each of which has a payload packet associated with each of the first parts, and each of the first parts has a second part of some parts associated with each of the first parts, the second part of which should contain supplemental information messages to be considered for reliability checks and to be assigned to the payload packet associated with each of the first parts. In other words, some parts include a first part and a second part, each of which has a payload packet associated with each of the second parts, the first part being for verifying the payload packet, and the second part being for verifying the supplemental information messages assigned to the payload packet of the respective associated first part.

[0080] More generally, according to one embodiment, each supplemental information message is associated with one of the payload packets, and the device is configured to assign each supplemental information message of a data stream having any type from a set of types of supplemental information messages to one of the parts, that one part is associated with another of the parts, and that other of the parts is associated with the payload packet associated with each supplemental information message.

[0081] According to one embodiment, each supplemental information packet is associated with one of the payload packets, and the device is configured to assign each supplemental information packet of a data stream carrying any type of supplemental information message to one of the parts, the one part of which is associated with another part, and the other part of which is associated with the payload packet associated with each supplemental information packet. According to one embodiment, the device 20 is configured to derive a syntactic element from a data stream that indicates the number of parts, the syntactic element having a value, and the device is configured to infer that the number of parts corresponds to the value plus 1, multiplied by 2. According to one embodiment, the device 20 is configured to derive a syntactic element from a data stream that indicates the number of parts, the syntactic element has a value, and the device is configured to infer that the number of parts corresponds to the value plus 1.

[0082] According to one embodiment, one part and one further part follow each other in a hierarchical order defined among several parts, for example, one part follows one further part in a hierarchical order. According to one embodiment, in the apparatus, the position index of one part in a hierarchical order defined between parts corresponds to the position index of a further part in the hierarchical order plus half the number of parts. Although the descriptions of Figures 1 to 7 relate to the apparatus, the block diagrams in these figures can be considered alternatively to flow diagrams of each method, where each block represents a step in that method. Therefore, Figures 1 to 7 further provide examples of each method. The embodiments of the present invention will be described again below, in other words, specific and further embodiments of the present invention will be described. The embodiments described with respect to Figures 1 to 7 may be considered generalizations of the embodiments described below, but the following description may further include additional embodiments of the present invention that can be carried out independently of the embodiments described above. Any of the features and details described with respect to the embodiments described below may be optionally incorporated into the embodiments described above.

[0083] Although the following explanation refers to video data streams, it is clear that the same concepts can be applied equivalently to any type of media data stream, or any type of data stream that includes sampled data. A further example of sampled data may be waveform signals. In addition to audio signals, another example of waveform signals is biomedical waveform signals. To authenticate a video, the client must be provided with information about the certificate carrying the public key that may be used to verify and provide one or more signatures of the video, including how the hashing was performed (including the hashing method used, how the data is organized, and what was actually hashed and how).

[0084] The following describes existing solutions for content validation. Subsequent embodiments of the present invention may be described in terms of modifications to these existing solutions, and therefore, details of these existing solutions may form features of embodiments of the present invention. For this purpose, existing solutions involve including three SEI messages within each coded video sequence (CVS) of the video bitstream. 1. Digitally Signed Content Initialization SEI Message: Carries a C2PA manifest containing information such as the hashing method used, a URL to obtain a certificate, or the public key and UUID of the content used when calculating the signature, so that different bitstreams (e.g., different audio and video streams) that are individually signed and belong to the same content can be identified as belonging to the same content (thus preventing the use of audio from different content in a particular video, and avoiding the use of something someone said at a different time in a new video), and also carries the number of substreams used to generate the signature. The latter refers to splitting the content into different substreams, each with its own signature, so that if less important parts of a video are dropped (e.g., to layer-encode a high-resolution enhancement layer in case of network congestion), the received parts can still be authenticated. 2. Digitally signed content selection SEI message: Identifies the substream ID to which the picture slice (i.e., VCL NAL unit) belongs. 3. Digitally signed content verification SEI message: Provides a signature corresponding to each substream.

[0085] The existing solutions for these three SEI messages and verification processes are reproduced below. Details of these solutions may be optionally implemented in embodiments of the present invention, for example, in terms of modifications to these solutions, unless otherwise specified. Digitally signed content initialization SEI message: JPEG2026086354000002.jpg78168 dsci_hash_method_type indicates the secure hash algorithm used to compute message digests for a subset of SPS, PPS, APS, PH, and VCL NAL units of the coded video sequence. Based on these message digests and the digital signature present in the digitally signed content verification SEI message, the decoder can verify that the coded video was generated by the content originator indicated by dsci_key_register_idx if the flag in dsci_use_key_register_idx_flag is equal to 1. The supported values ​​for the syntax element dsci_hash_method_type, the block size used to compute the message digest, and the size of the computed message digest are specified. Values ​​for dsci_hash_method_type not listed in Table 1 are reserved for future use by ITU-T|ISO / IEC and should not be present in payload data conforming to this version of this specification. The decoder shall ignore trusted initialization SEI messages containing reserved values ​​for dsci_hash_method_type. The secure hash algorithms listed in Table 1 are specified in the Secure Hash Standard FIPS PUB 180-4.

[0086] Table 1 - Support values ​​for dsci_hash_method_type JPEG2026086354000003.jpg51169 dsci_key_source_uri contains a URI having the syntax and semantics specified in IETF Internet Standard 66. If dsci_key_retrieval_mode_idc is equal to 0, dsci_key_source_uri specifies a C2PA manifest store as defined in the C2PA Technical Specification. If dsci_key_retrieval_mode_idc is equal to 1, the following applies: -If dsci_use_key_register_idx_flag is equal to 0, the URI identifies the content provider's certificate which may be used to verify the signature present in the following digitally signed content verification SEI message. - If not (dsci_use_key_register_idx_flag is equal to 1), the URI identifies the certificate register and the content provider's certificate that can be used to verify the signature present in the following digitally signed content verification SEI message, as indicated by dsci_key_register_idx.

[0087] Adding 1 to dsci_num_verification_substreams_minus1 indicates the number of substreams in which a signature may exist within the following digitally signed content verification SEI message, calculated from the message digest. The variable NumVerificationSubstream is derived as follows: NumVerificationSubstream=dsci_num_verification_substreams_minus1+1. A dsci_key_retrieval_mode_idc equal to 0 indicates that the URI contained in dsci_key_source_uri specifies a C2PA manifest store as defined in the C2PA Technical Specification. A dsci_key_retrieval_mode_idc equal to 1 indicates that the URI contained in dsci_key_source_uri, and dsci_key_register_idx, if present, specify a certificate. In this version of the Specification, dsci_key_retrieval_mode_idc shall be in the range of 0 to 1. The decoder shall also accept other values ​​for dsci_key_retrieval_mode_idc, but shall ignore the content of the digitally signed content initialization SEI message, associated digitally signed content selection SEI message, and associated digitally signed content verification SEI message.

[0088] A dsci_use_key_register_idx_flag equal to 1 indicates that the URI contained in dsci_key_source_uri specifies a certificate register and that the syntax element dsci_key_register_idx is present in the SEI message. A dsci_use_key_register_idx_flag equal to 0 indicates that the URI contained in dsci_key_source_uri specifies a certificate and that the syntax element dsci_key_register_idx is not present in the SEI message. When dsci_key_retrieval_mode_idc is equal to 0, the media asset for which the active manifest provides content binding is a digitally signed content initialization SEI message, as specified in the C2PA technical specification. The following constraints apply to the C2PA manifest store identified by dsci_key_source_uri.

[0089] - The active manifest shall contain exactly one c2pa.hash.data, which has strong binding force on content assertions as specified in the C2PA technical specifications. The exclusion range specified in -c2pa.hash.data shall match the dsci_key_source_uri byte in the digitally signed content initialization SEI message. dsci_key_register_idx contains an index that specifies the content provider's certificate in the certificate register indicated by dsci_key_source_uri, which, if present, can be used to verify the signature present in the following digitally signed content verification SEI message.

[0090] The certificate indicated by dsci_key_register_idx when the syntax elements dsci_key_retrieval_mode_idc, dsci_use_key_register_idx_flag, dsci_key_source_uri, and dsci_use_key_register_idx_flag are equal to 1 shall specify the digital signing method using the relevant parameters (if applicable) and the public key of the content provider. When dsci_key_retrieval_mode_idc is equal to 1, the format in which this information is provided is outside the scope of this specification. It is proposed to use a digital signature algorithm compliant with FIPS 186-5, “Digital Signature Standard”. A dsci_content_uuid_present_flag equal to 1 indicates that the syntactic element dsci_content_uuid exists. A dsci_content_uuid_present_flag equal to 0 indicates that the syntactic element dsci_content_uuid does not exist. When dsci_key_retrieval_mode_idc is equal to 0, dsci_content_uuid_present_flag is considered equal to 1.

[0091] dsci_content_uuid, if present, indicates the identifier of the video content and shall have a value specified as a UUID in accordance with ISO / IEC 11578:1996, Annex A.

[0092] If a digitally signed content initialization SEI message exists within the AU, the calculation of the NumVerificationSubstream message digest is initialized according to the FIPS PUB 180-4 specification for the specified dsci_hash_method_type. Each SPS, PPS, APS, PH, and VCL NAL unit following the digitally signed content initialization SEI message is associated with one of the NumVerificationSubstream message digests, where the verification substream ID is either indicated by the digitally signed content selection SEI message or inferred to be equal to 0 if no digitally signed content selection SEI message exists for the PU. The message used to calculate the k-th message digest is obtained by concatenating all SPS, PPS, APS, and VCL NAL units associated with the k-th verification substream, where k is in the range of 0 or greater and dsci_num_verification_substreams_minus1 or less. The message digest calculation is block-based, and the block size is specified in Table 1 depending on the value of dsci_hash_method_type. For each SPS, PPS, APS, PH, and VCL NAL unit, the associated message digest is updated according to the algorithm specified in FIPS PUB 180-4 for the given dsci_hash_method_type. Note that since the message digest is computed for the concatenation of all SPS, PPS, APS, PH, and VCL NAL units in the verification substream, some of the processing blocks will typically span two or more consecutive NAL units.

[0093] Digitally signed content selection SEI message: JPEG2026086354000004.jpg18166 dscs_verification_substream_id indicates the verification substream to which the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture are assigned. If a digitally signed content initialization SEI message is present in the current coded video sequence, but a digitally signed content selection SEI message is not present in the coded picture, the value of dscs_verification_substream_id is inferred to be equal to 0. The value of dscs_verification_substream_id must be greater than or equal to 0 and less than or equal to dsci_num_verification_substream_minus1.

[0094] As specified below, the message digest of the verification substream having an ID equal to dscs_verification_substream_id is updated with the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture, according to the dsci_hash_method_type specified in the preceding digitally signed content initialization SEI message.

[0095] Digitally signed content verification SEI message: JPEG2026086354000005.jpg29168 dscv_verification_substream_id indicates the verification substream to which the SEI message applies.

[0096] Adding 1 to dscv_signature_length_in_octets_minus1 specifies the length of the syntax element dscv_signature in octets (one octet consists of 8 bits). dscv_signature contains the digital signature for the verification substream indicated by dscv_verification_substream_id.

[0097] The verification of a bitstream signature consists of the following ordered steps: 1. The calculation of the message digest, known as CurrDigest, is ultimately determined as follows: -The concatenation of SPS, PPS, APS, PH, and VCL NAL units for the verification substream having an ID equal to dscv_verification_substream_id is padded according to the FIPS PUB 180-4 specification. Note that it is sufficient to pad only the last NAL unit of the verification substream. - The calculation of the message digest, CurrDigest, is finalized according to the FIPS PUB 180-4 specification. The length of the message digest (in bits) is shown in Table 1.

[0098] 2. The reference message digest RefDigest is determined as follows: -If dscv_verification_substream_id is greater than 0, the reference message digest RefDigest is the last calculated message digest of the verification substream having an ID equal to dscv_verification_substream_id-1. A bitstream conformance requirement is that any digitally signed content verification SEI associated with a verification substream ID equal to dscv_verification_substream_id-1 exists before the digitally signed content verification SEI message having a verification substream ID equal to dscv_verification_substream_id. -Instead, if the current digitally signed content verification SEI message is the first digitally signed content verification SEI in the coded video sequence with a verification ID equal to 0, and the preceding coded video sequence did not contain any digitally signed content initialization SEI messages (including the case where the current coded video sequence is the first coded video sequence in the bitstream), then RefDigest is set to equal a bit string consisting of DigestSize bits equal to 1, where DigestSize is the size of the message digest as specified in Table 1. - Otherwise, the reference message digest RefDigest is the last calculated message digest of the validation substream that has an ID equal to 0.

[0099] 3. The identifier string IdString is constructed by concatenating the binary representation of the reference message digest RefDigest, the current message digest, and dsci_hash_method_type, as well as dsci_content_uuid2 if present. The number of bits in RefDigest is determined by the value of dsci_hash_method_type that was valid when calculating the value of RefDigest, and the number of bits in CurrDigest is determined by the current value of dsci_hash_method_type, the value of dsci_hash_method_type is represented by 8 bits, and if present, the value of dsci_content_uuid is represented by 128 bits.

[0100] 4. The IdString represents the message used to verify the signature. The signature verification algorithm and the public key used to verify the signature are indicated by the syntax elements dsci_use_key_register_idx_flag, dsci_key_source_uri, and, if dsci_use_key_register_idx_flag is equal to 1, dsci_key_register_idx.

[0101] Note 1 - Since the bit stream used for signature verification includes RefDigest, it can be verified that the SPS, PPS, APS, PH, and VCL NAL units used to calculate the current message digest are correct, and it can also be additionally verified that no additional SPS, PPS, APS, PH, and VCL NAL units have been added to the bitstream and no SPS, PPS, APS, PH, and VCL NAL units have been removed from the bitstream. Note 2 - When the decoder tunes to the bitstream, it cannot correctly calculate the value of RefDigest, and therefore cannot verify the configured IdString for the first digitally signed Content Verification SEI message. However, it is possible to verify the signature starting from the second digitally signed Content Verification SEI message. After verification, the message digest of the verification substream with an ID equal to dscv_verification_substream_id is reinitialized according to the FIPS PUB 180-4 specification for the specified dsci_hash_method_type.

[0102] Embodiments of the present invention will be described below. It is not sufficient to simply authenticate data that directly affects the picture decoded by the decoder, namely VCL NAL units. Several processes can be linked to the video data by using metadata that describes post-processing steps that can significantly alter the content. One example is a neural network post-filter SEI message, which can output a picture very different from the one output by the decoder using only VCL NAL units. Furthermore, some descriptive metadata, such as copyright or content labeling, may not affect the output sample itself, but could potentially alter important information that has other implications.

[0103] Another angle of attack could involve face generation techniques that could be manipulated and exploited to illegally alter decoded pictures. Furthermore, masking / concealment attacks, i.e., concealing decoded samples within the output, could potentially be carried out by introducing malicious film grain or manipulated display overlays. Additionally, all frame-packing-related SEI messages could be manipulated through malicious means in their respective applications to mask or conceal decoded samples. Furthermore, SEI messages with registered user data and those without should be signed to ensure the authenticity of application-specific solutions.

[0104] It may not always be important to still sign such metadata with VCL NAL units, because it prevents some metadata from being irrelevant and some systems that do not need such metadata from dropping it, which goes against the design of video encoding standards, which would allow such data to be placed in a less important bitstream and dropped without affecting the decoding process.

[0105] The same applies to the repetition of SEI messages, which is a robustness measure in video encoding standards that allows for the repeated placement of several SEI messages with the same content within a particular portion of the bitstream. Any such repetition makes it possible to be robust against packet loss affecting redundant data, and any solution for media authenticity must be robust to the same extent, i.e., the ability to test the authenticity of received data should not be hindered by the loss of redundant data.

[0106] In the first embodiment, information is added to the bitstream that allows for the identification of whether or not such metadata (i.e., SEI message) should be considered for hashing / signature calculation. Its instantiation is shown below. Digitally signed content initialization SEI message JPEG2026086354000006.jpg82165 A dsci_sei_messages_digitally_signed_flag equal to 1 specifies that non-VCL NAL units corresponding to CVS SEI messages are used to compute the message digest. A dsci_sei_messages_digitally_signed_flag equal to 0 specifies that non-VCL NAL units corresponding to CVS SEI messages are not used to compute the message digest. In this regard, there are various options for how to handle the numerous SEI message types.

[0107] In one embodiment, if the signaling indicates that SEI messages should be considered in order to sign the video bitstream by dsci_sei_messages_digitally_signed_flag, then all SEI messages are included in the message digest. Alternatively, this process may apply only to a predefined set of SEI messages identified as relevant to signing. For example, in one embodiment, the video coding standard may define a list RelevantForSigningSEIMessages containing one or more SEI message types, such as those listed below.

[0108] • NN Post-Filter related: NNPFC, NNPFA SEI messages, Film grain synthesis: FGS • SEI processing sequence that instructs the order of post-processing of SEI messages: SEI message: SPO • Masking related: FGS SEI messages, display overlay SEI messages, • Face generation related: GFV SEI message, GFVE SEI message • Packing related: Frame packing arrangement SEI message, packed area information SEI message, packing SEI message per area, configuration rectangle SEI message • SEI messages indicating registered and unregistered user data • Law, regulation, and annotation-related SEI messages: Copyright, content labeling, or text description SEI messages, AI usage restriction SEI messages In an alternative embodiment, the SEI messages to be included in the authenticity processing can also be identified through a range of values ​​for an SEI message type that can also host reserved space for future SEI messages.

[0109] As a further alternative embodiment, signaling can indicate different possible combinations of SEI messages to be signed. For example, this can be done by adding syntactic elements that point to different combinations, as follows: JPEG2026086354000007.jpg92164 A dsci_sei_messages_digitally_signed_flag equal to 1 specifies that non-VCL NAL units corresponding to CVS SEI messages are used to compute the message digest. A dsci_sei_messages_digitally_signed_flag equal to 0 specifies that non-VCL NAL units corresponding to CVS SEI messages are not used to compute the message digest.

[0110] dsci_sei_messages_digitally_signed_idc identifies the SEI message used to calculate the message digest as specified in Table X. The value of dsci_sei_messages_digitally_signed_idc must be in the range from 0 to Y. Instead of using two syntactic elements, you can use only `idc` if the value 0 means that no SEI messages are used in the message digest.

[0111] As a further alternative, the SEI type used to compute the message digest can be signaled in the bitstream. An example is shown below. JPEG2026086354000009.jpg117165dsci_num_sei_message_types_digitally_signed_minus1 plus 1 specifies the number of SEI message types used to calculate the message digest. dsci_digitally_signed_sei_type[i] identifies the i-th SEI type used to compute the message digest.

[0112] Note that if the type value is greater than 255, the SEI message type can use more than 8 bits. Therefore, the above syntax can be extended to have 16 bits instead of 8 bits. Alternatively, a further syntax element (e.g., dsci_digitally_signed_sei_type_length_inbytes_minus1) can be added to indicate the length of the syntax element dsci_digitally_signed_sei_type[i] (e.g., 8*(dsci_digitally_signed_sei_type_length_inbytes_minus1+1) bits). Alternatively, instead of pre-identifying which SEI messages will be used for message digests, each SEI message used for message digests can be preceded by a prefix SEI message, or nesting SEI messages containing any SEI message can be used to indicate that such SEI messages will be used for message digests.

[0113] It should be noted that SEI messages are encapsulated within a single SEI NAL unit, which may contain more than one SEI message. Currently, authentication is performed based on the NAL unit (the entire NAL unit). In the case of SEI messages, in one embodiment only the SEI message may be considered, but as an alternative, the entire SEI NAL unit may also be considered. For this purpose, it may be useful to determine whether or not an SEI NAL unit is used based on a first SEI message. For this purpose, a constraint is applied that SEI messages with SEI message types that should not be validated should not be included in the same NAL unit as SEI messages with SEI message types that should be validated.

[0114] A further aspect to consider is the assignment of SEI messages to specific substreams. Different options exist. 1. SEI messages belong to the highest-level substream ID specified. 2. SEI messages belong to the same substream ID as the PU containing such SEI messages. 3. When an SEI message is instructed to be used for message digest / signature generation, the number of substreams is twice the number signaled, and the substreams of the SEI message are instructed by a digitally signed content selection SEI message.

[0115] Regarding option 1, the following changes may be necessary. 1.2 Digitally Signed Content Selection SEI Message 1.2.1 Digitally Signed Content Selection SEI Message Syntax JPEG2026086354000010.jpg18159 1.2.2 Digitally Signed Content Selection SEI Message Semantics A digitally signed content selection SEI message provides a mechanism for associating an encoded picture with one of the verification substreams specified within a digitally signed content initialization SEI message.

[0116] [...] dscs_verification_substream_id indicates the verification substream to which the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture are assigned. If a digitally signed content initialization SEI message is present in the current coded video sequence, but a digitally signed content selection SEI message is not present in the coded picture, the value of dscs_verification_substream_id is inferred to be equal to 0 for the SPS, PPS, APS, PH, and VCL NAL units. When dsci_sei_messages_digital_signed_flag is equal to 1, the value of dscs_verification_substream_id is inferred to be equal to dsci_num_verification_substream_minus1 of the SEI messages specified by the DSCI SEI message (where the subset of SEI messages used for message digesting may be specified by the DSCI SEI message). The value of dscs_verification_substream_id must be between 0 and dsci_num_verification_substream_minus1.

[0117] As an alternative, the syntax of the DSCI SEI message can be extended as follows to enable signaling of whether option 1 (the top-level substream of the SEI message) or option 2 (the same SEI message substream as the VCL NAL unit of the coded picture) is being used. JPEG2026086354000011.jpg87169 A dsci_sei_messages_digital_signed_highest_substream_flag equal to 1 specifies that the SEI messages used to compute the message digest are assigned to the highest-level verification substream. A dsci_sei_messages_digital_signed_highest_substream_flag equal to 0 specifies that the SEI messages used to compute the message digest are assigned to the same verification substream as the VCL NAL unit of the associated coded picture.

[0118] 1.2.2 Digitally Signed Content Selection SEI Message Semantics A digitally signed content selection SEI message provides a mechanism for associating an encoded picture with one of the verification substreams specified within a digitally signed content initialization SEI message. [...] dscs_verification_substream_id indicates the verification substream to which the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture are assigned. If a digitally signed content initialization SEI message is present in the current coded video sequence, but a digitally signed content selection SEI message is not present in the coded picture, the value of dscs_verification_substream_id is inferred to be equal to 0 for the SPS, PPS, APS, PH, and VCL NAL units. The value of dscs_verification_substream_id must be greater than or equal to 0 and less than or equal to dsci_num_verification_substream_minus1.

[0119] A further alternative is to do as described above in Option 3. This can be instantiated by default as follows: JPEG2026086354000012.jpg94169 A dsci_sei_messages_digital_signed_separate_substream_flag equal to 1 specifies that the SEI messages used to compute the message digest are assigned to a separate verification substream specified in the semantics of the DSCS SEI message. A dsci_sei_messages_digital_signed_separate_substream_flag equal to 0 specifies that the SEI messages used to compute the message digest are assigned to the same verification substream as the VCL NAL unit of the associated coded picture.

[0120] 1.2.2 Digitally Signed Content Selection SEI Message Semantics A digitally signed content selection SEI message provides a mechanism for associating an encoded picture with one of the verification substreams specified within a digitally signed content initialization SEI message. [...] dscs_verification_substream_id indicates the verification substream to which the SPS, PPS, APS, PH, and VCL NAL units of the current coded picture are assigned. If dsci_sei_messages_digital_signed_flag is equal to 1 and dsci_sei_messages_digital_signed_highest_substream_flag is equal to 0, the SEI message used for message digest belongs to the same verification substream. If dsci_sei_messages_digital_signed_flag is equal to 1 and dsci_sei_messages_digital_signed_highest_substream_flag is equal to 1, the SEI message used for message digest belongs to a verification substream with an ID equal to dscs_verification_substream_id plus dsci_num_verification_substream_minus1+1. If a digitally signed content initialization SEI message is present in the current coded video sequence, but a digitally signed content selection SEI message is not present in the coded picture, the value of dscs_verification_substream_id is inferred to be equal to 0 for SPS, PPS, APS, PH, and VCL NAL units. If dsci_sei_messages_digital_signed_flag is equal to 1, dsci_sei_messages_digital_signed_highest_substream_flag is equal to 1, and no digitally signed content selection SEI message is present for the coded picture, the SEI message used for message digest belongs to a verification substream with an ID equal to dsci_num_verification_substreams_minus1 plus 1. The value of dscs_verification_substream_id is greater than or equal to 0 and less than or equal to dsci_num_verification_substream_minus1. Alternatively, as mentioned above, syntactic elements can be introduced to distinguish between options 1, 2, and 3.

[0121] [Video coding method] The following describes a video encoding scheme in which embodiments of the present invention can be optionally implemented. In other words, the device 20 according to any of the embodiments described above may be a video decoder implemented according to any of the embodiments of the decoder 20 described below. Similarly, the device 10 may be a video encoder according to any of the embodiments of the encoder 10 described below. The following description of the figures begins with a presentation of a description of an encoder and decoder for a block-based predictive codec for encoding video pictures, in order to form an example of a coding framework in which embodiments of the present invention may be incorporated. Each encoder and decoder will be described with reference to Figures 8, 9, and 10. Hereafter, embodiments of the concept of the present invention will be presented, along with a description of how such concepts can be incorporated into the respective encoders and decoders in Figures 8 and 9. However, subsequent figures and embodiments described below may also be used to form encoders and decoders that do not operate according to the underlying coding framework of the encoders and decoders in Figures 8, 9, and 10.

[0122] Figure 8 illustrates a device for predictively encoding picture 12 into a data stream 14 using transform-based residual coding as an example. The device or encoder is indicated by reference numeral 10. Figure 9 shows the corresponding decoder 20, i.e., a device 20 configured to predictively decode picture 12' from the data stream 14 using transform-based residual decoding, where an apostrophe is used to indicate that picture 12' reconstructed by decoder 20 deviates from picture 12 originally encoded by device 10 in terms of coding loss introduced by quantization of the predictive residual signal. While Figures 8 and 9 use transform-based predictive residual coding as an example, embodiments of this application are not limited to this type of predictive residual coding. This also applies to other details described with respect to Figures 8 and 9, as outlined below.

[0123] Encoder 10 is configured to apply a spatial-spectral transform to the predicted residual signal and encode the thus obtained predicted residual signal into the data stream 14. Similarly, decoder 20 is configured to decode the predicted residual signal from the data stream 14 and apply a spectral-spatial transform to the thus obtained predicted residual signal. The encoder 10 may internally include a predictive residual signal generator 22, which generates a predictive residual 24 to measure the deviation of the predictive signal 26 from the original signal, i.e., from the picture 12. The predictive residual signal generator 22 may, for example, be a subtractor that subtracts the predictive signal from the original signal, i.e., from the picture 12. The encoder 10 then further includes a converter 28 that applies a spatial-spectral transform to the predictive residual signal 24 to obtain a spectral domain predictive residual signal 24', which is then quantized by a quantizer 32 also provided in the encoder 10. The thus quantized predictive residual signal 24'' is encoded in the bitstream 14. For this purpose, the encoder 10 may optionally include an entropy coder 34 that entropy codes the predictive residual signal to be transformed and quantized in the datastream 14. The predicted signal 26 is generated by the prediction stage 36 of the encoder 10 based on the predicted residual signal 24'' encoded in the data stream 14 and decodeable therefrom. For this purpose, the prediction stage 36 may internally include an inverse quantizer 38 that inversely quantizes the predicted residual signal 24'' to obtain a spectral domain predicted residual signal 24'''' corresponding to the signal 24' excluding quantization losses, as shown in Figure 8, and an inverse converter 40 that subsequently applies an inverse transform, i.e., spectral-spatial transform, to the latter predicted residual signal 24'''' to obtain a predicted residual signal 24'''' corresponding to the original predicted residual signal 24 excluding quantization losses. The coupler 42 of the prediction stage 36 then recombines the predicted signal 26 and the predicted residual signal 24'''' by addition or other means to obtain a reconstructed signal 46, i.e., a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to signal 12'. Next, the prediction module 44 in the prediction stage 36 generates a prediction signal 26 based on the signal 46, for example, by using spatial prediction, i.e., intra-picture prediction, and / or temporal prediction, i.e., inter-picture prediction.

[0124] Similarly, the decoder 20 may be internally composed of components corresponding to the prediction stage 36, as shown in Figure 9, and may be interconnected in a manner corresponding to the prediction stage 36. In particular, the entropy decoder 50 of the decoder 20 entropy decodes the spectral domain prediction residual signal 24'' quantized from the data stream, and then the inverse quantizer 52, inverse converter 54, coupler 56, and prediction module 58, which are interconnected and work together in the manner described above with respect to the module of the prediction stage 36, recover a signal reconstructed based on the prediction residual signal 24'', and as a result, as shown in Figure 9, the output of the coupler 56 yields the reconstructed signal, i.e., picture 12'.

[0125] Although not specifically described above, it is readily apparent that encoder 10 can set several coding parameters, including prediction mode and motion parameters, according to several optimization methods, such as several rate and distortion-related criteria, i.e., methods for optimizing coding cost. For example, encoder 10 and decoder 20, and their corresponding modules 44 and 58, may support different prediction modes, such as intra-coding mode and inter-coding mode. The granularity at which the encoder and decoder switch between these prediction mode types may correspond to the subdivision of picture 12 and 12' into coding segments or coding blocks, respectively. At the level of these coding segments, for example, a picture may be subdivided into intra-coded blocks and inter-coded blocks. The intra-coded blocks are predicted based on the spatial, already coded / decoded neighborhood of each block, as outlined in more detail below. Several intracoding modes may exist, and each intracoding segment may be selected, including directional or angular intracoding modes, to satisfy each segment by extrapolating neighboring sample values ​​along a specific direction specific to each directional intracoding mode into each intracoding segment. The intracoding modes may also include one or more further modes, such as a DC coding mode in which the prediction for each intracoding block assigns DC values ​​to all samples within each intracoding segment, and / or a planar intracoding mode in which the prediction for each block is approximated or determined to be a spatial distribution of sample values ​​described by a two-dimensional linear function over the sample positions of each intracoding block, having a plane driving slope and offset defined by a two-dimensional linear function based on neighboring samples. In comparison, the intercoding blocks may be predicted, for example, in time.In the case of an intercoded block, a motion vector may be signaled within the data stream, the motion vector indicating the spatial displacement of the portion of the previously coded picture to which picture 12 belongs, and the previously coded / decoded picture in that portion is sampled to obtain the predictive signal for each intercoded block. In addition to the residual signal coding contained in the data stream 14, such as the entropy coded transformation coefficient level representing the quantized spectral domain predictive residual signal 24'', the data stream 14 may have further selective parameters encoded within the data stream 14, such as coding mode parameters for assigning coding modes to various blocks, several predictive parameters of the blocks such as motion parameters for the intercoded segments, and parameters for controlling and signaling the subdivision of pictures 12 and 12' into segments, respectively. The decoder 20 uses these parameters to subdivision the picture in the same way that the encoder did, thereby assigning the same predictive mode to the segments, performing the same predictive actions, and yielding the same predictive signals.

[0126] Figure 10 shows the relationship between, on the one hand, the reconstructed signal, i.e., the reconstructed picture 12', and on the other hand, the combination of the predicted residual signal 24'''' and the predicted signal 26, which are signaled in the data stream 14. As already mentioned above, the combination can be additive. In Figure 10, the predicted signal 26 is shown as a subdivision of the picture area into intracoded blocks shown exemplarily with hatching and interconnected blocks shown exemplarily without hatching. The subdivision can be any subdivision, such as a regular subdivision of the picture area into rows and columns of square or non-square blocks, or a multi-tree subdivision of picture 12 into multiple leaf blocks of varying sizes from a tree root block, such as a quadtree subdivision, a mixture of which is shown in Figure 10, where the picture area is first subdivided into rows and columns of a tree root block, and then further subdivided into one or more leaf blocks according to a recursive multi-tree subdivision.

[0127] Here too, the data stream 14 may have an intra-coding mode encoded within the data stream 14 for the intra-coded blocks 80, so that one of several supported intra-coding modes is assigned to each intra-coded block 80. For the intercoded blocks 82, the data stream 14 may have one or more motion parameters encoded within the data stream 14. Generally speaking, the intercoded block 82 is not limited to being encoded in time. Alternatively, the intercoded block 82 may be any block predicted from parts encoded before the current picture 12 itself, such as a picture encoded before the video to which the picture 12 belongs, or a picture in another view or a hierarchically lower layer if the encoder and decoder are scalable encoders and decoders, respectively.

[0128] The predicted residual signal 24'''' in Figure 10 is also shown as a subdivision of the picture area into blocks 84. These blocks are sometimes called transformation blocks to distinguish them from coded blocks 80 and 82. In practice, Figure 10 shows that encoder 10 and decoder 20 may use two different subdivisions of picture 12 and picture 12' into blocks, namely, one subdivision into coded blocks 80 and 82 and the other subdivision into transformation block 84. Both subdivisions may be the same, i.e., each coded block 80 and 82 may simultaneously form a transformation block 84, but Figure 10 shows, for example, that the subdivision into transformation block 84 forms an extension of the subdivision into coded blocks 80 and 82, thereby such that any boundary between the two blocks 80 and 82 covers the boundary between the two blocks 84, or in other words, each block 80 and 82 coincides with one of the transformation blocks 84 or coincides with a cluster of transformation blocks 84. However, the subdivisions may also be determined or selected independently of each other, so that the transformation block 84 can alternatively cross the block boundary between blocks 80 and 82. Therefore, as far as subdivision to transformation block 84 is concerned, the same description as that presented for subdivision to blocks 80 and 82 applies: namely, block 84 may be the result of regular subdivision of the picture area into blocks (with or without row and column arrangement), the result of recursive multi-tree subdivision of the picture area, or a combination thereof, or any other type of block formation. Incidentally, it should be noted that blocks 80, 82, and 84 are not limited to squares, rectangles, or any other shape.

[0129] Figure 10 further shows that the combination of the prediction signal 26 and the prediction residual signal 24'''' directly yields the reconstructed signal 12'. However, it should be noted that, according to an alternative embodiment, more than one prediction signal 26 may be combined with the prediction residual signal 24'''' to yield the picture 12'.

[0130] In Figure 10, the transformation blocks 84 are assumed to have the following significance: The converters 28 and inverse converters 54 perform these transformations in units of these transformation blocks 84. For example, many codecs use some kind of DST or DCT for all transformation blocks 84. Some codecs allow skipping transformations, and as a result, for some of the transformation blocks 84, the predicted residual signal is encoded directly in the spatial domain. However, according to embodiments described later, the encoder 10 and decoder 20 are configured to support several transformations. For example, the transformations supported by the encoder 10 and decoder 20 may include: • DCT-II (or DCT-III), where DCT represents the Discrete Cosine Transform. • DST-IV, where DST represents the discrete sine transform. · DCT-IV · DST-VII • Identity transformation (IT) Naturally, the converter 28 may support all forward versions of these conversions, while the decoder 20 or the inverse converter 54 may support their corresponding reverse or inverse versions. • Inverse DCT-II (or Inverse DCT-III) · Reverse DST-IV ·Inverse DCT-IV · Reverse DST-VII • Identity transformation (IT) The following explanation provides further details on which transformations may be supported by the encoder 10 and decoder 20. Note that in any case, the set of supported transformations may include only one transformation, such as a single spectral-to-spatial or spatial-to-spectral transformation.

[0131] As outlined above, Figures 8, 9, and 10 are presented as examples of how the concepts of the present invention, further described below, may be implemented to form specific examples of encoders and decoders according to this application. To that extent, the encoders and decoders in Figures 8 and 9 may represent possible embodiments of encoders and decoders described herein. However, Figures 8 and 9 are merely examples. Nevertheless, an encoder according to an embodiment of this application may perform block-based encoding of picture 12 using concepts outlined in more detail below, which may differ from the encoder in Figure 8, for example, being a still picture encoder rather than a video encoder, not supporting interpretation, or performing subdivision into blocks 80 in a manner different from that illustrated in Figure 10. Similarly, a decoder according to an embodiment of the present application may perform block-based decoding of picture 12' from data stream 14 using the coding concept further outlined below, but may differ from decoder 20 in Figure 9, for example, in that it is a still picture decoder rather than a video decoder, does not support intra-prediction, subdivides picture 12' into blocks in a different manner than described with respect to Figure 10, and / or derives prediction residuals from data stream 14 in a spatial domain rather than a transformation domain.

[0132] The embodiments provide an H.264 / AVC decoder, an H.264 / AVC video data stream, a video encoder for providing the H.264 / AVC video data stream, or an H.265 / HEVC decoder, an H.265 / HEVC video data stream, a video encoder for providing the H.265 / HVC video data stream, or an H.266 / VVC decoder, an H.266 / VVC video data stream, a video encoder for providing the H.266 / VVC video data stream.

[0133] The following describes embodiments of the present invention that can be optionally implemented within or combined with the framework described above with respect to Figures 8 to 10. All embodiments of the present invention as disclosed herein may be optionally implemented in combination with any of the four video encoding modifications described below. That is, the data stream 14 may be a data stream according to any of these modifications, the device 20 may be configured to decode video from the data stream 14 according to any of the three modifications, and the device 14 may be configured to provide the data stream according to any of the four modifications. The first and second modifications may be optimally combined with each other.

[0134] According to the first of two modifications of the present invention, the video is decoded from the video data stream by block-based prediction and transformation-based residual decoding, by decoding the predicted residual data of the residual blocks into and from the video data stream. According to the first modification, the predicted residual data of the residual block is decoded using context-adaptive variable-length decoding, and the decoding is performed as follows: - A first syntactic element indicating the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a trailing-one numerical value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, -One or more second syntactic elements indicating the sign of a non-zero transformation coefficient having an absolute value of 1 when traversing the coefficients along the scan order, -One or more third syntactic elements indicating the values ​​of non-zero conversion coefficients, excluding the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, -A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, -One or more fifth syntactic elements that indicate the position of a non-zero conversion coefficient along the scan order by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients, This is done using, or As an alternative to using context-adaptive variable-length decoding, decoding the predicted residual data of the residual block is performed using context-adaptive binary arithmetic decoding, and context-adaptive binary arithmetic decoding is performed. - Decoding an importance map indicating the location of non-zero conversion coefficients within a conversion block representing a residual block by decoding an importance flag indicating whether a non-zero conversion coefficient is located at the current position in a forward scan traversing the conversion coefficients of the conversion block, and if a non-zero conversion coefficient is located at the current position, and the current position is not the end of the forward scan, by decoding a final importance flag indicating whether the non-zero conversion coefficient located at the current position is the last non-zero conversion coefficient in the forward scan sequence, and - Decode the values ​​of the non-zero conversion coefficients sequentially using the reverse scan order, which is the reverse of the forward scan order. It is done by [the specified method].

[0135] According to the first of two modifications of the present invention, the video is encoded in the video data stream by block-based predictive and transform-based residual encoding, by encoding the predicted residual data of the residual blocks into the video data stream. According to the first modification, the predicted residual data of the residual block is encoded using context-adaptive variable-length encoding, and the encoding is performed as follows: - A first syntactic element indicating the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a trailing-one numerical value indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, -One or more second syntactic elements indicating the sign of a non-zero transformation coefficient having an absolute value of 1 when traversing the coefficients along the scan order, -One or more third syntactic elements indicating the values ​​of non-zero conversion coefficients, excluding the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, -A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, -One or more fifth syntactic elements that indicate the position of a non-zero conversion coefficient along the scan order by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients, This is done using, or As an alternative to using context-adaptive variable-length encoding, encoding the predicted residual data of the residual block is performed using context-adaptive binary arithmetic encoding, and context-adaptive binary arithmetic encoding is, - Encoding an importance map indicating the location of non-zero conversion coefficients within a conversion block representing a residual block by encoding an importance flag indicating whether a non-zero conversion coefficient is located at the current position in a forward scan traversing the conversion coefficients of the conversion block, and if a non-zero conversion coefficient is located at the current position, and the current position is not the end of the forward scan, by encoding a final importance flag indicating whether the non-zero conversion coefficient located at the current position is the last non-zero conversion coefficient in the forward scan order, and - Encode the values ​​of the non-zero conversion coefficients sequentially using the reverse scan order, which is the reverse of the forward scan order. It is done by [the specified method].

[0136] According to the second modification, the video decoder (or device for decoding video from a video data stream) comprises an encoded picture buffer (CPB) and a decoded picture buffer (DPB). The video decoder, Receiving a data stream containing video pictures encoded in a data stream in a coded order as a sequence of access units (AUs) (for example, the term "access unit" refers to a portion of a video data stream containing coded video data or information relating to one time frame of the video), The current AU, which has been deleted from the CPB, is decoded using picture-to-picture prediction from the referenced picture stored in the DPB to obtain the decoded picture, and Inserting the decoded picture into the DPB, Assigning a classification to each reference picture stored in the DPB as one of the following: short-term reference picture, long-term reference picture, or picture not used for reference. Read DPB mode information from the current AU. If the DPB mode information indicates a first mode, remove one or more reference pictures classified as short-term pictures from the DPB in accordance with the First-In, First-Out (FIFO) strategy. If the DPB mode information indicates a second mode, read the memory management control information, which includes at least one command in the current AU, and execute at least one command to change the classification assigned to at least one of the reference pictures stored in the DPB, and to use the classification of the reference picture in the DPB to manage the removal of the reference picture from the DPB. It is configured to perform the following actions.

[0137] According to the second modification, the video data stream is to be decoded by being fed into a decoder having an encoded picture buffer (CPB). According to the second modification, the video encoder (or device for encoding video into a video data stream) is configured to encode the pictures of the video encoded in the coded order into the data stream as a sequence of access units (AUs), When the device encodes AU, Encoding the current picture into the current AU using picture-to-picture predictions from the referenced picture stored in the decoded picture buffer (DPB), Inserting a decoded version of the current picture in the DPB into the DPB itself. Assigning a classification to each reference picture stored in the DPB as one of the following: short-term reference picture, long-term reference picture, or picture not used for reference. To write DPB mode information into the current AU. If the DPB mode information indicates a first mode, then, according to the FIFO strategy, remove one or more reference pictures classified as short-term pictures from the DPB. If the DPB mode information indicates a second mode, write memory management control information including at least one command into the current AU, the command instructs to change the classification assigned to at least one of the reference pictures stored in the DPB, and the classification of the reference picture in the DPB is used to manage the removal of the reference picture from the DPB. It is configured to perform the following actions.

[0138] According to a third modification, the video is decoded from the video data stream by block-based predictive decoding and transformation-based residual decoding, which involves decoding the predicted residual data of the residual blocks from the video data stream using context-adaptive binary arithmetic decoding of the quantization index of the transformation coefficients of the transformation blocks representing the residual blocks, and sequential inverse quantization of the quantization index, where the value of the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. According to a third modification, the video is encoded within the video data stream by block-based predictive encoding and transform-based residual encoding, which is done by encoding the predicted residual data of the residual blocks into the data stream using context-adaptive binary arithmetic encoding of the quantization index of the transformation coefficients of the transformation block representing the residual block, and sequential quantization of the transformation coefficients to obtain the quantization index, where the quantizer for quantizing the current transformation coefficient depends on the parity of the quantization index of the previous quantization index.

[0139] All three variations relate to video encoders, video decoders, methods for decoding video, methods for encoding video, and the video data streams obtained by each encoding method.

[0140] The following describes further optional details and characteristics of the first and second variations. Embodiments of the first and second modified versions may conform to H.264 / AVC. Embodiments of the third modified version may conform to H.266 / VVC. One embodiment of the first and second modifications relates to the handling of decoded pictures and their buffering in a DPB, which is a decoded picture buffer.

[0141] According to one embodiment, two types of reference pictures can be distinguished: short-term and long-term. The encoder does the same when emulating the decoder's DPB-filled state at each point in time during decoding. A reference picture can be marked as "not used for reference" when it is no longer needed for predictive referencing. The conversion between these three states (short-term, long-term, not used for reference) is controlled by the decoded reference picture marking process. Two alternative decoded reference picture marking mechanisms exist: an implicit sliding window process and an explicit memory management control operation (MMCO) process. For each currently decoded picture or each currently decoded AU, the data stream signals which process should be used for DPB management. The sliding window process marks short-term reference pictures as "not used for reference" when the number of reference frames equals a given maximum number (max-num-ref-frames in SPS). Short-term reference pictures are stored in a first-in, first-out (FIFO) manner, so the most recently decoded short-term picture is retained in the DPB. The explicit MMCO process is controlled by several MMCO commands. If this mode is selected for the current AU or the picture currently being decoded, the bitstream will contain one or more of these commands for it, or in this case for the AU. An MMCO command may be any of the following: 1) mark one or more short-term or long-term reference pictures as "not used for reference", 2) mark all pictures as "not used for reference", or 3) mark the current reference picture or an existing short-term reference picture as long-term and assign a long-term picture index to that long-term picture. The reference picture marking operation, as well as any output for presentation and removal of the picture from the DPB, may be performed after the picture has been decoded.

[0142] Details of several possible but optional reference picture marking mechanisms are described below. 1) The first aspect concerns frame number gaps and non-existent pictures. Although not described above, each reference picture in a DPB may be associated with a frame number. Normally, this number increments by 1 for each reference picture, but frame number gaps may be allowed by setting a corresponding high-level (such as sequence-level) flag, sometimes called a parameter-gaps-in-frame-num-allowed-flag, to 1, to allow, for example, an encoder or MANE (Media Aware Network Element) to deliver a bitstream where the frame number increments by 2 or more for each reference picture preceding in the decoding order. This can be advantageous for supporting temporal scalability. A sequence of AUs with gaps in frame numbers may be received, and non-existent pictures may be created to fill the gaps. The non-existent pictures are assigned a frame number value in the gap and are considered reference pictures during decoded reference picture marking, but are not used for output (and therefore not displayed). The absence of a picture ensures that the state of the DPB for the frame numbers of pictures present in the DPB is the same for decoders that received the picture and decoders that did not receive the picture.

[0143] Another possible aspect of the first and second modifications concerns the loss of reference pictures when using sliding windows. If a reference picture is lost, assuming the loss has been detected, it may be possible to hide the picture and, if possible, report the loss to the encoder, provided a feedback channel is available. If gaps in frame numbers are not allowed, discontinuities in frame number values ​​indicate an unintended loss of a reference picture. If gaps in frame numbers are allowed, discontinuities in frame number values ​​may occur due to either the intentional deletion of a time layer or subsequence, or accidental picture loss, and the decoder should infer picture loss only if a non-existent picture was referenced in the interprediction process. The picture order number of a hidden picture may be unknown, in which case the decoder may use an incorrect reference picture without detecting any error when decoding the B picture.

[0144] Further possible embodiments of the first and second modifications relate to the loss of reference pictures due to MMCO. If a reference picture containing an MMCO command that marks a short-term reference picture as "not used for reference" is lost, the state of the reference picture in the DPB may become incorrect, and as a result, the reference picture list for several pictures following the lost picture may become incorrect. If a picture containing an MMCO command related to a long-term reference picture is lost, there is a risk that the number of long-term reference pictures in the DPB will differ from the number if that picture had been received, resulting in an "incorrect" sliding window process for all subsequent pictures. That is, the encoder and decoder may have different numbers of short-term reference pictures, and as a result, the operation of the sliding window process may not be synchronized. Further exacerbating the situation is that the decoder may not necessarily recognize that the sliding window process is out of sync.

[0145] The MMCO commands are shown below. In different embodiments, one or more or all of these commands may apply. Further options for the implementation of decoders and encoders by the first and second modifications are described below, optionally in combination with the aforementioned options for DPB management, relating to entropy decoding of several syntactic elements, such as residual data in the form of conversion coefficients into the bitstream. Reversible entropy coding of irreversibly quantized conversion coefficients is an important part of efficient video codecs. One such method is called context-adaptive variable-length coding (CAVLC), in which the encoder switches between different variable-length code (VLC) tables for different syntactic elements in a context-adaptive manner, depending on the values ​​of syntactic elements previously sent within the same slice. Encoders and decoders may use CAVLC. Each CAVLC coded bit in the bitstream can be associated with a single syntactic element, since each syntactic element is coded into the bitstream by writing the corresponding codeword selected for that syntactic element from a context-adaptively selected code table into the bitstream. Therefore, relevant information regarding the scan-order transformation coefficient levels present in the bitstream is available in a form that can be directly accessed as syntactic elements when CAVLC is used. Encoders and decoders can signal transformation coefficients in the bitstream using CAVLC. The following syntactic elements, for example, syntactic elements with the following semantics, may be used.

[0146] - A single syntactic element (indicated by CoeffToken) that specifies the total number of non-zero conversion coefficient levels within the conversion block. - One or more syntactic elements (indicated by CoeffToken) that indicate the number of subsequent transformation coefficient levels with a value of 1, for example, the sequence of syntactic elements that occur last when scanning syntactic elements in scan order up to the last non-zero syntactic element, all of which are 1, and their signs (indicated by trailing_ones_sign_flag) - One or more syntactic elements for each non-zero conversion coefficient, excluding the subsequent conversion coefficient that indicates the conversion coefficient level value. - A single syntactic element indicating the total number of zero-value conversion coefficient levels. - A syntactic element that indicates the number of consecutive zero-value conversion coefficient levels in the scan sequence from the current scan position until a non-zero value conversion coefficient level is encountered.

[0147] Alternatively or additionally, the encoder may choose to use CABAC, and therefore between context-adaptive binary arithmetic coding and CAVLC, signaling the choice in the bitstream, and the decoder may read this signal and decode the residual data in a specified manner.

[0148] Further options for implementing decoders and encoders by the first and second modifications are described below, which may optionally be combined with either the DPB management or the aforementioned CAVLC, and relate to a quarter-pixel interpolation filter. To enable interpretation with finer granularity than a normal full-pixel sample grid, a sample interpolation process is used to derive sample values ​​at sub-pixel sample positions that may range from half-pixel to quarter-pixel positions. One method of performing quarter-pixel interpolation may be used by encoders and decoders and is as follows: First, a 6-tap FIR filter is used to generate sample values ​​at half-pixel positions, and then the half-pixel position sample values ​​generated by interpolation are averaged to generate sample values ​​at quarter-pixel positions for the luminance component.

[0149] Further embodiments by modifications of the first and second are described below. According to one embodiment, the device may further comprise a decoded picture buffer (DPB), and may be configured to decode the current AU using picture predictions from referenced reference pictures stored in the DPB to obtain the decoded picture, insert the decoded picture into the DPB, assign each reference picture stored in the DPB a classification as one of short-term reference picture, long-term reference picture, and picture not used for reference, read DPB mode information from the current AU, if the DPB mode information indicates a first mode, remove one or more reference pictures classified as short-term pictures from the DPB according to a FIFO strategy, and if the DPB mode information indicates a second mode, read memory management control information including at least one command in the current AU, change the classification assigned to at least one of the reference pictures stored in the DPB, and execute at least one command to use the classification of reference pictures in the DPB to manage the removal of reference pictures from the DPB.

[0150] In one embodiment, the device may be configured to read an instruction from the current AU indicating whether the decoded picture should not be used for inter-picture prediction, to insert the decoded picture into the DPB if it is not indicated that the decoded picture should not be used for inter-picture prediction or should not be output directly, and to output the decoded picture directly in the DPB without buffering it if it is indicated that the decoded picture should not be used for inter-picture prediction and should be output directly.

[0151] According to one embodiment, the device may be further configured to assign a frame index to each reference picture in a DPB classified as a long-running picture, and to use a given reference picture in a DPB classified as a long-running picture as a referenced reference picture in the DPB when the frame index assigned to the given reference picture is referenced in the current AU.

[0152] In one embodiment, the device may be configured to perform one or more of the following: if at least one command in the current AU is a first command, reclassify reference pictures in a DPB classified as short-term reference pictures as pictures not used for reference; if at least one command in the current AU is a second command, reclassify reference pictures in a DPB classified as long-term reference pictures as pictures not used for reference; if at least one command in the current AU is a third command, reclassify reference pictures in a DPB classified as short-term pictures as long-term reference pictures and assign a frame index to the reclassified reference pictures; if at least one command in the current AU is a fourth command, set an upper frame index limit according to the fourth command and reclassify all reference pictures in a DPB classified as long-term pictures that have been assigned a frame index exceeding the upper frame index limit as pictures not used for reference; if at least one command in the current AU is a sixth command, classify the current picture as a long-term picture, a picture not used for reference, and assign a frame index to the reclassified reference pictures.

[0153] In one embodiment, the device may be configured to remove from the DPB any reference pictures that are classified as not being used for reference and therefore should no longer be output. In one embodiment, the device may be configured to read an entropy coding mode indicator from a data stream and decode the predicted residual data from the current AU using the context-adaptive variable-length coding mode if the entropy coding mode indicator indicates the context-adaptive variable-length coding mode, and using the context-adaptive binary arithmetic coding mode if the entropy coding mode indicator indicates the context-adaptive binary arithmetic coding mode. In one embodiment, the device may be configured to derive a quarter-pixel value in a referenced picture based on the motion vector in the current AU and using a 6-tap FIR filter, such that it derives a half-pixel value and averages the half-pixel values ​​of neighboring pixels.

[0154] In one embodiment, the apparatus is The current AU is decoded using picture-to-picture predictions from the referenced picture stored in the DPB to obtain the decoded picture, and the decoded picture is inserted into the DPB. Assigning a classification to each reference picture stored in the DPB as one of the following: short-term reference picture, long-term reference picture, or picture not used for reference. Read DPB mode information from the current AU. If the DPB mode information indicates a first mode, remove one or more reference pictures classified as short-term pictures from the DPB in accordance with the First-In, First-Out (FIFO) strategy. If the DPB mode information indicates a second mode, read the memory management control information, which includes at least one command in the current AU, and execute at least one command to change the classification assigned to at least one of the reference pictures stored in the DPB, and to use the classification of the reference picture in the DPB to manage the removal of the reference picture from the DPB. It can be configured to perform the following actions.

[0155] According to one embodiment, the apparatus is Read instructions from the current AU regarding whether the decoded picture will be used for picture-to-picture prediction. If it is not indicated that the decoded picture should not be used for inter-picture prediction or should not be output directly, insert the decoded picture into the DPB; if it is indicated that the decoded picture should not be used for inter-picture prediction and should be output directly, output the decoded picture directly in the DPB without buffering it. It can be configured to perform the following actions.

[0156] In one embodiment, the apparatus is Assigning a frame index to each referenced picture in the DPB classified as a long-term picture, and If a specified reference picture in a DPB classified as a long-running picture is referenced within the DPB when the frame index assigned to that specified reference picture is referenced within the current AU, then it should be used as the referenced reference picture within the DPB. It can be further configured to perform the following: According to one embodiment, the apparatus is If at least one command in the current AU is the first command, Reclassifying reference pictures within a DPB that were previously categorized as short-term reference pictures as pictures not used for reference. If at least one command in the current AU is the second command, Reclassifying reference pictures within a DPB that were previously categorized as long-term reference pictures as pictures not used for reference. If at least one command in the current AU is a third command, Reclassifying reference pictures within a DPB that were previously categorized as short-term pictures as long-term reference pictures, and assigning frame indices to the reclassified reference pictures. If at least one command in the current AU is the fourth command, The fourth command sets an upper frame index limit, and all referenced pictures in the DPB that are classified as long-term pictures and have frame indexes exceeding the upper frame index limit are reclassified as pictures not used for reference. If at least one command in the current AU is the sixth command, The current picture, which is a long-term picture, is reclassified as a picture that is not used for reference, and a frame index is assigned to the reclassified reference picture. It can be configured to perform one or more of the following: In one embodiment, the apparatus is It can be configured to remove any reference pictures from the DPB that are classified as pictures not used for reference and therefore should no longer be output.

[0157] The following describes the further optional details and characteristics of the third variant. Multiple Reference Lines (MRLs): For intra-prediction, in addition to adjacent lines of neighboring samples, one of two non-adjacent reference lines, corresponding to a position two or three lines away from the current block, can also be used as a reference line for in-picture prediction of luminance samples. Adaptive MV Resolution (AMVR): Selection of MV resolution is performed at the CU level. For interpreted CUs, the selected MV resolution is indicated and can be quarter, half, an integer, or 4 in units of luminance samples. If a half-luminance sample resolution is selected, an alternative luminance interpolation filter is used for the half-sample positions within this block, i.e., a different interpolation filter is used when the resolution is a quarter or half pixel relative to the MV.

[0158] History-Based MV Prediction (HMVP): In addition to spatially and temporally neighborly MV prediction, a new candidate type is added for merge mode and MV prediction in the AMVP candidate list. HMVP candidates are established using a 5-entry table that is maintained and updated using a first-in, first-out (FIFO) rule. The motion vector candidate list is generated using spatially and temporally neighborly and HMVP candidates. Affine Motion: An affine motion model with CU-level signaling is used for luminance. The CU-level affine motion can be either a 4-parameter or 6-parameter model. The 4-parameter model uses two MVs corresponding to two control points located at the upper-left and upper-right corners of the CU, while the 6-parameter model uses three MVs corresponding to three control points located at the upper-left, lower-left, and upper-right corners. When the CU is coded in affine motion mode, the luminance block of the CU is divided into 4x4 subblocks, and the MV at the central sample position of each subblock is calculated according to the affine motion model and set as the subblock MV based on the control points. The subblock MV is rounded to 1 / 16 luminance sample accuracy during computation, and a set of 6-tap interpolation filters is applied to generate the prediction for each subblock. For non-affine cases, merge mode and AMVP mode are used for predicting and coding the affine motion parameters.

[0159] Coefficient encoding: When encoding a coefficient level, first a flag (SigFlag) is indicated to specify whether the coefficient level is non-zero. If the flag is equal to 1 (the coefficient level is non-zero), then a further flag (Gt1-Flag) is indicated to specify whether the absolute level is greater than 1. In that case (when the absolute level is greater than 1), there are two further flags: a parity level flag (Parity-Flag) that specifies the parity of the converted coefficient level, and another flag (Gt3-Flag) that specifies whether the absolute value of the converted coefficient level is greater than 3.

[0160] The fourth modification of video decoding is performed by decoding the video from the video data stream using block-based predictive decoding and transformation-based residual decoding, the block-based predictive decoding and transformation-based residual decoding are performed by decoding the predicted residual data of the intra-predicted blocks from the video data stream using context-adaptive binary arithmetic decoding, the context-adaptive binary arithmetic decoding is performed Decoding the coordinates of the position within the transformation block representing the predicted residual data encountered when traversing the transformation coefficients of the transformation block according to a predetermined scan order, and Sequentially decoding the values ​​of the included and ranked transformation coefficients in a predetermined scan order between the last non-zero transformation coefficient and the first transformation coefficient scanned, and The predetermined scan order is selected from diagonal scan order, horizontal scan order, and vertical scan order by using a mapping that maps each of the multiple intra-prediction modes to the corresponding one among the diagonal scan order, horizontal scan order, and vertical scan order, according to the intra-prediction mode of the intra-predicted block. It is done by [the specified method].

[0161] A fourth modification relating to the present invention performs video encoding by encoding video into a video data stream using block-based predictive encoding and transform-based residual encoding, the block-based predictive encoding and transform-based residual encoding are performed by encoding intra-predicted block predictive residual data into the video data stream using context-adaptive binary arithmetic encoding, Encoding the coordinates of the position within the transformation block representing the predicted residual data encountered when traversing the transformation coefficients of the transformation block according to a predetermined scan order, and Sequentially encoding the values ​​of the included and ranked transformation coefficients in a predetermined scan order between the last non-zero transformation coefficient and the first transformation coefficient scanned, as well as The predetermined scan order is selected from diagonal scan order, horizontal scan order, and vertical scan order by using a mapping that maps each of the multiple intra-prediction modes to the corresponding one among the diagonal scan order, horizontal scan order, and vertical scan order, according to the intra-prediction mode of the intra-predicted block. It is done by [the specified method].

[0162] The present invention includes a video encoder, a video decoder, and a method for decoding video according to the above-described video decoding modification, as well as a method for encoding video according to a fourth modification of the above-described video, and a video data stream obtained by each encoding method.

[0163] The following describes further optional details and features of specific video encoding and video decoding variations of embodiments of the present invention. A fourth modified embodiment may conform to H.265 / HEVC. [PCM] Prediction, transformation, quantization, and entropy coding are bypassed, and samples are represented directly by a predefined number of bits. When PCM is enabled, the number of bits (minus 1) used to represent luminance samples and chroma samples is indicated, respectively. Furthermore, the minimum and maximum block sizes (difference from the minimum) for a 2N×2N block in which PCM can be optimally used are indicated. When a 2N×2N block is being parsed, if the size of that block is between the described minimum and maximum values, the flag indicates whether PCM mode is being used. When PCM mode is used, PCM samples are byte-aligned, the luminance samples of the block are represented / parsed directly in a raster scan with the indicated PCM luminance bit length, and then the chroma samples of the PCM block are represented / parsed in a raster scan with the indicated PCM chroma bit length, with the first half of the chroma samples being Cb samples and the remainder being Cr samples. Furthermore, the CABAC decoding engine terminates each time the PCM flag is parsed and initialized after decoding the PCM syntax.

[0164] [Intramode Dependency Coefficient Scan Order] The scan order of coefficients within an intracoded block is dynamically changed based on the transformation block size and intra mode. The scan is performed for every 4x4 subblock / region of the transformation block (e.g., a 4x4 transformation block size uses only one coefficient region, an 8x8 transformation block uses four coefficient regions, and a 16x16 transformation block size uses sixteen regions). The choice of scan method in intra mode depends on the transformation block size. For 16x16 and 32x32 transformation block sizes, the scan is performed diagonally (starting from 0,0 -> 1,0 -> 0,1 -> 2,0 -> 1,1 -> 0,2 -> 3,0, etc. - for example, decreasing the y value by step 1 and increasing the x value by step 1 at each step, restarting the y value from x+1 when y is equal to 0, and setting x to 0 as it starts from value 0,0). For 4x4 (luminance or chroma) and 8x8 (luminance) conversion block sizes, the coefficient scan order depends on the intra-mode associated with the intra-direction of the prediction of neighboring blocks. Vertical scanning is used when the prediction direction is nearly horizontal, and horizontal scanning is used when the prediction direction is nearly vertical. For other prediction directions, diagonal upper-right scanning is used.

[0165] [Conversion Quantization Bypass Mode] For each coding unit or coding block, it may be indicated whether both transformation and quantization are bypassed, which indicates that the residual signals from inter-picture or intra-picture predictions are directly entropy coded for that block. In that mode, the in-loop filter is skipped.

[0166] [Further embodiments] The embodiments of the present invention will be described again below in more general terms. The following embodiments may be optionally combined with any of the features described above. In the following further embodiments, the following options may apply to all embodiments. The data stream may be a video data stream or an audio data stream. The device for decoding the data stream may be a device for decoding video from a video data stream, or a device for decoding an audio signal from an audio data stream. The device for encoding the data stream may be a device for encoding video into a video data stream, or an audio signal into an audio data stream. The payload packet may be a coded video payload packet carrying video data, for example, encoded video data. The term “supplementary information packet” may be used interchangeably with the term “supplementary information payload packet.” The expressions “check the data stream for reliability” and “verify the data stream” may be used interchangeably.

[0167] Different embodiments can be combined; that is, any feature defined with respect to any of the embodiments may be combined with any of the further embodiments. 1. A device 20 for decoding a data stream 14, wherein the data stream includes a plurality of payload packets 16 carrying payload data, and further includes supplemental information packets 18, the supplemental information packets carrying supplemental information messages 19, and the device, From the data stream, derive an instruction 51 indicating whether one or more supplemental information messages should be considered to validate the data stream 40. A predetermined portion 13 of the data stream is determined by including one or more supplemental information messages in the predetermined portion, when instruction 51 indicates that one or more supplemental information messages should be considered in order to validate the data stream, 30. Obtaining a digital signature 43 to verify a predetermined portion 13 from the instructions in the data stream 21 Apparatus 20 is configured to perform the following actions.

[0168] 2. The apparatus according to Embodiment 1, wherein the apparatus is configured to verify a predetermined portion of a data stream by checking whether that portion conforms to a digital signature. 3. The apparatus according to Embodiment 1 or 2, wherein the apparatus is configured to provide a predetermined part and a digital signature for verifying the predetermined part. 4. A device 20 for decoding a data stream 14, wherein the device is configured to check the data stream for reliability, and the data stream includes a plurality of payload packets 16 carrying payload data, and further includes supplemental information packets 18, the supplemental information packets carrying supplemental information messages 19, and the device, From the data stream, derive an instruction 51 indicating whether supplemental information messages should be considered for validating the data stream 21. A predetermined portion 13 of the data stream that is to be verified is determined by including one or more of the supplemental information messages 19 within the predetermined portion 13 when instruction 51 indicates that supplemental information messages should be considered in order to verify the data stream 30. To obtain a digital signature 43 based on data stream 14, Check whether a given portion 13 of the data stream conforms to the digital signature 43 41 Apparatus 20 is configured to perform the following actions.

[0169] 5. Check whether a given portion 13 of the data stream conforms to the digital signature 43. The process involves applying a hash function to a predetermined portion 13 to obtain a hash value, Check whether the hash value conforms to digital signature 43, The apparatus according to any one of Embodiments 1 to 4, including the apparatus described above. 6. Check whether the hash value conforms to the digital signature 43. Forming a verification string 48 based on the hash value, Decrypting digital signature 43 to obtain the check value, The verification string 48 is checked to see if it matches the check value, The apparatus according to Embodiment 5, including the apparatus described in Embodiment 5.

[0170] 7. The predetermined portion 13 is determined by including one or more supplemental information packets in the predetermined portion 13 when the instruction indicates that one or more supplemental information messages should be considered in order to verify the predetermined portion 13. The apparatus according to any one of Embodiments 1 to 6.

[0171] 8. The apparatus according to any one of embodiments 1 to 7, configured to include in the predetermined portion 13 all supplemental information packets that carry any supplemental information messages to be included in the predetermined portion 13. 9. The apparatus according to any one of embodiments 1 to 8, wherein the decision of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a predetermined portion 13 is conditional on the result of checking whether or not the supplemental information payload packet contains a supplemental information message of any supplemental information message type from a set of supplemental information message types. 10. The instruction is a flag having a first state and a second state, and the device controls a predetermined portion 13 of the data stream. If the flag has the first state, one or more supplemental information messages to be included in a predetermined part 13 are determined based on a predefined set of supplemental information messages among the supplemental information messages. If the flag has the second state, none of the supplementary information messages should be included in the predetermined part 13. The apparatus according to any one of embodiments 1 to 9, configured to be determined by

[0172] 11. The apparatus according to Embodiment 10, wherein a predefined set of supplemental information messages includes all supplemental information messages of supplemental information packets that reference multiple payload packets 16. 12. The apparatus according to Embodiment 10, wherein a predefined set of supplemental information messages includes all supplemental information messages of any type from a set of types of supplemental information messages. 13. When the instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the system is further configured to derive identification information of one or more supplemental information messages that should be included in a given portion 13 from the data stream. The apparatus according to any one of Embodiments 1 to 12.

[0173] 14. The apparatus according to Embodiment 13, wherein, when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the apparatus is configured to derive from the data stream a syntactic element that identifies a set of types of supplemental information messages that should be included in a predetermined portion 13. 15. The apparatus according to any one of embodiments 1 to 14, configured to derive from the data stream syntactic elements that indicate whether one or more supplemental information messages should be considered for validation of the data stream and that signal identification information of one or more supplemental information messages to be included in a predetermined portion 13.

[0174] 16. A syntactic element has multiple first and second states, and the device, If a syntactic element has one of the first states, one or more supplementary information messages to be included in a predetermined part 13 are determined according to the state of the syntactic element, If the syntactic element has a second state, none of the one or more supplemental information messages shall be included in the predetermined part 13. The apparatus according to embodiment 15, configured to perform the following:

[0175] 17. The apparatus according to Embodiment 13, wherein, when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the apparatus is configured to derive from the data stream a first syntactic element indicating the number of supplemental information message types to be considered for a given portion 13, and a number of second syntactic elements, each identifying each of the supplemental information message types that should be included in the given portion 13. 18. The determination of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a predetermined portion 13 is further configured to be conditional on the result of checking whether the supplemental information packet is preceded by a prefix supplemental information packet that instructs the predetermined portion 13 to include subsequent supplemental information packets. The apparatus described in Embodiment 7. 19. The decision of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a predetermined portion 13 is further configured to be conditional on the result of checking whether the supplemental information packet contains a supplemental information message that instructs the predetermined portion 13 to include the supplemental information packet. The apparatus described in Embodiment 7.

[0176] 20. The decision of whether or not to include one of several supplemental information messages in a given portion is further configured to be conditional on the result of checking whether the supplemental information message is preceded by a prefix supplemental information message that instructs the given portion 13 to include a subsequent supplemental information message. The apparatus described in Embodiment 7. 21. The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a given portion is further configured to be conditional on the result of checking whether the supplemental information packet contains a nesting supplemental information message that instructs the supplemental information packet to be included in the given portion 13. The apparatus described in Embodiment 7.

[0177] 22. The decision of whether or not to include one of several supplemental information messages in a given portion is further configured to be conditional on the result of checking whether the supplemental information message is included in a nesting supplemental information message that instructs the given portion 13 to include the supplemental information message. The apparatus described in Embodiment 7. 23. When determining the predetermined portion 13, The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a predetermined portion is configured to be conditional on the result of checking whether the supplemental information packet contains any supplemental information message of any type from the set of types of supplemental information messages that should be included in the predetermined portion 13. The apparatus according to any one of Embodiments 1 to 22.

[0178] 24. When determining the designated portion, The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a predetermined portion is configured to be conditional on the result of checking whether the first supplemental information message among one or more supplemental information messages carried within the supplemental information packet is one of a set of types of supplemental information messages that should be included in the predetermined portion 13. The apparatus according to any one of Embodiments 1 to 23.

[0179] 25. Deriving several parts from a data stream, wherein the data stream is verifiable in units of several parts, and several parts include a predetermined part 13. Assigning each of the payload packets 16 to one of several parts, An apparatus according to any one of embodiments 1 to 24, configured to perform the following: 26. When determining the predetermined portion 13, When the instruction indicates that one or more supplemental information messages should be considered to validate the data stream, the predetermined part 13 is configured to include one supplemental information message from among multiple supplemental information messages having any type from a set of types of supplemental information messages, and the predetermined part 13 has a predetermined rank in an order defined among several parts. The apparatus described in Embodiment 25.

[0180] 27. The apparatus according to embodiment 26, wherein a predetermined rank is the highest rank in an order defined among several parts. 28. When determining the predetermined portion 13, The payload packet 16 assigned to a predetermined portion 13 is included in the predetermined portion 13, When the instruction indicates that one or more supplemental information messages should be considered to validate the data stream, a predetermined portion 13 includes supplemental information messages for the data stream associated with the payload packet 16 assigned to the predetermined portion 13, having any type from the set of supplemental information message types. The apparatus according to embodiment 25, configured to perform the following. 29. The apparatus according to Embodiment 25 or 28, wherein each supplemental information packet is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information packet carrying any type of supplemental information from a set of types of supplemental information to one or more portions to which the payload packet associated with each supplemental information packet is allocated.

[0181] 30. When determining the predetermined portion 13, When the instructions indicate that supplementary information should be considered to validate the data stream, based on the instructions in the data stream, The predetermined portion 13 includes all supplemental information of a data stream having any type from a set of supplemental information types, wherein the predetermined portion 13 has a predetermined rank, for example, the highest rank in an order defined among several portions. A predetermined portion 13 includes all supplemental information of a data stream associated with a payload packet 16 assigned to the predetermined portion 13, having any type from the set of supplemental information types. The apparatus according to embodiment 25, 28, or 29, which is configured to be determined from among the following.

[0182] 31. The apparatus according to Embodiment 25, wherein each supplemental information message is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information message of a data stream having any type from a set of types of supplemental information messages to one of the parts, the one part of which is associated with one of the further parts, and the further part is associated with the payload packet 16 associated with each supplemental information message. 32. The apparatus according to Embodiment 25, wherein each supplemental information packet is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information packet of a data stream carrying any type of a set of types of supplemental information messages to one of the parts, the one part of which is associated with another of the parts, and the other of the parts is associated with the payload packet 16 associated with each supplemental information packet. 33. The system is configured to derive syntactic elements that indicate the number of parts from a data stream, where the syntactic elements have values, and the device, It is configured to infer that some parts of the number correspond to the value plus 1, multiplied by 2. The apparatus according to Embodiment 31 or 32.

[0183] 34. Configured to derive a syntax element from a data stream that indicates the number of several parts, the syntax element has a value, and the apparatus is configured to infer that the number of several parts corresponds to the value plus 1. The apparatus according to Embodiment 31 or 32. 35. One part and a further one of the parts are successive to each other within a hierarchical order defined among several parts. For example, one of the parts is successive to a further one of the parts within the hierarchical order. The apparatus according to Embodiment 33 or 34. 36. The position index of one part within the hierarchical order defined among parts corresponds to the position index of a further one of the parts within the hierarchical order plus half of the number of several parts. The apparatus according to Embodiment 33 or 34.

[0184] 37. Each supplementary information message is associated with one of the payload packets 16. The apparatus according to any one of Embodiments 1 to 36. [[ID=##]]38. Each supplementary information message is associated with one of the associated payload packets 16, and the associated payload packet is part of the same picture unit or the same access unit as the supplementary information packet. The apparatus according to any one of Embodiments 1 to 37. 39. When determining a predetermined part 13, checking whether one or more supplementary information messages to be included in the predetermined part 13 include multiple instances of the same supplementary information message, and when one or more supplementary information messages to be included in the predetermined part 13 include multiple instances of the same supplementary information message, including only one instance of the multiple instances of the same supplementary information message, The apparatus is configured to perform the above operations. The apparatus according to any one of Embodiments 1 to 38.

[0185] It should be noted that there may be a small error in the original text. In item 38, "関連付けられた1つに関連付けられており" is a bit repetitive in expression. The translation tries to convey the meaning as accurately as possible while maintaining the integrity of the original text structure and tags.When checking whether one or more supplementary information messages to be included in a predetermined portion 13 include multiple instances of the same supplementary information message, the apparatus according to Embodiment 39, which is configured to perform the check for a single picture unit or access unit. 41. The apparatus according to Embodiment 39, which is configured to position one instance of multiple instances of the same supplementary information message at a predetermined position within the predetermined portion 13. 42. The predetermined position is defined with respect to the start position or end position of a picture unit included in the predetermined portion 13, or is defined with respect to a predetermined payload packet included in the predetermined portion 13, for example, a predetermined payload packet or a predetermined supplementary information packet. The apparatus according to Embodiment 41.

[0186] 43. The apparatus according to any one of Embodiments 39 to 42, which is configured to position one or more supplementary information messages to be included in the predetermined portion 13 according to a predefined order among the one or more supplementary information messages within the predetermined portion 13. 44. An apparatus 10 for encoding a data stream, the apparatus being configured to make the data stream checkable for reliability and to encode a plurality of payload packets 16 carrying payload data within the data stream, and further encode supplementary information packets 18 within the data stream, the supplementary information packets carrying supplementary information messages, and the apparatus being inserting an instruction 51 for indicating whether one or more supplementary information messages should be considered for verifying the data stream into the data stream 23. Determine a predetermined portion 13 of a data stream by including one or more supplemental information messages within the predetermined portion 13 when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, and determine that the predetermined portion 13 should be checkable with respect to reliability, Obtaining a digital signature 43 based on a predetermined portion 13 41' Apparatus 10 is configured to perform the following actions.

[0187] 45. When obtaining a digital signature 43, The process involves applying a hash function to a predetermined portion 13 to obtain a hash value, Obtaining a digital signature 43 based on the hash value, The apparatus according to embodiment 44, configured to perform the following: 46. ​​When obtaining a digital signature 43, Form a verification string 48 based on the hash value. Sign the verification string 48 to obtain the digital signature 43. The apparatus according to embodiment 45, configured to perform the following. 47. The predetermined portion 13 is determined by including one or more supplemental information packets in the predetermined portion 13 when the instruction indicates that one or more supplemental information messages should be considered in order to verify the predetermined portion 13. The apparatus according to any one of embodiments 44 to 46.

[0188] 48. The apparatus according to any one of embodiments 44 to 47, configured to include in the predetermined portion 13 all supplemental information packets that carry any supplemental information messages to be included in the predetermined portion 13. 49. The apparatus according to any one of embodiments 44 to 48, wherein the decision of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a predetermined portion 13 is conditional on the result of checking whether the supplemental information packet includes a supplemental information message of any supplemental information message type from a set of supplemental information message types. 50. The instruction is a flag having a first state and a second state, and the device, The flag is set to the first state, and one or more supplementary information messages that should be included in a given part 13 are determined based on a predefined set of supplementary information messages, The flag is set to the second state, and none of the supplementary information messages are included in the designated part 13, The apparatus according to any one of embodiments 44 to 49, configured to be determined from among the following.

[0189] 51. The apparatus according to Embodiment 50, wherein a predefined set of supplemental information messages includes all supplemental information messages of supplemental information packets that reference multiple payload packets 16. 52. The apparatus according to Embodiment 50, wherein a predefined set of supplemental information messages includes all supplemental information messages of any type from a set of types of supplemental information messages.

[0190] 53. The apparatus according to any one of embodiments 44 to 52, further configured to insert into the data stream identification information of one or more supplemental information messages to be included in a predetermined portion 13 when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream.

[0191] 54. The apparatus according to Embodiment 53, wherein, when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, a syntactic element is configured to insert into the data stream a set of types of supplemental information messages that should be included in a predetermined portion 13 from the data stream. 55. The apparatus according to any one of embodiments 44 to 54, configured to insert into the data stream a syntactic element that indicates whether one or more supplemental information messages should be considered for validation of the data stream and signals identification information of one or more supplemental information messages to be included in a predetermined portion 13. 56. A syntactic element has multiple first states and second states, and the device, Setting a syntactic element to one of the first states in accordance with one or more supplementary information messages that should be included in a predetermined part 13, The syntactic element is set to the second state, so that none of the one or more supplementary information messages are included in the predetermined part 13, The apparatus according to embodiment 55, configured to distinguish between the following.

[0192] 57. The apparatus according to Embodiment 53, wherein, when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, the apparatus is configured to insert into the data stream a first syntactic element indicating the number of supplemental information message types to be considered for a given portion 13, and a number of second syntactic elements, each identifying each of the supplemental information message types that should be included in the given portion 13. 58. The decision of whether or not to include one supplemental information packet from a plurality of supplemental information packets in a given portion is further configured to be conditional on the result of checking whether the supplemental information packet is preceded by a prefix supplemental information packet that instructs the given portion 13 to include subsequent supplemental information packets. The apparatus described in Embodiment 53. 59. Further configured to condition a determination of whether to include one of a plurality of supplementary information packets in a predetermined part 13 on a result of checking whether a supplementary information message including an instruction for the supplementary information packet to be included in the predetermined part 13 is included. The apparatus according to Embodiment 47.

[0193] 60. Further configured to condition a determination of whether to include one of a plurality of supplementary information messages in a predetermined part on a result of checking whether the supplementary information message is preceded by a prefix supplementary information message instructing to include a subsequent supplementary information message in the predetermined part 13. The apparatus according to Embodiment 47. 61. Further configured to condition a determination of whether to include one of a plurality of supplementary information packets in a predetermined part 13 on a result of checking whether a nesting supplementary information message including an instruction for the supplementary information packet to be included in the predetermined part 13 is included. The apparatus according to Embodiment 47. 62. Further configured to condition a determination of whether to include one of a plurality of supplementary information messages in a predetermined part on a result of checking whether the supplementary information message is included in a nesting supplementary information message instructing to include a supplementary information packet in the predetermined part 13. The apparatus according to Embodiment 47.

[0194] 63. When determining the predetermined part 13, configured to condition a determination of whether to include one of a plurality of supplementary information packets in a predetermined part on a result of checking whether the supplementary information packet includes any supplementary information message of a type in a set of types of supplementary information messages that should be included in the predetermined part 13. The apparatus according to any one of Embodiments 44 to 62. 64. When determining the predetermined portion 13, The decision of whether or not to include one supplemental information packet from among multiple supplemental information packets in a predetermined portion is configured to be conditional on the result of checking whether the first supplemental information message among one or more supplemental information messages carried within the supplemental information packet is one of a set of types of supplemental information messages that should be included in the predetermined portion 13. The apparatus according to any one of embodiments 44 to 63.

[0195] 65. The apparatus according to any one of embodiments 44 to 64, wherein each supplemental information packet is configured to provide supplemental information packets such that it does not contain both a supplemental information message of one type from a set of types of supplemental information packets that should be included in a predetermined portion 13, and a supplemental information message of a type that is not included in the set of types. 66. In a data stream, to indicate several parts, wherein the data stream is verifiable in units of several parts, and several parts include a predetermined part 13. Assigning each of the payload packets 16 to one of several parts, An apparatus according to any one of embodiments 44 to 65, configured to perform the following: 67. When determining the predetermined portion 13, When the instruction indicates that one or more supplemental information messages should be considered to validate the data stream, the given portion 13 includes one supplemental information message from among multiple supplemental information messages having any type from a set of supplemental information message types, and the given portion 13 has a predetermined rank in an order defined among several portions. The apparatus described in Embodiment 66.

[0196] 68. The apparatus according to embodiment 67, wherein a predetermined rank is the highest rank in an order defined among several parts. 69. When determining the predetermined portion 13, The payload packet 16 assigned to a predetermined portion 13 is included in the predetermined portion 13, When the instruction indicates that one or more supplemental information messages should be considered to validate the data stream, a predetermined portion 13 includes supplemental information messages for the data stream associated with the payload packet 16 assigned to the predetermined portion 13, having any type from the set of supplemental information message types. The apparatus according to embodiment 66, configured to perform the following: 70. The apparatus according to embodiment 66 or 67, wherein each supplemental information packet is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information packet carrying any type of supplemental information from a set of types of supplemental information to one or more portions to which the payload packet associated with each supplemental information packet is assigned.

[0197] 71. When determining the predetermined portion 13, When the instructions indicate that supplementary information should be considered to validate the data stream, The predetermined portion 13 includes all supplemental information of a data stream having any type from a set of supplemental information types, wherein the predetermined portion 13 has a predetermined rank, for example, the highest rank in an order defined among several portions. A predetermined portion 13 includes all supplemental information of a data stream associated with a payload packet 16 assigned to the predetermined portion 13, having any type from the set of supplemental information types. The apparatus according to embodiment 66, 67, or 68, which is configured to insert instructions into the data stream that are instructions for distinguishing between the two.

[0198] 72. The apparatus according to Embodiment 66, wherein each supplemental information message is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information message of a data stream having any type from a set of types of supplemental information messages to one of the parts, the one part of which is associated with another of the parts, and the other of the parts is associated with the payload packets 16 associated with each supplemental information message. 73. The apparatus according to Embodiment 33, wherein each supplemental information packet is associated with one of the payload packets 16, and the apparatus is configured to assign each supplemental information packet of a data stream carrying any type of a set of types of supplemental information messages to one of the parts, the one part of which is associated with another of the parts, and the other of the parts is associated with the payload packet 16 associated with each supplemental information packet. 74. The data stream is configured to insert syntactic elements that indicate the number of parts, and the syntactic elements have a value, and the device, The value is configured to be set to the value obtained by dividing the number of some parts by 2 and subtracting 1. The apparatus according to embodiment 72 or 73.

[0199] 75. The system is configured to derive syntactic elements that indicate the number of parts from a data stream, where the syntactic elements have values, and the device, The value is configured to be set to a value obtained by subtracting 1 from the number of some parts. The apparatus according to embodiment 72 or 73. 76. The apparatus according to embodiment 74 or 75, wherein one part and one further part succeed to one another in a hierarchical order defined among several parts, for example, one part succeeds to one further part in a hierarchical order. 77. The apparatus according to embodiment 74 or 75, wherein the position index of one part in a hierarchical order defined between parts corresponds to the position index of another part in the hierarchical order plus half the number of parts.

[0200] 78. The apparatus according to any one of embodiments 44 to 77, wherein each supplemental information message is associated with one of the payload packets 16. 79. The apparatus according to any one of embodiments 44 to 78, wherein each supplemental information message is associated with one of the associated payload packets 16, and the associated payload packet is part of the same picture unit or the same access unit as the supplemental information packet. 80. When determining a predetermined part 13, check whether one or more supplemental information messages that should be included in the predetermined part 13 contain multiple instances of the same supplemental information message, When one or more supplementary information messages to be included in a predetermined part 13 include multiple instances of the same supplementary information message, it is necessary to include only one instance of the multiple instances of the same supplementary information message. An apparatus according to any one of embodiments 44 to 79, configured to perform the following:

[0201] 81. The apparatus according to embodiment 80, configured to perform a check on a single picture unit or access unit when checking whether one or more supplemental information messages that should be included in a predetermined portion 13 include multiple instances of the same supplemental information message. 82. The apparatus according to embodiment 80, configured to position one instance of a plurality of identical supplemental information messages at a predetermined location within a predetermined portion 13. 83. The designated position is It is defined for the start or end position of the picture unit included in the predetermined part 13, or A predetermined payload packet included in a predetermined portion 13, for example, a predetermined payload packet or a predetermined supplementary information packet, is defined as follows: The apparatus described in Embodiment 82.

[0202] 84. The apparatus according to any one of embodiments 80 to 83, configured to position one or more supplemental information messages to be included in a predetermined portion 13 according to a predefined order of one or more supplemental information messages within a predetermined portion 13. 85. The apparatus according to any one of embodiments 44 to 84, wherein when a supplemental information packet contains multiple instances of the same supplemental information packet, the apparatus is configured to insert multiple instances into a data stream without being interleaved by another of the supplemental information packets. 86. The apparatus according to any one of embodiments 44 to 85, configured to provide a data stream such that when the supplemental information messages of a data stream include multiple instances of a first supplemental information message and multiple instances of a second supplemental information message, one instance each of the first supplemental information message and the second supplemental information message is included in one of the supplemental information packets.

[0203] 87. A method for decoding a data stream 14, wherein the data stream includes a plurality of payload packets 16 carrying payload data, and further includes supplemental information packets 18, the supplemental information packets carrying supplemental information messages 19, and the method From the data stream, derive an instruction 51 indicating whether one or more supplemental information messages should be considered to validate the data stream 40. A predetermined portion 13 of the data stream is determined by including one or more supplemental information messages in the predetermined portion, when instruction 51 indicates that one or more supplemental information messages should be considered in order to validate the data stream, 30. Obtaining a digital signature 43 to verify a predetermined portion 13 from the instructions in the data stream 21 Methods that include...

[0204] 88. A method for decoding a data stream 14, the method comprising checking the data stream for reliability, wherein the data stream comprises a plurality of payload packets 16 carrying payload data, and further comprises supplemental information packets 18, the supplemental information packets carrying supplemental information messages 19, and the method From the data stream, derive an instruction 51 indicating whether supplemental information messages should be considered for validating the data stream 21. A predetermined portion 13 of the data stream that is to be verified is determined by including one or more of the supplemental information messages 19 within the predetermined portion 13 when instruction 51 indicates that supplemental information messages should be considered in order to verify the data stream 30. To obtain a digital signature 43 based on data stream 14, Check whether a given portion 13 of the data stream conforms to the digital signature 43 41 Methods that include...

[0205] 89. A method for encoding a data stream, the method comprising: making the data stream verifiable with respect to reliability; encoding a plurality of payload packets 16 carrying payload data within the data stream; further encoding supplemental information packets within the data stream, wherein the supplemental information packets carry supplemental information messages, the method Insert an instruction 51 into the data stream indicating whether one or more supplemental information messages should be considered to validate the data stream. Determine a predetermined portion 13 of a data stream by including one or more supplemental information messages within the predetermined portion 13 when an instruction indicates that one or more supplemental information messages should be considered in order to validate the data stream, and determine that the predetermined portion 13 should be checkable with respect to reliability, Obtaining a digital signature 43 based on a predetermined portion 13 41' Methods that include...

[0206] 90. A data stream having data signals encoded within the data stream, the data stream being checkable in terms of reliability, and the data stream being A plurality of payload packets 16 that carry payload data, wherein the data stream further includes supplemental information packets, and the supplemental information packets carry supplemental information messages, Instructions indicating whether one or more supplemental information messages should be considered regarding the reliability of the data stream, A data stream that includes this. 91. The data stream according to Embodiment 90, wherein the data stream is provided using the method of Embodiment 89.

[0207] 92. The data stream has video encoded in the data stream, and the device is configured to decode the video from the data stream by block-based prediction and transformation-based residual decoding, and the decoding is, This is done by decoding the predicted residual data of residual blocks from the data stream using context-adaptive variable-length decoding, and the decoding is performed by A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a trailing-one number indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of a non-zero transformation coefficient having an absolute value of 1 when traversing the coefficients along the scan order, One or more third syntactic elements indicating the values ​​of non-zero conversion coefficients, excluding the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements that indicate the position of a non-zero conversion coefficient along the scan order by indicating the number of consecutive zero-value conversion coefficients in the scan order between consecutively encountered non-zero conversion coefficients, This is done using The apparatus according to any one of Embodiments 1 to 43.

[0208] 93. The data stream has video encoded within the data stream, and the device is configured to decode the video from the data stream by block-based predictive decoding and transform-based residual decoding, This is done by decoding the predicted residual data of the residual block from the data stream using context-adaptive binary arithmetic decoding, and context-adaptive binary arithmetic decoding is performed by The importance map indicating the location of non-zero conversion coefficients within a conversion block representing a residual block is decoded by decoding an importance flag indicating whether a non-zero conversion coefficient is located at the current position in a forward scan traversing the conversion coefficients of the conversion block, and if a non-zero conversion coefficient is located at the current position, and if the current position is not the end of the forward scan, by decoding a final importance flag indicating whether the non-zero conversion coefficient located at the current position is the last non-zero conversion coefficient in the forward scan order, and The values ​​of the non-zero conversion coefficients are decoded sequentially using a reverse scan order, which is the reverse of the forward scan order. This is done by The apparatus according to any one of Embodiments 1 to 43.

[0209] 94. The data stream has video encoded within the data stream, and the device is configured to decode the video from the data stream by block-based predictive decoding and transform-based residual decoding, This is done by decoding the predicted residual data of intra-predicted blocks from the data stream using context-adaptive binary arithmetic decoding, and context-adaptive binary arithmetic decoding is performed by Decoding the coordinates of the position within the transformation block representing the predicted residual data encountered when traversing the transformation coefficients of the transformation block according to a predetermined scan order, and Sequentially decoding the values ​​of the included and ranked transformation coefficients in a predetermined scan order between the last non-zero transformation coefficient and the first transformation coefficient scanned, and The predetermined scan order is selected from diagonal scan order, horizontal scan order, and vertical scan order by using a mapping that maps each of the multiple intra-prediction modes to the corresponding one among the diagonal scan order, horizontal scan order, and vertical scan order, according to the intra-prediction mode of the intra-predicted block. This is done by The apparatus according to any one of Embodiments 1 to 43.

[0210] 95. The data stream has video encoded in the data stream, and the device is configured to decode the video from the data stream by block-based predictive decoding and transform-based residual decoding, and the decoding is Decode the predicted residual data of the residual blocks from the data stream using context-adaptive binary arithmetic decoding of the quantization index of the transformation coefficients of the transformation block representing the residual block, and sequential inverse quantization of the quantization index where the value of the current transformation coefficient depends on the parity of the quantization index of the previous quantization index. This is done by The apparatus described in Embodiments 1 to 43.

[0211] 96. The device is an encoder configured to encode video into a data stream by block-based predictive encoding and transformation-based residual encoding, wherein the encoding is This is done by encoding the predicted residual data of the residual block into a data stream using context-adaptive variable-length coding, and the encoding is performed by A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a trailing-one number indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of a non-zero transformation coefficient having an absolute value of 1 when traversing the coefficients along the scan order, When traversing the coefficients along the scan order, one or more third syntactic elements indicating the values ​​of non-zero conversion coefficients, excluding the number of non-zero conversion coefficients with an absolute value of 1, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements that indicate the position of a non-zero conversion coefficient along the scan order by indicating the number of consecutive zero-value conversion coefficients in the scan order between non-zero conversion coefficients encountered consecutively in the scan order, This is done using The apparatus according to any one of embodiments 44 to 86.

[0212] 97. The device is an encoder configured to encode video into a data stream by block-based predictive coding and transform-based residual coding, where block-based predictive coding and transform-based residual coding are, This is done by encoding the predicted residual data of the residual block within the data stream using context-adaptive binary arithmetic coding, and context-adaptive binary arithmetic coding is performed by, The importance map indicating the location of non-zero transformation coefficients within a transformation block representing a residual block is encoded by encoding an importance flag indicating whether a non-zero transformation coefficient is located at the current position in a forward scan traversing the transformation coefficients of the transformation block, and if a non-zero transformation coefficient is located at the current position, and the current position is not the end of the forward scan, encoding a final importance flag indicating whether the non-zero transformation coefficient located at the current position is the last non-zero transformation coefficient in the forward scan order, and The values ​​of the non-zero conversion coefficients are sequentially encoded in a reverse scan order, which is the reverse of the forward scan order. This is done by The apparatus according to any one of embodiments 44 to 86.

[0213] 98. The device is configured to encode video into a data stream by block-based predictive coding and transform-based residual coding, and block-based predictive coding and transform-based residual coding are This is done by encoding the predicted residual data of the intra-predicted block into a data stream using context-adaptive binary arithmetic coding, and context-adaptive binary arithmetic coding is performed by, Encoding the coordinates of the position within the transformation block representing the predicted residual data encountered when traversing the transformation coefficients of the transformation block according to a predetermined scan order, and Sequentially encoding the values ​​of the included and ranked transformation coefficients in a predetermined scan order between the last non-zero transformation coefficient and the first transformation coefficient scanned, as well as The predetermined scan order is selected from diagonal scan order, horizontal scan order, and vertical scan order by using a mapping that maps each of the multiple intra-prediction modes to the corresponding one among the diagonal scan order, horizontal scan order, and vertical scan order, according to the intra-prediction mode of the intra-predicted block. This is done by The apparatus described in Embodiments 44 to 86.

[0214] 99. The device is configured to encode video into a data stream by block-based predictive encoding and transform-based residual encoding, Encoding predicted residual data of residual blocks into a data stream using context-adaptive binary arithmetic coding of the quantized index of the transformation coefficients of the transformation block representing the residual block, and sequential quantization of the transformation coefficients to obtain the quantized index, where the quantizer for quantizing the current transformation coefficient depends on the parity of the quantized index of the previous quantized index. This is done by The apparatus described in Embodiments 44 to 86.

[0215] 100. The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding, and the encoding is This is done by encoding the predicted residual data of the residual block into the data stream using context-adaptive variable-length coding, and the encoding is performed by A first syntactic element that indicates the total number of non-zero conversion coefficients in the conversion block representing the residual block, and a trailing-one number indicating the number of non-zero conversion coefficients that have an absolute value of 1 when traversing the coefficients along the scan order, One or more second syntactic elements indicating the sign of a non-zero transformation coefficient having an absolute value of 1 when traversing the coefficients along the scan order, When traversing the coefficients along the scan order, one or more third syntactic elements indicating the values ​​of non-zero conversion coefficients, excluding the number of non-zero conversion coefficients with an absolute value of 1, A fourth syntactic element indicating the total number of zero-value conversion coefficient levels in the conversion block from the first non-zero conversion coefficient encountered in the scan order, One or more fifth syntactic elements that indicate the position of a non-zero conversion coefficient along the scan order by indicating the number of consecutive zero-value conversion coefficients in the scan order between non-zero conversion coefficients encountered consecutively in the scan order, This is done using The apparatus according to any one of Embodiments 1 to 43.

[0216] 101. The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding, and block-based predictive coding and transformation-based residual coding are This is done by encoding the predicted residual data of the residual block within the data stream using context-adaptive binary arithmetic coding, and context-adaptive binary arithmetic coding is performed by, The importance map indicating the location of non-zero transformation coefficients within a transformation block representing a residual block is encoded by encoding an importance flag indicating whether a non-zero transformation coefficient is located at the current position in a forward scan traversing the transformation coefficients of the transformation block, and if a non-zero transformation coefficient is located at the current position, and the current position is not the end of the forward scan, encoding a final importance flag indicating whether the non-zero transformation coefficient located at the current position is the last non-zero transformation coefficient in the forward scan order, and The values ​​of the non-zero conversion coefficients are sequentially encoded in a reverse scan order, which is the reverse of the forward scan order. This is done by The apparatus according to any one of Embodiments 1 to 43.

[0217] 102. The data stream has video encoded within the data stream by block-based predictive coding and transformation-based residual coding, and block-based predictive coding and transformation-based residual coding are This is done by encoding the predicted residual data of the intra-predicted block into the data stream using context-adaptive binary arithmetic coding, Encoding the coordinates of the position within the transformation block representing the predicted residual data encountered when traversing the transformation coefficients of the transformation block according to a predetermined scan order, and Sequentially encoding the values ​​of the included and ranked transformation coefficients in a predetermined scan order between the last non-zero transformation coefficient and the first transformation coefficient scanned, as well as The predetermined scan order is selected from diagonal scan order, horizontal scan order, and vertical scan order by using a mapping that maps each of the multiple intra-prediction modes to the corresponding one among the diagonal scan order, horizontal scan order, and vertical scan order, according to the intra-prediction mode of the intra-predicted block. This is done by The apparatus according to any one of Embodiments 1 to 43.

[0218] 103. The data stream has video encoded within the data stream by block-based predictive encoding and transform-based residual encoding, and block-based predictive encoding and transform-based residual encoding, Encoding predicted residual data of residual blocks into a data stream using context-adaptive binary arithmetic coding of the quantized index of the transformation coefficients of the transformation block representing the residual block, and sequential quantization of the transformation coefficients to obtain the quantized index, where the quantizer for quantizing the current transformation coefficient depends on the parity of the quantized index of the previous quantized index. This is done by The apparatus according to any one of Embodiments 1 to 43. 104. A data stream generated using the method of Embodiment 89. 105. A computer program for carrying out the method of Embodiment 87, or Embodiment 88, or Embodiment 89, when executed on a computer or signal processor.

[0219] [Alternative examples of embodiments] While some aspects are described as features within the context of the apparatus, it is clear that such descriptions may also be considered descriptions of corresponding features of the method. Similarly, while some aspects are described as features within the context of the method, it is clear that such descriptions may also be considered descriptions of corresponding features relating to the function of the apparatus. In particular, block diagrams illustrating the function of the apparatus may also be understood as descriptions of the respective methods, including the functions described by the blocks in the block diagram as steps of the method.

[0220] Data signals or data streams provided by embodiments of the present invention (e.g., media data streams, video data streams, audio data streams) can be stored in a digital storage medium, such as a non-temporary or temporary digital storage medium, or can be transmitted over a transmission medium such as a wireless or wired transmission medium. In other words, further embodiments provide a computer product, such as a data stream product or a bitstream product, such as a non-temporary digital storage medium, which includes, for example, a data signal or data stream according to any of the embodiments described herein, stored therein.

[0221] Further embodiments provide a method for storing data, the method comprising the step of storing a data stream in a digital storage medium, for example, a non-temporary digital storage medium, the data stream carrying the data. For example, the data stream is according to any of the embodiments described herein. For example, encodes the data according to any of the encoding methods described herein.

[0222] Further embodiments provide a method for transmitting any data stream from any of the embodiments described herein.

[0223] Features described for devices for receiving or processing signals (e.g., receivers, decoders) should be understood to function as features of the respective signals, such as data streams, as descriptions of the respective features of devices for providing signals (e.g., encoders) and vice versa. In particular, those skilled in the art will understand that any information received by a receiver or derived from a signal by a receiver, such as data types, structures, or items, is inserted into the signal by the corresponding provider, and vice versa.

[0224] Some or all of the method steps may be performed by (or using) a hardware device such as a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more method steps may be performed by such a device.

[0225] Depending on specific implementation requirements, embodiments of the present invention may be implemented in hardware or software, or at least partially in hardware or at least partially in software. Implementations may be carried out using digital storage media containing electronically readable control signals, such as floppy disks, DVDs, Blu-rays, CDs, ROMs, PROMs, EPROMs, EEPROMs, or flash memory, which cooperate (or can cooperate) with a programmable computer system to perform the respective methods. Therefore, the digital storage media may be computer-readable.

[0226] Some embodiments of the present invention include a data carrier having an electronically readable control signal that can cooperate with a programmable computer system so that one of the methods described herein can be performed. Generally, embodiments of the present invention can be implemented as a computer program product having program code, the program code behaves to perform one of the methods when the computer program product is executed on a computer. The program code may be stored, for example, in a machine-readable carrier. Other embodiments include a computer program stored in a machine-readable carrier for performing one of the methods described herein. In other words, therefore, one embodiment of the method of the present invention is a computer program having program code for performing one of the methods of the present invention when the computer program is executed on a computer.

[0227] Accordingly, a further embodiment of the method of the present invention is a data carrier (or digital storage medium, or computer-readable medium) containing a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-temporary. Therefore, a further embodiment of the method of the present invention is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals may be configured to be transmitted, for example, over a data communication connection, such as the Internet. Further embodiments include processing means configured or adapted to perform one of the methods described herein, such as a computer or a programmable logic device. Further embodiments include a computer on which a computer program for performing one of the methods described herein is installed.

[0228] Further embodiments of the present invention include an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0229] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the method herein. In some embodiments, a field-programmable gate array may cooperate with a microprocessor to perform one of the methods herein. Generally, the method is preferably performed by any hardware device. The apparatus described herein may be implemented using hardware devices, or using a computer, or using a combination of hardware devices and a computer. The methods described herein may be performed using hardware devices, or using a computer, or using a combination of hardware devices and a computer.

[0230] As can be seen in the detailed description above, various features are grouped together in the examples for the purpose of simplifying this disclosure. This method of disclosure should not be interpreted as reflecting an intention that the claimed examples require more features than are explicitly described in each claim. Rather, as reflected in the claims below, the subject matter may consist of fewer features than all the features of a single disclosed example. Thus, the claims below are incorporated into the detailed description, and each claim may stand alone as a separate example. While each claim may stand alone as a separate example, a dependent claim may refer to a specific combination of one or more other claims in the claim, but it should be noted that other examples may also include combinations of a dependent claim with the subject matter of another dependent claim, or combinations of each feature with other dependent or independent claims. Such combinations are proposed herein unless otherwise stated that a particular combination is not intended. Furthermore, it is also intended that the features of the claim may be included in any other independent claim, even if this claim is not directly dependent on an independent claim.

[0231] The embodiments described above are merely illustrative of the principles of the present disclosure. Modifications and variations of the configurations and details described herein will be apparent to those skilled in the art. Therefore, it is intended to be limited only by the pending claims and not by the descriptions of the embodiments herein and by the specific details presented herein.

Claims

1. A device (20) for decoding an audio data stream (14), wherein the audio data stream includes a plurality of payload packets (16) carrying payload data, and further includes supplemental information packets (18), the supplemental information packets carrying supplemental information messages (19), and the device, From the audio data stream, an instruction (51) is derived (21) indicating whether one or more supplemental information messages should be considered in order to verify the audio data stream (40). A predetermined portion (13) of the audio data stream is determined by including the one or more supplemental information messages in the predetermined portion (30) when the instruction (51) indicates that the one or more supplemental information messages should be considered in order to verify the audio data stream. Obtaining a digital signature (43) to verify the predetermined portion (13) from the instructions in the audio data stream (21) A device (20) configured to perform the following actions.

2. The apparatus according to claim 1, wherein the apparatus is configured to verify the predetermined portion of the audio data stream by checking whether the predetermined portion conforms to the digital signature.

3. The apparatus according to claim 1 or 2, wherein the apparatus is configured to provide the predetermined portion and the digital signature for verifying the predetermined portion.

4. Checking whether the predetermined portion (13) of the audio data stream conforms to the digital signature (43) is The predetermined portion (13) is subjected to a hash function to obtain a hash value, Checking whether the hash value conforms to the digital signature (43), The apparatus according to any one of claims 1 to 3, including the apparatus described above.

5. Checking whether the hash value conforms to the digital signature (43) is A verification string (48) is formed based on the aforementioned hash value, Deciphering the aforementioned digital signature (43) to obtain a check value, The verification string (48) is checked to see if it matches the check value, The apparatus according to claim 4, including the apparatus described in claim 4.

6. The predetermined portion (13) is determined by including one or more of the supplemental information packets in the predetermined portion (13) when the instruction indicates that one or more supplemental information messages should be considered in order to verify the predetermined portion (13). The apparatus according to any one of claims 1 to 5.

7. The apparatus according to any one of claims 1 to 6, wherein all supplemental information packets that carry any supplemental information messages to be included in the predetermined portion (13) are configured to be included in the predetermined portion (13).

8. The apparatus according to any one of claims 1 to 7, wherein the decision of whether or not to include one supplemental information packet from a plurality of supplemental information packets in the predetermined portion (13) is conditional on the result of checking whether the supplemental information packet contains a supplemental information message of any supplemental information message type from a set of supplemental information message types.

9. The instruction is a flag having a first state and a second state, and the device controls the predetermined portion (13) of the audio data stream, If the flag has the first state, one or more supplemental information messages to be included in a predetermined portion (13) are determined based on a predefined set of supplemental information messages among the supplemental information messages. If the flag has the second state, none of the supplementary information messages shall be included in the predetermined portion (13). The apparatus according to any one of claims 1 to 8, configured to be determined by

10. The apparatus according to claim 9, wherein the predefined set of supplemental information messages includes all supplemental information messages of the supplemental information packets that refer to the plurality of payload packets (16).

11. The apparatus according to claim 9, wherein the predefined set of supplemental information messages includes all supplemental information messages of any type from the set of types of supplemental information messages.

12. If the instruction indicates that one or more supplemental information messages should be considered in order to verify the audio data stream, the system is further configured to derive identification information of the one or more supplemental information messages that should be included in the predetermined portion (13) from the audio data stream. The apparatus according to any one of claims 1 to 11.

13. The apparatus according to claim 12, wherein, when the instruction indicates that one or more supplemental information messages should be considered in order to validate the audio data stream, the apparatus is configured to derive from the audio data stream syntactic elements that identify a set of types of supplemental information messages to be included in the predetermined portion (13).

14. The apparatus according to any one of claims 1 to 13, configured to derive from the audio data stream syntactic elements that indicate whether the one or more supplemental information messages should be considered for verification of the audio data stream and that signal identification information of the one or more supplemental information messages to be included in the predetermined portion (13).

15. The syntactic element has a plurality of first and second states, and the device, If the syntactic element has one of the first states, one or more supplementary information messages to be included in the predetermined part (13) are determined according to the state of the syntactic element, If the syntactic element has the second state, none of the one or more supplementary information messages are included in the predetermined part (13), The apparatus according to claim 14, configured to perform the following:

16. The apparatus according to claim 12, wherein, when the instruction indicates that one or more supplemental information messages should be considered in order to validate the audio data stream, the apparatus is configured to derive from the audio data stream a first syntactic element indicating the number of supplemental information message types to be considered for the predetermined portion (13), and a number of second syntactic elements, each identifying each of the supplemental information message types to be included in the predetermined portion (13).

17. An audio data stream encoding device (10) is configured to make the audio data stream verifiable in terms of reliability, and to encode a plurality of payload packets (16) that carry payload data within the audio data stream, further encoding supplemental information packets (18) within the audio data stream, wherein the supplemental information packets carry supplemental information messages, and the device is configured to do the following: Insert an instruction (51) into the audio data stream indicating whether one or more supplemental information messages should be considered in order to verify the audio data stream (23), Determine (30') a predetermined portion (13) of the audio data stream by including one or more supplemental information messages within the predetermined portion (13) when the instruction indicates that one or more supplemental information messages should be considered in order to verify the audio data stream, and that the predetermined portion (13) should be checkable with respect to reliability, Obtaining a digital signature (43) based on a predetermined portion (13) (41') A device (10) configured to perform the following actions.

18. A method for decoding an audio data stream (14), wherein the audio data stream includes a plurality of payload packets (16) carrying payload data, and further includes supplemental information packets (18), the supplemental information packets carrying supplemental information messages (19), and the method From the audio data stream, an instruction (51) is derived (21) indicating whether one or more supplemental information messages should be considered in order to verify the audio data stream (40). The predetermined portion (13) of the audio data stream is determined by including the one or more supplemental information messages in the predetermined portion when the instruction (51) indicates that the one or more supplemental information messages should be considered in order to verify the audio data stream (30). Obtaining a digital signature (43) to verify the predetermined portion (13) from the instructions in the audio data stream (21) Methods that include...

19. A method for encoding an audio data stream, the method comprising: making the audio data stream verifiable in terms of reliability; encoding a plurality of payload packets (16) carrying payload data within the audio data stream, further encoding supplemental information packets within the audio data stream, wherein the supplemental information packets carry supplemental information messages, and the method Insert an instruction (51) into the audio data stream indicating whether one or more supplemental information messages should be considered in order to verify the audio data stream. Determine (30') a predetermined portion (13) of the audio data stream by including one or more of the supplemental information messages in the predetermined portion (13) when the instruction indicates that one or more supplemental information messages should be considered in order to verify the audio data stream, and that the predetermined portion (13) should be checkable with respect to reliability, Obtaining a digital signature (43) based on the predetermined portion (13) (41') Methods that include...

20. An audio data stream having data signals encoded within the audio data stream, wherein the audio data stream is checkable in terms of reliability, and the audio data stream is A plurality of payload packets (16) that carry payload data, wherein the audio data stream further includes supplemental information packets, and the supplemental information packets carry supplemental information messages, Instructions indicating whether one or more supplemental information messages should be considered regarding the reliability of the audio data stream, An audio data stream containing [the specified element].

Images