Checking authenticity of media data stream

Abstract

To provide a good tradeoff between a security level, implementation effort, and signaling overhead by checking the authenticity of media data streams.SOLUTION: A method for checking the authenticity of a media data stream in which a media stream is encoded includes steps of: subjecting a predetermined portion of the media data stream or data derived therefrom to a hash function to obtain a hash value; deriving a digital signature from the media data stream; and checking whether the hash value matches the digital signature to determine whether the media data stream can be trusted.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Embodiments of the present invention relate, for example, to an apparatus for checking a media data stream for authenticity, an apparatus for rendering a media data stream in which a media stream that can be checked for authenticity is encoded, a method for checking a media data stream for authenticity, a method for rendering a media data stream in which a media stream that can be checked for authenticity is encoded. Further embodiments relate to a decoder for decoding a media stream, such as a video or audio stream, from a media data stream, and an encoder for encoding such a media data stream. Further embodiments relate to a media data stream, such as a video or audio data stream. [Background technology]

[0002] Today, the production, distribution, and consumption of video content and other media content, such as audio, play an important role in people's lives. International standards such as ITU-T Recommendations H.264, H.265, and H.266 enable the globally reliable and interoperable exchange of video content. They therefore form a key technology for modern, interconnected societies. For audio content, further standards are available, such as MPEG-AAC (Advanced Audio Coding). Recent rapid developments in artificial intelligence (AI) enable new methods of artificial media content generation, such as the generation of video and / or audio content. In this way, new data formats and innovative ways of user experience can be achieved. Summary of the Invention [Problem to be solved by the invention]

[0003] However, at the same time, AI-based methods also carry the risk of being used in deceptive and potentially harmful ways. An example of such misuse is deepfakes, which create false perceptions about the origin or authorship of multimedia content. This can lead to devastating consequences, such as infringement of copyright or personal rights, fraud through falsified evidence, or undermining public trust in the integrity of public institutions. Therefore, there is a need for a concept to check video data streams for authenticity and provide a good trade-off between security level, implementation effort, and signaling overhead. [Means for solving the problem]

[0004] This object is achieved by the subject matter of the independent claims. An embodiment of the present invention relies on the idea of ​​rendering a media data stream whose authenticity can be verified and whose authenticity can be checked by hashing a predetermined portion of the media data stream and calculating a digital signature based on the obtained hash value. For example, the media data stream may be a video data stream, an audio data stream, or a multimedia data stream containing video and audio. The digital signature is transmitted within the media data stream. The authenticity of the media data stream can then be checked by subjecting the predetermined portion of the media data stream to a hash function to obtain a hash value and checking the hash value against the digital signature. Basing the authenticity check on the predetermined portion allows for a flexible and robust design of the authenticity check, for example, by allowing the predetermined portion to include or exclude certain parts of the data stream, thus enabling the authenticity check, and / or by maintaining certain functionality, such as random access, by selecting the predetermined portion according to a randomly accessible portion of the data stream, e.g., video, coded video sequence, or CVS. Furthermore, hashing the predetermined portion provides the advantage of reducing the data size of the digital signature and keeping the signaling overhead for transmitting the digital signature low. For example, a basic principle underlying embodiments of the present invention is that of digitally signing a compressed video bitstream, such as a video bitstream or an audio bitstream.

[0005] One embodiment of the present invention provides an apparatus for checking the authenticity of a media data stream in which a media stream is encoded (e.g., a video data stream in which video is encoded, or an audio data stream in which an audio stream is encoded). The apparatus is configured to apply a predetermined portion of the media data stream, or data derived therefrom, to a hash function to obtain a hash value, derive a digital signature from the media data stream, and check whether the hash value matches the digital signature to determine whether the media data stream is authentic.

[0006] A further embodiment of the present invention provides a decoder for decoding a media stream from a media data stream. The decoder is configured to decode a digital signature from a video data stream. For example, the media data stream may be a video data stream in which video is encoded or an audio data stream in which an audio stream is encoded. The decoder is further configured to decode from the media data stream the digital signature to be subjected to a check regarding the authenticity of the media data stream by applying a hash function to a predetermined portion of the video data stream, or data derived therefrom, to obtain a hash value, and checking whether the hash value matches the digital signature to determine whether the media data stream can be trusted. A further embodiment of the present invention provides an apparatus for rendering a media data stream in which a media stream checkable for authenticity is encoded, the apparatus being configured to subject a predetermined portion of the media data stream, or data derived therefrom, to a hash function to obtain a hash value, calculate a digital signature based on the hash value to digitally sign the hash function, and insert the digital signature into the media data stream, thereby making it possible to determine whether the media data stream is authentic by checking whether the hash value matches the digital signature.

[0007] A further embodiment of the present invention provides a method for checking the authenticity of a media data stream in which the media stream is encoded, the method including subjecting a predetermined portion of the media data stream, or data derived therefrom, to a hash function to obtain a hash value, deriving a digital signature from the media data stream, and checking whether the hash value matches the digital signature to determine whether the media data stream can be trusted. A further embodiment of the present invention provides a method for decrypting a media stream from a media data stream, the method including decrypting from the media data stream a digital signature to be subjected to a check regarding the authenticity of the media data stream, the check including subjecting a predetermined portion of the media data stream, or data derived therefrom, to a hash function to obtain a hash value, and checking whether the hash value matches the digital signature to determine whether the media data stream can be trusted.

[0008] A further embodiment of the present invention provides a method for rendering a media data stream in which a media stream checkable for authenticity is encoded, the method including subjecting a predetermined portion of the media data stream, or data derived therefrom, to a hash function to obtain a hash value, calculating a digital signature based on the hash value to digitally sign the hash function, and inserting the digital signature into the media data stream, thereby making it possible to determine whether the media data stream is trustworthy by checking whether the hash value matches the digital signature. A further embodiment of the present invention provides a media data stream having an encoded media stream, the media data stream being rendered checkable for authenticity using the above method, in particular the media data stream including a digital signature for a predetermined portion of the media data stream.

[0009] For example, the above-mentioned video coding standards do not support any method by which a standard-compliant decoder can determine whether a standard-compliant compressed bitstream was actually generated by a trusted source or whether it was generated by someone merely falsely claiming to be such a source, for example, by using deepfakes. Because these standards may already be widely deployed in devices around the world, often with dedicated hardware supporting their efficient use, embodiments of the present invention provide a technical solution that can modify already deployed mechanisms for standard-compliant decoding of bitstreams to enable trusted data exchange without changing them. Embodiments of the present invention ensure robustness and flexibility against future developments in the field of security-related hashing and signature algorithms, and provide a method that easily and independently enables data transmission from content providers to content consumers based on a mutual understanding of trust between both parties. Embodiments follow any general design principles of the underlying specification texts of the corresponding standards to enable easy implementation and deployment of the proposed technology. Furthermore, embodiments provide solutions for trusted data exchange that can be used in combination in accordance with the core features and functionality of the underlying video coding standards when used in practical applications.

[0010] It should be noted that although the above considerations are illustratively described with respect to video data streams, the same or similar considerations apply to other types of media data streams, such as audio data streams. A further aspect of the present invention provides a concept for checking the authenticity of a multimedia data stream that includes multiple media data streams, e.g., referred to as media substreams of the multimedia data stream. An embodiment of this aspect relies on providing a common digital signature within the multimedia data stream that, in combination with hash values ​​derived from the individual media substreams, enables verification of the combination of substreams as well as the individual media substreams present within the multimedia data stream. In particular, the embodiment relies on including a hash value within the multimedia data stream for at least one or more removable substreams of the multiple media substreams. The transmission of the hash value enables verification of the data stream against the common digital signature even if one of the removable substreams is removed, e.g., in response to a substream selection by a client. For example, the transmitted hash value can be used to form a check value that is verified against the common digital signature.

[0011] One embodiment of the present invention provides an apparatus for checking the authenticity of a multimedia data stream, the apparatus being configured to derive substream information from the multimedia data stream indicating that the multimedia data stream is composed of multiple media substreams, each of which encodes a media signal. The apparatus is further configured to, for each of a set of one or more first media substreams included in the multimedia data stream, subject a first predetermined portion of each first media substream, or first data derived therefrom, to a separate first hash function to obtain a separate calculated hash value. The apparatus is configured to derive signature information from the multimedia data stream, including a common digital signature, and derive a separate transmitted hash value for each of one or more removable media substreams of the multiple media substreams. The apparatus is configured to check whether the set of one or more first media substreams is authentic based on the common digital signature and further based on the separate calculated hash value for each of the set of one or more first media substreams and further based on the separate transmitted hash value for at least one of the one or more removable media substreams.

[0012] A further embodiment of the present invention provides a decoder for decoding a multimedia data stream and checking the authenticity of the multimedia data stream. The decoder is configured to decode substream information from the multimedia data stream indicating that the multimedia data stream is composed of multiple media substreams, each of which has a media signal encoded therein. The decoder is also configured to decode signature information from the multimedia data stream, the signature information including a common digital signature and further including, for each of one or more removable media substreams of the multiple media substreams, a separate transmitted hash value. The decoder is further configured to subject the common digital signature to an authenticity check of a first set of one or more first media substreams. The check includes, for each of the set of one or more first media substreams included in the multimedia data stream, subjecting a first predetermined portion of the respective first media substream, or first data derived therefrom, to a separate first hash function to obtain a separate calculated hash value. The check further includes performing a check based on the common digital signature and further based on an individual calculated hash value for each of the set of one or more first media substreams and further based on an individual transmitted hash value for at least one of the one or more removable media substreams.

[0013] A further embodiment of the present invention provides an apparatus for rendering a multimedia data stream that can be checked for authenticity. The apparatus is configured to insert substream information into the multimedia data stream, indicating that the multimedia data stream is composed of multiple media substreams, each of which has a media signal encoded therein. The apparatus is further configured to, for each of the multiple media substreams, subject a first predetermined portion of the individual media substream, or first data derived therefrom, to a respective first hash function to obtain a respective calculated hash value. The apparatus is further configured to encode signature information into the multimedia data stream. The signature information includes a common digital signature and, for each of one or more removable media substreams of the multiple media substreams, a respective transmitted hash value equal to the calculated hash value of the respective removable media signal. The apparatus is further configured to digitally sign the multimedia data stream by calculating a common digital signature based on the respective calculated hash value for each of the multiple media substreams.

[0014] A further embodiment of the present invention provides a method for checking authenticity of a multimedia data stream, the method including: deriving, from the multimedia data stream, substream information indicating that the multimedia data stream is composed of a plurality of media substreams, each of which has a media signal encoded therein; for each of a set of one or more first media substreams included in the multimedia data stream, subjecting a first predetermined portion of each first media substream, or first data derived therefrom, to a respective first hash function to obtain a respective calculated hash value; deriving, from the multimedia data stream, signature information including a common digital signature and, for each of one or more removable media substreams of the plurality of media substreams, a respective transmitted hash value; and checking whether the set of one or more first media substreams is authentic based on the common digital signature, the respective calculated hash value for each of the set of one or more first media substreams, and the respective transmitted hash value for at least one of the one or more removable media substreams. A further embodiment of the present invention provides a method for decoding a multimedia data stream and checking the multimedia data stream for authenticity, the method comprising: decoding, from the multimedia data stream, substream information indicating that the multimedia data stream is made up of a plurality of media substreams, each of which has a media signal encoded therein; decoding, from the multimedia data stream, signature information including a common digital signature and, for each of one or more removable media substreams of the plurality of media substreams, an individual transmitted hash value; and decoding, from the multimedia data stream, signature information including a common digital signature and, for each of one or more removable media substreams of the plurality of media substreams, an individual transmitted hash value. and subjecting the common digital signature to an authenticity check of the first set of one or more first media substreams, the authenticity check comprising, for each of the set of one or more first media substreams included in the system, subjecting a first predetermined portion of the respective first media substream, or first data derived therefrom, to a respective first hash function to obtain a respective calculated hash value; and performing a check based on the common digital signature, the respective calculated hash value for each of the set of one or more first media substreams, and the respective transmitted hash value for at least one of the one or more removable media substreams.

[0015] A further embodiment of the present invention provides a method for rendering a multimedia data stream that can be checked for authenticity, the method comprising: inserting into the multimedia data stream substream information indicating that the multimedia data stream is composed of a plurality of media substreams, each having a media signal encoded therein; for each of the plurality of media substreams, subjecting a first predetermined portion of the respective media substream, or first data derived therefrom, to a respective first hash function to obtain a respective calculated hash value; encoding into the multimedia data stream signature information including a common digital signature and, for each of one or more removable media substreams of the plurality of media substreams, a respective transmitted hash value equal to the calculated hash value of the respective removable media signal; and digitally signing the multimedia data stream by calculating a common digital signature based on the respective calculated hash value for each of the plurality of media substreams. Advantageous implementations are defined by the subject matter of the dependent claims. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will now be described with reference to the accompanying drawings. [Brief explanation of the drawings]

[0016] [Figure 1] 1 illustrates an apparatus for checking a data stream for reliability, according to one embodiment. [Figure 2] 1 illustrates a decoder according to one embodiment. [Figure 3] 1 illustrates a verification module, according to one embodiment. [Figure 4] 1 illustrates an apparatus for rendering a data stream that can be checked for authenticity, according to one embodiment. [Figure 5] 1 shows a schematic diagram of a trust verification process according to one embodiment. [Figure 6A] 1 illustrates data selection for a signature process, according to one embodiment. [Figure 6B]1 illustrates content hashing for a signature process, according to one embodiment. [Figure 6C] 1 illustrates a signature generation of a signature process according to one embodiment. [Figure 7] 1 illustrates a signature verification of the verification process according to one embodiment. [Figure 8] 1 illustrates the formation of a hash value, according to one embodiment. [Figure 9A] 1 illustrates a method for locating chunks according to an embodiment. [Figure 9B] 1 illustrates a method for locating chunks according to an embodiment. [Figure 9C] 1 illustrates a method for locating chunks according to an embodiment. [Figure 10A] 1 illustrates chunk-structured packet selection (eg, H.264) via markers into a single chunk using chunk-structured GOPs, according to one embodiment. [Figure 10B] 1 illustrates chunk-structured packet selection (e.g., H.265 / H.266) via markers into a single chunk using chunk-structured GOPs, according to one embodiment. [Figure 11A] 1 illustrates chunk-structured packet selection (eg, H.264) via markers using chunk-structured GOPs that divide a temporal layer into two chunks, according to one embodiment. [Figure 11B] 1 illustrates chunked packet selection via markers (e.g., H.265 / H.266) using chunked GOPs that divide a temporal layer into two chunks, according to one embodiment. [Figure 12] 1 illustrates chunk organization into chunks and recommended hash dependencies and recommended chunk dependencies in two-layer protection with bitstream packets, according to one embodiment. [Figure 13] 1 illustrates possible configurations of an identification string IdString according to one embodiment. [Figure 14] 1 illustrates an apparatus for predictively coding pictures into a data stream, illustratively using transform-based residual coding, according to one embodiment. [Figure 15]10 illustrates a corresponding decoder configured to predictively decode pictures from a data stream, also using transform-based residual decoding, according to one embodiment. [Figure 16] 1 illustrates the relationship between a reconstructed signal and a combination of a prediction residual signal and a prediction signal signaled in a data stream according to one embodiment. [Figure 17] 1 illustrates a method for checking a data stream for reliability, according to one embodiment. [Figure 18] 1 illustrates a method for decoding a media stream according to one embodiment. [Figure 19] 1 illustrates a method for rendering a data stream that can be checked for authenticity, according to one embodiment. [Figure 20] 1 illustrates a joint verification scheme for multiple media data streams, according to one embodiment. [Figure 21] 1 illustrates a temporal hash value pair secured by a joint signature, according to one embodiment. [Figure 22] 1 illustrates an apparatus for checking a multimedia data stream for authenticity, according to one embodiment. [Figure 23] 1 illustrates an apparatus for rendering a multimedia data stream that can be checked for authenticity, according to one embodiment. [Figure 24] 1 illustrates a method for checking a multimedia data stream for authenticity, according to one embodiment. [Figure 25] 1 illustrates a method for decoding a multimedia data stream according to one embodiment. [Figure 26] 1 illustrates a method for rendering a multimedia data stream that can be checked for authenticity, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Before describing the embodiments of the present invention below based on the accompanying drawings, it should be noted that elements and structures having the same effect are given the same reference numerals so that their descriptions can be applied to each other or can be exchanged. Moreover, the features of different embodiments described herein can be combined with each other unless otherwise specified.

[0018] FIG. 1 illustrates an apparatus 16 for checking the authenticity of a data stream 14 according to one embodiment. For example, the data stream 14 may contain video encoded therein, e.g., the data stream 14 may be a video data stream. In another example, the data stream 14 may contain audio encoded therein, e.g., the data stream 14 may be an audio data stream. The apparatus 16 subjects a predetermined portion 13 of the data stream 14 to a hash function 31 to obtain a hash value 33, or alternatively, the apparatus 16 subjects data 62 derived from the predetermined portion 13 to the hash function 31 to obtain the hash value 33. The latter option is exemplarily visualized in FIG. 1 by an optional block 61 that can derive data 62 to be subjected to the hash function 31 from the predetermined portion 13. The apparatus 16 derives a digital signature 43 from the data stream 14. Furthermore, the apparatus 16 includes a verification block 41 that checks whether the hash value 33 matches the digital signature 43 to determine whether the data stream is authentic. For example, if the digital signature 43 matches the hash value 33, the data stream, or a given portion thereof, is considered to be authentic. For example, authenticity may mean that the content and / or content provider of a data stream or a given portion thereof is successfully verified as authentic.

[0019] 2 shows a decoder 20 for decoding a media stream 11 from a data stream 14 according to one embodiment. The decoder 20 comprises a decryption module 63 for decrypting a digital signature 43, for example a digital signature 43 as described with reference to FIG. 1, from the data stream 14. In other words, the digital signature 43 is intended to subject the data stream 14 to a check on the authenticity of the data stream 14 performed by a device 16. Thus, for example, the decoder 20 may provide the digital signature 43 to the device 16. The device 16 may optionally be part of the decoder 20, or alternatively, the device 16 may be a separate entity. Thus, the device 16 may be independent from the decoder 20, or vice versa. Optionally, decoder 20 may further provide device 16 with predetermined portion 13, or alternatively data derived therefrom, for example data 62 as described with respect to device 16. In other words, some functionality described with respect to device 16, such as the location of predetermined portion 13, may optionally be performed by decoder 20. In other words, according to one embodiment, device 16 of Figure 1 may be a decoder, for example as shown in Figure 2. According to an alternative embodiment, device 16 may receive digital signature 43 and predetermined portion 13, for example from decoder 10.

[0020] According to one embodiment, the decoding module 63 decodes the media stream 11 from the data stream 14 . According to an embodiment in which the data stream is a video data stream, the decoder 20 reconstructs the video 11 with respect to the predetermined portion 13 to obtain a reconstructed portion of the video. In other words, for example, the decoder 20 reconstructs the portion of the video represented by the predetermined portion 13 to obtain a reconstructed portion of the video. According to this embodiment, the decoder 20 subjects the reconstructed portion to a hash function 31. For example, the reconstruction can be represented by block 61 in FIG. 2, in which case blocks 61 and 63 can be combined. In other words, the data 62 derived from the predetermined portion 13 can be the reconstructed portion of the video or data derived therefrom.

[0021] Further details of device 16 are provided below, but these details may optionally also be applied to decoder 20 of FIG. For example, the predetermined portion 33 may be a contiguous portion of the data stream 14. Alternatively, the predetermined portion may be made up of multiple subsections or sections of the data stream that may be interspersed with or interspersed with additional portions of the data stream that are not part of the predetermined portion. For example, the predetermined portion 13 may be referred to as a chunk. In subjecting the predetermined portion 13 to the hash function 31, the device 16 may subject the predetermined portion to the hash function 31 in the form of raw data from the data stream. This may mean, for example, that data is parsed from the data stream 14 and subjected to the hash function without further decoding.

[0022] According to one embodiment, the hash value 33 depends on all bits of the predetermined portion 13 of the data stream. According to one embodiment, the hash value 33 depends on all bits of a given portion of the data stream in the entropy coded domain. According to an embodiment in which the data stream is a video data stream, the predetermined portion 13 of the data stream 14 spans more than one access unit of the data stream. The hash value depends on bits of more than one access unit. For example, an access unit can refer to a portion of the data stream in which one, e.g., exactly one time frame of video is encoded. According to an embodiment in which the data stream is a video data stream, the predetermined portion 33 consists of one or more video coding layer portions of the data stream in which motion vectors and intra prediction modes for prediction blocks and transform coefficients for residual blocks are coded.

[0023] 3 shows a verification module 41 according to one exemplary embodiment. According to this embodiment, the verification module 41 comprises a decryption block 45 that decrypts the digital signature 43 to obtain a check value 47, and the verification module 41 further comprises a verification block 49 that checks whether the hash value 33 matches, e.g., whether it matches, the check value 47. For example, the generation of the digital signature 43 may be performed on the encoder side by forming a check value and signing it using a private key. For example, the signature may include a further hashing, i.e. hashing the check value using a further hash function to obtain a further hash value and signing the further hash value. In this example, it may not be possible to reconstruct the check value from the digital signature 43 on the decoder side, but instead it can only be checked if the check value formed using the hash value 33 matches the digital signature. In this case, verification by the verification module 41 may include hashing the check value using a further hash function to obtain a further hash value and checking whether the further hash value matches the digital signature, for example by decrypting the digital signature using the public key and checking whether the resulting decrypted further hash value is equal to the further hash value.

[0024] In other words, according to one embodiment, checking whether the hash value 33 matches or matches the check value 47 may involve forming a verification string using the hash value 33, e.g., by concatenating the hash value 33 with further information, such as a further hash value or a hash function identifier, as described below, and hashing the verification string, e.g., using the further hash function. The verification module 41 may then check whether the hashed verification string is equal to the check value 47 decrypted from the digital signature. On the encoder side, according to this embodiment, the digital signature may be generated by forming a verification string in the same way as on the decoder side, hashing it using the further hash function, and signing the hashed verification string to obtain the digital signature 43.

[0025] According to alternative embodiments, check value 47 may correspond to hash value 33 or may correspond to the concatenation of hash value 33 with further information, such as a further hash value or a hash function identifier. In other words, decryption of the digital signature in this case may yield hash value 33 as part of check value 47 (or the entire check value 47). In this case, the digital signature may be larger due to the omission of further hashing. For example, whether one or the other of the above alternatives is used may depend on the hash function selected.

[0026] For example, verification block 49 may check whether hash value 33 is equal to check value 47 or a portion thereof to determine whether hash value 33 matches check value 47. In other words, from another perspective, check value 47 may be the entire value or a portion thereof obtained from decrypting the digital signature, and verification block 49 may check whether hash value 33 matches or is equal to the check value.

[0027] In other words, what it means for a digital signature to match is, for example, according to one embodiment, a digital signature is matched by a predetermined value, e.g., a hash value, if the predetermined value and a check value obtained by decrypting digital signature 42 are equal. Alternatively, digital signature 42 is matched by a predetermined value, e.g., a hash value, if the predetermined value and a predetermined portion of the check value associated with the predetermined value are equal. According to one embodiment, the verification module 41 performs the check by using an asymmetric decryption scheme using a public key.

[0028] FIG. 4 illustrates an apparatus 15 according to one embodiment for rendering a data stream 14, e.g., a video data stream 14 in which a video that can be checked for authenticity is encoded, or an audio data stream in which an audio stream that can be checked for authenticity is encoded. The apparatus 15 is configured to apply a predetermined portion 13 of the data stream, or data 62 derived therefrom, to a hash function 31 to obtain a hash value 33. In this respect, the description of the apparatus 16 in FIG. 1 also applies, e.g., with respect to optional block 61. The apparatus 15 comprises a signature module 71 configured to calculate a digital signature 43 based on the hash value 33 and digitally sign the hash value 33. The apparatus 15 further comprises an insertion unit 77 configured to insert the digital signature 43 into the data stream 14, thereby enabling a determination of whether the data stream is authentic by checking whether the hash value 33 matches the digital signature 43.

[0029] It should be noted that any description of device 16 may optionally equally apply to device 15, in the sense that any information derived by device 16 from data stream 14 may be inserted by device 15 into data stream 14. For example, with respect to a description of how device 16 uses supplemental information messages as markers to locate predetermined portion 13, device 15 may insert these messages into data stream 14 accordingly.

[0030] According to one embodiment, in which the data stream is a video data stream, the device 15 reconstructs the video 11 with respect to the predetermined portion 13 to obtain a reconstructed portion of the video. In other words, for example, the decoder 20 reconstructs the portion of the video represented by the predetermined portion 13 to obtain a reconstructed portion of the video. According to this embodiment, the encoder 20 subjects the reconstructed portion to a hash function 31. According to one embodiment, device 15 is an encoder configured to encode a media stream into a data stream 14 and to encode a digital signature into data stream 14 .

[0031] According to one embodiment, signing module 71 calculates digital signature 43 based on hash value 33 by forming check value 47 based on hash value 33 and encrypting check value 47 to obtain digital signature 43. In other words, signing module 71 may be the counterpart of verification module 41 of FIG. For example, in encrypting check value 47, signing module 71 may use the private key of a public-private key pair of an asymmetric cryptography scheme, or may use the private key and the public key, and verification block 49 may use the public key in decrypting the digital signature.

[0032] In the following, further aspects, details and features of embodiments are described which may optionally be implemented individually or in any combination in any of the above-described embodiments of device 15 and device 16. According to one embodiment, the device 16 is configured to provide a hash value for subjecting the data stream combined with the media stream to an authenticity check, together with a further hash value obtained by subjecting a portion of the media stream accompanying the data stream, or further data derived therefrom, to a hash function or a different hash function.

[0033] In other words, the actual check, e.g., hashing and / or verification of the hash value against the digital signature, may be performed by yet another entity, rather than by device 16 or decoder 20 itself. For example, device 16 may provide either predetermined portion 13 or hash value 33, along with the digital signature and, optionally, further information obtained from data stream 14 for verification, e.g., any of the information described as derived from a supplemental information message as disclosed herein, to another entity, which may then perform the check.

[0034] FIG. 5 shows a schematic diagram of the authenticity verification process according to one embodiment. According to this embodiment, the encoder, e.g., device 15, possesses a private (and public) key for a fixed signature algorithm. The decoder, e.g., decoder 20, possesses only the public key, not the private key. According to one embodiment, the bitstream, e.g., data stream 14, itself may contain a pointer to the public key, but the public key itself must be obtained by invoking a trusted, independent method, rather than being obtained from the bitstream alone. For example, the bitstream may contain information identifying the encoder as a particular entity or as belonging to a particular entity. Given this information, the decoder may obtain the public key corresponding to this entity from a third-party trust center (e.g., by downloading it from a trusted, secure URL, such as a PKI). Furthermore, a fixed cryptographic hash function is agreed upon by the encoder and decoder. Here, the hash function may be pre-fixed or obtained from the bitstream and third-party trust center in the same way as the public key, or obtained from the bitstream, in which case the hash function may be part of the digital signature. Next, a unique byte range is determined from the bitstream for which a cryptographic hash value will be calculated using a given hash function by the decoder. Finally, a digital signature is transmitted in the bitstream that can be considered an alleged digital signature of the hash value calculated for the given byte range. To check authenticity, the decoder then processes this digital signature using a given public key. If the result of this processing is found to be a digital signature of the calculated hash value of the byte range, the decoder can consider this byte range as information that reliably belongs to the entity associated with the given public key, but should consider it fake if the opposite is true.

[0035] Below, individual aspects of embodiments of the disclosed concepts are described, the details of which features may be optionally combined with any of the embodiments described with respect to FIGS. First, a signature process that may be implemented by example device 15, according to one embodiment, will be described with respect to Figures 6A-6C.

[0036] 6A illustrates chunk construction according to one embodiment. For example, device 15 may construct a given portion 13 from multiple portions or sections of data stream 14. For example, as shown in FIG. 6A, chunk 13 may be part of multiple chunks, and given portion 13 may be referred to as the current chunk. For example, an order may be defined between the chunks, in other words the chunks may form a sequence, and a chunk 13 may have a preceding chunk 13' and / or a succeeding chunk 13''. For example, each of the chunks may be checkable for authenticity, e.g., data stream 14 may include an individual digital signature for each chunk, in other words, data stream 14 may be checkable for authenticity on a chunk-by-chunk basis.

[0037] Figure 6B illustrates hashing of chunks according to one embodiment. As shown in Figure 6B, chunk 13 may be subjected to a hash function 31, e.g., referred to as a hashing core, to derive a hash value 33. Optionally, the hash function may be selected using an index, e.g., HIdx, as described in more detail below.

[0038] 6C illustrates signature generation (71) according to one embodiment, as may optionally be performed, for example, by signing module 71 of FIG. 4. According to this embodiment, signing module 71 forms check value 47 by concatenating previous hash value 33′ of previous chunk 13′ with hash value 33 of current chunk 13 and hash function identifier 37 indicating the hash function used to derive hash value 33. Optionally, if previous chunk 13′ does not exist, for example, if predetermined portion 13 is the initial (first) portion of an encoded media sequence, e.g., an encoded video sequence, previous hash value 33′ may take a predetermined value, for example, as described in more detail below. According to the embodiment of FIG. 6C, the signing module comprises a signing block 73 that signs check value 47 using the private key of a public-private key pair of an asymmetric cryptography scheme to derive digital signature 43.

[0039] The following describes a verification process that may be implemented by example device 16, according to one embodiment. For example, for verification, device 16 can implement the chunk configuration of Figure 6A. For example, device 16 can derive the configuration of chunks 13 from the data stream. In other words, data stream 14 can include instructions or information that reveal the configuration of chunks, as described in more detail below. Additionally, device 16 may perform hashing of the chunks of Figure 6B. For example, an indication of the hash function may be transmitted within data stream 14.

[0040] 7 illustrates a signature verification (41) according to one embodiment, which may be optionally performed, for example, by the verification module 41. According to this embodiment, the decryption block 45 decrypts the digital signature 43 derived from the data stream 14 using the public key 79 of the public-private key pair of the asymmetric cryptography to derive the check value 47. According to this embodiment, the check value 47 includes a first portion 53, a second portion 53′, and, optionally, an instruction 37 for a hash function. The first portion 53 is related to the hash value 33 of the chunk 13, and the second portion 53′ is related to the hash value 33′ of the previous chunk 13′, for example, in that the related values ​​are checked against each other to verify authenticity. According to this embodiment, upon reviewing the block 51, the verification block 49 checks whether the first portion 53 is equal to the hash value 33. For example, if they are equal, it can be concluded that the given portion is authentic. Additionally, looking at block 51', verification block 49 checks whether second portion 53' is equal to hash value 33'. For example, if they are equal, it can be concluded that no additional data has been inserted between chunks 13', 13. Additionally, looking at block 51'', verification block 49 checks whether the hash function index is equal to the hash function index used to derive the hash value. For example, if they are equal, it can be ensured that the verification scheme has not been circumvented, as will be explained in more detail below.

[0041] Note that in other embodiments, fewer or additional pieces of information may be part of the check value. More generally, the verification process may include an identifier for the hash function, as described with respect to Figures 6C and 7. For example, the identifier may indicate the type and / or parameterization of the hash function 31. In other words, according to one embodiment, the verification module 41 is configured to further check the authenticity of the data stream by checking whether the hash function is correct by checking whether the parameterization or identifier of the hash function matches the digital signature.

[0042] Below, individual aspects of embodiments of the disclosed concepts are described, the details of which features may be optionally combined with any of the embodiments described with respect to FIGS. <Robustness against future security risks by enabling assignment of digital signature methods and hash functions via a trust center> The ability of the above process to guarantee authenticity depends on the following properties of the signature algorithm and hash algorithm:

[0043] First, it must be impossible to compute the private key given only the public key, since anyone in possession of the private key can generate video content that is reliably considered to belong to the entity associated with a given public key. Second, the cryptographic hash function used must be robust to collision attacks, since otherwise the byte ranges above could be manipulated while still matching the encoded and digitally signed hash value.

[0044] It has been observed that efforts to break the security of signature algorithms or hash functions are constantly underway worldwide, while at the same time new algorithms are being proposed that can be considered secure, at least for a certain time horizon. To take this observation into account, one aspect of the present invention is to propose that the signature algorithm and / or hash function be signaled within the bitstream and obtained from an external trust center by using an index that points to a specific entity to which content should be reliably assigned. The latter entity can then flexibly upgrade the signature algorithm and / or hash function to be used by decoders that attempt to verify that a given bitstream certainly belongs to it. For a corresponding security upgrade of the bitstream, the content providing entity only needs to update the digital signature of the hash value, leaving all other parts of the bitstream unchanged. More generally, and referring to the description of device 16 with respect to Figures 1-7, according to one embodiment, device 16 derives an asymmetric decryption scheme using first information derived from data stream 14.

[0045] According to one embodiment, the first information includes a decryption scheme identifier, for example, an identifier that identifies the decryption scheme. According to an alternative embodiment, the first information includes a first pointer to a first location, e.g., a network location, from which the asymmetric decryption scheme can be determined. For example, the first information may be a URI that points to a network location from which the certificate can be obtained. According to a further alternative embodiment, the first information includes an identifier of an entity that encoded the video into the data stream. According to one embodiment, device 16 derives the public key using second information derived from the data stream.

[0046] According to one embodiment, the second information includes a second pointer to a second location from which the public can be obtained. For example, the second information may include, for example, an identifier of a public key among a plurality of keys derivable from a network location, e.g., the network location to which the pointer of the first information points. According to an alternative embodiment, the second information includes an identifier of the entity that encoded the video into the data stream. According to one embodiment, device 16 is configured to derive a public key by deriving from data stream 14 a first syntax element, e.g., twci_use_key_register_idx_flag, and a second syntax element, e.g., twci_key_source_uri, where the second syntax element indicates a pointer to a location, e.g., a URI, from which the public key can be derived. According to this embodiment, device 16 is configured to infer, if the first syntax element has a first state, that the location identifies exactly one public key and derive that exactly one public key from the location, and, if the first syntax element has a second state, infer that the location indicates a list of keys, derive from the data stream a third syntax element, e.g., twci_key_register_idx, that indicates a pointer to an entry in the list of keys, and derive the public key from the entry in the list of keys indicated by the pointer.

[0047] For example, the location to which the pointer points may hold one or more certificates. For example, a certificate may contain one or more keys. A certificate may additionally indicate one or more decryption schemes, e.g., one or more hash functions, for one or more keys. For example, a certificate may optionally be associated with a particular content provider.

[0048] According to one embodiment, the location pointed to by the pointer holds exactly one certificate containing one key, so that the information in the second syntax element may be sufficient to obtain the public key. According to another embodiment, the location pointed to by the pointer holds a certificate containing one or more keys, in which case the third syntax element can point to a key within the certificate that will be used for the trust check. According to another embodiment, the location pointed to by the pointer holds one or more certificates, each containing exactly one key. In this case, the third syntax element may point to a certificate from which the public key and optionally the decryption scheme will be derived. A combination with the previous embodiment is also possible, in that the third syntax element indicates one certificate of the one or more certificates and, within the indicated certificate, one key of the one or more keys of the indicated certificate.

[0049] According to one embodiment, device 16 derives hash function 31 using third information derived from the data stream. According to one embodiment, the third information includes a hash function identifier or a third pointer to a third location from which the hash function can be determined. According to an alternative embodiment, the third information includes an identifier of the entity that encoded the video into the data stream.

[0050] <Robustness against future collision attacks on hash functions by digitally signing supported hash functions> As another solution to the problem of potentially successful future collision attacks against hash functions, it has also been proposed that the bitstream may contain an index to a particular hash function; for security reasons, this index is also part of the message whose digital signature is part of the bitstream, and can therefore also be reliably verified. To check authenticity, the decoder then performs a joint signature verification of the received digital signature against the pair of hash value and hash function index. An embodiment utilizing joint signature verification is described above with respect to FIGS. 6C and 7.

[0051] <Chunk-level digital signatures for efficient compression of video content> While the above method allows for the reliable exchange of digital video content using widely deployed video coding standards, it also imposes an additional rate burden on the bitstream to which it is applied, since additional information, in particular the digital signature, must be transmitted. For example, if the digital signature is based on the RSA-2048 algorithm, the digital signature requires a size of 400 bytes within the bitstream.

[0052] Therefore, instead of transmitting a new digital signature for each sub-portion of a bitstream belonging to, for example, one coded slice or picture (e.g., a video coding layer NAL unit), it may be more rate-efficient to transmit a single digital signature in which the authenticity of several coded slices or pictures is jointly guaranteed by the above method. However, in the core functionality of widely deployed video coding standards such as H.264 or H.265, for a single bitstream, different decoders may only have access to a portion of the coded slices or pictures of the bitstream. Then, if only one hash value calculated for the entire bitstream is sufficient to verify the authenticity, any decoder that does not have access to all coded slices or pictures is excluded from applying the above method for authenticity verification. As a further observation, it should be noted that, on the one hand, there are generally a great many ways in which a subset of the set of all coded slices or pictures can be combined into a sub-bitstream by a decoder, and, on the other hand, there may be an efficient way for an encoder to determine, based on a given content and a given total bitstream, which subset can be expected to occur in a given situation. Therefore, for the purpose of authenticity verification by the above method, a flexible syntax is proposed that allows flexible assignment of coded slices or pictures to one of a possibly multiple set of coded slices or pictures. In the present application, such a set of coded slices or pictures for which joint authenticity verification can be performed by the above method is called a chunk of data. In the current setting, such a chunk of data is therefore characterized by the property that all of its bytes are used to calculate a hash value, and for this chunk a digitally signed value is transmitted that must be verified against the calculated hash value using a public key for authenticity verification.

[0053] <Bitstream integrity across multiple chunks> To ensure that the decoder can verify the continuity between single chunks used and to prevent attacks by removing, inserting, or shuffling data chunks in the protected bitstream, the present invention proposes that the hash value of the previous chunk be incorporated into the digital signature of the current chunk, as illustrated in Figures 6C and 7. Thus, for the current chunk, a digital signature of the combination of the hash value of the current chunk and the claimed hash value of the previous chunk must be part of the bitstream. Then, to verify the authenticity of the integrity of the current chunk with the previous chunk, the calculated hash values ​​of the previous chunk and the current chunk are jointly verified against this digital signature using a public key. Note that the hash value of the current chunk and the hash value of the previous chunk must be jointly signed with the current chunk so that only an entity that owns the private key can generate their co-occurrence as the digital signature of the current chunk.

[0054] A very important application scenario for such a guarantee of temporal integrity is the fact that video bitstreams made trustworthy by the above method should also enable random access capability for their decoding. However, if a bitstream is already considered trustworthy simply by checking the temporal subsegments between random access points individually and independently of each other, fake content can easily be generated by patching together temporal subsegments from different bitstreams originally generated from completely different video sequences. Note that the proposed solution, which guarantees the joint authenticity of multiple chunks or random access segments, still guarantees random access capability. A decoder that switches to decoding at a given random access segment but ignores earlier parts of the bitstream can simply ignore the hash values ​​identifying these earlier parts of the bitstream and focus only on verifying that part of the received bitstream.

[0055] More generally, described in this section and with reference to Figures 6C and 7 are embodiments in which checks on the authenticity of data stream 14 can be performed sequentially on multiple portions or chunks of data stream 14. In the following, generalized embodiments of the present disclosure are described. For example, one portion of the plurality of portions may be a portion of a subsequent segment of data stream 14. In other words, two portions of the plurality of portions that are checked sequentially may be portions from subsequent segments of the data stream, as shown, for example, in Figure 6A. Additionally or alternatively, portions of the plurality of portions may be portions from the same temporal segment of data stream 14, i.e., sub-portions of portions may be interleaved within the data stream, as described, for example, with respect to Figures 11A and 11B.

[0056] In other words, each portion may be associated with one of one or more sub-streams on which reliability checks may be performed, as described below. For example, the sequential check may be performed by sequentially checking subsequent segments of data stream 14, and within a segment, sequentially checking substreams according to a defined order between the substreams. For example, segments of data stream 14 may be defined or depicted as described below with respect to FIGS. 9A-10B. Thus, for example, in the following embodiments, the previous portion 13' may be understood as the previous portion in the sequential check, whether it is a portion of a lower-ranked substream of the same segment of the data stream 14 or a portion of a previous segment of the data stream 14.

[0057] According to one embodiment, the verification module 41 checks whether the hash value 33 and a further hash value 33' obtained by applying the hash function 31 to a previous portion 13' of the data stream or further data derived therefrom, for example as described with respect to Figures 6C and 7, matches the digital signature 43. For example, the further data may be derived from the previous portion 13', as described with respect to the given portion 13. According to this embodiment, the hash value 33 and the further hash value 33' may be checked against respective relevant portions, eg substrings, of the check value 47, eg as described with respect to FIG.

[0058] Alternatively, hash value 33 and further hash value 33' may be combined, e.g., concatenated, and the resulting combined hash value may be checked against check value 47, or a portion thereof, e.g., a substring. In other words, according to one embodiment, device 16 may combine hash value 33 and further hash value 33' to obtain a combined hash value and check whether the combined hash value matches digital signature 43.

[0059] 8 shows an alternative embodiment for forming a hash value to be used as input to the verification module 41. According to this embodiment, the device 16 comprises a combiner 82 that forms a combination 85, e.g., a concatenation, of the predetermined portion 13 and the previous hash value 33′. This combination 85 is subjected to a hash function 31, resulting in a combined hash value 33* that is then verified by the verification module 41. For example, according to this embodiment, the verification module 41 can check the combined hash value 33* against a combined check value, e.g., a value or string obtained from decrypting (45) the signature 43, or a portion thereof, e.g., whether the combined hash value 33* is equal to the value or string obtained from decrypting (45) the signature 43, or a portion thereof. In this alternative embodiment, instead of the predetermined portion 13, data 62 derived therefrom, as described with reference to FIG. 1, can be used as input to the combiner 82.

[0060] More generally, according to one embodiment, the verification module 41 checks whether a combined hash value 33* derived by hashing the predetermined portion 13, or data 62 derived therefrom, and a further hash value 33' obtained by subjecting a previous portion 13' of the data stream, or further data derived therefrom, to a hash function 31, matches the digital signature 43. In other words, according to this embodiment, the hash value obtained for the previous portion may be combined, e.g. concatenated, with the predetermined portion 13, or data 62 derived therefrom, and the obtained combination may be subjected to the hash function 31 to derive a hash value for the predetermined portion.

[0061] According to one embodiment, the device 16 can combine a predetermined portion 13 of the data stream 14 with a further hash value 33' obtained by applying a hash function 31 to the previous portion 13' or further data derived therefrom to obtain a combined hash value and check whether the combined hash value matches the digital signature. It should be noted here again that the method for deriving the hash value to be subjected to the verification module 41 may be performed in the same way on the encoder side and on the decoder side, for example in the device 15 and the device 16.

[0062] The following description discloses an embodiment where the data stream 14 does not include any portion before the predetermined portion 13, for example, where the data stream 14 is accessed at the predetermined portion by random access. According to one embodiment, two different digital signatures are transmitted in the data stream 14 for a given portion 13. According to this embodiment, the device 16 performs a check on authenticity on several portions of the data stream in sequence. According to one embodiment, if the data stream includes a previous portion relative to the given portion, device 16 is configured to perform a check regarding authenticity with respect to the first of two different digital signatures. For example, in this case, a check is performed according to any of the embodiments described above, such as the embodiment of Figure 7 or Figure 8. If the data stream does not include a previous portion relative to the given portion, device 16 checks whether hash value 33 matches the second of the two different digital signatures to determine whether the data stream can be trusted.

[0063] According to a further embodiment, the device 16 is configured to check, if the data stream includes a previous portion for a given portion, whether the hash value 33 and a further hash value 33' obtained by applying the hash function 31 to the previous portion 13' of the data stream or further data derived therefrom, match the digital signature 43, or whether the hash value 33 and a still further hash value transmitted in the data stream for the given portion match the digital signature (e.g., in concatenated form or further in the hash region) and the still further hash value is equal to a still further hash value 33' obtained by applying the hash function 31 to the previous portion 13' of the data stream or further data derived therefrom. Additionally, according to this embodiment, if the data stream includes a portion 13' before the predetermined portion 13, device 16 checks whether a combined hash value derived by hashing the predetermined portion 13 on the one hand and a further hash value 33' obtained by applying a hash function to the previous portion 13' of the data stream or further data derived therefrom matches the first of two different digital signatures, or whether a combined hash value derived by hashing the hash value and a still further hash value transmitted in the data stream for the predetermined portion matches the digital signature (e.g., in concatenated form or in a further hash field) and the still further hash value is equal to a still further hash value obtained by applying a hash function to the previous portion of the data stream or further data derived therefrom. According to this embodiment, if the data stream does not include a portion before the predetermined portion 13, device 16 checks whether the hash value matches the digital signature to determine whether the data stream can be trusted.

[0064] For example, if the data stream does not include a previous portion for a given portion, device 16 can determine whether the data stream can be trusted by checking whether the hash value matches the digital signature, for example, by checking whether the hash value and any further hash values ​​transmitted in the data stream for the given portion match the digital signature (e.g., in concatenated form or in further hash fields).

[0065] <Special processing of the first chunk of the bitstream to generate secure padding in the signature> According to one embodiment, at the beginning of a video where no preceding chunks exist, the proposed structure of the digital signature requires that some padding values ​​be inserted in the positions reserved for the hash values ​​of the previous chunks. It is proposed that these padding values ​​are specified solely for security against attacks that attempt to generate fake content matching such padding values, and that bitstreams or chunks whose actual calculated hash value matches such padded hash values ​​do not need to be accepted as authentic. In the event that an encoder generates a chunk that it wishes to mark as authentic but whose hash value happens to match the reserved padding hash value, the encoder can simply add so-called stuffing bytes to the bitstream, which, on the one hand, do not change the reconstructed content obtained from the bitstream, but, on the other hand, due to the design properties of typical hash functions, result in a hash value that is completely different from the hash value reserved for padding.

[0066] <Supporting delayed-time consistency verification for signature algorithms with small signatures> To further save bitrate when transmitting a digital signature, a signature algorithm that requires fewer bytes for the signature, such as the elliptic curve-based ECDSA P-256, can be selected. However, when doing so, the value that needs to be signed by the proposed technique, which is obtained by concatenating at least two hash values, may require too many bytes for the signature algorithm used. For this purpose, the encoder can hash this value before signature generation using a private key. Thus, for verification, the decoder first calculates a hashed value from the bitstream, then forms a concatenation with the hashed value of the previous chunk and, optionally, the index of the cryptographic hash function, then calculates a second hashed value of the combination, and compares this value with the transmitted digital signature using its public key.

[0067] Note that with this mechanism, the above-mentioned reliable random accessibility still holds, with the difference that any decoder that starts decoding the bitstream at a segment that is not the start segment cannot directly verify the authenticity of this segment, but can only do so when verifying the joint authenticity of the next segment together with the first segment. Thus, in terms of what it means to match a digital signature in general terms, for example, according to one embodiment, a digital signature is matched by a given value if it is equal to a check value in a further hash field arrived at by a further hash function applied to the given value or a concatenation of values ​​including the given value.

[0068] <Exclusion of post-processing tools sent to ensure reliability> The application of the present reliability verification algorithm focuses on the part of the bitstream containing video coding layer NAL units that directly belong to coded sample values ​​(e.g., coded slices or pictures). The reason for this is that other parts of the bitstream are often transmitted out-of-band. However, while this does not pose a significant reliability problem if such other information simply consists of information like sequence or picture parameter sets, it can become very problematic for the target application if this information is, for example, an SEI message that suggests the application of post-processing tools. An example of the latter kind of information is a neural network-based postfilter SEI message that can transmit the parameters of a postfilter. With the help of this SEI message, it may be possible to significantly modify the displayed content compared to the originally intended content assumed when generating a reliable bitstream by the above method.

[0069] For these reasons, one possible solution is that the proposed reliability verification be completely exclusive with some SEI messages, such as the neural network-based postfilter. This means that if a bitstream indicates that the reliability verification of the present invention should be applied, the neural network-based postfilter SEI message of this bitstream must indicate that the neural network-based postfilter will not be used. Instead, it is suggested that the specification text should include a note that the entire verification process produced by the proposed TWC-SEI is only intended to ensure the authenticity of the decoded image (and not the output image after the application of possible post-processing tools).

[0070] As another possible solution, it is proposed that instead of parts of the bitstream, chunks of the entire reconstructed picture or slice should be used to calculate the hash value in the above proposed process. Any post-processing applied to such chunks of reconstructed video sample values, not intended by the original generator of the content, would then result in this chunk being marked as untrusted.

[0071] <Enabling the proposed authenticity verification process via supplemental extension messages> It has been proposed to add three Supplemental Extension Information (SEI) messages, namely, Trustworthy Content Start SEI message (TWC-Start SEI), Trustworthy Content Selection SEI message (TWC-Select SEI), and Trustworthy Content Verification SEI message (TWC-Verify SEI) (see Figures 10A to 12), which can achieve the above trust verification process, to the SEI specification texts of H.264, H.265, and H.266.

[0072] The TWC-Start SEI first contains the syntax element twci_hash_method_type, which references the specific hash function to be used to generate a hash value from the bytes of a Video Coding Layer (VCL) NAL unit. The number of different chunks or substreams for which the hash value is to be calculated is then represented by the syntax element twci_num_verification_substreams_minus1. Finally, the URI from which the public key can be obtained, possibly combined with an index to a specific key listed in the URI, is coded by the syntax elements twci_key_source_uri, twci_use_key_register_idx_flag, and twci_key_register_idx. Here, the envisaged application when twci_use_key_register_idx_flag is true, is a situation where the URI represents a large trust center where possibly multiple content providers can register their keys.

[0073] If a reliable content selection SEI message is signaled, the hash value of the substream corresponding to twcs_verification_substream_id shall be updated using the hash function specified in the preceding TWC-Start SEI message. If no TWC-Start SEI message was sent, the bitstream shall also not contain a reliable content selection SEI message. For a given access unit, if a TWC-Start SEI message is sent but no reliable content selection SEI message is sent, the verification substream corresponding to index 0 shall be updated using the hash function specified in the preceding TWC-Start SEI message.

[0074] When a TWC verification SEI message is sent within an access unit, it contains a digital signature of the assumed hash value corresponding to the verification substream to which the video coding layer of the access unit belongs. Thus, if the access unit also contains a trusted content selection SEI message, this is the verification substream specified by this SEI message. If it does not contain a trusted content selection SEI message, this is the 0th verification substream. This digital signature should then be considered as a joint signature of the verification substream, the previous verification substream (in bitstream order), and the hash function index, as described above. Here, if the verification substream index is equal to k, where k>0, the previous verification substream is verification substream k-1. If the verification substream index is equal to 0, the previous substream is the last substream with index 0.

[0075] For the TWC validation SEI message, two cases need to be distinguished. If the underlying specification allows SEI messages to be sent only at the beginning of an access unit (prefix SEI only), especially before any VCL NAL units are sent, the TWC validation SEI is also sent before any VCL-NAL units are sent for the current access unit. This is the case in the H.264 specification. Furthermore, the TWC validation SEI indicates that the bytes of the VCL NAL units belonging to the current access unit must still be used in the on-the-fly calculation of a hash value that is performed on all VCL-NAL units belonging to the same chunk as the current VCL-NAL unit according to the corresponding TWC-Selection SEI. Then, the TWC validation SEI contains a digital signature that serves to verify (by invoking a public key) the authenticity of the current chunk of the VCL-NAL unit together with the preceding chunk of the VCL-NAL unit and, optionally, together with a hash function index, according to the process described above and as illustrated in Figures 5 to 7.

[0076] If the underlying specification also allows an SEI message to be sent within an access unit after a VCL-NAL unit (suffix SEI), the TWC validation SEI is sent as a suffix SEI. This is the case in the H.265 and H.266 specifications. The TWC validation SEI then again contains the same type of digital signature as before, which is applied to all preceding VCL-NAL units that belong to the same chunk as the VCL-NAL unit of the current access unit.

[0077] The reasons for the proposed differentiation in the positioning of the TWC validation SEI are as follows. First, conceptually, it seems more appropriate to position the TWC validation SEI after all bytes that will be used to calculate the hash value verified by the TWC validation SEI. Otherwise, an encoder generating trusted content would have to switch back and forth in the bitstream during encoding, i.e., first finish calculating the hash value, and then switch to the prefix SEI position to insert a digital signature of the hash value (using its private key) into the TWC validation SEI. Therefore, it also seems natural to position the TWC validation SEI as a suffix SEI whenever a suffix SEI is already supported. On the other hand, to conform to the general design philosophy of the underlying standards and specification texts to enable quick and easy deployment of the proposed technology, it does not seem reasonable to insert an entirely new mechanism, a suffix SEI, for authenticity verification if the standard does not already support it. The reason is that the security quality of the latter verification procedure, which is the main target of the proposed technology, is not affected by the issue of whether the TWC verification SEI is positioned as a prefix SEI or a suffix SEI.

[0078] The features, functionality, and advantages previously described in this section will now be restated in more general terms, with further details and variations. Any of the aforementioned features, functionality, and advantages may optionally be combined individually or in combination with any of the embodiments described below. The details and features described below may be combined with any of the embodiments described above, for example, with any of the embodiments described with respect to Figures 1-8.

[0079] For example, data stream 14 may include or consist of a sequence of packets, e.g., payload packets; in the case of the above-mentioned video codecs, for example, data stream 14 may include or consist of a sequence of packets, e.g., network abstraction layer units (NAL units). The sequence of packets may include packets carrying coded media data, which may be referred to as coded media packets. In the case of a video data stream, these packets may be referred to as coded video packets, or, as in the above-mentioned video codecs, as coded video layer (CVL) NAL units. In addition, the sequence of packets may include packets carrying supplemental information messages, e.g., packets carrying information indicating coding parameters for the decoding process and / or coding options for decoding the data stream. In the above-mentioned video codecs, these latter packets may be referred to as supplemental enhancement information (SEI) messages. In the above-mentioned video codecs, these messages may be carried as so-called non-VCL NAL units. In other words, although the above description uses the terminology of the video codecs mentioned above, e.g., "SEI messages," in some instances, it should be understood that similar syntax can be used for other video and audio codecs. For example, the described SEI messages may be implemented more generally in the form of supplemental information messages or packets interspersed among the coded media packets. For example, a predetermined portion may include a number of coded media packets and, optionally, one or more supplemental information messages or supplemental information packets carrying one or more supplemental information messages.

[0080] According to one embodiment, the digital signature 42 is transmitted in a supplemental information message of the data stream 14, for example, in a supplemental information message following a predetermined portion 13 in the data stream 14 or interspersed between predetermined portions 13 in the data stream. According to this embodiment, the decryption module 63 decrypts the digital signature 42 from the supplemental information message. According to one embodiment, device 16 locates a predetermined portion 13 within data stream 14 by using one or more supplemental information messages interspersed within data stream 14 to determine the predetermined portion to be a section of the data stream that spans between or extends from one or more supplemental information messages.

[0081] 9A illustrates a method for locating a predetermined portion 13 according to one embodiment. According to this embodiment, device 16 locates the predetermined portion 13 within the data stream through the use of a first supplemental information message 25 and a second supplemental information message 27 interspersed within the data stream. For example, device 26 determines the predetermined portion to be a section of the data stream that spans or is located between the first and second supplemental information messages. In other words, predetermined portion 13 may be a section of data stream 14 that spans from the first supplemental information message 25 to the second supplemental information message 27. Alternatively, predetermined portion 13 may be a portion of this section, i.e., a contiguous portion, or may be composed of multiple sub-portions.

[0082] 9A, the first supplemental information message 25 may be a prefix supplemental information message and the second supplemental information message 27 may be a suffix supplemental information message. For example, the prefix supplemental information message may be a supplemental information message that precedes the coded video packet, and the suffix supplemental information message may be a supplemental information message that is appended to the coded video packet. In other words, the suffix supplemental information message may be used to indicate the end of the section of the data stream 14 in which the given portion 13 is located. For example, the second supplemental info message may carry a digital signature 43 and / or the first supplemental info message may carry an indication of a hash function and / or an indication of a public key. For example, the first supplemental info message may be an initialization supplemental info message described below, and the second supplemental info message may be a content verification supplemental info message described below.

[0083] 9B shows another method of locating the predetermined portion 13 according to one embodiment. According to this embodiment, the predetermined portion 13 is determined using one supplementary information message 21 as a part or section of the data stream 14 extending from the supplementary information message 21. For example, according to this embodiment, the section of data stream 14 in which predetermined portion 13 is located extends to one end of data stream 14 or to one end of a coded media sequence, e.g., a coded video sequence, of data stream 14. For example, the coded media sequence may be an independently decodable section of the data stream, but instead the section may end with another occurrence of the supplemental information message 21. Alternatively, the end of a given portion 13 may be determined based on the length of the section.

[0084] 9C illustrates another method of locating a predetermined portion 13 according to one embodiment, which is an alternative to the embodiment of FIG. 9A. According to this embodiment, device 16 locates a predetermined portion 13 within a data stream through the use of a first supplemental information message 25 and a second supplemental information message 27 interspersed within the data stream. According to this embodiment, device 16 may determine the predetermined portion to be a section of the data stream that spans or is located between a first supplemental information message 25 and a point 19 in the data stream that is located downstream of the second supplemental information message 27. In other words, according to this embodiment, the section in which the predetermined portion is located may extend across the second supplemental information message, for example, this embodiment may be used for codecs that do not expect a suffix supplemental information message as described above.

[0085] According to one embodiment, device 16 determines a point 19 in data stream 14 downstream of second supplemental information message 27 as being at one end of a video coding layer portion having an access unit of the data stream containing the second supplemental information message, e.g., point 19 is the end of the coded video data or VCL data in the access unit containing the second supplemental information message. For example, the second supplemental info message may carry a digital signature 43 and / or the first supplemental info message may carry an indication of a hash function and / or an indication of a public key. For example, the first supplemental info message may be an initialization supplemental info message and the second supplemental info message may be a content verification supplemental info message, as described below. In other words, the above-described variations for locating a predetermined portion 13 within a data stream 14 may be used to determine the section of the data stream 14 in which the predetermined portion 13 is located. Optionally, one or more further criteria may be applied to select data to be included in the predetermined portion from the sections indicated by one or more supplemental information messages. For example, packets to be included in a given portion may be selected by package type.

[0086] A further selection criterion may be the temporal layer to which the coded video data belongs. For example, each frame of a video may be associated with one of one or more temporal layers defined in the data stream. For example, frames of a first temporal layer may be interspersed in time with frames of a second temporal layer, such that, for example, a substream consisting of the first temporal layer has a lower frame rate than a substream including the first and second temporal layers.

[0087] FIG. 10A illustrates the configuration of a predetermined portion 13 according to one embodiment, where a group of pictures (GOP) is assigned to the predetermined portion 13. For example, the group of pictures may include multiple pictures that may be assigned to different temporal layers of a data stream, shown as TL0-TL4 in FIG. 10A. According to this embodiment, a check for authenticity may be performed jointly on all pictures of the GOP in that they are all part of the predetermined portion that is subjected to the hash function 31. According to the embodiment of FIG. 10A, the predetermined portion 13 may be identified using the first supplemental information message 25 and the second supplemental information message 27, as described with respect to FIG. 9C, and the predetermined portion extends from the first supplemental information message to the end of the video coding layer of the access unit in which the second supplemental information message is located.

[0088] 10A-12, each section assigned to any one of TL0-TL4 may represent data for one access unit or picture unit, e.g., coded video (layer) data. For example, the illustrated supplemental information messages may belong to the same access unit or picture unit as the subsequent coded video data, except for the second supplemental information message 27 in the case of Figures 10B, 11B, and 12, where the second supplemental information message 27 may belong to the access unit or picture unit preceding it. For example, the embodiment of FIG. 10A may be an example of a simple GOP signature that may be implemented in H.264.

[0089] Figure 10B shows an alternative to the embodiment of Figure 10A in that the predetermined portion 13 ends with a second supplemental information message 27. According to the embodiment of Figure 10B, the predetermined portion 13 can be located as described with respect to Figure 9A. For example, the embodiment of Figure 10B can be an example of a simple GOP signature that can be implemented in H.265 or H.266.

[0090] The following describes embodiments in which a check for authenticity is performed on one or more substreams of a data stream. These substreams may be referred to as verification substreams. For example, the concept of verification substreams may allow different portions, e.g., packets, from one section of a data stream to be assigned to different verification substreams. For example, the verification substreams may be checked for authenticity separately or independently of each other. However, in examples, a hierarchy may be defined between substreams such that verification of a lower substream may require verifying data, e.g., hash values, of a higher substream. Furthermore, for example, at least the highest substream may be verified independently from the other substreams, such that such a substream may remain verifiable without updating its digital signature and without additional substreams after being extracted from the data stream. Thus, for example, the definition of a verification substream may correspond to the definition of a substream of independently decodable data stream 14.

[0091] For example, the verification substreams, eg, the number or count thereof, may be indicated in a supplemental information message, eg, the first supplemental information message 25, eg, an initialization supplemental information message. For example, the concept of a substream may be combined with the concept of defining the beginning and end of a portion on which a reliability check may be performed, such as a predetermined portion 13. In other words, the predetermined portion 13 may be a part of one of the substreams.

[0092] FIG. 11A illustrates the configuration of an exemplary number of portions of two substreams according to one embodiment. In particular, in FIG. 11A, a first portion 131 of a data stream is assigned to a first substream, and a second portion 132 is assigned to a second substream. In FIG. 11A, the allocation of pictures or access units to substreams is performed according to temporal layers, with access units of temporal layers 0-2 assigned to the first portion and access units of temporal layers 3-4 assigned to the second portion. However, this aspect should be understood as an example, and other allocations based on temporal layers or other criteria may be used. In other words, the separation by temporal layers should not be understood as limiting the mechanism for signaling the allocation of packets, pictures, access units, or units in general of a data stream to substreams.

[0093] According to one embodiment, device 16 locates a portion of data stream 14 (see portion 132 in FIG. 11A ) to check a given one of the substreams, e.g., the second substream in the example of FIG. 11A , for reliability using one or more third supplemental information messages 29. According to this embodiment, device 16 determines portion 132 to be the portion of data stream 14 formed by a subsection of the data stream, or the portion of these subsections that follows each of one or more third supplemental information messages 29 and extends to the coded video layer end of the access unit or picture unit in which the respective third supplemental information message is located. See, for example, FIG. 11A , where each of the pictures assigned to second portion 132 is preceded by one of the third supplemental information messages 29. In other words, for example, the allocation of sections of a data stream to one or more substreams may be performed at a picture-by-picture or access-by-access granularity.

[0094] For example, a picture unit carries coded video data of one picture and optionally supplementary information for decoding the picture, and an access unit carries one or more picture units belonging to one common time frame, e.g. pictures of different layers such as a base layer and an enhancement layer, and optionally supplementary information for the access units. For example, the third supplemental info message 29 is a TWC selection supplemental info message as described below, and the third supplemental info message 29 has a substream_id with a predetermined value associated with a predetermined substream.

[0095] The first substream in FIG. 11A is another example of a mechanism by which a portion for checking a further predetermined one of the substreams, here portion 131 for checking the first substream for reliability, can be located. According to one embodiment, device 16 determines the portion of the further predetermined substream by checking, for each picture unit or access unit within a section of the data stream (e.g., the section is determined as described above, e.g., as in FIGS. 9A-10B), whether the picture unit or access unit includes a supplemental information message of a predetermined type; if not, the picture unit or access unit, or a portion thereof, is assigned to that portion. If the picture unit or access unit includes a supplemental information message of the predetermined type, device 16 can check whether the supplemental information message indicates that the picture unit or access unit belongs to the further predetermined substream. If so, the picture unit or access unit, or a portion thereof, is assigned to that portion; otherwise, the picture unit or access unit, or a portion thereof, is not assigned to that portion.

[0096] More generally, according to one embodiment, device 16 is configured to derive a summary supplemental information message from the data stream, the summary supplemental information message indicating one or more substreams of the data stream for each of which it is possible to check the data stream for reliability based on one or more portions within an individual substream. According to this embodiment, device 16 can perform a check on the reliability of the data stream for a subset of one or more substreams of the one or more substreams.

[0097] According to one embodiment, device 16 performs a reliability check on a given substream of a subset of one or more substreams by locating a portion 132 of the given substream of the data stream through the use of one or more third supplemental information messages 29 interspersed within the data stream. According to this embodiment, device 16 determines the portion of the given substream to be a section of the data stream formed by concatenating subsections of the data stream that follow each of one or more third supplemental information messages and extend to the end of the video coding layer of the access unit in which the respective third supplemental information message is contained. According to this embodiment, optionally, device 16 can additionally use a second supplemental information message 27 (e.g., the second supplemental information message described above) to locate portion 132 by forming a section by additionally concatenating subsections that precede the second supplemental information message and extend to the start of the video coding layer of the access unit in which the second supplemental information message is contained, or that follow the second supplemental information message and extend to the end of the video coding layer of the access unit in which the second supplemental information message is contained.

[0098] As already mentioned, for each substream, the reliability check may be performed in units of one or more portions, e.g., temporally subsequent portions of data stream 14. For example, in the case of the example of FIG. 11A, the beginning and end of the portions of the substream are determined in the manner described with reference to FIG. 9C or FIG. 10A. See, for example, the first and second supplemental information messages 27*, e.g., TWC verification supplemental information messages, signaled before subportion 131*. For example, first and second supplemental information messages 27* may be part of the same access unit or picture unit as subportion 131*. As mentioned above, the end point 191 of the section of data stream 14 in which first portion 131 is located may end at the video coding layer end of the access unit or picture unit in which first and second supplemental information messages 27* are located. Similarly, for second portion 132, the termination point 192 of the section of data stream 14 in which second portion 132 is located may terminate at the video coding layer termination of the access unit or picture unit in which second secondary supplemental information message 27** is located. For example, second secondary supplemental information message 27** may be part of the same access unit or picture unit as sub-portion 132*. For example, the embodiment of FIG. 11A may be an example of a grouped temporal layer signature that may be implemented in H.264.

[0099] Figure 11B shows an alternative to the embodiment of Figure 11A in that the termination of the portions is determined according to the embodiment of Figure 9A or Figure 10B. For example, the section of data stream 14 in which first portion 131 is located may terminate at a first secondary supplemental information message 27*, and the section of data stream 14 in which second portion 132 is located may terminate at a second secondary supplemental information message 27**. For example, the embodiment of FIG. 11B may be an example of a grouped temporal layer signature that may be implemented in H.265 or H.266. In the following, for the case of multiple verification substreams, optional concepts are presented for verifying authenticity across part boundaries, in units where digital signatures are signaled.

[0100] Figure 12 shows an example of verification dependencies for portions of two substreams. Data stream 14 in Figure 12 may follow the scheme of Figure 11B, but shows two subsequent time segments. In other words, Figure 12 shows, for each of the two substreams, two portions in which the respective substreams can be checked for authenticity: portions 131 and 133 of the first substream and portions 132 and 134 of the second substream, where first portion 131 and second portion 132 belong to a first segment of data stream 14 that precedes, for example, in transmission order or stream order, the second segment of data stream 14 to which third portion 133 and fourth portion 134 belong.

[0101] For example, the substreams may have dependencies defined between them, e.g., a second substream may depend on a first substream for verification, and for example, the substreams may have a hierarchical order defined between them, where a lower-ranked substream depends on a higher-ranked substream.

[0102] 12 further illustrates four portions of check values ​​471 to 474 to which the above description of check value 47 can optionally be applied. For the highest ranked substream, e.g., the first substream in FIG. 12, Verification Substream 0, boundary crossing verification can be performed as described above, i.e., using the hash value of the portion of the same substream belonging to the preceding segment for verification. See, for example, third portion 133, where the hash value of first portion 131 is included in check value 473.

[0103] The check value 471 in the first portion 131 may, for example, contain a default value instead of the hash value of the previous segment, as described above, since the previous segment is not available. In the case of a second substream that is dependent on a first substream, instead of using a hash value of a previous portion of the same substream, the check value may include, for example, a hash value of the first substream of the same segment of the data stream, see check values ​​472 and 474 of second portion 132 and fourth portion 134.

[0104] 12 allows for the first substream to be verified independently of the second substream, for example, after extracting the first substream from data stream 14 without the second substream, or after dropping the second substream from data stream 14. Furthermore, in the case of a second substream, it can be verified that the first substream on which it depends has not been manipulated.

[0105] The following describes syntax and semantics according to one embodiment in which the media data stream is a video data stream. The syntax and semantics described below may be optionally modified or extended according to any of the aforementioned embodiments. The aforementioned embodiments may be combined with any of the details and features of the syntax and semantics described below. In particular, similar syntax may be used for other media, such as audio data streams. <Reliable Content Initialization SEI message> <Syntax of reliable content initialization SEI message> JPEG2025165877000002.jpg40161

[0106] <Reliable Content Initialization SEI Message Semantics> The Trusted Content Initialization SEI message, the Trusted Content Selection SEI message, and the Trusted Content Verification SEI message provide mechanisms for verifying that the coded video was generated by a trusted content provider. The Trusted Content Initialization SEI message provides information about the secure hash algorithm used to compute a message digest that is used together with the digital signature present in the Trusted Content Verification SEI message to verify the authenticity of the VCL NAL units present in the coded video sequence. This message also provides information about the digital signature algorithm used and the content provider's public key.

[0107] If any of a trusted content initialization SEI message, a trusted content selection SEI message, or a trusted content verification SEI message is present in a coded video sequence, it is a requirement for bitstream conformance that a trusted content initialization SEI message be present in all access units of the coded video sequence, including IDR access units and CRA pictures. It is a requirement for bitstream conformance that any trusted content selection SEI message and any trusted content verification SEI message in an access unit be preceded by a trusted content initialization SEI message.

[0108] The reliable content initialization SEI message applies to the current coded picture and all subsequent coded pictures until one or more of the following conditions become true: - The bitstream ends. - The start of a new coded video sequence. - Receiving a new reliable content initialization SEI message.

[0109] twci_hash_method_type indicates the secure hash algorithm used to compute a message digest of a subset of the VCL NAL units of the coded video sequence. Based on these message digests and the digital signature present in the Trusted Content Verification SEI message, a decoder can verify that the coded video was generated by the content originator indicated by the syntax elements twci_use_key_register_idx_flag, twci_key_source_uri, and, if twci_key_register_idx_flag is equal to 1, additionally indicated by twci_key_register_idx. Supported values ​​of the syntax element twci_hash_method_type, the size of the block used to compute the message digest, and the size of the computed message digest are specified in Table 1. Values ​​of twci_hash_method_type not listed in the table are reserved for future use by ITU-T|ISO / IEC and shall not be present in payload data conforming to this version of this specification. Decoders shall ignore Trusted Initialization SEI messages that contain reserved values ​​of twci_hash_method_type. The secure hash algorithms listed in Table 1 are specified in the "Secure Hash Standard," FIPS PUB 180-4. Table 1 Supported values ​​for twci_has_method_type JPEG2025165877000003.jpg61169 twci_num_verification_substreams_minus1+1 indicates the number of substreams on which the message digest is calculated and the number of signatures that may be present in the subsequent trusted content verification SEI message. The variable NumVerificationSubstream is derived as follows: NumVerificationSubstream=twci_num_verification_substreams_minus1+1. twci_use_key_register_idx_flag equal to 1 indicates that the URI contained in twci_key_source_uri specifies a certificate register and the syntax element twci_key_register_idx is present in the SEI message. twci_use_key_register_idx_flag equal to 0 indicates that the URI contained in twci_key_source_uri specifies a certificate and the syntax element twci_key_register_idx is not present in the SEI message.

[0110] twci_key_source_uri contains a URI with syntax and semantics specified in IETF Internet Standard 66. If twci_use_key_register_idx_flag is equal to 0, the URI identifies the content provider's certificate that can be used to verify the signature present in the following trusted verification SEI message. Otherwise (twci_use_key_register_idx_flag is equal to 1), the URI identifies the certificate register and content provider's certificate that can be used to verify the signature present in the following trusted verification SEI message indicated by twci_key_register_idx.

[0111] twci_key_register_idx contains an index within the certificate register indicated by twci_key_source_uri that specifies the content provider's certificate, which can be used to verify the signature present in the following trusted verification SEI message. The certificate indicated by the syntax elements twci_use_key_register_idx_flag, twci_key_source_uri, and, if twci_use_key_register_idx_flag is equal to 1, also by twci_key_register_idx, shall specify a digital signature method with associated parameters (if applicable) and the public key of the content provider. The format in which this information is provided is outside the scope of this specification. It is suggested to use a digital signature algorithm that complies with the "Digital Signature Standard", FIPS 186-5.

[0112] When a reliable content initialization SEI message is received, the calculation of the NumVerificationSubstream message digests is initialized according to the FIPS PUB 180-4 specification for the specified twci_hash_method_type. Each VCL NAL unit following the reliable content initialization SEI message is associated with one of the NumVerificationSubstream message digests. The verification substream id is indicated by the reliable content selection SEI message or is inferred to be equal to 0 if no reliable content selection SEI message is present for the coded picture. The message used to calculate the kth message digest (k is in the range 0 to twci_num_verification_substreams_minus1) is obtained by concatenating all VCL NAL units associated with the kth verification substream. The message digest calculation is performed on a block-by-block basis, and the size of the block is specified in Table 1 depending on the value of twci_hash_method_type. For each VCL NAL unit, the associated message digest is updated according to the algorithm specified in FIPS PUB 180-4 for the specified twci_hash_method_type. Note that since the message digest is calculated over the concatenation of all VCL NAL units of the verification substream, some of the processing blocks typically span two or more consecutive VCL NAL units.

[0113] <Trusted Content Selection SEI Message> <Syntax of the trusted content selection SEI message> JPEG2025165877000004.jpg19169 <Semantics of the Reliable Content Selection SEI Message> The trusted content selection SEI message provides a mechanism for associating a coded picture with one of the verification substreams indicated in the trusted content initialization SEI message. It is a bitstream conformance requirement that any reliable content selection SEI message be preceded by a reliable content initialization SEI message within the same coded video sequence.

[0114] twcs_verification_substream_id indicates the verification substream to which the VCL NAL units of the current coded picture are assigned. If there was a reliable content initialization SEI message in the current coded video sequence but no reliable content selection SEI message for the coded picture, the value of twcs_verification_substream_id is inferred to be equal to 0. The value of twcs_verification_substream_id shall be in the range from 0 to twci_num_verification_substream_minus1, inclusive.

[0115] The message digest of the verification substream with id equal to twcs_verification_substream_id is updated in the VCL NAL unit of the current coded picture according to the twci_hash_method_type specified in the preceding Reliable Content Initialization SEI message, as specified in Section 1.1.2.

[0116] <Trusted Content Validation SEI Message> <Syntax of Trusted Content Verification SEI Message> JPEG2025165877000005.jpg25169 <Reliable Content Validation SEI Message Semantics> The trusted content verification SEI message provides a mechanism for verifying the authenticity of video content. It is a bitstream conformance requirement that any trusted content verification SEI message be preceded by a trusted content initialization SEI message within the same coded video sequence. When a coded video sequence includes a reliable content initialization SEI message, it is a requirement of bitstream conformance that the last coded picture of the verification substream in the coded video sequence is associated with the reliable content verification SEI message.

[0117] twcv_signature_length_in_octets_minus1+1 specifies the length of the syntax element twcv_signature in octets (an octet consists of 8 bits). twcv_signature contains the digital signature for the verification substream indicated by twcs_verification_substream_id, which is sent in a trusted content selection SEI message preceding the trusted content verification SEI message in the same access unit or is assumed to be equal to 0. If VerificationSubstreamId is the value of twcs_verification_substream_id associated with a trusted content verification SEI message, then verification consists of the following ordered steps:

[0118] 1. The computation of the message digest, called CurrDigest, is completed as follows: The concatenation of VCL NAL units for the verification substream with id equal to VerificationSubstreamId is padded according to the specifications of FIPS PUB 180-4. Note that it is sufficient to pad the last VCL NAL unit of the verification substream. - The calculation of the message digest CurrDigest is completed according to the FIPS PUB 180-4 specification. The length of the message digest (in bits) is shown in Table 1.

[0119] 2. The reference message digest, RefDigest, is determined as follows: If VerificationSubstreamId is greater than 0, the reference message digest RefDigest is the last calculated message digest for the verification substream with id equal to VerificationSubstreamId-1. It is a bitstream conformance requirement that any trusted content verification SEI associated with verification substream id equal to VerificationSubstreamId-1 precedes any trusted content verification SEI message with verification substream id equal to VerificationSubstreamId.

[0120] - Otherwise, if the current trusted content verification SEI message is the first trusted content verification SEI message in the coded video sequence with verification id equal to 0 and the preceding coded video sequence did not contain a trusted content initialization SEI message (this includes the case where the current coded video sequence is the first coded video sequence in the bitstream), then RefDigest is set equal to a bitstring consisting of DigestSize bits equal to 1, where DigestSize is the size of the message digest as specified in Table 1. Otherwise, the reference message digest, RefDigest, is the last calculated message digest for the validation substream with id equal to 0.

[0121] 3. The identification string IdString is constructed by concatenating the reference message digest RefDigest, the current message digest, and the binary representation of twci_hash_method_type, as illustrated in FIG. The number of bits in RefDigest is determined by the value of twci_hash_method_type that was in effect when the value of RefDigest was calculated, and the number of bits in CurrDigest is determined by the current value of twci_hash_method_type, which is represented by 8 bits.

[0122] 4. The identification string IdString represents the message used to verify the signature. The signature verification algorithm and public key used to verify the signature are indicated by the syntax elements twci_use_key_register_idx_flag, twci_key_source_uri, and, if twci_use_key_register_idx_flag is equal to 1, also by twci_key_register_idx.

[0123] NOTE 1 - Because the bitstream used for signature verification includes the RefDigest, it is possible not only to verify that the VCL NAL unit used to compute the current message digest is correct, but also to additionally verify that no additional VCL NAL units were added to the bitstream, nor that no VCL NAL units were removed from the bitstream.

[0124] NOTE 2 - When a decoder tunes into the bitstream, it cannot correctly calculate the value of RefDigest and therefore cannot verify the configured IdString for the first trusted content verification SEI message. However, it can verify the signature starting from the second trusted content verification SEI message. After verification, the message digest of the verification substream with id equal to VerificationSubstreamId is reinitialized for the specified twci_hash_method_type according to the FIPS PUB 180-4 specification.

[0125] <Video coding method> In the following, an embodiment of a video encoding scheme is described, which is an example of a video codec that can be used in combination with the above scheme for authenticity checking of a video data stream. In other words, device 15 can be a video encoder, such as encoder 10 described below, and device 16 can be a video decoder, such as decoder 20 described below. The encoding codec used to encode the video into the video data stream may be one of those mentioned above, such as H.264, H.265, or H.266, but may generally be any type of video codec. An embodiment of such a codec is presented below. Here, it is block-based predictive coding using block-based transform coding, but as mentioned above, it may be another one.

[0126] Figure 14 shows an apparatus for predictively coding picture 12 into data stream 14, illustratively using transform-based residual coding. The apparatus or encoder is indicated using reference numeral 10. Figure 15 shows a corresponding decoder 20, i.e., apparatus 20 configured to predictively decode picture 12' from data stream 14, also using transform-based residual decoding, with an apostrophe used to indicate that picture 12' reconstructed by decoder 20 deviates from picture 12 originally coded by apparatus 10 in terms of coding loss introduced by quantization of the predicted residual signal.

[0127] The encoder 10 is configured to subject the prediction residual signal to a spatial-to-spectral transformation and to encode the prediction residual signal thus obtained into a data stream 14. Similarly, the decoder 20 is configured to decode the prediction residual signal from the data stream 14 and to subject the prediction residual signal thus obtained to a spectral-to-spatial transformation.

[0128] Internally, the encoder 10 may comprise a prediction residual signal former 22 that generates a prediction residual 24 to measure the deviation of a prediction signal 26 from the original signal, i.e., from the picture 12. The prediction residual signal former 22 may, for example, be a subtractor that subtracts the prediction signal from the original signal, i.e., from the picture 12. The encoder 10 then further comprises a transformer 28 that subjects the prediction residual signal 24 to a spatial-to-spectral transformation to obtain a spectral-domain prediction residual signal 24′, which is subjected to quantization by a quantizer 32 also included in the encoder 10. The prediction residual signal 24″ thus quantized is coded into the bitstream 14. For this purpose, the encoder 10 may optionally comprise an entropy coder 34 that entropy codes the prediction residual signal that is transformed and quantized into the data stream 14. The prediction signal 26 is generated by a prediction stage 36 of the encoder 10 based on the prediction residual signal 24″ that is coded into the data stream 14 and decodable therefrom. For this purpose, the prediction stage 36 may internally comprise, as shown in FIG. 14 , an inverse quantizer 38 that inversely quantizes the prediction residual signal 24″ to obtain a spectral-domain prediction residual signal 24′″ that corresponds to the signal 24′ without the quantization losses, followed by an inverse transformer 40 that subjects the latter prediction residual signal 24′″ to an inverse transform, i.e., a spectral-to-spatial transform, to obtain a prediction residual signal 24′″ that corresponds to the original prediction residual signal 24 without the quantization losses. A combiner 42 of the prediction stage 36 then recombines the prediction signal 26 and the prediction residual signal 24′″, for example by addition, to obtain a reconstructed signal 46, i.e., a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to the signal 12′. A prediction module 44 of the prediction stage 36 then generates a prediction signal 26 based on the signal 46, for example by using spatial prediction, i.e., intra-picture prediction, and / or temporal prediction, i.e., inter-picture prediction.

[0129] Similarly, decoder 20 is internally composed of components that correspond to prediction stage 36 and are interconnected in a manner corresponding to prediction stage 36, as shown in Figure 15. In particular, entropy decoder 50 of decoder 20 is able to entropy decode quantized spectral domain prediction residual signal 24'' from the data stream, with inverse quantizer 52, inverse transformer 54, combiner 56, and prediction module 58 interconnected and cooperating in the manner described above with respect to the modules of prediction stage 36 to recover a reconstructed signal based on prediction residual signal 24'' such that the output of combiner 56 provides the reconstructed signal, i.e., picture 12', as shown in Figure 15.

[0130] Although not specifically described above, it is readily apparent that the encoder 10 can set several coding parameters, including, for example, prediction modes, motion parameters, etc., according to several optimization schemes, such as, for example, a method for optimizing rate- and distortion-related criteria, i.e., encoding cost. For example, the encoder 10 and the decoder 20 and corresponding modules 44, 58 can each support different prediction modes, such as intra-coding and inter-coding modes. The granularity at which the encoder and decoder switch between these prediction mode types may correspond to the subdivision of the pictures 12 and 12′, respectively, into coding segments or coding blocks. For example, in units of these coding segments, a picture may be subdivided into intra-coded blocks and inter-coded blocks. The intra-coded blocks are predicted based on the individual blocks' spatially already-coded / decoded neighbors, as outlined in more detail below. Several intra-coding modes may exist and be selected for individual intra-coded segments, including directional or angular intra-coding modes in which each segment is filled by extrapolating neighboring sample values ​​along a specific direction specific to the individual directional intra-coding mode to the individual intra-coded segment. The intra-coding modes may also include one or more further modes, such as, for example, a DC coding mode in which the prediction of an individual intra-coded block assigns a DC value to all samples in the individual intra-coded segment, and / or a planar intra-coding mode in which the prediction of an individual block is approximated or determined to be a spatial distribution of sample values ​​described by a two-dimensional linear function over the sample positions of the individual intra-coded block with a planar driving slope and offset defined by the two-dimensional linear function based on neighboring samples. In comparison, inter-coded blocks may, for example, be predicted temporally.For inter-coded blocks, motion vectors may be signaled within the data stream, indicating the spatial displacement of portions of previously coded pictures of the video to which picture 12 belongs, and the previously coded / decoded pictures are sampled to obtain prediction signals for the respective inter-coded blocks. This means that in addition to the residual signal coding included in data stream 14, such as entropy-coded transform coefficient levels representing the quantized spectral-domain prediction residual signal 24", data stream 14 may also have coded therein optional further parameters, such as coding mode parameters for assigning coding modes to various blocks, motion parameters for inter-coded segments, and some prediction parameters for the blocks, as well as parameters for controlling and signaling the subdivision of pictures 12 and 12' into their respective segments. Decoder 20 uses these parameters to subdivide the picture in the same way as the encoder did, assign the same prediction modes to the segments, and perform the same predictions, resulting in the same prediction signals.

[0131] FIG. 16 illustrates the relationship between, on the one hand, a reconstructed signal, i.e., a reconstructed picture 12′, and, on the other hand, a combination of a prediction residual signal 24″″ and a prediction signal 26 signaled in the data stream 14, according to one embodiment. As already mentioned above, the combination may be additive. In FIG. 16, the prediction signal 26 is illustrated as a subdivision of the picture area into intra-coded blocks, exemplarily shown with hatching, and inter-coded blocks, exemplarily shown without hatching. The subdivision may be any subdivision, such as a regular subdivision of the picture area into rows and columns of square or non-square blocks, or a multi-tree subdivision of the picture 12 from a tree-root block into multiple leaf blocks of various sizes, such as a quad-tree subdivision, or a mixture thereof, as shown in FIG. 16, in which the picture area is first subdivided into rows and columns of tree-root blocks, which are then further subdivided into one or more leaf blocks according to a recursive multi-tree subdivision.

[0132] Again, data stream 14 may have encoded therein an intra-coding mode that assigns one of several supported intra-coding modes for intra-coded blocks 80 to each individual intra-coded block 80. For inter-coded blocks 82, data stream 14 may have encoded therein one or more motion parameters. Generally speaking, inter-coded blocks 82 are not limited to being temporally coded. Instead, inter-coded blocks 82 may be any block predicted from a previously coded portion beyond current picture 12 itself, such as a previously coded picture of the video to which picture 12 belongs, or, if the encoder and decoder are scalable encoder and decoder, respectively, a picture of another view or a hierarchically lower layer.

[0133] The prediction residual signal 24"" in Figure 16 is also shown as a subdivision of the picture area into blocks 84. These blocks are sometimes called transform blocks to distinguish them from the coding blocks 80 and 82. In fact, Figure 16 shows that the encoder 10 and decoder 20 may use two different subdivisions of the picture 12 and the picture 12' into blocks: one subdivision into coding blocks 80 and 82, respectively, and the other subdivision into transform blocks 84. While both subdivisions may be the same, i.e., each coding block 80 and 82 may simultaneously form a transform block 84, Figure 16 also shows the case where, for example, the subdivision into transform blocks 84 forms an extension of the subdivision into coding blocks 80, 82, such that any boundary between the two blocks 80 and 82 covers the boundary between the two blocks 84, or instead, each block 80, 82 coincides with one of the transform blocks 84 or with a cluster of transform blocks 84. However, these subdivisions may also be determined or selected independently of one another, such that the transformation blocks 84 may instead cross the block boundaries between the blocks 80, 82. Thus, as far as the subdivision into transformation blocks 84 is concerned, similar statements are true as those presented with regard to the subdivision into blocks 80, 82, i.e., the blocks 84 may be the result of a regular subdivision of the picture area into blocks (with or without arrangement into rows and columns), a recursive multi-tree subdivision of the picture area, or a combination thereof, or any other type of blocking. It should be noted that the blocks 80, 82, and 84 are not limited to quadratic shapes, rectangles, or any other shape.

[0134] 16 further illustrates that the combination of the prediction signal 26 and the prediction residual signal 24'''' directly results in the reconstructed signal 12'. However, it should be noted that, according to alternative embodiments, more than one prediction signal 26 can be combined with the prediction residual signal 24''' to result in the picture 12'.

[0135] In Figure 16, the transform blocks 84 have the following meaning: The transformer 28 and the inverse transformer 54 perform transforms in units of these transform blocks 84. For example, many codecs use some kind of DST or DCT for all transform blocks 84. Some codecs allow for skipping the transform for some of the transform blocks 84, so that the prediction residual signal is directly coded in the spatial domain. However, according to the embodiments described below, the encoder 10 and the decoder 20 are configured so that they support several transforms. For example, the transforms supported by the encoder 10 and the decoder 20 may include:

[0136] DCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform DST-IV, where DST stands for discrete sine transform DCT-IV DST-VII Identity Transformation (IT) Of course, the transformer 28 supports all of the forward transform versions of these transforms, while the decoder 20 or inverse transformer 54 supports the corresponding backward or inverse versions. Inverse DCT-II (or Inverse DCT-III) Reverse DST-IV ·Inverse DCT-IV Reverse DST-VII Identity Transformation (IT) Different examples for coding the residual blocks representing spatial residual blocks in the transform domain and their transform blocks respectively are given below. Although a codec may support only one of them, the video data stream may also contain an entropy coding mode identifier indicating whether the prediction residual data of the residual blocks should be decoded from the video data stream using a context-adaptive variable length coding mode or a context-adaptive binary arithmetic coding mode, examples of which can be derived from the following description.

[0137] <Context-based adaptive variable length coding (CAVLC)> This is the method used to code residual blocks of zigzag-ordered 4x4 (and 2x2) transform coefficients. CAVLC is designed to take advantage of several properties of quantized 4x4 blocks. 1. After prediction, transformation, and quantization, blocks are usually sparse (contain mostly zeros). CAVLC uses run-level coding to represent strings of zeros compactly. 2. The highest non-zero coefficient after zigzag scanning is often a sequence of + / - 1. CAVLC compactly signals the number of frequent + / - 1 coefficients ("trailing 1" or "T1"). 3. The number of non-zero coefficients in adjacent blocks is correlated. The number of coefficients is coded using a lookup table. The choice of lookup table depends on the number of non-zero coefficients in the adjacent blocks. 4. The levels (magnitudes) of non-zero coefficients tend to be high at the beginning of the sorted sequence (near the DC coefficient) and decrease towards higher frequencies. CAVLC exploits this by adapting the VLC lookup table selection of the "level" parameter depending on the magnitude of the most recently coded level.

[0138] CAVLC coding of a block of transform coefficients proceeds as follows. 1. Encode the number of coefficients and trailing ones (coeff_token). The first VLC, coeff_token, encodes both the total number of non-zero coefficients (TotalCoeffs) and the number of trailing + / - 1 values ​​(T1). TotalCoeffs is 0 (no coefficients in a 4x4 block). 1T1 can be anywhere from 0 to 16 (16 non-zero coefficients). T1 can be anywhere from 0 to 3; if there are more than three trailing + / - 1's, only the last three are treated as a "special case" and the rest are coded as normal coefficients. Note: The coded_block_pattern (see above) indicates which 8x8 blocks in a macroblock contain non-zero coefficients; however, within a coded 8x8 block, there may be 4x4 sub-blocks that do not contain any coefficients, and therefore TotalCoeff can be 0 in any 4x4 sub-block. In practice, this value of TotalCoeff occurs most frequently and is assigned the shortest VLC.

[0139] There are four choices of lookup tables to use to encode the coeff_token, described as Num-VLC0, Num-VLC1, Num-VLC2, and Num-FLC (three variable length code tables and one fixed length code). The choice of table depends on the number of non-zero coefficients in the previously coded blocks above and to the left, Nu and NL. The parameter N is calculated as follows: If blocks U and L are available (i.e., within the same coded slice), then N=(Nu+NL) / 2; if only block U is available, then N=NU; if only block L is available, then N=NL; and if neither is available, then N=0.

[0140] N selects a lookup table (Table 34), and in this way the VLC selection is adapted depending on the number of coded coefficients in the neighboring block (context adaptation). Num-VLC0 is "biased" towards a small number of coefficients: low values ​​of TotalCoeff (0 and 1) are assigned particularly short codes, and high values ​​of TotalCoeff are assigned particularly long codes. Num-VLC1 is biased towards a medium number of coefficients (relatively short codes are assigned to TotalCoeff values ​​around 2-4), Num-VLC2 is biased towards a higher number of coefficients, and FLC assigns a fixed 6-bit code to all values ​​of TotalCoeff. Table 34. Coeff_token lookup table selection JPEG2025165877000006.jpg29112

[0141] 2. Encode the code of each T1. For each T1 (trailing + / -1) signaled by the coeff_token, encode the sign with a single bit (0=+, 1=-). These are coded in reverse order, starting with the most frequent T1.

[0142] 3. Encode the levels of the remaining non-zero coefficients. The level (sign and magnitude) of each remaining non-zero coefficient in the block is coded in reverse order, starting with the most frequent and working back towards the DC coefficient. The selection of the VLC table for coding each level is adapted according to the magnitude of each successive coding level (context adaptation). There are seven VLC tables to choose from: Level_VLC0 to Level_VLC6. Level_VLC0 is biased towards lower magnitudes. Level_VLC1 is biased towards slightly higher magnitudes, etc. The table selection is adapted as follows: (a) Initialize the table to Level_VLC0 (unless there are more than 10 non-zero coefficients and not less than 3 trailing + / - 1's, then start at Level_VLC1). (b) Encode the most frequent non-zero coefficients. (c) If the magnitude of this coefficient is greater than a predetermined threshold, move to the next VLC table. In this way, the level selection corresponds to the magnitude of the most recently coded coefficient. The thresholds are listed in Table 35. The first threshold is 0, which means that the table is always incremented after the first coefficient level is coded. Table 35. Thresholds for determining whether to increment the level table number JPEG2025165877000007.jpg46126

[0143] 4. Encode the total number of zeros before the last coefficient. TotalZeros is the sum of all zeros preceding the highest non-zero coefficient in the sorted array. This is coded in the VLC. The reason for sending a separate VLC t indicating TotalZeros is that many blocks contain some non-zero coefficients at the beginning of the array, and (as we'll see below) this technique means that the run of zeros at the beginning of the array does not need to be coded.

[0144] 5. Code each run of zeros. The number of zeros preceding each non-zero coefficient (run_before) is coded in reverse order: the run_before parameter is coded for each non-zero coefficient, starting with the most frequent, with two exceptions: (a) If there are no more zeros left to encode (ie, Σ[run_before]=TotalZeros), then there is no need to encode any more run_before values. (b) There is no need to code the run_before of the last (least frequent) non-zero coefficient. The VLC for each run of zeros is selected according to (a) the number of zeros not yet encoded (ZerosLeft) and (b) run_before. For example, if only 2 zeros remain to be encoded, run_before can only take 3 values (0, 1, or 2), and thus the VLC need not exceed 2 bits in length. If there are 6 zeros yet to be encoded, run_before can take 7 values (0 to 6), and the VLC table needs to be made larger accordingly.

[0145] <Example of CAVLC> In all the following examples, it is assumed that table Num-VLC0 is used to encode the coeff_token. · Example 1 4×4 block: JPEG2025165877000008.jpg2459 Shuffled block: 0, 3, 0, 1, -1, -1, 0, 1, 0... TotalCoeff = 5 (indexed from highest frequency [4] to lowest frequency [0]) TotalZeros = 3 T1s = 3 (although there are actually 4 trailing 1s, only 3 can be encoded as a "special case") Encoding: JPEG2025165877000009.jpg77164 The transmitted bitstream for this block is 000010001110010111101101. <s

[0146] Decoding: The output array is "constructed" from the decoded values as shown below. The values added to the output array at each stage are shown within []. JPEG2025165877000010.jpg77169 The decoder inserted two zeros, however TotalZeros equals 3, so another zero is inserted before the lowest coefficient to form the final output array. [0], 3, 0, 1, -1, -1, 0, 1

[0147] Example 2 4x4 blocks: JPEG2025165877000011.jpg2353 Sorted blocks: -2, 4, 3, -3, 0, 0, -1, ... TotalCoeffs=5 (indexed from most frequent [4] to least frequent [0]) TotalZeros=2 T1s=1 <encoding>: JPEG2025165877000012.jpg62168 The transmitted bitstream for this block is 000000011010001001000010111001100.

[0148] NOTE 1: Level (3) with a value of -3 is coded as a special case. If there are less than three T1s, the first non-T1 level does not have a value of + / -1 (it would otherwise have been coded as T1). To save bits, this level is incremented if negative (decremented if positive), so that + / -2 maps to + / -1, + / -3 maps to + / -2, and so on. In this way, shorter VLCs are used.

[0149] Note 2: After encoding level (3), the level_VLC table is incremented because the magnitude of this level is greater than the first threshold (which is 0). After encoding level (1), with a magnitude of 4, the table number is incremented again because level (1) is greater than the second threshold (which is 3). Note that the final level (-2) uses a different code than the first encoding level (also -2). <Decryption>: JPEG2025165877000013.jpg78169 Now all zeros have been decoded, so the output array is: -2, 4, 3, -3, 0, 0, -1 (This example shows how bits are saved by encoding all zeros; even though there are five non-zero coefficients, only one run needs to be encoded.)

[0150] <cabac> In CABAC, encoding and decoding can be done as follows. 1) The position of the first non-zero transform coefficient encountered when traversing the transform coefficients along a predetermined scan order is coded / decoded. 2) The transform coefficients, including the non-zero transform coefficients in the coded / decoded positions that follow in scan order, are coded / decoded from the data stream.

[0151] Alternatively, in CABAC, encoding and decoding may be performed as follows: 1) A significance map indicating the location of a non-zero transform coefficient is coded / decoded by using a significance flag and a last significance flag. In a forward scan traversing the locations of the transform coefficients, a significance flag indicating whether a non-zero transform coefficient is located at a particular location is coded / decoded, and if so, and if the location is not the end of the forward scan, a last significance flag indicating whether the non-zero transform coefficient located at the particular location is the last non-zero transform coefficient of the forward scan is coded / decoded; 2) The values ​​of the non-zero transform coefficients are coded / decoded sequentially in a reverse scan order, which reverses the forward scan order.

[0152] In CABAC, encoding and decoding can alternatively be done as follows: 1) encoding / decoding coordinates of a location within a transform block representing prediction residual data at which a last non-zero transform coefficient is encountered when traversing the transform coefficients of the transform block along a predetermined scan order; 2) sequentially encoding / decoding the values ​​of ranked transform coefficients included between the last non-zero transform coefficient and the first scanned transform coefficient along a predetermined scanning order; 3) A predetermined scan order from among the diagonal scan order, horizontal scan order, and vertical scan order is selected according to the intra prediction mode of the intra predicted block using a mapping that maps each of a plurality of corresponding intra prediction modes, for example, 33, to the corresponding diagonal scan order, horizontal scan order, and vertical scan order.

[0153] In CABAC, encoding and decoding can alternatively be done as follows: 1) context-adaptive binary arithmetic coding / decoding of quantization indices of transform coefficients of a transform block representing a residual block; and 2) by using a successive quantization to obtain a quantization index, where the value of the current transform coefficient is quantized depending on the parity of the quantization index of the previous quantization index, or a successive inverse quantization of a quantization index, where the value of the current transform coefficient depends on the parity of the quantization index of the previous quantization index; This function encodes / decodes prediction residual data of residual blocks from a video data stream.

[0154] 17 shows a flowchart of a method 160 for checking the authenticity of a media data stream in which video is encoded, according to one embodiment, performed, for example, by device 16. The method includes subjecting a predetermined portion of the media data stream, or data derived therefrom, to a hash function to obtain a hash value (131), deriving a digital signature from the media data stream (161), and checking whether the hash value matches the digital signature to determine whether the media data stream can be trusted (141).

[0155] 18 shows a flowchart of a method 200 for decrypting video from a media data stream according to one embodiment, performed, for example, by device 20. The method includes decrypting (163) from the media data stream a digital signature to be subjected to a check regarding the authenticity of the media data stream, the check including subjecting a predetermined portion of the media data stream, or data derived therefrom, to a hash function to obtain a hash value, and checking whether the hash value matches the digital signature to determine whether the media data stream can be trusted.

[0156] 19 shows a flowchart of a method 150 for rendering a media data stream in which video that can be checked for authenticity is encoded, performed, for example, by device 15. The method includes subjecting a predetermined portion of the media data stream, or data derived therefrom, to a hash function to obtain a hash value (131), calculating a digital signature based on the hash value to digitally sign the hash function (171), and inserting the digital signature into the media data stream, thereby making it possible to determine whether the media data stream is authentic by checking whether the hash value matches the digital signature (177).

[0157] Further embodiments The following embodiments refer to video data streams, but may alternatively be embodied for any type of media data stream, e.g., an audio data stream, unless the embodiments refer to video-specific features. When referring to a "video coded layer," reference may be made to a portion of a media data stream that comprises coded media data of the media data stream. 1. An apparatus 16 for checking the authenticity of a video data stream 14 in which video is encoded, the apparatus comprising: applying a predetermined portion 13 of the video data stream, or data derived therefrom, to a hash function 31 to obtain a hash value 33; Deriving a digital signature 43 from the video data stream; configured to determine 41 whether the video data stream is authentic by checking whether the hash value 33 matches the digital signature 43; a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing a residual block, and a trailing ones number indicating the number of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; one or more second syntax elements that indicate the signs of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; and one or more third syntax elements indicating values ​​of the non-zero transform coefficients, excluding the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order; and a fourth syntax element indicating the total number of zero-valued transform coefficient levels in the transform block following the first encountered non-zero transform coefficient in scan order; and encoding the video into the video data stream by block-based predictive coding and transform-based residual coding; and encoding predictive residual data of residual blocks into the video data stream by use of context-adaptive variable length coding using one or more fifth syntax elements that indicate a position of non-zero transform coefficients along the scan order by indicating a number of consecutive zero-valued transform coefficients in the scan order between successively encountered non-zero transform coefficients.

[0158] 2. The apparatus of embodiment 1, wherein the digital signature is transmitted within a supplemental enhancement information message of the video data stream. 3. Checking whether the hash value matches the digital signature to determine whether the video data stream is authentic; Decrypt the digital signature to obtain the check value 47,45 3. The apparatus of embodiment 1 or 2, configured to check 49 whether the hash value 33 matches a check value 47. 4. Applying a predetermined portion 22 of the video data stream, or data derived therefrom, to a hash function to obtain a hash value; reconstructing the video with respect to the predetermined portion to obtain a reconstructed portion of the video; 4. An apparatus according to any one of embodiments 1 to 3, configured to subject the reconstructed portion to a hash function.

[0159] 5. The device is Decodes video from the video data stream, 5. The apparatus according to any one of embodiments 1 to 4, wherein the apparatus is a decoder 20 configured to decode a digital signature from a video data stream. 6. configured to decode the digital signature from the supplemental enhancement information message of the video data stream; 6. The device of embodiment 5.

[0160] 7. configured to locate a predetermined portion within a video data stream by using one or more SEI messages 21, 25, 27 interspersed within the video data stream to determine the predetermined portion to be a section of the video data stream that spans or extends between one or more SEI messages; 7. The device of embodiment 5 or 6. 8. configured to decrypt a digital signature from one of the one or more SEI messages of a video data stream; 8. The device of embodiment 7. 9. configured to locate a predetermined portion 13 within a video data stream by using prefix SEI messages 25 and suffix SEI messages 27 interspersed within the video data stream to determine the predetermined portion to be a section of the video data stream that spans or is located between the prefix SEI message and the suffix SEI message; 7. The device of embodiment 5 or 6.

[0161] 10. configured to locate a predetermined portion 13 within a video data stream by using prefix SEI messages 25 and suffix SEI messages 27 interspersed within the video data stream, by determining that the predetermined portion is located between the prefix SEI message and the suffix SEI message; 7. The device of embodiment 5 or 6. 11. The apparatus of embodiment 9 or 10, configured to decrypt a digital signature from a suffix SEI message. 12. An apparatus as described in embodiment 5 or 6, configured to locate a predetermined portion 13 within a video data stream by using a first SEI message 25 and a second SEI message 27 interspersed within the video data stream to determine that the predetermined portion is a section of the video data stream that spans or is located between the first SEI message and the second SEI message, or a section of the video data stream that spans between the first SEI message and the second SEI message, or a point in the data stream that is located downstream of the first SEI message and the second SEI message.

[0162] 13. An apparatus as described in embodiment 5 or 6, configured to locate a predetermined portion 13 within a video data stream by using a first SEI message 25 and a second SEI message 27 interspersed within the video data stream to determine that the predetermined portion is located between the first SEI message and the second SEI message, or between a point in the data stream located downstream of the first SEI message and the second SEI message. 14. An apparatus as described in embodiment 12 or 13, configured to determine a point in the data stream located downstream of the second SEI message based on the length of the section or as being located at the end of a video coding layer portion having an access unit of the video data stream in which the second SEI message is contained.

[0163] 15. The apparatus of any one of embodiments 12-14, configured to decrypt the digital signature from the second SEI message. 16. Deriving from the video data stream a summary SEI message indicating one or more substreams of the video data stream, each of which is capable of checking the video data stream for reliability based on one or more portions within an individual substream; For a given substream of the subset, configured to perform checking the video data stream for reliability with respect to a subset of one or more of the one or more substreams by locating a portion of the video data stream by using one or more first and second SEI messages interspersed within the video data stream to determine that the portion of the given substream is a section of the video data stream formed by concatenating a subsection of the video data stream that follows each of the one or more first SEI messages and extends to a video coding layer end of the access unit in which the respective first SEI message is contained, and a subsection that precedes the second SEI message and extends to a video coding layer start of the access unit in which the second SEI message is contained, or that follows the second SEI message and extends to a video coding layer end of the access unit in which the second SEI message is contained; 7. The device of embodiment 5 or 6.

[0164] 17. Deriving from the video data stream a summary SEI message indicating one or more substreams of the video data stream, for each of which the video data stream can be checked for reliability based on one or more portions within an individual substream; For a given substream of the subset, configured to perform checking the video data stream for reliability with respect to a subset of one or more of the one or more substreams by locating a portion of the video data stream of interest by using one or more first SEI messages interspersed within the video data stream to determine that the portion of the given substream is a section of the video data stream formed by concatenating subsections of the video data stream that follow each of the one or more first SEI messages and extend up to a video coding layer end of the access unit in which the respective first SEI message is contained; 7. The device of embodiment 5 or 6.

[0165] 18. Deriving from the video data stream a summary SEI message indicating one or more substreams of the video data stream, for each of which the video data stream can be checked for reliability based on one or more portions within an individual substream; For a picture unit of the video data stream, checking whether the video data stream includes a sub-stream selection SEI message for the picture unit; if the video data stream does not include a sub-stream selection SEI message for the picture unit, assigning video coded layer payload packets of the picture unit to a first sub-stream of the one or more sub-streams; If the video data stream includes a substream selection SEI message for the picture unit, by assigning video coded layer payload packets of the picture unit to one or more substreams, the substreams indicated by the substream selection SEI message; determining a respective portion of the video data stream for one or more substreams; 7. The apparatus of embodiment 5 or 6, configured to perform checking the video data stream for reliability with respect to a subset of the one or more substreams of the one or more substreams.

[0166] 19. Checking the video data stream for reliability with respect to a subset of one or more substreams; An apparatus as described in any of embodiments 16 to 18, configured to further perform by subjecting the determined portions for one or more substreams of the subset of one or more substreams to a hash function to derive respective hash values. 20. An apparatus as described in any of embodiments 16 to 19, further configured to determine the portion of each substream of a subset of one or more substreams by using a first SEI message 25 and a second SEI message 27 interspersed within the video data stream to determine that the portion is located between the first SEI message and the second SEI message, or between a point in the data stream located downstream of the first SEI message and the second SEI message.

[0167] 21. An apparatus described in any of embodiments 16 to 20, further configured to determine that each portion of a substream of a subset of one or more substreams terminates at or before one end of the encoded video sequence. 22. One or more substreams have a defined order therein, and the apparatus determines that a given portion is part of a given substream of a subset of the one or more substreams; whether the hash value and a further hash value obtained by applying a hash function to the previous portion of the predetermined portion, or further data derived from it, match the digital signature; or by checking whether a combined hash value derived by hashing the predetermined portion and a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, matches the digital signature; configured to sequentially perform a check of the video data stream for reliability for each portion of the video data stream derived for one or more substreams; the previous part precedes the given part in the defined sequence order between the parts; An apparatus described in any one of embodiments 17 to 20.

[0168] 23. Whether the hash value and a further hash value obtained by applying a hash function to an earlier portion of the video data stream, or further data derived therefrom, match the digital signature; or and checking whether a combined hash value derived by hashing the predetermined portion and a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, matches the digital signature. configured to sequentially perform checking the video data stream for authenticity with respect to a plurality of portions of the video data stream; An apparatus according to any one of embodiments 1 to 21.

[0169] 24. Combining the hash value with a further hash value to obtain a combined hash value, and checking whether the combined hash value matches the digital signature; or configured to apply a hash function to the predetermined portion and a previous portion of the video data stream, or further data derived therefrom, to obtain a combined hash value by applying the resulting further hash value to the predetermined portion and a previous portion of the video data stream, or further data derived therefrom, and to check whether the combined hash value matches the digital signature; 24. A device according to embodiment 22 or 23. 25. The device of embodiment 24, wherein the combination is a linkage. 26. The device of embodiment 22, 23, 24, or 25, configured to check whether parameters or identifiers of a hash function match the digital signature to further determine whether the video data stream is authentic.

[0170] 27. Two different digital signatures are transmitted within the video data stream for a given portion, and the device: If the video data stream contains a previous part for a given part, whether the hash value and a further hash value obtained by applying a hash function to an earlier portion of the video data stream, or further data derived therefrom, matches the first of two different digital signatures; or checking whether a combined hash value derived by hashing the predetermined portion and a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, matches a first of the two different digital signatures; If the video data stream does not contain any previous parts for a given part, By checking whether the hash value matches the second of two different digital signatures to determine whether the video data stream can be trusted. configured to sequentially perform checking the video data stream for authenticity with respect to a plurality of portions of the video data stream; An apparatus described in any one of embodiments 1 to 26.

[0171] 28. The device is If the video data stream contains a previous part for a given part, checking whether the hash value and a further hash value obtained by applying a hash function to an earlier portion of the video data stream or further data derived therefrom match the digital signature, or whether the hash value and a still further hash value obtained by applying a hash function to an earlier portion of the video data stream or further data derived therefrom, which is transmitted in the video data stream for a given portion, match the digital signature and the still further hash value is equal to a still further hash value obtained by applying a hash function to the earlier portion of the video data stream or further data derived therefrom; checking whether a combined hash value derived by hashing the predetermined portion on the one hand and a further hash value obtained by applying a hash function to an earlier portion of the video data stream or further data derived therefrom on the other hand matches the first of the two different digital signatures, or whether a combined hash value obtained by applying a hash function to the hash value and a further hash value transmitted in the video data stream for the predetermined portion matches the digital signature and the further hash value is equal to a further hash value obtained by applying a hash function to an earlier portion of the video data stream or further data derived therefrom, If the video data stream does not contain any previous parts for a given part, By checking if the hash value matches the digital signature to determine if the video data stream can be trusted. 27. An apparatus as described in any one of embodiments 1 to 26, configured to sequentially perform checking of the video data stream for reliability for multiple portions of the video data stream.

[0172] 29. The device If the video data stream does not contain any previous parts for a given part, 29. The apparatus of claim 28, configured to determine whether the video data stream is trustworthy by checking whether the hash value matches the digital signature by checking whether the hash value and a further hash value transmitted within the video data stream for a given portion match the digital signature.

[0173] 30. The predetermined value is equal to the check value obtained by decrypting the digital signature or a predetermined portion of the check value related to the predetermined value; or If the predetermined value is equal to a check value in a further hash field, arrived at by a further hash function applied to the predetermined value or to a concatenation of values ​​including the predetermined value, The digital signature is matched by a predetermined value, An apparatus described in any one of embodiments 1 to 29. 31. The apparatus of any one of embodiments 1 to 30, configured to perform the check by using an asymmetric decryption scheme using a public key. 32. The apparatus of embodiment 31, configured to derive an asymmetric decryption scheme using first information derived from the data stream. 33. The apparatus of embodiment 32, wherein the first information includes a decryption scheme identifier or a first pointer to a first location from which the asymmetric decryption scheme can be determined, or an identifier of an entity that encoded the video into the video data stream. 34. The apparatus of any one of embodiments 31-33, configured to derive a public key using second information derived from the data stream. 35. The apparatus of embodiment 34, wherein the second information includes a second pointer to a second location from which the public key can be obtained, or an identifier of the entity that encoded the video into the video data stream.

[0174] 36. An apparatus configured to derive a public key by deriving a first syntax element and a second syntax element from a video data stream, the second syntax element indicating a pointer to a location from which the public key can be derived, the apparatus comprising: if the first syntax element has a first state, inferring that the location identifies exactly one public key, deriving exactly one public key from the location; An apparatus as described in any of embodiments 31 to 35, configured to infer that the position indicates a list of keys when the first syntax element has the second state, derive from the video data stream a third syntax element indicating a pointer to an entry in the list of keys, and derive a public key from the entry in the list of keys indicated by the pointer.

[0175] 37. The apparatus of any one of embodiments 1-36, configured to derive a hash function using third information derived from the data stream. 38. The apparatus of embodiment 37, wherein the third information includes a hash function identifier or a third pointer to a third location from which the hash function can be determined, or an identifier of the entity that encoded the video into the video data stream. 39. An apparatus as described in any one of embodiments 1-38, wherein the hash value depends on all bits of a predetermined portion of the video data stream. 40. An apparatus as described in any one of embodiments 1 to 39, wherein the hash value depends on all bits of a predetermined portion of the video data stream within the entropy-coded domain. 41. An apparatus as described in any of embodiments 1 to 40, wherein the predetermined portion of the video data stream spans more than one access unit of the video data stream such that the hash value depends on bits of more than one access unit.

[0176] 42. By checking whether the hash function is correct by checking whether the parameterization or identifier of the hash function matches the digital signature. An apparatus as described in any of embodiments 1 to 41, configured to further check the authenticity of the video data stream. 43. An apparatus described in any of embodiments 1 to 42, wherein the predetermined portion consists of one or more video coding layer portions of a video data stream in which motion vectors and intra prediction modes for predictive blocks and transform coefficients for residual blocks are coded. 44. Providing a hash value for checking the authenticity of a video data stream combined with a media stream, together with a further hash value obtained by subjecting a portion of the media stream accompanying the video data stream, or further data derived therefrom, to a hash function or a different hash function; An apparatus described in any one of embodiments 1 to 43.

[0177] 45. A first syntax element indicating the total number of non-zero transform coefficients in a transform block representing a residual block, and a trailing ones number indicating the number of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; one or more second syntax elements that indicate the signs of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; and one or more third syntax elements indicating values ​​of the non-zero transform coefficients, excluding the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order; and a fourth syntax element indicating the total number of zero-valued transform coefficient levels in the transform block following the first encountered non-zero transform coefficient in scan order; and one or more fifth syntax elements indicating the position of the non-zero transform coefficients along the scan order by indicating the number of consecutive zero-valued transform coefficients in the scan order between consecutively encountered non-zero transform coefficients. or decoding a significance flag indicating whether a non-zero transform coefficient is located at a current position in a forward scan traversing the transform coefficients of the transform block, and if so and if the current position is not the last in the forward scan, decoding a last significance flag indicating whether the non-zero transform coefficient located at the current position is the last non-zero transform coefficient in the forward scan, thereby decoding a significance map indicating the positions of the non-zero transform coefficients in the transform block representing the residual block; sequentially decoding the values ​​of the non-zero transform coefficients in a reverse scan order by reversing the forward scan order, thereby By using context-adaptive binary arithmetic decoding, By decoding the prediction residual data of the residual block from the video data stream, 45. The apparatus of any one of embodiments 1 to 44, wherein the apparatus is a decoder configured for decoding video from a video data stream by block-based predictive decoding and transform-based residual decoding.

[0178] 46. ​​Decode coordinates of a location within the transform block representing prediction residual data at which a last non-zero transform coefficient is encountered when traversing the transform coefficients of the transform block along a predetermined scan order; sequentially decoding values ​​of ranked transform coefficients included between the last non-zero transform coefficient and the first scanned transform coefficient along a predetermined scan order; By using context-adaptive binary arithmetic decoding, by selecting a predetermined scan order from among a diagonal scan order, a horizontal scan order, and a vertical scan order according to an intra prediction mode of the intra predicted block using a mapping that maps each of the 33 intra prediction modes to a corresponding diagonal scan order, a horizontal scan order, and a vertical scan order. By decoding the prediction residual data of the intra-predicted block from the video data stream, A decoder configured to decode video from a video data stream by block-based predictive decoding and transform-based residual decoding, An apparatus described in any one of embodiments 1 to 44.

[0179] 47. By using context-adaptive binary arithmetic coding / decoding of quantization indices of transform coefficients of a transform block representing a residual block and successive inverse quantization of quantization indices in which the value of a current transform coefficient depends on the parity of the quantization indices of the previous quantization indices. By decoding the prediction residual data of the residual block from the video data stream, 45. The apparatus of any one of embodiments 1 to 44, wherein the apparatus is a decoder configured for decoding video from a video data stream by block-based predictive decoding and transform-based residual decoding.

[0180] 48. A decoder 20 configured to decode video from a video data stream 14, the decoder comprising: Decrypting (63) the digital signature 43 from the video data stream; applying a predetermined portion 13 of the video data stream, or data 62 derived therefrom, to a hash function 31 to obtain a hash value 33; By checking whether the hash value matches the digital signature to determine whether the video data stream is authentic (41), configured to subject the digital signature to a check on the authenticity of the video data stream; The decoder a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing a residual block, and a trailing ones number indicating the number of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; one or more second syntax elements that indicate the signs of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; and one or more third syntax elements indicating values ​​of the non-zero transform coefficients, excluding the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order; and a fourth syntax element indicating the total number of zero-valued transform coefficient levels in the transform block following the first encountered non-zero transform coefficient in scan order; and one or more fifth syntax elements indicating the position of the non-zero transform coefficients along the scan order by indicating the number of consecutive zero-valued transform coefficients in the scan order between consecutively encountered non-zero transform coefficients. By decoding the prediction residual data of the residual block from the video data stream, A decoder configured for decoding video from a video data stream by block-based predictive decoding and transform-based residual decoding.

[0181] 49. A decoder as described in embodiment 48, wherein the digital signature is transmitted within a supplemental enhancement information message of the video data stream. 50. Checks on the authenticity of the video data stream decrypting the digital signature to obtain a check value; 50. A decoder as described in embodiment 48 or 49, comprising checking whether the hash value matches the check value. 51. Applying a predetermined portion of a video data stream, or data derived therefrom, to a hash function to obtain a hash value, reconstructing the video with respect to a predetermined portion to obtain a reconstructed portion of the video; 51. A decoder according to any one of embodiments 48 to 50, comprising: subjecting the reconstructed portion to a hash function.

[0182] 52. configured to decode a digital signature from a supplemental enhancement information message of a video data stream; A decoder according to any one of embodiments 48 to 51. 53. Checks on the authenticity of the video data stream A decoder as described in any of embodiments 48 to 52, comprising locating a predetermined portion within a video data stream by determining, through the use of one or more SEI messages interspersed within the video data stream, that the predetermined portion is a section of the video data stream that spans or extends between one or more SEI messages. 54. Configured to decrypt a digital signature from one of one or more SEI messages of a video data stream; A decoder as described in embodiment 53.

[0183] 55. Checks on the authenticity of the video data stream A decoder as described in any of embodiments 48 to 52, comprising locating a predetermined portion within a video data stream by using prefix SEI messages and suffix SEI messages interspersed within the video data stream to determine that the predetermined portion is a section of the video data stream that spans or is located between the prefix SEI messages and the suffix SEI messages. 56. The check on the authenticity of the video data stream includes locating a predetermined portion 13 in the video data stream by using prefix SEI messages 25 and suffix SEI messages 27 interspersed within the video data stream, determining that the predetermined portion is located between the prefix SEI message and the suffix SEI message; A decoder according to any one of embodiments 48 to 52. 57. The apparatus of embodiment 55 or 56 configured to decrypt a digital signature from a suffix SEI message.

[0184] 58. Configured to locate a predetermined portion within a video data stream by using a first SEI message and a second SEI message interspersed within the video data stream to determine the predetermined portion to be a section of the video data stream that spans or is located between the first SEI message and the second SEI message, or a section of the video data stream that spans between a point in the data stream located downstream of the first SEI message and the second SEI message; A decoder according to embodiment 54 or 55.

[0185] 59. An apparatus as described in embodiment 54 or 55, configured to locate a predetermined portion 13 within a video data stream by using a first SEI message 25 and a second SEI message 27 interspersed within the video data stream to determine that the predetermined portion is located between the first SEI message and the second SEI message, or between a point in the data stream located downstream of the first SEI message and the second SEI message.

[0186] 60. A decoder as described in embodiment 58 or 59, wherein the point in the data stream located downstream of the second SEI message is determined based on the length of the section or as being located at the end of the video coding layer portion having the access unit of the video data stream in which the second SEI message is contained. 61. The apparatus of any one of embodiments 58-60 configured to decrypt the digital signature from the second SEI message.

[0187] 62. A method for deriving a summary SEI message from a video data stream, the summary SEI message indicating one or more substreams of the video data stream, the method being capable of checking the reliability of the video data stream based on one or more portions within each individual substream; A check on the reliability of the video data stream is performed for a given substream of the subset, locating one or more portions within the video data stream by using one or more first SEI messages and second SEI messages interspersed within the video data stream to determine that the predetermined portion is a section of the video data stream formed by concatenating a subsection of the video data stream that follows each of the one or more first SEI messages and extends to a video coding layer end of the access unit that contains the respective first SEI message and a subsection that precedes the second SEI message and extends to a video coding layer start of the access unit that contains the second SEI message or that follows the second SEI message and extends to a video coding layer end of the access unit that contains the second SEI message; A decoder as described in any of embodiments 48 to 55, comprising performing a check on the video data stream regarding reliability for a subset of one or more substreams of the one or more substreams.

[0188] 63. A method for deriving a summary SEI message from a video data stream that indicates one or more substreams of the video data stream, the summary SEI message being capable of checking the reliability of the video data stream based on one or more portions within each individual substream; A check on the reliability of the video data stream is performed for a given substream of the subset, locating a portion of the given substream of the video data stream by using one or more first SEI messages interspersed within the video data stream to determine that the portion of the given substream is a section of the video data stream formed by concatenating subsections of the video data stream that follow each of the one or more first SEI messages and extend up to a video coding layer end of the access unit in which the respective first SEI message is contained; An apparatus as described in any of embodiments 48 to 55, comprising performing a check of the video data stream regarding reliability for a subset of one or more substreams of the one or more substreams.

[0189] 64. A method for deriving a summary SEI message from a video data stream that indicates one or more substreams of the video data stream, the summary SEI message being capable of checking the reliability of the video data stream based on one or more portions within each individual substream; The check on the reliability of the video data stream includes, for a picture unit of the video data stream, checking whether the video data stream includes a sub-stream selection SEI message for the picture unit; if the video data stream does not include a sub-stream selection SEI message for the picture unit, assigning video coded layer payload packets of the picture unit to a first sub-stream of the one or more sub-streams; If the video data stream includes a substream selection SEI message for the picture unit, by assigning video coded layer payload packets of the picture unit to one or more substreams, the substreams indicated by the substream selection SEI message; determining a respective portion of the video data stream for one or more substreams; An apparatus as described in any of embodiments 48 to 55, comprising performing a check of the video data stream regarding reliability for a subset of one or more substreams of the one or more substreams.

[0190] 65. The method of claim 1, wherein the check for reliability of the video data stream includes checking the video data stream for reliability with respect to a subset of the one or more substreams; An apparatus as described in any of embodiments 62 to 64, further comprising applying the determined portions for one or more substreams of the subset of one or more substreams to a hash function to derive respective hash values.

[0191] 66. An apparatus described in any of embodiments 62 to 65, further configured to determine the portion of each substream of a subset of one or more substreams by using a first SEI message 25 and a second SEI message 27 interspersed within the video data stream to determine that the portion is located between the first SEI message and the second SEI message, or between a point in the data stream located downstream of the first SEI message and the second SEI message. 67. An apparatus as described in any of embodiments 62 to 66, further configured to determine that each portion of a substream of a subset of one or more substreams terminates at or before one end of the encoded video sequence.

[0192] 68. One or more substreams have a defined order therein, and the apparatus is configured to determine that a given portion is part of a given substream of a subset of the one or more substreams; Checks on the authenticity of the video data stream whether the hash value and a further hash value obtained by applying a hash function to the previous portion of the predetermined portion, or further data derived from it, match the digital signature; or by checking whether a combined hash value derived by hashing the predetermined portion and a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, matches the digital signature; sequentially performing a check of the video data stream for authenticity for each portion of the video data stream derived for one or more substreams, An apparatus as described in any of embodiments 63 to 66, including the previous portion being before the given portion in a defined sequence order between the portions.

[0193] 69. Checks on the authenticity of the video data stream and whether the hash value and a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, match the digital signature; or by checking whether a combined hash value derived by hashing the predetermined portion and a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, matches the digital signature; sequentially performing a check of the video data stream for authenticity with respect to a plurality of portions of the video data stream; A decoder according to any one of embodiments 48 to 67.

[0194] 70. Checks on the authenticity of the video data stream combining the hash value with a further hash value to obtain a combined hash value and checking whether the combined hash value matches the digital signature; or combining the predetermined portion and a previous portion of the video data stream, or further data derived therefrom, by applying a hash function to the resulting further hash value to obtain a combined hash value, and checking whether the combined hash value matches the digital signature. 70. A decoder according to embodiment 68 or 69. 71. The decoder of embodiment 70, wherein the combination is a concatenation. 72. Checks on the authenticity of the video data stream checking whether the identifier in the hash value matches the digital signature to further determine whether the video data stream is authentic. A decoder according to any one of embodiments 68 to 71.

[0195] 73. Two different digital signatures are transmitted within the video data stream for a given portion; Checks on the authenticity of the video data stream If the video data stream contains a previous part for a given part, whether the hash value and a further hash value obtained by applying a hash function to an earlier portion of the video data stream, or further data derived therefrom, matches the first of two different digital signatures; or checking whether a combined hash value derived by hashing the predetermined portion and a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, matches a first of the two different digital signatures; If the video data stream does not contain any previous parts for a given part, By checking whether the hash value matches the second of two different digital signatures to determine whether the video data stream can be trusted. sequentially performing a check of the video data stream for authenticity with respect to a plurality of portions of the video data stream; A decoder according to any one of embodiments 48 to 72.

[0196] 74. Checks on the authenticity of the video data stream If the video data stream contains a previous part with respect to the given part, checking whether the hash value and a further hash value obtained by applying a hash function to a previous portion of the video data stream or further data derived therefrom match the digital signature, or whether a still further hash value, which is equal to a further hash value obtained by applying a hash function to a previous portion of the video data stream or further data derived therefrom, transmitted in the video data stream for the given portion, matches the digital signature; and checking whether a combined hash value derived by hashing the predetermined portion on the one hand and a further hash value obtained by applying a hash function to an earlier portion of the video data stream or further data derived therefrom on the other hand matches a first of the two different digital signatures, or whether a combined hash value derived by hashing the hash value and a still further hash value equal to a further hash value obtained by applying a hash function to an earlier portion of the video data stream or further data derived therefrom, which is transmitted in the video data stream for the predetermined portion, matches the digital signature; If the video data stream does not contain any previous parts for a given part, By checking if the hash value matches the digital signature to determine if the video data stream can be trusted. sequentially performing a check of the video data stream for authenticity with respect to a plurality of portions of the video data stream; A decoder according to any one of embodiments 48 to 72.

[0197] 75. Checks on the authenticity of the video data stream If the video data stream does not contain any previous parts for a given part, A decoder as described in embodiment 74, comprising determining whether the video data stream is trustworthy by checking whether the hash value matches the digital signature by checking whether the hash value and a further hash value transmitted within the video data stream for a given portion match the digital signature.

[0198] 76. The predetermined value is equal to the check value obtained by decrypting the digital signature or a predetermined portion of the check value related to the predetermined value; or the digital signature is matched by a predetermined value if the predetermined value is equal to a check value in a further hash field arrived at by a further hash function applied to the predetermined value or to a concatenation of values ​​including the predetermined value; A decoder according to any one of embodiments 48 to 75. 77. The check on the authenticity of the video data stream includes the use of an asymmetric decryption method using a public key; A decoder according to any one of embodiments 48 to 76.

[0199] 78. The decoder of embodiment 77 configured to derive an asymmetric decryption scheme using first information derived from the data stream. 79. A decoder as described in embodiment 78, wherein the first information includes a decryption scheme identifier or a first pointer to a first location from which an asymmetric decryption scheme can be determined, or an identifier of an entity that encoded the video into the video data stream. 80. A decoder according to any one of embodiments 48 to 79, configured to derive a public key using second information derived from the data stream.

[0200] 81. A decoder as described in embodiment 80, wherein the second information includes a second pointer to a second location from which the public key can be obtained, or an identifier of the entity that encoded the video into the video data stream. 82. An apparatus configured to derive a public key by deriving from a video data stream a first syntax element and a second syntax element indicating a pointer to a location from which the public key can be derived, comprising: if the first syntax element has a first state, inferring that the location identifies exactly one public key, deriving exactly one public key from the location; A decoder as described in any of embodiments 77 to 81, configured to infer that, if the first syntax element has a second state, the position indicates a list of keys and to derive from the video data stream a third syntax element indicating a pointer to an entry in the list of keys.

[0201] 83. A decoder according to any one of embodiments 48-82, configured to derive a hash function using third information derived from the data stream. 84. A decoder as described in embodiment 83, wherein the third information includes a hash function identifier or a third pointer to a third location from which the hash function can be determined, or an identifier of the entity that encoded the video into the video data stream. 85. A decoder according to any one of embodiments 48 to 84, wherein the hash value depends on all bits of a predetermined portion of the video data stream. 86. A decoder according to any one of embodiments 48 to 85, wherein the hash value depends on all bits of a given portion of the video data stream within the entropy-coded domain.

[0202] 87. A decoder described in any of embodiments 48 to 86, wherein a predetermined portion of the video data stream spans more than one access unit of the video data stream such that the hash value depends on bits of more than one access unit. 88. Checks on the authenticity of the video data stream checking whether the hash function is correct by checking whether the parameterization or identifier of the hash function matches the digital signature; A decoder according to any one of embodiments 48 to 87. 89. A decoder described in any of embodiments 48 to 88, wherein the predetermined portion consists of one or more video coding layer portions of a video data stream in which motion vectors and intra prediction modes for predictive blocks and transform coefficients for residual blocks are coded. 90. Providing a hash value for checking the authenticity of a video data stream combined with a media stream, together with a further hash value obtained by subjecting a portion of the media stream accompanying the video data stream, or further data derived therefrom, to a hash function or a different hash function; A decoder according to any one of embodiments 48 to 89.

[0203] 91. A decoder may include a first syntax element indicating the total number of non-zero transform coefficients in a transform block representing a residual block, and a trailing ones number indicating the number of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order. one or more second syntax elements that indicate the signs of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; and one or more third syntax elements indicating values ​​of the non-zero transform coefficients, excluding the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order; and a fourth syntax element indicating the total number of zero-valued transform coefficient levels in the transform block following the first encountered non-zero transform coefficient in scan order; one or more fifth syntax elements that indicate the position of the non-zero transform coefficients along the scan order by indicating the number of consecutive zero-valued transform coefficients in the scan order between consecutively encountered non-zero transform coefficients; or decoding a significance flag indicating whether a non-zero transform coefficient is located at a current position in a forward scan traversing the transform coefficients of the transform block, and if so and if the current position is not the last in the forward scan, decoding a last significance flag indicating whether the non-zero transform coefficient located at the current position is the last non-zero transform coefficient in the forward scan, thereby decoding a significance map indicating the positions of the non-zero transform coefficients in the transform block representing the residual block; sequentially decoding the values ​​of the non-zero transform coefficients in a reverse scan order by reversing the forward scan order, thereby By using context-adaptive binary arithmetic decoding, By decoding the prediction residual data of the residual block from the video data stream, Block-based predictive decoding and transform-based residual decoding configured to decode video from a video data stream; A decoder according to any one of embodiments 48 to 90.

[0204] 92. Decode coordinates of a location within the transform block representing prediction residual data at which a last non-zero transform coefficient is encountered when traversing the transform coefficients of the transform block along a predetermined scan order; sequentially decoding values ​​of ranked transform coefficients included between the last non-zero transform coefficient and the first scanned transform coefficient along a predetermined scan order; By using context-adaptive binary arithmetic decoding, by selecting a predetermined scan order from among a diagonal scan order, a horizontal scan order, and a vertical scan order according to an intra prediction mode of the intra predicted block using a mapping that maps each of the 33 intra prediction modes to a corresponding diagonal scan order, a horizontal scan order, and a vertical scan order. By decoding the prediction residual data of the intra-predicted block from the video data stream, configured to decode video from a video data stream by block-based predictive decoding and transform-based residual decoding; A decoder according to any one of embodiments 48 to 90.

[0205] 93. By using context-adaptive binary arithmetic decoding of quantization indices of transform coefficients of a transform block representing a residual block and successive inverse quantization of quantization indices in which the value of a current transform coefficient depends on the parity of the quantization index of the previous quantization index. By decoding the prediction residual data of the residual block from the video data stream, A decoder according to any one of embodiments 48 to 90, configured for decoding video from a video data stream by block-based predictive decoding and transform-based residual decoding.

[0206] 94. A device 15 for rendering a video data stream 14 in which video that can be checked for authenticity is encoded, the device comprising: applying a predetermined portion 13 of the video data stream, or data derived therefrom, to a hash function 31 to obtain a hash value 33; calculating a digital signature 43 based on the hash value 33 and digitally signing the hash value 33; configured to insert 77 a digital signature 43 into the video data stream 14, thereby making it possible to determine whether the video data stream is authentic by checking whether the hash value 33 matches the digital signature 43; a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing a residual block, and a trailing ones number indicating the number of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; one or more second syntax elements that indicate the signs of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; and one or more third syntax elements indicating values ​​of the non-zero transform coefficients, excluding the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order; and a fourth syntax element indicating the total number of zero-valued transform coefficient levels in the transform block following the first encountered non-zero transform coefficient in scan order; and one or more fifth syntax elements indicating the position of the non-zero transform coefficients along the scan order by indicating the number of consecutive zero-valued transform coefficients in the scan order between consecutively encountered non-zero transform coefficients. By encoding the prediction residual data of the residual block into the video data stream, Block-based predictive coding and transform-based residual coding A device that encodes video into a video data stream by encoding the video into a video data stream. 95. The apparatus of embodiment 94, configured to insert a digital signature into a supplemental enhancement information message of a video data stream. 96. Form a check value based on the hash value; configured to calculate a digital signature based on the hash value by encrypting the check value to obtain the digital signature; 96. A device as described in embodiment 94 or 95.

[0207] 97. Applying a predetermined portion of a video data stream, or data derived therefrom, to a hash function to obtain a hash value, reconstructing the video with respect to a predetermined portion to obtain a reconstructed portion of the video; An apparatus described in any of embodiments 94 to 96, comprising: subjecting the reconstructed portion to a hash function. 98. A device encodes video into a video data stream; an encoder configured to encode a digital signature into a video data stream; An apparatus described in any one of embodiments 94 to 97.

[0208] 99. The method is configured to encode a digital signature into a supplemental enhancement information message of a video data stream. The device described in embodiment 98. 100. By use of one or more SEI messages interspersed within the video data stream, locating a predetermined portion within the video data stream by determining the predetermined portion to be a section of the video data stream that spans or extends between one or more SEI messages; 100. An apparatus as described in embodiment 98 or 99. 101. A method for encoding a digital signature into one of one or more SEI messages of a video data stream; An apparatus as described in embodiment 100.

[0209] 102. Using prefix and suffix SEI messages interspersed within the video data stream, locating a predetermined portion within the video data stream by determining the predetermined portion to be a section of the video data stream that spans or is located between the prefix and suffix SEI messages; 100. An apparatus as described in embodiment 98 or 99.

[0210] 103. Using prefix SEI messages 25 and suffix SEI messages 27 interspersed within the video data stream, determining that the predetermined portion is located between the prefix SEI message and the suffix SEI message, thereby locating the predetermined portion within the video data stream; 100. An apparatus as described in embodiment 98 or 99. 104. The apparatus of embodiment 102 or 103 configured to encode a digital signature into a suffix SEI message.

[0211] 105. Using a first SEI message and a second SEI message interspersed within the video data stream, locating the predetermined portion within the video data stream by determining the predetermined portion to be a section of the video data stream that spans or is located between the first SEI message and the second SEI message, or a section of the video data stream that spans between the first SEI message and a point within the data stream that is located downstream of the second SEI message; 100. A device as described in embodiment 98 or 99.

[0212] 106. Using a first SEI message 25 and a second SEI message 27 interspersed within the video data stream, locating a predetermined portion within the video data stream by determining that the predetermined portion is located between the first SEI message and the second SEI message, or between the first SEI message and a point within the data stream located downstream of the second SEI message; 100. An apparatus as described in embodiment 98 or 99. 107. An apparatus as described in embodiment 106, which determines a point in the data stream located downstream of the second SEI message based on the length of the section or as being located at the end of a video coding layer portion having an access unit of the video data stream in which the second SEI message is contained. 108. The apparatus of embodiment 106 or 107 configured for encoding a digital signature into the second SEI message.

[0213] 109. Inserting into the video data stream a summary SEI message indicating one or more substreams of the video data stream, for each of which the video data stream can be checked for reliability based on one or more portions within an individual substream; For a given substream of the subset, locating a portion of a given substream by using one or more first and second SEI messages interspersed within the video data stream to determine that the portion of the given substream is a section of the video data stream formed by concatenating a subsection of the video data stream that follows each of the one or more first SEI messages and extends to a video coding layer end of the access unit that contains the respective first SEI message and a subsection that precedes the second SEI message and extends to a video coding layer start of the access unit that contains the second SEI message or that follows the second SEI message and extends to a video coding layer end of the access unit that contains the second SEI message; inserting the digital signature into the second SEI message; 100. An apparatus as described in embodiment 98 or 99.

[0214] 110. Inserting into the video data stream a summary SEI message indicating one or more substreams of the video data stream, for each of which the video data stream can be checked for reliability based on one or more portions within an individual substream; For a given substream of the subset, locating the portion of the given substream by use of one or more first SEI messages interspersed within the video data stream, by determining the portion of the given substream to be a section of the video data stream formed by concatenating subsections of the video data stream that follow each of the one or more first SEI messages and extend up to the video coding layer end of the access unit in which the respective first SEI message is contained; 100. An apparatus as described in embodiment 98 or 99.

[0215] 111. Checks on the authenticity of the video data stream are and further computing and digitally signing a digital signature based on the hash value and a further hash value obtained by subjecting a previous portion of the video data stream, or further data derived therefrom, to a hash function; or An apparatus as described in any of embodiments 94 to 110, wherein the method is performed sequentially on multiple portions of a video data stream by calculating a digital signature based on a combined hash value obtained by hashing a predetermined portion and a further hash value obtained by applying a hash function to a previous portion of the video data stream or further data derived from it, and digitally signing with the combined hash value.

[0216] 112. Combining the hash value with a further hash value to obtain a combined hash value, calculating a digital signature, and digitally signing the combined hash value; or configured to combine the predetermined value with a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, to obtain a combined hash value, and to calculate a digital signature and digitally sign the combined hash value. An apparatus as described in embodiment 111. 113. The device of embodiment 112, wherein the combination is a linkage. 114. The apparatus of any of embodiments 111-113, configured to calculate a digital signature based on an identifier of the hash function and further digitally sign therewith.

[0217] 115. A further hash value obtained by applying a hash function to the hash value and to a previous portion of the video data stream, or further data derived therefrom; or by computing a first of two different digital signatures based on a combined hash value derived by hashing the predetermined portion and the further hash value, computing a second of the same digital signatures based on the hash values, and digitally signing the combination of the hash value, the further hash value, and the first of the digital signatures; configured to insert two different digital signatures into the video data stream for predetermined portions; This allows checking the video data stream for authenticity. If the video data stream contains a previous part with respect to the given part, checking whether the hash value and the further hash value match a first of two different digital signatures, or whether a combined hash value obtained by applying a hash function to the predetermined portion and a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, matches a first of two different digital signatures; If the video data stream does not contain any previous parts for a given part, An apparatus as described in any of embodiments 94 to 114, wherein determining whether a video data stream is trustworthy is made possible by checking whether a hash value matches the second of two different digital signatures.

[0218] 116. configured to calculate a digital signature based on the hash value and a further hash value obtained by applying a hash function to a previous portion of the video data stream, or further data derived therefrom, or a combined hash value derived by hashing the predetermined portion and the further hash value, digitally signing the combination of the hash value and the further hash value, and inserting the digital signature into the video data stream for the predetermined portion, together with a yet further hash value equal to the further hash value; This allows checking the video data stream for authenticity. If the video data stream contains a previous part with respect to the given part, checking whether the hash value and the further hash value match the digital signature or whether the hash value and a still further hash value transmitted in the video data stream for the given portion match the digital signature and the still further hash value is equal to a still further hash value obtained by applying a hash function to a previous portion of the video data stream or further data derived therefrom; checking whether a combined hash value derived by hashing the predetermined portion on the one hand and the further hash value on the other hand matches the first of the two different digital signatures, or whether a combined hash value obtained by applying a hash function to the hash value and a further hash value transmitted in the video data stream for the predetermined portion matches the digital signature and the further hash value is equal to a further hash value obtained by applying the hash value to a previous portion of the video data stream or further data derived therefrom, If the video data stream does not contain any previous parts for a given part, This is done by checking whether the hash value matches the digital signature to determine whether the video data stream is authentic. An apparatus described in any one of embodiments 94 to 114.

[0219] 117. If the video data stream does not contain any previous parts for a given part, determining whether the video data stream is authentic by checking whether the hash value and any further hash values ​​transmitted within the video data stream for a given portion match the digital signature; An apparatus as described in embodiment 116.

[0220] 118. Forming a check value based on the hash value so that the hash value matches a predetermined portion of the check value, and encrypting the check value to obtain a digital signature; or forming a check value based on the hash value by further hashing the predetermined value or a concatenation of values ​​including the predetermined value, and encrypting the check value to obtain a digital signature; An apparatus described in any of embodiments 94 to 117, configured to calculate a digital signature based on a hash value.

[0221] 119. The apparatus of any of embodiments 94-118, configured to calculate a digital signature based on a hash value by using an asymmetric encryption scheme using a private key or a public key and a private key. 120. The apparatus of embodiment 119 configured for inserting first information from which an asymmetric encryption scheme can be derived into a data stream. 121. The device of embodiment 120, wherein the first information includes an encryption scheme identifier or a first pointer to a first location from which the asymmetric encryption scheme can be determined, or an identifier of the device. 122. An apparatus as described in any one of embodiments 94 to 121, configured to insert second information from which a public key can be derived into a video data stream. 123. The device of embodiment 122, wherein the second information includes a second pointer to a second location from which the public key can be obtained or an identifier of the device.

[0222] 124. An apparatus described in any of embodiments 119 to 123, configured to insert a first syntax element and a second syntax element into a video data stream, wherein the second syntax element indicates a pointer to a location from which a public key can be derived, the first syntax element having a first state and a second state, the first state indicating that the location identifies exactly one public key and the second state indicating that the location indicates a list of keys, and when the apparatus sets the first syntax element to the second state, configured to insert into the video data stream a third syntax element indicating a pointer to an entry in the list of keys.

[0223] 125. The apparatus of any one of embodiments 94-124 configured to insert third information from which a hash function can be derived into the data stream. 126. The device of embodiment 125, wherein the third information includes a hash function identifier or a third pointer to a third location from which the hash function can be determined, or an identifier of the device. 127. The apparatus of any one of embodiments 94-126, wherein the hash value depends on all bits of a predetermined portion of the video data stream. 128. An apparatus as described in any of embodiments 94-127, wherein the hash value depends on all bits of a predetermined portion of the video data stream within the entropy-coded domain.

[0224] 129. An apparatus described in any of embodiments 94 to 128, wherein a predetermined portion of the video data stream spans more than one access unit of the data stream such that the hash value depends on bits of more than one access unit. 130. Further calculate a digital signature based on the parameterization of the hash function or the identifier; An apparatus described in any of embodiments 94 to 129, thereby digitally signing. 131. An apparatus described in any of embodiments 94 to 130, wherein the predetermined portion consists of one or more video coding layer portions of a video data stream in which motion vectors and intra prediction modes for predictive blocks and transform coefficients for residual blocks are coded.

[0225] 132. A first syntax element indicating the total number of non-zero transform coefficients in a transform block representing a residual block, and a trailing ones number indicating the number of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; one or more second syntax elements that indicate the signs of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; and one or more third syntax elements indicating values ​​of the non-zero transform coefficients, excluding the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order; and a fourth syntax element indicating the total number of zero-valued transform coefficient levels in the transform block following the first encountered non-zero transform coefficient in scan order; one or more fifth syntax elements that indicate the position of the non-zero transform coefficients along the scan order by indicating the number of consecutive zero-valued transform coefficients in the scan order between consecutively encountered non-zero transform coefficients; or encoding a significance map indicating the locations of non-zero transform coefficients in a transform block representing a residual block by encoding a significance flag indicating whether a non-zero transform coefficient is located at a current position in a forward scan traversing the transform coefficients of the transform block, and if so and if the current position is not the last in the forward scan, encoding a last significance flag indicating whether the non-zero transform coefficient located at the current position is the last non-zero transform coefficient in the forward scan; sequentially encoding the values ​​of the non-zero transform coefficients in a reverse scan order by reversing the forward scan order; By using context-adaptive binary arithmetic coding, By encoding the prediction residual data of the residual block into the video data stream, Block-based predictive coding and transform-based residual coding An apparatus described in any of embodiments 94 to 131, which is an encoder configured to encode video into a video data stream.

[0226] 133. Encoding coordinates of a location within the transform block representing prediction residual data at which a last non-zero transform coefficient is encountered when traversing the transform coefficients of the transform block along a predetermined scan order; sequentially encoding the values ​​of ranked transform coefficients included between the last non-zero transform coefficient and the first scanned transform coefficient along a predetermined scan order; selecting a predetermined scan order from among a diagonal scan order, a horizontal scan order, and a vertical scan order according to an intra prediction mode of the intra predicted block, using a mapping that maps each of the 33 intra prediction modes to a corresponding diagonal scan order, a horizontal scan order, and a vertical scan order; By using context-adaptive binary arithmetic coding, By encoding the prediction residual data of intra-predicted blocks into the video data stream, block-based prediction coding and transform-based residual coding An apparatus described in any of embodiments 94 to 132, which is an encoder configured to encode video into a video data stream.

[0227] 134. By using context-adaptive binary arithmetic coding of quantization indices of transform coefficients of a transform block representing a residual block and sequential quantization of the transform coefficients such that a quantizer quantizing a current transform coefficient obtains a quantization index that depends on the parity of the quantization index of the previous quantization index; By encoding the prediction residual data of the residual block into the video data stream, An apparatus described in any of embodiments 94 to 131, which is an encoder configured to encode video into a video data stream by block-based predictive coding and transform-based residual coding.

[0228] 135. A method (160) for checking the authenticity of a video data stream (14) in which video is encoded, comprising: The device, applying a hash function 31 to a predetermined portion of the video data stream, or data derived therefrom, to obtain a hash value 33 (131), and deriving a digital signature 43 from the video data stream (161); configured to check whether the hash value 33 matches the digital signature 43 to determine (141) whether the video data stream is authentic; a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing a residual block, and a trailing ones number indicating the number of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; one or more second syntax elements that indicate the signs of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; and one or more third syntax elements indicating values ​​of the non-zero transform coefficients, excluding the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order; and a fourth syntax element indicating the total number of zero-valued transform coefficient levels in the transform block following the first encountered non-zero transform coefficient in scan order; and one or more fifth syntax elements indicating the position of the non-zero transform coefficients along the scan order by indicating the number of consecutive zero-valued transform coefficients in the scan order between consecutively encountered non-zero transform coefficients. By encoding the prediction residual data of the residual block into the video data stream, Block-based predictive coding and transform-based residual coding A method for encoding video into a video data stream by encoding the video into a video data stream.

[0229] 136. A method (200) for decoding video from a video data stream (14), comprising: Decrypting (163) the digital signature 43 from the video data stream; applying a hash function 31 to a predetermined portion 13 of the video data stream, or data 62 derived therefrom, to obtain a hash value 33 (131); By checking if the hash value matches the digital signature to determine if the video data stream is trustworthy 141 configured to subject the digital signature to a check on the authenticity of the video data stream; The method is a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing a residual block, and a trailing ones number indicating the number of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; one or more second syntax elements that indicate the signs of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; and one or more third syntax elements indicating values ​​of the non-zero transform coefficients, excluding the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order; and a fourth syntax element indicating the total number of zero-valued transform coefficient levels in the transform block following the first encountered non-zero transform coefficient in scan order; and one or more fifth syntax elements indicating the position of the non-zero transform coefficients along the scan order by indicating the number of consecutive zero-valued transform coefficients in the scan order between consecutively encountered non-zero transform coefficients. By decoding the prediction residual data of the residual block from the video data stream, A method comprising decoding video from a video data stream by block-based predictive decoding and transform-based residual decoding.

[0230] 137. A method 15 for rendering a video data stream 14 in which video that can be checked for authenticity is encoded, comprising: applying a hash function 31 to a predetermined portion 13 of the video data stream, or data derived therefrom, to obtain a hash value 33 (131); Calculating a digital signature 43 based on the hash value 33 and digitally signing 171 the hash value 33; configured to insert a digital signature 43 into the video data stream 14, thereby making it possible to determine (177) whether the video data stream is authentic by checking whether the hash value 33 matches the digital signature 43; a first syntax element indicating a total number of non-zero transform coefficients in a transform block representing a residual block, and a trailing ones number indicating the number of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; one or more second syntax elements that indicate the signs of non-zero transform coefficients that have an absolute value of one when traversing the coefficients along the scan order; and one or more third syntax elements indicating values ​​of the non-zero transform coefficients, excluding the number of non-zero transform coefficients having an absolute value of one when traversing the coefficients along the scan order; and a fourth syntax element indicating the total number of zero-valued transform coefficient levels in the transform block following the first encountered non-zero transform coefficient in scan order; and one or more fifth syntax elements indicating the position of the non-zero transform coefficients along the scan order by indicating the number of consecutive zero-valued transform coefficients in the scan order between consecutively encountered non-zero transform coefficients. By encoding the prediction residual data of the residual block into the video data stream, Block-based predictive coding and transform-based residual coding A method for encoding video into a video data stream by encoding the video into a video data stream. 138. A video data stream generated by the method of embodiment 137.

[0231] <Media data stream authenticity check> Further optional details and features of the above-described embodiments are described below. Furthermore, additional aspects and advantages that can be optionally applied to the above-described embodiments are described. One aspect described below is that a media data stream may include multiple substreams for different types of media that can complement each other, such as video, audio, and text, e.g., subtitles. One aspect described below in particular is the concept of checking authenticity across multiple media streams, e.g., checking a multimedia stream that includes multiple media streams for authenticity. However, the following description is not limited to such multimedia streams, but also relates to authenticity checks of the media stream itself. In particular, in the following, the terms "multimedia stream" or "multimedia content" may relate to any stream or content that includes one or more of the specified types of media. For example, the above embodiments can be understood to serve to illustrate possible examples for processing a video media substream among the media substreams of a multimedia data stream described below, and to that extent, all these details shall be treated as possible extensions of the embodiments described below for checking the authenticity of a multimedia data stream.

[0232] The following description highlights the challenges posed by AI-enabled deepfakes in the realm of multimedia content, including text, images, audio, and video. Because these sophisticated forgeries blur the line between reality and fabrication, they raise significant cybersecurity and trust issues within the digital ecosystem. Embodiments address the urgent need for reliable verification methods to counter the threats posed by deepfakes. The following description also provides a rationale for implementation and outlines possible technical approaches for authentication of digital content. Multimedia content, including text, images, audio, and video, supported globally by ITU-T standards for trusted exchange, is essential to modern life. However, rapid advances in AI have given rise to the creation of sophisticated deepfakes, blurring the line between real and fake content and raising serious cybersecurity and copyright concerns. These AI-generated deepfakes, which are increasingly difficult to detect, pose threats to personal and national security by being used for fraud and deception, mimicking identities and stealing credentials. The evolving capabilities of generative AI exacerbate these challenges, undermining trust and security on digital platforms and impacting societies worldwide, particularly those with limited digital literacy.

[0233] The International Telecommunication Union (ITU), an agency of the United Nations, brings together member states, businesses, academic societies, and various institutions from around the world. Its core mission is to foster the development and dissemination of practical and innovative solutions in information and communication technology (ICT) that advance the common good. For example, embodiments may be implemented in new standards or normative amendments to existing ITU-T Recommendations to provide techniques for authentication of multimedia content.

[0234] Below, we provide a brief technical description of a method for authenticating any digital content, including video, images, voice, audio, or text. The described approach is considered to serve as an example embodiment. <Technical explanation of the mechanism for verifying the reliability of multimedia> For example, embodiments aim to enhance verification of the authenticity of multimedia content, allowing users to verify the authenticity of content from creators such as governments, corporations, or news organizations. Embodiments are based on digital signatures of data streams. For example, a content creator (encoder) signs the content using a private key, while a recipient (decoder) verifies the authenticity using a corresponding public key. In examples, the public key required for verification is not derived directly from the data stream but is obtained through a trusted, independent method, such as a third-party trust center. For example, this process can utilize ITU Recommendation X.509 for securely obtaining a digital certificate that validates the encoder's identity. Additionally, the encoder and decoder can agree on a cryptographic hash function for computing a unique digital signature over a specified range of bytes in the data stream. Verification occurs when the decoder uses the public key to successfully match the digital signature with the calculated hash value, establishing the authenticity of the content.

[0235] Below is an example of a technical solution to one problem that falls within the scope of the proposed work item. More precisely, it addresses the problem of authenticity verification. For example, the proposed technology should allow a user to verify that the multimedia content they consume was in fact produced by the entity that identified itself in the multimedia data being provided. Such an entity could be, for example, a government or public institution, a company, a specific person, or a news organization. For example, embodiments may be built on the basic technical principle of digitally signing a data stream. For example, in general terms, this may work as follows: The encoder, i.e., the entity generating the data stream, possesses the private (and public) key for the fixed signature algorithm. The decoder, i.e., the entity attempting to verify the data stream generated from the encoder, possesses only the public key, not the private key. The data stream itself may contain a pointer to the public key, but the public key itself must be obtained by invoking a trusted, independent method, not from the data stream alone. For example, the data stream may contain information that identifies the encoder as a particular entity. Given this information, the decoder can obtain the digital certificate (public key and all other parameters required for verification) corresponding to this entity from a third-party trust center, where the relevant data can be transmitted, for example, by implementing ITU Recommendation X.509. One example is for the decoder to obtain the digital certificate from a source provided by the ITU, where interested content providers (encoders) can register their digital certificates.

[0236] Furthermore, the encoder and decoder agree on a cryptographic hash function, where the hash function may be fixed in advance, or obtained in the same way as the public key from the bitstream and a third-party trust center, or obtained from the datastream, where it may be part of a digital signature. The decoder then determines from the datastream unique byte ranges for which a cryptographic hash value will be calculated using the given hash function. Finally, a digital signature is sent in the data stream that can be considered an alleged digital signature of the hash value calculated over a given byte range. To check its authenticity, the decoder processes this digital signature using a given public key. If the result of this processing turns out to be a digital signature of the calculated hash value of a byte range, the decoder can consider this byte range as information that reliably belongs to the entity associated with the given public key, but should consider it fake if the opposite is true.

[0237] The overall process described here is illustrated in FIG. Thus, what has been described so far is a general concept for validating a media data stream. Below, we describe an embodiment that extends this concept to validating a multimedia stream that includes multiple media data streams, for example, multiple video data streams, audio data streams, and text data streams.

[0238] Enabling joint validation of multiple data streams In a typical multimedia application, the data stream to be verified may consist of several substreams, each generated by a different codec for a specific type of data, and assembled together at a higher level, e.g., at the system level. An example is the combination of visual data (image or video signal), sound (audio signal), and captions (text signal). Here, the video data can be video data encoded using the H.264 / AVC standard, the audio data can be encoded using the MPEG-4 AAC-LC (ISO / IEC 14496-3) standard, and the captions can be encoded via a text format such as ASCII or UTF-8 (ISO / IEC 10646, Unicode). If a user intends to consume the combination of all substreams and is interested in authenticity verification, a method for jointly authentically verifying all these substreams is required. On the other hand, it is possible that a user intends to consume only some of the substreams, resulting in only using some of the substreams. For example, a native speaker may not require captions. In another example, if the data consists of recorded audio, some users may only want to hear the audio signal, while in other applications, some users may only want to consume the visual signal. Embodiments provide techniques that allow joint verification of a flexible number of data substreams. For example, to this end, unencrypted, i.e., directly readable, versions of one or more hash values ​​associated with each of the substreams can be made part of the higher-level information.

[0239] Furthermore, for example, a joint digital signature corresponding to the claimed digital signature of all specified concatenations of these hash values ​​can be made part of the information at the higher level, along with information about the public key used, as described in the previous section. The user then performs a verification process in two steps. First, using the public key, the user verifies that the concatenation of hash values ​​matches the claimed digital signature. This verification process does not require the user to use the actual substreams, since all information necessary for such verification is provided at the higher level. Next, for each substream that the user actually intends to consume, and therefore to verify, the user calculates hash values ​​from these substreams, as in the previous section, and checks whether these hash values ​​are equal to the hash values ​​that are part of the information at the higher level. Note that because the digital signature provided at the higher level is a joint signature of all substreams, only an entity possessing the private key for the applied signature algorithm can combine multiple substreams into one that is verified as reliable by the proposed method. An overview of the method presented in this section is given by Figure 20. For example, according to an embodiment, for each codec used for different types of data, a specific way in which hash values ​​are generated for each substream is specified, and individual modifications to each specification are generated to be developed so that the specific requirements in each case (e.g., flexible multi-channel configurations for audio signals, support for temporal scalability for video signals, etc.) are met.

[0240] Enabling temporal consistency across multiple temporal segments To ensure that a decoder can verify the continuity between different temporal segments of multimedia data and to prevent attacks based on removing, inserting, or shuffling data chunks within the protected data stream, according to an embodiment, the hash values ​​of the preceding temporal segments are also incorporated into the digital signature of the current temporal segment, as described above, e.g., with reference to FIG. 12. Therefore, if the multimedia data contains multiple temporal segments, the hash values ​​of the individual substreams belonging to the previous temporal segment should also be included as information for the current segment, and this information should be included unencrypted. Furthermore, the digital signature of the previous temporal segment should actually be a joint digital signature of both the current and previous temporal segments. A user wishing to verify temporal integrity with the previous segment then checks that the hash values ​​that can be calculated for each substream of the previous segment actually match the hash values ​​that are part of the information contained in the upper layer of the current temporal segment. Note that because the digital signature of the current segment is a joint signature, only an entity possessing a private key can combine multiple substreams into one that is verified as reliable by the proposed method. An example of the proposed method for verifying temporal integrity is shown in FIG. 21.

[0241] In the following, what has been described above with respect to joint verification of multimedia data streams will be restated in more general terms, and any of the aforementioned features may optionally be combined with any of the embodiments described below. FIG. 22 illustrates an apparatus 8 for checking a multimedia data stream 9 for authenticity, according to one embodiment. From the multimedia data stream 9, the apparatus 8 derives substream information 89 indicating that the multimedia data stream includes or is composed of multiple media substreams, each of which has a media signal encoded therein, such as media substreams 17 and 17′ shown in FIG. 22. The multiple media substreams include a set of one or more first media substreams, exemplified by the single first media substream 17 in FIG. 21, and one or more removable media substreams, exemplified by the single removable media substream 17′ in FIG. 21. The multimedia data stream 8 further includes a separate transmitted hash value 94 for each of the one or more removable media substreams. The multimedia data stream 9 may, in an example, include one or more of the one or more removable substreams. In other words, a removable substream need not necessarily be included in the data stream 9 in order to be removable. This situation is illustrated in FIG. 22 by showing the removable substream 14' in dashed lines.

[0242] For example, the multiple media substreams may include one or more video data streams, one or more audio data streams, and / or one or more text data streams. For example, a removable substream may be a text stream, or one of one or more audio streams, or a video stream.

[0243] For each of a set of one or more first media substreams 14 included in the multimedia data stream, the device 8 subjects the first predetermined portion 13 of the respective first media substream, or first data 62 derived therefrom, to a respective first hash function 31 to obtain a respective calculated hash value 33. For example, the determination of the predetermined portion 13 and the generation of the hash value 33 may be performed as described above with respect to Figures 1 to 19 for the data stream 14, which may be an example of any of the first media substreams 17 and / or removable substreams 17'. The multimedia data stream 9 further comprises a common digital signature 91. For example, the common digital signature 91 and the transmitted hash value 94 may be part of the signature information signaled in the multimedia data stream 9.

[0244] The device 8 comprises a verification module 90 that checks whether the set of one or more first media substreams 17 can be trusted. The verification module 90 performs this check using a common digital signature 91, an individual calculated hash value 33 for each of the set of one or more first media substreams, and an individual transmitted hash value 94 for at least one of the one or more removable media substreams. In other words, the first set of media substreams can be defined as a set of substreams among the plurality of media substreams that should be checked for reliability. For example, the first set of media substreams can include one or more removable substreams and / or one or more non-removable substreams. In other words, the first set of media substreams can include one or more of the one or more removable substreams. For example, at least one of the one or more removable media substreams for which the transmitted hash value is used is a removable substream, and the removable substream should not be checked for authenticity in the sense of checking the authenticity of the removable substream. However, the hash value of this substream may be needed to check the authenticity of the substream combination, and the transmitted hash value may be used for this purpose. For example, at least one of the one or more removable substreams may or may not be present in data stream 9.

[0245] For example, the transmitted hash value 94 may be generated by subjecting a predetermined portion of the respective removable substream to a hash function, such as hash function 31. For example, the predetermined portion may be determined as described for the first media substream 17. For example, the authenticity check of the first set of media substreams can be performed as follows: A verification module 90 can form a verification string or check value and check whether the verification string matches a common digital signature 91, for example using a public and private key pair of an asymmetric cryptography scheme. For example, the generation of the common digital signature 91 may be performed on the encoder side by forming a check value and signing it using the private key. For example, the signature may include a further hashing, i.e. hashing the check value using a further hash function to obtain a further hash value, and signing the further hash value. In this example, it may not be possible to reconstruct the check value from the digital signature on the decoder side, but instead it is only possible to check whether the check value formed using the hash value matches the digital signature. In this case, verification by the verification module 90 may include hashing the check value using a further hash function to obtain a further hash value, and checking whether the further hash value matches the digital signature, for example by decrypting the digital signature using the public key and checking whether the resulting decrypted further hash value is equal to the further hash value.

[0246] The verification module 90 can form the verification string using a respective hash value for each of the multiple media substreams indicated by the substream information 89. Regarding which hash value the verification module uses for each substream, either calculated or transmitted, several variations may be possible. A first variant will be described with reference to Figure 20. For example, according to this variant, no distinction is necessary between removable and non-removable substreams, and for example, all of the multiple media substreams may be treated equally.

[0247] In the example of FIG. 20 , the multiple media substreams include media data streams denoted using reference symbols 17<1, 17<2, and 17<3, and multimedia data stream 9 includes, for each media data stream, a separate transmitted hash value denoted using reference symbols 94<1, 94<2, and 94<3. For each of media data streams 17<1, 17<2, and 17<3, device 8 may calculate a separate hash value denoted using reference symbols 33<1, 33<2, and 33<2, for example, as described with respect to first media substream 17 above. According to this embodiment, verification string 97 is formed by concatenating the transmitted hash values, and verification string 97 is checked against digital signature 43 using public key 79 (see block 108 of FIG. 20 ). If this check verifies transmitted hash value 94 as authentic, transmitted hash value 94 may be used to verify calculated hash value 33 (see block 102 of FIG. 20 ).

[0248] In other words, common signature 91, in combination with transmitted hash value 94, allows the combination of substreams to be verified because the transmitted hash values ​​are jointly signed to obtain signature 91. In addition, the transmitted hash values ​​allow the individual substreams to be verified. Even if a substream is removed, the other substreams can still be verified because data stream 9 contains the transmitted hash values ​​of all substreams. Thus, according to one embodiment of the first variant, the data stream 9 includes a separate transmitted hash value for each of the multiple media substreams indicated by the substream information 89 .

[0249] The second variant takes advantage of the situation where some substreams may be treated as non-removable; in other words, the common digital signature 91 may be generated under the assumption that one or more of the substreams are not removable in order to maintain checkability regarding the authenticity of the multimedia data stream 9. Since such substreams are always available for authenticity checks, verification of the substream combination, e.g., block 108 of FIG. 20, can be based on the calculated hash value 33 instead of the transmitted hash value for the non-removable media substreams. Therefore, transmission of the transmitted hash may be omitted for these substreams. Therefore, the verification string 97 to be subjected to the check 108 against the common signature 91 may be formed by using, for non-removable media substreams, calculated hash values ​​derived by applying a hash function to the respective portions of the non-removable media substreams, and by using the transmitted hash value for the removable substreams. Alternatively, if a calculated hash value is available for the removable substream, the calculated hash value may be used for the verification substream 97. In other words, according to one embodiment, the verification substream 97 is formed by concatenating the calculated hash value derived for the first media substream and the transmitted hash value for the removable substream.

[0250] According to an alternative embodiment, the verification substream 97 is formed by concatenating a calculated hash value derived for the first media substream, calculated hash values ​​derived for one or more of the removable substreams present in the multimedia data stream 9, if any, and transmitted hash values ​​for the removable substreams not present in the multimedia data stream 9, if any.

[0251] According to one embodiment, the device 8 selects, for each of the one or more removable media substreams, the set of one or more first media substreams based on the presence of a respective removable media substream, such that the set of one or more first media substreams consists of any removable media substreams present in the multimedia data stream and any media substreams that do not belong to the one or more removable media substreams, e.g. any non-removable substreams. According to one embodiment, the media signal encoded into a plurality of media substreams comprises one or more video signals and / or one or more audio signals and / or one or more text signals.

[0252] In the following, embodiments are described in which the public key can be used to derive the check value from the common digital signature 91. In other words, according to these embodiments, the verification module 90 can derive the check value by decrypting the common signature 91 using the public key. According to one embodiment, the verification module 90 checks whether the set of one or more first media substreams can be trusted by checking whether, for each of the set of one or more first media substreams, the individual calculated hash value 33, or, if the individual first media substream is a removable media substream, the individual transmitted hash value 94, and, for each of at least one of the one or more removable media substreams, the individual transmitted hash value, matches mutually distinct portions of a check value derived from the common digital signature by checking equality between the individual transmitted hash value and the individual calculated hash value for each of the first media substreams (if any) included in the check for match with mutually distinct positions where the individual transmitted hash values ​​differ from each other.

[0253] According to one embodiment, the verification module 90 is configured to, in checking whether the set of one or more first media substreams is trustworthy, subject, for each of the set of one or more first media substreams, an individual calculated hash value, or, if the individual first media substream is a removable media substream, an individual transmitted hash value, and, for each of at least one of the one or more removable media substreams, the individual transmitted hash value, to further hashing to obtain a common hash value, and for each of the first media substreams (if any) included in the subjecting the individual transmitted hash value to further hashing, check whether the common hash value matches a check value derived from the common digital signature by checking equality between the individual transmitted hash value and the individual calculated hash value. According to one embodiment, the validation module 90 is configured to derive a check value from the common digital signature by decrypting the common digital signature.

[0254] In the following, embodiments are described that can provide verification of the temporal integrity of one or more of the media substreams, e.g., as described above with respect to the portion-by-portion authenticity check for data stream 14. Features related to this aspect can optionally be combined with any of the embodiments described above. According to one embodiment, at least one media substream of the one or more first media streams 17 is a segmented media substream in which individual media signals are temporally segmented and encoded. According to this embodiment, the device 8 locates within the predetermined segment a first predetermined portion that will be subjected to a hash function to calculate the calculated hash value 33. In other words, the predetermined portion is within the predetermined segment.

[0255] For example, to verify temporal integrity, the verification module 90 may include a separate additional hash value in the verification string or check value for one or more or all of the first media substreams. The additional hash value for one of the media substreams may be a hash value of a temporal segment that precedes or precedes the predetermined segment, for example, the immediately preceding temporal segment. For example, the additional hash value to be included in the verification string may be calculated using a further predetermined portion of the first media substream located in the previous segment, or the transmitted hash value, which may be referred to as the transmitted hash value for temporal integrity.

[0256] An example is shown in Figure 20 above. The temporal hash value pair for the current temporal segment 6 may include a hash value 33 derived from the current temporal segment and a further hash value 33' of a previous temporal segment 6'. According to one embodiment, device 8 derives from data stream 9, for a segmented media substream, a transmitted hash value 94 for a given segment and a further transmitted hash value, e.g., called a transmitted hash value for temporal integrity. If no previous segment for a given segment exists in the multimedia data stream, verification module 90 uses the transmitted hash value for temporal integrity for the segmented media substream to check whether the first set of media substreams is authentic. In other words, if no previous segment exists, the hash value used for the segmented substream to form the check value may be the transmitted hash value for temporal consistency.

[0257] For example, the transmitted hash value for temporal consistency may be a joint hash value of a given segment and a previous segment of a segmented substream, formed, for example, by hashing the concatenation of respective hash values ​​derived by applying a hash function to each portion of the given segment and the previous segment.

[0258] According to one embodiment, if a previous segment for a given segment exists in the multimedia data stream, device 8 may obtain a further calculated hash value for the segmented media substream by subjecting a further predetermined portion of the segmented media substream, or further data derived therefrom, to a respective first hash function for the segmented media substream to locate the further predetermined portion of the segmented media substream within the previous predetermined segment. According to this embodiment, verification module 90 may additionally use the further calculated hash value for the segmented media substream to check whether the set of one or more first media substreams is trustworthy. In other words, if a previous segment exists, a further hash value can be calculated that can be used for verification.

[0259] As already mentioned above, the verification of the check value may involve the use of a further hash function to hash a check value or verification string formed using the calculated or transmitted hash value of the substream. According to one embodiment, the verification module 90 subjects, for each of a set of one or more first media substreams, the individual calculated hash value, or, if the individual first media substreams are removable media substreams, the individual transmitted hash value, or, if the individual first media substreams are segmented substreams, the transmitted hash value for temporal integrity, for each of at least one of the one or more removable media substreams, to further hashing to obtain a common hash value, and checks whether the common hash value matches a check value derived from the common digital signature.

[0260] Additionally, according to this embodiment, verification of the individual substreams of the set of first media substreams may be performed by checking whether the individual transmitted hash value or the transmitted hash value for temporal integrity matches the individual calculated hash value for each first media substream (if any) included in subjecting the individual transmitted hash value or the transmitted hash value for temporal integrity to further hashing. The latter check may be performed by checking the equality of the respective values. Alternatively, in the case of the transmitted hash value for temporal integrity, it may be performed by checking the equality with a hash value obtained by hashing the concatenation of the calculated hash values ​​for the given segment and the previous segment. For example, for an embodiment in which the check value is derivable from the digital signature 91 using the public key, the temporal integrity can be verified as follows.

[0261] According to one embodiment, the verification module 90 checks whether the set of one or more first media substreams can be trusted by checking whether, for each of the set of one or more first media substreams, the individual calculated hash value, or, if the individual first media substream is a removable media substream, the individual transmitted hash value, or, if the individual first media substream is a segmented substream, the transmitted hash value for temporal integrity, for each of at least one of the one or more removable media substreams, matches mutually distinct portions of a check value derived from the common digital signature by checking equality between the individual transmitted hash value or the transmitted hash value for temporal integrity and the individual calculated hash value for each of the first media substreams (if any) included in the check for match with mutually distinct positions where the individual transmitted hash value or the transmitted hash value for temporal integrity differs from each other.

[0262] It should be noted that any of the aspects described with respect to authenticity checking of data stream 14 may optionally be applied to authenticity checking of one or more substreams of multimedia substream 17. Some aspects are described below in terms of substreams. It should be understood that additional features described above with respect to device 16 may optionally be applied to device 8. For example, the reliability check may be performed on the raw data stream or on a reconstructed media signal or media stream reconstructed from the individual substreams. According to one embodiment, each first hash value depends on all bits of a respective first predetermined portion. According to one embodiment, the device 8 is configured to reconstruct the media signal with respect to the first predetermined portion to obtain a reconstructed media portion, and to subject the reconstructed media portion to the respective first hash function, where the first predetermined portion, or data derived therefrom, is subjected to the respective first hash function to obtain a first hash value. According to one embodiment, device 8 is a multimedia decoder configured to decode a media signal of each of a first set of one or more media substreams from a respective first media substream of the multimedia data stream and to decode a common digital signature from the video data stream. For example, device 8 may correspond to a decoder 20 which, in addition to being a video decoder, may be capable of decoding one or more of an audio data stream and a text data stream.

[0263] According to one embodiment, the device 8 can perform the check by using an asymmetric decryption scheme using a public key. According to one embodiment, device 8 may derive an asymmetric decryption scheme using first information derived from the data stream. According to one embodiment, the first information includes a decryption scheme identifier or a first pointer to a first location from which the asymmetric decryption scheme can be determined, or an identifier of an entity that encoded the video into the video data stream.

[0264] According to one embodiment, the device 8 is configured to derive the public key using second information derived from the data stream. According to one embodiment, the second information includes a second pointer to a second location where the public key can be obtained, or an identifier of the entity that encoded the video into the video data stream. According to one embodiment, the device 8 is configured to derive a respective first hash function using third information derived from the data stream. According to one embodiment, the third information includes a hash function identifier or a third pointer to a third location from which the hash function can be determined, or an identifier of the entity that encoded the video into the video data stream. According to one embodiment, the common digital signature 91 is transmitted in a header or information box or at an upper layer of the multimedia data stream outside the multiple media substreams.

[0265] 23 illustrates an apparatus 7 for rendering a multimedia data stream 9 that can be checked for authenticity. The apparatus 7 further comprises an inserting unit 99 configured to insert into the multimedia data stream 9 substream information 89 indicating that the multimedia data stream is composed of multiple media substreams, e.g., media substreams 17, 17′ shown in FIG. 23 , each having a media signal encoded therein. The apparatus 7 is configured to, for each of the multiple media substreams, subject a first predetermined portion 13, 13′ of the respective media substream, or first data 62, 62′ derived therefrom, to a respective first hash function 31 to obtain a respective calculated hash value 33′, and to encrypt signature information into the multimedia data stream 9. The signature information includes a common digital signature 91 and, for each of one or more removable media substreams 13′ of the multiple media substreams, a respective transmitted hash value equal to the calculated hash value 33′ of the respective removable media signal. The apparatus 7 includes a signing module 98 further configured to sign the multimedia data stream by calculating 98 a common digital signature 91 using the individual calculated hash values ​​for each of the plurality of media substreams. For example, the signing module forms a concatenation of the calculated hash values ​​and signs the concatenation, e.g., by hashing and signing it. It should be noted that any description of apparatus 8 may, optionally, equally apply to apparatus 7, in the sense that any information derived from data stream 9 by apparatus 8 may be inserted into data stream 9 by apparatus 7. Examples of the interaction between signing module 98 and verification module 99 have already been described above. It should further be noted that the determination of hash value 33 may equally be performed by apparatus 7 as described for apparatus 8.

[0266] FIG. 24 illustrates a method 260 for checking a multimedia data stream 9 for authenticity, according to one embodiment, the method comprising: deriving 261 substream information 89 from the multimedia data stream, the substream information indicating that the multimedia data stream is composed of a plurality of media substreams 17, 17′, each of which has a media signal encoded therein; and for each of a set of one or more first media substreams 17 included in the multimedia data stream, subjecting a first predetermined portion 13 of the respective first media substream, or first data 62 derived therefrom, to a separate first hash function 31 to generate a separate calculated hash. the method further comprising: obtaining (231) a common digital signature 91 and a hash value 33; deriving (262) signature information from the multimedia data stream, the common digital signature 91 and, for each of one or more removable media substreams 17', an individual transmitted hash value 94; and checking (241) whether the set of one or more first media substreams is trustworthy based on the common digital signature 91, the individual calculated hash value 33 for each of the set of one or more first media substreams, and the individual transmitted hash value 94 for at least one of the one or more removable media substreams.

[0267] FIG. 25 illustrates a method 300 for decoding a multimedia data stream and checking the multimedia data stream for authenticity, according to one embodiment, the method including: decoding 361 substream information from the multimedia data stream indicating that the multimedia data stream is composed of a plurality of media substreams, each of which has a media signal encoded therein; decoding 362 signature information from the multimedia data stream, the signature information including a common digital signature and, for each of one or more removable media substreams of the plurality of media substreams, a separate transmitted hash value; and decoding 363 the signature information from the multimedia data stream. and subjecting the common digital signature to an authenticity check of the first set of one or more first media substreams (390), including subjecting a first predetermined portion of, or first data derived therefrom, a respective first hash function to obtain a respective calculated hash value (391), for each of the set of one or more first media substreams included in the one or more first media substreams; and performing a check based on the common digital signature and, for each of the set of one or more first media substreams, the respective calculated hash value and, for at least one of the one or more removable media substreams, the respective transmitted hash value.

[0268] FIG. 26 provides a method 250 for rendering a multimedia data stream 9 that can be checked for authenticity, according to one embodiment, the method comprising inserting 299 substream information 89 into the multimedia data stream indicating that the multimedia data stream is composed of a plurality of media substreams 17, 17′, each of which has a media signal encoded therein; and for each of the plurality of media substreams, passing a first predetermined portion 13, 13′ of the respective media substream, or first data 62, 62′ derived therefrom, through a respective first hash function 31. and digitally signing (298) the multimedia data stream by calculating (98) the common digital signature 91 based on the individual calculated hash value 33' of the respective removable media signal.

[0269] Further optional details and features, as well as advantages, scopes and objectives of the above embodiments are described below. To enable reliable authentication of the origin of multimedia content, embodiments may provide new standards or may be implemented in existing ITU Recommendations that are regularly revised. Embodiments may make any ITU multimedia exchange format resistant to any potential attempts to spread deepfakes. With the development of advanced AI technologies, it has become increasingly easy to artificially generate multimedia content, such as audio, text, images, or video data, using so-called deepfakes. These are essentially fakes that can deceive users and thus raise significant cybersecurity and trust issues. It has been observed that for existing ITU-T Recommendations widely used for the global exchange of multimedia data, there are no generally accepted mechanisms that can provide reliable verification of the origin and authenticity of these data. Accordingly, embodiments may provide solutions for one or more of the following objectives:

[0270] · Establishing solutions that allow reliable authentication of the origin of coded digital data such as audio, text, images or video. Establish these solutions so that the main requirements and design features intended by the ITU-T Recommendations used to encode these data are still met. Allowing users to reliably verify that whenever they are presented with different types of multimedia data, for example a combination of video and audio, such a combination has in fact been created by a trusted and identifiable source and is not fake. Enabling easy access to advanced authentication technologies for content providers and consumers worldwide, preferably by establishing ITU itself as a secure and trusted source of digital validation certificates necessary for trusted multimedia exchange. · To enable the possibility to trace the origin of multimedia content even if the originally coded data has been subjected to multiple stages of editing and / or re-coding. Establishing technical methods that can detect the core characteristics of multimedia content that should be preserved under appropriate editing and / or re-encoding stages.

[0271] Below, we provide a framework for a specific implementation according to an embodiment that can be used for a new recommendation based on the technical description (for example, called Rec. ITU-T H.MMAuth). For example, the above embodiments aim to provide a technical solution for verifying the authenticity of multimedia content, allowing users to verify the authenticity of content created by its creator, such as a government, a company, or a news organization. The solution is based on digital signatures of the data stream. The content creator (encoder) signs the content using a private key, while the recipient (decoder) verifies the authenticity using the corresponding public key. The public key required for verification is not derived directly from the data stream, but is obtained in a trusted, independent manner, such as from a third-party trust center. The use of ITU Recommendation X.509 is specified for the secure acquisition of a digital certificate that validates the encoder's identity. In addition, cryptographic hash functions are specified for the encoder and decoder to calculate unique digital signatures over specified byte ranges in the data stream. Verification occurs when the decoder uses the public key to successfully match the digital signature with the calculated hash value, establishing the authenticity of the content.

[0272] Embodiments may refer to one or more of digital signatures, multimedia integrity, authentication verification, public key infrastructure, and ITU Recommendation X.509. Rapid advances in artificial intelligence technology, particularly in areas like deep learning, are facilitating the spread of misinformation via deepfakes. These AI-generated synthetic media, which can include video, images, text, and audio, are becoming more sophisticated and difficult to distinguish for both humans and current detection methods. Such advances could significantly increase threats to cybersecurity, raise concerns about digital piracy, and undermine the trust of digital platforms. Deepfakes can serve as tools for deception, fraud, and harm. By replicating voices and appearances, malicious actors can use deepfakes to steal credentials and impersonate IT personnel or senior management through convincing video or audio simulations. Such deceptive tactics can be directed at individual employees or at sophisticated automated systems that rely on voice recognition for authentication. The impact of these cyberattacks can be far-reaching, even endangering countries by targeting politicians.

[0273] As the technology behind generative AI continues to evolve, it becomes increasingly difficult to distinguish between authentic and fabricated digital content, which not only amplifies cybersecurity threats but also undermines the trust and safety of digital ecosystems such as social media platforms by blurring the line between reality and fabrication. The impact of deepfakes is not limited by geographic or developmental boundaries, affecting both developed and emerging countries, where the spread of manipulated content can cause significant social damage, especially in areas with limited digital literacy. The above embodiments can provide a technical framework for verifying the integrity and authenticity of multimedia content.

[0274] The following ITU-T Recommendations and other references contain provisions that, through reference herein, may constitute provisions of embodiments. In other words, an embodiment may implement one or more of the referenced recommendations. The versions indicated were in effect at the time of filing. [ITU-T H.264] Recommendation ITU-T H.264, Advanced video coding for generic audiovisual service [ITU Rec.X.509] Information technology-Open Systems Interconnection-The Directory: Public-key and attribute certificate frameworks [ISO / IEC 14496-3] The audio data coded by the MPEG-4 AAC-LC (ISO / IEC 14496-3) standard [ISO / IEC 10646] Universal coded character set, the captions coded via a text-format

[0275] For example, the methodology uses digital signatures applied to a data stream for verification. Here, the individual or entity generating the content (the encoder) utilizes a private key to authenticate the material, while the content recipient (the decoder) utilizes a matching public key for validation. This public key, essential to the validation process, is not necessarily extracted directly from the data stream. Instead, it can be obtained through a trusted, separate method, such as through a third-party validation center. It is outlined that secure acquisition of a digital certificate verifying the identity of the encoder according to ITU Recommendation X.509 may be required. It is further detailed that both the encoder and decoder create separate digital signatures for determined segments of the data stream using a cryptographic hash function. The authenticity of the content is confirmed when the decoder verifies the digital signature as an exact match with a hash value calculated using the public key.

[0276] Further embodiments In the following, embodiments are described in general terms that can optionally be combined with any of the features described above, with the numbering in the list below starting from the most recent. 1. A device 8 for checking a multimedia data stream 9 for authenticity, comprising: The device, deriving substream information 89 from the multimedia data stream indicating that the multimedia data stream is composed of a plurality of media substreams 17, 17', each of which has a media signal encoded therein; for each of a set of one or more first media substreams 17 included in the multimedia data stream, subjecting a first predetermined portion 13 of the respective first media substream, or first data 62 derived therefrom, to a respective first hash function 31 to obtain a respective calculated hash value 33; From multimedia data streams, common digital signatures91 and deriving signature information for each of the one or more removable media substreams 17′ of the plurality of media substreams, the signature information including a respective transmitted hash value 94; Common digital signature 91 and a separate calculated hash value 33 for each of the set of one or more first media substreams; for at least one of the one or more removable media substreams, based on the individual transmitted hash value 94; An apparatus configured to check whether a first set of one or more media substreams is reliable.

[0277] 2. The apparatus of embodiment 1, wherein the common digital signature is transmitted in a header or information box or at an upper layer of the multimedia data stream outside the multiple media substreams. 3. In checking whether the set of one or more first media sub-streams is reliable, 3. The apparatus of claim 1 or 2, wherein the apparatus is configured to check whether the individual calculated hash value for each of a set of one or more first media substreams, or, if the individual first media substreams are removable media substreams, the individual transmitted hash value, and, for each of at least one of the one or more removable media substreams, the individual transmitted hash value, matches mutually distinct portions of a check value derived from the common digital signature by checking equality between the individual transmitted hash value and the individual calculated hash value for each of the first media substreams (if any) included in the check for match with mutually distinct positions of the individual transmitted hash values.

[0278] 4. In checking whether the set of one or more first media substreams is authentic, the method is configured to subject, for each of the set of one or more first media substreams, the individual calculated hash value, or, if the individual first media substream is a removable media substream, the individual transmitted hash value, and for each of at least one of the one or more removable media substreams, the individual transmitted hash value, to further hashing to obtain a common hash value, and for each of the first media substreams (if any) included in subjecting the individual transmitted hash value to further hashing, check whether the common hash value matches a check value derived from the common digital signature by checking equality between the individual transmitted hash value and the individual calculated hash value; 3. The device of embodiment 1 or 2. 5. The apparatus of embodiment 3 or 4, configured to derive a check value from the common digital signature by decrypting the common digital signature. 6. An apparatus as described in any of embodiments 1 to 5, configured to select, for each of one or more removable media substreams, a set of one or more first media substreams based on the presence of a separate removable media substream, such that the set of one or more first media substreams consists of any removable media substreams present in the multimedia data stream and any media substreams that do not belong to the one or more removable media substreams.

[0279] 7. An apparatus as described in any one of embodiments 1 to 6, wherein the media signal encoded into multiple media substreams includes one or more video signals and / or one or more audio signals and / or one or more text signals. 8. At least one media substream of the one or more first media streams is a segmented media substream in which a separate media signal is temporally segmented and encoded; Deriving the signature information A common digital signature; deriving signature information for the segmented media substream, the signature information including a transmitted hash value for a given segment and a transmitted hash value for temporal consistency; The device, for each of a set of one or more first media substreams included in the multimedia data stream, subjecting a first predetermined portion of each first media substream, or first data derived therefrom, to a respective first hash function to obtain a respective calculated hash value, locating the first predetermined portion of the segmented media substream within a predetermined segment; If no previous segment exists in the multimedia data stream for a given segment, For the segmented media substreams, based on the transmitted hash values ​​for temporal consistency, 8. An apparatus as in any one of embodiments 1 to 7, configured to check whether a first set of one or more media substreams is reliable.

[0280] 9. The device is If a previous segment exists in the multimedia data stream for a given segment, For segmented media substreams, a further predetermined portion of the segmented media substream; or applying the further data derived therefrom to a respective first hash function of the segmented media substream to locate a further predetermined portion of the segmented media substream within a previous predetermined segment to obtain a further calculated hash value; For the segmented media substreams, based on the further calculated hash values: 9. The apparatus of embodiment 8, configured for checking whether a first set of one or more media substreams is reliable.

[0281] 10. for each of a set of one or more first media substreams, by checking whether the individual calculated hash value, or, if the individual first media substreams are removable media substreams, the individual transmitted hash value, or, if the individual first media substreams are segmented substreams, the transmitted hash value for temporal integrity, or, for each of at least one of the one or more removable media substreams, the individual transmitted hash value, matches mutually distinct portions of a check value derived from the common digital signature by checking equality between the individual transmitted hash value or the transmitted hash value for temporal integrity and the individual calculated hash value for each of the first media substreams (if any) included in the check for match with mutually distinct positions of the individual transmitted hash value or the transmitted hash value for temporal integrity; or by further hashing, for each of a set of one or more first media substreams, the individual calculated hash value, or, if the individual first media substream is a removable media substream, the individual transmitted hash value, and, if the individual first media substream is a segmented substream, the transmitted hash value for temporal integrity; and, for each of at least one of the one or more removable media substreams, the individual transmitted hash value to obtain a common hash value; and checking for equality between the individual transmitted hash value or the transmitted hash value for temporal integrity and the individual calculated hash value for each first media substream (if any) included in the individual transmitted hash value or the transmitted hash value for temporal integrity by subjecting the individual transmitted hash value or the transmitted hash value for temporal integrity to the further hash value, and checking whether the common hash value matches a check value derived from the common digital signature; 10. The apparatus of embodiment 9, configured to check whether a first set of one or more media substreams is reliable.

[0282] 11. Subjecting the first predetermined portion, or data derived therefrom, to a respective first hash function to obtain a first hash value; configured to reconstruct the media signal with respect to a first predetermined portion to obtain a reconstructed media portion, and subject the reconstructed media portion to a respective first hash function; The device according to any one of embodiments 1 to 3. 12. The apparatus decodes a media signal for each of a first set of one or more media substreams from a respective first media substream of the multimedia data stream; a multimedia decoder configured to decode a common digital signature from a multimedia data stream; An apparatus according to any one of embodiments 1 to 11.

[0283] 13. The apparatus of any one of embodiments 1 to 12, configured to perform the check by using an asymmetric decryption scheme using a public key. 14. The apparatus of embodiment 13, configured to derive an asymmetric decryption scheme using first information derived from the data stream. 15. The apparatus of embodiment 14, wherein the first information includes a decryption scheme identifier or a first pointer to a first location from which the asymmetric decryption scheme can be determined, or an identifier of an entity that encoded the video into the video data stream. 16. The apparatus of any one of embodiments 1-15, configured to derive a public key using second information derived from the data stream. 17. The apparatus of embodiment 16, wherein the second information includes a second pointer to a second location from which the public key can be obtained, or an identifier of the entity that encoded the video into the video data stream. 18. An apparatus as described in any one of embodiments 1-17, configured to derive a distinct first hash function using third information derived from the data stream. 19. The apparatus of embodiment 18, wherein the third information includes a hash function identifier or a third pointer to a third location from which the hash function can be determined, or an identifier of the entity that encoded the video into the video data stream. 20. The apparatus of any one of embodiments 1-19, wherein each first hash value depends on all bits of a respective first predetermined portion.

[0284] 21. A decoder for decoding a multimedia data stream and checking the multimedia data stream for authenticity, comprising: The decoder decoding, from the multimedia data stream, substream information indicating that the multimedia data stream is composed of multiple media substreams, each of which has a media signal encoded therein; From multimedia data streams, common digital signatures and and decrypting the signature information, including the transmitted hash value, for each of the one or more removable media substreams of the plurality of media substreams; for each of a set of one or more first media substreams included in the multimedia data stream, subjecting a first predetermined portion of each first media substream, or first data derived therefrom, to a respective first hash function to obtain a respective calculated hash value; A common digital signature; a separate calculated hash value for each of the set of one or more first media substreams; and for at least one of the one or more removable media substreams, based on the respective transmitted hash value. performing a check, A decoder configured to perform subjecting a common digital signature to an authenticity check of a first set of one or more first media substreams.

[0285] 22. A decoder as described in embodiment 21, wherein the common digital signature is transmitted in a header or information box or at an upper layer of the multimedia data stream outside the multiple media substreams. 23. Checking whether a set of one or more first media sub-streams is reliable includes: 23. A decoder as described in embodiment 21 or 22, comprising checking whether, for each of a set of one or more first media substreams, the individual calculated hash value, or, if the individual first media substream is a removable media substream, the individual transmitted hash value, and for each of at least one of the one or more removable media substreams, the individual transmitted hash value, matches mutually distinct portions of a check value derived from a common digital signature by checking equality between the individual transmitted hash value and the individual calculated hash value for each of the first media substreams (if any) included in the check for match with mutually distinct positions where the individual transmitted hash values ​​differ from each other.

[0286] 24. Checking whether the set of one or more first media sub-streams is reliable includes: 23. A decoder as described in embodiment 21 or 22, comprising subjecting, for each of a set of one or more first media substreams, the individual calculated hash value, or, if the individual first media substream is a removable media substream, the individual transmitted hash value, and for each of at least one of the one or more removable media substreams, the individual transmitted hash value, to further hashing to obtain a common hash value, and checking whether the common hash value matches a check value derived from the common digital signature by checking for equality between the individual transmitted hash value and the individual calculated hash value for each of the first media substreams (if any) included in subjecting the individual transmitted hash value to further hashing. 25. A decoder according to embodiment 23 or 24, wherein the check value is derived from the common digital signature by decrypting the common digital signature. 26. A decoder as described in any of embodiments 21 to 25, configured to select, for each of one or more removable media substreams, a set of one or more first media substreams based on the presence of a separate removable media substream, such that the set of one or more first media substreams consists of any removable media substreams present in the multimedia data stream and any media substreams that do not belong to the one or more removable media substreams.

[0287] 27. A decoder according to any one of embodiments 21 to 26, wherein the media signal encoded into multiple media substreams includes one or more video signals and / or one or more audio signals and / or one or more text signals. 28. At least one media substream of the one or more first media streams is a segmented media substream in which a separate media signal is temporally segmented and encoded, and the signature information is A common digital signature; For segmented media substreams, a transmitted hash value for a given segment and a transmitted hash value for temporal consistency; For each of a set of one or more first media substreams included in the multimedia data stream, subjecting a first predetermined portion of each first media substream, or first data derived therefrom, to a respective first hash function to obtain a respective calculated hash value, locating the first predetermined portion of the segmented media substream within a predetermined segment; If no previous segment exists in the multimedia data stream for a given segment, For the segmented media substreams, based on the transmitted hash values ​​for temporal consistency, 28. The decoder of any one of embodiments 21 to 27, further comprising: additionally checking whether the set of one or more first media sub-streams is reliable.

[0288] 29. Checking If a previous segment exists in the multimedia data stream for a given segment, For segmented media substreams, a further predetermined portion of the segmented media substream; or applying the further data derived therefrom to a respective first hash function of the segmented media substream to locate a further predetermined portion of the segmented media substream within a previous predetermined segment to obtain a further calculated hash value; For the segmented media substreams, based on the further calculated hash values:

[0013] 9. The decoder of embodiment 8, comprising: checking whether the first set of one or more media sub-streams is reliable.

[0289] 30. Checking whether a set of one or more first media sub-streams is reliable includes: checking whether, for each of a set of one or more first media substreams, the individual calculated hash values, or, if the individual first media substreams are removable media substreams, the individual transmitted hash values, or, if the individual first media substreams are segmented substreams, the transmitted hash values ​​for temporal integrity, and, for each of at least one of the one or more removable media substreams, the individual transmitted hash values, match mutually distinct portions of a check value derived from the common digital signature by checking equality between the individual transmitted hash values ​​or the transmitted hash values ​​for temporal integrity and the individual calculated hash values ​​for each of the first media substreams (if any) included in the check for match with mutually distinct positions where the individual transmitted hash values ​​or the transmitted hash values ​​for temporal integrity differ from each other; or 30. A decoder as described in embodiment 29, comprising subjecting, for each of a set of one or more first media substreams, the individual calculated hash value, or, if the individual first media substream is a removable media substream, the individual transmitted hash value, and, if the individual first media substream is a segmented substream, the transmitted hash value for temporal integrity, for each of at least one of the one or more removable media substreams, the individual transmitted hash value to further hashing to obtain a common hash value, and checking whether the common hash value matches a check value derived from the common digital signature by checking equality between the individual transmitted hash value or the transmitted hash value for temporal integrity and the individual calculated hash value for each first media substream (if any) included in the individual transmitted hash value or the transmitted hash value for temporal integrity.

[0290] 31. Applying a first predetermined portion, or data derived therefrom, to a separate first hash function to obtain a first hash value; reconstructing the media signal with respect to a first predetermined portion to obtain a reconstructed media portion; 31. A decoder according to any one of embodiments 21 to 30, comprising: subjecting the reconstructed media portions to a separate first hash function. 32. A decoder according to any one of embodiments 21 to 31, configured to perform the check by using an asymmetric decryption scheme using a public key.

[0291] 33. The decoder of embodiment 32 configured to derive an asymmetric decryption scheme using first information derived from the data stream. 34. A decoder as described in embodiment 33, wherein the first information includes a decryption scheme identifier or a first pointer to a first location from which an asymmetric decryption scheme can be determined, or an identifier of an entity that encoded the media signal into a multimedia data stream. 35. A decoder according to any one of embodiments 21 to 34, configured to derive a public key using second information derived from the data stream. 36. A decoder as described in embodiment 35, wherein the second information includes a second pointer to a second location from which the public key can be obtained, or an identifier of the entity that encoded the media signal into the multimedia data stream. 37. A decoder according to any one of embodiments 21 to 36, configured to derive a distinct first hash function using third information derived from the data stream. 38. A decoder as described in embodiment 37, wherein the third information includes a hash function identifier or a third pointer to a third location from which the hash function can be determined, or an identifier of the entity that encoded the media signal into the multimedia data stream. 39. A decoder according to any one of embodiments 21-38, wherein each first hash value depends on all bits of a respective first predetermined portion.

[0292] 40. An apparatus 7 for rendering a multimedia data stream 9 that can be checked for authenticity, the apparatus comprising: inserting 99 into the multimedia data stream substream information 89 indicating that the multimedia data stream is composed of multiple media substreams 17, 17', each of which has a media signal encoded therein; for each of the plurality of media sub-streams, subjecting a first predetermined portion 13, 13' of the respective media sub-stream, or first data 62, 62' derived therefrom, to a respective first hash function 31 to obtain a respective calculated hash value 33'; Common digital signature 91 and for each of the one or more removable media substreams 13' of the plurality of media substreams, a respective transmitted hash value equal to the calculated hash value 33' of the respective removable media signal; Encode it into a multimedia data stream, for each of the plurality of media substreams, based on the respective calculated hash value; An apparatus configured to digitally sign a multimedia data stream by computing a common digital signature.

[0293] 41. The apparatus of embodiment 40, wherein the common digital signature is transmitted in a header or information box or at an upper layer of the multimedia data stream outside of the multiple media substreams. 42. by deriving a common digital signature 91 by signing a check value that includes, for each of the set of one or more first media substreams, the respective calculated hash value, or, if the respective first media substreams are removable media substreams, the respective transmitted hash value, and the respective transmitted hash value for each of at least one of the one or more removable media substreams; 42. The apparatus of embodiment 40 or 41, configured to enable checking reliability for a set of one or more first media substreams. 43. by subjecting, for each of the set of one or more first media substreams, the respective calculated hash value, or, if the respective first media substreams are removable media substreams, the respective transmitted hash value, and, for each of at least one of the one or more removable media substreams, the respective transmitted hash value, to further hashing to obtain a common hash value, and signing the common hash value to derive a common digital signature; 42. The apparatus of embodiment 40 or 41, configured to enable checking reliability for a set of one or more first media substreams. 44. The apparatus of embodiment 42 or 43, configured to derive a common digital signature from the check value by encrypting the check value.

[0294] 45. An apparatus as described in any of embodiments 40 to 44, wherein the media signal encoded into multiple media substreams includes one or more video signals and / or one or more audio signals and / or one or more text signals. 46. ​​At least one media substream of the one or more first media streams is a segmented media substream in which a separate media signal is temporally segmented and encoded; The signature information is A common digital signature; For segmented media substreams, a transmitted hash value for a given segment and a transmitted hash value for temporal consistency; The device, 46. ​​An apparatus as described in any of embodiments 40 to 45, configured to locate a first predetermined portion of a segmented media substream within a predetermined segment by subjecting, for each of a set of one or more first media substreams included in a multimedia data stream, a first predetermined portion of each first media substream, or first data derived therefrom, to a respective first hash function to obtain a respective calculated hash value.

[0295] 47. Deriving a common digital signature 91 by signing a check value including, for each of a set of one or more first media substreams, the respective calculated hash value, or, if the respective first media substreams are removable media substreams, the respective transmitted hash value, or, if the respective first media substreams are segmented substreams, the transmitted hash value for temporal integrity, for at least one of the one or more removable media substreams; by subjecting, for each of the set of one or more first media substreams, the individual calculated hash value, or, if the individual first media substreams are removable media substreams, the individual transmitted hash value, and, if the individual first media substreams are segmented substreams, the transmitted hash value for temporal consistency, for each of at least one of the one or more removable media substreams, to further hashing to obtain a common hash value, and signing the common hash value to derive a common digital signature; 47. The apparatus of embodiment 46, configured to enable checking reliability for a set of one or more first media substreams.

[0296] 48. In subjecting a first predetermined portion, or data derived therefrom, to a respective first hash function to obtain a first hash value, reconstructing the media signal with respect to a first predetermined portion to obtain a reconstructed media portion; 43. An apparatus as described in any one of embodiments 40 to 42, configured to subject the reconstructed media portions to a separate first hash function. 49. An apparatus encrypts a media signal of each of a first set of one or more media substreams into a separate first media substream of a multimedia data stream; a multimedia encoder configured to encrypt a common digital signature into a multimedia data stream; An apparatus described in any one of embodiments 40 to 48.

[0297] 50. The apparatus of any of embodiments 40-49 configured to perform digital signing by use of an asymmetric encryption scheme using a private key or, optionally, a public key. 51. The apparatus of embodiment 50 configured for inserting first information from which an asymmetric encryption scheme can be derived into a data stream. 52. The device of embodiment 51, wherein the first information includes an encryption scheme identifier or a first pointer to a first location from which the asymmetric encryption scheme can be determined, or an identifier of the device. 53. An apparatus described in any one of embodiments 40 to 52, configured to insert second information from which a public key can be derived into a video data stream. 54. The device of embodiment 53, wherein the second information includes a second pointer to a second location from which the public key can be obtained or an identifier of the device. 55. The apparatus of any one of embodiments 40-54, configured to insert third information into the data stream from which the first hash function can be derived. 56. The device of embodiment 55, wherein the third information includes a hash function identifier or a third pointer to a third location from which the hash function can be determined, or an identifier of the device. 57. The apparatus of any of embodiments 40-54, wherein each first hash value depends on all bits of a respective first predetermined portion.

[0298] 58. A method 260 for checking a multimedia data stream 9 for authenticity, the method comprising: deriving 261 substream information 89 from the multimedia data stream indicating that the multimedia data stream is composed of a plurality of media substreams 17, 17', each of which has a media signal encoded therein; for each of a set of one or more first media substreams 17 comprised in the multimedia data stream, subjecting 231 a first predetermined portion 13 of the respective first media substream, or first data 62 derived therefrom, to a respective first hash function 31 to obtain a respective calculated hash value 33; From multimedia data streams, Common digital signature 91 and deriving 262 signature information including a respective transmitted hash value 94 for each of one or more removable media substreams 17' of the plurality of media substreams; Common digital signature 91 and a separate calculated hash value 33 for each of the set of one or more first media substreams; and for at least one of the one or more removable media substreams, based on the individual transmitted hash value 94; checking 241 whether the first set of one or more media sub-streams is reliable.

[0299] 59. A method 300 for decoding a multimedia data stream and checking the multimedia data stream for authenticity, the method comprising: decoding 361 substream information from the multimedia data stream indicating that the multimedia data stream is composed of multiple media substreams, each of which has a media signal encoded therein; From multimedia data streams, A common digital signature; decrypting 362 the signature information, including the respective transmitted hash value, for each of the one or more removable media substreams of the plurality of media substreams; 391, for each of a set of one or more first media substreams included in the multimedia data stream, subjecting a first predetermined portion of each first media substream, or first data derived therefrom, to a respective first hash function to obtain a respective calculated hash value; A common digital signature; a separate calculated hash value for each of the set of one or more first media substreams; and for at least one of the one or more removable media substreams, based on the respective transmitted hash value. performing a check, and applying 390 the common digital signature to an authenticity check of the first set of one or more first media sub-streams.

[0300] 60. A method 250 for rendering a multimedia data stream 9 that is checkable for authenticity, the method comprising: inserting 299 substream information 89 into the multimedia data stream indicating that the multimedia data stream is composed of a plurality of media substreams 17, 17', each of which has a media signal encoded therein; for each of the plurality of media sub-streams, subjecting 281 a first predetermined portion 13, 13' of the respective media sub-stream, or first data 62, 62' derived therefrom, to a respective first hash function 31 to obtain a respective calculated hash value 33'; Common digital signature 91 and for each of the one or more removable media substreams 13' of the plurality of media substreams, a respective transmitted hash value equal to the calculated hash value 33' of the respective removable media signal; 271 encoding the multimedia data stream; for each of the plurality of media substreams, based on the respective calculated hash value; and digitally signing 298 the multimedia data stream by calculating 98 a common digital signature 91. 61. A multimedia data stream generated using the method of embodiment 60; and / or can be checked for authenticity using the method of embodiment 59; and / or a multimedia data stream that is decodable using the method of embodiment 58.

[0301] Alternative Implementations While some aspects are described as features in the context of an apparatus, it will be apparent that such description may also be considered a description of corresponding features of a method. Although some aspects are described as features in the context of an apparatus, it will be apparent that such description may also be considered a description of corresponding features in terms of the functionality of the apparatus. In particular, block diagrams illustrating apparatuses may also be considered illustrations of individual methods including the steps described by the blocks in the block diagram. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit, and in some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0302] The coded image signal of the present invention can be stored on a digital storage medium or can be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet. In other words, a further embodiment provides a video bitstream product comprising a video bitstream according to any of the embodiments described herein, e.g., a digital storage medium having stored thereon a video bitstream.

[0303] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or software, or at least partly in hardwar...

Claims

1. A decoder (20) for decoding an audio stream (11) from an audio data stream (14), said decoder comprising: subjecting a predetermined portion (13) of the audio data stream, or data derived therefrom, to a hash function (31) to obtain a hash value (33); by checking whether the hash value matches the digital signature to determine (41) whether the audio data stream is authentic; A decoder configured to decode (63) from said audio data stream a digital signature (43) to be subjected to a check (16) on the authenticity of said audio data stream.

2. 2. The decoder of claim 1, wherein the digital signature is transmitted within a supplemental information message of the audio data stream.

3. said check on the authenticity of said audio data stream comprising: decrypting the digital signature to obtain a check value; and checking whether the hash value matches the check value.

4. configured to decrypt the digital signature from a supplemental information message of the audio data stream; A decoder according to any one of claims 1 to 3.

5. said check on the authenticity of said audio data stream comprising:

5. A decoder as claimed in any preceding claim, comprising locating said predetermined portion within the audio data stream by using one or more supplemental information messages interspersed within the audio data stream to determine said predetermined portion to be a section of the audio data stream that spans between or extends from said one or more supplemental information messages.

6. configured to decrypt the digital signature from one of the one or more supplemental information messages of the audio data stream.

6. A decoder according to claim 5.

7. said check on the authenticity of said audio data stream comprising:

5. A decoder as claimed in any preceding claim, comprising locating said predetermined portion within the audio data stream by using prefix and suffix supplemental information messages interspersed within the audio data stream to determine said predetermined portion to be a section of the audio data stream that spans or is located between said prefix and suffix supplemental information messages.

8. 8. A decoder according to claim 7, configured to decode the digital signature from the suffix supplemental information message.

9. configured to locate the predetermined portion within the audio data stream by using first and second supplemental information messages interspersed within the audio data stream to determine the predetermined portion to be a section of the audio data stream that spans or is located between the first and second supplemental information messages; A decoder according to any one of claims 6 to 8.

10. configured to locate the predetermined portion within the audio data stream by using first and second supplemental information messages interspersed within the audio data stream to determine the predetermined portion to be a section of the video data stream that spans or is located between the first SEI message and a point in the data stream located downstream of the second SEI message; A decoder according to claim 6 or 7.

11. A decoder according to claim 9 or 10, arranged for decrypting the digital signature from the second supplementary information message.

12. configured to derive from said audio data stream summary supplemental information messages indicating one or more substreams of said audio data stream for which it is possible to check said audio data stream for reliability based on one or more portions within said individual substreams, A decoder according to any one of claims 1 to 11.

13. said check on the authenticity of said audio data stream comprising: whether the hash value and further data derived by subjecting an earlier portion of the video data stream to the hash function match the digital signature; or and checking whether a combined hash value derived by hashing the predetermined portion and a further hash value obtained by subjecting a previous portion of the audio data stream, or further data derived therefrom, to the hash function matches the digital signature, sequentially performing a check of the audio data stream for authenticity for a plurality of portions of the audio data stream; A decoder according to any one of claims 1 to 12.

14. said check on the authenticity of said audio data stream comprising: combining the predetermined portion with a further hash value obtained by subjecting a previous portion of the audio data stream, or further data derived therefrom, to the hash function to obtain a combined hash value, and checking whether the combined hash value matches the digital signature.

14. A decoder according to claim 13.

15. 15. The decoder of claim 14, wherein the combination is a concatenation.

16. said check on the authenticity of said audio data stream comprising: checking whether an identifier of the hash value matches the digital signature to further determine whether the audio data stream is authentic. A decoder according to any one of claims 1 to 15.

17. The electronic signature is the predetermined value is equal to a check value obtained by decrypting the digital signature or a predetermined portion of the check value that is related to the predetermined value; or if the predetermined value is equal to the check value in a further hash field reached by a further hash function applied to the predetermined value or to a concatenation of values ​​including the predetermined value, matched by the predetermined value, A decoder according to any one of claims 1 to 16.

18. said check on the authenticity of said audio data stream comprising: including the use of asymmetric decryption methods using public keys; A decoder according to any one of claims 1 to 17.

19. 20. A decoder as claimed in claim 18, configured to derive the asymmetric decryption scheme using first information derived from the data stream.

20. 20. The decoder of claim 19, wherein the first information comprises a decryption scheme identifier or a first pointer to a first location from which the asymmetric decryption scheme can be determined, or an identifier of the entity that encoded the audio into the audio data stream.

21. A decoder according to any one of claims 18 to 20, configured to derive said public key using second information derived from said data stream.

22. 22. The decoder of claim 21 , wherein the second information comprises a second pointer to a second location from which the public key can be obtained, or an identifier of the entity that encoded the audio into the audio data stream.

23. A decoder according to any preceding claim, configured to derive the hash function using third information derived from the data stream.

24. 24. The decoder of claim 23, wherein the third information comprises a hash function identifier or a third pointer to a third location from which the hash function can be determined, or an identifier of the entity that encoded the audio into the audio data stream.

25. A decoder according to any preceding claim, wherein the hash value depends on all bits of the predetermined portion of the audio data stream.

26. said check on the authenticity of said audio data stream comprising: checking whether the hash function is correct by checking whether the parameterization or identifier of the hash function matches the digital signature. A decoder according to any one of claims 1 to 25.

27. providing said predetermined portion for subjecting said audio data stream combined with said media stream to an authenticity check together with further data derived from a portion of said media stream accompanying said audio data stream; A decoder according to any one of claims 1 to 26.

28. An apparatus (15) for rendering an audio data stream (14) in which an audio stream that can be checked for authenticity is encoded, said apparatus comprising: applying a predetermined portion (13) of the audio data stream, or data derived therefrom, to a hash function (31) to obtain a hash value (33); calculating a digital signature (43) based on the hash value and digitally signing the hash value; 7. An apparatus configured to insert the digital signature into the audio data stream, thereby enabling a determination of whether the audio data stream can be trusted by checking whether the hash value matches the digital signature (77).

29. 29. The apparatus of claim 28, configured to insert the digital signature into a supplemental information message of the audio data stream.

30. forming a check value based on the hash value; by encrypting the check value to obtain the digital signature; 30. Apparatus according to claim 28 or 29, configured to calculate the digital signature based on the hash value.

31. The device, encoding said audio into said audio data stream; An apparatus according to any of claims 28 to 30, being an encoder configured for encoding said digital signature into said audio data stream.

32. configured to encode the digital signature into a supplemental information message of the audio data stream; 32. The apparatus of claim 31.

33. locating the predetermined portion within the audio data stream by using one or more supplemental information messages interspersed within the audio data stream to determine the predetermined portion to be a section of the audio data stream that spans between or from the one or more supplemental information messages; 33. Apparatus according to claim 31 or 32.

34. configured to encode the digital signature into one of the one or more supplemental information messages of the audio data stream.

34. The apparatus of claim 33.

35. locating the predetermined portion within the audio data stream by using prefix and suffix supplemental information messages interspersed within the audio data stream to determine the predetermined portion to be a section of the audio data stream that spans or is located between the prefix and suffix supplemental information messages; 33. Apparatus according to claim 31 or 32.

36. 36. The encoder of claim 35 configured to encode the digital signature into the suffix supplemental information message.

37. locating the predetermined portion within the audio data stream by using first and second supplemental information messages interspersed within the audio data stream to determine the predetermined portion to be a section of the audio data stream that spans or is located between the first and second supplemental information messages; 33. Apparatus according to claim 31 or 32.

38. locating the predetermined portion within the audio data stream by using first and second supplemental information messages interspersed within the audio data stream to determine the predetermined portion to be a section of the audio data stream that spans or is located between the first supplemental information message and a point in the data stream located downstream of the second supplemental information message; 33. Apparatus according to claim 31 or 32.

39. 36. An encoder according to claim 35, configured to encode the digital signature into the second supplemental information message.

40. configured to insert into the audio data stream summary supplemental information messages indicating one or more substreams of the audio data stream for which it is possible to check the audio data stream for reliability based on one or more portions within the individual substreams, 33. Apparatus according to claim 31 or 32.

41. said check on the authenticity of said audio data stream comprising: further calculating the digital signature based on the hash value and further data obtained by subjecting an earlier portion of the video data stream to the hash function; or 40. Apparatus according to any one of claims 28 to 39, adapted to be performed sequentially on a plurality of portions of the audio data stream by calculating the digital signature based on a combined hash value obtained by hashing the predetermined portion and a further hash value obtained by subjecting a previous portion of, or further data derived from, the audio data stream to the hash function.

42. the further hash value obtained by subjecting the predetermined value and a previous portion of the audio data stream, or further data derived therefrom, to the hash function; configured to apply the combination to obtain a combined hash value, and to calculate the digital signature to digitally sign the combined hash value.

42. The apparatus of claim 41.

43. 43. The device of claim 42, wherein the combination is a linkage.

44. Apparatus according to any of claims 41 to 43, configured to further calculate the digital signature based on an identifier of the hash function.

45. forming a check value based on the hash value such that the hash value matches a predetermined portion of the check value, and encrypting the check value to obtain the digital signature; or forming a check value based on the hash value by further hashing the predetermined value or a concatenation of values ​​including the predetermined value, and encrypting the check value to obtain the digital signature; An apparatus according to any of claims 28 to 44, configured to calculate the digital signature based on the hash value.

46. An apparatus according to any one of claims 28 to 45, configured to calculate the digital signature based on the hash value by use of an asymmetric cryptography scheme using a public key and a private key.

47. 47. The apparatus of claim 46, configured to insert first information into the data stream from which the asymmetric encryption scheme can be derived.

48. 48. The device of claim 47, wherein the first information comprises an encryption scheme identifier or a first pointer to a first location from which the asymmetric encryption scheme can be determined, or an identifier of the device.

49. Apparatus according to any of claims 28 to 48, configured for inserting second information from which the public key can be derived into the audio data stream.

50. 50. The device of claim 49, wherein the second information comprises a second pointer to a second location from which the public key can be obtained or an identifier of the device.

51. Apparatus according to any of claims 28 to 50, configured for inserting third information into the data stream from which the hash function can be derived.

52. 52. The device of claim 51, wherein the third information comprises a hash function identifier or a third pointer to a third location from which the hash function can be determined, or an identifier of the device.

53. Apparatus according to any of claims 28 to 52, wherein the hash value depends on all bits of the predetermined portion of the audio data stream.

54. further calculating the digital signature based on a parameterization or an identifier of the hash function; An apparatus according to any one of claims 28 to 53, whereby digital signing is performed.

55. A method (200) for decoding an audio stream from an audio data stream, said method comprising: and decoding (163) from said audio data stream a digital signature to be subjected to a check on the authenticity of said audio data stream, said check comprising: subjecting a predetermined portion of the audio data stream, or data derived therefrom, to a hash function to obtain a hash value (131); checking whether the hash value matches the digital signature to determine whether the audio data stream is authentic (141).

56. A method (150) for rendering an audio data stream in which an audio stream that can be checked for authenticity is encoded, said method comprising: applying a predetermined portion of the audio data stream, or data derived therefrom, to a hash function to obtain a hash value (131); calculating a digital signature based on the hash value and digitally signing the hash value (171); and inserting (177) the digital signature into the audio data stream, thereby making it possible to determine whether the audio data stream can be trusted by checking whether the hash value matches the digital signature.

57. 57. An audio data stream produced by the method of claim 56.

Images