Editable signature video data

Abstract

Editable signature video data is disclosed. A signature video bitstream obtained by predictive encoding of a video sequence includes data units and associated signature units, wherein each data unit represents at most one macroblock in a video frame of the video sequence, and each signature unit includes a digital signature of a bit string derived from a fingerprint of each exactly one associated data unit. The bitstream is edited by receiving a request to replace a region, determining a first set of macroblocks containing the region and a second set of macroblocks referencing macroblocks in the first set, adding an archive object including fingerprints of data units of the first set and data units of the second set, the data units of the first set and the data units of the second set representing the first set of macroblocks and the second set of macroblocks, editing the data units of the first set according to the request to replace, and re-encoding the data units of the second set.

Description

Technical Field

[0001] This disclosure relates to the field of security measures for protecting video data from unauthorized activities, particularly concerning the storage and transmission of data. It proposes methods and apparatus for editing signed video bitstreams and for verifying that such editing can result in signed video bitstreams. Background Technology

[0002] Digital signatures provide a layer of verification and security for digital messages transmitted over insecure channels. With the aid of digital signatures, the authenticity or integrity of a message can be verified, and non-repudiation can be ensured. In particular, regarding video coding, secure and efficient methods for digitally signing predictive coded video sequences have been described in the prior art. For example, see the inventors' earlier patent applications EP21201360.1 (EP4164173A1) and EP21201362.7 (EP4164230A1). Also see US20140010366A1, which proposes an encrypted video verification technique particularly suitable for predictive coded video data with a picture group structure.

[0003] Video sequences may need to be edited after they have been signed. Beyond visual improvements, the purpose of editing can be to ensure privacy by cropping, masking, blurring, or similar image manipulations that make visual features less identifiable. For most available methods, this will require recoding and re-signing the edited frames as a whole. Even if adjacent frames are not directly affected by the editing, recoding and re-signing should preferably be extended to multiple adjacent frames to avoid interfering with any predictive coding dependencies (inter-frame / intra-frame references). These steps can consume significant computational resources and may result in inconvenient latency for the user.

[0004] US7437007B1 discloses a method for performing region of interest (ROI) editing of a video stream in a compressed domain. The compressed video stream comprises compressed video stream frames, which represent video stream frames having unwanted portions and ROI portions. According to the method, the compressed video stream frames are edited to modify the unwanted portions and obtain a compressed video stream frame including the ROI portion, while maintaining the original structure of the video stream. To achieve this, the editing includes, for both one-way predictive coding (P) frames and two-way predictive coding (B) frames, skipping macroblocks located above, below, and to the right of the ROI portion. The video stream considered in US7437007B1 is not a signed video stream. Summary of the Invention

[0005] It is an object of the present disclosure to make available a method of editing a signed video bitstream obtained by predictive encoding of a video sequence, which method greatly avoids the need to re-sign the bitstream outside the portion affected by the edit, as is the case with some available methods. It is a particular object to make available such a video editing method which preserves the signature of all macroblocks except the edited macroblocks and any further macroblocks that directly or indirectly reference these macroblocks. It is a further object to enable video editing without any significant impairment to the data security of the original signed video bitstream. It is a further object to provide a method of verifying a signed video bitstream obtained by predictive encoding of a video sequence. It is a still further object to provide apparatuses and computer programs for these purposes.

[0006] These objects are achieved by the present invention as defined by the independent claims. The dependent claims relate to advantageous embodiments of the invention.

[0007] In a first aspect of the present disclosure, a method of editing a signed video bitstream is provided, which signed video bitstream has been obtained by predictive encoding of a video sequence. The signed video bitstream shall comprise data units and associated signature units. Each data unit represents (i.e. encodes) at most one macroblock in a video frame of the predictively encoded video sequence. Each signature unit comprises a digital signature of a bit string derived from a plurality of fingerprints of each exactly one associated data unit. The signature unit can optionally comprise the bit string to which the digital signature belongs. For a signed video bitstream having these features, the method comprises the steps of receiving a request to replace a region of at least one video frame (e.g. replacing the pixel values of the region with a privacy mask in one or more video frames), determining a first set of macroblocks containing said region and a second set of macroblocks directly or indirectly referencing the macroblocks in the first set, adding to the signed video bitstream an archival object comprising fingerprints of the first set of data units and the second set of data units, which respectively represent the determined first set of macroblocks and the second set of macroblocks, editing the first set of data units in accordance with the request to replace, and re-encoding the second set of data units. The signature unit can optionally comprise the derived bit string to which the digital signature belongs ("documented method").

[0008] Because each data unit represents at most one macroblock, data units and macroblocks are in a one-to-one relationship, or some macroblocks are represented by two or more data units and can be reconstructed from two or more data units. No data unit represents multiple macroblocks (nor parts of multiple macroblocks), and thus the immediate effect of editing some macroblocks is limited to one or more data units (a first group) of the edited macroblocks. Moreover, since each fingerprint is the fingerprint of exactly one associated data unit, the need for re-signing data units after editing is limited. More precisely, the method according to the first aspect preserves any predictive coding dependencies connecting pairs or groups of macroblocks, i.e. by determining and re-encoding a group (a second group) of data units representing macroblocks that directly or indirectly reference the edited macroblocks. If the re-encoding is limited to the second group of data units, the method will effectively exploit the available computational effort. This allows the method to be performed with satisfactory performance on ordinary processing devices.

[0009] The method according to the first aspect involves further advantages at the recipient side. Thanks to the archive object, from which the fingerprints of the first group of data units and the second group of data units can be retrieved, the recipient will be able to verify all data units of the signed video bitstream that have not been affected by the edit. This allows to preserve an important part of the existing signature; in this respect, the video editing method according to the first aspect can be said to be the least disruptive. The verification at the recipient will be explained in detail within the second aspect of the present disclosure.

[0010] In some embodiments, the archive object further comprises the locations of the first group of macroblocks and the second group of macroblocks. The locations can refer to the locations of the macroblocks in the frame, e.g. in frame coordinates. If a static macroblock partitioning is used, the locations of the macroblocks can be expressed as macroblock sequence numbers or another identifier. This provides a way to help the recipient of the edited video bitstream to determine whether a particular macroblock has been changed, and thus to select the appropriate way to obtain the fingerprints of the data units representing said macroblock.

[0011] Some embodiments provide an advantageous procedure for editing the first group of data units. More precisely, such an edit can comprise decoding the data units into reconstructed macroblocks; providing edited macroblocks by performing the requested replacement on the reconstructed macroblocks; and providing edited data units by encoding the edited macroblocks. If the region to be replaced covers the entire reconstructed macroblock, in particular, the edited data units can be provided by encoding a subset of the region, e.g. the intersection of the macroblock and the region.

[0012] In such an embodiment, the resulting macroblock after replacement may optionally be encoded as an intra-frame coded macroblock (I-block), thereby obtaining independently decodeable data units. This illustrates the fact that replacement can introduce abrupt temporal changes into a video sequence; this often reduces the temporal continuity of the video sequence, causing most known predictive coding techniques to perform poorly.

[0013] Within the same process used to edit the first set of data units, the reconstructed macroblocks can be used to decode the data units in the second set. This decoding is predictive because the second set of data units represents the second set of macroblocks, which directly or indirectly reference the first set of macroblocks. The output of decoding the data units in the second set will be used in the re-encoding step.

[0014] In different embodiments, the re-encoding of the second set of data units can be performed either by predictive encoding based on the edited first set of data units or by non-predictive encoding. Choosing one of these options can correspond to achieving a desired balance between quality and bit rate. Furthermore, the second set of data units can be re-encoded using simplified data compression. In lossy video coding formats, data compression is achieved by discarding some information in the video sequence, such as through the quantization of pixel values. The degree of quantization can, for example, correspond to the value of the quantization parameter (QP), which represents the fineness of the quantized pixel values. The degree of quantization can further depend on entries in the definition of the scaling matrix. Assuming the video sequence is encoded at a predetermined conventional data compression level, it is foreseeable that the second set of macroblocks will be encoded at a reduced data compression level relative to conventional data compression. Using a reduced data compression level implies discarding less information in the macroblock during the encoding step, for example, by mapping the macroblock's pixel values ​​to a relatively finer set of quantized pixel values. In other words, the macroblocks in the second set are encoded at a relatively higher bit rate than when using a conventional data compression level, and can be played back with less residual error. The additional memory or bandwidth costs are likely acceptable, especially in use cases where editing rarely occurs and / or occurs in isolated portions of a video sequence.

[0015] In some embodiments, the data security of an edited signed video bitstream is improved by providing one or more signature units associated with the edited first set of data units and / or the re-encoded second set of data units in the bitstream. This avoids scenarios of unauthorized modification of the edited first set of data units and the re-encoded second set of data units. The one or more signature units can be new signature units added to the bitstream, or edited versions of signature units included in the signed video bitstream prior to editing.

[0016] According to a first aspect of the generalization, a video editing method is provided that is performed on a signed video sequence comprising data units and associated signature units. Each data unit represents at most N macroblocks in a video frame of a predictively coded video sequence. In the present text, N is a small integer, such as 1, 2, 3, 4, 5 or at most 10. Each signature unit comprises a digital signature of a bit string and optionally the bit string itself, the bit string being derived from a plurality of fingerprints of exactly one associated data unit. The video editing method comprises receiving a request to replace a region of at least one video frame; determining a first set of macroblocks that contains the region and a second set of macroblocks that directly or indirectly reference a macroblock of the first set; adding an archival object to the signed video bitstream, the archival object comprising fingerprints of the first set of data units and the second set of data units, the first set of data units and the second set of data units representing the determined first set of macroblocks and the second set of macroblocks, respectively; editing the first set of data units according to the request to replace; and re-encoding the second set of data units.

[0017] According to this generalization, because each data unit represents at most N macroblocks, the immediate effect of editing one macroblock is limited to at most N data units of the edited macroblock. Moreover, because each fingerprint is of exactly one associated data unit, the need for re-signing data units after editing is limited. In particular, the archival object will hold at most N times as many fingerprints as the cardinality of the first set of macroblocks and the second set of macroblocks, so that the video editing method will have a feasible complexity.

[0018] In a second aspect of the disclosure, a method is provided for verifying a signed video bitstream that is obtained by predictively coding a video sequence. It is understood that the signed video bitstream comprises data units, signature units each associated with some of the data units, and an archival object. Each data unit represents at most one macroblock in a frame of a predictively coded video sequence. Each signature unit comprises a digital signature of a bit string and optionally the bit string itself. Finally, the archival object comprises at least one archival fingerprint of a data unit. The method for verifying the signed video bitstream comprises obtaining a fingerprint of each data unit associated with a signature unit by computing the fingerprint of the data unit or retrieving the archival fingerprint from the archival object; deriving a bit string from the obtained fingerprints; and verifying the data units associated with the signature unit using the digital signature in the signature unit. The final verification step can comprise verifying the derived bit string using the digital signature. Alternatively ("document method"), the verification step comprises verifying the bit string in the signature unit using the digital signature, and comparing the derived bit string and the verified bit string.

[0019] The archive object may have already been added by executing the editing method according to the first aspect, but the method according to the second aspect can be executed without reliable knowledge of such prior processing. Accordingly, the method according to the second aspect achieves the verification of the authenticity of the video sequence because it verifies that the digital signature (and any bit string) carried in the signature unit does indeed match the fingerprint of the associated data unit. Therefore, the data unit cannot have been modified.

[0020] The second aspect of the method includes two options: obtaining the fingerprint of a data unit either by direct computation or by retrieval from the archive object. This supports minimally destructive processing of existing fingerprints during the editing phase (first aspect). The fact that each data unit represents at most one macroblock tends to limit the number of fingerprints that need to be archived for a given replacement request, which in turn limits the size of the archive object.

[0021] In some embodiments, the archive object further indicates the location of the macroblock represented by the data unit to which the archive fingerprint belongs. In other words, the archive fingerprint is the fingerprint of the data unit, and the data unit represents (encodes) a macroblock whose location is indicated in the archive object. During the execution of the method according to the second aspect, in order to obtain the fingerprint of the data unit, it is determined, based on the location indicated by the archive object, whether to compute the fingerprint or retrieve the fingerprint from the archive object.

[0022] A third aspect of this disclosure relates to apparatuses arranged to perform the methods of the first and / or second aspects. These apparatuses may be embedded in systems with different primary purposes (e.g., video recording, video content management, video playback), or they may be dedicated to the editing and verification processes, respectively. The apparatuses within the third aspect of this disclosure generally share the effects and advantages of the first and second aspects, and they may be implemented with equivalent technical variations.

[0023] The present invention further relates to a computer program comprising instructions for causing a computer to perform the methods described above. The computer program may be stored or distributed on a data carrier. As used herein, "data carrier" can be a temporary data carrier such as modulated electromagnetic waves or light waves, or a non-temporary data carrier. Non-temporary data carriers include volatile and non-volatile memories, such as permanent and non-permanent storage media of magnetic, optical, or solid-state types. Still within the scope of "data carrier," such memory may be fixedly mounted or portable.

[0024] In a further aspect of the disclosure, a signed video bitstream is provided, comprising data units and associated signature units. Each data unit represents at most N macroblocks in a video frame of a predictively coded video sequence. Herein, N is a small integer, such as 1, 2, 3, 4, 5 or at most 10. Each signature unit comprises a digital signature of a bit string derived from a plurality of fingerprints of each exactly one associated data unit, and optionally the bit string itself. Since the immediate effect of editing one macroblock is limited to at most N data units of the edited macroblock, and since each fingerprint is of exactly one associated data unit, the signed video bitstream is suitable for editing. This limits the propagation of edits to a limited number of data units, such that less data units need to be re-signed after editing.

[0025] It should be noted that, as used in the present disclosure, a "macroblock" can advantageously be a coded macroblock. However, the present invention is also applicable to non-predictively coded video, and in the more general case, a macroblock can thus be any contiguous group of pixels. Since the signing of the video is done on decoded frames, the grouping of pixels need not be confined to any coding group partitioning.

[0026] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless explicitly defined in this specification. All references to a / an / the item, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one and possibly also more than one of the respective items, apparatus, component, means, step, etc. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless explicitly stated. BRIEF DESCRIPTION OF DRAWINGS

[0027] Aspects and embodiments will now be described, by way of example, with reference to the accompanying drawings, in which:

[0028] Figure 1A and Figure 1B Two example modes of intra-frame referencing within macroblocks in a video frame are shown;

[0029] Figure 2 A sequence of frames representing one group of pictures (GoP) with example modes of inter-frame referencing is shown;

[0030] Figure 3A , Figure 3B , Figure 3C and Figure 3D Four example correspondence modes between data units and the macroblocks they represent are shown;

[0031] Figure 4It is illustrated that an editing operation, which directly affects some macroblocks in the first frame (first column) and causes corresponding changes in the further frames (second and third column) until the end of the GoP;

[0032] Figure 5A It is shown a video sequence in the state before the edit (upper half) and a signed video bitstream comprising data units and associated signature units (lower half);

[0033] Figure 5B It is shown the same video sequence and signed video bitstream after the edit;

[0034] Figure 6 is a flowchart of a method of editing a signed video bitstream according to embodiments herein;

[0035] Figure 7 is a flowchart of a method of verifying a signed video bitstream according to embodiments herein;

[0036] Figure 8 It is shown a device suitable for performing the methods illustrated in Figure 6 and Figure 7 is a flowchart of a method of verifying a signed video bitstream according to embodiments herein;

[0037] Figure 9 It is shown a plurality of such devices connected through a local and / or wide area network; and

[0038] Figure 10A ("document method") and Figure 10B It is shown some operations within the methods illustrated in Figure 7 a signed video bitstream and DETAILED DESCRIPTION

[0039] Aspects of the disclosure will now be described in more detail below with reference to the accompanying drawings, in which certain embodiments of the application are shown. These aspects can, however, be implemented in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of all aspects of the application to those skilled in the art. Throughout the description, same reference numerals refer to same elements.

[0040] In the terminology of the present disclosure, a "video bitstream" comprises any substantially linear data structure, which can resemble a sequence of bit values. A video bitstream can be carried by a transitory medium, e.g. modulated electromagnetic or light waves, as in some streaming use cases, or the video bitstream can be stored on a non-transitory medium, such as a volatile or non-volatile memory.

[0041] A video bitstream represents a video sequence, which can be understood as a sequence of video frames played back sequentially at nominal time intervals. Each video frame can be divided into macroblocks. Furthermore, in this disclosure, a "macroblock" can be a transform block or a prediction block within a frame of a video sequence, or a block serving both purposes. The use of "frame" and "macroblock" herein is intended to be consistent with the H.26x video coding standard or similar specifications. A "macroblock" can also be a coded block. As mentioned above, although the term macroblock is used, the grouping of pixels is not necessarily limited to any partition used for coding; rather, a macroblock can be any group of adjacent pixels. When applied to prediction-based coding, using the same partition used for coding can be advantageous.

[0042] Figure 1A and Figure 1B An example of dividing video frame 100 into 4×4 uniformly arranged macroblocks 101 is illustrated. It should be noted that this is a simplification for illustrative purposes. In practice, video frame 100 is typically divided into a much larger number of macroblocks; for example, macroblocks could be 8×8 pixels, 16×16 pixels, or 64×64 pixels. The curved arrows are consistently used herein to indicate intra-frame or inter-frame references to be used in predictive coding. Without departing from the scope of this disclosure, Figure 1A and Figure 1B The macroblock partitioning seen in the video can vary significantly, including non-square arrangements and / or arrangements of macroblocks 101 that are dissimilar to video frame 100 and / or mixed arrangements of different macroblocks 101 with different sizes or shapes. It is understood that some video coding formats support dynamic macroblock partitioning, meaning that the partitioning can differ for different video frames in a sequence. H.265 is an example of this.

[0043] exist Figure 1A In this context, the intra-frame reference mode is limited to a single-line macroblock 101. In practice, each macroblock 101 references the macroblock immediately to its left (if it exists in frame 100), meaning it is represented by data units that predictively represent the image data within the macroblock, i.e., the image data in that macroblock is represented relative to the image data in the macroblock to its left. Conceptually, and slightly simplified, the data units represent macroblock 101 based on changes or shifts relative to the left macroblock. Another possible understanding is that the data units represent corrections to predefined prediction operations derived from the left macroblock for macroblock 101. In alternatives within the scope of this disclosure, macroblock 101 may reference another macroblock 101 to its right, above, or below it.

[0044] exist Figure 1BIntra reference is more dense. In this text, each macroblock 101 refers to the macroblock immediately to its left (if it exists in frame 100) and to the macroblock immediately above it (if it exists in frame 100). Accordingly, macroblock 101 is represented by a data unit that, for example, predicts the image data in the macroblock in terms of a difference or in terms of a prediction of the image data based on a predefined interpolation operation acting on the image data in the left and upper macroblocks (or some other predefined combination). The interpolation can include post-processing operations such as smoothing. Further alternatives within the scope of the present disclosure are to use intra reference in the opposite direction as shown in Figure 1B Intra reference is more dense. In this text, each macroblock 101 refers to the macroblock immediately to its left (if it exists in frame 100) and to the macroblock immediately above it (if it exists in frame 100). Accordingly, macroblock 101 is represented by a data unit that, for example, predicts the image data in the macroblock in terms of a difference or in terms of a prediction of the image data based on a predefined interpolation operation acting on the image data in the left and upper macroblocks (or some other predefined combination). The interpolation can include post-processing operations such as smoothing. Further alternatives within the scope of the present disclosure are to use intra reference in the opposite direction as shown in

[0045] An image / video format with predefined modes of intra reference can be associated with a specified scan order, which represents a feasible sequence for decoding the macroblocks. In Figure 1A In the case of the macroblock scan order being non-unique, i.e. since each row can be decoded independently. For the modes according to Figure 1B , the macroblocks can be decoded from above in rows or from the left in columns; a video format with such a reference mode can specify a scan order in columns or in rows, so that any reconstruction error can be anticipated at the encoder side, which facilitates coding efficiency. Furthermore, there are video formats with arbitrary macroblock ordering or so-called slices.

[0046] Figure 2 A video sequence V comprising a sequence of frames 100 is shown. There are independently decodable frames (I-frames) and predicted frames in the video sequence V, the predicted frames comprising uni-directional predicted frames (P-frames) and bi-directional predicted frames (B-frames). The recommendation ITU-T H.264 (06 / 2019) “Advanced video coding for generic audiovisual services” of the International Telecommunication Union specifies a video coding standard, in which both forward predicted frames and bi-directional predicted frames are used. As seen in Figure 2 The independently decodable frames do not refer to any other frame, as seen in Figure 2 The uni-directional predicted frames (P-frames) in are forward predicted, since they directly refer to at least one other preceding or immediately preceding frame. The bi-directional predicted frames (B-frames) can additionally directly refer to a following or immediately following frame in the video sequence V. If a video sequence comprises a third frame that directly refers to a first frame and that in turn directly refers to a second frame, the first frame indirectly refers to the second frame. In predictive video coding, a group of pictures (GoP) is defined as a sub-sequence of video frames, which does not refer to any video frame outside of this sub-sequence; it can be decoded without referring to any other I-frame, P-frame or B-frame. Figure 2 The video frames in form a GoP. Figure 2 The GOPs in are minimal, since they cannot be subdivided into further GOPs.

[0047] In a simpler implementation, a video sequence V can consist only of independently decodable frames (I) and single-direction predicted frames (P). Such a video sequence can have the following appearance: IPPIPPPPIPPPIPPP, where each P-frame refers to the immediately preceding I-frame or P-frame. The following GOPs can be identified in this example: IPP, IPPPP, IPPP, IPP.

[0048] There are several options for coordinating inter- and intra-frame prediction coding. For example, if a static (fixed) macroblock partitioning is used in all video frames, inter- references like the ones illustrated can be defined at the level of one macroblock position at a time (e.g., the top-left macroblock in Figure 1A and Figure 1B Some video formats allow dynamic macroblock partitioning, e.g., a macroblock can be predicted from a corresponding pixel in a preceding video frame or from a spatially shifted pixel in the preceding frame. Alternatively, inter- references are defined for entire video frames. I-frames consist only of I-blocks, while P-frames can consist only of P-blocks or a mix of I- and P-blocks.

[0049] Turning to Figure 3A , Figure 3B , Figure 3C and Figure 3D Attention is now directed towards a video bitstream encoding the video sequence under consideration. Figure 3A , Figure 3B , Figure 3C and Figure 3D Figures illustrate different correspondence modes between data units 102 and the macroblocks 101 they represent (encode). For the purposes of the present disclosure, data units can have any suitable format and structure; no assumption is made other than that a data unit can be separated (or extracted) from a video bitstream, e.g., to allow processing without any need to decode that data unit or any surrounding data units. In addition to data units, a signed video bitstream further comprises signed units that can be separated from the signed video bitstream in the same or a similar manner. Details about signed units will be presented below with reference to Figure 5A presentation.

[0050] Under one option, as illustrated in Figure 3A , data units 102 correspond one-to-one to macroblocks 101 (dotted lines). This correspondence mode allows each macroblock 101 to always be reconstructed from one data unit 102. Furthermore, if a macroblock 101 is edited, no other data unit 102 needs to be modified in addition to the corresponding data unit 102 (although modifications to signed units and metadata, etc. can of course be necessary).

[0051] Alternatively, as illustrated in Figure 3BAs illustrated, each macroblock 101 is encoded by a plurality of data units 102. Each data unit 102 still represents at most one macroblock 101. Therefore, if each macroblock 101 is encoded by at most M data units 102, then any macroblock 101 can therefore be reconstructed from at most M data units 102. Edits made to macroblock 101 have a direct effect on at most M data units 102. Assume that M is a small integer such as 1, 2, 3, 4, 5, or at most 10.

[0052] In other options, such as Figure 3C As illustrated, each data unit 102 encodes multiple macroblocks 101. This means that the effect of editing macroblock 101 is not limited to macroblock 101 itself, but may require re-encoding and / or re-signing of data units 102 shared by the edited macroblock 101 and other macroblocks 101. Because performing re-signing on unnecessarily large datasets can be computationally wasteful, this correspondence is not applied in the preferred implementation of this disclosure, although implementing this correspondence is highly possible. If data unit 102 is allowed to be shared by at most a predefined number N macroblocks 101, the total increase in computational effort can remain limited. Again, N can be specified as a small integer such as 1, 2, 3, 4, 5, or at most 10.

[0053] Within the scope of this disclosure, Figure 3B and Figure 3C Combinations of the patterns seen in the data are possible. As a result, the techniques proposed in this paper can be applied to video bitstreams, where the ratio of macroblocks 101 to data units 102 can be 2:1, 1:1, 2:2, 3:3, 4:2, 2:4, or 4:4.

[0054] In other options, such as Figure 3D As illustrated, each data unit 102 allows any number of macroblocks 101 in the video sequence to be represented, and each macroblock 101 can be encoded by any number of data units 102. As indicated by the dashed lines, this correspondence, in the worst case, implies that even limited editing operations on macroblocks 101 would require a complete re-encoding and re-signing of the video sequence. The techniques disclosed herein cannot be applied to... Figure 3D The structure shown is practiced on video sequences.

[0055] Figure 4 The illustration shows the editing operations that will be described below.

[0056] Figure 5A A portion of the video sequence V is depicted, comprising a series of macroblocks 101 belonging to one or more video frames. For example, macroblocks 101 may occupy fixed positions within consecutive video frames (e.g., Figure 1Aand Figure 1B The upper left macroblock in the upper left macroblock of the macroblock 101). It is assumed that the macroblock 101 has been predictively encoded according to the reference indicated by the curved arrow. The video sequence V is encoded into a signed video bitstream B comprising data units 102 and signature units 103. For the purpose of illustration and not limitation, Figure 5A The data units 102 are shown such that they encode the video macroblocks 101 according to a corresponding mode which can be described as Figure 3A and Figure 3B a mix of the modes shown in and allowing regular or irregular alternations between these modes (e.g. based on macroblock size, where larger macroblocks 101 correspond to multiple data units 102 and smaller or more compressed macroblocks 101 correspond to single data units 102). Nonetheless, each data unit 102 represents at most one macroblock 101. The data units 102 can comply with a proprietary or standardized video encoding format such as ITU-T H.264, H.265 or AV1. The bitstream B can further comprise additional types of units (e.g. dedicated metadata units) without departing from the scope of the present disclosure.

[0057] Each signature unit 103 can be associated with a plurality of data units 102. In Figure 5A It is understood that, in the case of two consecutive signature units 103, the data units 102 between these two consecutive signature units 103 are associated with the subsequent signature unit 103; this is not an essential feature of the invention and other conventions are possible without departing from the scope of the present disclosure. The signature units 103 can be associated with a set of data units 102 all contained in one GoP, but other association modes - like the one seen in Figure 5A Moreover, the set of data units 102 associated with one signature unit 103 is preferably chosen taking into account the applicable macroblock scan order. For example, the set of data units 102 associated with a signature unit 103 can represent the number of macroblocks that will be scanned sequentially during decoding, whereby the number of macroblocks that need to be revisited if the signature unit 103 fails verification is minimized.

[0058] The signature unit 103 comprises at least one bit string (e.g. H1) and a digital signature of the bit string (e.g. s(H1)). As shown by the use of dashed lines, the presence of a bit string is optional. In case the signature unit 103 comprises multiple bit strings, the signature unit 103 can have one digital signature for all these bit strings, or multiple digital signatures for each single bit string or for a subgroup of bit strings. The bit string of which the digital signature is formed can be a combination of the fingerprints of the associated data unit 102, or can be a fingerprint of the combination of said fingerprints of the associated data unit 102. The combination of fingerprints (or "document") can be a list or other concatenation of the string representations of the fingerprints. In the ITU-T H.264 and H.265 formats, the signature unit can be included in the video bitstream as a Supplemental Enhancement Information (SEI) message. In the AV1 standard, the signature can be included in a metadata Open Bitstream Unit (OBU).

[0059] Each of the fingerprints can be a hash or a salted hash. A salted hash can be a hash of a combination of the data unit (or part of the data unit) and a cryptographic salt; the presence of a salt can stop an unauthorized party with access to multiple hashes from guessing what hash function is being used. Potentially useful cryptographic salts include the value of an activity internal counter, a random number, and the time and place of the signature. The hash can be generated by a hash function (or one-way function) h, which is a cryptographic function that provides a level of security deemed sufficient, taking into account the sensitivity of the video data to be signed and / or taking into account values that would be at risk if the video data were manipulated by an unauthorized party. Three examples are SHA-256, SHA3-512, and RSA-1024. The hash function should be pre-defined (e.g. it should be reproducible) so that when the receiving party is to verify the fingerprints, the fingerprints can be regenerated. In the example of a salted hash, the bit string is given by: Figure 5A

[0060] H1 = h([h1, h2, h3, h4, h5, h6])

[0061] and

[0062] H2 = h([h7, h8, h9, h 10 ]),

[0063] where h1, h2,... are hashes of data units, and [·] denotes concatenation. An example salted hash can be defined as:

[0064] H'1 = h([h1, h2, h3, h4, h5, h6], σ)

[0065] or

[0066] H"1 = h([h1, h2, h3, h4, h5, h6, σ]),​

[0067] where σ is a cryptographic salt. In the first example, the hash function h has a parameter dependency on the second argument, the salt having been assigned to the second argument.

[0068] In some embodiments, each of the fingerprints h1, h2,... is computed directly from the data unit 102, e.g. from the encoded transform coefficients or other video data therein. The fingerprints can be computed from the entire data unit or from a subset of the entire data unit that has been extracted according to a pre-agreed rule. In other embodiments, the fingerprints h1, h2,... are computed from the reconstructed macroblock that is obtained by decoding the data unit 102, e.g. the pixel values or other plaintext data. In yet other embodiments, the fingerprints h1, h2,... are computed neither at the plaintext level nor at the bitstream level, but from intermediate reconstruction data that is derived from the data unit. More precisely, if an encoder is used that comprises a frequency domain transform (e.g. DCT, DST, DFT, wavelet transform) followed by an encoding process (e.g. entropy, Huffman, Lempel-Ziv, run length, binary or non-binary arithmetic encoding such as context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC)), then the transform coefficients will normally be available as intermediate reconstruction data at the decoder side. The transform coefficients can be recovered from the encoded representation. If the encoder further comprises a quantization process immediately downstream of the transform, then the quantized transform coefficients will be available at the decoder side. In more complex codecs with a larger number of sequential processing stages, there can be additional types of intermediate reconstruction data, and these can be used for fingerprint computation. It is particularly convenient to use an intermediate reconstruction data that occurs identically in the encoding process as the quantized transform coefficients. Common to all embodiments reviewed in this paragraph, the fingerprints belong to exactly one data unit 102 that is associated with the signature unit 103.

[0069] Optionally, to discover unauthorized removal or insertion of data units, the fingerprints can be linked together sequentially. That is, each fingerprint depends on the next or previous fingerprint, e.g. the input for the hash includes the hash of the next or previous fingerprint. Linking can be implemented, for example, as follows: h1= h(X 102.1 ), h2= h([h1, X 102.2 ]), h3= h([h2, X 102.3 ]), etc., where X 102.1 , X 102.2 , X 102.3 denote data from a first, second and third data unit in the data unit 102.

[0070] Still referring to Figure 5AThe signing unit 103 in the recording device can make use of a cryptographic element (not shown) having a pre-stored private key in order to generate the digital signature s(H1). The recipient of the signed video bitstream can be assumed to hold the public key belonging to the same key pair (see also Figure 10A and Figure 10B ), which enables the recipient to verify that the signature produced by the cryptographic element is authentic, but does not enable the recipient to generate new signatures. The public key can also be included as metadata in the signed video bitstream, in which case it is not necessary to store it at the recipient side.

[0071] Reference is made to Figure 6 A method 600 of editing the signed video bitstream B obtained by predictive coding of a video sequence V will now be described. It is assumed that the non-optional steps of the method 600 are performed after the original signing of the video bitstream. For example, if the signed video bitstream is originally generated on a recording device, the editing method 600 can be performed in a video management system (VMS). Another example use case is that the signed video bitstream is generated at a device, stored in a memory, and then accessed again using the same device for editing. The editing can take place at a later point in time, for example, after the need to perform privacy screening has become known.

[0072] Although the device performing the editing method 600 as described can be an application or system dedicated to a specific purpose, it can have the basic functional structure shown in Figure 8 . As illustrated, the device 800 comprises a processing circuit 810, a memory 820 and an external interface 830. The memory 820 can be adapted to store a computer program 821 having instructions implementing the editing method 600. The external interface 830 can be a communication interface allowing the device 800 to communicate with similar devices (not shown) held by the recipient and / or the author of the video content, e.g., a recording device, or it can allow read and write operations in an external memory 890 adapted to store the video bitstream.

[0073] Figure 9 The transmission of the bitstream between multiple devices is illustrated. It is noted that the device performing the editing method 600 can be connected to the recipient device through a local area network (LAN) (connection line in the lower half of Figure 9 or through a wide area network 990. Attacks on the bitstream B can take place on either type of network, which justifies the signing.

[0074] Returning to Figure 6One embodiment of the method 600 starts with a step 612 of receiving a request to replace a region of at least one video frame 100 in the video sequence V. The request can be received from a human operator via a human-machine interface or in an automated manner, for example, in a message from a control application executing on the same device 800 or remotely. The region to be replaced can be a set of replacement pixel values, such as a privacy mask, to replace similarly located pixels in the video sequence V.

[0075] For the avoidance of doubt, it is noted that the video sequence V to be edited is encoded by predictive encoding into a signed video bitstream B comprising data units 102 and associated signature units 103, wherein each data unit represents at most one macroblock 101 in a video frame 100 of the predictively encoded video sequence V, and wherein each signature unit comprises a digital signature of a bit string derived from a plurality of fingerprints of each exactly one associated data unit. Such a bitstream format has already been referred to with respect to Figure 5A Example.

[0076] In a next step 614 of the method 600, a first set of macroblocks in which the region is contained and a second set of macroblocks directly or indirectly referencing the first set of macroblocks are determined. Recalling that bi-directionally predicted frames (B-frames) can be defined in some video encoding formats, it is understood that the second set of macroblocks can be located before or after the first set of macroblocks, or occupy both locations. It is understood that the first and second sets are defined to be disjoint. For example, it can be stipulated that a macroblock belongs to the second set only if it does not belong to the first set, i.e. only if it is not needed to form the set of macroblocks containing the region to be replaced. Thus, if the first set of macroblocks extends to the boundary of a GoP, the second set of macroblocks is normally empty. Furthermore, it is understood that the second set of macroblocks can contain more than one P-frame or more than two B-frames of macroblocks, depending on the initially used video encoder, which additional frames can use the replacement region as a reference.

[0077] If the region to be replaced is limited to a single video frame, the first set of macroblocks can be determined with reference to the macroblock partitioning of that frame only. More precisely, the first set is all macroblocks that the region overlaps with, i.e. that have a non-empty intersection with it in pixel space. If the region extends to multiple frames, this operation is repeated for each frame. In the special case where the region is identically repeated in all video frames and additionally the macroblock partitioning is constant throughout all said frames, the first set of macroblocks is a copy of the macroblocks determined for the initial frame (by the overlap criterion) for each subsequent frame. The second set of macroblocks can be determined based on the pattern of intra- and inter-frame references in the predictively encoded video sequence and the signature. Since such references by definition do not extend beyond the GoP boundary, the search for macroblocks to be included in the second set can be limited to that GoP or those GoPs to which the first set of macroblocks belongs.

[0078] The possible results of step 614 are illustrated in the diagram. Figure 4 In this context, each column represents a video frame of the video sequence V, and each row represents a macroblock at a specific location within that frame (e.g., the top-left macroblock). Furthermore, in... Figure 4 In the diagram, references between macroblocks are indicated by curved arrows, and the boundary between two consecutive GOPs (GOP1 and GOP2) is shown as a vertical dashed line. Note that in... Figure 4 In this context, inter-frame references are defined at a level within a macroblock location. Furthermore, diagonally hashed macroblocks are those directly affected by requests for replacement regions; they are all located in frame 1 and form the first group of macroblocks 401. Macroblocks with dotted shading are those that directly (frame 2) or indirectly (frame 3) reference macroblocks in the first group, and they are identified as the second group of macroblocks 402. Consistent with expectations, the second group of macroblocks does not extend beyond the GoP boundary.

[0079] Note that if intra-frame reference is introduced, then Figure 4 The combination of the first and second macroblock groups seen in the image can be changed, but it is not mandatory. For example, if the macroblock position corresponding to the first row references the macroblock position corresponding to the second row, then the first macroblock 401 and the second macroblock 402 will remain unchanged.

[0080] The next step 616 of method 600 will be about Figure 5B To explain, Figure 5B The image shows the video sequence V and the signed video bitstream B after region replacement has been performed. Figure 5B In the first group of macroblocks 101.2 and 101.3, diagonal hashing is used, while dotted shading is used for macroblocks 101.4 and 101.5 in the second group of macroblocks.

[0081] In step 616, archive object 104 is added to the signed video bitstream B. Archive object 104 includes fingerprints of a first set of data units and a second set of data units, i.e., data units representing the first set of macroblocks and the second set of macroblocks determined in step 614, respectively. While not strictly necessary, it is preferred that the fingerprints be individual fingerprints belonging to exactly one data unit. In embodiments where fingerprints are calculated from macroblocks reconstructed from data units, which is one of the above options, and where recoding is expected to faithfully preserve these macroblocks, the fingerprints of the second set of data units do not need to be included in archive object 104. At the level of the signed video bitstream B, archive object 104 has a similar format to data unit 102 and signature unit 103 because archive object 104 can be separated from the video bitstream without decoding.

[0082] exist Figure 5BIn the example, the first group of data units corresponds to the third, fourth and fifth data units 102, and the second group of data units corresponds to the sixth and seventh data units 102. Accordingly, the (two) archive objects 104 added to the signed video bitstream B in step 616 will together store the fingerprints h3, h4, h5, h6, h7. Optionally, each archive object 104 can comprise a digital signature of these fingerprints, or a digital signature of the combination of these fingerprints in that archive object 104, or it can comprise a digital signature of the combination of fingerprints as described. Further optionally, the archive object 104 can also comprise the locations of the first group of macroblocks and the second group of macroblocks whose signatures have been archived. The locations can refer to the locations of the macroblocks in the frame, e.g. in frame coordinates, and this in turn corresponds to locations in the bitstring. If a static macroblock partitioning is used, the locations of the macroblocks 101 can be expressed as macroblock sequence numbers or another identifier. The bitstring can be formed, for example, by concatenating the fingerprints in the same order as the macroblock sequence in the frame.

[0083] The execution flow of the editing method 600 then continues with step 618, in which the first group of data units is edited according to the request for replacement of the region. In other words, step 618 leaves the signed video bitstream B with some data units replaced or modified.

[0084] In some embodiments, step 618 can comprise decoding 618.1 the data units into reconstructed macroblocks; providing 618.2 edited macroblocks by performing the requested replacement on the reconstructed macroblocks; and providing edited data units by encoding 618.3 the edited macroblocks.

[0085] Optionally, the macroblocks resulting from the replacement 618.2 (e.g. Figure 5B macroblocks 101.2 and 101.3 in the frame 101) can be encoded into independently decodable data units. The independently decodable data units can correspond to I-frames in the H.264 or H.265 encoding specification, encoded macroblocks that do not reference another macroblock, or data units equivalent to these. This is in line with the implementation of a replacement that introduces a sudden temporal change in the video sequence, which reduces the performance of predictive encoding.

[0086] In the next step 620 of the method 600, the second group of data units is re-encoded. The generally desired goal is to decode the re-encoded second group of data units into macroblocks that are as similar as possible to the (original) second group of macroblocks. With reference to the decoder application, the goal is that the re-encoded second group of data units produces almost identical reference buffer contents and / or almost identical decoder state. However, while the macroblocks in the second group (e.g. Figure 5BThe macroblocks in the second group should advantageously be re-encoded 620 using reduced data compression. This will be understood in the context of the video sequence V being encoded with a predetermined regular level of data compression. More precisely, it is foreseen that the macroblocks in the second group are encoded with a reduced level of data compression compared to the regular data compression.

[0087] The second group of macroblocks can advantageously be re-encoded 620 using reduced data compression. This will be understood in the context of the video sequence V being encoded with a predetermined regular level of data compression. More precisely, it is foreseen that the macroblocks in the second group are encoded with a reduced level of data compression compared to the regular data compression.

[0088] For encoding the second group of macroblocks in step 620, the main options are non-predictive encoding and predictive encoding. If non-predictive encoding is used, the second group of macroblocks is encoded as independently decodable intra-coded blocks, also referred to as I-blocks. Thus, the encoded second group of macroblocks will be represented by independently decodable data units. Under this option, it is also possible to use lossless encoding for the second group of macroblocks; for example, the second group of macroblocks can be represented by a plain list of the original values for each position in the macroblock in the appropriate color space.

[0089] Under the second option (predictive encoding), step 620 can be performed by re-encoding the second group of data units using predictive encoding with reference to the edited first group of data units. More precisely, applying step 620 to a data unit can comprise: obtaining 620.1 a macroblock reconstructed from a further data unit directly referenced by said data unit; decoding 620.2 said data unit using the reconstructed macroblock; obtaining 620.3 a reconstructed edited version of the macroblock; and providing 620.4 the edited data unit by re-encoding said data unit with predictive encoding with reference to the reconstructed edited version of the macroblock. Sub-step 620.1 can comprise decoding the further data unit (see step 618.1), and this further data unit can belong to the first group of data units or to the second group of data units. In sub-step 620.3, the reconstructed edited version of the macroblock can correspond to the image data resulting from the above sub-step 618.2, i.e. by performing the requested replacement on the reconstructed macroblock. Sub-step 620.4 can comprise representing the macroblock in the second group (i.e. the macroblock being processed) in terms of the difference or correction with respect to the macroblock of the reconstructed edited version (which originates from said further data unit). As mentioned, this option is mainly useful if the editing performed in the first group of macroblocks is relatively limited, or if predictive encoding can not be satisfactorily performed.

[0090] In an optional final step 622 of the method 600, one or more signature units 105 associated with the edited first set of data units and the re-encoded second set of data units are provided in the signed video bitstream B. The signature units 105 provided in step 622 can have the same structure as the signature units 103 described above. Accordingly, the signature units 105 can comprise a bit string derived from the fingerprints of each of the edited data units and a digital signature of the bit string, or the signature units 105 can comprise only the digital signature. As Figure 5B As shown in Fig. 6, the signature units 105 provided in step 622 can be newly generated signature units. Alternatively, the signature units 105 are provided by editing existing signature units, in particular by extending it with a further digital signature.

[0091] As already mentioned, the steps of any of the methods disclosed herein do not necessarily have to be performed in the exact order as described, unless explicitly stated. This is in particular illustrated by the editing method 600, wherein step 616 can clearly be performed before, between or after the sub-sequence of steps 618 and 620, as desired.

[0092] In some embodiments, the method 600 further comprises an initial step 610 of providing at least one signature unit 103. It is understood that, in use cases considered to be of primary interest, step 610 is performed by a different entity than steps 612, 614, 616, 618, 620 and 622 of the method 600, and / or step 610 is performed at an earlier point in time. Either way, step 610 is separated from the subsequent steps 612, 614, 616, 618, 620 and 622 by a relatively insecure data transmission and / or a reasonable storage period of the proof signature to ensure the desired level of data security.

[0093] The optional step 610 can comprise the following sub-steps: computing 610.1 a plurality of fingerprints of the individual data units associated with the signature unit; deriving 610.2 a bit string from the plurality of fingerprints; and obtaining 610.3 a digital signature of the bit string, wherein the bit string is a combination of the plurality of fingerprints or a fingerprint of the combination. Suitable implementations of the fingerprint computation 610.1, the bit string derivation 610.2 and the digital signature 610.3 have been discussed in detail above. In particular, the bit string to which the digital signature in the signature unit 103 belongs can be a combination of the fingerprints of the associated data units 102, or it can be a fingerprint of the combination of the fingerprints of the associated data units 102. The combination (or "document") can be a list of individual string representations of the fingerprints or another concatenation.

[0094] Having thus completed the description of the editing method 600, attention is now directed to the receiver side. More precisely, reference is made to Fig. 7, which shows a method 700 of verifying the authenticity of a video bitstream B.Figure 7 The method 700 of verifying a signed video bitstream B will be described with reference to the flowchart in Fig. 7. Again, it is assumed that the signed video bitstream B has been obtained by predictive encoding of a video sequence V and optionally by subsequent editing operations. It is not necessary that the signed video bitstream B has been processed according to the editing method 600. Furthermore, it is assumed that the signed video bitstream comprises data units 102, associated signature units 103 and archive objects 104. Herein, each data unit 102 represents at most one macroblock 101 in a frame 100 of the predictively encoded video sequence V, each signature unit 103 comprises a digital signature (e.g. s(H1), s(H2)) of a bitstring (e.g. H1, H2) and optionally the bitstring itself, and the archive objects 104 comprise at least one fingerprint, which can be an archival fingerprint of a data unit that is not present in the bitstream B now and / or has been edited. This is irrelevant for the verification method 700 and generally cannot be determined at the receiver side whether a particular signature unit 103 is added in conjunction with an edit (e.g. by the editing method 600) or part of the original, unedited bitstream B.

[0095] In an optional first step 710 of the method 700, which is only performed in some embodiments ("documentation method"), the bitstring H1 in one signature unit 103 is verified using the digital signature s(H1) in order to verify in a manner known per se that the fingerprint contained therein is authentic. As Figure 10A As illustrated in Fig. 10, the verification can be performed using a cryptographic element 1001, which is located in the receiver device 800 and in which the public key is deposited. This can be described as an asymmetric signature setup, in which the signature and verification are different cryptographic operations corresponding to the private / public keys. Other combinations of symmetric and / or asymmetric verification operations are possible without departing from the scope of the present disclosure. If the result V1 of the bitstring verification is negative (reject), the execution of the method 700 ends. If, on the contrary, the result V1 is positive (accept), the execution of the method 700 continues to a second step 712.

[0096] In the second step 712, the fingerprint h1, h2,... of each data unit 102 associated with the signature unit 103 is obtained. An independent decision 712.1 can be made for each data unit as to how the fingerprint is obtained. More precisely, the fingerprint is either computed 712.2 from the data unit or retrieved 712.3 from the archive object 104 in the bitstream. As explained above, the fingerprint can be computed directly from the data unit 102 (of the subset) (e.g. from the transform coefficients or other video data therein) or from the reconstructed macroblock obtained by decoding the data unit 102. In the case of a subset of data units, the fingerprint can be computed from the subset as a whole or from each data unit in the subset individually. Figure 10A and Figure 10B As seen in Figs. 7 and 8, the fingerprint h1, h2,... is compared 712.4 with the corresponding signature s(H1), s(H2) in the signature unit 103. If the result V2 of the comparison is negative (reject), the execution of the method 700 ends. If, on the contrary, the result V2 is positive (accept), the execution of the method 700 continues to a third step 714. The fingerprints are calculated from data unit 102, while the remaining fingerprints h3, h4, h5, and h6 are retrieved from file object 104. Accordingly, only the fingerprints are retrieved. This will cause the verification in the upcoming fourth step 716 to fail, in which case the failure indicates that unauthorized manipulation of bitstream B has occurred.

[0097] In step 714, the bit string is derived from the fingerprint thus obtained. This can be accomplished according to pre-agreed rules, for example, through a process similar to that described in step 610.2. The bit string can be a combination of acquired fingerprints, or it can be a combination of fingerprints of said fingerprints.

[0098] Finally, in step 716, the digital signature in signature unit 103 is used to verify the data unit associated with the signature unit 103 under consideration. To avoid confusion, it is important to note that the verification of the data unit in step 716 is indirect, without any processing acting on the data unit itself.

[0099] In embodiments where the signature unit 103 does not contain the bit string H1, the derived bit string is verified using the digital signature s(H1). To perform step 716. For example, the derived bit string can be verified using a public key that belongs to the same key pair as the private key used to generate the digital signature s(H1). exist Figure 10B In this, this is achieved by deriving the bit string. The digital signature s(H1) is fed to a cryptographic entity 1001 that stores the public key, and the cryptographic entity 1001 outputs a binary result W1 representing the verification result.

[0100] Alternatively, in an embodiment where the signature unit 103 does indeed contain the bit string H1 (“Document Method”), the bit string H1 has already been initially verified in step 710, and then in step 716 the verified bit string H1 is compared with the derived bit string A comparison is performed. This comparison can be a bitwise equality check, such as by... Figure 10A As shown in function block 1002, it produces an output V2 that is either true or false. If the comparison result V2 is true, it can be concluded that, with respect to this signature unit 103, the signed video bitstream B is genuine.

[0101] The execution of the method 700 can then comprise repeating the relevant steps among the above steps 710, 712, 714 for any further signing units 103 in the signed video bitstream B. If the result is positive for all signing units 103, it is concluded that the signed video bitstream B is valid, and it can be further consumed or processed. In the opposite case, the signed video bitstream B will be considered as not authentic, and it can be quarantined to prevent any further use or processing.

[0102] It is noted that the verification of the data units in the first group is based on a different trust relationship than the verification of the data units in the second group. The data units in the first group are verified by trusting the entity that created the digital signature s(H1), which is the holder of the private key if asymmetric key encryption is used. The data units in the second group are verified by trusting the entity that edited the signature bitstream B and created the archive object.

[0103] In some embodiments of the verification method 700, the decision in sub-step 712.1 is guided by locations indicated in the archive object 104. These locations are the locations of the macroblocks 101 represented by the data units 102 involved in the archival fingerprints. Having access to these macroblock locations allows the recipient to perform a reliable integrity check based on the assumption that any macroblock 101 in the video frame 100 that cannot be reconstructed from the data units 102 in the signed video bitstream B is encoded by another data unit whose fingerprint can have to be retrieved from the archive object 104. If the archive object 104 does not indicate the locations of these macroblocks, the recipient can insert the missing fingerprints, e.g. by trial and error - fingerprints that cannot be computed from the data units 102 in the signed video bitstream B. The trial and error can comprise performing steps 714 and 716 for each possible way of inserting the archival fingerprints from the archive object 104 (each such way of insertion can be imagined as an arrangement of the locations of the missing macroblocks), and concluding that the signed video bitstream B is not authentic only when all these executions fail.

[0104] The aspects of the disclosure have been primarily described above with reference to several embodiments. However, as will be readily appreciated by those skilled in the art, other embodiments than those explicitly described above are equally possible within the scope of the application defined by the appended claims. In particular, it is noted that the above description of various embodiments has focused on predictively coded video. However, the same approach can be used for any video, not just predictively coded video, as long as the entity signing the video and the entity verifying the video have access to the reconstructed or decoded frames of the video. It can be seen that in the case of prediction-based coding, it is actually convenient to employ fingerprints that are also used as groups of pixels in the encoding. However, in general, the fingerprints can be computed from groups of pixels that are otherwise grouped. Once the frame is decoded, it does not matter which partitioning of the pixels was used for encoding. For example, depending on the type of edit that is intended, it can be useful to divide the decoded image into smaller or larger groups of pixels than were used for encoding. For example, if masking will always be done in rectangular form, then a coarser pixel partitioning can be sufficient for the signing process as compared to what would be used for encoding. On the other hand, if it is envisioned that masking can be done more closely following the contours of the objects to be masked, then a finer partitioning can be useful for the signing process.

Claims

1. A method of editing a signed video bitstream obtained by predictive coding of a video sequence, The signed video bitstream comprises data units and associated signature units, wherein, each data unit representing at most one macroblock in a video frame of said predictively coded video sequence, and wherein each signature unit comprises a digital signature of a bit string derived from a plurality of fingerprints of each exactly one associated data unit, said method comprising: receiving a request to replace a region of at least one video frame; determining a first set of macroblocks comprising said region and a second set of macroblocks directly or indirectly referencing macroblocks in the first set; adding an archival object to said signed video bitstream, said archival object comprising fingerprints of data units of the first and second sets, said data units of the first and second sets representing the determined first and second sets of macroblocks, respectively, wherein said archival object comprises locations of the first and second sets of macroblocks; editing said data units of the first set in accordance with said request to replace said region of said at least one video frame, wherein said editing of said data units of the first set comprises: decoding said data units into reconstructed macroblocks, providing edited macroblocks by performing said requested replacement on said reconstructed macroblocks, and providing edited data units by encoding said edited macroblocks; and re-encoding said data units of the second set.

2. The method of claim 1, wherein, said macroblocks after said replacement are coded into independently decodable data units.

3. The method of claim 1, further comprising decoding data units in the second set using said reconstructed macroblocks to facilitate said re-encoding.

4. The method of claim 1, wherein, said data units of the second set are re-encoded using a simplified data compression.

5. The method of claim 1, wherein, said data units of the second set are re-encoded into independently decodable data units.

6. The method of claim 1, further comprising: providing one or more signature units in said signed video bitstream associated with said edited data units of the first set and said re-encoded data units of the second set.

7. The method of claim 1, further comprising initially providing a signature unit by: computing a plurality of fingerprints of respective data units associated with said signature unit; deriving a bit string from said plurality of fingerprints; and obtaining a digital signature of that bit string, wherein, said bit string being a combination of said plurality of fingerprints or a fingerprint of said combination.

8. The method of claim 7, wherein, each fingerprint is computed from: a) from said data unit; or b) from a macroblock reconstructed from said data unit; or c) from intermediate reconstructed data derived from said data unit.

9. The method of claim 1, wherein, said received request is to replace regions in a plurality of video frames.

10. A method of verifying a signed video bitstream obtained by predictive coding of a video sequence, The signed video bitstream includes data units, associated signature units, and archive objects, wherein, each data unit representing at most one macroblock in a frame of a predictively coded video sequence, wherein each signature unit comprises a digital signature of a bit string, and wherein said archival object comprises at least one archival fingerprint of a data unit, said method comprising: obtaining a fingerprint of each data unit associated with a signature unit by: if the archive object indicates a location of a macroblock represented by a data unit to which the archival fingerprint belongs, retrieving an archival fingerprint from the archive object; or if the archive object does not indicate the location of the macroblock, computing a fingerprint of the data unit; deriving a bit string from the fingerprint obtained; and verifying the data unit associated with the signature unit using the digital signature in the signature unit, including: if the signature unit does not include a bit string, verifying the bit string derived using the digital signature; or if the signature unit includes a bit string, verifying the bit string in the signature unit using the digital signature, and comparing the bit string derived and the bit string verified.

11. The method of claim 10, wherein: the obtaining of a fingerprint includes determining whether to compute the fingerprint of a data unit or retrieve the fingerprint from the archive object based on the location indicated by the archive object.

12. An apparatus comprising processing circuitry arranged to perform the method of claim 1.

13. A non-transitory computer-readable recording medium comprising a computer program including instructions that, when executed by a computer, cause the computer to perform the method of claim 1.

Images