Apparatus and method for digital watermarking

Abstract

An apparatus (110) for embedding a sequence of message bits (101) into a video stream (103) is disclosed. The apparatus (110) is configured to determine one or more transformed frames of at least one of a plurality of frames (103a-103d) of the video stream (103) based on a wavelet kernel filter. Furthermore, the apparatus (110) is configured to encode the sequence of message bits (101) by, for each bit of the sequence of message bits (101), encoding the sequence of message bits (101) in accordance with a value of a respective message bit of the sequence of message bits (101), point-by-point multiplication is performed on absolute values of elements of subframes of the one or more transformed frames with elements of the first encoding matrix or elements of the second encoding matrix to obtain one or more modified transformed frames. The apparatus (120) is further configured to add the one or more modified transform frames to the at least one of the plurality of frames (103a to 103d) of the video stream (103) to embed the sequence of message bits (101) into the video stream (103).

Description

Technical Field

[0001] The present invention relates to data processing. More particularly, the present invention relates to an apparatus and method for digital watermarking of video streams. Background Art

[0002] As the use and distribution of digital multimedia data increases, content protection becomes increasingly important to prevent unrestricted copying and dissemination of copyrighted materials. Digital watermarking technology emerges as a complementary method to encryption for content protection of copyrighted materials. Digital watermarking technology embeds a secret, imperceptible signal, i.e., a watermark, into the original content. It usually exists with the original content and survives the transformation, conversion, and transcoding process.

[0003] In a video attack, a video stream displayed on a monitor is re-recorded using a video camera. In the process, the re-recorded video stream may undergo different kinds of significant changes, but still keep the video stream of good quality to enjoy its content. An example of such an attack could exist in a cinema room, where a malicious user uses a video camera (e.g. a smartphone camera) to record the movie being shown.

[0004] The main means to thwart such attacks is to embed an imperceptible watermark in the video content that survives re-recordings and contains information that helps identify possible leak points. Although there are some known techniques that attempt to combat such attacks, the main problem remains largely unsolved, as existing techniques have weak robustness and survivability and require precise external conditions to work. Besides this, the computational power and per-frame latency required to embed the watermark make these existing techniques unsuitable for live broadcast scenarios, where (near) real-time embedding of the watermark is an essential function and the entire content of the video is not known at the beginning of the embedding process. Furthermore, the re-recording process may introduce new challenges related to frame realignment. Most conventional watermarking techniques require realigning the video to its original shape. This process is not automated in most cases and requires human intervention to select the four corners of the display. Summary of the invention

[0005] An object of the present invention is to provide an apparatus and method for digital watermarking.

[0006] The above and other objects are achieved by the subject matter claimed in the independent claims. Other implementations are apparent from the dependent claims, the description and the drawings.

[0007] More specifically, a data processing device (also referred to as an encoding device) is provided for embedding (i.e., encoding) a message bit sequence (i.e., a digital watermark) into a video stream comprising a plurality of frames. The data processing device comprises: a processing circuit for modifying at least one frame of the plurality of frames of the video stream by:

[0008] determining one or more transformed frames of the at least one frame of the plurality of frames of the video stream based on a wavelet kernel filter;

[0009] The message bit sequence is encoded in the following manner: for each bit of the message bit sequence, according to the value of the corresponding message bit of the message bit sequence, the absolute value of the element of the subframe of the one or more transformed frames is multiplied point by point (also called element by element multiplication, i.e., element by element) with the element of the first coding matrix (also called the first coding mode) or the element of the second coding matrix (also called the second coding mode) to obtain one or more modified transformed frames;

[0010] The one or more modified transformed frames are added to the at least one original frame to embed the message bit sequence into the at least one frame of the video stream.Therefore, an improved encoding apparatus for embedding a digital watermark into a video stream is provided.

[0011] In another possible implementation, the processing circuit is configured to determine the one or more transformed frames of the at least one frame based on the wavelet kernel filter by convolving the wavelet kernel filter along one or more directions of the at least one original frame. The wavelet kernel filter may be defined by a one-dimensional array of filter coefficients.

[0012] In another possible implementation, the processing circuit is used to determine the one or more transformed frames by convolving the wavelet kernel filter along the horizontal direction of the at least one frame, convolving the wavelet kernel filter along the vertical direction of the at least one frame, and / or convolving the wavelet kernel filter along the diagonal direction of the at least one frame.

[0013] In another possible implementation manner, the video stream includes a YUV video stream, and the at least one original frame among the multiple frames of the YUV video stream is a Y frame (also referred to as a Y channel).

[0014] In another possible implementation, the processing circuit is further configured to reduce the size of at least one frame among the multiple frames to obtain a reduced-size version of the at least one frame, and to modify the reduced-size version of the at least one frame among the multiple frames of the video stream.

[0015] In another possible implementation manner, the processing circuit is configured to reduce the size of the at least one frame among the multiple frames by a factor of 2 or higher to obtain a reduced-size version of the at least one frame.

[0016] In another possible implementation, the processing circuit is further configured to upsize corresponding downsized versions of the one or more modified transformed frames to add the corresponding modified transformed frames to the at least one frame to embed the message bit sequence into the at least one frame of the video stream.

[0017] In another possible implementation, elements of the first encoding matrix (ie, encoding mode) have values ​​of +1 or −1, and the second encoding matrix (ie, encoding mode) is a result of point-wise multiplication of the elements of the first encoding matrix with −1.

[0018] In another possible implementation manner, the processing circuit is configured to generate the first encoding matrix and / or the second encoding matrix based on a random number generator.

[0019] In another possible implementation manner, the processing circuit is used to generate the first encoding matrix and / or the second encoding matrix to have a desired spatial frequency distribution and / or a desired orthogonality.

[0020] In another possible implementation, the processing circuit is further used to multiply at least a portion of the elements of the one or more modified transformed frames by a scaling factor less than 1 before adding the one or more modified transformed frames to the at least one original frame to embed the message bit sequence into the at least one original frame of the video stream.

[0021] According to a second aspect, a method for embedding a message bit sequence (i.e., a digital watermark) into a video stream comprising a plurality of frames is disclosed. The method comprises the following steps:

[0022] determining one or more transformed frames of at least one original frame of the plurality of frames of the video stream based on a wavelet kernel filter;

[0023] Encoding the message bit sequence in the following manner: for each bit of the message bit sequence, performing point-wise multiplication of absolute values ​​of elements of subframes of the one or more transformed frames with elements of a first encoding matrix or elements of a second encoding matrix according to values ​​of corresponding message bits of the message bit sequence, so as to obtain one or more modified transformed frames;

[0024] The one or more modified transformed frames are added to the at least one original frame to embed the message bit sequence into the at least one original frame of the video stream.

[0025] The encoding method according to the second aspect can be performed by the encoding device according to the first aspect. Therefore, other features of the encoding method according to the second aspect are directly derived from the functions of the encoding device according to the first aspect and its different implementations described above and below.

[0026] According to a third aspect, a data processing apparatus for decoding (i.e., retrieving) a message bit sequence (i.e., a digital watermark) from a video stream comprising a plurality of frames is disclosed. The data processing apparatus comprises: a processing circuit for decoding the message bit sequence from at least one of the plurality of frames of the video stream by:

[0027] determining one or more transformed frames of the at least one frame of the plurality of frames of the video stream based on a wavelet kernel filter;

[0028] determining a composite transformed frame based on the one or more transformed frames or a sum of the one or more transformed frames;

[0029] The message bit sequence is decoded from the composite transform frame by determining a first correlation value with a first coding matrix and a second correlation value with a second coding matrix for one or more subframes of the composite transform frame, and determining the message bit sequence based on the first correlation value and the second correlation value for each of the one or more subframes of the composite transform frame. Therefore, an improved decoding apparatus for extracting a digital watermark from a video stream is provided.

[0030] In another possible implementation, the processing circuit is configured to determine the one or more transformed frames of the at least one frame based on the wavelet kernel by convolving the wavelet kernel along one or more directions of the at least one frame.

[0031] In another possible implementation, the processing circuit is used to determine the one or more transformed frames by convolving the wavelet kernel filter along the horizontal direction of the at least one frame, convolving the wavelet kernel filter along the vertical direction of the at least one frame, and / or convolving the wavelet kernel filter along the diagonal direction of the at least one frame.

[0032] In another possible implementation manner, the video stream includes a YUV video stream, and the at least one frame among the multiple frames of the YUV video stream is a Y frame.

[0033] In another possible implementation, elements of the first encoding matrix (ie, encoding mode) have values ​​of +1 or −1, and the second encoding matrix (ie, encoding mode) is a result of point-wise multiplication of the elements of the first encoding matrix with −1.

[0034] In another possible implementation, the processing circuit is used to generate the first coding matrix (i.e., coding mode) and / or the second coding matrix (i.e., encoding mode), or receive the first coding matrix (i.e., decoding mode) and / or the second coding matrix (i.e., decoding mode) from an encoding device, and the encoding device is used to encode the message bit sequence into the video stream.

[0035] In another possible implementation manner, the processing circuit is used to generate the first encoding matrix (ie, the first decoding mode) and / or the second encoding matrix (ie, decoding mode) to have a desired spatial frequency distribution and / or a desired orthogonality.

[0036] According to a fourth aspect, a data processing method for decoding (i.e., retrieving) a message bit sequence (i.e., a digital watermark) from a video stream comprising a plurality of frames is disclosed. The data processing method comprises the following steps:

[0037] determining one or more transformed frames of at least one of the plurality of frames of the video stream based on a wavelet kernel filter;

[0038] determining a composite transformed frame based on the one or more transformed frames or a sum of the one or more transformed frames;

[0039] The message bit sequence is decoded from the composite transform frame in the following manner: a first correlation value with a first coding matrix and a second correlation value with a second coding matrix are determined for one or more subframes of the composite transform frame, and the message bit sequence is determined based on the first correlation value and the second correlation value of each subframe in the one or more subframes of the composite transform frame.

[0040] The decoding method according to the fourth aspect can be performed by the decoding device according to the third aspect. Therefore, other features of the decoding method according to the fourth aspect are directly derived from the functions of the decoding device according to the third aspect and its different implementations described above and below.

[0041] According to a fifth aspect, a computer program or a computer program product is provided, comprising a computer-readable storage medium carrying a program code, wherein when the program code is executed by a computer or a processor, the computer or the processor executes the method according to the second aspect or the method according to the fourth aspect.

[0042] Various aspects of the invention can be implemented in software and / or hardware.

[0043] The details of one or more embodiments are set forth in the following drawings and the description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The embodiments of the present invention will be described in detail below with reference to the following drawings, in which:

[0045] Figure 1 A schematic diagram of an encoding device provided by an embodiment and a decoding device provided by an embodiment, wherein the encoding device is used to encode a message into a video stream, and the decoding device is used to retrieve the message from the video stream;

[0046] Figure 2 is a schematic diagram showing a frame of a video stream having an embedded digital watermark;

[0047] Figure 3a and Figure 3b An exemplary first estimated coding matrix and a second estimated coding matrix are shown, that is, a coding mode estimated for an encoding device provided in one embodiment and a decoding device provided in one embodiment;

[0048] Figure 4a and Figure 4b A first encoding mode and a second encoding mode used by an encoding device provided by an embodiment and a decoding device provided by an embodiment are shown;

[0049] Figure 5 A flowchart illustrating a computer-implemented method for encoding a message into a video stream provided in one embodiment;

[0050] Figure 6 A computer-implemented flow diagram for retrieving messages from a video stream is provided to illustrate one embodiment.

[0051] In the following the same reference signs refer to identical or at least functionally equivalent features. DETAILED DESCRIPTION

[0052] In the following description, reference is made to the accompanying drawings that form a part of the present disclosure, which illustrate by way of illustration specific aspects of embodiments of the present invention or specific aspects in which embodiments of the present invention may be used. It should be understood that embodiments of the present invention may be used in other aspects and include structural changes or logical changes not described in the accompanying drawings. Therefore, the following detailed description is not to be construed as limiting, and the scope of the present invention is defined by the appended claims.

[0053] For example, it should be understood that the disclosure related to the described method can also be applicable to the corresponding device or system for performing the method, and vice versa. For example, if one or more specific method steps are described, the corresponding device may include one or more units, such as functional units, for performing the described one or more method steps (e.g., one unit performs the one or more steps, or multiple units each perform one or more of the multiple steps), even if such one or more units are not explicitly described or illustrated in the figure. On the other hand, for example, if a specific device is described in terms of one or more units (e.g., functional units), the corresponding method may include a step to perform the function of the one or more units (e.g., one step performs the function of the one or more units, or multiple steps each perform the function of one or more of the multiple units), even if such one or more steps are not explicitly described or illustrated in the figure. In addition, it should be understood that unless otherwise specifically stated, the features of the various exemplary embodiments and / or aspects described herein may be combined with each other.

[0054] Figure 1 The schematic diagram of an encoding device 110 provided by an embodiment and a decoding device 160 provided by an embodiment is shown, wherein the encoding device 110 is used to encode a message including a plurality of message bits 101 (also called a digital watermark) into a plurality of video frames 103a to 103d ( Figure 2 In a video frame 103 (shown in FIG. 1 ), a decoding device 160 is used to retrieve a message including a plurality of message bits 105 (ie, a digital watermark) from the video stream 103 . Figure 2 1 is a schematic diagram showing a plurality of exemplary frames 103a to 103d of a video stream 103 with an embedded digital watermark. In one embodiment, the video stream 103 may be a YUV video stream 103, and the encoding device may be used to embed the plurality of message bits 101 into one or more Y frames (or channels) of the YUV video stream 103.

[0055] The encoding device 110 and / or the decoding device 160 may be a data processing device, such as a server, a desktop computer, a laptop computer, a tablet computer, or other devices having computing resources for implementing the embodiments disclosed herein.

[0056] like Figure 1As shown, the encoding device 110 includes a processing circuit 120, such as one or more processors 120 for processing data. The processing circuit 120 of the encoding device 110 can be implemented by hardware and / or software. The hardware may include a digital circuit, or both an analog circuit and a digital circuit. The digital circuit may include components such as an application-specific integrated circuit (ASIC), a field-programmable array (FPGA), a digital signal processor (DSP) or a general-purpose processor. In addition, the encoding device 110 may include a communication interface 130 for wired and / or wireless communication with a decoding device 160, etc. through a communication channel 150. The encoding device 110 may also include a memory 140 for storing and retrieving data. The memory 140 of the encoding device 110 may store executable program code, and when the executable program code is executed by the processing circuit 120, the executable program code enables the encoding device 110 to perform the functions and methods described herein.

[0057] Likewise, the decoding device 160 includes a processing circuit 170, such as one or more processors 170 for processing data. The processing circuit 170 of the decoding device 160 may be implemented by hardware and / or software. The hardware may include a digital circuit, or both an analog circuit and a digital circuit. The digital circuit may include components such as an application-specific integrated circuit (ASIC), a field-programmable array (FPGA), a digital signal processor (DSP), or a general-purpose processor. In addition, the decoding device 160 may include a communication interface 180 for wired and / or wireless communication with the encoding device 110, etc. through a communication channel 150. The decoding device 160 may also include a memory 190 for storing and retrieving data. The memory 190 of the decoding device 160 may store executable program code, which, when executed by the processing circuit 170, causes the decoding device 160 to perform the functions and methods described herein.

[0058] As will be further referenced below Figure 3a , Figure 3b and Figure 4a , Figure 4bTo describe in more detail, in order to embed the message bit sequence 101 into the video stream 103 , the processing circuit 120 of the encoding device 110 is configured to modify at least one frame of the plurality of frames 103 a to 103 d of the video stream 103 in the following manner. Figure 4a and Figure 4b 4 shows a first coding matrix or pattern 401a and a second coding pattern or matrix 401b used by a coding device 110 provided by an embodiment and a decoding device 160 provided by an embodiment, Figure 3a and Figure 3b A first estimated encoding mode 301a and a second estimated encoding mode 301b are shown.

[0059] In a first stage of the embedding process, the processing circuit 120 of the encoding device 110 is used to determine one or more transformed frames of the at least one frame of the plurality of frames 103a to 103d of the video stream 103 based on a wavelet kernel filter. In one embodiment, the one or more transformed frames may have the same size, i.e., the same number of pixels, as the at least one frame of the plurality of frames 103a to 103d of the video stream 103. In one embodiment, the wavelet kernel filter may be a one-dimensional kernel filter. In one embodiment, the wavelet kernel filter may be a wavelet kernel filter based on a biorthogonal wavelet.

[0060] In one embodiment, the processing circuit 120 of the encoding device 110 is used to determine the one or more transformed frames of the at least one frame in the multiple frames 103a to 103d of the video stream 103 by convolving the wavelet kernel filter along one or more directions of the at least one frame in the multiple frames 103a to 103d of the video stream 103 based on the wavelet kernel filter. In one embodiment, the processing circuit 120 of the encoding device is used to determine the one or more transformed frames by convolution of the wavelet kernel filter along the horizontal direction of the at least one frame, convolution of the wavelet kernel filter along the vertical direction of the at least one frame, and convolution of the wavelet kernel filter along the diagonal direction of the at least one frame. It should be understood that in this case, the one or more transformed frames will include three transformed frames.

[0061] In the second stage of the embedding process, the processing circuit 120 of the encoding device 110 is used to encode the message bit sequence 101 in the following manner: for each bit of the message bit sequence 101, according to the value of the corresponding message bit of the message bit sequence 101, the absolute value of the element of the subframe of the one or more transform frames is multiplied point by point (also called element by element multiplication, i.e., element by element) with the element of the first encoding matrix 401a (also called the first encoding mode 401a) or the element of the second encoding matrix 401b (also called the second encoding mode 401b) to obtain one or more modified transform frames. In one embodiment, the multiple subframes can cover the entire corresponding transform frame. In one embodiment, the size M×N of the multiple subframes should be an integer fraction of the size of the corresponding transform frame. The height M and width N of the multiple subframes can be selected in different ways to maximize robustness (larger height and width, i.e., 8×8) or maximize the embeddable space (i.e., 4×4).

[0062] As described above, the exemplary first encoding mode 401a and the exemplary second encoding mode 401a are respectively Figure 4a and Figure 4b . In one embodiment, the elements of the first coding pattern 401a have a value of +1 or -1, and the second coding pattern 401b is the result of point-by-point multiplication of the elements of the first coding pattern 401a and -1 (the positive and negative signs of the values ​​of the first coding pattern 401a are inverted). In one embodiment, the processing circuit 120 of the encoding device 120 is used to generate the first coding pattern 401a and the second coding pattern 401b based on a random number generator. In one embodiment, the processing circuit 120 of the encoding device 110 is used to generate the first coding pattern 401a and the second coding pattern 401b to have a desired spatial frequency distribution and / or a desired degree of orthogonality.

[0063] In the third stage of the embedding process, the processing circuit 120 of the encoding device 110 is further configured to add the one or more modified transformed frames (generated by the two previous stages of the embedding process described above) to the at least one frame of the plurality of frames 103a to 103d of the video stream 103, so as to embed the message bit sequence 101 into the at least one frame of the plurality of frames 103a to 103d of the video stream 103. In one embodiment, the processing circuit 120 of the encoding device 110 is further configured to multiply at least a portion of the elements of the one or more modified transformed frames by a scaling factor less than 1 and greater than 0 before adding the one or more modified transformed frames to the at least one frame of the plurality of frames 103a to 103d of the video stream 103, so as to embed the message bit sequence 101 into the at least one frame of the plurality of frames 103a to 103d of the video stream 103.

[0064] In one embodiment, the processing circuit 120 of the encoding device 110 may also be configured to downscale the at least one frame among the plurality of frames 103a to 103d of the video stream 103 to obtain a downsized version of the at least one frame among the plurality of frames 103a to 103d of the video stream 103, and modify the downsized version of the at least one frame among the plurality of frames 103a to 103d of the video stream 103 in the manner described above. In one embodiment, the processing circuit 120 of the encoding device 110 is configured to downscale the at least one frame among the plurality of frames 103a to 103d of the video stream 103 by a factor of 2 or more to obtain a downsized version of the at least one frame among the plurality of frames 103a to 103d of the video stream 103. In an embodiment including such downsizing, the processing circuit 120 of the encoding device 120 may also be used to upscale the downsized versions of the one or more modified transformed frames to add the modified transformed frames to the at least one frame among the multiple frames 103a to 103d of the video stream 103 to embed the message bit sequence 101 into the at least one frame among the multiple frames 103a to 103d of the video stream 103.

[0065] In order to decode the digital watermark embedded into the video stream 103 by the encoding device 110 in the manner described above, the processing circuit 170 of the decoding device 160 is used to perform the decoding process described below.

[0066] In the first stage of the decoding process, the processing circuit 170 of the decoding device 160 is used to determine one or more transformed frames of the at least one frame in the plurality of frames 103a to 103d of the video stream 103 based on the wavelet kernel filter already described above. It should be understood that the encoding device 110 performs the same operation. Therefore, the one or more transformed frames may have the same size as the at least one frame in the plurality of frames 103a to 103d of the video stream 103, i.e., the same number of pixels. In one embodiment, the wavelet kernel filter may be a one-dimensional kernel filter.

[0067] In one embodiment, the processing circuit 170 of the decoding device 160 is used to determine the one or more transform frames of the at least one frame in the plurality of frames 103a to 103d of the video stream 103 by convolving the wavelet kernel filter along one or more directions of the at least one frame in the plurality of frames 103a to 103d of the video stream 103 based on a wavelet kernel filter. In one embodiment, the processing circuit 170 of the decoding device 160 is used to determine the one or more transform frames by convolution of the wavelet kernel filter along the horizontal direction of the at least one frame, convolution of the wavelet kernel filter along the vertical direction of the at least one frame, and convolution of the wavelet kernel filter along the diagonal direction of the at least one frame. It should be understood that in this case, the one or more transform frames will include three transform frames. In order to determine the composite transform frame based on the one or more transform frames, the processing circuit 170 of the decoding device 160 is also used to determine, for example, the sum of the three transform frames (for a single transform frame, the processing circuit 170 of the decoding device 160 can use the single transform frame as the composite transform frame).

[0068] In the second stage of the decoding process, the processing circuit 170 of the decoding device 160 is used to decode (i.e., retrieve) the message bit sequence 105 from the composite transform frame by determining, for one or more subframes of the composite transform frame, a first correlation value with the first coding matrix 401a and a second correlation value with the second coding matrix 401b, and determining the message bit sequence 105 based on the first correlation value and the second correlation value of each of the one or more subframes of the composite transform frame. It should be understood that in order to extract the digital watermark 105, the processing circuit 170 of the decoding device 160 is used to use the same coding matrix or pattern 401a, 401b as the encoding device 110 used to embed the digital watermark 101. In one embodiment, the processing circuit 170 of the decoding device can be used to generate the first coding matrix 401a and / or the second coding matrix 401a itself (for example, based on a seed value received from the encoding device 110), or to receive the first coding matrix 401a and / or the second coding matrix 401b from the encoding device 110. In other words, the processing circuit 170 of the decoding device 160 is used to extract the message bit sequence 105 from the video stream 103 using a correlation technique in order to distinguish between the coding mode 401a for encoding, for example, a 0 bit and the coding mode 401b for encoding, for example, a 1 bit of the original message bit sequence 101. More specifically, for each subframe of size M×N, a first correlation value and a second correlation value are determined.

[0069] It should be understood that the embodiments disclosed herein can be fine-tuned to embed the watermark 110 into the most solid part of the frame content, namely the edges. Unlike smooth surfaces, the edges in the video frame are the basic pillars that allow the recognition of shapes and are mostly preserved. Edges are associated with specific frequencies. Generally speaking, images are composed of high frequencies, medium frequencies, and low frequencies. High frequencies represent fine details in the image, medium frequencies can be associated with most edges, and low frequencies represent slow gradient changes (sunset gradients of light) that appear in the image. The embodiments disclosed herein focus specifically on medium frequencies for the following two main reasons. First, medium frequencies provide a good compromise between robustness and embeddable space. Although low frequencies are the most robust, they do not allow for a satisfactory embeddable space. Second, medium frequencies provide a good compromise between robustness and imperceptibility. High-frequency information belongs to the tiny optical details of the image and is very suitable for hiding information. Unfortunately, however, this high-frequency information is very vulnerable to common attacks such as scaling and noise.

[0070] As described above, the embodiments disclosed herein make special use of the intermediate frequency by wavelet transforms implemented by wavelet kernel filters, for example, for convolution. Unlike Fourier transforms, wavelet transforms are very suitable for extracting time-local frequency information. Although the encoding device 110 and the decoding device 160 provided in the embodiment can also use mature wavelet transforms such as DTCWT and CWT, the computational cost of these full-scale wavelet transforms is very high, so it is difficult to use them in real-time embedding scenes. For this reason, the embodiments disclosed herein use the above-mentioned wavelet kernel filters to use convolutions with lower computational requirements. As described above, in one embodiment, different convolutions in different directions can be combined to extract frequency information in different directions. The results of these convolutions provide a transform frame representing frequency information, for example, along the main horizontal, vertical and diagonal directions. Other sub-directions can be added to obtain a more accurate frequency representation. Each element of the transform frame contains information about the frequency magnitude (obtained by its absolute value) and its phase at that point.

[0071] As described above, in one embodiment, the encoding device 110 and the decoding device 160 can use a pseudo-random pattern of values ​​+1 and -1 as the encoding matrix 401a, 401b, wherein the size of the encoding matrix 401a, 401b is M×N, and the second encoding matrix 401b can be generated by flipping the positive and negative signs of each element of the first encoding matrix 401a. M and N are parameters that can be adjusted to meet different properties (such as robustness or increase embedding space). As described above, in one embodiment, the distribution of the values ​​of the encoding patterns 401a, 401b can be selected to achieve different properties. The encoding patterns 401a, 401b consisting of alternating rows of 1 and -1 can create decoding invariance in the horizontal direction during the extraction process. This means that even if there is no perfect alignment on the x-axis, the signal can still be extracted due to this property.

[0072] The coding modes 401a, 401b used by the encoding device 110 and the decoding device 160 can improve performance in the following three ways. First, more symbol coefficients can be used to represent the bits, which increases redundancy and thus improves the robustness of the scheme. Second, the degree of randomness of the coding modes 401a, 401b can increase imperceptibility because the user can perceive it as random noise. Third, choosing opposite modes maximizes the chance of distinguishing the first mode 401a (for encoding, for example, a 0 bit) from the second mode 401b (for encoding, for example, a 1 bit) during decoding.

[0073] Figure 5 Flowchart of an embodiment of a computer-implemented method 500 for embedding (i.e., encoding) a message bit sequence 101 (i.e., a digital watermark 101) into a video stream 103 comprising a plurality of frames 103a to 103d. The method 500 comprises a step 501 of determining one or more transformed frames of at least one of the plurality of frames 103a to 103d of the video stream 103 based on a wavelet kernel filter, such as the wavelet kernel filter described above. In addition, the method 500 comprises a step 503 of encoding the message bit sequence by performing, for each bit of the message bit sequence, a point-wise multiplication of an absolute value of an element of a subframe of the one or more transformed frames with an element of a first encoding matrix 401a or an element of a second encoding matrix 401b according to a value of a corresponding message bit of the message bit sequence to obtain one or more modified transformed frames. The method 500 further comprises step 505 of adding the one or more modified transformed frames to the at least one frame of the plurality of frames 103a to 103d of the video stream 103 to embed the message bit sequence 101 into the at least one frame of the plurality of frames 103a to 103d of the video stream 103 .

[0074] Figure 6Flowchart of an embodiment of a computer-implemented data processing method 600 for decoding (i.e., retrieving) a message bit sequence 105 from a video stream 103 including a plurality of frames 103a to 103d. The data processing method 600 comprises a step 601 of determining one or more transform frames of at least one of the plurality of frames 103a to 103d of the video stream 103 based on a wavelet kernel filter, such as the wavelet kernel filter described above. In addition, the method 600 comprises a step 603 of determining a composite transform frame based on the one or more transform frames or a sum of the one or more transform frames. The method 600 further comprises a step 605 of decoding the message bit sequence 105 from the composite transform frame by determining, for one or more subframes of the composite transform frame, a first correlation value with the first encoding matrix 401a described above and a second correlation value with the second encoding matrix 401b described above, and determining the message bit sequence 105 based on the first correlation value and the second correlation value of each of the one or more subframes of the composite transform frame.

[0075] Those skilled in the art will understand that the “blocks” (“units”) in the various figures (methods and devices) represent or describe the functions of an embodiment of the present invention (and not necessarily separate “units” in hardware or software), and therefore equally describe the functions or features of the device embodiments and the method embodiments (units are equivalent to steps).

[0076] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the above-mentioned device embodiments are merely exemplary. For example, the unit division is only a logical function division and can be other divisions in the actual implementation. For example, multiple units or components can be merged or integrated into another system, or some features can be ignored or not performed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be implemented through some interfaces. The indirect coupling or communication connection between devices or units can be implemented in electronic, mechanical or other forms.

[0077] The units described as discrete parts may be physically separated or not, and the parts shown as units may be physical units or not, and may be located in one location or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0078] In addition, the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

Claims

1. A data processing device (110) for embedding a message bit sequence (101) into a video stream (103) comprising a plurality of frames (103a to 103d), the data processing device (110) comprising: A processing circuit (120) for modifying at least one of the plurality of frames (103a to 103d) of the video stream (103) by: determining one or more transformed frames of the at least one frame among the plurality of frames (103a to 103d) of the video stream (103) based on a wavelet kernel filter; The message bit sequence (101) is encoded in the following manner: for each bit of the message bit sequence (101), according to the value of the corresponding message bit of the message bit sequence (101), the absolute value of the element of the subframe of the one or more transformed frames is multiplied point by point with the element of the first encoding matrix (401a) or the element of the second encoding matrix (401b) to obtain one or more modified transformed frames; The one or more modified transformed frames are added to the at least one frame of the plurality of frames (103a to 103d) of the video stream (103) to embed the message bit sequence (101) into the at least one frame of the plurality of frames (103a to 103d) of the video stream (103).

2. The data processing device (110) according to claim 1, wherein: The processing circuit (120) is used to determine the one or more transformed frames of the at least one frame among the multiple frames (103a to 103d) of the video stream (103) based on the wavelet kernel filter by convolving the wavelet kernel filter along one or more directions of the at least one frame among the multiple frames (103a to 103d) of the video stream (103).

3. The data processing device (110) according to claim 2, wherein: The processing circuit (120) is used to determine the one or more transformed frames by convolving the wavelet kernel filter along the horizontal direction of the at least one frame, convolving the wavelet kernel filter along the vertical direction of the at least one frame, and / or convolving the wavelet kernel filter along the diagonal direction of the at least one frame.

4. The data processing device (110) according to any one of the preceding claims, wherein: The video stream (103) comprises a YUV video stream (103), and the at least one frame among the plurality of frames (103a to 103d) of the YUV video stream (103) is a Y frame.

5. The data processing device (110) according to any one of the preceding claims, wherein: The processing circuit (120) is also used to reduce the size of at least one frame among the multiple frames (103a to 103d) of the video stream (103) to obtain a reduced-size version of the at least one frame among the multiple frames (103a to 103d) of the video stream (103), and to modify the reduced-size version of the at least one frame among the multiple frames (103a to 103d) of the video stream (103).

6. The data processing device (110) according to claim 5, wherein: The processing circuit (120) is used to downsize the at least one frame among the multiple frames (103a to 103d) of the video stream (103) by a factor of 2 or higher to obtain a downsized version of the at least one frame among the multiple frames (103a to 103d) of the video stream (103).

7. The data processing device (110) according to claim 5 or 6, wherein: The processing circuit (120) is further configured to upsize the downsized versions of the one or more modified transformed frames to add the modified transformed frames to the at least one frame of the plurality of frames (103a to 103d) of the video stream (103) to embed the message bit sequence (101) into the at least one frame of the plurality of frames (103a to 103d) of the video stream (103).

8. The data processing device (110) according to any one of the preceding claims, wherein: The elements of the first encoding matrix (401a) have a value of +1 or -1, and the second encoding matrix (401b) is a result of point-wise multiplication of the elements of the first encoding matrix (401a) and -1.

9. The data processing device (110) according to claim 8, wherein: The processing circuit (120) is used to generate the first encoding matrix (401a) and / or the second encoding matrix (401b) based on a random number generator.

10. The data processing device (110) according to claim 9, wherein: The processing circuit (120) is used to generate the first encoding matrix (401a) and / or the second encoding matrix (401b) to have a desired spatial frequency distribution and / or a desired degree of orthogonality.

11. The data processing device (110) according to any one of the preceding claims, wherein: The processing circuit (120) is also used to multiply at least a portion of the elements of the one or more modified transformed frames by a scaling factor less than 1 before adding the one or more modified transformed frames to at least one frame of the multiple frames (103a to 103d) of the video stream (103) to embed the message bit sequence (101) into the at least one frame of the multiple frames (103a to 103d) of the video stream (103).

12. A method (500) for embedding a message bit sequence (101) into a video stream (103) comprising a plurality of frames (103a to 103d), the method (500) comprising: Determining (501) one or more transformed frames of at least one frame of the plurality of frames (103a to 103d) of the video stream (103) based on a wavelet kernel filter; The message bit sequence (101) is encoded (503) by: for each bit of the message bit sequence (101), performing point-wise multiplication of absolute values ​​of elements of subframes of the one or more transformed frames with elements of a first encoding matrix (401a) or elements of a second encoding matrix (401b) according to the value of a corresponding message bit of the message bit sequence (101) to obtain one or more modified transformed frames; The one or more modified transformed frames are added (505) to the at least one frame of the plurality of frames (103a to 103d) of the video stream (103) to embed the message bit sequence (101) into the at least one frame of the plurality of frames (103a to 103d) of the video stream (103).

13. A data processing device (160) for decoding a message bit sequence (105) from a video stream (103) comprising a plurality of frames (103a to 103d), the data processing device (160) comprising: A processing circuit (170) is configured to decode the message bit frame (105) from at least one frame of the plurality of frames (103a to 103d) of the video stream (103) by: determining one or more transformed frames of the at least one frame among the plurality of frames (103a to 103d) of the video stream (103) based on a wavelet kernel filter; determining a composite transformed frame based on the one or more transformed frames or a sum of the one or more transformed frames; The message bit sequence (105) is decoded from the composite transform frame in the following manner: a first correlation value with a first coding matrix (401a) and a second correlation value with a second coding matrix (401b) are determined for one or more subframes of the composite transform frame, and the message bit sequence (105) is determined based on the first correlation value and the second correlation value of each subframe in the one or more subframes of the composite transform frame.

14. The data processing device (160) according to claim 13, wherein: The processing circuit (170) is used to determine the one or more transformed frames of the at least one frame among the multiple frames (103a to 103d) of the video stream (103) based on the wavelet kernel filter by convolving the wavelet kernel filter along one or more directions of the at least one frame among the multiple frames (103a to 103d) of the video stream (103).

15. The data processing device (160) according to claim 13 or 14, wherein: The processing circuit (170) is used to determine the one or more transformed frames by convolving the wavelet kernel filter along the horizontal direction of the at least one frame, convolving the wavelet kernel filter along the vertical direction of the at least one frame and / or convolving the wavelet kernel filter along the diagonal direction of the at least one frame.

16. The data processing device (160) according to any one of claims 13 to 15, wherein: The video stream (103) comprises a YUV video stream (103), and the at least one frame among the plurality of frames (103a to 103d) of the YUV video stream (103) is a Y frame.

17. The data processing device (160) according to any one of claims 13 to 16, wherein: The elements of the first encoding matrix (401a) have a value of +1 or -1, and the second encoding matrix (401b) is a result of point-wise multiplication of the elements of the first encoding matrix (401a) and -1.

18. The data processing device (160) according to claim 17, wherein: The processing circuit (170) is used to generate the first coding matrix (401a) and / or the second coding matrix (401b), or receive the first coding matrix (401a) and / or the second coding matrix (401b) from the device (110), wherein the device (110) is used to encode the message bit sequence (101) into the video stream (103) based on the first coding matrix (401a) and / or the second coding matrix (401b).

19. The data processing device (160) according to claim 18, wherein: The processing circuit (170) is used to generate the first encoding matrix (401a) and / or the second encoding matrix (401b) to have a desired spatial frequency distribution and / or a desired degree of orthogonality.

20. A data processing method (600) for decoding a message bit sequence (105) from a video stream (103) comprising a plurality of frames (103a to 103d), the data processing method (600) comprising: Determining (601) one or more transformed frames of at least one frame of the plurality of frames (103a to 103d) of the video stream (103) based on a wavelet kernel filter; determining (603) a composite transformed frame based on the one or more transformed frames or a sum of the one or more transformed frames; The message bit sequence (105) is decoded (605) from the composite transform frame by determining, for one or more subframes of the composite transform frame, a first correlation value with a first coding matrix (401a) and a second correlation value with a second coding matrix (401b), and determining the message bit sequence (105) based on the first correlation value and the second correlation value for each of the one or more subframes of the composite transform frame.

21. A computer program product comprising a computer-readable storage medium for storing program code, which, when executed by a computer or a processor, causes the computer or the processor to perform the method (500) according to claim 12 or the method (600) according to claim 20.

Images