A Method and System for Watermark Embedding, Extraction, and Comparison Based on 3D Video

Abstract

The present invention relates to a watermark embedding, extraction and comparison method and system based on 3D video, and relates to the field of information security technology. Each two-dimensional video frame to be embedded in a three-dimensional video is divided into blocks to obtain a plurality of sub-blocks; any component in the sub-block color space is subjected to multi-level DT-CWT decomposition to obtain the sub-bands of the sub-blocks under each level of decomposition; the amplitude matrix of the sub-band is modified according to the watermark to be embedded to obtain the modified sub-band; the modified sub-band is subjected to inverse DT-CWT operation to obtain the embedded watermark sub-block corresponding to the sub-block; all embedded watermark sub-blocks corresponding to the two-dimensional video frame are spliced ​​to obtain the watermark-embedded two-dimensional video frame corresponding to the two-dimensional video frame; all watermarks corresponding to the three-dimensional video to be embedded are embedded in the two-dimensional video frame to obtain the watermark-embedded video and the zero watermark of the three-dimensional video to be embedded. The present invention can further improve the accuracy of copyright protection under the premise of ensuring the high robustness of the watermark.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to a 3D video-based watermark embedding, extraction and comparison method and system. Background Art

[0002] As 3D display technology matures, the threshold for the dissemination and production of 3D products is gradually lowered, and it has been widely used in many fields, such as movies, games, distance education and medical treatment. Ordinary consumers can also enjoy 3D technology at home. As an information carrier, on the one hand, 3D video can carry more information than 2D video and provide viewers with a more novel visual experience. On the other hand, the production cost of 3D video is high. Illegal copying and illegal use of 3D video will bring more serious losses to the copyright owner than 2D video. The DIBR (Depth-image-based-Rendering) method is a low-cost technology for generating 3D video. It shifts pixels horizontally and fills holes in a series of 2D video frames and corresponding depth maps to obtain virtual left and right stereo images, and then produces 3D video frames. The DIBR method can generate new forms of media content and can also be used as a tool to evaluate the robustness of watermarks. However, since the 3D video generated by the DIBR method comes from two carriers, namely the original two-dimensional frame and the depth map, it also brings difficulties to the copyright protection of 3D video. Moreover, the synthesized frame after DIBR operation has horizontal pixel shift relative to the original two-dimensional frame, which causes some distortion to the image content. This means that the protection scheme for 3D video should be DIBR-invariant, and the source of content to be protected is more complex than that of traditional two-dimensional video, so it is challenging. In addition, the research methods for copyright protection of two-dimensional multimedia technology products started early and are becoming more mature. Therefore, the research on copyright protection of 3D video is more practical and valuable. The copyright protection of 3D video has also become a hot issue in the field of digital media copyright protection in recent years.

[0003] In order to effectively protect the copyright of videos, protection methods can be divided into two categories:

[0004] Watermark embedding: Using the redundancy of the carrier, information such as text, logo and image is embedded in the carrier to be protected. If the watermark can be extracted from the carrier later, the copyright can be proved. The general methods are spatial domain and frequency domain. In the spatial domain, the classic methods are the least significant bit (LSB) method and the histogram method. For example, the histogram method constructs a histogram for the two-dimensional video frame and then selects a pixel range of appropriate size to embed the watermark according to the histogram. In the frequency domain, the two-dimensional video frame is decomposed by using discrete Fourier transform, discrete wavelet transform and dual-tree complex wavelet transform. Through the reversible quantization method, the coefficients of the low-frequency subband are slightly changed in combination with the watermark to achieve embedding.

[0005] Zero watermark feature construction: The zero watermark scheme does not directly embed the watermark into the 3D video content, but generates the relationship between the 3D video features and the watermark, ensuring that no direct information is embedded in the 3D video, thus not causing any impact on its content. Common methods include converting the 2D video frames into the frequency domain, formatting the wavelet coefficients therein for binarization; or calculating the moment results of the left and right stereo images based on various moments, such as Zernike moments, radial harmonic Fourier moments, and Chebyshev octet moments, extracting the features of the two frames of stereo images and constructing hash values ​​for them.

[0006] However, the above prior art has the following disadvantages:

[0007] (1) Although the robustness and distinguishability of existing two-dimensional frame-based digital watermarking schemes can meet the requirements, the two-dimensional frame-based digital watermarking schemes embed information in the video frames, which will inevitably destroy the association between pixels and cause irreversible distortion of the video content. This is not allowed in fields with high precision requirements.

[0008] (2) The zero watermark algorithm based on no direct information embedding can generate a mapping relationship between the watermark representing the video features and the watermark with copyright information, so there is no need to embed the watermark directly in the video frame to protect the copyright of the 3D video. Therefore, compared with the traditional watermark algorithm, the zero watermark protection scheme will not affect the quality of the video content. It is better than the watermark embedding scheme in terms of video quality and watermark robustness, but it cannot guarantee sufficient robustness to achieve good distinguishability, that is, the robustness and recognizability performance are still difficult to meet the requirements, because robustness and recognizability are contradictory. At present, we can only find the balance between them as much as possible and compromise when the performance of the two reaches an acceptable level. Therefore, ensuring the high robustness of the algorithm is still a challenge. Summary of the invention

[0009] The purpose of the present invention is to provide a watermark embedding, extraction and comparison method and system based on 3D video, which can further improve the accuracy of copyright protection under the premise of ensuring high robustness of the watermark.

[0010] To achieve the above object, the present invention provides the following solutions:

[0011] A watermark embedding method based on 3D video, comprising:

[0012] Divide each two-dimensional video frame to be embedded in the three-dimensional video into blocks to obtain a plurality of sub-blocks;

[0013] For any sub-block in any two-dimensional video frame, perform multi-level DT-CWT decomposition on any component in the color space of the sub-block to obtain sub-bands of the sub-block at each level of decomposition;

[0014] Modifying the amplitude matrix of the sub-band obtained by decomposing the sub-block within a set range of levels according to the watermark to be embedded to obtain a modified sub-band;

[0015] Performing an inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block;

[0016] Splicing all the watermark-embedded sub-blocks corresponding to the two-dimensional video frame to obtain a watermark-embedded two-dimensional video frame corresponding to the two-dimensional video frame;

[0017] Embedding the two-dimensional video frame according to all watermarks corresponding to the three-dimensional video to be embedded obtains the watermark-embedded video and the zero watermark of the three-dimensional video to be embedded.

[0018] Optionally, embedding a two-dimensional video frame according to all watermarks corresponding to the three-dimensional video to be embedded to obtain a zero watermark of the three-dimensional video to be embedded specifically includes:

[0019] The two-dimensional video frames corresponding to the watermarks to be embedded in the three-dimensional video are divided into blocks to obtain a plurality of watermark sub-blocks;

[0020] For any watermark sub-block embedded in any two-dimensional video frame, all components in the color space of the watermark sub-block are decomposed by multi-level DT-CWT to obtain sub-bands of the watermark sub-block at each level of decomposition;

[0021] Performing an SVD operation on the amplitude matrix of the sub-band obtained under the set level decomposition of the watermark sub-block to obtain the singular value corresponding to the watermark sub-block;

[0022] Generate a zero watermark of the watermark-embedded two-dimensional video frame according to the singular values ​​corresponding to all the watermark sub-blocks in the watermark-embedded two-dimensional video frame;

[0023] The zero watermark of the to-be-embedded three-dimensional video is generated by embedding the zero watermark of the two-dimensional video frame according to all watermarks corresponding to the to-be-embedded three-dimensional video.

[0024] Optionally, the step of modifying the amplitude matrix of the sub-band obtained by decomposing the sub-block within a set range of levels according to the watermark to be embedded to obtain the modified sub-band specifically includes:

[0025] Calculating the amplitude matrix and phase matrix of the sub-band obtained by decomposing the sub-block within a set series range;

[0026] According to the watermark to be embedded, the amplitude matrix of the sub-band obtained by decomposing the sub-block within a set range of levels is quantized to obtain a quantized sub-band amplitude matrix corresponding to the sub-block;

[0027] A modified subband is obtained according to the phase matrix and the quantized subband amplitude matrix corresponding to the subblock.

[0028] A watermark extraction method based on 3D video, comprising:

[0029] Divide each 2D video frame in the 3D video to be extracted into blocks to obtain a plurality of extraction sub-blocks;

[0030] For any extracted sub-block in any two-dimensional video frame, perform multi-level DT-CWT decomposition on any component in the color space of the extracted sub-block to obtain sub-bands of the extracted sub-block at each level of decomposition;

[0031] Determine the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each of the extracted sub-blocks in the three-dimensional video to be extracted within a set series range;

[0032] A zero watermark of the 3D video to be extracted is generated according to the 3D video to be extracted.

[0033] Optionally, the determining the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each of the extracted sub-blocks in the three-dimensional video to be extracted within a set series range specifically includes:

[0034] For any extracted sub-block, a sub-band amplitude matrix obtained by decomposing the extracted sub-block within a set level range is quantized to obtain a quantized sub-band amplitude matrix corresponding to the extracted sub-block;

[0035] Calculating the watermark embedded in the extracted sub-block according to the quantized sub-band amplitude matrix corresponding to the extracted sub-block;

[0036] The watermark embedded in the 3D video to be extracted is determined according to the watermarks embedded in all the extracted sub-blocks in the 3D video to be extracted.

[0037] A watermark comparison method based on 3D video, comprising:

[0038] According to the above-mentioned watermark embedding method based on 3D video, the three-dimensional video to be protected is processed to obtain an embedded watermark and a zero watermark;

[0039] According to the above-mentioned 3D video-based watermark extraction method, the 3D video to be authenticated is processed to obtain an extracted embedded watermark and an extracted zero watermark;

[0040] It is determined whether the to-be-identified 3D video infringes the copyright of the to-be-embedded 3D video according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark.

[0041] Optionally, the determining whether the to-be-identified 3D video infringes the copyright of the to-be-embedded 3D video according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark specifically includes:

[0042] Performing an XOR operation on the embedded watermark and the extracted embedded watermark to obtain a watermark bit error rate;

[0043] Performing an XOR operation on the zero watermark and the extracted zero watermark to obtain a zero watermark bit error rate;

[0044] Determine whether the watermark bit error rate is less than a first set threshold and whether the zero watermark bit error rate is less than a second set threshold, and obtain a first determination result;

[0045] If the first judgment result is yes, the 3D video to be identified infringes the copyright of the 3D video to be protected;

[0046] If the first judgment result is no, the 3D video to be identified does not infringe the copyright of the 3D video to be protected.

[0047] A watermark embedding system based on 3D video, comprising:

[0048] A first block division module is used to divide each 2D video frame to be embedded in the 3D video into multiple sub-blocks;

[0049] A first decomposition module is used to perform multi-level DT-CWT decomposition on any component in the color space of any sub-block in any two-dimensional video frame to obtain sub-bands of the sub-block at each level of decomposition;

[0050] A subband modification module, used for modifying the amplitude matrix of the subband obtained by decomposing the sub-block within a set series range according to the watermark to be embedded to obtain a modified subband;

[0051] A sub-block watermark embedding module, used for performing an inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block;

[0052] A video frame watermark embedding module, used for splicing all the embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain a watermark corresponding to the two-dimensional video frame and embedding it into a two-dimensional video frame;

[0053] The watermark determination module is used to embed the two-dimensional video frame according to all watermarks corresponding to the three-dimensional video to be embedded to obtain the watermark embedded video and the zero watermark of the three-dimensional video to be embedded.

[0054] A watermark extraction system based on 3D video, comprising:

[0055] The second block division module is used to divide each 2D video frame in the 3D video to be extracted into blocks to obtain a plurality of extraction sub-blocks;

[0056] A second decomposition module is used to perform multi-level DT-CWT decomposition on any component in the color space of any extracted sub-block in any two-dimensional video frame to obtain sub-bands of the extracted sub-block at each level of decomposition;

[0057] An embedded watermark extraction module, used for determining the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each of the extracted sub-blocks in the three-dimensional video to be extracted within a set series range;

[0058] The zero watermark extraction module is used to generate a zero watermark of the three-dimensional video to be extracted according to the three-dimensional video to be extracted.

[0059] A watermark comparison system based on 3D video, comprising:

[0060] A watermark embedding module, used to process the three-dimensional video to be protected according to the watermark embedding method based on 3D video described above to obtain an embedded watermark and a zero watermark;

[0061] A watermark extraction module, used for processing the three-dimensional video to be authenticated according to the watermark extraction method based on 3D video described above to obtain an extracted embedded watermark and an extracted zero watermark;

[0062] The copyright determination module is used to determine whether the to-be-identified 3D video infringes the copyright of the to-be-embedded 3D video according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark.

[0063] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the present invention divides each two-dimensional video frame to be embedded in a three-dimensional video into blocks to obtain a plurality of sub-blocks; performs multi-level DT-CWT decomposition on any component in the sub-block color space to obtain sub-bands of the sub-blocks under each level of decomposition; modifies the amplitude matrix of the sub-band obtained under the decomposition of the sub-block within a set level range according to the watermark to be embedded to obtain a modified amplitude matrix of the sub-band; obtains the modified sub-band according to the modified amplitude matrix of the sub-band; performs an inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block; splices all embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain a watermark-embedded two-dimensional video frame corresponding to the two-dimensional video frame; embeds all watermarks corresponding to the three-dimensional video to be embedded into the two-dimensional video frame to obtain a watermark-embedded video and a zero watermark of the three-dimensional video to be embedded, and combines the embedded watermark with the zero watermark to further improve the accuracy of copyright protection under the premise of ensuring high robustness of the watermark. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0065] Figure 1 This is the result diagram of the watermark embedding method based on discrete cosine transform;

[0066] Figure 2 This is the first example picture;

[0067] Figure 3 This is the second example picture;

[0068] Figure 4 Comparison diagram of video frames at different baseline distances;

[0069] Figure 5 An example of the directions of the DT-CWT decomposition subbands;

[0070] Figure 6 A flow chart of a watermark embedding process provided by an embodiment of the present invention;

[0071] Figure 7 A flow chart of a watermark comparison process provided by an embodiment of the present invention;

[0072] Figure 8 A diagram of the first frame of a video to be protected provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0074] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0075] When embedding watermarks in video frames, there is a trade-off between watermark invisibility and watermark robustness. The two restrict each other and it is impossible to achieve the best at the same time. Because the lower the presence of the watermark, the more vulnerable it is to damage, so the robustness may not meet the requirements. If it is necessary to be robust to various attacks, the watermark embedding strength needs to be increased, and the content of the video frame will be damaged. Therefore, research has always focused on how to meet the requirements of watermark invisibility while improving the robustness of the watermark. If the watermark embedding scheme is not robust enough, even if it achieves good invisibility, the watermark owned by the copyright owner cannot be extracted during copyright identification, and the copyright protection function cannot be achieved; if the video frame is robust to common operations, but the appearance is destroyed due to the embedding of the watermark, or a certain degree of distortion is produced, the scope of its application is greatly reduced.

[0076] Figure 1 The results of the watermark embedding method based on discrete cosine transform are shown: as the watermark embedding strength changes, the watermark robustness and invisibility change. The bit error rate (BER) and peak signal-to-noise ratio (PSNR) are used to measure the robustness and invisibility results. The lower the BER, the better the watermark integrity and the stronger the robustness of the watermark method; the higher the PSNR value, the lower the image distortion and the better the watermark invisibility. Figure 1 It can be seen that as the embedding strength increases, the left image has more obvious block boundaries, that is, the image quality is reduced, which affects the perception. However, although the right image has no obvious distortion, it cannot play a protective role. After a simple signal attack, the complete watermark cannot be extracted.

[0077] Although constructing zero watermark features for video frames will not damage the video content, the recognition is not high enough, which may cause misjudgment and omission problems when identifying copyright. Figure 2 Constructing the zero watermark, we get the feature sequence [0 11 1 0 0 1 1 1 1 0 1 0 0 0 1 1 1 0 1], which is Figure 3Construct a zero watermark and get the feature sequence [0 0 1 1 0 01 1 0 1 0 0 0 0 0 1 1 1 0 1]. This method is effective for Figure 2 and Figure 3 1600 features were generated for each video, and the top 20 features are selected for display. Comparing the two zero-watermark features, the similarity reached 89.8%, but from the human eye, it can be distinguished that these are two different videos. It can be seen that even if a certain zero-watermark method has good robustness against various signal attacks and geometric attacks, once it cannot distinguish whether the video belongs to the copyright owner, it loses a certain practical value. Moreover, robustness and distinguishability are also contradictory properties. Similar to the watermark scheme, a trade-off between the two needs to be made for different usage environments, and it is impossible to achieve the best at the same time.

[0078] In recent years, many studies have noticed the special properties of DIBR technology and proposed corresponding copyright protection schemes based on its characteristics. However, the robustness to higher baseline distances is not enough. The baseline distance is a parameter for synthesizing the left and right views of 2D video frames and depth maps. Figure 4 Shows the stereoscopic degree of video frames under the influence of different baseline distances. Figure 4 (a) is a 2D video frame, Figure 4 (b) is the left view of the synthesized 3D video frame with a baseline distance of 3%. Figure 4 (c) is the left view of the synthesized 3D video frame with a baseline distance of 5%. Figure 4 (d) is the left view of the synthesized 3D video frame with a baseline distance of 7% ( Figure 4 Take the left view as an example), such as Figure 4 As shown in the figure, the longer the baseline, the better the accuracy of the synthesized 3D video within a certain range, and the better the stereoscopic effect. However, the impact of lens distortion will increase. The more distorted the image content seen by the naked eye, the more difficult the binocular matching of features will be. Therefore, it is necessary to improve the robustness of the DIBR method within a reasonable baseline distance, that is, in the 3D video synthesized based on the DIBR method, the extraction of watermarks or the construction of zero watermark features will not be affected by a slightly longer baseline distance.

[0079] Based on this, an embodiment of the present invention provides a watermark embedding method based on 3D video, including:

[0080] Each 2D video frame to be embedded in the 3D video is divided into blocks to obtain a plurality of sub-blocks. In this embodiment, the 3D video to be embedded is based on DIBR 3D video.

[0081] For any sub-block in any two-dimensional video frame, a multi-level DT-CWT decomposition is performed on any component in the color space of the sub-block to obtain the sub-band of the sub-block under each level of decomposition. Within the acceptable content distortion range of 3D video, the dual-tree complex wavelet transform (DT-CWT) method is used to embed a watermark of a certain strength into the video frame. DT-CWT uses an independent dual discrete wavelet tree structure for multi-scale decomposition. In addition to obtaining the horizontal and diagonal directions, it can also obtain the sub-band information in the directions of ±15° and ±75°. Therefore, DT-CWT has directional selectivity, limited redundancy and approximate translation invariance. Compared with the traditional wavelet transform, it has a more detailed and rich description of the details of the image. The obtained description sub-band will not change or distort easily as the image content changes. Therefore, the DT-CWT method is selected as the technical means of the watermark embedding solution. Under the requirements of robustness, invisibility and security required by digital watermarking technology, DT-CWT can better meet its requirements. Each level of decomposition of DT-CWT generates 6 symmetrical subbands H with different directions (±15°, ±45° and ±75°). 3,dr , dr = 1, 2, 3, 4, 5, 6, so when the image pixels move left and right, up and down, or even turn at a certain angle, the DT-CWT processed video frame will not be very sensitive to these changes and has good robustness. Figure 5 As shown, the 6 subbands of DT-CWT can be divided into three pairs according to the symmetry of the direction: (H 3,1 ,H 3,6 ), (H 3,2 ,H 3,5 ) and (H 3,3 ,H 3,4 ). The first pair of coefficients mainly has the contour information of horizontal edges, the second pair mainly has the contour information of diagonal edges, and the third pair mainly has the contour information of vertical edges. Embedding watermarks in the low frequency of DT-CWT can effectively deal with signal attacks, such as smoothing operations such as Gaussian filtering, median filtering and mean filtering, daily JPEG compression, and noise attacks such as salt and pepper noise and Gaussian noise; and the multiple statistical differences in the subsequent watermark embedding quantization method enhance the robustness to geometric attacks such as scaling and edge clipping; the watermark embedding method of DT-CWT ensures robustness to DIBR transformation due to its approximate translation invariance.

[0082] The amplitude matrix of the sub-band obtained by decomposing the sub-block within a set range of levels is modified according to the watermark to be embedded to obtain the modified sub-band.

[0083] An inverse DT-CWT operation is performed on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block.

[0084] All the watermark-embedded sub-blocks corresponding to the two-dimensional video frame are spliced ​​to obtain a watermark-embedded two-dimensional video frame corresponding to the two-dimensional video frame.

[0085] Embedding the two-dimensional video frame according to all watermarks corresponding to the three-dimensional video to be embedded obtains the watermark-embedded video and the zero watermark of the three-dimensional video to be embedded.

[0086] In practical applications, performing multi-level DT-CWT decomposition on any component in the sub-block color space to obtain sub-bands of the sub-block at each level of decomposition specifically includes:

[0087] Convert the sub-block from the RGB color space to the YCbCr color space to obtain a converted sub-block.

[0088] The Y component of the conversion sub-block is subjected to multi-level DT-CWT decomposition to obtain sub-bands of the conversion sub-block at each level of decomposition. In practical applications, three-level decomposition can be performed.

[0089] In practical applications, the step of modifying the amplitude matrix of the sub-band obtained by decomposing the sub-block within a set range of levels according to the watermark to be embedded to obtain the modified sub-band specifically includes:

[0090] The amplitude matrix and phase matrix of the sub-band obtained by decomposing the sub-block within a set range of levels are calculated.

[0091] According to the watermark to be embedded, the amplitude matrix of the sub-band obtained by decomposing the sub-block within a set level range is quantized to obtain a quantized sub-band amplitude matrix corresponding to the sub-block.

[0092] A modified subband is obtained according to the phase matrix and the quantized subband amplitude matrix corresponding to the subblock.

[0093] In practical applications, the amplitude matrix and phase matrix of the sub-band obtained by calculating the sub-block under the set series range decomposition are specifically: according to the formula

[0094] Calculate the magnitude matrix,

[0095] According to the formula Calculate the phase matrix. The amplitude matrix is ​​obtained by finding the absolute value of the subband coefficients, and the phase matrix is ​​obtained by finding the angle of the subband coefficients. represents the amplitude matrix of the drth subband under the lvth level decomposition, represents the phase matrix of the drth subband under the lvth level decomposition, W and H represent the width and height of the video frame; M and N are the width and height of the subblock, represents the value of the last row and last column of the subband coefficient matrix, minus one because the length and width of this matrix start counting from 0. Then the length of the coefficient matrix is Width The absolute value symbol outside the brackets is used to obtain the absolute value and thus the amplitude. The letters in brackets have the same meaning as H lv,dr The same, ∠ symbol means the phase value. Specifically, if three DT-CWT decompositions are performed, lv = 2, 3; dr = 1, 2, 5, 6 can be set. The principle of the DIBR system is to move each pixel horizontally, so the vertical edge loses more information than the horizontal edge. Therefore, the operation is not performed on all sub-bands, but carefully selected according to the direction, and the first and second pairs of coefficients are selected for quantization of watermark embedding, that is, (H 3,1 ,H 3,6 ) and (H 3,2 ,H 3,5 ), avoid including too much vertical edge information to ensure robustness to DIBR transformation.

[0096] In practical applications, the sub-band amplitude matrix obtained by decomposing the sub-block in a set range of levels according to the watermark to be embedded is quantized to obtain the quantized sub-band amplitude matrix corresponding to the sub-block, which specifically includes: according to the formula

[0097] Quantify, where , Represents the i-th row in the quantized subband amplitude matrix of the first subband under the lv-th level decomposition of the sub-block, The i-th row in the modified amplitude matrix of the distance between the symmetrical sub-bands of the first sub-band under the lv-th level decomposition of the sub-block, Represents the i-th row in the quantized subband amplitude matrix of the 6th subband under the lv-th level decomposition of the sub-block, represents the i-th row in the amplitude matrix of the first subband of the sub-block at the lv-th level decomposition, represents the i-th row in the amplitude matrix of the 6th subband of the sub-block at the lv-th level decomposition, represents the i-th row in the modified amplitude matrix of the distance between the symmetrical sub-bands of the 6th sub-band under the lv-level decomposition of the sub-block, w represents the watermark value, The modified amplitude matrix of the distance between the symmetrical sub-bands of the drth sub-band under the lv-th level decomposition of the sub-block is represented by the i-th row and the k-th column, represents the i-th row and k-th column of the amplitude matrix of the dr-th subband under the lv-th level decomposition of the sub-block, Represents the i-th row in the quantized subband amplitude matrix of the second subband under the lv-th level decomposition of the sub-block, The i-th row in the modified amplitude matrix of the distance between the symmetrical sub-bands of the second sub-band under the lv-th level decomposition of the sub-block, Represents the i-th row in the quantized subband amplitude matrix of the 5th subband under the lv-th level decomposition of the sub-block, Represents the i-th row in the amplitude matrix of the second subband of the sub-block at the lv-th level decomposition, represents the i-th row in the amplitude matrix of the 5th subband of the sub-block at the lv-th level decomposition, The i-th row in the modified amplitude matrix of the distance between symmetric subbands of the 5th subband under the lv-th level decomposition of the sub-block. Specifically, if three DT-CWT decompositions are performed in total, lv=2,3; dr=1,2,5,6 can be set.

[0098] In practical applications, splicing all the embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain the watermark embedded two-dimensional video frame corresponding to the two-dimensional video frame specifically includes:

[0099] All the embedded watermark sub-blocks corresponding to the two-dimensional video frame are spliced ​​to obtain a complete two-dimensional video frame.

[0100] The complete two-dimensional video frame is converted from the YCbCr color space to the RGB color space to obtain a watermark corresponding to the two-dimensional video frame and embed it into the two-dimensional video frame.

[0101] In practical applications, embedding a two-dimensional video frame according to all watermarks corresponding to the three-dimensional video to be embedded to obtain a zero watermark of the three-dimensional video to be embedded specifically includes:

[0102] The two-dimensional video frames into which each watermark corresponding to the three-dimensional video to be embedded are respectively divided into blocks to obtain a plurality of watermark sub-blocks.

[0103] For any watermark sub-block embedded in any two-dimensional video frame, all components in the color space of the watermark sub-block are decomposed by multi-level DT-CWT to obtain sub-bands of the watermark sub-block at each level of decomposition.

[0104] An SVD operation is performed on the amplitude matrix of the sub-band obtained under the set level decomposition of the watermark sub-block to obtain the singular value corresponding to the watermark sub-block.

[0105] A zero watermark of the watermark-embedded two-dimensional video frame is generated according to singular values ​​corresponding to all the watermark sub-blocks in the watermark-embedded two-dimensional video frame.

[0106] The zero watermark of the three-dimensional video to be embedded is generated according to the zero watermark of the two-dimensional video frame embedded with all the watermarks corresponding to the three-dimensional video to be embedded. The DT-CWT method is selected to process the video frame after the watermark is embedded, and the singular value decomposition (SVD) is added to jointly construct the feature map of the zero watermark, which can further improve the robustness to DIBR transformation, signal attack and geometric attack. The DT-CWT method has been introduced before and will not be repeated here. The decomposition formula of SVD is as follows:

[0107] X=USV T

[0108] For a matrix X∈R n×n , the decomposed U and V are both unitary matrices, that is, UU T =E,VV T =E,U T and V T are the transposed matrices of U and V respectively, and E is the unit matrix. Among them, S is a matrix with all values ​​​​of 0 except the diagonal. The values ​​on the diagonal of the S matrix are singular values, which is equivalent to extracting the features of a matrix and representing the complete matrix with a small amount of data. The singular values ​​are arranged in order from large to small, and the speed of decrease is very fast. The latter is a singular value far less than 1, so often only the first few or a dozen singular values ​​are used to represent the information of the entire matrix. Therefore, SVD can simplify data, extract the most important information in the data, and achieve the effect of compression. And it has good stability, displacement invariance and certain rotation invariance, and is widely used in image processing. Later, the size relationship features of the singular values ​​will be combined with the features extracted by DT-CWT to construct robust and distinguishable zero watermark features.

[0109] The present invention is lossy in the watermark embedding step, but due to the dual verification and complementary characteristics of the watermark and zero watermark features, the watermark scheme does not need to sacrifice the accuracy of the image content for high robustness, because the zero watermark scheme will perform copyright retrieval; the zero watermark scheme does not need to sacrifice the robustness to signal attacks, geometric attacks and DIBR operations for high distinguishability, because the watermark scheme will perform copyright authentication. The detection accuracy of the present invention has been improved to a certain extent compared with the traditional single watermark embedding or zero watermark construction method, and the effectiveness of copyright protection is further improved under the premise of ensuring the accuracy of video content.

[0110] In practical applications, all components in the color space of the watermark sub-block are subjected to multi-level DT-CWT decomposition to obtain sub-bands of the watermark sub-block at each level of decomposition, specifically including:

[0111] Multi-level DT-CWT decomposition is performed on the R, G and B channels of the watermark sub-block respectively to obtain sub-bands of the watermark sub-block at each level of decomposition. Specifically, three-level DT-CWT decomposition can be performed.

[0112] In practical applications, the SVD operation is performed on the amplitude matrix of the sub-band obtained under the set level decomposition of the watermark sub-block to obtain the singular value corresponding to the watermark sub-block, specifically including:

[0113] When performing three-level DT-CWT decomposition, the amplitude matrices of the 1st, 2nd, 5th and 6th sub-bands obtained by the watermark sub-block under the third level decomposition are calculated.

[0114] The amplitude matrices of the 1st, 2nd, 5th and 6th sub-bands obtained in the third level decomposition of the watermark sub-block are determined as the matrix P.

[0115] Perform an SVD operation on P to obtain the singular value corresponding to the watermark sub-block.

[0116] In practical applications, generating a zero watermark of the watermark-embedded two-dimensional video frame according to singular values ​​corresponding to all the watermark sub-blocks in the watermark-embedded two-dimensional video frame specifically includes:

[0117] The median of the singular values ​​corresponding to all watermark sub-blocks embedded in the two-dimensional video frame is taken as the threshold.

[0118] The singular values ​​corresponding to all watermark sub-blocks embedded in the two-dimensional video frame are changed to 1 if they are greater than the threshold, and changed to 0 if they are less than or equal to the threshold, and finally a zero watermark of the two-dimensional video frame is obtained.

[0119] The embodiment of the present invention combines the embedded watermark with the zero watermark, which can further improve the accuracy of copyright protection while ensuring the high robustness of the watermark.

[0120] The embodiment of the present invention further provides a 3D video-based watermark extraction method for the above-mentioned watermark embedding method, including:

[0121] Each 2D video frame in the 3D video to be extracted is divided into blocks to obtain a plurality of extraction sub-blocks. In this embodiment, the 3D video to be extracted is based on DIBR 3D video.

[0122] For any extracted sub-block in any two-dimensional video frame, multi-level DT-CWT decomposition is performed on any component in the color space of the extracted sub-block to obtain sub-bands of the extracted sub-block at each level of decomposition.

[0123] The watermark embedded in the three-dimensional video to be extracted is determined according to the amplitude matrix of the sub-band obtained by decomposing each of the extracted sub-blocks in the three-dimensional video to be extracted within a set level range.

[0124] A zero watermark of the 3D video to be extracted is generated according to the 3D video to be extracted.

[0125] In practical applications, performing multi-level DT-CWT decomposition on any component in the color space of the extracted sub-block to obtain the sub-band of the extracted sub-block at each level of decomposition is the same as the step of "performing multi-level DT-CWT decomposition on any component in the color space of the sub-block to obtain the sub-band of the sub-block at each level of decomposition".

[0126] In practical applications, determining the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each of the extracted sub-blocks in the three-dimensional video to be extracted within a set series range specifically includes:

[0127] For any extracted sub-block, the amplitude matrix of the sub-band obtained by decomposing the extracted sub-block within a set level range is quantized to obtain the quantized sub-band amplitude matrix corresponding to the extracted sub-block. Specifically, if three DT-CWT decompositions are performed in total, lv=2, 3; dr=1, 2, 5, 6 can be set to calculate the quantized sub-band amplitude matrix.

[0128] The watermark embedded in the extracted sub-block is calculated according to the quantized sub-band amplitude matrix corresponding to the extracted sub-block. Specifically, when the value of the i-th row of the 1st sub-band amplitude matrix is ​​less than the value of the i-th row of the 6th sub-band amplitude matrix, the number of 0 watermarks embedded in the extracted sub-block is increased by 1, otherwise the number of 1 watermarks is increased by 1. When the value of the i-th row of the 2nd sub-band amplitude matrix is ​​less than the value of the i-th row of the 5th sub-band amplitude matrix, the number of 0 watermarks embedded in the extracted sub-block is increased by 1, otherwise the number of 1 watermarks is increased by 1. When all the values ​​in the amplitude matrix are judged, the watermark embedded in the extracted sub-block is obtained.

[0129] The watermark embedded in the 3D video to be extracted is determined according to the watermarks embedded in all the extracted sub-blocks in the 3D video to be extracted.

[0130] The embodiment of the present invention further provides a watermark comparison method based on 3D video, comprising:

[0131] According to the above-mentioned watermark embedding method based on 3D video, the three-dimensional video to be protected is processed to obtain an embedded watermark and a zero watermark.

[0132] According to the above-mentioned watermark extraction method based on 3D video, the three-dimensional video to be authenticated is processed to obtain the extracted embedded watermark and the extracted zero watermark.

[0133] It is determined whether the to-be-identified 3D video infringes the copyright of the to-be-embedded 3D video according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark.

[0134] In practical applications, determining whether the to-be-identified 3D video infringes the copyright of the to-be-embedded 3D video according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark specifically includes:

[0135] An XOR operation is performed on the embedded watermark and the extracted embedded watermark to obtain a watermark bit error rate B1.

[0136] An XOR operation is performed on the zero watermark and the extracted zero watermark to obtain a zero watermark bit error rate B2.

[0137] It is determined whether the watermark bit error rate is less than a first set threshold value T1 and whether the zero watermark bit error rate is less than a second set threshold value T2 to obtain a first determination result.

[0138] If the first judgment result is yes, the 3D video to be identified infringes the copyright of the 3D video to be protected.

[0139] If the first judgment result is no, the 3D video to be identified does not infringe the copyright of the 3D video to be protected.

[0140] In view of the above embedding method, an embodiment of the present invention further provides a watermark embedding system based on 3D video, comprising:

[0141] The first block division module is used to divide each 2D video frame to be embedded in the 3D video into multiple sub-blocks.

[0142] The first decomposition module is used to perform multi-level DT-CWT decomposition on any component in the color space of any sub-block in any two-dimensional video frame to obtain sub-bands of the sub-block under each level of decomposition.

[0143] The subband modification module is used to modify the amplitude matrix of the subband obtained by decomposing the sub-block within a set series range according to the watermark to be embedded to obtain a modified subband.

[0144] The sub-block watermark embedding module is used to perform an inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block.

[0145] The video frame watermark embedding module is used to splice all the corresponding embedded watermark sub-blocks in the two-dimensional video frame to obtain the watermark corresponding to the two-dimensional video frame and embed it into the two-dimensional video frame.

[0146] The watermark determination module is used to embed the two-dimensional video frame according to all watermarks corresponding to the three-dimensional video to be embedded to obtain the watermark embedded video and the zero watermark of the three-dimensional video to be embedded.

[0147] In view of the above extraction method, an embodiment of the present invention further provides a watermark extraction system based on 3D video, comprising:

[0148] The second block division module is used to divide each 2D video frame in the 3D video to be extracted into blocks to obtain a plurality of extraction sub-blocks.

[0149] The second decomposition module is used to perform multi-level DT-CWT decomposition on any component in the color space of any extracted sub-block in any two-dimensional video frame to obtain sub-bands of the extracted sub-block at each level of decomposition.

[0150] The embedded watermark extraction module is used to determine the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each extracted sub-block in the three-dimensional video to be extracted within a set series range.

[0151] The zero watermark extraction module is used to generate a zero watermark of the three-dimensional video to be extracted according to the three-dimensional video to be extracted.

[0152] In view of the above comparison method, an embodiment of the present invention further provides a watermark comparison system based on 3D video, including:

[0153] The watermark embedding module is used to process the three-dimensional video to be protected according to the watermark embedding method based on 3D video described above to obtain an embedded watermark and a zero watermark.

[0154] The watermark extraction module is used to process the three-dimensional video to be authenticated according to the watermark extraction method based on 3D video described above to obtain the extracted embedded watermark and the extracted zero watermark.

[0155] The copyright determination module is used to determine whether the to-be-identified 3D video infringes the copyright of the to-be-embedded 3D video according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark.

[0156] The present invention also provides a more specific implementation method of the entire process of watermark embedding, extraction and comparison:

[0157] For the watermark embedding process, all 2D video frames in the DIBR 3D video are first embedded with watermarks for subsequent authentication. Since the entire algorithm process uses zero watermark features for retrieval, the video frames are protected by two methods, so the difficulty of balancing the robustness and invisibility of the watermark embedding algorithm characteristics is reduced. It is not necessary to destroy the perception of the human eye in order to achieve high-intensity robustness. Therefore, the number of watermark bits embedded in each frame of the image does not need to be too long, and it can be extracted later to achieve the purpose of authentication. The specific process is as follows: Figure 6 shown.

[0158] Step 1: Input the 2D video frame to be protected, divide the 2D video frame into blocks, and obtain (I W / M)×(I H / N) sub-blocks B s (s=1,2,…(I W / M)×(I H / N)), M and N are sub-blocks B s One sub-block is embedded with one bit of watermark.

[0159] Step 2: DT-CWT coefficients are vectorized. s Convert from RGB color format to YCbCr color space, select the Y component for three-level DT-CWT decomposition, select the second and third sub-bands for subsequent quantization operations, and pair the 1, 6 directions with the 2, 5 directions: (H lv，1 , H lv，6 ) and (H lv，2 , H lv，5 ), where lv=2,3.

[0160] Step 3: Embed the watermarked video frame. Calculate the amplitude matrix and phase matrix of the subband, modify the amplitude matrix row by row, and modify (H lv，1 , H lv，6 ) lv，1 , (H lv，2 , H lv，5 ) lv，2 , the other two remain unchanged; when embedding 1, modify H lv，6 and H lv，5 . The amplitude matrix and phase matrix are calculated as follows:

[0161]

[0162]

[0163] Where lv indicates the decomposition level. Since three levels of decomposition are performed but only the coefficients of the last two levels are selected, lv = 2, 3; dr represents each subband: 1, 2, 3, 4, 5, 6; W and H represent the width and height of the video frame; M and N are the width and height of the subblock.

[0164] The amplitude matrix is ​​obtained by finding the absolute value of the subband coefficients, and the phase matrix is ​​obtained by finding the angle of the subband coefficients. The watermark is embedded by modifying the amplitude matrix, and the phase matrix is ​​used to restore the subband coefficients after the watermark is embedded.

[0165] For the modification of the amplitude matrix, each row of the amplitude matrix is ​​first regarded as a whole, and quantization operation is performed in units of rows, that is, the following operations are performed on all columns of a row of the amplitude matrix. The quantization calculation formula is as follows:

[0166]

[0167] in, The calculation steps are to widen the gap between the amplitude matrices of each row of the 1st and 6th subbands and the 2nd and 5th subbands. Since the amplitude matrix is ​​generally not too large, the division in the formula is rounding, so 256 is used as the divisor and rounded down twice. At this time After calculation, it is usually a very small value or even 0.

[0168]

[0169] The calculation of the 1st and 6th sub-bands is taken as an example. i represents the operation on all columns of the i-th row. When the embedded watermark is 0, that is, w is equal to 0, the quantization amplitude matrix of the 1st sub-band under the lv-level decomposition of the sub-block is assigned the previously obtained (at this time Generally, it is a very small number or even 0), while the 6th subband amplitude matrix remains unchanged, which further expands the difference between 1 and 6, that is, the amplitude matrix of the ith row of the 1st subband amplitude matrix is ​​0 or a very small value, while the amplitude matrix of the ith row of the 6th subband amplitude matrix remains unchanged. When the embedded watermark is 1, that is, w is equal to 1, the ith row of the amplitude matrix of the 1st subband is not modified, but the ith row of the amplitude matrix of the 6th subband is modified, and its amplitude matrix is ​​calculated by Similarly, the 2nd and 5th subbands are also modified according to the 0 and 1 bits of the watermark. Therefore, for the same block, each row of the amplitude matrix embeds the same watermark bit sequence.

[0170] Step 4: After quantizing each row of the amplitude matrix of each subband, restore it to the original subband format according to the modified amplitude matrix and the previously calculated phase matrix to obtain the modified subband. Then perform the inverse DT-CWT operation to obtain the watermarked blocks. Reassemble all the blocks into a complete 2D video frame and convert them from the YCbCr domain to the RGB domain to complete the watermark embedding operation for the entire video frame.

[0171] Next, we introduce the zero watermark construction method. The specific steps are as follows:

[0172] Step 1, DT-CWT+SVD. The video sequence embedded with the watermark is pre-processed by scaling, that is, the video frame is converted to a fixed length and width, and the size is set according to the actual situation. Generally, it is set to a square with equal length and width to be more robust to rotation and scaling attacks. Then the frame sequence in the 2D video is normalized and downsampled, and every n frames are merged into a group and the average is taken, which can effectively reduce the redundancy of time information and make full use of the time domain information.

[0173] Step 2: Divide each frame of the merged video sequence into blocks (I W / M)×(I H / N) sub-blocks B s (s=1, 2, ...M×N), M and N are B s The side length of .

[0174] Step 3: Block B s The R, G, and B channels of the image are decomposed by three levels of DT-CWT, and the sub-band coefficients in the third level horizontal direction are selected to pair the 1, 6 directions with the 2, 5 directions: P1 = (H 3，1 , H 3，6 ) and P2=(H 3，2 , H 3，5 ), and calculate the amplitude according to the following formula:

[0175]

[0176] is the amplitude matrix, where dr = 1, 2, 5, 6, represents the first sub-band matrix of the drth sub-band matrix at the third level of decomposition. Line The coefficient values ​​for the column.

[0177] Step 4: After obtaining the amplitude matrices of P1 and P2, put them in the same matrix P. The detailed structure of P is shown below. Perform SVD operation on it and save the first singular value to generate the intermediate eigenvector f m .

[0178]

[0179] Step 5, zero watermark feature. Take the first singular value of all sub-blocks, that is, the median of the intermediate feature vector, as the threshold t. The value greater than this threshold is set to 1, and the value less than or equal to this threshold is set to 0. Binarization is performed and finally concatenated into a zero watermark bit sequence f. A series of 0 or 1 bit sequences are obtained and registered in the database for retrieval in the future for copyright identification. f can represent a certain 2D video. The binarization process is as follows:

[0180] f m(c) represents the cth value of the intermediate feature vector generated previously, and f(c) represents the cth value of the zero watermark sequence finally constructed.

[0181] Step 6, third-party digital watermark information library. Finally, the video with embedded watermark, watermark and zero watermark features are saved in the database. When a video V that may infringe copyright is found on the Internet, a zero watermark feature is constructed for it, and then the zero watermark in the database is searched based on the zero watermark feature. If a zero watermark of a video with similar content is matched in the database, further authentication can be performed: extract the watermark of V. If the extracted watermark is very similar to the watermark stored in the database, the calculated ber is less than the set threshold, and infringement can be judged.

[0182] For watermark extraction and comparison, the early steps of video frame segmentation, color space conversion, and amplitude matrix calculation are basically the same as those in the watermark embedding stage, so only the watermark extraction is explained in detail:

[0183] Step 1: Divide the 2D video frame into blocks to obtain extracted sub-blocks.

[0184] Step 2: Convert the extracted sub-block to the YCbCr color space, select the Y component for three-level DT-CWT decomposition, and also select the second and third level sub-band coefficients of the extracted sub-block in the horizontal direction. Divide into three pairs: (H′ lv，1 , H lv，6 ), (H′ lv，2 , H lv，5 ) and (H′ lv，3 , H lv，4 ), the first two pairs are selected for watermark embedding operation, and the last pair is discarded because it represents the information of the vertical edge of the video frame content, which is easily distorted and twisted under the DIBR operation.

[0185] Step 3: (H′ lv，1 , H′ lv，6 ) and (H′ lv，2 , H′ lv，5 ) represents the coefficients and calculates the amplitude matrix and quantizes it. The calculation is performed row by row. The calculation process is the same as the watermark embedding process, so the quantized subband amplitude matrix of the extracted sub-block is obtained: the i-th row in the quantized subband amplitude matrix of the first sub-band of the extracted sub-block under the lv-th level decomposition Extract the i-th row in the quantized subband amplitude matrix of the 6th subband under the lv-level decomposition of the sub-block Extract the i-th row in the quantized subband amplitude matrix of the second subband under the lv-th level decomposition of the sub-block The i-th row in the quantized subband amplitude matrix of the 5th subband under the lv-level decomposition of the extracted sub-block Determine whether to extract 0 or 1 by statistical difference. Taking the amplitude matrices of the 1st and 6th subbands as an example:

[0186]

[0187] If is less than it means that the count of the 0 watermark increases by 1. If is greater than it means that the count of the 1 watermark increases by 1. After all rows are executed, determine whether to extract 0 or 1 according to the statistical size comparison result, and obtain the final extracted watermark sequence W'.

[0188] Step 4: Perform an exclusive OR operation on W' and the original watermark sequence W to obtain the final bit error rate:

[0189]

[0190] If the ber is less than the set empirical threshold, the copyright authentication is completed. Since this extraction scheme determines the watermark based on the final total count, the watermark extraction error is reduced. Instead of using a single watermark embedding or zero-watermark construction method, they are combined, making full use of the advantages of these two methods and complementarily reducing their disadvantages, expanding the application range and improving the copyright discrimination accuracy. Step 4 is specifically as follows: First, extract the zero-watermark features of the 3D video to be authenticated, then perform feature retrieval in the registered copyright information database. If the bit error rate is less than the T2 threshold, that is, the similarity is very high, then further extract the embedded watermark, and combine the results of the two to make an infringement judgment. The copyright authentication process is as Figure 7 shown. First, input the 3D video frames to be detected, then perform preprocessing such as scaling and downsampling, then perform DT-CWT transformation processing on the preprocessed video frames respectively to extract the watermark, calculate the watermark bit error rate B1 and construct zero-watermark features by DT-CWT+SVD, calculate the zero-watermark feature bit error rate B2, and determine whether B1<T1 and B2>T2. If so, it proves infringement, otherwise it proves non-infringement. Among them, T1 and T2 are thresholds set according to experience.

[0191] This embodiment has the following technical effects:

[0192] 1. The present invention relates to a copyright protection method based on watermark embedding and zero watermark hybrid of DIBR synthesized 3D video. Existing watermark schemes and zero watermark schemes do not adequately consider the characteristics of DIBR synthesized 3D video. After the DIBR operation, the synthesized frame has horizontal pixel shifts relative to the original two-dimensional frame, so the DIBR operation can be regarded as an attack that changes the content of the video frame, which means that the protection scheme for 3D video should be DIBR invariant. The present invention combines watermark embedding and zero watermark construction methods, and proposes a scheme that takes into account the advantages of both, and tries to avoid the shortcomings of both, thereby improving the accuracy of copyright protection.

[0193] 2. The present invention utilizes a watermark scheme for authentication and a zero-watermark scheme for retrieval, taking into account the advantages of both methods, eliminating the contradictions between the two methods to a certain extent, and improving the accuracy of copyright identification.

[0194] The embodiment of the present invention proposes a copyright protection method based on DIBR 3D video watermark embedding and zero watermark construction hybrid, firstly embed watermarks into 2D video frames, then extract zero watermark features from the watermarked video frames, and register the obtained features in a database, wherein both the watermark embedding method and the zero watermark construction method are robust to DIBR operations, that is, the synthesized 3D video can still extract watermarks and construct similar zero watermark features, and the entire process is invariant to the 3D video synthesized based on DIBR, that is, after the 2D video frames and depth maps embedded with watermarks are synthesized into left and right views by DIBR technology, watermarks can still be extracted from the virtual left and right views, and the constructed zero watermark features are not affected by the DIBR operation, thereby achieving the purpose of protecting the copyright of DIBR 3D videos. When a video that may infringe copyright is subsequently found on the network, firstly construct zero watermark features for it, and search the extracted features and feature sequences stored in the database, if similar zero watermark features can be matched, then further perform watermark extraction operations on the video, and when the extracted watermark is similar to the original watermark, the video can be finally judged as infringing copyright.

[0195] The present invention provides a specific embodiment of applying the above watermark comparison method:

[0196] The difference between video and image is that a video has hundreds of frames. If each frame is embedded with a watermark, it will take a long time and occupy too much memory. However, if the number of embedded frames is too small, it cannot cope with attacks such as frame loss and frame insertion. Therefore, in order to balance the processing efficiency and the need for copyright protection, a watermark embedding operation is performed every 10 frames, and the operation of each frame is the same. Take a video to be protected as an example. Figure 8 As shown, the size is 320*240. Since the subsequent 100 frames are similar to the first frame, only the first frame is shown.

[0197] The specific protection steps are as follows:

[0198] 1. If the watermarking operation is performed on every 10 video frames and the block size is 40*40, 48 watermark bits can be embedded: (320 / 40)×(240 / 40)=8×6. The watermark to be embedded is [0 1 0 1 0 1 0 1 1 1 0 0 0 1 0 1 1 1 0 0 0 1 0 1 0 0 0 0 1 1 1 0 1 1 0 1 0 0 0 0 0].

[0199] 2. Downsample the watermarked video and normalize it to 320*320 size. Average every 4 frames to get 25 video frames. Each video frame is also divided into 40 blocks. Construct zero watermark features according to the method described above. Connect them in series to get 25×8×8=1600-dimensional features.

[0200] 3. Register the watermarked 2D video frames, watermarks, and zero-watermark features of each video in the database.

[0201] 4. When a video that may infringe copyright is found on the Internet, a zero watermark feature is first constructed for it, and then matched and retrieved with the feature sequence stored in the database. If there is matching content in the database, further operation is performed to extract the watermark and calculate the ber with the original watermark. When it is less than the set threshold, infringement can be determined and copyright protection can be achieved.

[0202] The above example describes the application of this method in a specific example, and the experiment also proves the feasibility of this method. This method combines the watermark embedding method with the zero watermark construction method to achieve a trade-off between robustness, watermark invisibility and distinguishability, meet the requirements of copyright protection, and improve the accuracy of copyright protection.

[0203] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0204] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A watermark embedding method based on 3D video, characterized in that: include: Divide each two-dimensional video frame to be embedded in the three-dimensional video into blocks to obtain a plurality of sub-blocks; For any sub-block in any two-dimensional video frame, perform multi-level DT-CWT decomposition on any component in the color space of the sub-block to obtain sub-bands of the sub-block at each level of decomposition; Modifying the amplitude matrix of the sub-band obtained by decomposing the sub-block within a set range of levels according to the watermark to be embedded to obtain a modified sub-band; Performing an inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block; Splicing all the watermark-embedded sub-blocks corresponding to the two-dimensional video frame to obtain a watermark-embedded two-dimensional video frame corresponding to the two-dimensional video frame; Embedding a two-dimensional video frame according to all watermarks corresponding to the three-dimensional video to be embedded to obtain a watermark-embedded video and a zero watermark of the three-dimensional video to be embedded; The step of modifying the amplitude matrix of the sub-band obtained by decomposing the sub-block within a set range of levels according to the watermark to be embedded to obtain the modified sub-band specifically includes: Calculating the amplitude matrix and phase matrix of the sub-band obtained by decomposing the sub-block within a set series range; According to the watermark to be embedded, the amplitude matrix of the sub-band obtained by decomposing the sub-block within a set range of levels is quantized to obtain a quantized sub-band amplitude matrix corresponding to the sub-block; Obtaining a modified subband according to the phase matrix and the quantized subband amplitude matrix corresponding to the subblock; The calculation of the amplitude matrix and phase matrix of the sub-band obtained by decomposing the sub-block within the set series range is specifically: According to the formula Compute the magnitude matrix, where represents the amplitude matrix of the drth subband under the lvth level decomposition, W and H represent the width and height of the two-dimensional video frame, M and N are the width and height of the sub-block, represents the value of the last row and last column of the subband coefficient matrix, and || represents the absolute value; According to the formula Compute the phase matrix, where represents the phase matrix of the drth subband under the lvth level decomposition, Representation calculation The phase value of According to the watermark to be embedded, the sub-band amplitude matrix obtained by decomposing the sub-block within a set level range is quantized to obtain the quantized sub-band amplitude matrix corresponding to the sub-block, specifically: According to the formula and Quantify, where Represents the i-th row in the quantized subband amplitude matrix of the first subband under the lv-th level decomposition of the sub-block, The i-th row in the modified amplitude matrix of the distance between the symmetrical sub-bands of the first sub-band under the lv-th level decomposition of the sub-block, Represents the i-th row in the quantized subband amplitude matrix of the 6th subband under the lv-th level decomposition of the sub-block, represents the i-th row in the amplitude matrix of the 6th subband at the lv-th level decomposition of the sub-block, w represents the watermark value, represents the i-th row in the amplitude matrix of the first subband of the sub-block at the lv-th level decomposition, The i-th row in the modified amplitude matrix representing the distance between the symmetrical sub-bands of the 6th sub-band under the lv-th level decomposition of the sub-block; Represents the i-th row in the quantized subband amplitude matrix of the second subband under the lv-th level decomposition of the sub-block, The i-th row in the modified amplitude matrix of the distance between the symmetrical sub-bands of the second sub-band under the lv-th level decomposition of the sub-block, Represents the i-th row in the quantized subband amplitude matrix of the 5th subband under the lv-th level decomposition of the sub-block, represents the i-th row in the amplitude matrix of the 5th subband of the sub-block at the lv-th level decomposition, Represents the i-th row in the amplitude matrix of the second subband of the sub-block at the lv-th level decomposition, The i-th row in the modified amplitude matrix representing the distance between the symmetrical sub-bands of the 5th sub-band under the lv-th level decomposition of the sub-block; The modified amplitude matrix of the distance between the symmetrical sub-bands of the drth sub-band under the lv-th level decomposition of the sub-block is represented by the i-th row and the k-th column, represents the i-th row and k-th column of the amplitude matrix of the dr-th subband under the lv-th level decomposition of the sub-block, Indicates rounding down.

2. A watermark embedding method based on 3D video according to claim 1, characterized in that: Embedding a two-dimensional video frame according to all watermarks corresponding to the three-dimensional video to be embedded to obtain a zero watermark of the three-dimensional video to be embedded specifically includes: The two-dimensional video frames corresponding to the watermarks to be embedded in the three-dimensional video are divided into blocks to obtain a plurality of watermark sub-blocks; For any watermark sub-block embedded in any two-dimensional video frame, all components in the color space of the watermark sub-block are decomposed by multi-level DT-CWT to obtain sub-bands of the watermark sub-block at each level of decomposition; Performing an SVD operation on the amplitude matrix of the sub-band obtained under the set level decomposition of the watermark sub-block to obtain the singular value corresponding to the watermark sub-block; Generate a zero watermark of the watermark-embedded two-dimensional video frame according to the singular values ​​corresponding to all the watermark sub-blocks in the watermark-embedded two-dimensional video frame; The zero watermark of the to-be-embedded three-dimensional video is generated by embedding the zero watermark of the two-dimensional video frame according to all watermarks corresponding to the to-be-embedded three-dimensional video.

3. A watermark extraction method based on 3D video, characterized in that: include: Divide each 2D video frame in the 3D video to be extracted into blocks to obtain a plurality of extraction sub-blocks; The three-dimensional video to be extracted is obtained by the watermark embedding method based on 3D video according to any one of claims 1-2; For any extracted sub-block in any two-dimensional video frame, perform multi-level DT-CWT decomposition on any component in the color space of the extracted sub-block to obtain sub-bands of the extracted sub-block at each level of decomposition; Determine the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each of the extracted sub-blocks in the three-dimensional video to be extracted within a set series range; A zero watermark of the 3D video to be extracted is generated according to the 3D video to be extracted.

4. A watermark extraction method based on 3D video according to claim 3, characterized in that: The step of determining the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each of the extracted sub-blocks in the three-dimensional video to be extracted within a set range of levels specifically includes: For any extracted sub-block, a sub-band amplitude matrix obtained by decomposing the extracted sub-block within a set level range is quantized to obtain a quantized sub-band amplitude matrix corresponding to the extracted sub-block; Calculating the watermark embedded in the extracted sub-block according to the quantized sub-band amplitude matrix corresponding to the extracted sub-block; The watermark embedded in the 3D video to be extracted is determined according to the watermarks embedded in all the extracted sub-blocks in the 3D video to be extracted.

5. A watermark comparison method based on 3D video, characterized in that: include: According to the 3D video-based watermark embedding method according to any one of claims 1-2, the 3D video to be protected is processed to obtain an embedded watermark and a zero watermark; According to the 3D video-based watermark extraction method according to any one of claims 3-4, the 3D video to be authenticated is processed to obtain an extracted embedded watermark and an extracted zero watermark; It is determined whether the to-be-identified 3D video infringes the copyright of the to-be-embedded 3D video according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark.

6. A watermark comparison method based on 3D video according to claim 5, characterized in that: The determining, according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark, whether the to-be-identified 3D video infringes the copyright of the to-be-embedded 3D video specifically includes: Performing an XOR operation on the embedded watermark and the extracted embedded watermark to obtain a watermark bit error rate; Performing an XOR operation on the zero watermark and the extracted zero watermark to obtain a zero watermark bit error rate; Determine whether the watermark bit error rate is less than a first set threshold and whether the zero watermark bit error rate is less than a second set threshold, and obtain a first determination result; If the first judgment result is yes, the 3D video to be identified infringes the copyright of the 3D video to be protected; If the first judgment result is no, the 3D video to be identified does not infringe the copyright of the 3D video to be protected.

7. A watermark embedding system based on 3D video, characterized in that: The 3D video-based watermark embedding method according to claim 1, wherein the 3D video-based watermark embedding system comprises: A first block division module is used to divide each 2D video frame to be embedded in the 3D video into multiple sub-blocks; A first decomposition module is used to perform multi-level DT-CWT decomposition on any component in the color space of any sub-block in any two-dimensional video frame to obtain sub-bands of the sub-block at each level of decomposition; A subband modification module, used for modifying the amplitude matrix of the subband obtained by decomposing the sub-block within a set series range according to the watermark to be embedded to obtain a modified subband; A sub-block watermark embedding module, used for performing an inverse DT-CWT operation on the modified sub-band to obtain an embedded watermark sub-block corresponding to the sub-block; A video frame watermark embedding module, used for splicing all the embedded watermark sub-blocks corresponding to the two-dimensional video frame to obtain a watermark corresponding to the two-dimensional video frame and embedding it into a two-dimensional video frame; The watermark determination module is used to embed the two-dimensional video frame according to all watermarks corresponding to the three-dimensional video to be embedded to obtain the watermark embedded video and the zero watermark of the three-dimensional video to be embedded.

8. A watermark extraction system based on 3D video, characterized in that: The 3D video-based watermark extraction method according to claim 3, wherein the 3D video-based watermark extraction system comprises: The second block division module is used to divide each 2D video frame in the 3D video to be extracted into blocks to obtain a plurality of extraction sub-blocks; A second decomposition module is used to perform multi-level DT-CWT decomposition on any component in the color space of any extracted sub-block in any two-dimensional video frame to obtain sub-bands of the extracted sub-block at each level of decomposition; An embedded watermark extraction module, used for determining the watermark embedded in the three-dimensional video to be extracted according to the amplitude matrix of the sub-band obtained by decomposing each of the extracted sub-blocks in the three-dimensional video to be extracted within a set series range; The zero watermark extraction module is used to generate a zero watermark of the three-dimensional video to be extracted according to the three-dimensional video to be extracted.

9. A watermark comparison system based on 3D video, characterized in that: include: A watermark embedding module, configured to process the three-dimensional video to be protected according to the watermark embedding method based on 3D video according to any one of claims 1 to 2 to obtain an embedded watermark and a zero watermark; A watermark extraction module, configured to process the three-dimensional video to be authenticated according to the watermark extraction method based on 3D video according to any one of claims 3-4 to obtain an extracted embedded watermark and an extracted zero watermark; The copyright determination module is used to determine whether the to-be-identified 3D video infringes the copyright of the to-be-embedded 3D video according to the embedded watermark, the extracted embedded watermark, the zero watermark and the extracted zero watermark.

Images