Video watermarking method based on adaptive embedding strength and ring tensor
Through the video watermarking method of adaptive embedding intensity and ring tensor, combined with video time redundancy and grayscale symbiosis matrix evaluation, the problem of difficult balance between robustness and inesis in the prior art is solved, and the video is achieved Effective authentication of copyright and improved robustness in complex scenarios.
Patent Information
- Application Number
- CN202410926006.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-12
AI Technical Summary
Existing frequency domain-based video watermarking methods are poorly robust in dealing with geometric attacks and time-synchronous attacks, and it is difficult to balance robustness and insensitivity, especially in complex scenarios. Video copyright certification.
Using video watermarking methods with adaptive embedding intensity and ring tensor, scene division of video through video time redundancy, texture complexity is evaluated using grayscale symbiosis matrix, and embedding intensity is selected in the Y channel and U channel of video frame The copyright watermark and synchronous watermark are embedded respectively in it, and the extracted copyright watermark is trained using the neural network model to improve its accuracy.
While ensuring the unawareness of the watermark, it improves its resistance to conventional attacks and geometric attacks, enhances the robust authentication ability of video copyright, and achieves robust authentication in complex scenarios.
Smart Images

Figure CN118945399A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the field of information hiding, in particular to a video watermarking method based on adaptive embedding strength and ring tensor. Background Art
[0002] In recent years, with the continuous advancement of network technology and editing software, digital videos are facing serious problems of illegal tampering, copying and distribution, which may lead to the spread of false information, invasion of privacy and even affect national security. Therefore, how to effectively protect the copyright of digital video content has always been the focus of academic and industrial circles.
[0003] Digital watermarking is a technical means to protect video intellectual property rights. It achieves active defense by embedding invisible watermarks in videos. Common methods include video watermarking technologies based on spatial domain, compression domain, neural network and frequency domain. Video watermarking based on spatial domain embeds watermarks by directly modifying pixels. Its algorithm is simple and computationally inefficient. However, this method is very sensitive to operations such as noise and filtering, and the watermark is easily attacked or removed. Video watermarking based on compression domain embeds watermarks in compressed data, reducing the overhead of decompression and recompression, and is suitable for real-time applications. However, due to the involvement of video coding standards, the implementation is relatively complex, and there are differences between different coding standards, making the method not universal and limited by the coding itself. Video watermarking based on neural networks can adapt to different video contents by training models without manual intervention, and achieve end-to-end training. However, this method requires a large number of training data sets, has high requirements on computing performance, and is difficult to effectively resist unknown attacks. In contrast, the method based on frequency domain embeds watermarks into the stable features of the medium or low frequencies of the video. It performs well in resisting conventional video processing attacks such as recompression, noise and filtering, and has attracted more and more attention from researchers.
[0004] At present, the video watermarking method based on frequency domain still has the following shortcomings:
[0005] (1) The robustness against geometric attacks and time synchronization attacks is poor, especially the ability to resist multiple attack combinations is limited, and robust authentication cannot be achieved.
[0006] (2) Most methods use a fixed embedding strength without considering the scene changes in the video and the texture complexity of different scenes, making it difficult to strike a balance between robustness and imperceptibility. Summary of the invention
[0007] The technical problem to be solved by the present invention is to provide a video watermarking method based on adaptive embedding strength and annular tensor. First, the original video is divided into scenes using video temporal redundancy, and the texture complexity of each scene is evaluated using the gray-level co-occurrence matrix to select the appropriate embedding strength, thereby achieving a balance between imperceptibility and robustness. Secondly, the method of annularly dividing the video frame is adopted to embed the copyright watermark and the synchronization watermark in the Y channel and U channel rings of the video frame respectively. Finally, the extracted copyright watermark is trained through a neural network model to further improve the accuracy of the watermark, thereby realizing robust authentication of video copyright in complex scenes.
[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:
[0009] A video watermarking method based on adaptive embedding strength and ring tensor, comprising the following steps:
[0010] S1, embedding watermark into the video to be processed;
[0011] S1.1 Video scene division and texture complexity evaluation:
[0012] S1.2 Ring segmentation of the video to be processed;
[0013] S1.3 Synchronous watermark embedding;
[0014] S1.4 embeds copyright watermark into the processed video;
[0015] S2, extracting the video containing the watermark;
[0016] S2.1 divides the scene according to the watermark embedding S1.1 to obtain the texture complexity of each scene and determine the embedding strength;
[0017] S2.2 converts the watermarked video frame into the YCbCr space, selects the U channel for ring segmentation and discrete Fourier transform and singular value decomposition, obtains the maximum singular value containing the watermark, extracts the synchronization watermark according to the maximum singular value, and corrects the time series of the video frame;
[0018] S2.3 converts the time-corrected video frames into YCbCr space, selects the Y channel for ring segmentation and DCT transformation, forms a tensor of frequency domain ring coefficients with the same radius in consecutive frames, decomposes the tensor using high-order singular value decomposition, obtains a watermarked time factor matrix, and extracts the copyright watermark from the first column elements;
[0019] S2.4 inputs the extracted copyright watermark into a pre-trained neural network model to correct the watermark information and improve the clarity.
[0020] A further improvement of the technical solution of the present invention is that S1.1 specifically includes the following steps:
[0021] (1) Convert the video frame into grayscale space and perform discrete wavelet transform on the inscribed circle of the video frame to obtain four subbands of the same size;
[0022] (2) Calculate the mean frame difference between adjacent frames in the low-frequency sub-band to determine whether scene division is required;
[0023] (3) For videos that need to be divided into scenes, the frame difference mean is normalized and the video is divided into different scenes according to the threshold;
[0024] (4) The gray-level co-occurrence matrix is used to statistically analyze the homogeneity, energy, and correlation of the gray-level inscribed circle of the video frame to obtain the texture complexity of each scene, and the appropriate embedding strength is selected based on the texture complexity.
[0025] A further improvement of the technical solution of the present invention is that S1.2 specifically includes the following steps:
[0026] (1) First, determine the central pixel position of the video frame and calculate the spatial distance from the surrounding pixels to the central pixel position in the video frame;
[0027] (2) Pixels with the same distance are divided into the same ring.
[0028] A further improvement of the technical solution of the present invention is that S1.3 specifically includes the following steps:
[0029] (1) Convert the video frame from RGB space to YCbCr space, and perform ring segmentation on the U channel according to the method in Section 1.2 to obtain a ring;
[0030] (2) Perform discrete Fourier transform on each ring and extract frequency domain information;
[0031] (3) Perform singular value decomposition on the frequency domain information of each ring to obtain a singular value matrix;
[0032] (4) The quaternary synchronization watermark is embedded into the maximum singular value of each singular value matrix, and the watermarked matrix is subjected to inverse singular value decomposition and inverse discrete Fourier transform to obtain the U channel containing the watermark.
[0033] A further improvement of the technical solution of the present invention is that S1.4 specifically includes the following steps:
[0034] (1) Select the Y channel and perform ring segmentation according to the method in S1.2, and use discrete cosine transform to process the ring to obtain frequency domain coefficients;
[0035] (2) The frequency domain coefficients of the rings with the same radius in four consecutive frames are combined into a tensor;
[0036] (3) Decompose the tensor using high-order singular value decomposition to obtain the temporal factor matrix of the video;
[0037] (4) The copyright watermark is embedded by modifying the pair of coefficients with the strongest correlation in the first column of the factor matrix, and the watermarked factor matrix and other dimensional matrices are reconstructed into a watermarked tensor, which is then processed by inverse discrete cosine transform to obtain the watermarked Y channel; finally, the three channels of the video frame are merged and converted to RGB space to obtain the final watermarked video.
[0038] A further improvement of the technical solution of the present invention is that in S2.4, the neural network model includes 16 convolutional layers, wherein the convolution kernel size is 3×3, the activation function is the ReLu function, and the mean square error between the trained watermark and the original watermark is used as the loss function, and the loss function is defined as follows:
[0039]
[0040] Among them, W c is the original watermark, and n is the watermark length.
[0041] Due to the adoption of the above technical solution, the technical progress achieved by the present invention is:
[0042] 1. The present invention adopts a video texture complexity evaluation method based on frame difference method scene division and gray level co-occurrence matrix statistics to adjust the watermark embedding strength according to different scenes, thereby ensuring the imperceptibility of the watermark while improving the ability to resist conventional attacks;
[0043] 2. The present invention utilizes the redundancy of the video time dimension and adopts a ring-shaped division method for video frames to improve the robustness of the watermark against geometric attacks;
[0044] 3. The present invention embeds two watermarks: synchronization watermark and copyright watermark. The quaternary synchronization watermark is embedded into the maximum singular value to correct the time sequence of the video and improve the robustness of the watermark against time synchronization attacks; the copyright watermark is embedded into the factor matrix of the torus tensor to achieve the purpose of robust video authentication.
[0045] 4. The present invention uses a neural network to train the extracted copyright watermark to further improve the accuracy of the copyright watermark. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is an embedding flow chart of a video watermarking method based on adaptive embedding strength and ring tensor in the present invention;
[0047] Figure 2 It is an extraction flow chart of a video watermarking method based on adaptive embedding strength and ring tensor in the present invention;
[0048] Figure 3 It is a structural diagram of the copyright watermark correction network model in the present invention;
[0049] Figure 4 is the PSNR of different videos after watermarking in the present invention;
[0050] Figure 5 It is the change of the watermarked video NC under different noise attacks in the present invention;
[0051] Figure 6 It is the change of the watermarked video NC under different cropping areas in the present invention;
[0052] Figure 7 This is the change of the watermarked video NC at different rotation angles in the present invention. DETAILED DESCRIPTION
[0053] The present invention is further described in detail below with reference to the accompanying drawings and embodiments:
[0054] A video watermarking method based on adaptive embedding strength and ring tensor, including watermark embedding and watermark extraction;
[0055] S1, embedding watermark into the video to be processed;
[0056] Watermark embedding includes video scene segmentation and texture complexity assessment, ring segmentation, synchronous watermark embedding, and copyright watermark embedding. The process diagram of watermark embedding is as follows: Figure 1 As shown, the specific steps are as follows:
[0057] S1.1 Video scene division and texture complexity evaluation:
[0058] (1) The video frame V to be processed f Convert to grayscale space and select the inscribed circle part G f ; Convert the grayscale inscribed circle video frame into a two-dimensional matrix H f , and apply DWT transform (discrete wavelet transform) to extract low-frequency information. The calculation formula is as follows:
[0059] [LL,LH,HL,HH]=dwt2(H f )(1.1)
[0060] Among them, LL is the low-frequency sub-band, and LH, HL, and HH are high-frequency sub-bands;
[0061] (2) In order to determine the difference between adjacent frames, the mean of their frequency domain differences is calculated. The calculation formula for the mean frame difference is as follows:
[0062]
[0063] Among them, LL i (x,y) and LL i+1 (x, y) represents the frequency domain value of the i-th frame and the i+1-th frame (x, y), M and N are the width and height of the two-dimensional matrix; the average frame difference between video frames is used to judge the continuity of the scene: when the average frame difference does not exceed T 1 When T is exceeded, the video frames are considered to have little change over time and can be considered to belong to the same scene without scene division. 1 When the scene is divided, it is necessary to consider the division of scenes;
[0064] (3) Scene segmentation is achieved by normalizing the mean of frame differences. The calculation method is as follows:
[0065]
[0066] Among them, max and min represent obtaining the maximum and minimum values respectively; when V norm (i) Exceeding T 2 , then these frames are considered as the same scene; otherwise, if V norm (i) Less than T 2 , it is divided into different scenes. 1 and T 2 The selection is based on the analysis of 10 standard video datasets, calculating the D (i) and V norm (i) is determined by taking the average value;
[0067] (4) In order to balance the imperceptibility and robustness of the watermark, the gray-level co-occurrence matrix is used to calculate the texture feature statistics of the video frame. Different weights are assigned to each statistic according to the normalized homogeneity, energy, and correlation. The texture feature is calculated as follows:
[0068]
[0069] Among them, F represents the texture complexity of different scenes, n represents the total number of video frames of the same scene, and (a, b, c) represent the homogeneity G respectively. 1 Energy G 2 , Correlation G 3 The weight coefficient of , and satisfies a+b+c=1; given threshold T 3When the scene complexity F exceeds the threshold, it indicates that the texture of the scene is relatively simple, insensitive to various attacks, and more stable, so a lower embedding strength is used; conversely, the texture of the scene is relatively complex and has weak resistance to conventional attacks such as noise and filtering, so a higher embedding strength is required.
[0070] S1.2 Ring segmentation of the video to be processed;
[0071] (1) Calculate the center of the video frame and calculate the spatial distance between the surrounding pixel positions and the central pixel position. The calculation method of the video frame center is as follows:
[0072]
[0073] Among them, V M and V N are the width and height of the video respectively, and the spatial distance is calculated as follows:
[0074]
[0075] Among them, D(x,y) represents the distance from the surrounding pixels to the central pixel (x 0 ,y 0 ), (x i ,y i ) are the surrounding pixels except the central pixel in the video frame;
[0076] (2) A ring is defined as all pixels whose distance from the surrounding pixels to the center of the video frame is equal. The ring is calculated as follows:
[0077] R i ={(x i ,y i )|D i-1 ≤D(x,y)<D i} (1.7)
[0078] Among them, D i-1 and D i are the i-th ring R i The starting radius and ending radius of the ring are assumed to be R. w , then D i The calculation is as follows:
[0079] D i =D 0 +iR w (1.8)
[0080] Among them, D 0 Indicates the starting radius of the video frame, i=1,2,3,…i max ,i max Indicates the maximum number of rings that can be divided, and the calculation formula is:
[0081]
[0082] in Indicates the maximum distance from the surrounding pixels to the center position.
[0083] S1.3 Synchronous watermark embedding;
[0084] (1) Convert the video frame from RGB space to YCbCr space, select the U channel and perform ring segmentation according to the method in S1.2 to obtain the ring
[0085] (2) In order to improve the ability of the synchronized watermark to resist geometric processing, DFT transform is used to extract the frequency domain information of the ring. The calculation method is as follows:
[0086]
[0087] Among them, L R is the DFT coefficient, j is the imaginary unit, R m and R n It represents the width and height of the ring after it is converted into a two-dimensional matrix. u and v are the frequency domain coordinates after DFT.
[0088] (3) Perform SVD (singular value decomposition) on the frequency domain information of the ring to ensure robustness while improving imperceptibility. The SVD decomposition is as follows:
[0089] L R =L U λL V (1.11)
[0090] Among them, λ is the singular value matrix, which contains the main components of the matrix and has high stability. U and L V is an orthogonal matrix;
[0091] (4) Embed the quaternary synchronization watermark into the maximum singular value of the singular value matrix and modify the formula as follows:
[0092]
[0093] where λ m and Represent the original maximum singular value and the maximum singular value with watermark, Δ=mod(λ m ,q s ) represents the remainder operation, W s is the synchronization watermark, q s To synchronize the watermark embedding strength; perform inverse SVD and inverse DFT processing on the matrix after embedding the watermark to obtain the U channel containing the watermark.
[0094] 1.4 Embed copyright watermark into the video to be processed;
[0095] (1) Select the Y channel and perform ring segmentation according to the method in S1.2, using DCT (discrete cosine transform) to transform the Y channel with a width of R. w The ring is processed to obtain the frequency domain coefficient C k ;
[0096] (2) The frequency domain coefficients of the circular rings with the same radius in four consecutive frames are combined into a tensor. The tensor is defined as follows:
[0097] A i ={C k (u,v)|1<k<K} (1.13)
[0098] in k represents the kth video frame;
[0099] (3) Use HOSVD (high-order singular value decomposition) to decompose the tensor and obtain its time factor matrix. The tensor decomposition is calculated as follows:
[0100] A i ≈A C × 1 A X × 2 A V × 3 A U (1.14)
[0101] Among them, A C is the core tensor, A X , A V , A U is the factor matrix, A U A factor matrix representing the time dimension;
[0102] (4) The pair of elements with the highest correlation in the first column of the adjustment factor matrix is used to embed the copyright watermark. The calculation formula for copyright watermark embedding is as follows:
[0103] When W c (i) = 1
[0104]
[0105] When W c (i) = 0
[0106]
[0107] Among them, W c A is the copyright watermark bit, U and Represent the original factor matrix and the watermarked factor matrix respectively, u 1 and u 2 The two positions with the highest correlation in the first column of the factor matrix, sign represents the sign function; the watermarked factor matrix and other dimensional matrices are reconstructed into a watermarked tensor, and then the inverse DCT is performed to obtain the watermarked Y channel, and the three video channels are merged and converted to RGB space to obtain the final watermarked video V w .
[0108] S2, extracting the video containing the watermark;
[0109] The steps of extracting watermarks are opposite to those of embedding watermarks, but the steps in the video processing are the same. Both require ring segmentation of the video frame, and then frequency domain transformation and matrix decomposition of the Y channel and U channel to extract the maximum singular value containing the watermark. and the watermark factor matrix like Figure 2 As shown, the specific steps are as follows:
[0110] S2.1 divides the scene according to the watermark embedding S1.1 to obtain the texture complexity of each scene and determine the embedding strength;
[0111] S2.2 extracts the synchronization watermark and corrects the time sequence of the video frame according to the synchronization watermark. The extraction formula of the synchronization watermark is as follows:
[0112]
[0113] Among them, W′ s represents the extracted synchronization watermark sequence, mod is the modulo function;
[0114] By judging whether the synchronization watermark conforms to the 1234 sequence, that is, the original quaternary synchronization watermark 123412341234, if repeated watermark bits appear, the repeated frames are directly deleted; if lost watermark bits appear, a frame is supplemented with a full 0 pixel frame, thereby ensuring the time synchronization of the watermarked video and ensuring that the copyright watermark can be accurately extracted;
[0115] S2.3 After the video sequence is corrected, the copyright watermark is extracted according to the following formula:
[0116]
[0117] Among them, W′ c For the extracted copyright watermark;
[0118] S2.4 Extract the copyright watermark information W′ c Input into the trained network model, the network model structure is as follows Figure 3As shown in the figure, it mainly includes 16 convolution layers, where the convolution kernel size is 3×3, the activation function is the ReLu function, and the mean square error between the trained watermark and the original watermark is used as the loss function. The loss function is defined as follows:
[0119]
[0120] Among them, W c is the original watermark, and n is the watermark length.
[0121] The present invention has conducted several experiments, including the PSNR of standard videos such as containers after embedding watermarks and the average NC of different watermarked videos under noise attack, cropping attack, and rotation attack. Figure 4 , Figure 5 , Figure 6 , Figure 7 shown. Figure 4 PSNR (peak signal-to-noise ratio) after embedding watermarks in different videos. Figure 5 The change of NC (normalized correlation coefficient) of watermarked video under different noise attacks. Figure 6 The changes of NC of watermarked video under different cropping areas are shown in Figure 2. Figure 7 The changes of the watermarked video NC at different rotation angles.
[0122] The above-described embodiments are merely descriptions of preferred implementations of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solution of the present invention should fall within the protection scope of the present invention, and the technical contents for which protection is sought in the present invention have been fully recorded in the claims.
Claims
1. A video watermarking method based on adaptive embedding strength and ring tensor, characterized by: The following steps are involved: S1, embedding watermark into the video to be processed; S1.1 Video scene division and texture complexity evaluation: S1.2 Ring segmentation of the video to be processed; S1.3 Synchronous watermark embedding; S1.4 embedding copyright watermark into the processed video; S2, extracting the video containing the watermark; S2.1 divides the scene according to the watermark embedding S1.1 to obtain the texture complexity of each scene and determine the embedding strength; S2.2 converts the watermarked video frame into the YCbCr space, selects the U channel for ring segmentation and discrete Fourier transform and singular value decomposition, obtains the maximum singular value containing the watermark, extracts the synchronization watermark according to the maximum singular value, and corrects the time series of the video frame; S2.3 converts the time-corrected video frames into YCbCr space, selects the Y channel for ring segmentation and DCT transformation, forms a tensor of frequency domain ring coefficients with the same radius in consecutive frames, decomposes the tensor using high-order singular value decomposition, obtains a watermarked time factor matrix, and extracts the copyright watermark from the first column elements; S2.4 inputs the extracted copyright watermark into a pre-trained neural network model to correct the watermark information and improve the clarity.
2. The video watermarking method based on adaptive embedding strength and ring tensor according to claim 1, characterized in that: S1.1 specifically includes the following steps: (1) Convert the video frame into grayscale space and perform discrete wavelet transform on the inscribed circle of the video frame to obtain four subbands of the same size; (2) Calculate the mean frame difference between adjacent frames in the low-frequency sub-band to determine whether scene division is required; (3) For videos that need to be divided into scenes, the frame difference mean is normalized and the video is divided into different scenes according to the threshold; (4) The gray-level co-occurrence matrix is used to statistically analyze the homogeneity, energy, and correlation of the gray-level inscribed circle of the video frame to obtain the texture complexity of each scene, and the appropriate embedding strength is selected based on the texture complexity.
3. The video watermarking method based on adaptive embedding strength and ring tensor according to claim 1, characterized in that: S1.2 specifically includes the following steps: (1) First, determine the central pixel position of the video frame and calculate the spatial distance from the surrounding pixels to the central pixel position in the video frame; (2) Pixels with the same distance are divided into the same ring.
4. The video watermarking method based on adaptive embedding strength and ring tensor according to claim 1, characterized in that: S1.3 specifically includes the following steps: (1) Convert the video frame from RGB space to YCbCr space, and perform ring segmentation on the U channel according to the method in Section 1.2 to obtain a ring; (2) Perform discrete Fourier transform on each ring and extract frequency domain information; (3) Perform singular value decomposition on the frequency domain information of each ring to obtain a singular value matrix; (4) The quaternary synchronization watermark is embedded into the maximum singular value of each singular value matrix, and the watermarked matrix is subjected to inverse singular value decomposition and inverse discrete Fourier transform to obtain the U channel containing the watermark.
5. The video watermarking method based on adaptive embedding strength and ring tensor according to claim 1, characterized in that: S1.4 specifically includes the following steps: (1) Select the Y channel and perform ring segmentation according to the method in S1.2, and use discrete cosine transform to process the ring to obtain frequency domain coefficients; (2) The frequency domain coefficients of the rings with the same radius in four consecutive frames are combined into a tensor; (3) Decompose the tensor using high-order singular value decomposition to obtain the temporal factor matrix of the video; (4) The copyright watermark is embedded by modifying the pair of coefficients with the strongest correlation in the first column of the factor matrix, and the watermarked factor matrix and other dimensional matrices are reconstructed into a watermarked tensor, which is then processed by inverse discrete cosine transform to obtain the watermarked Y channel; finally, the three channels of the video frame are merged and converted to RGB space to obtain the final watermarked video.
6. The video watermarking method based on adaptive embedding strength and ring tensor according to claim 1, characterized in that: In S2.4, the neural network model includes 16 convolutional layers, where the convolution kernel size is 3×3, the activation function is the ReLu function, and the mean square error between the trained watermark and the original watermark is used as the loss function. The loss function is defined as follows: Among them, W c is the original watermark, and n is the watermark length.
