Video watermarking methods

Abstract

This application provides a video watermarking method, which includes: encoding an original watermark to obtain a first watermark sequence to be embedded; decoding an original video to obtain a first video frame sequence; selecting a second video frame sequence from the first video frame sequence to which a watermark needs to be embedded; embedding the first watermark sequence into the second video frame sequence; and obtaining a watermarked video after encoding. The video watermarking method provided in this application improves the invisibility of the watermark embedding by selecting a second video frame sequence from the first video frame sequence to which a watermark needs to be embedded, embedding the first watermark sequence into the second video frame sequence, and obtaining a watermarked video after encoding.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a video watermarking method. Background Technology

[0002] With the popularity of video and film services and the widespread use of mobile shooting devices such as smartphones and cameras, it has become increasingly easy for people to publish and access video content on the mobile internet. At the same time, the protection of digital video copyrights has gained attention.

[0003] Video watermarking technology provides an excellent solution for copyright protection of digital videos. Traditional video watermarking algorithms are divided into spatial domain algorithms and transform domain algorithms. Among them, spatial domain algorithms directly modify the pixels of video frames to embed the watermark, which is highly efficient. Transform domain algorithms perform mathematical transformations on video frames to embed the watermark information into the transform domain, and then transform the transform domain back to the spatial domain, thereby achieving the purpose of adding watermarks to the video. They have better robustness and invisibility, but have high computational complexity.

[0004] Watermark invisibility and robustness are the two most critical performance indicators for video watermarking algorithms. Regarding watermark invisibility, key considerations include selecting a suitable embedding frame and embedding position. For robustness, key attack types to consider include editing attacks, geometric attacks, and desynchronization attacks. Relatively speaking, the ability of video watermarking algorithms to resist geometric attacks is currently the most pressing and critical issue in the field, especially since videos displayed on different network terminals may experience resolution changes, partial subtitle appearances, or obstructions; current video watermarking solutions struggle to perfectly address this problem. Summary of the Invention

[0005] To address the aforementioned problems with related technologies, this application provides a video watermarking method.

[0006] In a first aspect, embodiments of this application provide a video watermarking method, including:

[0007] The original watermark is encoded to obtain the first watermark sequence to be embedded;

[0008] The original video is decoded to obtain the first video frame sequence;

[0009] Select the second video frame sequence from the first video frame sequence to embed the watermark;

[0010] The first watermark sequence to be embedded is embedded into the second video frame sequence, and the video with the embedded watermark is obtained after encoding processing.

[0011] In some embodiments, any video frame in the first video frame sequence includes an R channel, a G channel, and a B channel, and the step of selecting a second video frame sequence from the first video frame sequence to embed a watermark includes:

[0012] For any video frame in the first video frame sequence, two channels are selected from the R channel, the G channel, and the B channel as the first channel and the second channel, respectively;

[0013] Based on the first and second channels of the current video frame in the first video frame sequence, and the first and second channels of the previous video frame, determine whether to select the current video frame as the video frame for which a watermark needs to be embedded.

[0014] The second video frame sequence is obtained by sequentially filtering all video frames in the first video frame sequence.

[0015] In some embodiments, determining whether to select the current video frame as the video frame for which a watermark needs to be embedded includes:

[0016] Extract the first feature sequence of the first channel and the second feature sequence of the second channel of the current video frame; extract the first feature sequence of the first channel and the second feature sequence of the second channel of the previous video.

[0017] Determine the first correlation value between the first feature sequence of the first channel of the current video frame and the first feature sequence of the first channel of the previous video frame; determine the second correlation value between the second feature sequence of the second channel of the current video frame and the second feature sequence of the second channel of the previous video frame.

[0018] The first correlation value and the second correlation value are averaged, and the average value is compared with a preset threshold.

[0019] If the average value is less than the preset threshold, the current video frame is selected as the video frame for which a watermark needs to be embedded; or, if the average value is greater than or equal to the preset threshold, the current video frame is not selected as the video frame for which a watermark needs to be embedded.

[0020] In some embodiments, embedding the first watermark sequence to be embedded into the second video frame sequence includes:

[0021] For each video frame in the second video frame sequence, watermark embedding is performed based on the third channel of each video frame;

[0022] The third channel is a channel other than the first channel and the second channel.

[0023] In some embodiments, the watermark embedding based on the third channel of each video frame includes:

[0024] The chroma matrix of the third channel is divided into blocks;

[0025] Calculate the mean chromaticity value of each block and arrange them in ascending or descending order to form the first chromaticity mean matrix;

[0026] Perform a discrete cosine transform on the first chromaticity mean matrix to obtain the first transform coefficient matrix;

[0027] Multiple coefficient pairs are selected from the first transformation coefficient matrix using the first key, and the watermark embedding is completed by modifying the coefficient values ​​of the coefficient pairs.

[0028] In some embodiments, the watermark embedding is accomplished by modifying the coefficient values ​​of the coefficient pairs, satisfying the following modification principles:

[0029] If w i =0:

[0030] If w i =1:

[0031] Among them, (a 1i ,b 1i ) represents a coefficient pair formed by the coefficient values ​​corresponding to the two positions on the main diagonal in the first transformation coefficient matrix, w i Let α represent the value of the i-th position in the first watermark sequence to be embedded, where i = 1, 2, ..., L, L represents the length of the first watermark sequence to be embedded, and α represents the watermark embedding strength.

[0032] In some embodiments, the method further includes:

[0033] The video from which the watermark is to be extracted is decoded to obtain a third video frame sequence;

[0034] Select the fourth video frame sequence from the third video frame sequence from which the watermark needs to be extracted;

[0035] Extract the second watermark sequence from each video frame in the fourth video frame sequence, and determine the third watermark sequence based on the second watermark sequence corresponding to each video frame in the fourth video frame sequence and perform decoding processing.

[0036] In some embodiments, extracting the second watermark sequence from each video frame in the fourth video frame sequence includes:

[0037] The chroma matrix of the third channel of each video frame in the fourth video frame sequence is divided into blocks, the mean of the chroma values ​​of each block is calculated, and the blocks are arranged in ascending or descending order to form a second chroma mean matrix.

[0038] Perform a discrete cosine transform on the second chromaticity mean matrix to obtain the second transform coefficient matrix;

[0039] Multiple coefficient pairs are selected from the second transformation coefficient matrix using the second key, and the second watermark sequence is extracted based on the coefficient values ​​of the coefficient pairs.

[0040] In some embodiments, the extraction of the second watermark sequence based on the coefficient values ​​of coefficient pairs satisfies the following rules:

[0041]

[0042] Among them, (a 2i ,b 2i () represents a coefficient pair in the second transformation coefficient matrix, consisting of the coefficient values ​​corresponding to the two positions on the main diagonal. This represents the value of the i-th element in the extracted second watermark sequence, where i = 1, 2, ..., L, and L represents the length of the second watermark sequence.

[0043] In some embodiments, determining the third watermark sequence based on the second watermark sequence corresponding to each video frame in the fourth video frame sequence includes:

[0044] Based on the third correlation value between the second watermark sequences corresponding to any two video frames in the fourth video frame sequence, a correlation matrix is ​​determined.

[0045] The elements in each row vector of the relevance matrix are summed, and the third watermark sequence is determined based on the maximum sum.

[0046] Secondly, embodiments of this application also provide a video watermarking processing apparatus, comprising:

[0047] The first encoding module is used to encode the original watermark and obtain the first watermark sequence to be embedded.

[0048] The first decoding module is used to decode the original video and obtain the first video frame sequence;

[0049] The first filtering module is used to filter out the second video frame sequence that needs to be embedded with watermarks from the first video frame sequence;

[0050] An embedding module is used to embed the first watermark sequence to be embedded into the second video frame sequence, and obtain the video after encoding processing.

[0051] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the video watermarking methods described in the first aspect above.

[0052] Fourthly, embodiments of this application also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the video watermarking methods described in the first aspect above.

[0053] Fifthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements any of the video watermarking methods described in the first aspect above.

[0054] The video watermarking method provided in this application improves the invisibility of watermark embedding by selecting a second video frame sequence that needs to be embedded with watermark from a first video frame sequence, embedding the first watermark sequence to be embedded into the second video frame sequence, and obtaining the video with embedded watermark after encoding processing. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart illustrating the video watermarking method provided in an embodiment of this application;

[0057] Figure 2 This is a schematic diagram of the video watermarking processing device provided in the embodiments of this application;

[0058] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0059] Online video content is characterized by its ease of copying, distribution, processing, and rapid transmission. The phenomenon of copyrighted online video works being disseminated at will is widespread, making the protection of digital video copyrights an urgent matter.

[0060] Video watermarking technology offers a good solution to the problem of digital video copyright protection. It embeds a watermark containing copyright information imperceptibly into the video; in the event of a copyright dispute, the ownership of the video can be identified by extracting the watermark. Although video watermarking technology has a history of over twenty years and has made great progress, there are still some problems that need to be solved in academic research and engineering applications.

[0061] Traditional video watermarking algorithms are divided into two categories: spatial domain algorithms and transform domain algorithms.

[0062] The video watermarking spatial domain algorithm directly modifies the pixels of video frames to embed the watermark, resulting in high operating efficiency.

[0063] The Transform Domain Algorithm for Video Watermarking embeds watermark information into the transform domain by performing mathematical transformations on video frames. Then, it inversely transforms the transform domain back to the spatial domain, thus achieving the purpose of adding a watermark to the video. It exhibits good robustness and invisibility, but its computational complexity is high. Transformation methods include: Discrete Fourier Transform (DFT), Discrete Cosine Transform (DCT), Discrete Wavelet Transform (DWT), Singular Value Decomposition (SVD), Discrete Multiwavelet Transformation (DMT), Contourlet Transform (CT), and Dual-Tree Complex Wavelet Transform (DT CWT). The advantages of the Transform Domain Algorithm for Video Watermarking are its good invisibility (i.e., minimal impact on video quality after watermark embedding) and its high robustness (i.e., resistance to a certain degree of attacks and compression). However, the computational complexity is high.

[0064] Watermark invisibility and watermark robustness are the two most important performance indicators for video watermarking algorithms. The former refers to the watermark embedded in the video being imperceptible to the human eye, meaning that viewers cannot detect the watermark's existence during video playback; the latter refers to the ability of the watermark to be accurately extracted even if the video containing the watermark is subjected to a certain degree of intentional or unintentional attack during its distribution and use.

[0065] To ensure the watermark's invisibility, researchers have been exploring how to select a good watermark embedding frame and embedding position. To ensure the watermark's robustness, the main attack types considered include editing attacks, geometric attacks, and desynchronization attacks. Relatively speaking, the ability of video watermarking algorithms to resist geometric attacks is the most pressing issue in the field of video watermarking technology. In particular, videos displayed on different network terminals may have changes in resolution, partial subtitles, or occlusions, and current video watermarking algorithms struggle to perfectly address this problem.

[0066] To address the aforementioned problems in related technologies, this application provides a video watermarking method. By selecting a second video frame sequence from a first video frame sequence that needs to be embedded with a watermark, embedding the first watermark sequence into the second video frame sequence, and then encoding the watermarked video, the invisibility of the watermark embedding is improved.

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0068] Figure 1 This is one of the flowcharts illustrating the video watermarking method provided in this application embodiment, such as... Figure 1 As shown, the method includes at least the following steps:

[0069] Step 101: Encode the original watermark to obtain the first watermark sequence to be embedded.

[0070] Specifically, encoding the original watermark can be performed by converting the watermark information into a specific encoded form through a mapping relationship. This process can include steps such as defining the mapping relationship, analyzing the original watermark, mapping and encoding, and generating the encoded watermark. By encoding the original watermark, a first watermark sequence to be embedded is obtained, which facilitates subsequent embedding.

[0071] For example, the original watermark can be a 0-1 string of a certain length, a -1 / +1 string of a certain length, a string of a certain length consisting of letters, numbers, underscores, etc., or a small binary image of a certain size. The encoding form can be a 0-1 binary sequence.

[0072] Specifically, any of the above-mentioned original watermarks can be mapped into a 0-1 binary sequence of a specific length through a mapping relationship. Then, the 0-1 binary sequence is scrambled using key1, and combined with error correction coding, to obtain a first watermark sequence of length L to be embedded, represented as W = [w1, w2, ..., w...]. L ].

[0073] This application does not impose any restrictions on the encoding format of the original watermark, how to define the mapping relationship between the original watermark and the above-mentioned 0-1 binary sequence, or error correction encoding.

[0074] Step 102: Decode the original video to obtain the first video frame sequence.

[0075] Specifically, the original video is encoded video data, so it needs to be decoded according to its encoding method to obtain a complete video frame sequence.

[0076] In this application, after decoding the original video, all video frames are arranged in chronological order to obtain the first video frame sequence. Each video frame in the video frame sequence can be regarded as a complete image.

[0077] Step 103: Select the second video frame sequence from the first video frame sequence that needs to be embedded with a watermark.

[0078] Specifically, in step 102, to improve the invisibility of the watermark, it is not necessary to embed the watermark in every single video frame of the first video frame sequence. Therefore, a selection of video frames is made from the first video frame sequence, and these selected video frames can also be arranged in chronological order to form the second video frame sequence.

[0079] Step 104: Embed the first watermark sequence to be embedded into the second video frame sequence, and obtain the video after encoding processing.

[0080] Specifically, based on the first watermark sequence to be embedded obtained in step 101, it is embedded into the second video frame sequence. The video frames before watermark embedding are replaced with the video frames after watermark embedding, and the video after watermark embedding is obtained after encoding processing.

[0081] The encoding process in step 104 should correspond to the decoding process in step 102, that is, the same encoding format and bitrate parameters should be used to complete the encoding process of the video after embedding the watermark.

[0082] The video watermarking method provided in this application improves the invisibility of watermark embedding by selecting a second video frame sequence that needs to be embedded with watermark from a first video frame sequence, embedding the first watermark sequence to be embedded into the second video frame sequence, and obtaining the video with embedded watermark after encoding processing.

[0083] In some embodiments, any video frame in the first video frame sequence includes an R channel, a G channel, and a B channel. Step 103, selecting a second video frame sequence from the first video frame sequence to embed a watermark, specifically includes:

[0084] For any video frame in the first video frame sequence, two channels are selected from the R channel, the G channel, and the B channel as the first channel and the second channel, respectively;

[0085] Based on the first and second channels of the current video frame in the first video frame sequence, and the first and second channels of the previous video frame, determine whether to select the current video frame as the video frame for which a watermark needs to be embedded.

[0086] The second video frame sequence is obtained by sequentially filtering all video frames in the first video frame sequence.

[0087] Specifically, RGB channels are a color mode used to represent color images, representing three basic color channels: red, green, and blue. For the original first video frame sequence, each video frame contains R, G, and B channels.

[0088] When selecting the second video frame sequence to embed a watermark, the R, G, and B channels of any video frame in the first video frame sequence are processed separately. Two channels are selected to choose the video frames for which the watermark needs to be embedded. The two selected channels are denoted as the first channel and the second channel, respectively, and the unselected channel is denoted as the third channel. For example, if the first and second channels are the R and G channels, then the third channel is the B channel.

[0089] For the current video frame in the first video frame sequence, the first and second channels of the current video frame, as well as the first and second channels of the previous video frame, are used to determine whether to select the current video frame as the video frame for which a watermark needs to be embedded.

[0090] The first video frame sequence is filtered in a certain order (e.g., chronological order), and all video frames that need to be watermarked are arranged in a certain order (e.g., chronological order) to obtain the second video frame sequence that needs to be watermarked.

[0091] The video watermarking method provided in this application uses the RGB channel to filter the video frames that need to be embedded with watermarks during the video watermark embedding process. This can effectively protect the copyright and integrity of the video. During watermark extraction, the embedded watermark information can be extracted from the RGB image of the video frame, thereby providing a basic basis for video copyright authentication, etc.

[0092] In some embodiments, determining whether to select the current video frame as the video frame for which a watermark needs to be embedded includes:

[0093] Extract the first feature sequence of the first channel and the second feature sequence of the second channel of the current video frame; extract the first feature sequence of the first channel and the second feature sequence of the second channel of the previous video.

[0094] Determine the first correlation value between the first feature sequence of the first channel of the current video frame and the first feature sequence of the first channel of the previous video frame; determine the second correlation value between the second feature sequence of the second channel of the current video frame and the second feature sequence of the second channel of the previous video frame.

[0095] The first correlation value and the second correlation value are averaged, and the average value is compared with a preset threshold.

[0096] If the average value is less than the preset threshold, the current video frame is selected as the video frame for which a watermark needs to be embedded; or, if the average value is greater than or equal to the preset threshold, the current video frame is not selected as the video frame for which a watermark needs to be embedded.

[0097] Specifically, taking the selected first and second channels as R and G channels, and the unselected third channel as B channel as an example, based on the first and second channels of the current video frame in the first video frame sequence, and the first and second channels of the previous video frame, it is determined whether to select the current video frame as the video frame for which a watermark needs to be embedded. This can specifically include the following steps:

[0098] Step a: Extract the first feature sequence of the R channel and the second feature sequence of the G channel of the current video frame, and extract the first feature sequence of the R channel and the second feature sequence of the G channel of the previous video.

[0099] Specifically, when selecting video frames for which watermarks need to be embedded, this application focuses on ensuring that the selected video frames for which watermarks need to be embedded have a significant visual difference from the previous video frame in terms of timing. In this way, the rapid switching between the previous video frame and the selected current video frame during video playback can make the watermark more concealed and less likely to be detected.

[0100] RGB channel feature sequences refer to a sequence in which the pixel values ​​of each channel are arranged in a certain order. This can be achieved by preprocessing the channel image, such as adjusting brightness and contrast, and then performing spatial transformation on the channel image to convert the two-dimensional image into a one-dimensional feature sequence, for example, by connecting the pixel values ​​by row or column.

[0101] A chromaticity matrix is ​​a matrix used to represent color information in a video frame and can be used to extract RGB channel feature sequences. In video processing, color typically consists of two components: luminance and chrominance. Luminance represents the brightness of a video frame, while chrominance represents its color information. The chromaticity matrix constructs a numerical matrix for the chromaticity components of a video frame, where each element represents the relative intensity of a specific color component within the video frame.

[0102] In one implementation, for the current video frame in the first video frame sequence, the R-channel and G-channel chroma matrices are first divided into blocks with a block size of m1×n1. Then, the number of blocks is K=l1×k1, where l1=M / m1, k1=N / n1, and M and N represent the height and width of the chroma matrix (i.e., the video frame), respectively.

[0103] Then, the average chroma value of each block is calculated, and the blocks are arranged in order to obtain the first feature sequence R1 = [r] of the current video frame's R channel. 11 ,r 12 ,…,r 1K ] and the second feature sequence G1 = [g] of the current video frame G channel 11 ,g 12 ,…,g 1K ], with a length of K.

[0104] For the video frame preceding the current video frame in the second video frame sequence, perform the same operation to obtain the first feature sequence R0 = [r] of the previous video frame's R channel. 01 ,r 02 ,…,r 0K ] and the second feature sequence G0 = [g] of the G channel of the previous video frame 01 ,g 02 ,…,g 0K ], with a length of K.

[0105] Step b: Determine the first correlation value between the first feature sequence of the current video frame R channel and the first feature sequence of the previous video frame R channel, and determine the second correlation value between the second feature sequence of the current video frame G channel and the second feature sequence of the previous video frame G channel.

[0106] Specifically, the correlation value is calculated for the feature sequences of the current video frame and the previous video frame.

[0107] Taking the example of obtaining feature sequences based on the chromaticity matrix:

[0108] Calculate the first correlation value f between the first feature sequence of the R channel of the current video frame and the first feature sequence of the R channel of the previous video frame. r as follows:

[0109]

[0110] Calculate the second correlation value f between the second feature sequence of the G channel of the current video frame and the second feature sequence of the G channel of the previous video frame. g as follows:

[0111]

[0112] Step c: Take the average of the first correlation value and the second correlation value, and compare the average value with a preset threshold.

[0113] Specifically, after obtaining the correlation values, the average of the first and second correlation values ​​is calculated, and the average is compared with a preset threshold to determine whether to select the current video frame as the video frame for which watermarking needs to be embedded. Using the feature sequences of two channel components to determine the correlation value between the current video frame and the previous video frame can effectively avoid randomness compared to using the feature sequence of only one channel component.

[0114] Step d: If the average value is less than the preset threshold, select the current video frame as the video frame for which a watermark needs to be embedded; or, if the average value is greater than or equal to the preset threshold, do not select the current video frame as the video frame for which a watermark needs to be embedded.

[0115] Specifically, let the preset threshold be Th1, which satisfies:

[0116] when At that time, the current video frame is selected as the video frame for which the watermark needs to be embedded;

[0117] when At that time, the current video frame is not selected as the video frame for which the watermark needs to be embedded.

[0118] By performing steps a to d on each video frame in the first video frame sequence, all video frames that need to be embedded with watermarks can be filtered out and arranged in a certain order (e.g., chronological order) to obtain the second video frame sequence.

[0119] The video watermarking method provided in this application extracts the feature sequences of the current video frame and its previous video frame in the first video frame sequence, and calculates the correlation value of the feature sequences of the same channel components. Using the relationship between the average value of the correlation values ​​of the two channel components and a preset threshold, the current video frame whose average correlation value is less than the preset threshold is selected as the video frame to be embedded with watermark. This ensures that the selected video frame to be embedded with watermark has a large visual difference from its previous video frame in time sequence. In this way, the rapid switching between the previous video frame and the selected current video frame during video playback can make the watermark more concealed and less likely to be detected.

[0120] In some embodiments, embedding the first watermark sequence to be embedded into the second video frame sequence includes:

[0121] For each video frame in the second video frame sequence, watermark embedding is performed based on the chroma matrix of the third channel of each video frame.

[0122] The third channel is a channel other than the first channel and the second channel.

[0123] Specifically, in the aforementioned steps, the first and second channels are used for selecting video frames that need to be embedded with watermarks, while the unselected third channel is used in the watermark embedding step.

[0124] For each video frame in the second video frame sequence that needs to be embedded with a watermark, the watermark embedding in the first watermark sequence to be embedded is completed based on the third channel.

[0125] In some embodiments, the watermark embedding based on the third channel of each video frame includes:

[0126] The chroma matrix of the third channel is divided into blocks;

[0127] Calculate the mean chromaticity value of each block and arrange them in ascending or descending order to form the first chromaticity mean matrix;

[0128] Perform a discrete cosine transform on the first chromaticity mean matrix to obtain the first transform coefficient matrix;

[0129] Multiple coefficient pairs are selected from the first transformation coefficient matrix using the first key, and the watermark embedding is completed by modifying the coefficient values ​​of the coefficient pairs.

[0130] Specifically, taking the third channel as channel B as an example, for each video frame in the second video frame sequence, watermark embedding is performed based on the third channel, which may include:

[0131] Step a: Divide the chroma matrix of channel B into blocks.

[0132] Step b: Calculate the chromaticity mean of each block and form the first chromaticity mean matrix in ascending or descending order.

[0133] Specifically, the chroma matrix of the B channel is divided into blocks, and the block size is set to m2×n2. Then the number of blocks is l2×k2, where l2=M / m2, k2=N / n2, and M and N represent the height and width of the chroma matrix (i.e., video frame), respectively.

[0134] Calculate the mean chromaticity value of each block and arrange them in order (ascending or descending) to form a first chromaticity mean matrix of size l2×k2.

[0135] Step c: Perform a discrete cosine transform on the first chromaticity mean matrix to obtain the first transformation coefficient matrix with size l2×k2.

[0136] Step d: Select several coefficient pairs from the first transformation coefficient matrix using the first key, and embed a watermark by modifying the coefficient values ​​of the coefficient pairs.

[0137] Specifically, L pairs of coefficients are selected from the first transformation coefficient matrix using the first key key1, denoted as [(a 11 ,b 11 ),(a 12 ,b 12 ),…,(a 1L ,b 1L )], where a 1i and b 1i These are the coefficient values ​​corresponding to two positions symmetrical about the main diagonal in the first transformation coefficient matrix, i = 1, 2, ..., L, where L represents the length of the first watermark sequence to be embedded.

[0138] Secondly, watermark embedding of the second video frame sequence is achieved by modifying the coefficient pairs.

[0139] In some embodiments, watermark embedding is accomplished by modifying the coefficient values ​​of coefficient pairs, satisfying the following modification rules:

[0140] If w i =0:

[0141] If w i =1:

[0142] Among them, (a 1i ,b 1i ) represents a coefficient pair formed by the coefficient values ​​corresponding to the two positions on the main diagonal in the first transformation coefficient matrix, w iLet represent the value of the i-th position in the first watermark sequence to be embedded, where i = 1, 2, ..., L, L represents the length of the first watermark sequence to be embedded, and α represents the watermark embedding strength, which is a number greater than zero.

[0143] This completes the watermark embedding of one video frame in the second video frame sequence. It should be noted that there is a certain mapping relationship between the second video frame sequence and the first watermark sequence to be embedded. For example, if the number of video frames in the second video frame sequence is equal to the number of watermarks in the first watermark sequence, the nth watermark in the first watermark sequence is embedded into the nth video frame of the second video frame sequence. As another example, if the number of video frames in the second video frame sequence is m times the number of watermarks in the first watermark sequence, the nth watermark in the first watermark sequence is embedded into the (n-1)×m+1 to n×mth video frames of the second video frame sequence. This embodiment does not impose any limitations on this.

[0144] In some embodiments, the video watermarking method further includes:

[0145] The video from which the watermark is to be extracted is decoded to obtain a third video frame sequence;

[0146] Select the fourth video frame sequence from the third video frame sequence from which the watermark needs to be extracted;

[0147] The second watermark sequence is extracted from the fourth video frame sequence, and the third watermark sequence is obtained after decoding.

[0148] Specifically, in addition to video watermark embedding, the video watermark processing method provided in this application embodiment may also include video watermark extraction. Video watermark extraction corresponds to video watermark embedding and may specifically include:

[0149] Step a: Decode the video from which the watermark is to be extracted to obtain the third video frame sequence.

[0150] Specifically, if the video from which the watermark is to be extracted is the video with the embedded watermark obtained in the aforementioned steps and the video after it has been attacked, then the decoding process in step a is the same as the decoding process in step 102. This application mainly focuses on this situation. After decoding the video from which the watermark is to be extracted, a third video frame sequence is obtained.

[0151] Step b: Select the fourth video frame sequence from the third video frame sequence from which the watermark needs to be extracted.

[0152] Specifically, since watermarking is not embedded in every video frame of the original video, the third video frame sequence after decoding is filtered during the watermark extraction process to obtain the video frames from which the watermark needs to be extracted. After arranging them in a certain order (e.g., chronological order), the fourth video frame sequence from which the watermark needs to be extracted is obtained.

[0153] It should be noted that step b and the aforementioned step 103, which involves selecting the second video frame sequence from the first video frame sequence to embed the watermark, can be the same. However, to ensure the robustness of the watermark, the preset threshold (denoted as Th2) in the watermark extraction process can be slightly larger than the preset threshold (denoted as Th1) in step 103, to ensure that a certain proportion of the video frames in the second video frame sequence identified in step 103 are included in the fourth video frame sequence identified in this step.

[0154] Step c: Extract the second watermark sequence from each video frame in the fourth video frame sequence, and determine the third watermark sequence based on the second watermark sequence corresponding to each video frame in the fourth video frame sequence and perform decoding processing.

[0155] Specifically, for each video frame in the fourth video frame sequence, a second watermark sequence can be extracted. Optionally, the chroma matrix of the corresponding channel (third channel) in the aforementioned watermark embedding process can be used for watermark extraction. Multiple second watermark sequences are analyzed and processed to obtain a third watermark sequence, which is then decoded to obtain a watermark with the same type and format as the original watermark.

[0156] In some embodiments, extracting the second watermark sequence from each video frame in the fourth video frame sequence includes:

[0157] The chroma matrix of the third channel of each video frame in the fourth video frame sequence is divided into blocks, the mean of the chroma values ​​of each block is calculated, and the blocks are arranged in ascending or descending order to form a second chroma mean matrix.

[0158] Perform a discrete cosine transform on the second chromaticity mean matrix to obtain the second transform coefficient matrix;

[0159] Multiple coefficient pairs are selected from the second transformation coefficient matrix using the second key, and the second watermark sequence is extracted based on the coefficient values ​​of the coefficient pairs.

[0160] Specifically, the same method as in the aforementioned watermark embedding process is adopted, that is, the chroma matrix of the third channel of each video frame in the fourth video frame sequence is divided into blocks, the mean value of the chroma values ​​of each block is calculated, and the second chroma mean matrix of size l2×k2 is obtained by arranging them in ascending or descending order.

[0161] Then, a discrete cosine transform is performed on the second chromaticity mean matrix to obtain the second transformation coefficient matrix with size l2×k2.

[0162] Furthermore, a second key is used to select several coefficient pairs from the second transformation coefficient matrix, and the second watermark sequence is extracted based on the coefficient values ​​of the multiple coefficient pairs.

[0163] Specifically, L pairs of coefficients are selected from the second transformation coefficient matrix using the key key2, denoted as [(a 21 ,b 21 ),(a 22 ,b 22 ),…,(a 2L ,b 2L )], where a 2i and b 2i These are the coefficient values ​​corresponding to two positions symmetrical about the main diagonal in the second transformation coefficient matrix, i = 1, 2, ..., L, where L represents the length of the second watermark sequence, which is the same as the length of the first watermark sequence to be embedded in step 101.

[0164] Next, the second watermark sequence is extracted based on the coefficient values ​​of multiple coefficient pairs. This completes the extraction of the second watermark sequence for one video frame in the fourth video frame sequence. Repeating the above operation for each video frame in the fourth video frame sequence yields multiple second watermark sequences.

[0165] In some embodiments, the extraction of the second watermark sequence based on the coefficient values ​​of coefficient pairs satisfies the following rules:

[0166]

[0167] Among them, (a 2i ,b 2i () represents a coefficient pair in the second transformation coefficient matrix, consisting of the coefficient values ​​corresponding to the two positions on the main diagonal. This represents the value of the i-th element in the extracted second watermark sequence, where i = 1, 2, ..., L, and L represents the length of the second watermark sequence.

[0168] In some embodiments, determining the third watermark sequence based on the second watermark sequence corresponding to each video frame in the fourth video frame sequence includes:

[0169] Based on the third correlation value between the second watermark sequences corresponding to any two video frames in the fourth video frame sequence, a correlation matrix is ​​determined.

[0170] The elements in each row vector of the relevance matrix are summed to determine the third watermark sequence.

[0171] Specifically, the multiple second watermark sequences extracted through the aforementioned steps may differ, therefore further analysis is needed to select one as the final third watermark sequence. This may include:

[0172] Step a: Determine the correlation matrix based on the third correlation value between the second watermark sequences corresponding to any two video frames in the fourth video frame sequence.

[0173] Specifically, assume that the aforementioned steps extracted P second watermark sequences, denoted as {W1, W2, ..., W...} P}, where W i =[w i1 ,w i2 ,…,w iL ], w ij ∈{0,1}, i=1,2,…,L, where L represents the length of the extracted second watermark sequence.

[0174] Calculate the third correlation value between any two second watermark sequences, satisfying:

[0175]

[0176] Among them, w ik w represents the value at the k-th position in the i-th extracted second watermark sequence. jk It is the value at the k-th position in the j-th extracted second watermark sequence, c ij Let represent the third correlation value between the i-th extracted second watermark sequence and the j-th extracted second watermark sequence, k = 1, 2, ..., L, i, j = 1, 2, ..., P.

[0177] Based on the pairwise correlation values ​​between the P second watermark sequences, the correlation matrix C is obtained as follows:

[0178]

[0179] Among them, c ij Let represent the third correlation value between the i-th extracted second watermark sequence and the j-th extracted second watermark sequence, k = 1, 2, ..., L, i, j = 1, 2, ..., P.

[0180] Step b: Summate the elements in each row vector of the relevance matrix, and determine the third watermark sequence based on the maximum sum.

[0181] Specifically, the elements (correlation values) in each row vector of the correlation matrix C are summed to obtain P sums, denoted as [S1, S2, ..., S]. P ],in, c ikIt is the value in the i-th row and k-th column of the correlation matrix C, where i = 1, 2, ..., P.

[0182] Determine [S1,S2,…S] P The position q of the maximum sum in S is... q ≥S i If q∈[1,P], i=1,2,…,P and i≠q, then the second watermark sequence S is determined. q This is the final third watermark sequence.

[0183] The video watermarking method provided in this application involves repeatedly embedding the watermark in several watermark embedding frames, repeatedly extracting it in several watermark extraction frames, and statistically analyzing it. This significantly enhances the robustness of the algorithm against desynchronization attacks (such as frame insertion, frame deletion, frame swapping, video editing, etc.), and the watermark is basically unaffected by such desynchronization attacks. At the same time, the watermark is embedded and extracted based on the energy distribution of image blocks, ensuring the robustness of the watermark against attacks such as resolution changes, local cropping, and desynchronization.

[0184] In summary, through the implementation of the video watermarking processing method provided in this application, the performance of video watermarking has been greatly improved in terms of invisibility, robustness against geometric attacks and desynchronization attacks.

[0185] The video watermarking apparatus provided in this application is described below. The video watermarking apparatus described below can be referred to in correspondence with the video watermarking method described above.

[0186] Figure 2 This is a schematic diagram of the watermark processing device provided in the embodiments of this application, as shown below. Figure 2 As shown, the device includes at least:

[0187] The first encoding module 201 is used to encode the original watermark and obtain the first watermark sequence to be embedded.

[0188] The first decoding module 202 is used to decode the original video and obtain the first video frame sequence;

[0189] The first filtering module 203 is used to filter out the second video frame sequence that needs to be embedded with watermark from the first video frame sequence;

[0190] The embedding module 204 is used to embed the first watermark sequence to be embedded into the second video frame sequence, and obtain the video after encoding processing.

[0191] In some embodiments, any video frame in the first video frame sequence includes an R channel, a G channel, and a B channel, and the first filtering module 203 is specifically used for:

[0192] For any video frame in the first video frame sequence, two channels are selected from the R channel, the G channel, and the B channel as the first channel and the second channel, respectively;

[0193] Based on the first and second channels of the current video frame in the first video frame sequence, and the first and second channels of the previous video frame, determine whether to select the current video frame as the video frame for which a watermark needs to be embedded.

[0194] The second video frame sequence is obtained by sequentially filtering all video frames in the first video frame sequence.

[0195] In some embodiments, the first filtering module 203 is specifically used for:

[0196] Extract the first feature sequence of the first channel and the second feature sequence of the second channel of the current video frame; extract the first feature sequence of the first channel and the second feature sequence of the second channel of the previous video.

[0197] Determine the first correlation value between the first feature sequence of the first channel of the current video frame and the first feature sequence of the first channel of the previous video frame; determine the second correlation value between the second feature sequence of the second channel of the current video frame and the second feature sequence of the second channel of the previous video frame.

[0198] The first correlation value and the second correlation value are averaged, and the average value is compared with a preset threshold.

[0199] If the average value is less than the preset threshold, the current video frame is selected as the video frame for which a watermark needs to be embedded; or, if the average value is greater than or equal to the preset threshold, the current video frame is not selected as the video frame for which a watermark needs to be embedded.

[0200] In some embodiments, the embedding module 204 is specifically used for:

[0201] For each video frame in the second video frame sequence, watermark embedding is performed based on the third channel of each video frame;

[0202] The third channel is a channel other than the first channel and the second channel.

[0203] In some embodiments, the embedding module 204 is specifically used for:

[0204] The chroma matrix of the third channel is divided into blocks;

[0205] Calculate the mean chromaticity value of each block and arrange them in ascending or descending order to form the first chromaticity mean matrix;

[0206] Perform a discrete cosine transform on the first chromaticity mean matrix to obtain the first transform coefficient matrix;

[0207] Multiple coefficient pairs are selected from the first transformation coefficient matrix using the first key, and the watermark embedding is completed by modifying the coefficient values ​​of the coefficient pairs.

[0208] In some embodiments, the watermark embedding is accomplished by modifying the coefficient values ​​of the coefficient pairs, satisfying the following modification principles:

[0209] If w i =0:

[0210] If w i =1:

[0211] Among them, (a 1i ,b 1i ) represents a coefficient pair formed by the coefficient values ​​corresponding to the two positions on the main diagonal in the first transformation coefficient matrix, w i Let α represent the value of the i-th position in the first watermark sequence to be embedded, where i = 1, 2, ..., L, L represents the length of the first watermark sequence to be embedded, and α represents the watermark embedding strength.

[0212] In some embodiments, the device further includes:

[0213] The second decoding module is used to decode the video from which the watermark is to be extracted, and obtain the third video frame sequence.

[0214] The first extraction module is used to filter out the fourth video frame sequence from the third video frame sequence from which the watermark needs to be extracted;

[0215] The second extraction module is used to extract a second watermark sequence from each video frame in the fourth video frame sequence, and to determine a third watermark sequence and perform decoding processing based on the second watermark sequence corresponding to each video frame in the fourth video frame sequence.

[0216] In some embodiments, the second extraction module is specifically used for:

[0217] The chroma matrix of the third channel of each video frame in the fourth video frame sequence is divided into blocks, the mean of the chroma values ​​of each block is calculated, and the blocks are arranged in ascending or descending order to form a second chroma mean matrix.

[0218] Perform a discrete cosine transform on the second chromaticity mean matrix to obtain the second transform coefficient matrix;

[0219] Multiple coefficient pairs are selected from the second transformation coefficient matrix using the second key, and the second watermark sequence is extracted based on the coefficient values ​​of the coefficient pairs.

[0220] In some embodiments, the extraction of the second watermark sequence based on the coefficient values ​​of coefficient pairs satisfies the following rules:

[0221]

[0222] Among them, (a 2i ,b 2i () represents a coefficient pair in the second transformation coefficient matrix, consisting of the coefficient values ​​corresponding to the two positions on the main diagonal. This represents the value of the i-th element in the extracted second watermark sequence, where i = 1, 2, ..., L, and L represents the length of the second watermark sequence.

[0223] In some embodiments, the second extraction module is specifically used for:

[0224] Based on the third correlation value between the second watermark sequences corresponding to any two video frames in the fourth video frame sequence, a correlation matrix is ​​determined.

[0225] The elements in each row vector of the relevance matrix are summed, and the third watermark sequence is determined based on the maximum sum.

[0226] The methods and apparatuses provided in the various embodiments of this application are based on the same concept. Since the methods and apparatuses solve problems in similar ways, the implementations of the apparatuses and methods can refer to each other, and repeated details will not be repeated.

[0227] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 3 As shown, the electronic device may include: a processor 301, a communications interface 302, a memory 303, and a communication bus 304. The processor 301, communications interface 302, and memory 303 communicate with each other via the communication bus 304. The processor 301 can call logical instructions stored in the memory 303 to execute a video watermarking method, such as:

[0228] The original watermark is encoded to obtain the first watermark sequence to be embedded;

[0229] The original video is decoded to obtain the first video frame sequence;

[0230] Select the second video frame sequence from the first video frame sequence to embed the watermark;

[0231] The first watermark sequence to be embedded is embedded into the second video frame sequence, and the video with the embedded watermark is obtained after encoding processing.

[0232] Furthermore, the logical instructions in the aforementioned memory 303 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0233] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to perform the video watermarking methods provided by the above methods, such as including:

[0234] The original watermark is encoded to obtain the first watermark sequence to be embedded;

[0235] The original video is decoded to obtain the first video frame sequence;

[0236] Select the second video frame sequence from the first video frame sequence to embed the watermark;

[0237] The first watermark sequence to be embedded is embedded into the second video frame sequence, and the video with the embedded watermark is obtained after encoding processing.

[0238] Furthermore, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the video watermarking methods provided by the methods described above, including, for example:

[0239] The original watermark is encoded to obtain the first watermark sequence to be embedded;

[0240] The original video is decoded to obtain the first video frame sequence;

[0241] Select the second video frame sequence from the first video frame sequence to embed the watermark;

[0242] The first watermark sequence to be embedded is embedded into the second video frame sequence, and the video with the embedded watermark is obtained after encoding processing.

[0243] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0244] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0245] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method of video watermarking processing, characterized by, include: The original watermark is encoded to obtain the first watermark sequence to be embedded; The original video is decoded to obtain the first video frame sequence; Select the second video frame sequence from the first video frame sequence to embed the watermark; The first watermark sequence to be embedded is embedded into the second video frame sequence, and the video with the watermark embedded is obtained after encoding processing. Wherein, any video frame in the first video frame sequence includes an R channel, a G channel, and a B channel, and the step of selecting the second video frame sequence to which the watermark needs to be embedded from the first video frame sequence includes: For any video frame in the first video frame sequence, two channels are selected from the R channel, the G channel, and the B channel as the first channel and the second channel, respectively; Based on the first and second channels of the current video frame in the first video frame sequence, and the first and second channels of the previous video frame, determine whether to select the current video frame as the video frame for which a watermark needs to be embedded. The second video frame sequence is obtained by sequentially filtering all video frames in the first video frame sequence. The step of embedding the first watermark sequence to be embedded into the second video frame sequence includes: For each video frame in the second video frame sequence, watermark embedding is performed based on the third channel of each video frame, including: dividing the chroma matrix of the third channel into blocks; calculating the mean of the chroma values ​​of each block and arranging them in ascending or descending order to form a first chroma mean matrix; performing a discrete cosine transform on the first chroma mean matrix to obtain a first transform coefficient matrix; selecting multiple coefficient pairs from the first transform coefficient matrix using a first key, and completing the watermark embedding by modifying the coefficient values ​​of the coefficient pairs; The third channel is a channel other than the first channel and the second channel.

2. The video watermarking method of claim 1, wherein, The step of determining whether to select the current video frame as the video frame for which a watermark needs to be embedded includes: Extract the first feature sequence of the first channel and the second feature sequence of the second channel of the current video frame; extract the first feature sequence of the first channel and the second feature sequence of the second channel of the previous video. Determine the first correlation value between the first feature sequence of the first channel of the current video frame and the first feature sequence of the first channel of the previous video frame; determine the second correlation value between the second feature sequence of the second channel of the current video frame and the second feature sequence of the second channel of the previous video frame. The first correlation value and the second correlation value are averaged, and the average value is compared with a preset threshold. If the average value is less than the preset threshold, the current video frame is selected as the video frame for which a watermark needs to be embedded; or, if the average value is greater than or equal to the preset threshold, the current video frame is not selected as the video frame for which a watermark needs to be embedded.

3. The video watermarking method of claim 1, wherein, The watermark embedding is accomplished by modifying the coefficient values ​​of the coefficient pairs, satisfying the following modification principles: ； ； wherein, denotes a coefficient pair of the first transform coefficient matrix corresponding to the two positions with respect to the main diagonal, denotes a value of a bit of the first watermark sequence to be embedded, denotes a value of a bit of the first watermark sequence to be embedded, , denotes a length of the first watermark sequence to be embedded, denotes a watermark embedding strength.

4. The video watermarking method of claim 1, wherein, The method further includes: The video from which the watermark is to be extracted is decoded to obtain a third video frame sequence; Select the fourth video frame sequence from the third video frame sequence from which the watermark needs to be extracted; Extract the second watermark sequence from each video frame in the fourth video frame sequence, and determine the third watermark sequence based on the second watermark sequence corresponding to each video frame in the fourth video frame sequence and perform decoding processing.

5. The video watermarking method according to claim 4, characterized in that, Extracting the second watermark sequence from each video frame in the fourth video frame sequence includes: The chroma matrix of the third channel of each video frame in the fourth video frame sequence is divided into blocks, the mean of the chroma values ​​of each block is calculated, and the blocks are arranged in ascending or descending order to form a second chroma mean matrix. Perform a discrete cosine transform on the second chromaticity mean matrix to obtain the second transform coefficient matrix; Multiple coefficient pairs are selected from the second transformation coefficient matrix using the second key, and the second watermark sequence is extracted based on the coefficient values ​​of the coefficient pairs.

6. The video watermarking method according to claim 5, characterized in that, The extraction of the second watermark sequence based on the coefficient values ​​of coefficient pairs satisfies the following rules: ； in, This represents a coefficient pair in the second transformation coefficient matrix, consisting of the coefficient values ​​corresponding to the two positions on the main diagonal. This indicates the extracted second watermark sequence. The value of the bit. , This indicates the length of the second watermark sequence.

7. The video watermarking method according to claim 5, characterized in that, Determining the third watermark sequence based on the second watermark sequence corresponding to each video frame in the fourth video frame sequence includes: Based on the third correlation value between the second watermark sequences corresponding to any two video frames in the fourth video frame sequence, a correlation matrix is ​​determined. The elements in each row vector of the relevance matrix are summed, and the third watermark sequence is determined based on the maximum sum.

Images