Video Image Watermark Embedding and Extraction Method Based on Deep Learning

Abstract

The present invention discloses a method for embedding and extracting video image watermarks based on deep learning, which relates to the technical field of video image watermark processing. It includes establishing a video database based on the original video before editing, performing feature recognition on multiple video blocks in the original video, selecting two target video blocks, determining the watermark content based on the similarity of the recognition content of the two target video blocks, extracting the watermark in the original video, integrating to obtain the original watermark data, and training through a deep learning model to obtain a target watermark fitting model. According to the watermark requirement data in the finished video, target watermark data is output, the embedding bitstream sequence of the target watermark data is obtained, and the finished video is embedded to obtain the finished video with the embedded watermark. The present invention is applicable to the watermark extraction and embedding process of new videos synthesized by multi-video editing, and there is a correlation between the watermarks in the new video and the watermarks in the original video, providing a digital watermark embedding and extraction method that can trace the source.

Description

Technical Field

[0001] The present invention relates to the technical field of video image watermark processing, and particularly to a method for embedding and extracting video image watermarks based on deep learning. Background Art

[0002] Digital watermark technology embeds watermark information into carriers such as documents, images, audio, and videos without affecting the perception of the carrier, and it is not easily discovered or tampered with. The embedder of the watermark information can use a set algorithm key to extract the watermark, and evaluate the difference between the extracted watermark information and the original watermark information through corresponding evaluation criteria, so as to determine whether the data containing the watermark information has been attacked or tampered with. As a kind of digital watermark technology, video watermark technology is an important means for protecting video products such as video copyright protection, content authentication, broadcast monitoring, and copy control. When synthesizing and editing multiple video clips, in order to ensure copyright ownership and prevent piracy, watermarks will be embedded in the synthesized video. Before synthesis or editing, it is also necessary to extract the watermarks in the video to ensure that the synthesized content does not contain the watermarks or logos of the original video.

[0003] Existing digital watermark generation algorithms are divided into two types: compressed domain watermark algorithms and original domain watermark algorithms. Among them, the compressed domain algorithms mainly include the discrete cosine transform coefficient steganography algorithm and the motion vector steganography algorithm, which compensate the steganography effect through methods of motion compensation and energy analysis, and have high execution efficiency. However, since there is a corresponding watermark steganography algorithm for each video format, the generality of the algorithm is poor; and because the time for embedding the watermark is after video encoding, the space for embedding the watermark is small, resulting in poor robustness of the watermark.

[0004] In view of the above technical defects, a solution is proposed. Summary of the Invention

[0005] The purpose of the present invention is to be applicable to the process of watermark extraction and embedding for a new video synthesized by multi-video editing, and there is a correlation between the watermark in the new video and the watermark in the original video, and a method for digital watermark embedding and extraction that can trace the source is provided.

[0006] In order to achieve the above purpose, the present invention adopts the following technical solution: A method for embedding and extracting video image watermarks based on deep learning, including the following steps:

[0007] Step 1: Obtain the original video before editing, establish a video database, decompose the video frames of the original video into n×n video blocks, perform feature recognition on multiple video blocks in the original video, and obtain the feature recognition results of each video block.

[0008] Step 2: Select two target video blocks from the multiple video blocks with an interval greater than the preset number of frames, determine the watermark content based on the similarity of the recognition contents of the two target video blocks, and extract the watermark in the original video based on the watermark content and the position of each recognition content in each feature recognition result in the corresponding video block, and integrate to obtain the original watermark data;

[0009] Step 3: Integrate the original watermark data corresponding to all the original videos in the video database into a dataset, train it through a deep learning model to obtain a target watermark fitting model, obtain the clipped finished video, and then obtain the watermark requirement data in the finished video. The watermark requirement data includes the target watermark content and the position of the target watermark content in the finished video, and substitute it into the target watermark fitting model to output the target watermark data;

[0010] Step 4: Obtain the embedding bitstream sequence of the target watermark data, perform macroblock segmentation processing on the finished video to obtain the discrete cosine transform coefficients of each macroblock, divide the resolution size of the video into multiple ranges, and select different frame processing methods according to different application scenarios and the video resolution size to obtain the embedding frames;

[0011] Step 5: Embed the finished video according to the discrete cosine transform coefficients and the embedding bitstream sequence, re-encode and save the finished video with the target embedded watermark data to obtain the finished video with the embedded watermark.

[0012] Further, the specific process of obtaining the feature recognition result of each video block is as follows:

[0013] S101: Obtain each video block in the original video, calculate the sum of the pixel values of each video block, establish a time horizontal axis according to the sequence of video frames, and establish a vertical axis with the numerical values of the pixel values to obtain a plane coordinate system. Mark the sum of the pixel values one by one in the plane coordinate system and connect them with a smooth curve to obtain a pixel value fluctuation curve;

[0014] S102: Segment the pixel value fluctuation curve, calculate the slope Ki of each segmented curve. According to the preset slope threshold Kmin, if the slope Ki is greater than the slope threshold Kmin, then obtain the sum of the pixels of two adjacent video blocks on this segmented curve as the recognition object;

[0015] S103: If the sum of the pixel values of the first video block in the recognition object is greater than the sum of the pixel values of the second video block, then extract bit 0, otherwise extract bit 1;

[0016] S104: Repeat the above step to obtain an n×n sequence and generate a pseudo-random bit sequence s with a length of n;

[0017] S105. Perform a bit-by-bit logical exclusive OR operation on the n-by-n sequence using a pseudo-random bit sequence s to obtain the feature recognition results. Each feature recognition result includes the recognition content of each frame of the video block and the position of each recognition content in the corresponding video block.

[0018] Furthermore, the specific process of integrating the original watermark data is as follows:

[0019] S201 , selecting two target video blocks with an interval greater than a preset number of frames from the multiple video blocks, obtaining the sums S1 and S2 of pixel values corresponding to the target video blocks, and calculating a pixel value difference ΔS=|S1−S2|;

[0020] S202: Obtain a preset difference judgment threshold Smin. If the pixel value difference ΔS is less than or equal to the difference judgment threshold Smin, the two target video blocks are selected as analysis objects.

[0021] S203: Obtain the corresponding feature recognition results for each analysis object, define an analysis area based on the position of each recognized content in the feature recognition results in the corresponding video block, and perform similarity calculation on the analysis area to obtain a similarity comparison value. The calculation formula is as follows:

[0022] ,in and is the image mean corresponding to the two analysis areas, and is the variance, is the covariance, C1 and C2 are preset constants;

[0023] S204. Obtain a preset similarity evaluation value SSIMi. If the similarity comparison value is greater than or equal to the similarity evaluation value SSIMi, use the analysis area in the analysis object as the watermark content, extract the watermark content and the position of the watermark content in the corresponding video block, and integrate them to obtain the original watermark data.

[0024] Furthermore, the specific process of obtaining the target watermark fitting model is as follows:

[0025] S301. Obtain original watermark data corresponding to all original videos in a video database, the original watermark data including the watermark content and the location of the watermark content in the corresponding video block; obtain the video block corresponding to the original watermark data; crop the video block to obtain video images; integrate them into a data set; and generate corresponding label text as training samples;

[0026] S302. Divide the training samples into a training set and a test set according to a ratio of 8:2. After initializing the training parameters, start training the model. Use a network structure with full convolution and cross-layer skip connections similar to a residual network to extract features from the video images in the training set, and obtain a feature map of a certain size.

[0027] S303. Divide the video images into grids, use a multi-label multi-classification logistic regression layer for class prediction, and at the same time use a multi-scale fusion method in the model for border position prediction, and perform detection on feature maps of multiple scales.

[0028] S304. Load the initialized training parameters, perform model training on a GPU server, use the test set to detect the effect of the trained model, and save the optimal model weight parameters to obtain the target watermark fitting model.

[0029] Furthermore, the specific process of obtaining the embedding bitstream sequence of the target watermark data is as follows:

[0030] S401. Binarize the target watermark data to obtain the binary bitstream sequence of the target watermark data.

[0031] S402. Perform convolutional code encoding according to the binary bitstream sequence to obtain the embedding bitstream sequence of the target watermark data.

[0032] Furthermore, the specific process of obtaining the finished video with the embedded watermark is as follows:

[0033] S501. Obtain the discrete cosine transform coefficients of each macroblock. According to the discrete cosine transform coefficients of each macroblock, obtain a coefficient matrix, and perform a zigzag scan on the coefficient matrix to obtain a scan matrix.

[0034] S502. Obtain the embedding bitstream sequence of the target watermark data, and determine whether the scan matrix needs to replace data. Re-encode and save the finished video of the target embedded watermark data to obtain the finished video with the embedded watermark.

[0035] In summary, due to the adoption of the above technical solutions, the beneficial effects of the present invention are:

[0036] The video image watermark embedding and extraction method based on deep learning establishes a video database according to the original video before editing, performs feature recognition on multiple video blocks in the original video, selects two target video blocks, determines the watermark content based on the similarity of the recognition content of the two target video blocks, integrates to obtain the original watermark data, and trains through a deep learning model to obtain a target watermark fitting model. According to the watermark requirement data in the finished video, target watermark data is output, the embedding code stream sequence of the target watermark data is obtained, and the finished video is embedded to obtain the finished video with the embedded watermark. The present invention is applicable to the watermark extraction and embedding process of new videos synthesized by multi-video editing, and there is a correlation between the watermarks in the new video and the watermarks in the original video, providing a digital watermark embedding and extraction method that can trace the source. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 FIG. shows a schematic flowchart of the video image watermark extraction method of the present invention;

[0038] Figure 2 FIG. shows a schematic flowchart of the video image watermark embedding method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0040] Embodiment

[0041] As Figure 1-2 shown, the video image watermark embedding and extraction method based on deep learning includes the following steps:

[0042] Step 1: Obtain the original video before editing, establish a video database, decompose the video frames of the original video into n×n video blocks, perform feature recognition on multiple video blocks in the original video, and obtain the feature recognition results of each video block;

[0043] The specific process of obtaining the feature recognition results of each video block is as follows:

[0044] S101: Obtain each video block in the original video, calculate the sum of pixel values of each video block, establish a time horizontal axis according to the sequence of video frames, and establish a vertical axis with the numerical values of pixel values to obtain a plane coordinate system. Mark the sum of pixel values one by one in the plane coordinate system and connect them with a smooth curve to obtain a pixel value fluctuation curve;

[0045] S102. Segment the pixel value fluctuation curve, calculate the slope Ki of each segmented curve. According to the preset slope threshold Kmin, if the slope Ki is greater than the slope threshold Kmin, then obtain the sum of the pixels of two adjacent video blocks on this segmented curve as the recognition object;

[0046] S103. If the sum of the pixel values of the first video block in the recognition object is greater than the sum of the pixel values of the second video block, then the extracted bit is 0, otherwise the extracted bit is 1;

[0047] S104. Repeat the above step to obtain an n×n sequence, and generate a pseudo-random bit sequence s with a length of n;

[0048] S105. Perform a bitwise logical exclusive OR operation on the n×n sequence using the pseudo-random bit sequence s to obtain the feature recognition result. Each feature recognition result includes the recognition content of each video block and the position of each recognition content in the corresponding video block.

[0049] Step 2. Select two target video blocks with an interval greater than the preset number of frames from multiple video blocks, and determine the watermark content based on the similarity of the recognition contents of the two target video blocks. Based on the watermark content and the position of each recognition content in the corresponding video block in each feature recognition result, integrate to obtain the original watermark data;

[0050] The specific process of integrating to obtain the original watermark data is as follows:

[0051] S201. Select two target video blocks with an interval greater than the preset number of frames from multiple video blocks, and obtain the sum of the corresponding pixel values S1 and S2 of the target video blocks, and calculate the pixel value difference ΔS = |S1 - S2|;

[0052] S202. Obtain the preset difference judgment threshold Smin. If the pixel value difference ΔS is less than or equal to the difference judgment threshold Smin, then the two target video blocks are selected as the analysis objects;

[0053] S203. Obtain the corresponding feature recognition results in each analysis object, delimit the analysis area according to the position of each recognition content in the corresponding video block in the feature recognition result, and calculate the similarity comparison value for the analysis area. The calculation formula is as follows:

[0054] , where and are the image means corresponding to the two analysis areas, and are the variances, is the covariance, and C1 and C2 are preset constants;

[0055] S204. Obtain the preset similarity evaluation value SSIMi. If the similarity comparison value is greater than or equal to the similarity evaluation value SSIMi, then take the analysis area in the analysis object as the watermark content, extract the watermark content and the position of the watermark content in the corresponding video block, and integrate to obtain the original watermark data.

[0056] Step 3. Integrate the original watermark data corresponding to all the original videos in the video database into a data set, train it through a deep learning model to obtain a target watermark fitting model, obtain the clipped finished video, and then obtain the watermark requirement data in the finished video. The watermark requirement data includes the target watermark content and the position of the target watermark content in the finished video, and substitute it into the target watermark fitting model to output the target watermark data;

[0057] The specific process of obtaining the target watermark fitting model is as follows:

[0058] S301. Obtain the original watermark data corresponding to all the original videos in the video database. The original watermark data includes the watermark content and the position of the watermark content in the corresponding video block. Obtain the video block corresponding to the source of the original watermark data, crop the video block to obtain a video image, integrate it into a data set, and generate a corresponding label text as a training sample;

[0059] S302. Divide the training samples into a training set and a test set according to a ratio of 8:2. After initializing the training parameters, start training the model. Use a network structure with full convolution and cross-layer jump links similar to a residual network to extract features from the video images in the training set to obtain a feature map of a certain size;

[0060] S303. Divide the video image into grids, use a multi-label multi-classification logistic regression layer for class prediction, and at the same time use a multi-scale fusion method in the model for border position prediction, and perform detection on feature maps of multiple scales;

[0061] S304. Load the initialized training parameters, perform model training on the GPU server, use the test set to detect the effect of the trained model, and save the optimal model weight parameters to obtain the target watermark fitting model.

[0062] Step 4. Obtain the embedding bitstream sequence of the target watermark data, perform macroblock segmentation processing on the finished video to obtain the discrete cosine transform coefficients of each macroblock, divide the video resolution size into multiple ranges, and select different frame processing methods according to different application scenarios and the video resolution size to obtain the embedding frames;

[0063] The specific process of obtaining the embedding bitstream sequence of the target watermark data is as follows:

[0064] S401. Binarize the target watermark data to obtain a binary code stream sequence of the target watermark data;

[0065] S402. Perform convolutional code encoding according to the binary code stream sequence to obtain an embedded code stream sequence of the target watermark data.

[0066] Step Five. According to the discrete cosine transform coefficients and the embedded code stream sequence, embed into the finished video, re-encode and save the finished video with the target embedded watermark data to obtain the finished video with the embedded watermark.

[0067] The specific process of obtaining the finished video with the embedded watermark is as follows:

[0068] S501. Obtain the discrete cosine transform coefficients of each macroblock, obtain a coefficient matrix according to the discrete cosine transform coefficients of each macroblock, and perform zigzag scanning on the coefficient matrix to obtain a scanning matrix;

[0069] S502. Obtain the embedded code stream sequence of the target watermark data, and determine whether the scanning matrix needs to replace data, re-encode and save the finished video with the target embedded watermark data to obtain the finished video with the embedded watermark.

[0070] The present invention establishes a video database based on the original video before editing, performs feature recognition on multiple video blocks in the original video, selects two target video blocks, determines the watermark content based on the similarity of the recognition content of the two target video blocks, integrates to obtain the original watermark data, and trains through a deep learning model to obtain a target watermark fitting model, outputs the target watermark data according to the watermark demand data in the finished video, obtains the embedded code stream sequence of the target watermark data, and embeds into the finished video to obtain the finished video with the embedded watermark. The present invention is applicable to the watermark extraction and embedding process of a new video synthesized by multi-video editing, and there is a correlation between the watermark in the new video and the watermark in the original video, providing a digital watermark embedding and extraction method that can trace the source.

[0071] The setting of the size of the threshold is for the convenience of comparison. Regarding the size of the threshold, it depends on the amount of sample data and the number of base numbers set by those skilled in the art for each group of sample data; as long as it does not affect the proportional relationship between the parameters and the quantized values.

[0072] The above formulas are all dimensionless and take their numerical calculations. The formula is obtained by collecting a large amount of data and performing software simulation to obtain a formula closest to the actual situation. The preset parameters in the formula are set by those skilled in the art according to the actual situation;

[0073] As described above, it is only the preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, should be covered within the protection scope of the present invention.

Claims

1. A method for embedding and extracting video image watermarks based on deep learning, characterized in that, It includes the following steps: Step 1: Obtain the original video before editing, establish a video database, decompose the video frames of the original video into n×n video blocks, perform feature recognition on multiple video blocks in the original video, and obtain the feature recognition results of each video block; The specific process of obtaining the feature recognition results of each video block is as follows: S101: Obtain each video block in the original video, calculate the sum of pixel values of each video block, establish a time horizontal axis according to the sequence of video frames, and establish a vertical axis with the numerical values of pixel values to obtain a plane coordinate system. Mark the sum of pixel values one by one in the plane coordinate system and connect them with a smooth curve to obtain a pixel value fluctuation curve; S102: Segment the pixel value fluctuation curve, calculate the slope Ki of each segmented curve. According to the preset slope threshold Kmin, if the slope Ki is greater than the slope threshold Kmin, obtain the sum of pixel values of two adjacent video blocks on this segmented curve as the recognition object; S103: If the sum of pixel values of the first video block in the recognition object is greater than the sum of pixel values of the second video block, the extracted bit is 0, otherwise the extracted bit is 1; S104: Repeat the above step to obtain an n×n sequence and generate a pseudo-random bit sequence s with a length of n; S105: Perform a bitwise logical exclusive OR operation on the n×n sequence using the pseudo-random bit sequence s to obtain the feature recognition result. Each feature recognition result includes the recognition content of each video block in each frame and the position of each recognition content in the corresponding video block; Step 2: Select two target video blocks with an interval greater than the preset number of frames from the multiple video blocks, determine the watermark content based on the similarity of the recognition contents of the two target video blocks, and integrate to obtain the original watermark data based on the watermark content and the position of each recognition content in the corresponding video block in each feature recognition result; The specific process of integrating to obtain the original watermark data is as follows: S201: Select two target video blocks with an interval greater than the preset number of frames from the multiple video blocks, obtain the sum of pixel values S1 and S2 corresponding to the target video blocks, and calculate the pixel value difference ΔS = |S1 - S2|; S202: Obtain the preset difference judgment threshold Smin. If the pixel value difference ΔS is less than or equal to the difference judgment threshold Smin, the two target video blocks are selected as the analysis objects; S203: Obtain the corresponding feature recognition results in each analysis object, delimit the analysis area according to the position of each recognition content in the corresponding video block in the feature recognition result, and calculate the similarity comparison value for the analysis area. The calculation formula is as follows: , where and are the image means corresponding to two analysis regions, and are variances, is the covariance, and C1 and C2 are preset constants; S204: Obtain the preset similarity evaluation value SSIMi. If the similarity comparison value is greater than or equal to the similarity evaluation value SSIMi, use the analysis area in the analysis object as the watermark content, extract the watermark content and the position of the watermark content in the corresponding video block, and integrate to obtain the original watermark data; Step 3: Integrate the original watermark data corresponding to all the original videos in the video database into a dataset, train it through a deep learning model to obtain a target watermark fitting model, obtain the edited finished video, and then obtain the watermark requirement data in the finished video. The watermark requirement data includes the target watermark content and the position of the target watermark content in the finished video, and substitute it into the target watermark fitting model to output the target watermark data; Step 4: Obtain the embedding bitstream sequence of the target watermark data, perform macroblock segmentation processing on the finished video to obtain the discrete cosine transform coefficients of each macroblock, divide the video resolution size into multiple ranges, and select different frame processing methods according to different application scenarios and video resolution sizes to obtain the embedding frames; Step 5: According to the discrete cosine transform coefficients and the embedding bitstream sequence, embed the finished video, re-encode and save the finished video with the target embedded watermark data to obtain the finished video with the embedded watermark.

2. The method for embedding and extracting video image watermark based on deep learning according to claim 1, wherein The specific process of obtaining the target watermark fitting model is as follows: S301: Obtain the original watermark data corresponding to all the original videos in the video database. The original watermark data includes the watermark content and the position of the watermark content in the corresponding video block. Obtain the video block corresponding to the source of the original watermark data, crop the video block to obtain a video image, integrate it into a dataset, and generate a corresponding label text as a training sample; S302: Divide the training samples into a training set and a test set in a ratio of 8:

2. After initializing the training parameters, start training the model. Use a network structure with full convolution and cross-layer jump links similar to the residual network to extract features from the video images in the training set to obtain a feature map of a certain size; S303: Divide the video image into grids, use a multi-label multi-classification logistic regression layer to make class predictions, and at the same time use a multi-scale fusion method in the model to make border position predictions and perform detections on feature maps of multiple scales; S304: Load the initialized training parameters, perform model training on the GPU server, use the test set to detect the effect of the trained model, and save the optimal model weight parameters to obtain the target watermark fitting model.

3. The method for embedding and extracting video image watermark based on deep learning according to claim 1, characterized in that, The specific process of obtaining the embedding bitstream sequence of the target watermark data is as follows: S401: Perform binarization processing on the target watermark data to obtain the binary bitstream sequence of the target watermark data; S402: Perform convolutional code encoding according to the binary bitstream sequence to obtain the embedding bitstream sequence of the target watermark data.

4. The method for embedding and extracting video image watermark based on deep learning according to claim 1, characterized in that, The specific process of obtaining the finished video with the embedded watermark is as follows: S501: Obtain the discrete cosine transform coefficients of each macroblock. According to the discrete cosine transform coefficients of each macroblock, obtain a coefficient matrix, and perform a zigzag scan on the coefficient matrix to obtain a scan matrix; S502: Obtain the embedding bitstream sequence of the target watermark data, and determine whether the scan matrix needs to replace data. Re-encode and save the finished video with the target embedded watermark data to obtain the finished video with the embedded watermark.

Images