Video image watermark embedding and extracting method based on deep learning

Abstract

The invention discloses a video image watermark embedding and extracting method based on deep learning, which relates to the technical field of video image watermark processing, and comprises the following steps: establishing a video database according to an original video before editing, carrying out feature recognition on a plurality of video blocks in the original video, selecting two target video blocks, and carrying out feature extraction on the two target video blocks; and based on the similarity of the identification contents of the two target video blocks, determining the watermark content, extracting the watermark in the original video, performing integration to obtain raw watermark data, performing training through a deep learning model to obtain a target watermark fitting model, and outputting the target watermark data according to the watermark demand data in the finished video. The method is suitable for the watermark extraction and embedding process of a new video obtained by editing and synthesizing multiple videos, the watermark in the new video and the watermark in the original video have relevance, the watermark extraction and embedding process is simple, and the watermark extraction and embedding efficiency is improved. The invention provides a source-traceable digital watermark embedding and extracting method.

Description

Technical Field

[0001] The present invention relates to the technical field of video image watermark processing, and in particular to a video image watermark embedding and extraction method based on deep learning. Background Art

[0002] Digital watermarking technology embeds watermark information into documents, images, audio, video and other carriers without affecting the perception of the carrier, making it difficult to be discovered or tampered with. The embedder of the watermark information can use the set algorithm key to extract the watermark, evaluate the difference between the extracted watermark information and the original watermark information through the corresponding evaluation criteria, and then determine whether the data containing the watermark information has been attacked or tampered with. Video watermarking technology, as a type of digital watermarking technology, is an important means of protecting video products such as video copyright protection, content authentication, broadcast monitoring, and copy control. When synthesizing and editing multiple video clips, in order to ensure copyright ownership and prevent theft, a watermark will be embedded in the synthesized video. Before synthesis or editing, it is also necessary to extract the watermark in the video to ensure that the synthesized content does not contain the watermark or logo of the original video; There are two types of existing digital watermark generation algorithms: compressed domain watermark algorithm and original domain watermark algorithm. The compressed domain algorithms mainly include discrete cosine transform coefficient steganography algorithm and motion vector steganography algorithm. They compensate for the steganographic effect through motion compensation and energy analysis, and have high execution efficiency. However, since there is a corresponding watermark steganography algorithm for each video format, the algorithm has poor versatility. In addition, since the watermark is embedded after the video is encoded, there is less space for embedding the watermark, resulting in poor robustness of the watermark. In view of the above-mentioned technical defects, a solution is now proposed. Summary of the invention

[0003] The purpose of the present invention is to be applicable to the watermark extraction and embedding process of a new video obtained by synthesizing multiple video clips, and there is a correlation between the watermark in the new video and the watermark in the original video, providing a digital watermark embedding and extraction method that can trace the source.

[0004] In order to achieve the above object, the present invention adopts the following technical solution: a video image watermark embedding and extraction method based on deep learning, comprising the following steps: Step 1: obtaining the original video before editing, establishing a video database, decomposing the video frames of the original video into n times n video blocks, performing feature recognition on multiple video blocks in the original video, and obtaining a feature recognition result for each video block; Step 2: selecting two target video blocks with an interval greater than a preset number of frames from the multiple video blocks, and determining the watermark content based on the similarity of the identification contents of the two target video blocks, and extracting the watermark in the original video based on the watermark content and the position of each identification content in each feature recognition result in the corresponding video block, and integrating to obtain the original watermark data; Step 3: Integrate the original watermark data corresponding to all the original videos in the video database into a data set, train it through a deep learning model, obtain a target watermark fitting model, obtain the edited finished video, and then obtain the watermark requirement data in the finished video. The watermark requirement data includes the target watermark content and the position of the target watermark content in the finished video, and substitute it into the target watermark fitting model to output the target watermark data; Step 4: Obtain the embedded code stream sequence of the target watermark data, perform macroblock segmentation on the finished video, obtain the discrete cosine transform coefficient of each macroblock, divide the video resolution into multiple ranges, and select different frame processing methods according to different application scenarios and video resolution sizes to obtain embedded frames; Step 5: embed the finished video according to the discrete cosine transform coefficients and the embedded code stream sequence, re-encode and save the finished video with the target embedded watermark data, and obtain the finished video with the embedded watermark.

[0005] Furthermore, the specific process of obtaining the feature recognition result of each video block is as follows: S101, obtaining each video block in the original video, calculating the sum of the pixel values ​​of each video block, establishing a time horizontal axis according to the sequence of video frames, and establishing a vertical axis according to the numerical values ​​of the pixel values, obtaining a plane coordinate system, calibrating the sum of the pixel values ​​one by one in the plane coordinate system, and connecting them through a smooth curve to obtain a pixel value fluctuation curve; S102, segmenting the pixel value fluctuation curve, calculating the slope Ki of each segmented curve, and according to a preset slope threshold Kmin, if the slope Ki is greater than the slope threshold Kmin, obtaining the sum of the pixels of two adjacent video blocks on the segmented curve as the identification object; S103, if the sum of the pixel values ​​of the first video block in the identified object is greater than the sum of the pixel values ​​of the second video block, the extracted bit is 0, otherwise the extracted bit is 1; S104, repeat the previous step to obtain an n times n sequence, and generate a pseudo-random bit sequence s with a length of n; S105, using a pseudo-random bit sequence s to perform a bit-by-bit logical XOR operation on the n-by-n sequence to obtain the characteristic feature recognition result, each characteristic recognition result including the recognition content of each frame of the video block and the position of each recognition content in the corresponding video block.

[0006] Furthermore, the specific process of integrating the original watermark data is as follows: S201, selecting two target video blocks with an interval greater than a preset number of frames from the multiple video blocks, obtaining the sums of pixel values ​​S1 and S2 corresponding to the target video blocks, and calculating a pixel value difference ΔS=|S1-S2|; S202, obtaining a preset difference judgment threshold Smin, if the pixel value difference ΔS is less than or equal to the difference judgment threshold Smin, then the two target video blocks are selected as analysis objects; S203, obtaining the corresponding feature recognition result in each analysis object, defining the analysis area according to the position of each recognition content in the feature recognition result in the corresponding video block, and performing similarity calculation on the analysis area to obtain a similarity comparison value, the calculation formula of which is as follows: ,in and is the image mean corresponding to the two analysis areas, and is the variance, is the covariance, C1 and C2 are preset constants; S204, obtaining a preset similarity evaluation value SSIMi. If the similarity comparison value is greater than or equal to the similarity evaluation value SSIMi, the analysis area in the analysis object is used as the watermark content, the watermark content and the position of the watermark content in the corresponding video block are extracted, and the original watermark data is obtained by integration.

[0007] Furthermore, the specific process of obtaining the target watermark fitting model is as follows: S301, obtaining original watermark data corresponding to all original videos in the video database, wherein the original watermark data includes watermark content and the position of the watermark content in the corresponding video block, obtaining the video block corresponding to the source of the original watermark data, cropping the video block to obtain the video image, integrating it into a data set, and generating the corresponding label text as a training sample; S302, dividing the training samples into a training set and a test set in a ratio of 8:2, and starting to train the model after initializing the training parameters, using a network structure of full convolution and a cross-layer skip link structure similar to a residual network to extract features from the video images in the training set, and obtaining a feature map of a certain size; S303, dividing the video image into grids, using a multi-label multi-classification logistic regression layer to make category predictions, and using a multi-scale fusion method in the model to make border position predictions, and performing detection on feature maps of multiple scales; S304, loading the initialization training parameters, performing model training on the GPU server, using the test set to detect the effect of the trained model, and saving the optimal model weight parameters to obtain the target watermark fitting model.

[0008] Furthermore, the specific process of obtaining the embedded code stream sequence of the target watermark data is as follows: S401, binarizing the target watermark data to obtain a binary code stream sequence of the target watermark data; S402: Perform convolutional coding on the binary code stream sequence to obtain an embedded code stream sequence of the target watermark data.

[0009] Furthermore, the specific process of obtaining the finished video with embedded watermark is as follows: S501, obtaining discrete cosine transform coefficients of each macroblock, obtaining a coefficient matrix according to the discrete cosine transform coefficients of each macroblock, and performing a zigzag scan on the coefficient matrix to obtain a scan matrix; S502, obtaining the embedded code stream sequence of the target watermark data, and determining whether the scan matrix needs to replace data, re-encoding and saving the finished video with the target embedded watermark data, and obtaining the finished video with the embedded watermark.

[0010] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: The video image watermark embedding and extraction method based on deep learning establishes a video database according to the original video before editing, performs feature recognition on multiple video blocks in the original video, selects two target video blocks, and determines the watermark content based on the similarity of the recognition content of the two target video blocks, extracts the watermark in the original video, integrates to obtain the original watermark data, and trains through a deep learning model to obtain a target watermark fitting model, outputs the target watermark data according to the watermark requirement data in the finished video, obtains the embedded code stream sequence of the target watermark data, embeds the finished video to obtain a finished video with the embedded watermark. The present invention is suitable for the watermark extraction and embedding process of a new video obtained by synthesizing multiple video clips, and there is a correlation between the watermark in the new video and the watermark in the original video, providing a digital watermark embedding and extraction method that can trace the source. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of the process of extracting watermark from a video image of the present invention is shown; Figure 2 The schematic diagram of the process of video image watermark embedding method of the present invention is shown. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] Example like Figure 1-2 As shown, the video image watermark embedding and extraction method based on deep learning includes the following steps: Step 1: obtain the original video before editing, establish a video database, decompose the video frames of the original video into n times n video blocks, perform feature recognition on multiple video blocks in the original video, and obtain feature recognition results for each video block; The specific process of obtaining the feature recognition results of each video block is as follows: S101, obtaining each video block in the original video, calculating the sum of the pixel values ​​of each video block, establishing a time horizontal axis according to the sequence of video frames, and establishing a vertical axis according to the numerical values ​​of the pixel values, obtaining a plane coordinate system, calibrating the sum of the pixel values ​​one by one in the plane coordinate system, and connecting them through a smooth curve to obtain a pixel value fluctuation curve; S102, segmenting the pixel value fluctuation curve, calculating the slope Ki of each segmented curve, and according to a preset slope threshold Kmin, if the slope Ki is greater than the slope threshold Kmin, obtaining the sum of the pixels of two adjacent video blocks on the segmented curve as the identification object; S103, if the sum of the pixel values ​​of the first video block in the identified object is greater than the sum of the pixel values ​​of the second video block, the extracted bit is 0, otherwise the extracted bit is 1; S104, repeat the previous step to obtain an n times n sequence, and generate a pseudo-random bit sequence s with a length of n; S105, using the pseudo-random bit sequence s to perform a bit-by-bit logical XOR operation on the n-by-n sequence to obtain a feature recognition result, each feature recognition result including the recognition content of each frame of the video block and the position of each recognition content in the corresponding video block.

[0014] Step 2: Select two target video blocks with an interval greater than a preset number of frames from the multiple video blocks, and determine the watermark content based on the similarity of the recognition content of the two target video blocks, and extract the watermark in the original video based on the watermark content and the position of each recognition content in each feature recognition result in the corresponding video block, and integrate to obtain the original watermark data; The specific process of integrating the original watermark data is as follows: S201, selecting two target video blocks with an interval greater than a preset number of frames from multiple video blocks, obtaining the sums of pixel values ​​S1 and S2 corresponding to the target video blocks, and calculating a pixel value difference ΔS=|S1-S2|; S202, obtaining a preset difference judgment threshold Smin, if the pixel value difference ΔS is less than or equal to the difference judgment threshold Smin, then the two target video blocks are selected as analysis objects; S203, obtaining the corresponding feature recognition result in each analysis object, defining the analysis area according to the position of each recognition content in the feature recognition result in the corresponding video block, and performing similarity calculation on the analysis area to obtain a similarity comparison value, the calculation formula of which is as follows: ,in and is the image mean corresponding to the two analysis areas, and is the variance, is the covariance, C1 and C2 are preset constants; S204, obtaining a preset similarity evaluation value SSIMi. If the similarity comparison value is greater than or equal to the similarity evaluation value SSIMi, the analysis area in the analysis object is used as the watermark content, the watermark content and the position of the watermark content in the corresponding video block are extracted, and the original watermark data is obtained by integration.

[0015] Step 3: Integrate the original watermark data corresponding to all the original videos in the video database into a data set, train it through a deep learning model, obtain a target watermark fitting model, obtain the edited finished video, and then obtain the watermark requirement data in the finished video. The watermark requirement data includes the target watermark content and the position of the target watermark content in the finished video, and substitute it into the target watermark fitting model to output the target watermark data; The specific process of obtaining the target watermark fitting model is as follows: S301, obtaining original watermark data corresponding to all original videos in the video database, the original watermark data including the watermark content and the position of the watermark content in the corresponding video block, obtaining the video block corresponding to the source of the original watermark data, cropping the video block to obtain the video image, integrating it into a data set, and generating the corresponding label text as a training sample; S302, dividing the training samples into a training set and a test set in a ratio of 8:2, and starting to train the model after initializing the training parameters, using a network structure of full convolution and a cross-layer skip link structure similar to a residual network to extract features from the video images in the training set, and obtaining a feature map of a certain size; S303, dividing the video image into grids, using a multi-label multi-classification logistic regression layer to make category predictions, and using a multi-scale fusion method in the model to make border position predictions, and performing detection on feature maps of multiple scales; S304, loading the initialization training parameters, performing model training on the GPU server, using the test set to detect the effect of the trained model, and saving the optimal model weight parameters to obtain the target watermark fitting model.

[0016] Step 4: Obtain the embedded code stream sequence of the target watermark data, perform macroblock segmentation on the finished video, obtain the discrete cosine transform coefficient of each macroblock, divide the video resolution into multiple ranges, and select different frame processing methods according to different application scenarios and video resolution sizes to obtain embedded frames; The specific process of obtaining the embedded code stream sequence of the target watermark data is as follows: S401, binarizing the target watermark data to obtain a binary code stream sequence of the target watermark data; S402: Perform convolutional coding according to the binary code stream sequence to obtain an embedded code stream sequence of target watermark data.

[0017] Step 5: embed the finished video according to the discrete cosine transform coefficients and the embedded code stream sequence, re-encode and save the finished video with the target embedded watermark data, and obtain the finished video with the embedded watermark.

[0018] The specific process of getting the finished video with embedded watermark is as follows: S501, obtaining discrete cosine transform coefficients of each macroblock, obtaining a coefficient matrix according to the discrete cosine transform coefficients of each macroblock, and performing a zigzag scan on the coefficient matrix to obtain a scan matrix; S502, obtaining the embedded code stream sequence of the target watermark data, and determining whether the scan matrix needs to replace data, re-encoding and saving the finished video with the target embedded watermark data, and obtaining the finished video with the embedded watermark.

[0019] The present invention establishes a video database according to the original video before editing, and performs feature recognition on multiple video blocks in the original video, selects two target video blocks, and determines the watermark content based on the similarity of the recognition content of the two target video blocks, extracts the watermark in the original video, integrates to obtain the original watermark data, and trains through a deep learning model to obtain a target watermark fitting model, outputs the target watermark data according to the watermark requirement data in the finished video, obtains the embedded code stream sequence of the target watermark data, embeds the finished video to obtain a finished video with an embedded watermark, and the present invention is suitable for the watermark extraction and embedding process of a new video obtained by synthesizing multiple video clips, and there is a correlation between the watermark in the new video and the watermark of the original video, and provides a digital watermark embedding and extraction method that can trace the source.

[0020] The threshold is set to facilitate comparison. The size of the threshold depends on the amount of sample data and the number of bases set by technicians in this field for each group of sample data; as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0021] The above formulas are all dimensionless and numerical calculations. The formula is a formula obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formula are set by technicians in this field according to actual conditions. The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A video image watermark embedding and extraction method based on deep learning, characterized in that: The following steps are involved: Step 1: obtaining the original video before editing, establishing a video database, decomposing the video frames of the original video into n times n video blocks, performing feature recognition on multiple video blocks in the original video, and obtaining a feature recognition result for each video block; Step 2: selecting two target video blocks with an interval greater than a preset number of frames from the multiple video blocks, and determining the watermark content based on the similarity of the identification contents of the two target video blocks, and extracting the watermark in the original video based on the watermark content and the position of each identification content in each feature recognition result in the corresponding video block, and integrating to obtain the original watermark data; Step 3: Integrate the original watermark data corresponding to all the original videos in the video database into a data set, train it through a deep learning model, obtain a target watermark fitting model, obtain the edited finished video, and then obtain the watermark requirement data in the finished video. The watermark requirement data includes the target watermark content and the position of the target watermark content in the finished video, and substitute it into the target watermark fitting model to output the target watermark data; Step 4: Obtain the embedded code stream sequence of the target watermark data, perform macroblock segmentation on the finished video, obtain the discrete cosine transform coefficient of each macroblock, divide the video resolution into multiple ranges, and select different frame processing methods according to different application scenarios and video resolution sizes to obtain embedded frames; Step 5: embed the finished video according to the discrete cosine transform coefficients and the embedded code stream sequence, re-encode and save the finished video with the target embedded watermark data, and obtain the finished video with the embedded watermark.

2. The video image watermark embedding and extraction method based on deep learning according to claim 1 is characterized in that: The specific process of obtaining the feature recognition results of each video block is as follows: S101, obtaining each video block in the original video, calculating the sum of the pixel values ​​of each video block, establishing a time horizontal axis according to the sequence of video frames, and establishing a vertical axis according to the numerical values ​​of the pixel values, obtaining a plane coordinate system, calibrating the sum of the pixel values ​​one by one in the plane coordinate system, and connecting them through a smooth curve to obtain a pixel value fluctuation curve; S102, segmenting the pixel value fluctuation curve, calculating the slope Ki of each segmented curve, and according to a preset slope threshold Kmin, if the slope Ki is greater than the slope threshold Kmin, obtaining the sum of the pixels of two adjacent video blocks on the segmented curve as the identification object; S103, if the sum of the pixel values ​​of the first video block in the identified object is greater than the sum of the pixel values ​​of the second video block, the extracted bit is 0, otherwise the extracted bit is 1; S104, repeat the previous step to obtain an n times n sequence, and generate a pseudo-random bit sequence s with a length of n; S105, using a pseudo-random bit sequence s to perform a bit-by-bit logical XOR operation on the n-by-n sequence to obtain the characteristic feature recognition result, each characteristic recognition result including the recognition content of each frame of the video block and the position of each recognition content in the corresponding video block.

3. The video image watermark embedding and extraction method based on deep learning according to claim 1 is characterized in that: The specific process of integrating the original watermark data is as follows: S201, selecting two target video blocks with an interval greater than a preset number of frames from the multiple video blocks, obtaining the sums of pixel values ​​S1 and S2 corresponding to the target video blocks, and calculating a pixel value difference ΔS=|S1-S2|; S202, obtaining a preset difference judgment threshold Smin, if the pixel value difference ΔS is less than or equal to the difference judgment threshold Smin, then the two target video blocks are selected as analysis objects; S203, obtaining the corresponding feature recognition result in each analysis object, defining the analysis area according to the position of each recognition content in the feature recognition result in the corresponding video block, and performing similarity calculation on the analysis area to obtain a similarity comparison value, the calculation formula of which is as follows: ,in and is the image mean corresponding to the two analysis areas, and is the variance, is the covariance, C1 and C2 are preset constants; S204, obtaining a preset similarity evaluation value SSIMi. If the similarity comparison value is greater than or equal to the similarity evaluation value SSIMi, the analysis area in the analysis object is used as the watermark content, the watermark content and the position of the watermark content in the corresponding video block are extracted, and the original watermark data is obtained by integration.

4. The video image watermark embedding and extraction method based on deep learning according to claim 1 is characterized in that: The specific process of obtaining the target watermark fitting model is as follows: S301, obtaining original watermark data corresponding to all original videos in the video database, wherein the original watermark data includes watermark content and the position of the watermark content in the corresponding video block, obtaining the video block corresponding to the source of the original watermark data, cropping the video block to obtain the video image, integrating it into a data set, and generating the corresponding label text as a training sample; S302, dividing the training samples into a training set and a test set in a ratio of 8:2, and starting to train the model after initializing the training parameters, using a network structure of full convolution and a cross-layer skip link structure similar to a residual network to extract features from the video images in the training set, and obtaining a feature map of a certain size; S303, dividing the video image into grids, using a multi-label multi-classification logistic regression layer to make category predictions, and using a multi-scale fusion method in the model to make border position predictions, and performing detection on feature maps of multiple scales; S304, loading the initialization training parameters, performing model training on the GPU server, using the test set to detect the effect of the trained model, and saving the optimal model weight parameters to obtain the target watermark fitting model.

5. The video image watermark embedding and extraction method based on deep learning according to claim 1 is characterized in that: The specific process of obtaining the embedded code stream sequence of the target watermark data is as follows: S401, binarizing the target watermark data to obtain a binary code stream sequence of the target watermark data; S402: Perform convolutional coding on the binary code stream sequence to obtain an embedded code stream sequence of the target watermark data.

6. The video image watermark embedding and extraction method based on deep learning according to claim 1 is characterized in that: The specific process of getting the finished video with embedded watermark is as follows: S501, obtaining discrete cosine transform coefficients of each macroblock, obtaining a coefficient matrix according to the discrete cosine transform coefficients of each macroblock, and performing a zigzag scan on the coefficient matrix to obtain a scan matrix; S502, obtaining the embedded code stream sequence of the target watermark data, and determining whether the scan matrix needs to replace data, re-encoding and saving the finished video with the target embedded watermark data, and obtaining the finished video with the embedded watermark.

Images