Robust video watermarking method and system based on frame domain attack training

Abstract

The invention discloses a robust video watermarking method and system based on frame domain attack training. According to the invention, a model training framework composed of an encoder module, a noise attack module, a generative adversarial module and a decoder module is constructed; and taking minimization of the total loss function as a target, training the model training framework by using training data, and introducing a plurality of noise attacks and watermark removal attacks in the training process to improve the robustness of the model. In practical application, the trained encoder module is utilized to add watermark information to a target video, and the trained decoder module is utilized to extract the watermark information from the watermark video. According to the method, the noise attack and the watermark removal attack are introduced in the training process, so that the stability and the extraction precision of the watermark in a complex attack environment can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of digital watermark and information security technology, and in particular relates to an algorithm for enhancing the anti-attack capability of video watermark in a complex and changeable propagation environment, and is particularly used to improve the robustness and stability of video watermark in various attack scenarios such as frame domain attack, noise interference and occlusion. Background Art

[0002] As piracy becomes increasingly rampant, video, as the most efficient and universal carrier of information dissemination, has gradually become the main target coveted by pirates. Therefore, how to solve the problem of video copyright protection has become increasingly important. Among them, video watermarking technology has become an important method of copyright protection due to its robustness and imperceptibility. However, most of the existing video watermarking algorithms are either limited by the inability of traditional algorithms to cope with new video attack methods, or only embed watermarks into single-frame images, resulting in the inability to make good use of the redundant space in the video, and the ability to resist frame domain attacks needs to be improved.

[0003] Therefore, in view of the fact that the current video watermarking algorithm is insufficient in resisting frame domain attacks, it is urgent to propose a robust watermarking algorithm based on frame domain attack training to improve the stability and extraction accuracy of the watermark in complex attack environments. Summary of the invention

[0004] The purpose of the present invention is to solve the problems existing in the prior art and provide a robust video watermarking method and system based on frame domain attack training.

[0005] The specific technical solutions adopted by the present invention are as follows:

[0006] In a first aspect, the present invention provides a robust video watermarking method based on frame domain attack training, which comprises:

[0007] S1. Construct a model training framework consisting of an encoder module, a noise attack module, a generative adversarial module, and a decoder module;

[0008] S2. With the goal of minimizing the total loss function, the model training framework is trained using training data. During the training process, the encoder module first uses the mask generated by the attention mechanism to embed the watermark information into the original video sample to generate a watermark video. The original video sample and the watermark video are output through the discriminator. The probability score of carrying the watermark information is respectively output; then the noise attack module randomly performs a noise attack on the watermark video and generates a first distorted video, and the noise attack type includes at least a frame domain attack, a Dropout attack, a Gaussian noise attack and an occlusion attack. The generator performs a de-watermarking attack on the watermark video and generates a second distorted video. The generated first distorted video and the second distorted video are input into the decoder module to respectively extract the first watermark information and the second watermark information using the mask generated by the attention mechanism; the total loss function is weighted by the expectation of the probability score output by the discriminator for the watermark video, the cross entropy loss between the first watermark information and the true value of the watermark information, and the cross entropy loss between the second watermark information and the true value of the watermark information;

[0009] S3. In practical applications, the trained encoder module is used to add watermark information to the target video, and the trained decoder module is used to extract watermark information from the watermark video.

[0010] As a preferred embodiment of the above-mentioned first aspect, the encoder module takes the original video as input, and first generates a first attention mask through an attention module composed of two cascaded convolutional layers and a Softmax layer inside the module, and then splices the multiplication result of the watermark information and the first attention mask with the original video along the channel dimension. The splicing result is reduced in dimension by two layers of convolution to obtain a 3-channel residual mask, and the residual mask is weighted with the original video to obtain the final watermarked video, thereby generating a watermarked video.

[0011] As a preferred embodiment of the above-mentioned first aspect, the noise attack module is used to take the watermarked video as input, simulate the noise attack on the watermarked video, and generate a distorted video; wherein the noise attack types include cropping attack, scaling attack, compression attack, Gaussian noise attack, Dropout attack, occlusion attack, frame copy attack and frame deletion attack, and different noise types need to be randomly combined when performing a noise attack on a single watermarked video.

[0012] As a preferred embodiment of the above-mentioned first aspect, the generative adversarial module includes a generator and a discriminator. The generator is used to perform a watermark removal attack on the watermarked video to generate a distorted video; the discriminator is used to discriminate whether the watermarked video and the original video contain watermark information, and output the probability scores of the two videos carrying the watermark information respectively.

[0013] As a preferred embodiment of the first aspect, the discriminator is formed by a cascade of a convolutional layer and a classification layer; and the generator is formed by a cascade of a convolutional layer and a residual connection layer.

[0014] As a preferred embodiment of the above-mentioned first aspect, the decoder module takes different distorted videos as input, and each distorted video first generates a second attention mask through an attention module composed of two convolutional layers and a Softmax layer cascaded. At the same time, each distorted video obtains a feature map of the same dimension as the second attention mask after two layers of convolution. The second attention mask is multiplied by the feature map and average pooling is performed in the frame dimension. An average value is obtained for each channel, and the average values ​​of all channels are arranged in order as the extraction result of the watermark information.

[0015] In a second aspect, the present invention provides a robust video watermarking system based on frame domain attack training, which comprises:

[0016] A storage module, used to store the trained encoder module and the trained decoder module obtained according to the robust video watermarking method based on frame domain attack training as described in any one of the first aspects above;

[0017] A watermark generation module is used for calling the trained encoder module to embed target watermark information into a video frame to obtain a watermarked video for a target video;

[0018] The watermark extraction module is used to extract watermark information from the watermark video by calling the trained decoder module.

[0019] In a third aspect, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the robust video watermarking method based on frame domain attack training as described in any one of the first aspects above.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the robust video watermarking method based on frame domain attack training as described in any one of the first aspects above can be implemented.

[0021] In a fifth aspect, the present invention provides a computer electronic device comprising a memory and a processor;

[0022] The memory is used to store computer programs;

[0023] The processor is used to implement the robust video watermarking method based on frame domain attack training as described in any one of the first aspects above when executing the computer program.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1) In view of the fact that current video watermarking algorithms are insufficient in resisting frame domain attacks, the present invention proposes a robust watermarking method based on frame domain attack training. This method can improve the stability and extraction accuracy of watermarks in complex attack environments by introducing noise attacks and removing watermark attacks during the training process.

[0026] 2) In view of the weak ability of existing algorithms to resist frame domain attacks, the present invention designs a frame deletion and frame duplication attack layer in the deep learning network. The algorithm is specially trained for frame domain attacks that may be encountered during the video transmission process, making the watermark more stable and reliable in dealing with such attacks and suitable for dynamic scenarios in practical applications.

[0027] 3) The present invention introduces a variety of attack methods, including Dropout attack, Gaussian noise attack and occlusion attack, to simulate different pixel-level attack methods, greatly enriching the model's resistance to various complex attacks.

[0028] 4) The attack simulation of the present invention is closer to the real scene. It randomly combines and mixes different attack processing methods during the training process, so that the generated video sequence is closer to the attack situation in the real application. Under the joint action of multiple attack layers, the model has stronger generalization ability after training. The experimental results show that the algorithm has stronger robustness in distortion processing such as frame deletion attack, Gaussian noise attack and frame duplication attack. At the same time, adding a new attack layer will not cause a more obvious visual impact on the watermarked video generated by the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the steps of a robust video watermarking method based on frame domain attack training;

[0030] Figure 2 Schematic diagram of the model training framework;

[0031] Figure 3 This is the encoder module flow chart;

[0032] Figure 4 Schematic diagram of the spatial attention mechanism;

[0033] Figure 5 This is the flow chart of the attention module;

[0034] Figure 6 This is a schematic diagram of the layer structure of the noise attack module;

[0035] Figure 7 is the flow chart of the decoder module;

[0036] Figure 8 This is a comparison chart of watermark extraction accuracy of different algorithms;

[0037] Fig. 9 Schematic diagram of the module composition of the robust video watermarking system based on frame domain attack training;

[0038] Fig.10 A block diagram of computer electronic equipment. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0040] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0041] In view of the current situation that the video watermark algorithm is not able to resist frame domain attacks, the present invention proposes a robust watermark method based on frame domain attack training to improve the stability and extraction accuracy of the watermark in complex attack environments. The following describes the specific implementation of the robust video watermark method based on frame domain attack training in conjunction with an embodiment.

[0042] like Figure 1 As shown, in a preferred embodiment of the present invention, a robust video watermarking method based on frame domain attack training is provided, which includes steps S1 to S3, and the specific implementation method of each step is as follows.

[0043] S1. Construct a model training framework consisting of an encoder module, a noise attack module, a generative adversarial module, and a decoder module.

[0044] like Figure 2As shown in the figure, a schematic diagram of the structure of the above model training framework is shown. In this framework, the original video is first input into the encoder module, and a mask is generated through the spatial attention mechanism to embed the watermark into each frame of the video, thereby generating a watermarked video. Subsequently, the watermarked video enters the noise attack module to simulate various distortion processes such as frame domain, occlusion, and compression. At the same time, the watermarked video is also passed to the generator in the generative adversarial module for de-watermarking attack. Both the noise attack module and the generator in the generative adversarial module will produce distorted videos. In addition, the watermarked video and the original video are also synchronously input into the discriminator to determine whether the video carries watermark information. Finally, the decoder module uses the attention mechanism to extract the watermark information from the distorted video. By comprehensively optimizing the model through the loss terms of different links, the extraction reliability and anti-attack robustness of the watermark in complex environments can be improved. The specific structure and data processing method of each module in the above model training framework are described in detail below.

[0045] In an embodiment of the present invention, the above-mentioned encoder module takes the original video as input, and first generates a first attention mask through an attention module composed of two convolutional layers and a Softmax layer cascaded in the module, and then splices the multiplication result of the watermark information and the first attention mask with the original video along the channel dimension. The splicing result is reduced in dimension by two layers of convolution to obtain a 3-channel residual mask, and the residual mask is weighted with the original video to obtain the final watermarked video, thereby generating a watermarked video.

[0046] like Figure 3 As shown, the internal process of the encoder module in an embodiment of the present invention is demonstrated. The original video is first input into the encoder module, and an attention mask is generated for each frame of the image through the attention mechanism in the attention module. The watermark information is embedded into the image using the attention mask, and the watermark image is generated frame by frame, and the watermark video is synthesized.

[0047] It should be noted that the process of generating the attention mask in the above attention module is a spatial attention mechanism. In this mechanism, before watermark embedding, the embedding weight coefficients of different positions need to be determined. The attention module generates the attention mask through the spatial attention mechanism to determine the embedding weight coefficient of each bit. The principle of the spatial attention mechanism is as follows Figure 4 As shown, it is assumed that the number of watermark information bits to be embedded is D, the image width and height of the video frame are W and H respectively, and the total number of frames is T. Figure 4 The yellow cube on the left side of the middle image indicates that the attention mask of a single frame image contains D channels, and the red matrix represented by ① is of size W×H, representing the weight data distribution in a channel (channel 1 in the figure); the horizontal pink area represented by ② represents the weight distribution of a pixel (pixel block 1 in the figure) in each channel. For example, the weight of the first channel is 0.1, the weight of the second channel is 0.2, and so on.

[0048] Based on the above spatial attention mechanism, the attention module structure involved in the embodiment of the present invention is as follows Figure 5 As shown in the figure, it contains two convolutional layers and a Softmax layer. The first convolutional layer convolves the original video frame of T×W×H×3 dimension from 3 channels to 32 channels to obtain a T×W×H×32-dimensional tensor. The second convolutional layer convolves the 32 channels to the number of channels of the required watermark information bits D to obtain a T×W×H×D-dimensional tensor. The Softmax function is then applied to the D-channel data tensor to generate the final T×W×H×D-dimensional attention mask.

[0049] Last seen Figure 3 The attention module is used to multiply the T×W×H×D-dimensional attention mask and add the D-bit watermark information to obtain a single-channel four-dimensional tensor (dimension T×W×H×1), and the single-channel four-dimensional tensor is connected to the three-channel original video to obtain a four-channel four-dimensional tensor (dimension T×W×H×4), and the four-channel four-dimensional tensor is increased to 32 channels (dimension T×W×H×32) through the first layer of convolution kernel, and then reduced to 3 channels through the second layer of convolution layer of the same size to obtain a residual mask (dimension T×W×H×3), and finally the residual mask is superimposed on the original video according to a certain weight coefficient to obtain the final watermark video. The weight coefficient is an optimizable hyperparameter, which is optimized to 0.016 in the embodiment of the present invention.

[0050] In the embodiment of the present invention, the core of the noise attack module is a series of noise attack layers, each of which can be used to add a type of noise attack to the watermarked video. The present invention optimizes the types of noise attacks to a certain extent, and specially designs the following three types of noise attacks:

[0051] First, the present invention designs two unique frame domain attack layers, which target different characteristics of video frame content respectively: the first is the frame copy attack layer, which randomly selects several frames in a multi-frame video sequence and copies their content to achieve the effect of covering part of the video screen, simulating common tampering behaviors in actual applications; the second is the frame deletion attack layer, which deletes certain consecutive frames in the video sequence and removes part of the video content, thereby enhancing the algorithm's ability to deal with frame loss or clipping attacks.

[0052] At the same time, in order to further enhance the watermark's resistance to random interference, the present invention combines the Dropout regularization technology in deep neural networks and proposes a Dropout attack layer. This layer randomly retains pixel blocks of different proportions in the original video frame and the encoded watermarked video frame, and randomly selects the positions of these pixel blocks, and then reassembles them into a new noise image, thereby effectively simulating random attacks in practice.

[0053] In addition, the present invention adds a Gaussian noise attack layer, and by adding Gaussian noise of different intensities to the video, the robustness of the algorithm in the face of noise interference is enhanced, ensuring that the watermark information can still be accurately extracted under high noise conditions.

[0054] Finally, in order to deal with the occlusion operation commonly used in video editing, the present invention introduces an occlusion attack layer, which simulates the situation that the watermark in the video may be occluded by objects to ensure that in a complex video editing environment, the watermark will not be unable to be restored due to partial occlusion.

[0055] The present invention introduces an improved noise attack module between the encoder module and the decoder module of the watermark video, adds these four attacks (frame domain attack, Dropout attack, Gaussian noise attack and occlusion attack), and the generated watermark video will be distorted, which can ensure that the algorithm model training framework is fully trained and significantly improve its robustness under various frame domain attacks.

[0056] In the embodiment of the present invention, the noise attack module uses the watermark video as input, simulates the noise attack on the watermark video and generates a distorted video. All noise attack types used include cropping attack, scaling attack, compression attack, Gaussian noise attack, Dropout attack, occlusion attack, frame duplication attack and frame deletion attack. Figure 6 As shown in Figure 1, the attack operations performed in each noise attack layer are as follows:

[0057] 1) Cropping attack layer: 80%-100% of the frame is randomly selected for cropping, and the model is trained to embed the watermark with sufficient spatial redundancy so that the watermark is still preserved after cropping.

[0058] 2) Scaling attack layer: The video is scaled to a random ratio (between 80% and 100%) to ensure the embedding consistency of the watermark data when the video ratio changes.

[0059] 3) Compression attack layer: simulate the quantization process of H.264 compression, return some high-frequency components to zero after applying DCT, and return to RGB space through inverse DCT, so that the training model is resistant to compression.

[0060] 4) Gaussian noise attack layer: Introduce a noise matrix whose mean and variance conform to Gaussian distribution to increase the robustness of the algorithm under noise interference.

[0061] 5) Dropout attack layer: reassemble pixel blocks from video frames and generate distorted frames using the pixel differences between the original and watermarked frames to enhance adaptability to frame structure changes.

[0062] 6) Occlusion attack layer: Rectangular occlusions of uncertain sizes are randomly added to the video frames to simulate the difficulty of watermark extraction under occlusion.

[0063] 7) Frame duplication attack layer: Randomly copy frame content to cover adjacent frames, and train the model to maintain watermark integrity when repeating or modifying content.

[0064] 8) Frame deletion attack layer: Randomly delete frames from the video sequence to enhance the watermark's survivability under content reduction conditions.

[0065] During the training process, different noise types need to be randomly combined when performing noise attacks on a single watermarked video. By randomly combining multiple attacks for distortion processing, a more robust video watermark model can be constructed.

[0066] In an embodiment of the present invention, the above-mentioned generative adversarial module includes a generator and a discriminator, and the function of the module is similar to the adversarial training structure in the GAN network. The generator is used to remove the watermark attack on the watermark video generated by the encoder module to generate a distorted video; and the discriminator is used to discriminate whether the watermark video generated by the encoder module and the original video contain watermark information, and output the probability scores of the watermark information in the two videos respectively. In an embodiment of the present invention, the discriminator can be formed by cascading a convolution layer and a classification layer, and the convolution and classification layers are used to judge whether the video contains a watermark. If the probability score is higher than the threshold, it is judged to contain a watermark, and if it is lower than the threshold, it is judged to be free of watermark. In addition, the generator will simulate the watermark removal attack and forge the watermark video into a watermark-free video. The above-mentioned generator is cascaded by a convolution layer and a residual connection layer, and the internal processing flow is: taking the watermark video as input, the watermark video passes through two convolution layers to obtain a tensor of the same dimension as the watermark video, and the tensor is re-added to the watermark video to obtain the output distorted video after the watermark attack is removed.

[0067] Therefore, within the generative adversarial module, the generator and the discriminator can also adopt the alternating training mode of the GAN network and be trained in advance. The loss of the discriminator can be set as the Wasserstein loss between the probability scores of the original video and the watermarked video output, and the loss of the generator can be set as the negative of the cross entropy loss of the true value M of the watermark information and the second watermark information. The generator and the discriminator in the generative adversarial module use the game to prompt the encoder to embed the watermark more covertly, thereby improving the imperceptibility and visual quality of the watermark.

[0068] In an embodiment of the present invention, the last module in the above-mentioned model training framework is a decoder module. The decoder module takes different distorted videos (distorted videos output by the generator and distorted videos output by the noise attack module) as input, and each distorted video first generates a second attention mask through an attention module composed of two convolutional layers and a Softmax layer cascaded, and each distorted video is convoluted to obtain a feature map of the same dimension as the second attention mask after two layers of convolution, and the second attention mask is multiplied with the feature map and average pooled on the frame dimension (ie, channel dimension), and each channel obtains an average value, and the average values ​​of all channels are arranged in order as the extraction result of the watermark information.

[0069] like Figure 7 As shown in the figure, the specific model structure of the above decoder module is shown. Its single input is a distorted watermark video (the distorted video output by the generator and the distorted video output by the noise attack module need to be input in turn). Assuming that the dimension of the distorted watermark video is T×W×H×3, the decoder module will generate an attention mask through the attention mechanism after receiving the distorted watermark video. Specifically, by comparing the weight values ​​of each bit, the weight value of each bit of each pixel block is determined, and finally the watermark information is extracted. The specific internal process of the module is as follows:

[0070] 1. The three-channel T×W×H×3-dimensional distorted watermark video is passed through the attention module consistent with the encoder module to generate a T×W×H×D-dimensional attention mask;

[0071] 2. The three-channel T×W×H×3-dimensional distorted watermark video is processed through two convolutional layers. The first convolutional layer expands the number of channels of the video to 32, and obtains a T×W×H×32-dimensional tensor. The second convolutional layer further converts the 32-channel convolution result into a D-channel to obtain a T×W×H×D-dimensional feature tensor.

[0072] 3. Use the T×W×H×D-dimensional attention mask generated by the attention mechanism to multiply the T×W×H×D-dimensional feature tensor element by element to strengthen or weaken the influence of each bit channel and obtain a T×W×H×D-dimensional attention weighted feature map.

[0073] 4. Perform average pooling on the T×W×H×D dimensional attention weighted feature map in the frame dimension, that is, perform average pooling on the channel dimension. The T×W×H feature values ​​in each channel are averaged to obtain an average value. After arranging this average value bit by bit, the recovered watermark information can be obtained.

[0074] S2. With the goal of minimizing the total loss function, the model training framework is trained using the training data.

[0075] During the training process of the above model training framework, for each input original video sample, it carries watermark information that needs to be embedded, and the watermark information will be used as a true value label later. After the original video sample and watermark information are input, the encoder module first uses the mask generated by the attention mechanism to embed the watermark information into the original video sample to generate a watermarked video. The original video sample and the watermarked video are output through the discriminator. The probability score of carrying the watermark information is output; then the noise attack module randomly performs a noise attack on the watermarked video and generates a first distorted video, and the generator performs a de-watermarking attack on the watermarked video and generates a second distorted video. The generated first distorted video and the second distorted video are input into the decoder module to extract the first watermark information and the second watermark information respectively using the mask generated by the attention mechanism.

[0076] Finally, based on the input and output of each module, the total loss function can be calculated. In this embodiment, the total loss function is weighted by the expectation of the probability score output by the discriminator for the watermarked video, the cross entropy loss between the first watermark information and the true value of the watermark information, and the cross entropy loss between the second watermark information and the true value of the watermark information. The total loss function L all The formula form is as follows:

[0077] L all =aL c +βL adv +λL w

[0078] in:

[0079] The above L c is the expected probability score of the watermark information output by the discriminator for the watermarked video, expressed as:

[0080] L c =E V，M [C(E(V,M))]

[0081] Where: E V,M () represents the expectation, C(E(V,M)) represents the probability score of the watermark information output by the discriminator for the watermarked video, V and M represent the original video and watermark information respectively, and E(V,M) represents the watermarked video output by the encoder module based on V and M;

[0082] The above L w is the cross entropy loss between the first watermark information and the true value of the watermark information, expressed as

[0083] L w =E V，M [CrossEntropy(M,D(N(E(V,M))))]

[0084] Where: E V,M () represents the expectation, CrossEntropy(M,D(N(E(V,M)))) represents the calculation of the cross entropy between the true value of the watermark information M and D(A(E(V,M))), and D(N(E(V,M))) represents the first watermark information extracted by the decoder module D from the distorted video N(E(V,M)) generated after processing by the noise attack module N;

[0085] The above L adv is the cross entropy loss between the second watermark information and the true value of the watermark information, expressed as:

[0086] L adv =E V，M [CrossEntropy(M,D(A(E(V,M))))]

[0087] Where: E V,M () represents expectation, CrossEntropy(M,D(A(E(V,M)))) represents the calculation of cross entropy between the true value M of the watermark information and D(A(E(V,M))), and D(A(E(V,M))) represents the second watermark information extracted by the decoder module D from the distorted video A(E(V,M)) generated after processing by the generator A.

[0088] During the model training process, each module can be updated in reverse based on the total loss value of each round to optimize the model parameters.

[0089] S3. In practical applications, the trained encoder module is used to add watermark information to the target video, and the trained decoder module is used to extract watermark information from the watermark video.

[0090] It should be noted that the noise attack module and the generative adversarial module are only introduced during the training process, and these two modules may not be introduced in practical applications. The watermark video generated by the watermark embedder using the encoder module can be transmitted to the receiver, and the receiver directly inputs the received watermark video into the decoder module for watermark information extraction.

[0091] In order to better demonstrate the technical effect of the watermark method shown in S1 to S3 above (referred to as the method of the present invention), its effect is verified on specific data. At the same time, for horizontal comparison, the method of the present invention is compared with two common algorithms HiDDeN and RivaGAN in the prior art. At the same time, in practical applications, the method of the present invention only needs to use the trained encoder module to add watermark information to the target video, and use the trained decoder module to extract the watermark information from the watermark video. However, in order to demonstrate the anti-attack robustness of the method of the present invention, this embodiment introduces different noise attacks into the watermark video generated by the encoder module, and the watermark video after the noise attack is input into the decoder module for watermark information extraction. The first noise attack is a Gaussian attack, and the experimental group is recorded as the present invention (including a Gaussian attack layer); the second noise attack is a combination of a Gaussian attack and a Dropout attack, and the experimental group is recorded as the present invention (including a Gaussian attack layer + a Dropout attack layer); the third noise attack is a combination of a Gaussian attack, a Dropout attack, and an occlusion attack, and the experimental group is recorded as the present invention (including a Gaussian attack layer + a Dropout attack layer + an occlusion attack layer). The final experimental results are as follows:

[0092] Table 1 shows the PSNR and SSIM indicators of different models. Even if three more noise layers are added in the method of the present invention, the PSNR value of the watermarked video generated by the encoding is still improved. Especially when only the Gaussian attack layer is added, the video quality effect is most significantly improved, and the SSIM indicator of the watermarked video is also significantly better than other methods.

[0093] Table 1 Comparison of imperceptibility of different algorithms

[0094]

[0095] It can be seen from the above table that the present invention significantly improves the robustness against noise attacks.

[0096] In addition to the visual effect comparison, this embodiment also compares the watermark extraction accuracy of the algorithm under different attack intensities, such as Figure 8 As shown, the vertical axis is the watermark bit accuracy, and the horizontal axis is the attacks of different intensities. The results show that under the four types of attacks, the watermark accuracy of the method of the present invention is improved.

[0097] Similarly, based on the same inventive concept, Fig. 9 As shown, the present invention also provides a robust video watermark system based on frame domain attack training, which includes:

[0098] A storage module, used for storing the trained encoder module and the trained decoder module obtained according to the robust video watermarking method based on frame domain attack training as described above;

[0099] A watermark generation module is used for calling the trained encoder module to embed target watermark information into a video frame to obtain a watermarked video for a target video;

[0100] The watermark extraction module is used to extract watermark information from the watermark video by calling the trained decoder module.

[0101] It should be noted that the above-mentioned storage module, watermark generation module and watermark extraction module can be respectively arranged on different computer devices, and the watermark generation module and the watermark extraction module only need to be able to communicate with the storage module respectively.

[0102] Similarly, based on the same inventive concept, Fig.10 As shown, the present invention also provides a computer electronic device provided in the above embodiment, which includes a memory and a processor;

[0103] The memory is used to store computer programs;

[0104] The processor is used to implement the above-mentioned robust video watermarking method based on frame domain attack training when executing the computer program.

[0105] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0106] Therefore, based on the same inventive concept, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the robust video watermarking method based on frame domain attack training as described above can be implemented.

[0107] Therefore, based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can implement the robust video watermarking method based on frame domain attack training as described above.

[0108] It is understandable that the above storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. The storage medium may also be a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc., which can store program codes.

[0109] It can be understood that the above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0110] It should also be noted that those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the various embodiments provided in this application, the division of steps or modules in the system and method is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or steps can be combined or integrated together, and a module or step can also be split.

[0111] The above-described embodiments are only some preferred implementations of the present invention, but are not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A robust video watermarking method based on frame domain attack training, characterized in that: include: S1. Construct a model training framework consisting of an encoder module, a noise attack module, a generative adversarial module, and a decoder module; S2. With the goal of minimizing the total loss function, the model training framework is trained using training data. During the training process, the encoder module first uses the mask generated by the attention mechanism to embed the watermark information into the original video sample to generate a watermark video. The original video sample and the watermark video are output through the discriminator. The probability score of carrying the watermark information is respectively output; then the noise attack module randomly performs a noise attack on the watermark video and generates a first distorted video, and the noise attack type includes at least a frame domain attack, a Dropout attack, a Gaussian noise attack and an occlusion attack. The generator performs a de-watermarking attack on the watermark video and generates a second distorted video. The generated first distorted video and the second distorted video are input into the decoder module to respectively extract the first watermark information and the second watermark information using the mask generated by the attention mechanism; the total loss function is weighted by the expectation of the probability score output by the discriminator for the watermark video, the cross entropy loss between the first watermark information and the true value of the watermark information, and the cross entropy loss between the second watermark information and the true value of the watermark information; S3. In practical applications, the trained encoder module is used to add watermark information to the target video, and the trained decoder module is used to extract watermark information from the watermark video.

2. The robust video watermarking method based on frame domain attack training as claimed in claim 1, characterized in that: The encoder module takes the original video as input. The module first generates a first attention mask through an attention module composed of two cascaded convolutional layers and a Softmax layer. The multiplication result of the watermark information and the first attention mask is then spliced ​​with the original video along the channel dimension. The splicing result is reduced in dimension by two layers of convolution to obtain a 3-channel residual mask. The residual mask is weighted with the original video to obtain the final watermarked video, thereby generating a watermarked video.

3. The robust video watermarking method based on frame domain attack training as claimed in claim 1, characterized in that: The noise attack module is used to take the watermark video as input, simulate the noise attack on the watermark video, and generate a distorted video; The noise attack types include cropping attack, scaling attack, compression attack, Gaussian noise attack, Dropout attack, occlusion attack, frame duplication attack and frame deletion attack. When performing noise attack on a single watermarked video, different noise types need to be randomly combined.

4. The robust video watermarking method based on frame domain attack training as claimed in claim 1, characterized in that: The generative adversarial module includes a generator and a discriminator. The generator is used to perform a watermark removal attack on the watermarked video to generate a distorted video; the discriminator is used to discriminate whether the watermarked video and the original video contain watermark information, and output the probability scores of the two videos carrying the watermark information respectively.

5. The robust video watermarking method based on frame domain attack training as claimed in claim 4, characterized in that: The discriminator is formed by cascading a convolutional layer and a classification layer; the generator is formed by cascading a convolutional layer and a residual connection layer.

6. The robust video watermarking method based on frame domain attack training as claimed in claim 1, characterized in that: The decoder module takes different distorted videos as input. Each distorted video first generates a second attention mask through an attention module composed of two convolutional layers and a Softmax layer in cascade. At the same time, each distorted video obtains a feature map of the same dimension as the second attention mask after two layers of convolution. The second attention mask is multiplied by the feature map and average pooling is performed in the frame dimension. An average value is obtained for each channel, and the average values ​​of all channels are arranged in order as the extraction result of watermark information.

7. A robust video watermarking system based on frame domain attack training, characterized in that include: A storage module, used for storing the trained encoder module and the trained decoder module obtained according to the robust video watermarking method based on frame domain attack training according to any one of claims 1 to 6; A watermark generation module is used for calling the trained encoder module to embed target watermark information into a video frame to obtain a watermarked video for a target video; The watermark extraction module is used to extract watermark information from the watermark video by calling the trained decoder module.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the robust video watermarking method based on frame domain attack training as claimed in any one of claims 1 to 6 can be implemented.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the robust video watermarking method based on frame domain attack training as claimed in any one of claims 1 to 6 can be implemented.

10. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to implement the robust video watermarking method based on frame domain attack training as claimed in any one of claims 1 to 6 when executing the computer program.

Images