Video watermarking method based on reversible neural network

Abstract

The invention discloses a video watermarking method based on a reversible neural network, and the method constructs a watermark embedding extraction network which comprises a watermark embedding module and a watermark extraction module which share network parameters, and a distortion module which is used for simulating an HEVC compression effect. Distortion layers in the watermark embedding module, the watermark extraction module and the distortion module are all based on a reversible neural network; pre-training the distortion layer based on the pre-training set, fixing network parameters of the distortion layer after the pre-training of the distortion layer is completed, training the watermark embedding extraction network based on the training set, and obtaining a trained watermark embedding extraction network after the training of the watermark embedding extraction network is completed; during testing, a test set is input into the watermark embedding module to obtain a watermark video, HEVC compression is performed on the watermark video to obtain an HEVC compressed video, and the HEVC compressed video is input into the watermark extraction module to obtain extraction end watermark information; the method has the advantages of being large in watermark embeddable capacity, good in watermark robustness and high in watermark extraction accuracy.

Description

Technical Field

[0001] The invention relates to a video watermarking technology, in particular to a video watermarking method based on a reversible neural network. Background Art

[0002] With the widespread use of multimedia digital devices, video has become the main carrier of information transmission, which has had a profound impact on the way people obtain information. However, the widespread dissemination of digital video content has also brought security issues such as copyright protection and information tampering. Therefore, video watermarking technology has been widely used, including but not limited to copyright protection, content authentication, broadcast monitoring, etc. In terms of copyright protection, video watermarks can identify the copyright owner of the video to prevent unauthorized copying and distribution. In terms of content authentication, video watermarks can verify the authenticity and integrity of the video to prevent tampering and forgery.

[0003] Traditional video watermarking technology has played an important role in copyright protection and information traceability. However, with the rapid development of artificial intelligence technology, especially the popularization of deep learning, higher requirements are placed on the robustness and transparency of video watermarking technology. With the introduction of deep learning technology, video watermarking technology has made breakthroughs in embedding, extraction and adversarial improvement through adaptive learning of complex data features, enhancing the robustness and concealment of video watermarks, and being able to effectively resist information tampering in complex scenes. However, compared with the research on watermarking technology in the field of images, due to the complexity of videos, robust watermarking for videos is still in its infancy. Existing research mainly relies on irreversible neural network frameworks such as generative adversarial networks (GANs). Although the encoder and decoder of such networks are trained simultaneously, they lack good coupling, resulting in the encoder embedding more features that the decoder cannot extract, thereby limiting the accuracy of the network. In addition, the existing video watermarking technology can only embed watermarks of smaller capacity, that is, the length of the watermark information is very limited, and it is usually impossible to simultaneously include multiple data such as author information, copyright statement and version number; and there is currently a lack of video watermarking work combined with reversible neural networks (INNs). Therefore, it is of great practical significance to study a method for video watermark embedding and extraction combined with reversible neural networks. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a video watermarking method based on a reversible neural network, which has a large watermark embedding capacity, good watermark robustness and high watermark extraction accuracy.

[0005] The technical solution adopted by the present invention to solve the above technical problems is: a video watermarking method based on a reversible neural network, characterized by comprising the following steps:

[0006] Step 1: Select an original data set, which includes Num original videos; then preprocess each original video in the original data set and the original watermark randomly generated for each original video to obtain Num video clip samples and the embedding end watermark information corresponding to each video clip sample, and form a first preprocessed data set; then divide the first preprocessed data set into a training set and a test set;

[0007] Compress Num video clip samples through HEVC to obtain corresponding compressed videos as labels; then form a second preprocessed data set with the Num video clip samples and the labels corresponding to each video clip sample, and divide the second preprocessed data set into a pre-training set and a pre-test set;

[0008] Step 2: Build a watermark embedding and extraction network, which includes a watermark embedding module and a watermark extraction module that share network parameters, as well as a distortion module used to simulate the HEVC compression effect;

[0009] The watermark embedding module is based on a reversible neural network, which is composed of N1 first reversible neural blocks with the same structure connected in sequence. The video clip sample and its corresponding embedding end watermark information are forward input into the watermark embedding module. The watermark embedding module embeds the embedding end watermark information into the video clip sample through the N1 first reversible neural blocks. The watermark embedding module outputs the watermark video and redundant information.

[0010] The distortion module includes a DWT operation, a distortion layer composed of N2 second reversible neural blocks with the same structure connected in sequence based on a reversible neural network, and an IDWT operation. After the watermark video is subjected to the DWT operation, a low-frequency region and a high-frequency region composed of high-frequency parts with the same three dimensions are obtained. The low-frequency region and the high-frequency region are forwardly input into the distortion layer. The distortion layer compresses the low-frequency region and the high-frequency region through N2 second reversible neural blocks. The distortion layer outputs the compressed low-frequency region and the compressed high-frequency region. The compressed low-frequency region and the compressed high-frequency region are subjected to the IDWT operation to obtain a distorted video.

[0011] The watermark extraction module is based on a reversible neural network, which consists of N3 third reversible neural blocks with the same structure connected in sequence. The distorted video and a full zero vector with the same size as the watermark information at the embedding end are reversely input into the watermark extraction module. The watermark extraction module extracts the watermark from the distorted video as the watermark information at the extraction end through the N3 third reversible neural blocks. The watermark extraction module outputs the restored video and the watermark information at the extraction end.

[0012] Step 3: Based on the pre-training set, pre-train the distortion layer; after the distortion layer pre-training is completed, the network parameters of the distortion layer are fixed, and then the watermark embedding extraction network is trained based on the training set, and after the watermark embedding extraction network training is completed, a trained watermark embedding extraction network is obtained;

[0013] Step 4: Forward input each video clip sample in the test set and its corresponding embedded watermark information into the watermark embedding module of the trained watermark embedding and extraction network. The watermark embedding module outputs a watermarked video. Then, perform HEVC compression on the watermarked video to obtain a HEVC compressed video. Next, use the HEVC compressed video as the distorted video and input it together with a zero vector in the reverse direction into the watermark extraction module of the trained watermark embedding and extraction network. The watermark extraction module outputs the watermark information at the extraction end.

[0014] In the said Step 1, the process of preprocessing each original video is as follows: Randomly extract T consecutive video frames along the time axis from this original video. Then, randomly intercept a region of size W×H in each of the randomly extracted video frames. The T regions consecutive along the time axis form a video clip sample. Among them, the number of channels of the video clip sample is 3T, the width is W, and the height is H.

[0015] In the said Step 1, the process of preprocessing each original watermark is as follows: Pass this original watermark through a linear layer to obtain a watermark feature of size ; then reshape the watermark feature to obtain the embedded watermark information of size where the number of channels of the embedded watermark information is 1, the width is and the height is

[0016] In the said Step 2, the input of the i-th first invertible neural block is and The corresponding output is and where i = 1, 2, …, N1. When i = 1, represents a video clip sample input into the watermark embedding module, represents the corresponding embedded watermark information. When 1 < i ≤ N1, represents the video feature containing the watermark output by the (i - 1)-th first invertible neural block, represents the watermark feature of the fused video output by the (i - 1)-th first invertible neural block. When 1 ≤ i < N1, represents the video feature containing the watermark output by the i-th first invertible neural block, represents the watermark feature of the fused video output by the i-th first invertible neural block. When i = N1, represents the watermarked video output by the watermark embedding module, represents the redundant information output by the watermark embedding module, represents the dot product operation, exp(·) represents the exponential function with the natural constant e as the base, and Ui (·) represents the upsampling module in the i-th first reversible neural block, D 1 i (·) represents the first downsampling module in the i-th first reversible neural block, D 2 i (·) represents the second downsampling module in the i-th first reversible neural block, and have 3T channels, width W, and height H, and have 1 channel, width height U i (·) has 1 channel in the input, width height U i (·) has 1 channel in the output, width W, and height H, D 1 i (·) and D 2 i (·) have 3T channels in the input, width W, and height H, D 1 i (·) and D 2 i (·) have 1 channel in the output, width height

[0017] In step 2, the input of the k-th third reversible neural block is z k+1 and x noise k+1 and the corresponding output is z k and x noise k , x noise k = x noise k+1 - U k (z k ), where k = N3, N3 - 1, …, 2, 1. When k = N3, z k+1 represents the all-zero vector input to the watermark extraction module, x noise k+1 represents the distorted video input to the watermark extraction module. When 1 ≤ k < N3, z k+1 represents the watermark feature of the fused distorted video input to the k-th third reversible neural block, x noise k+1 represents the distorted video feature containing the watermark input to the k-th third reversible neural block. When 1 < k ≤ N3, z krepresents the watermark feature of the fused distorted video output by the kth third reversible neural block, x noise k represents the distorted video feature containing the watermark output by the kth third reversible neural block, when k=1, z k represents the watermark information of the extraction end output by the watermark extraction module, x noise k represents the restored video output by the watermark extraction module, U k (·) denotes the upsampling module in the kth third reversible neural block, D 1 k (·) represents the first downsampling module in the kth third reversible neural block, D 2 k (·) represents the second downsampling module in the kth third reversible neural block, x noise k+1 and x noise k The number of channels is 3T, the width is W, the height is H, z k+1 and z k The number of channels is 1 and the width is Gao Wei U k The input channel number of (·) is 1 and the width is Gao Wei U k The output of (·) has 1 channel, W width, H height, D 1 k (·) and D 2 k The number of input channels of (·) is 3T, the width is W, the height is H, and D 1 k (·) and D 2 k The output of (·) has 1 channel and a width of Gao Wei

[0018] The upsampling modules in all the first reversible neural blocks and the upsampling modules in all the third reversible neural blocks have the same structure, including the first convolution layer, the first upsampling layer, the second upsampling layer, the third upsampling layer, and the second convolution layer connected in sequence, the input end of the first convolution layer is the input end of the upsampling module in which it is located, and the output end of the second convolution layer is the output end of the upsampling module in which it is located, the first convolution layer expands the number of channels of the features it receives from 1 to 32, each upsampling layer simultaneously expands the width and height of the features it receives by 2 times, and the second convolution layer compresses the number of channels of the features it receives from 32 to 1.

[0019] The structures of the first downsampling module and the second downsampling module in all the first reversible neural blocks, and the first downsampling module and the second downsampling module in all the third reversible neural blocks are the same, including a third convolutional layer, a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth convolutional layer connected in sequence. The input end of the third convolutional layer is the input end of the downsampling module where it is located, and the output end of the fourth convolutional layer is the output end of the downsampling module where it is located. The third convolutional layer expands the number of channels of the received features from 24 to 32, each downsampling layer reduces the width and height of the received features by 2 times simultaneously, and the fourth convolutional layer compresses the number of channels of the received features from 32 to 1.

[0020] In step 2, the input of the m-th second reversible neural block is l m and h m and the corresponding output is l m+1 and h m+1 , l m+1 = l m + η(ε(F(h m (j)))), where m = 1, 2, …, N2. When m = 1, l m represents the low-frequency region input to the distortion layer, H m represents the high-frequency region input to the distortion layer. When 1 < m ≤ N2, l m represents the low-frequency features containing high frequencies output by the (m - 1)-th second reversible neural block, h m represents the high-frequency features containing low frequencies output by the (m - 1)-th second reversible neural block. When 1 ≤ m < N2, l m+1 represents the low-frequency features containing high frequencies output by the m-th second reversible neural block, h m+1 represents the high-frequency features containing low frequencies output by the m-th second reversible neural block. When m = N2, l m+1 represents the compressed low-frequency region output by the distortion layer, h m+1 represents the compressed high-frequency region output by the distortion layer, h m (j) represents the j-th high-frequency part of h m , h m+1 (j) represents the j-th high-frequency part of h m+1 . j = 1, 2, 3. ε(·) represents the concatenation operation on the three high-frequency parts, η(·) represents a convolutional layer with a convolutional kernel size of 3×3, σ(·) represents the sigmoid activation function, and F(·), E(·), and G(·) are all functions learned by the distortion layer and are all composed of Dense modules.

[0021] The Dense module includes the 5th convolutional layer, the 6th convolutional layer, the 7th convolutional layer, the 8th convolutional layer, and the 9th convolutional layer connected in sequence. The input end of the 5th convolutional layer is the input end of the Dense module to which it belongs, and the output end of the 9th convolutional layer is the output end of the Dense module to which it belongs. The input end of the 5th convolutional layer is jump-connected to the output ends of the 5th convolutional layer, the output ends of the 6th convolutional layer, the output ends of the 7th convolutional layer, and the output ends of the 8th convolutional layer, and the input end of the 6th convolutional layer is jump-connected to the output ends of the 6th convolutional layer, the output ends of the 7th convolutional layer, and the output ends of the 8th convolutional layer. The output end of the 6th to 9th convolutional layers is 3T+32, 3T+32×2, 3T+32×3, and 3T+32×4, respectively, and the number of input channels of the 5th convolutional layer and the output channel number of the 9th convolutional layer is 3T.

[0022] In step 3, the process of pre-training the distortion layer is:

[0023] Step 3.a1: Perform a DWT operation on each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of each video clip sample; perform a DWT operation on the label corresponding to each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of the label corresponding to each video clip sample;

[0024] Step 3.a2: Initialize the weights, learning rate and network parameters of the distortion layer;

[0025] Step 3.a3: forward input the low-frequency region and the high-frequency region of each video clip sample into the distortion layer, and the distortion layer outputs the compressed low-frequency region and the compressed high-frequency region;

[0026] Step 3.a4: The compressed low-frequency region and the compressed high-frequency region corresponding to each video clip sample are subjected to IDWT operation to obtain a distorted video;

[0027] Step 3.a5: The low-frequency region and the high-frequency region of the label corresponding to each video clip sample are reversely input into the distortion layer, and the distortion layer outputs the uncompressed low-frequency region and the uncompressed high-frequency region;

[0028] Step 3.a6: Perform an IDWT operation on each uncompressed low-frequency region and the corresponding uncompressed high-frequency region to obtain an uncompressed video;

[0029] Step 3.a7: Calculate the loss L of the distorted video and the corresponding label corresponding to each video clip sample noise , L noise =MSE(v noise ,y com )+MSE(y rev ,v origin ), where MSE(·) represents the mean square error loss function, v origin Represents a video clip sample, v noise represents the distorted video corresponding to the video clip sample, y rev The label corresponding to the video clip sample is reversed and input into the distortion layer to obtain the uncompressed video. y com represents the label corresponding to the video clip sample, DWT(·) represents discrete wavelet transform, IDWT(·) represents inverse discrete wavelet transform, represents the forward input distortion layer, represents the reverse input distortion layer;

[0030] Step 3.a8: Back propagate and update the gradient; then iterate the process from step 3.a3 to step 3.a7 until the mean of the loss function of the distorted video and the corresponding label corresponding to all video clip samples in the pre-training set converges, completing the distortion layer pre-training.

[0031] In step 3, the process of training the watermark embedding extraction network is as follows:

[0032] Step 3.b1: Initialize the weights, learning rate and network parameters of the watermark embedding extraction network;

[0033] Step 3.b2: Input each video clip sample in the training set and the corresponding embedding end watermark information, as well as the all-zero vector, into the watermark embedding extraction network, and the watermark embedding extraction network outputs the extraction end watermark information;

[0034] Step 3.b3: Calculate the total network loss L total , L t0tal =λ1L video +λ2L message +λ3L dis , where λ1, λ2 and λ3 are used to balance L video , L message and L dis Contribution weight parameter, L video represents the video loss used to quantify the mean square error between the video clip sample and the watermark video output by the watermark embedding module, L video=MSE(c,s), where MSE(·) represents the mean square error loss function, c represents the video clip sample, s represents the watermarked video output by the watermark embedding module, and L message represents the message loss used to quantify the mean square error between the watermark information at the embedding end and the watermark information at the extraction end output by the watermark extraction module, L message =MAE(M,M ex ), L represents the watermark information at the embedding end, M ex represents the watermark information output by the watermark extraction module, L dis represents the adversarial loss, L dis =CrossEntropy(dis(s),label), CrossEntropy(·) represents the cross entropy loss function, dis(·) represents the neural network structure discriminator used to determine whether the watermarked video is watermarked, label represents an all-1 vector label;

[0035] Step 3.b4: back propagate and update the gradient; then iterate the process from step 3.b2 to step 3.b3, set λ3 to 0 in the first 20 iterations, and set λ3 to non-zero in subsequent iterations; end the iteration process when the mean of all network total losses for the training set converges, and complete the watermark embedding extraction network training.

[0036] Compared with the prior art, the advantages of the present invention are:

[0037] The method of the present invention constructs a watermark embedding module and a watermark extraction module based on a reversible neural network, solves the problem of small watermark embedding capacity, avoids the problem of uncoupling between the watermark embedding module and the watermark extraction module, and improves the accuracy of watermark extraction; the method of the present invention also designs a differentiable HEVC video compression simulator as a distortion layer, which solves the problem of poor robustness of watermark to video compression. Through the above design, the constructed watermark embedding and extraction network shows robustness to operations such as video compression, frame exchange, and frame loss.

[0038] The method of the present invention uses a neural network structure discriminator for judging whether a watermarked video has a watermark when calculating the adversarial loss in the training phase, thereby improving the quality of the video after embedding the watermark. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the composition structure of the watermark embedding and extraction network constructed for the present invention;

[0040] Figure 2 A schematic diagram of the composition structure of the up-sampling module in the watermark embedding and extraction network constructed for the present invention;

[0041] Figure 3A schematic diagram of the composition structure of the down-sampling module in the watermark embedding and extraction network constructed for the present invention;

[0042] Figure 4 A schematic diagram of the composition structure of the distortion layer in the watermark embedding and extraction network constructed for the present invention;

[0043] Figure 5 A schematic diagram of the composition structure of the Dense module in the distortion layer of the watermark embedding extraction network constructed for the present invention. DETAILED DESCRIPTION

[0044] The present invention is further described in detail below with reference to the accompanying drawings.

[0045] The present invention proposes a video watermarking method based on a reversible neural network, which comprises the following steps:

[0046] Step 1: Select an original data set, which contains Num original videos; then preprocess each original video in the original data set and the original watermark randomly generated for each original video to obtain Num video clip samples and the embedded end watermark information corresponding to each video clip sample, and form a first preprocessed data set; then divide the first preprocessed data set into a training set and a test set. In this embodiment, the ratio of the training set to the test set is 8:2.

[0047] Num video clip samples are compressed by HEVC to obtain corresponding compressed videos as labels; then the Num video clip samples and the labels corresponding to each video clip sample constitute a second preprocessed data set, and the second preprocessed data set is divided into a pre-training set and a pre-test set. In this embodiment, the ratio of the pre-training set to the pre-test set is 8:2, and the pre-test set is only used to verify the effectiveness of the distortion layer.

[0048] It is further defined that the process of preprocessing each original video is as follows: randomly select T video frames that are continuous along the time axis from the original video; then randomly intercept a region of size W×H from each randomly selected video frame, and the T regions that are continuous along the time axis constitute a video clip sample, wherein the video clip sample has 3T channels, a width of W, and a height of H. In this embodiment, T is 8, and W×H is 128×128; in specific implementation, the position of the starting frame when randomly selecting continuous video frames from all original videos can be fixed, such as continuously selecting 8 video frames starting from the 5th frame, and the position of the intercepted region in the randomly selected video frame can also be fixed, such as intercepting the central region.

[0049] Further, the process of preprocessing each original watermark is as follows: the original watermark is passed through a linear layer to obtain a size of The watermark feature is reshaped to obtain a watermark of size The embedded end watermark information, where the number of channels of the embedded end watermark information is 1 and the width is Gao Wei In this embodiment, a 96-bit original watermark is used; the linear layer and reshape technology both adopt the existing technology.

[0050] Step 2: Build a watermark embedding extraction network, such as Figure 1 As shown, it includes a watermark embedding module (embed watermark) and a watermark extraction module (extract watermark) that share same network parameters, and a distortion module used to simulate the HEVC compression effect.

[0051] The watermark embedding module is based on a reversible neural network, which consists of N1 first reversible neural blocks with the same structure connected in sequence. Figure 1 The cover video c) and its corresponding embedding end watermark information ( Figure 1 The watermark M shown in FIG is forwarded to the watermark embedding module, which embeds the embedding end watermark information into the video clip sample through N1 first reversible neural blocks. The watermark embedding module outputs the watermark video ( Figure 1 The watermark video s) and redundant information ( Figure 1 The lost information r shown in .

[0052] The distortion module includes a DWT (discrete wavelet transform) operation, a distortion layer based on a reversible neural network consisting of N2 second reversible neural blocks with the same structure connected in sequence, and an IDWT (inverse discrete wavelet transform) operation to convert the watermarked video ( Figure 1 After the DWT operation, the watermark video s shown in FIG. 1 is subjected to the DWT operation to obtain a low-frequency region and a high-frequency region composed of high-frequency parts with the same three dimensions, realizing the conversion from the pixel domain to the transform domain, and the low-frequency region and the high-frequency region are forwardly input into the distortion layer, as shown in FIG. Figure 4 As shown, the distortion layer compresses the low-frequency area and the high-frequency area through N2 second reversible neural blocks, and the distortion layer outputs the compressed low-frequency area and the compressed high-frequency area. The compressed low-frequency area and the compressed high-frequency area are subjected to IDWT operation to obtain the distorted video ( Figure 1 The distorted video s shown in noise). In order to enhance the robustness of the watermark, especially for non-differentiable operations such as HEVC compression, the present invention designs a differentiable distortion layer to simulate the compression effect of HEVC. The distortion layer also selects a reversible neural network as the backbone network, but makes necessary adjustments to its internal structure to better simulate the impact of HEVC compression on the watermark. It is worth noting that the distortion layer needs to be pre-trained before embedding and extracting the watermark. Through the above design, the watermark embedding and extraction network can not only resist HEVC compression, but also maintain robustness in other attacks that may cause changes in video content.

[0053] The watermark extraction module is based on a reversible neural network, which consists of N3 third reversible neural blocks with the same structure connected in sequence. Figure 1 The distorted video s shown in noise ) and an all-zero vector (auxiliary variable z) of the same size as the embedding end watermark information are reversely input into the watermark extraction module. The watermark extraction module extracts the watermark from the distorted video as the extraction end watermark information through N3 third reversible neural blocks. The watermark extraction module outputs the restored video ( Figure 1 recover video) and extraction end watermark information ( Figure 1 The extractwatermark M shown in ex ). Here, in order to achieve blind extraction, that is, to successfully extract the watermark without other prior conditions, in the watermark extraction process, the watermark extraction module uses the all-zero vector as auxiliary information for extraction.

[0054] In this embodiment, N1, N2, and N3 have the same value, that is, N1=N2=N3=16.

[0055] In fact, the watermark embedding module and the watermark extraction module are modules with the same structure. Therefore, in actual design, it is sufficient to design two identical modules. When the forward input is used, watermark embedding is realized, and the module serves as a watermark embedding module; when the reverse input is used, watermark extraction is realized, and the module serves as a watermark extraction module.

[0056] In this embodiment, in step 2, each first reversible neural block in the watermark embedding module is a forward encoding process, and the input of the i-th first reversible neural block is and The corresponding output is and Where i=1,2,…,N1, when i=1 represents a video clip sample input to the watermark embedding module, express The corresponding embedded - side watermark information, when 1 < i ≤ N1 represents the watermark - containing video features output by the (i - 1) - th first reversible neural block, represents the watermark features of the fused video output by the (i - 1) - th first reversible neural block, when 1 ≤ i < N1 represents the watermark - containing video features output by the i - th first reversible neural block, represents the watermark features of the fused video output by the i - th first reversible neural block, when i = N1 represents the watermark video output by the watermark embedding module, represents the redundant information output by the watermark embedding module, represents the dot - product operation, exp(·) represents the exponential function with the natural constant e as the base, e = 2.71, U i (·) represents the up - sampling module in the i - th first reversible neural block, D 1 i (·) represents the first down - sampling module in the i - th first reversible neural block, D 2 i (·) represents the second down - sampling module in the i - th first reversible neural block, and have 3T channels, width W, and height H, and have 1 channel, width height U i (·) has an input with 1 channel, width height U i (·) has an output with 1 channel, width W, and height H, D 1 i (·) and D 2 i (·) have an input with 3T channels, width W, and height H, D 1 i (·) and D 2 i (·) have an output with 1 channel, width height In it will automatically expand the number of channels to 3T, which is the broadcasting mechanism. In the first reversible neural block, the up - sampling module and the two down - sampling modules are responsible for sampling the watermark information and the video information to the same dimension for fusion.

[0057] In this embodiment, in step 2, in the watermark extraction module, the reverse input is from the last third reversible neural block, and the input of the k - th third reversible neural block is zk+1 and x noise k+1 The corresponding output is z k and x noise k , x noise k = x noise k+1 -U k (z k ), where k = N3, N3 - 1, …, 2, 1. When k = N3, z k+1 represents the all-zero vector input to the watermark extraction module, x noise k+1 represents the distorted video input to the watermark extraction module. When 1 ≤ k < N3, z k+1 represents the watermark feature of the fused distorted video input to the k-th third reversible neural block, x noise k+1 represents the distorted video feature containing the watermark input to the k-th third reversible neural block. When 1 < k ≤ N3, z k represents the watermark feature of the fused distorted video output from the k-th third reversible neural block, x noise k represents the distorted video feature containing the watermark output from the k-th third reversible neural block. When k = 1, z k represents the watermark information at the extraction end output by the watermark extraction module, x noise k represents the restored video output by the watermark extraction module, U k (·) represents the upsampling module in the k-th third reversible neural block, D 1 k (·) represents the first downsampling module in the k-th third reversible neural block, D 2 k (·) represents the second downsampling module in the k-th third reversible neural block, x noise k+1 and x noise k have 3T channels, width W, and height H. z k+1 and z k have 1 channel, width height U k (·) has an input with 1 channel, width height U k (·) has an output with 1 channel, width W, and height H. D 1 k (·) and D 2k (·) has an input channel number of 3T, a width of W, and a height of H, D 1 k (·) and D 2 k (·) has an output channel number of 1, a width of a height of At x noise k = x noise k+1 - U k (z k ) where U k (z k ) will automatically expand the channel number to 3T.

[0058] In this embodiment, the upsampling modules in all the first reversible neural blocks and the upsampling modules in all the third reversible neural blocks have the same structure. As Figure 2 shown, it includes a first convolutional layer, a first upsampling layer, a second upsampling layer, a third upsampling layer, and a second convolutional layer connected in sequence. The input end of the first convolutional layer is the input end of the upsampling module where it is located, and the output end of the second convolutional layer is the output end of the upsampling module where it is located. The first convolutional layer expands the channel number of the received feature from 1 to 32. Each upsampling layer doubles the width and height of the received feature simultaneously. The second convolutional layer compresses the channel number of the received feature from 32 to 1.

[0059] In this embodiment, the first downsampling module and the second downsampling module in all the first reversible neural blocks, and the first downsampling module and the second downsampling module in all the third reversible neural blocks have the same structure. As Figure 3 shown, it includes a third convolutional layer, a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth convolutional layer connected in sequence. The input end of the third convolutional layer is the input end of the downsampling module where it is located, and the output end of the fourth convolutional layer is the output end of the downsampling module where it is located. The third convolutional layer expands the channel number of the received feature from 24 to 32. Each downsampling layer halves the width and height of the received feature simultaneously. The fourth convolutional layer compresses the channel number of the received feature from 32 to 1.

[0060] In this embodiment, in step 2, as Figure 4 shown, the input of the m-th second reversible neural block is l m and h m , and the corresponding output is l m+1 and h m+1 , l m+1 = l m + η(ε(F(h m(j)))) where m = 1, 2, …, N2, and when m = 1, l m represents the low-frequency region input to the distortion layer, h m represents the high-frequency region input to the distortion layer. When 1 < m ≤ N2, l m represents the low-frequency features containing high frequencies output by the (m - 1)-th second reversible neural block, h m represents the high-frequency features containing low frequencies output by the (m - 1)-th second reversible neural block. When 1 ≤ m < N2, l m+1 represents the low-frequency features containing high frequencies output by the m-th second reversible neural block, h m+1 represents the high-frequency features containing low frequencies output by the m-th second reversible neural block. When m = N2, l m+1 represents the compressed low-frequency region output by the distortion layer, h m +1 represents the compressed high-frequency region output by the distortion layer, h m (j) represents h m the j-th high-frequency part of, h m+1 (j) represents h m+1 the j-th high-frequency part of, where j = 1, 2, 3. ε(·) represents the concatenation operation on the three high-frequency parts. η(·) represents a convolutional layer with a convolution kernel size of 3×3, aiming to transform the concatenated part to the same dimension as the low-frequency region for convenient subsequent operations. σ(·) represents the sigmoid activation function. F(·), E(·), and G(·) are all functions learned by the distortion layer and are all composed of Dense modules. Since video compression mainly targets the high-frequency region of the video, the change in the low-frequency region of the video is relatively small. Based on this, the distortion layer simulates the compression of the low-frequency region by HEVC by continuously adding perturbations to the original low-frequency region, while for the high-frequency region, a more complex method is used for simulation.

[0061] such as Figure 5As shown, the Dense module includes the 5th convolutional layer, the 6th convolutional layer, the 7th convolutional layer, the 8th convolutional layer, and the 9th convolutional layer connected in sequence. The input end of the 5th convolutional layer is the input end of the Dense module to which it belongs, and the output end of the 9th convolutional layer is the output end of the Dense module to which it belongs. The input end of the 5th convolutional layer is jump-connected to the output ends of the 5th convolutional layer, the output ends of the 6th convolutional layer, the output ends of the 7th convolutional layer, and the output ends of the 8th convolutional layer, and the input end of the 6th convolutional layer is jump-connected to the output ends of the 6th convolutional layer, the output ends of the 7th convolutional layer, and the output ends of the 8th convolutional layer. The output end of the 6th to 9th convolutional layers is 3T+32, 3T+32×2, 3T+32×3, and 3T+32×4, respectively, and the input end of the 5th convolutional layer and the output end of the 9th convolutional layer are 3T.

[0062] Step 3: Based on the pre-training set, pre-train the distortion layer; after the pre-training of the distortion layer is completed, fix the network parameters of the distortion layer, and then train the watermark embedding extraction network based on the training set, and obtain the trained watermark embedding extraction network after the training of the watermark embedding extraction network is completed.

[0063] After the pre-training of the distortion layer is completed, in order to ensure the stability of the distortion layer and the convergence of the watermark embedding and extraction network, all network parameters of the distortion layer need to be fixed, so that the distortion layer can provide a stable distortion effect in the subsequent watermark embedding and extraction process.

[0064] Preferably, the process of pre-training the distortion layer is:

[0065] Step 3.a1: Perform a DWT operation on each video clip sample in the pre-training set to obtain the low-frequency area and high-frequency area of ​​each video clip sample; perform a DWT operation on the label corresponding to each video clip sample in the pre-training set to obtain the low-frequency area and high-frequency area of ​​the label corresponding to each video clip sample.

[0066] Step 3.a2: Initialize the weights, learning rate, and network parameters of the distortion layer.

[0067] Step 3.a3: The low-frequency region and the high-frequency region of each video clip sample are forward-inputted into the distortion layer, and the distortion layer outputs a compressed low-frequency region and a compressed high-frequency region.

[0068] Step 3.a4: The compressed low-frequency region and the compressed high-frequency region corresponding to each video clip sample are subjected to IDWT operation to obtain a distorted video.

[0069] Step 3.a5: The low-frequency region and the high-frequency region of the label corresponding to each video clip sample are reversely input into the distortion layer, and the distortion layer outputs the uncompressed low-frequency region and the uncompressed high-frequency region.

[0070] Step 3.a6: Perform an IDWT operation on each uncompressed low-frequency region and the corresponding uncompressed high-frequency region to obtain an uncompressed video.

[0071] Step 3.a7: Calculate the loss L of the distorted video and the corresponding label corresponding to each video clip sample noise , L noise =MSE(v noise ,y com )+MSE(y rev ,v origin ), where MSE(·) represents the mean square error loss function, v origin Represents a video clip sample, v noise represents the distorted video corresponding to the video clip sample, y rev The label corresponding to the video clip sample is reversed and input into the distortion layer to obtain the uncompressed video. y com represents the label corresponding to the video clip sample, DWT(·) represents discrete wavelet transform, IDWT(·) represents inverse discrete wavelet transform, represents the forward input distortion layer, Represents the reverse input distortion layer.

[0072] Step 3.a8: Back propagate and update the gradient; then iterate the process from step 3.a3 to step 3.a7 until the mean of the loss function of the distorted video and the corresponding label corresponding to all video clip samples in the pre-training set converges, completing the distortion layer pre-training.

[0073] In the pre-training process of the distortion layer, it is necessary to imitate HEVC compression as much as possible, so in the forward input process, the distorted video v noise With label y com As similar as possible. Accordingly, in the reverse input process, the desired uncompressed video y rev With the video clip sample v origin As close as possible, so a loss function L is defined noiseTo minimize the mean square error between the distorted video and the corresponding label corresponding to the video clip samples in the pre-training set. During the pre-training process, the learning rate decay and network parameter regularization techniques can be used to make the distortion layer pre-training more stable.

[0074] As a preferred method, the process of training the watermark embedding extraction network is:

[0075] Step 3.b1: Initialize the weights, learning rate and network parameters of the watermark embedding extraction network.

[0076] Step 3.b2: Input each video clip sample in the training set and the corresponding embedding end watermark information, as well as the all-zero vector into the watermark embedding extraction network, and the watermark embedding extraction network outputs the extraction end watermark information.

[0077] Step 3.b3: Calculate the total network loss L total , the total network loss L total It consists of three parts: In order to achieve the invisibility of the watermark, the video loss L is defined video ; In order to losslessly extract the watermark from the distorted video in the watermark extraction module, the message loss L is defined message In order to improve the invisibility of watermarks and enhance the image quality of videos, the present invention introduces a more complex discriminator to achieve the goal and defines the adversarial loss L dis . The total network loss L total The mathematical expression is: L total =λ1L video +λ2L message +λ3L dis , where λ1, λ2 and λ3 are used to balance L video , L message and L dis Contribution weight parameter, L video represents the video loss used to quantify the mean square error between the video clip sample and the watermark video output by the watermark embedding module, L video =MSE(c,s), where MSE(·) represents the mean square error loss function, c represents the video clip sample, s represents the watermarked video output by the watermark embedding module, and L message represents the message loss used to quantify the mean square error between the watermark information at the embedding end and the watermark information at the extraction end output by the watermark extraction module, L message =MSE(M,M ex ), M represents the watermark information of the embedding end, M ex Indicates the watermark information output by the watermark extraction module, L dis represents the adversarial loss, L dis= CrossEntropy(dis(s), label), CrossEntropy(·) represents the cross entropy loss function, dis(·) represents the neural network structure discriminator used to determine whether the watermarked video is watermarked ( Figure 1 The neural network structure discriminator adopts the existing technology. The neural network structure discriminator will output 0 or 1. The output 0 indicates that the judgment result is that the input watermarked video has a watermark, and the output 1 indicates that the judgment result is that the input watermarked video does not have a watermark. Label represents a full 1 vector label.

[0078] Step 3.b4: back propagate and update the gradient; then iterate the process from step 3.b2 to step 3.b3, set λ3 to 0 in the first 20 iterations, and set λ3 to non-zero in subsequent iterations; end the iteration process when the mean of all network total losses for the training set converges, and complete the watermark embedding extraction network training.

[0079] In the training process of the watermark embedding extraction network, a phased training strategy is adopted. In the first 20 rounds, i.e., the first 20 iterations, the focus is on minimizing the video loss L. video and message loss L message , and the adversarial loss L dis We will not consider it for now, that is, set the weight λ3 to 0, λ1 to 1, and λ2 to 10. In the subsequent iteration process, we will consider the adversarial loss L dis , where λ1 is set to 1, λ2 is set to 10, and λ3 is set to 0.0001. This phased training strategy aims to avoid the model collapse caused by the premature introduction of adversarial training and avoid the unnecessary complexity caused by it. This phased training strategy ensures that the model can learn stably in each training stage and gradually improves performance, ultimately achieving optimized training results.

[0080] Step 4: Forward each video clip sample in the test set and its corresponding embedding-end watermark information are input into the watermark embedding module of the trained watermark embedding extraction network, and the watermark embedding module outputs the watermarked video; then the watermarked video is compressed by HEVC to obtain the HEVC compressed video; then the HEVC compressed video is used as the distorted video and is reversely input into the watermark extraction module of the trained watermark embedding extraction network together with the all-zero vector, and the watermark extraction module outputs the extraction-end watermark information.

[0081] In order to further illustrate the feasibility and effectiveness of the method of the present invention, experiments were conducted on the method of the present invention.

[0082] The experimental dataset uses UCF-101 as the original dataset. This is a large-scale video dataset containing 101 action categories, with a total of 13,320 video samples, i.e. original videos. In view of the fact that there are a large number of video samples with similar scenes in the experimental dataset, 2,000 video samples with low scene similarity were carefully selected, and video clip samples were obtained after preprocessing. A part of the video clip samples constituted a pre-training set for pre-training of the distortion layer. At the same time, this part of the video clip samples constituted a training set for training the watermark embedding extraction network to improve the generalization ability and training efficiency of the watermark embedding extraction network in diverse scenarios. The remaining video clip samples constituted a test set for testing. At the same time, in order to comprehensively evaluate the performance of the watermark embedding extraction network, video samples from the two classic datasets, REDS and Kinetics 400, were used to form a test set after preprocessing for testing, in order to evaluate the recognition and processing capabilities of the watermark embedding extraction network for new data.

[0083] In order to enhance the adaptability of the watermark embedding extraction network to different video sizes and ratios, a random cropping strategy is implemented as a preprocessing process. Specifically, after the video samples are cropped, their dimensions are adjusted to 3×8×128×128, where 3 represents the number of color channels, 8 represents the number of video frames, and 128×128 represents the spatial resolution of each video frame. This random cropping method not only increases the diversity of training data, but also simulates various video sizes that may be encountered in practical applications, thereby improving the generalization ability of the watermark embedding extraction network.

[0084] In the experiment, indicators such as peak signal-to-noise ratio (PSNR), structural similarity (SSIM), learning perceptual image block similarity (LPIPS) and accuracy (ACC) are used to measure the performance of different watermarking methods in video picture quality, video perception quality and watermark extraction accuracy. Among them, higher PSNR and SSIM values ​​indicate higher video picture quality, lower LPIPS values ​​indicate better video perception quality, and higher ACC values ​​indicate that the model can extract the embedded watermark information with higher accuracy.

[0085] The comparison methods in the experiment are HiDDeN method and REVMARK method. HiDDeN method refers to the method of Zhu (J.Zhu, R.Kaplan, J.Johnson, and L.Fei-Fei, "HiDDen: hiding data with deep networks," Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 2018, pp.657-672. (hiding data with deep network)), and REVMARK method refers to the method of Zhang (Y.Zhang, J.Ni, W.Su, and X.Liao, "A novel deep video watermarking framework with enhanced robustness to H.264 / AVC compression," in Proceedings of the 31st ACMInternational Conference on Multimedia, New York, NY, USA, 2023, pp.8095-8104 (a novel deep video watermarking framework with enhanced robustness to H.264 / AVC compression)).

[0086] Table 1 shows the watermark extraction accuracy (ACC) of the existing HiDDeN method, REVMARK method and the proposed method (LRVMark) on the UCF-101 test set, Kinetics 400 dataset and REDS dataset for watermarked videos after different distortion attacks such as frame averaging, frame loss, frame swapping, Gaussian noise, H264 / AVC and HEVC. In Table 1, N refers to the number of frames in a time window, p represents the probability of frame loss (or swapping with its directly adjacent frames), std refers to standard deviation, and QP refers to quantization parameter (the larger the QP, the stronger the compression effect).

[0087] Table 1 Comparison of ACC (%) under different distortion attacks

[0088]

[0089] As can be seen from Table 1, the watermark extraction accuracy of the method of the present invention on the three data sets is better than that of the comparison method, and in the face of different distortion attacks, the watermark information can be extracted 100% in most cases. This not only shows that the method of the present invention performs well in robustness, but also verifies that the method of the present invention has good generalization ability on different data sets.

[0090] Table 2 shows the comparison of watermark video quality measured by different evaluation indicators using the existing HiDDeN method, REVMARK method and the proposed method (LRVMark) on the UCF-101 test set, Kinetics 400 dataset and REDS dataset when 96 bits of watermark information is embedded.

[0091] Table 2 Comparison of the quality of watermarked videos obtained under different watermarking methods

[0092]

[0093] It can be seen from Table 2 that the method of the present invention is relatively excellent in terms of video quality.

[0094] Table 3 shows the extraction accuracy of watermark information by the method of the present invention after different degrees of video compression when watermark information of different capacities is embedded.

[0095] Table 3 ACC (%), different compression strengths for a range of watermark lengths

[0096]

[0097] As can be seen from Table 3, it can be found that after embedding a 256-bit or even 1024-bit watermark, despite the HEVC compression process, the method of the present invention can still effectively extract the watermark information while ensuring the video quality.

Claims

1. A video watermarking method based on reversible neural network, characterized in that The following steps are involved: Step 1: Select an original data set, which contains Num original videos; then preprocess each original video in the original data set and the original watermark randomly generated for each original video to obtain Num video clip samples and the embedding end watermark information corresponding to each video clip sample, and form a first preprocessed data set; Then divide the first preprocessed data set into a training set and a test set; Compress Num video clip samples through HEVC to obtain the corresponding compressed video as a label; Then, Num video clip samples and the label corresponding to each video clip sample constitute a second preprocessed data set, and the second preprocessed data set is divided into a pre-training set and a pre-test set; Step 2: Build a watermark embedding and extraction network, which includes a watermark embedding module and a watermark extraction module that share network parameters, as well as a distortion module used to simulate the HEVC compression effect; The watermark embedding module is based on a reversible neural network, which is composed of N1 first reversible neural blocks with the same structure connected in sequence. The video clip sample and its corresponding embedding end watermark information are forward input into the watermark embedding module. The watermark embedding module embeds the embedding end watermark information into the video clip sample through the N1 first reversible neural blocks. The watermark embedding module outputs the watermark video and redundant information. The distortion module includes a DWT operation, a distortion layer composed of N2 second reversible neural blocks with the same structure connected in sequence based on a reversible neural network, and an IDWT operation. After the watermark video is subjected to the DWT operation, a low-frequency region and a high-frequency region composed of high-frequency parts with the same three dimensions are obtained. The low-frequency region and the high-frequency region are forwardly input into the distortion layer. The distortion layer compresses the low-frequency region and the high-frequency region through N2 second reversible neural blocks. The distortion layer outputs the compressed low-frequency region and the compressed high-frequency region. The compressed low-frequency region and the compressed high-frequency region are subjected to the IDWT operation to obtain a distorted video. The watermark extraction module is based on a reversible neural network, which consists of N3 third reversible neural blocks with the same structure connected in sequence. The distorted video and a full zero vector with the same size as the watermark information at the embedding end are reversely input into the watermark extraction module. The watermark extraction module extracts the watermark from the distorted video as the watermark information at the extraction end through the N3 third reversible neural blocks. The watermark extraction module outputs the restored video and the watermark information at the extraction end. Step 3: Pre-train the distortion layer based on the pre-training set; After the pre-training of the distortion layer is completed, the network parameters of the distortion layer are fixed, and then the watermark embedding extraction network is trained based on the training set. After the training of the watermark embedding extraction network is completed, a trained watermark embedding extraction network is obtained; Step 4: Each video clip sample in the test set and its corresponding embedding-end watermark information are forward-inputted into the watermark embedding module of the trained watermark embedding extraction network, and the watermark embedding module outputs the watermarked video; then the watermarked video is HEVC compressed to obtain the HEVC compressed video; The HEVC compressed video is then used as a distorted video and is reversely input into the watermark extraction module of the trained watermark embedding extraction network together with the all-zero vector. The watermark extraction module outputs the watermark information at the extraction end.

2. The video watermarking method based on reversible neural network according to claim 1 is characterized in that In step 1, the process of preprocessing each original video is as follows: randomly select T video frames that are continuous along the time axis from the original video; then randomly intercept a region of size W×H from each randomly selected video frame, and the T regions that are continuous along the time axis constitute a video clip sample, wherein the number of channels of the video clip sample is 3T, the width is W, and the height is H.

3. The video watermarking method based on reversible neural network according to claim 2 is characterized in that In step 1, the process of preprocessing each original watermark is as follows: the original watermark is passed through a linear layer to obtain a The watermark feature is then reshaped to obtain a size of The embedded end watermark information, where the number of channels of the embedded end watermark information is 1 and the width is Gao Wei 4. The video watermarking method based on reversible neural network according to claim 3 is characterized in that In step 2, the input of the i-th first reversible neural block is and The corresponding output is and Among them, \(i = 1, 2, \ldots, N_1\). When \(i = 1\), represents a video clip sample input into the watermark embedding module, represents the corresponding watermark information at the embedding end. When \(1 \lt i \leq N_1\), represents the watermark-containing video features output by the \((i - 1)\)-th first reversible neural block, represents the watermark features of the fused video output by the \((i - 1)\)-th first reversible neural block. When \(1 \leq i \lt N_1\), represents the watermark-containing video features output by the \(i\)-th first reversible neural block, represents the watermark features of the fused video output by the \(i\)-th first reversible neural block. When \(i = N_1\), represents the watermarked video output by the watermark embedding module, represents the redundant information output by the watermark embedding module, represents the dot product operation, \(\exp(\cdot)\) represents the exponential function with the natural constant \(e\) as the base, \(U\) i (\(\cdot\)) represents the upsampling module in the \(i\)-th first reversible neural block, \(D\) 1 i (\(\cdot\)) represents the first downsampling module in the \(i\)-th first reversible neural block, \(D\) 2 i (\(\cdot\)) represents the second downsampling module in the \(i\)-th first reversible neural block, and have 3T channels, width \(W\), and height \(H\), and have 1 channel, width height U i (\(\cdot\)) has an input with 1 channel, width height U i (\(\cdot\)) has an output with 1 channel, width \(W\), and height \(H\), \(D\) 1 i (\(\cdot\)) and \(D\) 2 i (\(\cdot\)) have an input with 3T channels, width \(W\), and height \(H\), \(D\) 1 i (\(\cdot\)) and \(D\) 2 i (\(\cdot\)) have an output with 1 channel, width height 5. The video watermarking method based on reversible neural network according to claim 4 is characterized in that In step 2, the input of the k-th third reversible neural block is z k+1 and x noise k+1 and the corresponding output is z k and x noise l , x noise k = x noise k+1 - U k (z k ), where k = N3, N3 - 1, …, 2, 1. When k = N3, z k+1 represents the all-zero vector input to the watermark extraction module, and x noise k+1 represents the distorted video input to the watermark extraction module. When 1 ≤ k < N3, z k+1 represents the watermark feature of the fused distorted video input to the k-th third reversible neural block, and x noise k+1 represents the distorted video feature containing the watermark input to the k-th third reversible neural block. When 1 < k ≤ N3, z k represents the watermark feature of the fused distorted video output by the k-th third reversible neural block, and x noise k represents the distorted video feature containing the watermark output by the k-th third reversible neural block. When k=1, z k represents the watermark information of the extraction end output by the watermark extraction module, x noise k represents the restored video output by the watermark extraction module, U k (·) denotes the upsampling module in the kth third reversible neural block, D 1 k (·) represents the first downsampling module in the kth third reversible neural block, D 2 k (·) represents the second downsampling module in the kth third reversible neural block, x noise k+1 and x noise k The number of channels is 3T, the width is W, the height is H, z k+1 and z k The number of channels is 1 and the width is Gao Wei U k The input channel number of (·) is 1 and the width is Gao Wei U k The output of (·) has 1 channel, W width, H height, D 1 k (·) and D 2 k The number of input channels of (·) is 3T, the width is W, the height is H, and D 1 k (·) and D 2 k The output of (·) has 1 channel and a width of Gao Wei 6. The video watermarking method based on reversible neural network according to claim 5 is characterized in that The upsampling modules in all the first reversible neural blocks and the upsampling modules in all the third reversible neural blocks have the same structure, including the first convolution layer, the first upsampling layer, the second upsampling layer, the third upsampling layer, and the second convolution layer connected in sequence, the input end of the first convolution layer is the input end of the upsampling module in which it is located, and the output end of the second convolution layer is the output end of the upsampling module in which it is located, the first convolution layer expands the number of channels of the features it receives from 1 to 32, each upsampling layer simultaneously expands the width and height of the features it receives by 2 times, and the second convolution layer compresses the number of channels of the features it receives from 32 to 1.

7. The video watermarking method based on reversible neural network according to claim 5 is characterized in that The first downsampling module and the second downsampling module in all the first reversible neural blocks, and the first downsampling module and the second downsampling module in all the third reversible neural blocks have the same structure, including the third convolutional layer, the first downsampling layer, the second downsampling layer, the third downsampling layer, and the fourth convolutional layer connected in sequence, the input end of the third convolutional layer is the input end of the downsampling module in which it is located, and the output end of the fourth convolutional layer is the output end of the downsampling module in which it is located, the third convolutional layer expands the number of channels of the features it receives from 24 to 32, each downsampling layer reduces the width and height of the features it receives by 2 times at the same time, and the fourth convolutional layer compresses the number of channels of the features it receives from 32 to 1.

8. The video watermarking method based on reversible neural network according to claim 5, 6 or 7, characterized in that In step 2, the input of the m-th second reversible neural block is l m and h m , and the corresponding output is l m+1 and h m+1 , l m+1 = l m + η(ε(F(h m (j)))), where m = 1, 2, …, N2, when m = 1, l m represents the low-frequency region input to the distortion layer, h m represents the high-frequency region input to the distortion layer. When 1 < m ≤ N2, l m represents the low-frequency features containing high frequencies output by the (m - 1)-th second reversible neural block, h m represents the high-frequency features containing low frequencies output by the (m - 1)-th second reversible neural block. When 1 ≤ m < N2, l m+1 represents the low-frequency features containing high frequencies output by the m-th second reversible neural block, h m+1 represents the high-frequency features containing low frequencies output by the m-th second reversible neural block. When m = N2, l m+1 represents the compressed low-frequency region output by the distortion layer, h m+1 represents the compressed high-frequency region output by the distortion layer. h m (j) represents the j-th high-frequency part of h m , h m+1 (j) represents the j-th high-frequency part of h m+1 , j = 1, 2, 3. ε(·) represents the concatenation operation on the three high-frequency parts. η(·) represents a convolutional layer with a convolutional kernel size of 3×3. σ(·) represents the sigmoid activation function. F(·), E(·) and G(·) are all functions learned by the distortion layer and are all composed of Dense modules.

9. The video watermarking method based on reversible neural network according to claim 1 is characterized in that In step 3, the process of pre-training the distortion layer is: Step 3.a1: Perform a DWT operation on each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of each video clip sample; perform a DWT operation on the label corresponding to each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of the label corresponding to each video clip sample; Step 3.a2: Initialize the weights, learning rate and network parameters of the distortion layer; Step 3.a3: forward input the low-frequency region and the high-frequency region of each video clip sample into the distortion layer, and the distortion layer outputs the compressed low-frequency region and the compressed high-frequency region; Step 3.a4: The compressed low-frequency region and the compressed high-frequency region corresponding to each video clip sample are subjected to IDWT operation to obtain a distorted video; Step 3.a5: The low-frequency region and the high-frequency region of the label corresponding to each video clip sample are reversely input into the distortion layer, and the distortion layer outputs the uncompressed low-frequency region and the uncompressed high-frequency region; Step 3.a6: Perform an IDWT operation on each uncompressed low-frequency region and the corresponding uncompressed high-frequency region to obtain an uncompressed video; Step 3.a7: Calculate the loss L of the distorted video and the corresponding label corresponding to each video clip sample noise , L noise =MSE(v noise ,y com )+MSE(y rev ,v origin ), where MSE(·) represents the mean square error loss function, v origin Represents a video clip sample, v noise Represents the distorted video corresponding to the video clip sample, y rev The label corresponding to the video clip sample is reversed and input into the distortion layer to obtain the uncompressed video, y com Indicates the label corresponding to the video clip sample; Step 3.a8: Back propagate and update the gradient; then iterate the process from step 3.a3 to step 3.a7 until the mean of the loss function of the distorted video and the corresponding label corresponding to all video clip samples in the pre-training set converges, completing the distortion layer pre-training.

10. The video watermarking method based on reversible neural network according to claim 1 is characterized in that In step 3, the process of training the watermark embedding extraction network is as follows: Step 3.b1: Initialize the weights, learning rate and network parameters of the watermark embedding extraction network; Step 3.b2: Input each video clip sample in the training set and the corresponding embedding end watermark information, as well as the all-zero vector, into the watermark embedding extraction network, and the watermark embedding extraction network outputs the extraction end watermark information; Step 3.b3: Calculate the total network loss L total , L total =λ1L video +λ2L message +λ3L dis , where λ1, λ2 and λ3 are used to balance L video , L message and L dis Contribution weight parameter, L video represents the video loss used to quantify the mean square error between the video clip sample and the watermark video output by the watermark embedding module, L video =MSE(c,s), where MSE(·) represents the mean square error loss function, c represents the video clip sample, s represents the watermarked video output by the watermark embedding module, and L message represents the message loss used to quantify the mean square error between the watermark information at the embedding end and the watermark information at the extraction end output by the watermark extraction module, L message =MSE(M,M ex ), M represents the watermark information of the embedding end, M ex represents the watermark information output by the watermark extraction module, L dis represents the adversarial loss, L dis =CrossEntropy(dis(s),label), CrossEntropy(·) represents the cross entropy loss function, dis(·) represents the neural network structure discriminator used to determine whether the watermarked video is watermarked, and label represents an all-1 vector label; Step 3.b4: back propagate and update the gradient; then iterate the process from step 3.b2 to step 3.b3, set λ3 to 0 in the first 20 iterations, and set λ3 to non-zero in subsequent iterations; end the iteration process when the mean of all network total losses for the training set converges, and complete the watermark embedding extraction network training.

Images