Video watermarking method based on reversible neural network

Abstract

The present invention discloses a video watermarking method based on a reversible neural network. The method constructs a watermark embedding and extraction network, comprising a watermark embedding module and a watermark extraction module that share network parameters, and a distortion module for simulating the HEVC compression effect. The watermark embedding module, the watermark extraction module and the distortion layer in the distortion module are all based on the reversible neural network. The distortion layer is pre-trained based on a pre-training set, and after the distortion layer pre-training is completed, the network parameters of the distortion layer are fixed. The watermark embedding and extraction network is then trained based on the training set, and after the watermark embedding and extraction network training is completed, a trained watermark embedding and extraction network is obtained. During testing, the test set is input into the watermark embedding module to obtain a watermarked video, HEVC compression is performed on the watermarked video to obtain an HEVC compressed video, and the HEVC compressed video is input into the watermark extraction module to obtain extraction-end watermark information. The method has the advantages of large watermark embedding capacity, good watermark robustness and high watermark extraction accuracy.

Description

Technical Field

[0001] The present invention relates to a video watermarking technology, in particular to a video watermarking method based on a reversible neural network. Background Art

[0002] With the widespread use of multimedia digital devices, video has become the primary medium for information transmission, profoundly impacting how people access information. However, the widespread dissemination of digital video content also raises security concerns such as copyright protection and information tampering. Consequently, video watermarking technology has gained widespread application, including but not limited to copyright protection, content authentication, and broadcast monitoring. In terms of copyright protection, video watermarks can identify the copyright owner of a video, preventing unauthorized copying and distribution. In terms of content authentication, video watermarks can verify the authenticity and integrity of a video, preventing tampering and forgery.

[0003] Traditional video watermarking technology has played an important role in copyright protection and information traceability. However, the rapid development of artificial intelligence (AI), particularly the prevalence of deep learning, has placed higher demands on the robustness and transparency of video watermarking technology. With the introduction of deep learning, video watermarking has achieved breakthroughs in embedding, extraction, and adversarial improvement by adaptively learning complex data features. This has enhanced the robustness and concealment of video watermarks, effectively resisting information tampering in complex scenarios. However, compared with watermarking technology research in the image field, due to the complexity of video, robust video watermarking is still in its infancy. Existing research primarily relies on irreversible neural network frameworks such as generative adversarial networks (GANs). Although the encoder and decoder of such networks are trained simultaneously, they lack good coupling, causing the encoder to embed many features that the decoder cannot extract, thereby limiting the network's accuracy. In addition, existing video watermarking technology can only embed watermarks of smaller capacity, that is, the length of the watermark information is very limited, and it is usually impossible to simultaneously include multiple data such as author information, copyright statement and version number; and there is currently a lack of video watermarking work combined with reversible neural networks (INN). Therefore, it is of great practical significance to study a method for video watermark embedding and extraction combined with reversible neural networks. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a video watermarking method based on a reversible neural network, which has a large watermark embedding capacity, good watermark robustness and high watermark extraction accuracy.

[0005] The technical solution adopted by the present invention to solve the above technical problems is: a video watermarking method based on a reversible neural network, characterized by comprising the following steps:

[0006] Step 1: Select an original dataset containing Num original videos; then preprocess each original video in the original dataset and the original watermark randomly generated for each original video to obtain Num video clip samples and the embedded end watermark information corresponding to each video clip sample, and form a first preprocessed dataset; then divide the first preprocessed dataset into a training set and a test set;

[0007] Compress Num video clip samples through HEVC to obtain corresponding compressed videos as labels; then form a second preprocessed data set with Num video clip samples and the labels corresponding to each video clip sample, and divide the second preprocessed data set into a pre-training set and a pre-test set;

[0008] Step 2: Build a watermark embedding and extraction network, which includes a watermark embedding module and a watermark extraction module that share network parameters, as well as a distortion module used to simulate the HEVC compression effect;

[0009] The watermark embedding module is based on a reversible neural network, which consists of N1 first reversible neural blocks with the same structure connected in sequence. The video clip sample and its corresponding embedding end watermark information are forward input into the watermark embedding module. The watermark embedding module embeds the embedding end watermark information into the video clip sample through N1 first reversible neural blocks. The watermark embedding module outputs the watermark video and redundant information.

[0010] The distortion module includes a DWT operation, a distortion layer composed of N2 second reversible neural blocks with the same structure connected in sequence based on a reversible neural network, and an IDWT operation. After the watermark video is subjected to the DWT operation, a low-frequency region and a high-frequency region composed of high-frequency parts with the same three dimensions are obtained. The low-frequency region and the high-frequency region are forward input into the distortion layer. The distortion layer compresses the low-frequency region and the high-frequency region through the N2 second reversible neural blocks. The distortion layer outputs the compressed low-frequency region and the compressed high-frequency region. After the compressed low-frequency region and the compressed high-frequency region are subjected to the IDWT operation, a distorted video is obtained.

[0011] The watermark extraction module is based on a reversible neural network, which consists of N3 third-party reversible neural blocks with the same structure connected in sequence. The distorted video and an all-zero vector with the same size as the watermark information at the embedding end are reversely input into the watermark extraction module. The watermark extraction module extracts the watermark from the distorted video as the extraction end watermark information through the N3 third-party reversible neural blocks. The watermark extraction module outputs the restored video and the extraction end watermark information;

[0012] Step 3: Pre-train the distortion layer based on the pre-training set; after the distortion layer pre-training is completed, the network parameters of the distortion layer are fixed, and then the watermark embedding extraction network is trained based on the training set. After the watermark embedding extraction network training is completed, the trained watermark embedding extraction network is obtained;

[0013] Step 4: Positively input each video clip sample in the test set and its corresponding embedded watermark information into the watermark embedding module of the trained watermark embedding and extraction network. The watermark embedding module outputs a watermarked video. Then, perform HEVC compression on the watermarked video to obtain a HEVC compressed video. Next, use the HEVC compressed video as a distorted video and input it together with a zero vector backward into the watermark extraction module of the trained watermark embedding and extraction network. The watermark extraction module outputs the extracted watermark information.

[0014] In the said Step 1, the process of preprocessing each original video is as follows: Randomly take T consecutive video frames along the time axis from this original video. Then, randomly intercept a region of size W×H in each randomly taken video frame. The T regions consecutive along the time axis form a video clip sample. Among them, the number of channels of the video clip sample is 3T, the width is W, and the height is H.

[0015] In the said Step 1, the process of preprocessing each original watermark is as follows: Pass this original watermark through a linear layer to obtain a watermark feature of size Then, reshape the watermark feature to obtain an embedded watermark information of size Among them, the number of channels of the embedded watermark information is 1, the width is and the height is

[0016] In the said Step 2, the input of the i-th first invertible neural block is and The corresponding output is and Among them, i = 1, 2, …, N1. When i = 1 represents a video clip sample input into the watermark embedding module, represents the corresponding embedded watermark information. When 1 < i ≤ N1 represents the video feature containing the watermark output by the (i - 1)-th first invertible neural block, represents the watermark feature of the fused video output by the (i - 1)-th first invertible neural block. When 1 ≤ i < N1 represents the video feature containing the watermark output by the i-th first invertible neural block, represents the watermark feature of the fused video output by the i-th first invertible neural block. When i = N1 represents the watermarked video output by the watermark embedding module, represents the redundant information output by the watermark embedding module, represents the dot product operation, exp(·) represents the exponential function with the natural constant e as the base, Ui (·) represents the upsampling module in the i-th first reversible neural block, D 1 i (·) represents the first downsampling module in the i-th first reversible neural block, D 2 i (·) represents the second downsampling module in the i-th first reversible neural block, and have a channel number of 3T, a width of W, and a height of H, and have a channel number of 1, a width of and a height of U i (·) has an input with a channel number of 1, a width of and a height of U i (·) has an output with a channel number of 1, a width of W, and a height of H, D 1 i (·) and D 2 i (·) have an input with a channel number of ST, a width of W, and a height of H, D 1 i (·) and D 2 i (·) have an output with a channel number of 1, a width of and a height of

[0017] In the said step 2, the input of the k-th third reversible neural block is z k+1 and x noise k+1 and the corresponding output is z k and x noise k , )]]x noise k = x noise k+1 - U k (z k ), where k = N3, N3 - 1,..., 2, 1, when k = N3, z k+1 represents the all-zero vector input to the watermark extraction module, x noise k+1 represents the distorted video input to the watermark extraction module, when 1 ≤ k < N3, z k+1 represents the watermark feature of the fused distorted video input to the k-th third reversible neural block, x noise k+1 represents the distorted video feature containing the watermark input to the k-th third reversible neural block, when 1 < k ≤ N3, z krepresents the watermark feature of the fused distorted video output by the kth third reversible neural block, x noise k represents the distorted video feature containing the watermark output by the kth third reversible neural block, when k=1, z k Represents the watermark information of the extraction end output by the watermark extraction module, x noise k represents the restored video output by the watermark extraction module, U k (·) denotes the upsampling module in the k-th third reversible neural block, D 1 k (·) denotes the first downsampling module in the kth third reversible neural block, D 2 k (·) denotes the second downsampling module in the kth third reversible neural block, x noise k+1 and x noise k The number of channels is 3T, the width is W, the height is H, z k+1 and z k The number of channels is 1 and the width is Gao Wei U k The input channel number of (·) is 1 and the width is Gao Wei U k The output of (·) has 1 channel, W width, H height, and D 1 k (·) and D 2 k The number of input channels of (·) is 3T, the width is W, the height is H, and D 1 k (·) and D 2 k The output of (·) has 1 channel and a width of Gao Wei

[0018] The upsampling modules in all the first reversible neural blocks and the upsampling modules in all the third reversible neural blocks have the same structure, including the first convolutional layer, the first upsampling layer, the second upsampling layer, the third upsampling layer, and the second convolutional layer connected in sequence. The input end of the first convolutional layer is the input end of the upsampling module in which it is located, and the output end of the second convolutional layer is the output end of the upsampling module in which it is located. The first convolutional layer expands the number of channels of the features it receives from 1 to 32, and each upsampling layer expands the width and height of the features it receives by 2 times at the same time. The second convolutional layer compresses the number of channels of the features it receives from 32 to 1.

[0019] The structures of the first downsampling module and the second downsampling module in all the first reversible neural blocks, and the first downsampling module and the second downsampling module in all the third reversible neural blocks are the same, including a third convolutional layer, a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth convolutional layer connected in sequence. The input end of the third convolutional layer is the input end of the downsampling module where it is located, and the output end of the fourth convolutional layer is the output end of the downsampling module where it is located. The third convolutional layer expands the number of channels of the features it receives from 24 to 32. Each downsampling layer reduces the width and height of the features it receives by a factor of 2 simultaneously. The fourth convolutional layer compresses the number of channels of the features it receives from 32 to 1.

[0020] In step 2, the input of the m-th second reversible neural block is l m and h m , and the corresponding output is l m+1 and h m+1 , l m+1 = l m + η(ε(F(h m (j)))), where m = 1, 2, …, N2. When m = 1, l m represents the low-frequency region input to the distortion layer, H m represents the high-frequency region input to the distortion layer. When 1 < m ≤ N2, l m represents the low-frequency features containing high frequencies output by the (m - 1)-th second reversible neural block, h m represents the high-frequency features containing low frequencies output by the (m - 1)-th second reversible neural block. When 1 ≤ m < N2, l m+1 represents the low-frequency features containing high frequencies output by the m-th second reversible neural block, h m+1 represents the high-frequency features containing low frequencies output by the m-th second reversible neural block. When m = N2, l <00000�4>represents the compressed low-frequency region output by the distortion layer, h m+1 represents the compressed high-frequency region output by the distortion layer. h m (j) represents the j-th high-frequency part of h m , h m+1 (j) represents the j-th high-frequency part of h m+1 . j = 1, 2, 3. ε(·) represents the concatenation operation on the three high-frequency parts. η(·) represents a convolutional layer with a convolutional kernel size of 3×3. σ(·) represents the sigmoid activation function. F(·), E(·) and G(·) are all functions learned by the distortion layer and are all composed of Dense modules.

[0021] The Dense module includes the 5th convolutional layer, the 6th convolutional layer, the 7th convolutional layer, the 8th convolutional layer, and the 9th convolutional layer connected in sequence. The input end of the 5th convolutional layer is the input end of the Dense module where it is located, and the output end of the 9th convolutional layer is the output end of the Dense module where it is located. The input end of the 5th convolutional layer is jump-connected to the output end of the 5th convolutional layer, the output end of the 6th convolutional layer, the output end of the 7th convolutional layer, and the output end of the 8th convolutional layer respectively. The input end of the 6th convolutional layer is jump-connected to the output end of the 6th convolutional layer, the output end of the 7th convolutional layer, and the output end of the 8th convolutional layer respectively. The output end of the 6th to 9th convolutional layers is connected to the output end of the 7th convolutional layer and the output end of the 8th convolutional layer respectively. The input end of the 8th convolutional layer is jump-connected to the output end of the 8th convolutional layer. The number of input channels of each convolutional layer from the 6th to the 9th convolutional layers increases by 32 each time it passes through a convolutional layer. That is to say, the number of input channels of the 6th to the 9th convolutional layers is 3T+32, 3T+32×2, 3T+32×3, and 3T+32×4 respectively, where the number of input channels of the 5th convolutional layer and the number of output channels of the 9th convolutional layer are 3T.

[0022] In step 3, the process of pre-training the distortion layer is as follows:

[0023] Step 3.a1: Apply a DWT to each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of each video clip sample; apply a DWT to the label corresponding to each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of the label corresponding to each video clip sample;

[0024] Step 3.a2: Initialize the weights, learning rate, and network parameters of the distortion layer;

[0025] Step 3.a3: Forward the low-frequency region and high-frequency region of each video clip sample into the distortion layer, and the distortion layer outputs the compressed low-frequency region and the compressed high-frequency region;

[0026] Step 3.a4: Apply IDWT to the compressed low-frequency region and the compressed high-frequency region corresponding to each video clip sample to obtain a distorted video;

[0027] Step 3.a5: Reversely input the low-frequency region and high-frequency region of the label corresponding to each video clip sample into the distortion layer, and the distortion layer outputs the uncompressed low-frequency region and uncompressed high-frequency region;

[0028] Step 3.a6: Perform an IDWT operation on each uncompressed low-frequency region and the corresponding uncompressed high-frequency region to obtain an uncompressed video;

[0029] Step 3.a7: Calculate the loss L of the distorted video and the corresponding label corresponding to each video clip sample noise , L noise =MSE(v noise ,y com )+MSE(y rev ,v origin ), where MSE(·) represents the mean square error loss function, v origin Represents a video clip sample, v noise represents the distorted video corresponding to the video clip sample, y rev The label corresponding to the video clip sample is input into the uncompressed video obtained by the distortion layer. y com represents the label corresponding to the video clip sample, DWT(·) represents discrete wavelet transform, IDWT(·) represents inverse discrete wavelet transform, represents the forward input distortion layer, represents the reverse input distortion layer;

[0030] Step 3.a8: Backpropagate and update the gradient; then iterate the process from step 3.a3 to step 3.a7 until the mean of the loss function of the distorted video and the corresponding label corresponding to all video clip samples in the pre-training set converges, completing the distortion layer pre-training.

[0031] In step 3, the process of training the watermark embedding and extraction network is as follows:

[0032] Step 3.b1: Initialize the weights, learning rate, and network parameters of the watermark embedding extraction network;

[0033] Step 3.b2: Input each video clip sample in the training set and the corresponding embedding end watermark information, as well as the all-zero vector, into the watermark embedding extraction network, which then outputs the extraction end watermark information.

[0034] Step 3.b3: Calculate the total network loss L total , L t0tal =λ1L video +λ2L message +λ3L dis , where λ1, λ2 and λ3 are used to balance L video , L message and L dis Contribution weight parameter, L video represents the video loss used to quantify the mean square error between the video clip samples and the watermarked video output by the watermark embedding module, L video=MSE(c,s), where MSE(·) represents the mean square error loss function, c represents the video clip sample, s represents the watermarked video output by the watermark embedding module, and L message It represents the message loss used to quantify the mean square error between the watermark information at the embedding end and the watermark information at the extraction end output by the watermark extraction module, L message =MAE(M,M ex ), L represents the embedded end watermark information, M ex Indicates the watermark information extracted by the watermark extraction module, L dis Denotes the adversarial loss, L dis =CrossEntropy(dis(s),label), CrossEntropy(·) represents the cross entropy loss function, dis(·) represents the neural network structure discriminator used to determine whether the watermarked video has a watermark, and label represents an all-one vector label;

[0035] Step 3.b4: Backpropagate and update the gradient; then iterate the process from step 3.b2 to step 3.b3, setting λ3 to 0 during the first 20 iterations and setting λ3 to non-zero during subsequent iterations; end the iteration process when the mean of the total network loss for all training sets converges, completing the watermark embedding extraction network training.

[0036] Compared with the prior art, the advantages of the present invention are:

[0037] The method of the present invention constructs a watermark embedding module and a watermark extraction module based on a reversible neural network, solving the problem of small watermark embedding capacity and avoiding the problem of decoupling between the watermark embedding module and the watermark extraction module, thereby improving watermark extraction accuracy. The method of the present invention also designs a differentiable HEVC video compression simulator as a distortion layer, solving the problem of poor watermark robustness to video compression. This design makes the constructed watermark embedding and extraction network robust to operations such as video compression, frame swapping, and frame loss.

[0038] The method of the present invention uses a neural network structure discriminator for judging whether a watermarked video has a watermark when calculating the adversarial loss in the training phase, thereby improving the quality of the video after embedding the watermark. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of the composition structure of the watermark embedding and extraction network constructed for the present invention;

[0040] Figure 2 A schematic diagram of the composition structure of the upsampling module in the watermark embedding and extraction network constructed for the present invention;

[0041] Figure 3A schematic diagram of the structure of the downsampling module in the watermark embedding and extraction network constructed for the present invention;

[0042] Figure 4 A schematic diagram of the composition structure of the distortion layer in the watermark embedding and extraction network constructed for the present invention;

[0043] Figure 5 Schematic diagram of the composition structure of the Dense module in the distortion layer of the watermark embedding and extraction network constructed for the present invention. DETAILED DESCRIPTION

[0044] The present invention will be described in further detail below with reference to the accompanying drawings and embodiments.

[0045] The present invention proposes a video watermarking method based on a reversible neural network, which includes the following steps:

[0046] Step 1: Select an original data set containing Num original videos; then preprocess each original video in the original data set and the original watermark randomly generated for each original video to obtain Num video clip samples and the embedded end watermark information corresponding to each video clip sample, and form a first preprocessed data set; then divide the first preprocessed data set into a training set and a test set. In this embodiment, the ratio of the training set to the test set is 8:2.

[0047] Num video clip samples are compressed by HEVC to obtain the corresponding compressed video as a label; then Num video clip samples and the label corresponding to each video clip sample constitute a second preprocessed data set, and the second preprocessed data set is divided into a pre-training set and a pre-test set. In this embodiment, the ratio of the pre-training set to the pre-test set is 8:2, and the pre-test set is only used to verify the effectiveness of the distortion layer.

[0048] Further defined, the process of preprocessing each original video is as follows: randomly selecting T consecutive video frames along the time axis from the original video; then randomly cropping a W×H region from each randomly selected video frame, with the T consecutive regions along the time axis forming a video clip sample, where the video clip sample has 3T channels, a width of W, and a height of H. In this embodiment, T is 8, and W×H is 128×128. In specific implementations, the starting frame position when randomly selecting consecutive video frames from all original videos can be fixed, such as selecting 8 consecutive video frames starting from the 5th frame. The position of the cropped region in the randomly selected video frame can also be fixed, such as cropping the center region.

[0049] Further definition, the process of preprocessing each original watermark is: the original watermark is passed through a linear layer to obtain a size of The watermark feature is reshaped to obtain a size of The embedded end watermark information, where the number of channels of the embedded end watermark information is 1 and the width is Gao Wei In this embodiment, a 96-bit original watermark is used; the linear layer and reshape technology both adopt existing technologies.

[0050] Step 2: Build a watermark embedding extraction network, such as Figure 1 As shown, it includes a watermark embedding module (embed watermark) and a watermark extraction module (extract watermark) that share same network parameters, as well as a distortion module used to simulate the HEVC compression effect.

[0051] The watermark embedding module is based on a reversible neural network, which consists of N1 first reversible neural blocks with the same structure connected in sequence. Figure 1 The cover video c) and its corresponding embedded end watermark information ( Figure 1 The watermark M shown in is forwardly input to the watermark embedding module, which embeds the embedding end watermark information into the video clip sample through N1 first reversible neural blocks. The watermark embedding module outputs the watermark video ( Figure 1 The watermark video s) and redundant information ( Figure 1 The lost information r) shown in .

[0052] The distortion module includes DWT (discrete wavelet transform) operation, a distortion layer composed of N2 second reversible neural blocks with the same structure connected in sequence based on a reversible neural network, and IDWT (inverse discrete wavelet transform) operation, which converts the watermarked video ( Figure 1 After the DWT operation, the watermark video s shown in the figure is converted into a low-frequency region and a high-frequency region composed of three high-frequency parts with the same dimensions, realizing the conversion from the pixel domain to the transform domain. The low-frequency region and the high-frequency region are forward-inputted into the distortion layer, as shown in Figure 4 As shown, the distortion layer compresses the low-frequency area and the high-frequency area through N2 second reversible neural blocks, and the distortion layer outputs the compressed low-frequency area and the compressed high-frequency area. The compressed low-frequency area and the compressed high-frequency area are subjected to IDWT operation to obtain the distorted video ( Figure 1 The distorted video s shown in noise). In order to enhance the robustness of the watermark, especially for non-differentiable operations such as HEVC compression, the present invention designs a differentiable distortion layer to simulate the compression effect of HEVC. The distortion layer also uses a reversible neural network as the backbone network, but makes necessary adjustments to its internal structure to better simulate the impact of HEVC compression on the watermark. It is worth noting that the distortion layer needs to be pre-trained before embedding and extracting the watermark. Through the above design, the watermark embedding and extraction network can not only resist HEVC compression, but also maintain robustness against other attacks that may cause changes in video content.

[0053] The watermark extraction module is based on a reversible neural network, which consists of N3 third reversible neural blocks with the same structure connected in sequence. Figure 1 The distorted video s shown in noise ) and an all-zero vector (auxiliary variable z) of the same size as the embedding end watermark information are reversely input into the watermark extraction module. The watermark extraction module extracts the watermark from the distorted video as the extraction end watermark information through N3 third reversible neural blocks. The watermark extraction module outputs the restored video ( Figure 1 recover video) and extract end watermark information ( Figure 1 The extractwatermark M shown in ex ). Here, in order to achieve blind extraction, that is, to successfully extract the watermark without other prior conditions, the watermark extraction module uses the all-zero vector as auxiliary information for extraction during the watermark extraction process.

[0054] In this embodiment, N1, N2, and N3 have the same value, that is, N1=N2=N3=16.

[0055] In fact, the watermark embedding module and the watermark extraction module have the same structure. Therefore, in actual design, it is sufficient to design two identical modules. When the forward input is used, watermark embedding is realized, and the module serves as a watermark embedding module; when the reverse input is used, watermark extraction is realized, and the module serves as a watermark extraction module.

[0056] In this embodiment, in step 2, each first reversible neural block in the watermark embedding module is a forward encoding process, and the input of the i-th first reversible neural block is and The corresponding output is and Where i = 1, 2, ..., N1, when i = 1 Represents a video clip sample input to the watermark embedding module, express The corresponding embedded - side watermark information, when 1 < i ≤ N1 represents the watermark - containing video features output by the (i - 1)-th first reversible neural block, represents the watermark features of the fused video output by the (i - 1)-th first reversible neural block, when 1 ≤ i < N1 represents the watermark - containing video features output by the i - th first reversible neural block, represents the watermark features of the fused video output by the i - th first reversible neural block, when i = N1 represents the watermark video output by the watermark embedding module, represents the redundant information output by the watermark embedding module, represents the dot - product operation, exp(·) represents the exponential function with the natural constant e as the base, e = 2.71, U i U(·) represents the up - sampling module in the i - th first reversible neural block, D 1 i D(·) represents the first down - sampling module in the i - th first reversible neural block, D 2 i D(·) represents the second down - sampling module in the i - th first reversible neural block, and have a channel number of 3T, a width of W, and a height of H, and have a channel number of 1, a width of a height of U i The input of U(·) has a channel number of 1, a width of a height of U i The output of U(·) has a channel number of 1, a width of W, and a height of H, D 1 i D(·) and D 2 i D(·) have an input with a channel number of 3T, a width of W, and a height of H, D 1 i D(·) and D 2 i D(·) have an output with a channel number of 1, a width of a height of In it will automatically expand the channel number to 3T, which is the broadcast mechanism. In the first reversible neural block, the up - sampling module and the two down - sampling modules are responsible for sampling the watermark information and the video information to the same dimension for fusion.

[0057] In this embodiment, in step 2, in the watermark extraction module, the reverse input is from the last third reversible neural block, and the input of the k - th third reversible neural block is zk+1 and x noise k+1 The corresponding output is z k and x noise k , x noise k = x noise k+1 -U k (z k ), where k = N3, N3 - 1, …, 2, 1. When k = N3, z k+1 represents the all-zero vector input to the watermark extraction module, x noise k+1 represents the distorted video input to the watermark extraction module. When 1 ≤ k < N3, z k+1 represents the watermark feature of the fused distorted video input to the k-th third reversible neural block, x noise k+1 represents the distorted video feature containing the watermark input to the k-th third reversible neural block. When 1 < k ≤ N3, z k represents the watermark feature of the fused distorted video output from the k-th third reversible neural block, x noise k represents the distorted video feature containing the watermark output from the k-th third reversible neural block. When k = 1, z k represents the watermark information at the extraction end output by the watermark extraction module, x noise k represents the restored video output by the watermark extraction module, U k (·) represents the upsampling module in the k-th third reversible neural block, D 1 k (·) represents the first downsampling module in the k-th third reversible neural block, D 2 k (·) represents the second downsampling module in the k-th third reversible neural block, x noise k+1 and x<0000​​​​​​​​​​​​​​​​​​​​​​k (·) has an input channel number of 3T, a width of W, and a height of H, D 1 k (·) and D 2 k (·) has an output channel number of 1 and a width of height of At x noise k = x noise k+1 - U k (z k ) where U k (z k ) will automatically expand the channel number to 3T.

[0058] In this embodiment, the upsampling modules in all the first reversible neural blocks and the upsampling modules in all the third reversible neural blocks have the same structure. As Figure 2 shown, it includes a first convolutional layer, a first upsampling layer, a second upsampling layer, a third upsampling layer, and a second convolutional layer connected in sequence. The input end of the first convolutional layer is the input end of the upsampling module where it is located, and the output end of the second convolutional layer is the output end of the upsampling module where it is located. The first convolutional layer expands the channel number of the received features from 1 to 32. Each upsampling layer doubles the width and height of the received features simultaneously. The second convolutional layer compresses the channel number of the received features from 32 to 1.

[0059] In this embodiment, the first downsampling module and the second downsampling module in all the first reversible neural blocks, and the first downsampling module and the second downsampling module in all the third reversible neural blocks have the same structure. As Figure 3 shown, it includes a third convolutional layer, a first downsampling layer, a second downsampling layer, a third downsampling layer, and a fourth convolutional layer connected in sequence. The input end of the third convolutional layer is the input end of the downsampling module where it is located, and the output end of the fourth convolutional layer is the output end of the downsampling module where it is located. The third convolutional layer expands the channel number of the received features from 24 to 32. Each downsampling layer halves the width and height of the received features simultaneously. The fourth convolutional layer compresses the channel number of the received features from 32 to 1.

[0060] In this embodiment, in step 2, as Figure 4 shown, the input of the m-th second reversible neural block is l m and h m , and the corresponding output is l m+1 and h m+1 , l m+1 = l m + η(ε(F(h m(j)))) where \(m = 1, 2, \ldots, N_2\), when \(m = 1\), \(l\) m represents the low - frequency region input to the distortion layer, \(h\) m represents the high - frequency region input to the distortion layer, when \(1 \lt m \leq N_2\), \(l\) m represents the low - frequency feature containing high - frequency output from the \((m - 1)\) - th second reversible neural block, \(h\) m represents the high - frequency feature containing low - frequency output from the \((m - 1)\) - th second reversible neural block, when \(1 \leq m \lt N_2\), \(l\) m+1 represents the low - frequency feature containing high - frequency output from the \(m\) - th second reversible neural block, \(h\) m+1 represents the high - frequency feature containing low - frequency output from the \(m\) - th second reversible neural block, when \(m = N_2\), \(l\) m+1 represents the compressed low - frequency region output from the distortion layer, \(h\) m +1 represents the compressed high - frequency region output from the distortion layer, \(h\) m \(h(j)\) represents m the \(j\) - th high - frequency part of \(h\), \(h\) m+1 \(h(j)\) represents m+1 the \(j\) - th high - frequency part of \(h\), \(j = 1, 2, 3\), \(\varepsilon(\cdot)\) represents the concatenation operation on the three high - frequency parts, \(\eta(\cdot)\) represents a convolutional layer with a convolution kernel size of \(3\times3\), the purpose is to transform the concatenated part to the same dimension as the low - frequency region for convenient subsequent operations, \(\sigma(\cdot)\) represents the sigmoid activation function, \(F(\cdot)\), \(E(\cdot)\) and \(G(\cdot)\) are all functions learned by the distortion layer and are all composed of Dense modules. Since video compression mainly targets the high - frequency region of the video, the change in the low - frequency region of the video is relatively small. Based on this, the distortion layer simulates the compression of the low - frequency region by HEVC by continuously adding perturbations to the original low - frequency region, and for the high - frequency region, a more complex method is used for simulation.

[0061] such as Figure 5As shown, the Dense module includes the 5th convolutional layer, the 6th convolutional layer, the 7th convolutional layer, the 8th convolutional layer, and the 9th convolutional layer connected in sequence. The input end of the 5th convolutional layer is the input end of the Dense module where it is located, and the output end of the 9th convolutional layer is the output end of the Dense module where it is located. The input end of the 5th convolutional layer is jump-connected to the output end of the 5th convolutional layer, the output end of the 6th convolutional layer, the output end of the 7th convolutional layer, and the output end of the 8th convolutional layer, and the input end of the 6th convolutional layer is jump-connected to the output end of the 6th convolutional layer, the output end of the 7th convolutional layer, and the output end of the 8th convolutional layer. The output end and the output end of the 8th convolutional layer, the input end of the 7th convolutional layer are jump-connected to the output end of the 7th convolutional layer and the output end of the 8th convolutional layer respectively, and the input end of the 8th convolutional layer is jump-connected to the output end of the 8th convolutional layer. The number of input channels of each convolutional layer from the 6th to the 9th convolutional layer increases by 32 each time it passes through a convolutional layer. That is to say, the number of input channels of the 6th to the 9th convolutional layers are 3T+32, 3T+32×2, 3T+32×3, and 3T+32×4 respectively, of which the number of input channels of the 5th convolutional layer and the number of output channels of the 9th convolutional layer are 3T.

[0062] Step 3: Based on the pre-training set, pre-train the distortion layer; after the distortion layer pre-training is completed, fix the network parameters of the distortion layer, and then train the watermark embedding extraction network based on the training set. After the watermark embedding extraction network training is completed, a trained watermark embedding extraction network is obtained.

[0063] After the distortion layer pre-training is completed, in order to ensure the stability of the distortion layer and the convergence of the watermark embedding and extraction network, all network parameters of the distortion layer need to be fixed, so that the distortion layer can provide a stable distortion effect in the subsequent watermark embedding and extraction process.

[0064] Preferably, the process of pre-training the distortion layer is:

[0065] Step 3.a1: Perform a DWT operation on each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of each video clip sample; perform a DWT operation on the label corresponding to each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of the label corresponding to each video clip sample.

[0066] Step 3.a2: Initialize the weights, learning rate, and network parameters of the distortion layer.

[0067] Step 3.a3: Forward the low-frequency region and the high-frequency region of each video clip sample into the distortion layer, and the distortion layer outputs the compressed low-frequency region and the compressed high-frequency region.

[0068] Step 3.a4: Perform an IDWT operation on the compressed low-frequency region and the compressed high-frequency region corresponding to each video clip sample to obtain a distorted video.

[0069] Step 3.a5: The low-frequency region and high-frequency region of the label corresponding to each video clip sample are reversely input into the distortion layer, and the distortion layer outputs the uncompressed low-frequency region and the uncompressed high-frequency region.

[0070] Step 3.a6: Perform an IDWT operation on each uncompressed low-frequency region and the corresponding uncompressed high-frequency region to obtain an uncompressed video.

[0071] Step 3.a7: Calculate the loss L of the distorted video and the corresponding label corresponding to each video clip sample noise , L noise =MSE(v noise ,y com )+MSE(y rev ,v origin ), where MSE(·) represents the mean square error loss function, v origin Represents a video clip sample, v noise represents the distorted video corresponding to the video clip sample, y rev The label corresponding to the video clip sample is input into the uncompressed video obtained by the distortion layer. y com represents the label corresponding to the video clip sample, DWT(·) represents discrete wavelet transform, IDWT(·) represents inverse discrete wavelet transform, represents the forward input distortion layer, represents the reverse input distortion layer.

[0072] Step 3.a8: Backpropagate and update the gradient; then iterate the process from step 3.a3 to step 3.a7 until the mean of the loss function of the distorted video and the corresponding label corresponding to all video clip samples in the pre-training set converges, completing the distortion layer pre-training.

[0073] During the pre-training process of the distortion layer, it is necessary to imitate HEVC compression as much as possible, so in the forward input process, the distorted video v noise with label y com As similar as possible. Accordingly, in the reverse input process, the uncompressed video y rev With the video clip sample v origin As close as possible, so a loss function L is defined noiseTo minimize the mean squared error between the distorted videos and the corresponding labels corresponding to the video clip samples in the pre-training set. During the pre-training process, learning rate decay and network parameter regularization techniques can be used to make the distortion layer pre-training more stable.

[0074] As a preferred method, the process of training the watermark embedding and extraction network is as follows:

[0075] Step 3.b1: Initialize the weights, learning rate, and network parameters of the watermark embedding extraction network.

[0076] Step 3.b2: Input each video clip sample in the training set and the corresponding embedding end watermark information, as well as the all-zero vector, into the watermark embedding extraction network, and the watermark embedding extraction network outputs the extraction end watermark information.

[0077] Step 3.b3: Calculate the total network loss L total , the total network loss L total It consists of three parts: In order to achieve the invisibility of the watermark, the video loss L is defined video In order to losslessly extract the watermark from the distorted video in the watermark extraction module, the message loss L is defined. message In order to improve the invisibility of watermarks and enhance the quality of video, this paper introduces a more complex discriminator to achieve the goal and defines the adversarial loss L dis The total network loss L total The mathematical expression is: L total =λ1L video +λ2L message +λ3L dis , where λ1, λ2 and λ3 are used to balance L video , L message and L dis Contribution weight parameter, L video represents the video loss used to quantify the mean square error between the video clip samples and the watermarked video output by the watermark embedding module, L video =MSE(c,s), where MSE(·) represents the mean square error loss function, c represents the video clip sample, s represents the watermarked video output by the watermark embedding module, and L message It represents the message loss used to quantify the mean square error between the watermark information at the embedding end and the watermark information at the extraction end output by the watermark extraction module, L message =MSE(M,M ex ), M represents the embedded end watermark information, M ex Indicates the watermark information extracted by the watermark extraction module, L dis Denotes the adversarial loss, L dis= CrossEntropy(dis(s),label), CrossEntropy(·) represents the cross entropy loss function, dis(·) represents the neural network structure discriminator used to determine whether the watermarked video has a watermark ( Figure 1 The neural network structure discriminator uses the existing technology. The neural network structure discriminator will output 0 or 1. Output 0 indicates that the judgment result is that the input watermarked video has a watermark, and output 1 indicates that the judgment result is that the input watermarked video does not have a watermark. Label represents an all-1 vector label.

[0078] Step 3.b4: Backpropagate and update the gradient; then iterate the process from step 3.b2 to step 3.b3, setting λ3 to 0 during the first 20 iterations and setting λ3 to non-zero during subsequent iterations; end the iteration process when the mean of the total network loss for all training sets converges, completing the watermark embedding extraction network training.

[0079] In the training process of the watermark embedding extraction network, a staged training strategy is adopted. In the first 20 rounds, that is, the first 20 iterations, the focus is on minimizing the video loss L video and message loss L message , and the adversarial loss L dis We will not consider it for now, that is, set the weight λ3 to 0, λ1 to 1, and λ2 to 10. In the subsequent iteration process, we will consider the adversarial loss L dis , where λ1 is set to 1, λ2 is set to 10, and λ3 is set to 0.0001. This phased training strategy aims to avoid model collapse caused by premature introduction of adversarial training and avoids the unnecessary complexity that this may cause. This phased training strategy ensures stable model learning and gradual performance improvement at each training stage, ultimately achieving optimized training results.

[0080] Step 4: Each video clip sample in the test set and its corresponding embedding-end watermark information are forward-inputted into the watermark embedding module of the trained watermark embedding extraction network, and the watermark embedding module outputs the watermarked video; then the watermarked video is HEVC compressed to obtain an HEVC compressed video; then the HEVC compressed video is used as a distorted video and is reversely inputted into the watermark extraction module of the trained watermark embedding extraction network together with the all-zero vector, and the watermark extraction module outputs the extraction-end watermark information.

[0081] In order to further illustrate the feasibility and effectiveness of the method of the present invention, experiments were conducted on the method of the present invention.

[0082] The experimental dataset uses UCF-101 as the original dataset. This is a large-scale video dataset containing 101 action categories, containing a total of 13,320 video samples, or original videos. Given the large number of video samples with similar scenes in the experimental dataset, 2,000 video samples with low scene similarity were carefully selected. After preprocessing, video clip samples were obtained. A portion of the video clip samples constituted the pre-training set for pre-training the distortion layer. At the same time, this portion of the video clip samples constituted the training set for training the watermark embedding and extraction network to improve the generalization ability and training efficiency of the watermark embedding and extraction network in diverse scenarios. The remaining video clip samples constituted the test set for testing. At the same time, to comprehensively evaluate the performance of the watermark embedding and extraction network, video samples from two classic datasets, REDS and Kinetics 400, were used after preprocessing to form the test set for testing. This was used to evaluate the watermark embedding and extraction network's ability to recognize and process new data.

[0083] To enhance the adaptability of the watermark embedding and extraction network to different video sizes and ratios, a random cropping strategy was implemented as a preprocessing process. Specifically, after cropping, the video samples were resized to 3×8×128×128 dimensions, where 3 represents the number of color channels, 8 represents the number of video frames, and 128×128 represents the spatial resolution of each video frame. This random cropping method not only increases the diversity of training data but also simulates the various video sizes that may be encountered in real applications, thereby improving the generalization ability of the watermark embedding and extraction network.

[0084] The experiment used metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), Learning Perceptual Patch Similarity (LPIPS), and Accuracy (ACC) to measure the performance of different watermarking methods in terms of video image quality, video perception quality, and watermark extraction accuracy. Higher PSNR and SSIM values ​​indicate higher video image quality, lower LPIPS values ​​indicate better video perception quality, and higher ACC values ​​indicate that the model can extract the embedded watermark information with higher accuracy.

[0085] The comparison methods in the experiment are HiDDeN method and REVMARK method. The HiDDeN method refers to the method of Zhu (J.Zhu,R.Kaplan,J.Johnson,and L.Fei-Fei,"HiDDen:hiding data with deep networks,"Proceedings of the European Conference on Computer Vision(ECCV),Munich,Germany,2018,pp.657-672.(Hiding data with deep networks)), and the REVMARK method refers to the method of Zhang (Y.Zhang,J.Ni,W.Su,and X.Liao,"A novel deep video watermarking framework with enhanced robustness to H.264 / AVC compression,"in Proceedings of the 31st ACMInternational Conference on Multimedia,New York,NY,USA,2023,pp.8095-8104(A novel deep video watermarking framework with enhanced robustness to H.264 / AVC compression)).

[0086] Table 1 shows the watermark extraction accuracy (ACC) of the existing HiDDeN method, REVMARK method, and the proposed method (LRVMark) on the UCF-101 test set, Kinetics 400 dataset, and REDS dataset, respectively, after watermarking videos subjected to various distortion attacks, including frame averaging, frame loss, frame swapping, Gaussian noise, H264 / AVC, and HEVC. In Table 1, N refers to the number of frames in a time window, p represents the probability of frame loss (or swapping with its directly adjacent frames), std refers to the standard deviation, and QP refers to the quantization parameter (the larger the QP, the stronger the compression effect).

[0087] Table 1 Comparison of ACC (%) under different distortion attacks

[0088]

[0089] As can be seen in Table 1, the watermark extraction accuracy of our method on all three datasets is superior to that of the comparison methods. Furthermore, in the face of various distortion attacks, 100% watermark information can be extracted in most cases. This not only demonstrates the robustness of our method but also verifies its good generalization ability across different datasets.

[0090] Table 2 shows the comparison of watermarked video quality measured by different evaluation indicators using the existing HiDDeN method, REVMARK method, and the proposed method (LRVMark) on the UCF-101 test set, Kinetics 400 dataset, and REDS dataset when 96-bit watermark information is embedded.

[0091] Table 2 Comparison of the quality of watermarked videos obtained under different watermarking methods

[0092]

[0093] It can be seen from Table 2 that the method of the present invention is relatively excellent in terms of video quality.

[0094] Table 3 shows the extraction accuracy of watermark information by the method of the present invention after different degrees of video compression when watermark information of different capacities is embedded.

[0095] Table 3 ACC (%), different compression strengths for a range of watermark lengths

[0096]

[0097] As can be seen from Table 3, it can be found that after embedding a 256-bit or even 1024-bit watermark, despite the HEVC compression process, the method of the present invention can still effectively extract the watermark information while ensuring the video quality.

Claims

1. A video watermarking method based on reversible neural network, characterized in that The following steps are involved: Step 1: Select an original data set containing Num original videos; then preprocess each original video in the original data set and the original watermark randomly generated for each original video to obtain Num video clip samples and the embedded end watermark information corresponding to each video clip sample, and form a first preprocessed data set; Then divide the first preprocessed data set into a training set and a test set; Compress Num video clip samples through HEVC and obtain the corresponding compressed video as a label; Then, Num video clip samples and the labels corresponding to each video clip sample constitute a second preprocessed data set, and the second preprocessed data set is divided into a pre-training set and a pre-test set; Step 2: Build a watermark embedding and extraction network, which includes a watermark embedding module and a watermark extraction module that share network parameters, as well as a distortion module used to simulate the HEVC compression effect; The watermark embedding module is based on a reversible neural network, which consists of N1 first reversible neural blocks with the same structure connected in sequence. The video clip sample and its corresponding embedding end watermark information are forward input into the watermark embedding module. The watermark embedding module embeds the embedding end watermark information into the video clip sample through N1 first reversible neural blocks. The watermark embedding module outputs the watermark video and redundant information. The distortion module includes a DWT operation, a distortion layer composed of N2 second reversible neural blocks with the same structure connected in sequence based on a reversible neural network, and an IDWT operation. After the watermark video is subjected to the DWT operation, a low-frequency region and a high-frequency region composed of high-frequency parts with the same three dimensions are obtained. The low-frequency region and the high-frequency region are forward input into the distortion layer. The distortion layer compresses the low-frequency region and the high-frequency region through the N2 second reversible neural blocks. The distortion layer outputs the compressed low-frequency region and the compressed high-frequency region. After the compressed low-frequency region and the compressed high-frequency region are subjected to the IDWT operation, a distorted video is obtained. The watermark extraction module is based on a reversible neural network, which consists of N3 third-party reversible neural blocks with the same structure connected in sequence. The distorted video and an all-zero vector with the same size as the watermark information at the embedding end are reversely input into the watermark extraction module. The watermark extraction module extracts the watermark from the distorted video as the extraction end watermark information through the N3 third-party reversible neural blocks. The watermark extraction module outputs the restored video and the extraction end watermark information; Step 3: Pre-train the distortion layer based on the pre-training set; After the pre-training of the distortion layer is completed, the network parameters of the distortion layer are fixed, and then the watermark embedding extraction network is trained based on the training set. After the training of the watermark embedding extraction network is completed, a trained watermark embedding extraction network is obtained; Step 4: Each video clip sample in the test set and its corresponding embedding end watermark information are forward-inputted into the watermark embedding module of the trained watermark embedding extraction network. The watermark embedding module outputs the watermarked video. The watermarked video is then compressed by HEVC to obtain an HEVC compressed video. Then, the HEVC compressed video is used as the distorted video and is reversely input into the watermark extraction module of the trained watermark embedding extraction network together with the all-zero vector. The watermark extraction module outputs the extraction end watermark information.

2. The video watermarking method based on reversible neural network according to claim 1 is characterized in that In step 1, the process of preprocessing each original video is as follows: randomly select T frames of video frames that are continuous along the time axis from the original video; then randomly intercept a region of size W×H from each randomly selected video frame, and the T regions that are continuous along the time axis constitute a video clip sample, where the number of channels of the video clip sample is 3T, the width is W, and the height is H.

3. The video watermarking method based on reversible neural network according to claim 2 is characterized in that In step 1, the process of preprocessing each original watermark is as follows: the original watermark is passed through a linear layer to obtain a The watermark feature is reshaped to obtain a size of The embedded end watermark information, where the number of channels of the embedded end watermark information is 1 and the width is Gao Wei 4. The video watermarking method based on reversible neural network according to claim 3 is characterized in that In step 2, the input of the i-th first reversible neural block is and The corresponding output is and where \(i = 1, 2, \ldots, N_1\), when \(i = 1\) represents a video clip sample input into the watermark embedding module, represents the corresponding watermark information at the embedding end, when \(1 < i \leq N_1\) represents the watermarked video features output by the \((i - 1)\) - th first reversible neural block, represents the watermark features of the fused video output by the \((i - 1)\) - th first reversible neural block, when \(1 \leq i < N_1\) represents the watermarked video features output by the \(i\) - th first reversible neural block, represents the watermark features of the fused video output by the \(i\) - th first reversible neural block, when \(i = N_1\) represents the watermarked video output by the watermark embedding module, represents the redundant information output by the watermark embedding module, represents the dot - product operation, \(\exp(\cdot)\) represents the exponential function with the natural constant \(e\) as the base, \(U\) i (\(\cdot\)) represents the up - sampling module in the \(i\) - th first reversible neural block, \(D\) 1 i (\(\cdot\)) represents the first down - sampling module in the \(i\) - th first reversible neural block, \(D\) 2 i (\(\cdot\)) represents the second down - sampling module in the \(i\) - th first reversible neural block, and have a channel number of \(3T\), a width of \(W\), and a height of \(H\), and have a channel number of \(1\), a width of a height of U i (\(\cdot\)) has an input channel number of \(1\), a width of a height of U i (\(\cdot\)) has an output channel number of \(1\), a width of \(W\), and a height of \(H\), \(D\) 1 i (\(\cdot\)) and \(D\) 2 i (\(\cdot\)) have an input channel number of \(3T\), a width of \(W\), and a height of \(H\), \(D\) 1 i (\(\cdot\)) and \(D\) 2 i (\(\cdot\)) have an output channel number of \(1\), a width of a height of 5. The video watermarking method based on reversible neural network according to claim 4 is characterized in that In the said step 2, the input of the k-th third reversible neural block is z k+1 and x noise k+1 and the corresponding output is z k and x noise k , x noise k = x noise k+1 - U k (z k ), where k = N3, N3 - 1, …, 2, 1. When k = N3, z k+1 represents the all-zero vector input to the watermark extraction module, and x noise k+1 represents the distorted video input to the watermark extraction module. When 1 ≤ k < N3, z k+1 represents the watermark feature of the fused distorted video input to the k-th third reversible neural block, and x noise k+1 represents the distorted video feature containing the watermark input to the k-th third reversible neural block. When 1 < k ≤ N3, z k [[ID=३4]]represents the watermark feature of the fused distorted video output from the k-th third reversible neural block, and x noise k represents the distorted video feature containing the watermark output from the k-th third reversible neural block. When k=1, z k Represents the watermark information of the extraction end output by the watermark extraction module, x noise k represents the restored video output by the watermark extraction module, U k (·) denotes the upsampling module in the k-th third reversible neural block, D 1 k (·) denotes the first downsampling module in the kth third reversible neural block, D 2 k (·) denotes the second downsampling module in the kth third reversible neural block, x noise k+1 and x noise k The number of channels is 3T, the width is W, the height is H, z k+1 and z k The number of channels is 1 and the width is Gao Wei U k The input channel number of (·) is 1 and the width is Gao Wei U k The output of (·) has 1 channel, W width, H height, and D 1 k (·) and D 2 k The number of channels of the input of (·) is 3T, the width is W, the height is H, and D 1 k (·) and D 2 k The output of (·) has 1 channel and a width of Gao Wei 6. The video watermarking method based on reversible neural network according to claim 5 is characterized in that The upsampling modules in all the first reversible neural blocks and the upsampling modules in all the third reversible neural blocks have the same structure, including the first convolutional layer, the first upsampling layer, the second upsampling layer, the third upsampling layer, and the second convolutional layer connected in sequence. The input end of the first convolutional layer is the input end of the upsampling module in which it is located, and the output end of the second convolutional layer is the output end of the upsampling module in which it is located. The first convolutional layer expands the number of channels of the features it receives from 1 to 32, and each upsampling layer expands the width and height of the features it receives by 2 times at the same time. The second convolutional layer compresses the number of channels of the features it receives from 32 to 1.

7. The video watermarking method based on reversible neural network according to claim 5 is characterized in that The first downsampling module and the second downsampling module in all the first reversible neural blocks, and the first downsampling module and the second downsampling module in all the third reversible neural blocks have the same structure, including the third convolutional layer, the first downsampling layer, the second downsampling layer, the third downsampling layer, and the fourth convolutional layer connected in sequence, the input end of the third convolutional layer is the input end of the downsampling module in which it is located, and the output end of the fourth convolutional layer is the output end of the downsampling module in which it is located. The third convolutional layer expands the number of channels of the features it receives from 24 to 32, each downsampling layer reduces the width and height of the features it receives by 2 times at the same time, and the fourth convolutional layer compresses the number of channels of the features it receives from 32 to 1.

8. The video watermarking method based on reversible neural network according to claim 5, 6 or 7, characterized in that In the said step 2, the input of the m-th second reversible neural block is l m and h m ; the corresponding output is l m+1 and h m+1 , l m+1 =l m +η(ε(F(h m )(j))), where m = 1, 2, …, N2, when m = 1, l m represents the low-frequency region input to the distortion layer, h m represents the high-frequency region input to the distortion layer, when 1 < m ≤ N2, l m represents the low-frequency feature containing high frequency output by the (m - 1)-th second reversible neural block, h m represents the high-frequency feature containing low frequency output by the (m - 1)-th second reversible neural block, when 1 ≤ m < N2, l m+1 represents the low-frequency feature containing high frequency output by the m-th second reversible neural block, h m+1 represents the high-frequency feature containing low frequency output by the m-th second reversible neural block, when m = N2, l m+1 represents the compressed low-frequency region output by the distortion layer, h m+1 represents the compressed high-frequency region output by the distortion layer, h m (j) represents the j-th high-frequency part of h m , h m+1 (j) represents the j-th high-frequency part of h m+1 , j = 1, 2, 3, ε(·) represents the concatenation operation on the three high-frequency parts, η(·) represents the convolutional layer with a convolutional kernel size of 3×3, σ(·) represents the sigmoid activation function, F(·), E(·) and G(·) are all functions learned by the distortion layer and are all composed of Dense modules.

9. The video watermarking method based on reversible neural network according to claim 1 is characterized in that In step 3, the process of pre-training the distortion layer is as follows: Step 3.a1: Apply a DWT to each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of each video clip sample; apply a DWT to the label corresponding to each video clip sample in the pre-training set to obtain the low-frequency region and high-frequency region of the label corresponding to each video clip sample; Step 3.a2: Initialize the weights, learning rate, and network parameters of the distortion layer; Step 3.a3: Forward the low-frequency region and high-frequency region of each video clip sample into the distortion layer, and the distortion layer outputs the compressed low-frequency region and the compressed high-frequency region; Step 3.a4: Apply IDWT to the compressed low-frequency region and the compressed high-frequency region corresponding to each video clip sample to obtain a distorted video; Step 3.a5: Reversely input the low-frequency region and high-frequency region of the label corresponding to each video clip sample into the distortion layer, and the distortion layer outputs the uncompressed low-frequency region and uncompressed high-frequency region; Step 3.a6: Perform an IDWT operation on each uncompressed low-frequency region and the corresponding uncompressed high-frequency region to obtain an uncompressed video; Step 3.a7: Calculate the loss L of the distorted video and the corresponding label corresponding to each video clip sample noise , L noise =MSE(v noise ,y com )+MSE(y rev ,v origin ), where MSE(·) represents the mean square error loss function, v origin Represents a video clip sample, v noise Represents the distorted video corresponding to the video clip sample, y rev Represents the label corresponding to the video clip sample. The uncompressed video obtained by reverse inputting the distortion layer, y com Indicates the label corresponding to the video clip sample; Step 3.a8: Backpropagate and update the gradient; then iterate the process from step 3.a3 to step 3.a7 until the mean of the loss function of the distorted video and the corresponding label corresponding to all video clip samples in the pre-training set converges, completing the distortion layer pre-training.

10. The video watermarking method based on reversible neural network according to claim 1 is characterized in that In step 3, the process of training the watermark embedding and extraction network is as follows: Step 3.b1: Initialize the weights, learning rate, and network parameters of the watermark embedding extraction network; Step 3.b2: Input each video clip sample in the training set and the corresponding embedding end watermark information, as well as the all-zero vector, into the watermark embedding extraction network, which then outputs the extraction end watermark information. Step 3.b3: Calculate the total network loss L total , L total =λ1L video +λ2L message +λ3L dis , where λ1, λ2 and λ3 are used to balance L video , L message and L dis Contribution weight parameter, L video represents the video loss used to quantify the mean square error between the video clip samples and the watermarked video output by the watermark embedding module, L video =MSE(c,s), where MSE(·) represents the mean square error loss function, c represents the video clip sample, s represents the watermarked video output by the watermark embedding module, and L message It represents the message loss used to quantify the mean square error between the watermark information at the embedding end and the watermark information at the extraction end output by the watermark extraction module, L message =MSE(M,M ex ), M represents the embedded end watermark information, M ex Indicates the watermark information extracted by the watermark extraction module, L dis Denotes the adversarial loss, L dis =CrossEntropy(dis(s),label), CrossEntropy(·) represents the cross entropy loss function, dis(·) represents the neural network structure discriminator used to determine whether the watermarked video has a watermark, and label represents an all-one vector label; Step 3.b4: Backpropagate and update the gradient; then iterate the process from step 3.b2 to step 3.b3, setting λ3 to 0 during the first 20 iterations and setting λ3 to non-zero during subsequent iterations; end the iteration process when the mean of the total network loss for all training sets converges, completing the watermark embedding extraction network training.

Images