Video watermark embedding and extracting method and system

Abstract

The invention discloses a video watermark embedding and extracting method and system, and the method comprises the steps: carrying out the preprocessing of an original video frame sequence, and obtaining a preprocessed watermark video frame; and inputting the watermark video frame into a decoder based on a 3D convolutional neural network to output the extracted watermark information, so that the problem of watermark flicker can be effectively solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and particularly relates to a video watermark embedding and extraction method and system. BACKGROUND

[0002] The existing video watermark method mainly has two technical defects in solving the watermark time continuity problem: one is to copy a single frame watermark to multiple frames, which does not consider the dynamic correlation between video frames, resulting in a lack of consistency of inter-frame watermark features and being unable to fundamentally suppress the flicker phenomenon; the other is to input multiple frames of images into a model to generate multiple frames of watermark images and only rely on a loss function for constraint, but the constraint dimension is single and it is difficult to accurately control the time domain continuity of inter-frame watermarks, so the watermark flicker problem cannot be effectively solved, which is a key technical problem to be solved in the current field. SUMMARY

[0003] The purpose of the application is to provide a video watermark embedding and extraction method and system to solve the deficiencies in the prior art. The application first pre-processes the original video frame sequence, and then outputs watermark information based on the decoder of the 3D convolutional neural network, which can effectively solve the watermark flicker problem.

[0004] One embodiment of the application provides a video watermark embedding and extraction method, which comprises: performing pre-processing on the original video frame sequence to obtain a pre-processed watermark video frame; inputting the watermark video frame into a decoder based on a 3D convolutional neural network to output extracted watermark information.

[0005] Optionally, the pre-processing of the original video frame sequence to obtain the pre-processed watermark video frame comprises: inputting the original video frame sequence into an encoder to output a continuous watermark residual, superimposing the watermark residual and the original video frame sequence to generate a video frame with watermark.

[0006] Optionally, the encoder is composed of an Unet module based on a 3D convolutional neural network and a visual perception threshold module; the watermark residual is obtained by the Unet module outputting features and the visual perception heat map generated by the visual perception threshold module, and the mathematical expression of the watermark residual generation is:

[0007] wherein, represents the watermark residual output by the encoder, represents a load factor, represents the original video frame sequence output by the visual perception threshold module, represents matrix element multiplication, video features representing fused watermark information output by the Unet module, network parameters representing the encoder, original video frame sequence, watermark information to be embedded.

[0008] Optionally, the method further comprises: inputting the watermark video frame into a decoder based on a 3D convolutional neural network, to generate extracted watermark information through a layer-by-layer down-sampling architecture and a terminal fully connected layer and a Sigmoid activation function; wherein a mathematical expression of the watermark information extraction is:

[0009] wherein, watermark information extracted from the watermark video frame, decoder based on a 3D convolutional neural network, network parameters of the decoder, watermark video frame sequence.

[0010] Optionally, after the pre-processing of the original video frame sequence is performed to obtain the pre-processed watermark video frame, the method further comprises: using a parameter-fixed optical flow calculation module to process the original video frame sequence and the watermark video frame sequence respectively to obtain an original video optical flow field and a watermark video optical flow field; using a loss function to constrain consistency of the original video optical flow field and the watermark video optical flow field, to realize that the original video optical flow field and the watermark video optical flow field are approximately consistent.

[0011] Optionally, the loss function is represented by:

[0012] wherein, watermark video optical flow field, original video optical flow field, is a minimum constant.

[0013] Yet another embodiment of the present application provides a video watermark embedding and extracting system, the system comprising: an execution module configured to perform pre-processing of an original video frame sequence to obtain a pre-processed watermark video frame; an output module configured to input the watermark video frame into a decoder based on a 3D convolutional neural network, to output extracted watermark information. ​

[0014] Optionally, the execution module comprises: The generating unit is configured to input the original video frame sequence into an encoder to output continuous watermark residuals, superimpose the watermark residuals and the original video frame sequence, and generate a video frame with watermark.

[0015] Another embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and the computer program is configured to implement the method of any one of the above embodiments when running.

[0016] Another embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the method of any one of the above embodiments.

[0017] Compared with the prior art, the present application firstly performs preprocessing on the original video frame sequence to obtain a preprocessed watermark video frame; and inputs the watermark video frame into a decoder based on a 3D convolutional neural network to output extracted watermark information, which can effectively solve the watermark flickering problem. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A hardware structure block diagram of a computer terminal of a video watermark embedding and extraction method provided by the embodiment of the present application is provided. Figure 2 A flowchart of a video watermark embedding and extraction method provided by the embodiment of the present application is provided. Figure 3 A frame diagram of a video watermark embedding and extraction provided by the embodiment of the present application is provided. Figure 4 A watermark embedding detail diagram provided by the embodiment of the present application is provided. Figure 5 An optical flow consistency constraint method diagram provided by the embodiment of the present application is provided. Figure 6 An overall frame diagram of a video watermark embedding and extraction method based on optical flow consistency constraint and two-stage training provided by the embodiment of the present application is provided. Figure 7 A structure diagram of a video watermark embedding and extraction system provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0019] The embodiments described below with reference to the drawings are exemplary and are only used to explain the present application, and cannot be explained as a limitation of the present application.

[0020] The embodiment of the present application firstly provides a video watermark embedding and extracting method, which can be applied to an electronic device, such as a computer terminal, specifically, a general computer, a tablet and the like.

[0021] The following will be described in detail taking a computer terminal as an example. Figure 1 A hardware structure block diagram of a computer terminal of the video watermark embedding and extracting method provided by the embodiment of the present application is shown in FIG. 1. Figure 1 As shown in the figure, the computer device comprises a processor, a memory and a network interface connected through a system bus, wherein the memory can comprise a non-volatile storage medium and an internal memory.

[0022] The non-volatile storage medium can store an operating system and a computer program. The computer program comprises program instructions, which, when executed, can make the processor execute any kind of video watermark embedding and extracting method.

[0023] The processor is used to provide computing and control capabilities to support the operation of the entire computer device.

[0024] The internal memory provides an environment for the running of the computer program in the non-volatile storage medium, which, when executed by the processor, can make the processor execute any kind of video watermark embedding and extracting method.

[0025] The network interface is used for network communication, such as sending assigned tasks and the like. Those skilled in the art can understand that Figure 1 The structure shown in FIG. 1 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can comprise more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0026] It should be understood that the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0027] Referring to Figure 2 , Figure 2A flowchart of a video watermark embedding and extraction method provided by an embodiment of the present application can include the following steps: S201: performing preprocessing on the original video frame sequence to obtain a preprocessed watermark video frame.

[0028] Specifically, the step of performing preprocessing on the original video frame sequence to obtain a preprocessed watermark video frame can include: inputting the original video frame sequence into an encoder to output continuous watermark residuals, and superimposing the watermark residuals and the original video frame sequence to generate a video frame with watermark.

[0029] For example, the present application can adopt a "multiple frame-in-multiple frame-out" watermark embedding mode, and the specific process is as shown in Figure 3 Figure 3 A framework diagram of a video watermark embedding and extraction method provided by an embodiment of the present application is shown in the figure, wherein, represents an original video frame sequence, represents watermark information to be embedded, Encoder represents an encoder, represents watermark residuals output by the encoder, represents a watermark video frame sequence, Decoder represents a decoder, represents watermark information extracted from the watermark video frame. Figure 3 An 8-frame continuous original video frame sequence is input into the encoder, and the encoder outputs 8-frame continuous watermark residuals; the watermark residuals are superimposed with the original video frame sequence to generate a video frame with watermark, and the mathematical expression is as follows:

[0030]

[0031] wherein, is the input continuous original video frame sequence, is the watermark residuals output by the encoder, is the network parameter of the encoder, is the watermark information to be embedded.

[0032] It should be noted that the encoder is composed of an Unet module constructed based on a 3D convolutional neural network and a visual perception threshold module; the watermark residuals are obtained through the Unet module output feature and the visual perception heat map generated by the visual perception threshold module, and the mathematical expression for generating the watermark residuals is as follows:

[0033] wherein, represents the watermark residuals output by the encoder, represents a load factor, ​a sequence of original video frames representing a visual perception threshold module output, representing element-wise multiplication of matrices, a video feature representing fused watermark information output by the Unet module, network parameters of the encoder, a sequence of original video frames, watermark information to be embedded.

[0034] Specifically, in addition to solving the watermark flicker problem, the present application also meets the core requirement of watermark invisibility, and the specific implementation details are as follows Figure 4 as shown, Figure 4 A watermark embedding detail diagram is provided for an embodiment of the present application, wherein the encoder is composed of a Unet module and a JND (Just Noticeable Difference, visual perception threshold) module based on a 3DCNN. The two modules work together to ensure watermark invisibility. In the diagram, the core function of the Unet module is to extract the spatiotemporal features of 8 consecutive video frames, and the last layer of the module uses a tanh activation function to constrain the feature distribution, realizing effective fusion of watermark information and video frame features. The core function of the JND module is to generate a visual perception heat map of the original video frames, which generates higher values in complex areas of the video picture (such as texture-dense areas and edge areas) and lower values in flat areas (such as solid color background areas). Multiplying the features output by the Unet module with the heat map generated by the JND module results in the final watermark residual; this design allows the watermark to be embedded in complex areas that are difficult for the human eye to perceive, embedding sufficient watermark information under the premise of being imperceptible to the human eye, and without the need for additional introduction of a visual quality loss function for constraint.

[0035] S202: inputting the watermark video frame into a decoder based on a 3D convolutional neural network to output extracted watermark information.

[0036] Specifically, the inputting of the watermark video frame into the decoder based on the 3D convolutional neural network to output the extracted watermark information can include: inputting the watermark video frame into the decoder based on the 3D convolutional neural network to generate the extracted watermark information through a layer-by-layer down-sampling architecture and a terminal fully connected layer and a Sigmoid activation function; wherein the mathematical expression of the watermark information extraction is:

[0037] wherein, extracted watermark information from the watermark video frame, a decoder based on a 3D convolutional neural network, network parameters of the decoder, a sequence of watermark video frames.

[0038] Specifically, the watermark extraction in this application also adopts the "multi-frame advance" mode, using a layer-by-layer downsampling architecture based on 3D Convolutional Neural Network (3DCNN). The network ends through a fully connected layer and a Sigmoid activation function to output the final extracted watermark information.

[0039] In one optional implementation, after performing preprocessing on the original video frame sequence to obtain the preprocessed watermarked video frames, the method may further include: Step 1: Using an optical flow calculation module with fixed parameters, process the original video frame sequence and the watermarked video frame sequence respectively to obtain the optical flow field of the original video and the optical flow field of the watermarked video.

[0040] Step 2: Utilize The loss function constrains the consistency between the original video optical flow field and the watermarked video optical flow field, so as to achieve approximately consistency between the original video optical flow field and the watermarked video optical flow field.

[0041] Specifically, the aforementioned The loss function is expressed as follows:

[0042] in, This represents the optical flow field of the watermarked video. This represents the optical flow field of the original video. It is a very small constant.

[0043] For details, see Figure 5 , Figure 5 This is a schematic diagram illustrating an optical flow consistency constraint method provided in an embodiment of the present invention. The diagram suppresses the flickering phenomenon of watermarked videos from the temporal dimension. The specific implementation logic is as follows: Using a fixed-parameter optical flow calculation module in an optical flow network, the original video frame sequence is processed respectively. With watermark video frame sequence The original video optical flow field was obtained through processing. With watermark video optical flow field The expression for optical flow calculation is: , where V is the input video frame sequence.

[0044] Among them, the introduction loss function, for and Constraining consistency allows for the gradual optimization of the optical flow field in the watermarked video during model training, ultimately achieving... and The watermark flicker phenomenon is effectively inhibited. Loss function table The constant is usually close to 0, and the loss function is smooth when the error approaches 0, avoiding gradient mutation.

[0045] It can be seen that the core idea of solving the watermark flicker problem of the application is as follows: the time domain continuity characteristic of the optical flow is used, and in the watermark video with obvious flicker phenomenon, the watermark embedding will disturb the original optical flow field of the video. Based on this, the optical flow consistency constraint mechanism is introduced to regulate the watermark generation process, so as to ensure that the optical flow field of the generated watermark video is consistent with the original video optical flow field, thereby essentially solving the flicker problem of the video watermark.

[0046] Compared with the prior art, the application first performs preprocessing on the original video frame sequence to obtain a preprocessed watermark video frame; the watermark video frame is input into a decoder based on a 3D convolutional neural network to output extracted watermark information, which can effectively solve the watermark flicker problem.

[0047] In an optional embodiment, in order to solve the problem that, for mainstream video compression standards such as H.264, H.265, etc., due to the non-differentiable characteristics of the compression process, the effective propagation of the gradient cannot be realized in the network training stage, and then it is difficult to complete the end-to-end optimization training of the model, so that the existing watermark algorithm has weak resistance to such non-differentiable noise, the application also provides a video watermark embedding and extraction method based on optical flow consistency constraint and two-stage training.

[0048] Referring to Figure 6 , Figure 6 The overall framework schematic diagram of the video watermark embedding and extraction method based on optical flow consistency constraint and two-stage training provided by the embodiment of the application, wherein, for the problem of video watermark resistance to non-differentiable noise, the application adopts a two-stage training strategy, in the first stage, a random differentiable network noise pool is constructed, which is input as a training disturbance to realize the overall end-to-end training of the model, so that the model reaches an initial convergence state and has the ability to resist most common differentiable noise; in the second stage, the parameters of the model encoder are frozen, and the decoder is trained alone, and H.264 and other non-differentiable noise are introduced in the noise pool for targeted training, so that the model is further converged on the basis of maintaining the existing performance, and finally the effective resistance to H.264 and other non-differentiable noise is obtained.

[0049] The overall network training strategy includes adopting an end-to-end training mode, combining the two-stage training strategy to optimize the network performance, and ensuring that the model meets the requirements of watermark invisibility, robustness and extraction accuracy, in particular: First stage: basic performance training (guaranteeing differentiable noise robustness and invisibility) In this stage, differentiable noise is selected to construct a noise pool, including common differentiable interference types such as Gaussian noise, Gaussian blur, frame averaging, frame loss, etc. The training process is divided into three steps: Initial convergence stage: To ensure the stability of model convergence, the noise layer and JND module are not enabled in the initial training stage (the first several rounds), and only the Unet module is used to generate video features. After being constrained by the tanh activation function, multiply by the load factor , get the watermark residual and superimpose it with the original video frame to generate a watermark video; continue training until the average accuracy of watermark extraction reaches more than 95% in the current round, and complete the initial convergence.

[0050] Differentiable noise robustness training: enable the noise layer, randomly apply differentiable noise in the noise pool to the watermark video frame, and optimize the model parameters through back propagation to enable the model to resist most differentiable noise; continue training until the average accuracy of watermark extraction reaches more than 95% again.

[0051] Invisibility optimization training: enable the JND module, combine the JND heat map in the previous text with the watermark residual generated by the Unet feature to optimize the watermark embedding position selection, so that the model can meet the invisibility requirement while maintaining robustness.

[0052] Second stage: special robustness training (resisting non-differentiable noise) After the first stage of training is completed and the basic performance of the model is stable, enter the second stage of training to focus on improving the model's resistance to non-differentiable noise.

[0053] Parameter freezing: freeze all network parameters of the encoder to avoid destroying the optimized watermark embedding logic in subsequent training; Non-differentiable noise introduction: add H.264 and other non-differentiable video compression noise to the noise pool; Decoder special training: only train the decoder, optimize the watermark extraction logic of the decoder through the disturbance feedback of non-differentiable noise, and make the model further converge to eventually have effective resistance to H.264 and other non-differentiable noise.

[0054] To improve the watermark extraction accuracy of the decoder, the present invention introduces binary cross-entropy loss ( ); combined with the loss of optical flow consistency constraint, construct the overall loss function ( ), realize multi-dimensional optimization of model performance. The specific expression is as follows:

[0055] Among them, denotes the binary cross-entropy loss function, is the bit length of the watermark information, and and are the true label (0 or 1) of the i-th watermark bit and the predicted probability of the i-th watermark bit by the decoder, respectively.

[0056] The overall loss function is:

[0057] wherein, , is the binary cross-entropy loss, is the weight coefficient of the loss, which can be adjusted according to the actual training requirements, and is used to balance the priority of the watermark extraction accuracy and the optical flow consistency constraint.

[0058] It should be noted that the present application is mainly implemented using the deep learning framework Pytorch, but can also be implemented using deep learning frameworks such as TensorFlow, Caffe, etc.

[0059] For convenience of explanation, refer to Table 1 and Table 2, Table 1 is a comparison table of decoding accuracy of different video watermarking methods for different noises; and Table 2 is a comparison table of generation quality of different video watermarking methods.

[0060] Table 1 Comparison table of decoding accuracy of different video watermarking methods for different noises

[0061] Table 2 Comparison table of generation quality of different video watermarking methods

[0062] It can be seen that the present application realizes significant improvement in the three core dimensions of flicker suppression, robustness enhancement and visual quality guarantee of video watermarking through innovative technical design and scientific experimental verification. The beneficial effects thereof will be described in detail below in combination with experimental data and technical principles: (1) Experimental design description In order to objectively verify the technical advantages of the present application, the mainstream video watermarking method is selected as the comparison benchmark, and standardized evaluation indicators and diversified noise attack scenes are used to ensure the fairness and reliability of the comparison results: Comparison method: Select REVMark and VideoSeal two public methods, both use the official open source training model. Among them, the input size of REVMark is 128x128, and the embedded 96bit watermark; the input size of VideoSeal is 256x256, and the embedded 96bit watermark; the input size of the application is 128x128, and the embedded 64bit watermark, all methods adjust the watermark intensity to the same level to eliminate the influence of intensity difference.

[0063] Training and testing data set: Model training is based on Kinetics-400 training set, and all performance tests are completed on Kinetics-400 test set.

[0064] Evaluation index: Three types of video quality evaluation indexes are adopted to comprehensively cover static and dynamic quality: 1. PSNR (peak signal-to-noise ratio) and SSIM (structural similarity): Focus on single-frame static picture quality, reflect the influence of watermark embedding on video pixel-level structure and brightness; 2. VMAF (Video Multi-Method Assessment Fusion): Accurately capture video dynamic quality, which can effectively identify time domain quality problems such as flicker and stall that pixel-level indicators cannot reflect.

[0065] Noise attack type: Covering two typical noises in space and time, covering common interference scenes in actual application: 1. Spatial noise: Gaussian noise (variance = 0.04), Gaussian blur (kernel size = 3, standard deviation = 2), random cropping (cropping ratio = 0.4); 2. Time domain noise: Frame loss (loss probability = 0.5), frame averaging (average frame number N = 3), frame exchange (exchange probability = 0.5); 3. Video compression noise: H.264 / AVC (CRF = 22).

[0066] (2) Analysis of experimental results 1. Robustness is significantly better than existing technology From Table 1, the decoding accuracy of different video watermarking methods under different noise can be seen, the decoding accuracy of the application is significantly better than the two comparison methods of REVMark and VideoSeal under all noise attack scenarios: When facing H.264 / AVC compression noise, the decoding accuracy of the application is 99.00%, which is about 4.82% and 2.70% higher than REVMark (94.18%) and VideoSeal (96.30%) respectively; In the presence of strong interference scenarios such as Gaussian noise (var = 0.04) and random cropping (p = 0.4), the decoding accuracy of the present application is 99.82% and 98.34% respectively, far exceeding the highest level (88.43%, 77.90%) of the comparative method; Under the time domain noise attack of frame loss (p = 0.5) and frame exchange (p = 0.5), the decoding accuracy of the present application is maintained at 99.96% and 100.00%, showing strong time domain anti-interference ability.

[0067] The above results prove that the present application has excellent robustness against complex noise in space, time and compression through innovative training strategy and model design.

[0068] 2. Excellent video visual quality, effectively solves the flicker problem From Table 2, the generation quality comparison of different video watermarking methods shows that the present application ensures high robustness while optimizing the video visual quality.

[0069] In terms of static quality, the PSNR (40.10115037) and SSIM (0.961496316) indexes of the present application are at the same excellent level as the comparative method, indicating that the watermark embedding does not cause significant damage to the pixel structure and brightness of single frame pictures; in terms of dynamic quality, the VMAF index (83.8619888) of the present application is significantly higher than that of REVMark (67.7277743) and VideoSeal (62.0741247), and the VMAF index can accurately capture problems such as flicker and interframe inconsistency in dynamic video playback. This result directly proves that the present application has effectively suppressed the flicker phenomenon of the watermark video by designing a light flow consistency constraint mechanism, achieving a significant improvement in dynamic visual quality.

[0070] (3) Summary of core beneficial effects The present application effectively solves the watermark flicker problem and improves the dynamic visual experience. The present application innovatively reveals the correlation between light flow field disturbance and watermark flicker, designs a light flow consistency constraint mechanism, and through the loss function forces the light flow field of the watermark video to be consistent with the original video, solving the time domain flicker defect, and the significant advantage of the VMAF index verifies the effectiveness of this design.

[0071] Robustness is comprehensive and leads the field, adapting to complex application scenarios. Through the two-stage training strategy, i.e. micro-noise basic training, non-differentiable noise special training and encoder collaborative design, the present application realizes strong resistance to various complex noises such as Gaussian noise, compression interference and frame operation without sacrificing visual quality, and the decoding accuracy is significantly better than existing methods in all scenarios, adapting to the copyright protection needs of video transmission, storage and dissemination in multiple scenarios.

[0072] Balancing the core performance, taking into account invisibility and practicality. The JND module and Unet feature fusion design of the application makes the watermark preferentially embedded in the complex area difficult for the human eye to perceive, while ensuring the invisibility of the watermark without additional loss function constraints on visual quality; combined with the comprehensive excellent performance of PSNR, SSIM and VMAF, the three-dimensional balance of robustness-invisibility-dynamic quality is realized, which has very high practical application value.

[0073] Another embodiment of the application provides a video watermark embedding and extraction system, as shown in Figure 7 The structure diagram of a video watermark embedding and extraction system, the system comprises: The execution module 701 is used for pre-processing the original video frame sequence to obtain a pre-processed watermark video frame; The output module 702 is used for inputting the watermark video frame into a decoder based on a 3D convolutional neural network to output extracted watermark information.

[0074] Specifically, the execution module comprises: The generation unit is used for inputting the original video frame sequence into an encoder to output continuous watermark residuals, and superimposing the watermark residuals and the original video frame sequence to generate a video frame with watermark.

[0075] Compared with the prior art, the application first pre-processes the original video frame sequence to obtain a pre-processed watermark video frame; inputs the watermark video frame into a decoder based on a 3D convolutional neural network to output extracted watermark information, which can effectively solve the watermark flicker problem.

[0076] The embodiment of the application further provides a storage medium, and the storage medium stores a computer program, wherein the computer program is set to realize the steps in the above method embodiment when running.

[0077] Specifically, in the embodiment, the above storage medium can be set to store a computer program for executing the following steps: S201: pre-processing the original video frame sequence to obtain a pre-processed watermark video frame; S202: inputting the watermark video frame into a decoder based on a 3D convolutional neural network to output extracted watermark information.

[0078] Specifically, in the embodiment, the above storage medium can include but is not limited to: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0079] Compared with the prior art, the pre-processing of the original video frame sequence is first performed to obtain the pre-processed watermark video frame; the watermark video frame is input into the decoder based on the 3D convolutional neural network to output the extracted watermark information, and the watermark flickering problem can be effectively solved.

[0080] The embodiment of the application further provides an electronic device comprising a memory and a processor, the memory storing a computer program, and the processor is configured to run the computer program to execute the steps in the above method embodiments.

[0081] Specifically, the electronic device can further comprise a transmission device connected with the processor and an input and output device connected with the processor.

[0082] Specifically, in the embodiment, the processor can be configured to execute the following steps through the computer program: S201: performing pre-processing on the original video frame sequence to obtain pre-processed watermark video frames; S202: inputting the watermark video frame into the decoder based on the 3D convolutional neural network to output the extracted watermark information.

[0083] Compared with the prior art, the pre-processing of the original video frame sequence is first performed to obtain the pre-processed watermark video frame; the watermark video frame is input into the decoder based on the 3D convolutional neural network to output the extracted watermark information, and the watermark flickering problem can be effectively solved.

[0084] It should be noted that, for the above-mentioned method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the application is not limited by the described action sequence, because according to the application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the application.

[0085] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0086] In several embodiments of the present application, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the division of the apparatus embodiments is merely illustrative, and the division of the units can be changed according to actual needs. For example, the units or components illustrated or discussed as separate parts can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0087] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0088] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0089] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0090] The embodiments of the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range can be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for embedding and extracting video watermarks, characterized in that, The method includes: Perform preprocessing on the original video frame sequence to obtain preprocessed watermarked video frames; The watermarked video frame is input into a decoder based on a 3D convolutional neural network to output the extracted watermark information.

2. The method according to claim 1, characterized in that, The step of performing preprocessing on the original video frame sequence to obtain preprocessed watermarked video frames includes: The original video frame sequence is input into the encoder to output a continuous watermark residual. The watermark residual is then superimposed on the original video frame sequence to generate a watermarked video frame.

3. The method according to claim 2, characterized in that, The encoder consists of a Unet module built based on a 3D convolutional neural network and a visual perception thresholding module; the watermark residual is obtained by combining the output features of the Unet module with the visual perception heatmap generated by the visual perception thresholding module, and the mathematical expression for generating the watermark residual is: in, This represents the watermark residual output by the encoder. Indicates the load factor. This represents the original video frame sequence output by the visual perception threshold module. This indicates that the matrices are multiplied element by element. This represents the video features output by the Unet module that incorporate watermark information. Represents the network parameters of the encoder. Indicates the original video frame sequence. This indicates that watermark information is to be embedded.

4. The method according to claim 3, characterized in that, The step of inputting the watermarked video frame into a decoder based on a 3D convolutional neural network to output the extracted watermark information includes: The watermarked video frame is input into a decoder based on a 3D convolutional neural network. Through a layer-by-layer downsampling architecture and a fully connected layer at the end followed by a sigmoid activation function, the extracted watermark information is generated. The mathematical expression for the watermark information extraction is as follows: in, This indicates the watermark information extracted from the watermarked video frame. This represents a decoder based on a 3D convolutional neural network. Indicates the network parameters of the decoder. This represents the sequence of watermarked video frames.

5. The method according to claim 4, characterized in that, After performing preprocessing on the original video frame sequence to obtain the preprocessed watermarked video frames, the method further includes: Using a fixed-parameter optical flow calculation module, the original video frame sequence and the watermarked video frame sequence are processed respectively to obtain the optical flow field of the original video and the optical flow field of the watermarked video. use The loss function constrains the consistency between the original video optical flow field and the watermarked video optical flow field, so as to achieve approximately consistency between the original video optical flow field and the watermarked video optical flow field.

6. The method according to claim 5, characterized in that, The The loss function is expressed as follows: in, This represents the optical flow field of the watermarked video. This represents the optical flow field of the original video. It is a very small constant.

7. A video watermark embedding and extraction system, characterized in that, The system includes: The execution module is used to perform preprocessing on the original video frame sequence to obtain preprocessed watermarked video frames; The output module is used to input the watermarked video frame into a decoder based on a 3D convolutional neural network to output the extracted watermark information.

8. The system according to claim 7, characterized in that, The execution module includes: The generation unit is used to input the original video frame sequence into the encoder to output a continuous watermark residual, and to superimpose the watermark residual with the original video frame sequence to generate a watermarked video frame.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to implement the method of any one of claims 1 to 6 when it is run.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to implement the method of any one of claims 1 to 6.

Images