Video watermark processing method and device, equipment, storage medium and product

Abstract

The invention provides a video watermark processing method and device, equipment, a storage medium and a product, and relates to the field of image processing, and the method comprises the steps: obtaining a chrominance component of an original video frame; embedding a symmetric watermark template into the chrominance component to generate a target video frame; processing the traceability watermark information through a deep learning watermark model to obtain a watermark-containing feature map; and embedding the feature map containing the watermark into the target video frame to generate a video containing the watermark. According to the invention, the video with leakage traceability can be generated.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a video watermarking method, apparatus, device, storage medium, and product. Background Technology

[0002] Currently, videos collected by home security monitoring and intelligent robots often contain sensitive information such as user behavior and living environment. These are prone to leakage during data sharing and dissemination, making it crucial to trace the source of leaks to ensure security and privacy. Video watermarking technology, by hiding source information within the video content, can provide effective support for tracing the source. However, with existing watermarking technologies, the source information directly embedded in the video is easily destroyed under complex processing or malicious attacks, making effective source tracing impossible. Summary of the Invention

[0003] This invention provides a video watermarking method, apparatus, device, storage medium, and product to solve the technical problem that the traceability information embedded in videos using existing watermarking technology is easily destroyed, making it impossible to effectively trace the source of leakage.

[0004] This invention provides a video watermarking method, comprising the following steps: Obtain the chroma components of the original video frames; Embed the symmetrical watermark template into the chroma components to generate the target video frame; The source watermark information is processed by a deep learning watermark model to obtain a watermark feature map; The watermarked feature map is embedded into the target video frame to generate a watermarked video.

[0005] According to a video watermarking method provided by the present invention, before embedding a symmetrical watermark template into the chroma component to generate a target video frame, the method further includes: Retrieve preset synchronization messages; The preset synchronization message is converted into a two-dimensional matrix, and the two-dimensional matrix is ​​spread into a symmetric watermark unit using a spreading matrix; The symmetric watermark unit is scrambled using a scrambling matrix to obtain the target symmetric watermark unit. The target symmetrical watermark unit is flipped to obtain a symmetrical watermark template.

[0006] According to a video watermarking method provided by the present invention, the step of processing the source watermark information through a deep learning watermarking model to obtain a watermarked feature map includes: Extract the image region to be embedded from the original video frame; The source watermark information is processed by the watermark processor of the deep learning watermark model to obtain a reconstructed watermark feature map. The decoder of the deep learning watermarking model determines the target embedding strategy based on the original video frames. The encoder of the deep learning watermarking model processes the image region to be embedded and the reconstructed watermark feature map according to the target embedding strategy to obtain a watermarked feature map.

[0007] According to a video watermarking method provided by the present invention, before processing the source watermark information through a deep learning watermarking model to obtain a watermarked feature map, the method further includes: The process involves acquiring watermarked video samples and an initial deep learning watermark model. The watermarked video samples include original video frame samples and source watermark information samples. The initial deep learning watermark model includes a watermark processor, an encoder, a combined noise layer, and a decoder. Based on the watermarked video samples, the initial deep learning watermark model is trained to obtain a deep learning watermark model.

[0008] According to a video watermarking method provided by the present invention, the decoder includes a watermark extraction branch; the step of training the initial deep learning watermarking model based on the watermarked video sample to obtain a deep learning watermarking model includes: The watermarked video sample is input into the initial deep learning watermarking model, and the source tracing watermark information sample is processed by the watermark processor to obtain a reconstructed feature map; The encoder processes the reconstructed feature map to obtain a first watermarked feature map; The first watermarked feature map is attacked by the combined noise layer to obtain the second watermarked feature map. The second watermarked feature map is processed by the watermark extraction branch of the decoder to obtain the target reconstructed feature map; Based on the original video frame samples, the reconstructed feature map, the first watermarked feature map, and the target reconstructed feature map, the overall loss value is determined, and the initial deep learning watermarking model is optimized according to the overall loss value to obtain a trained deep learning watermarking model.

[0009] According to a video watermarking method provided by the present invention, the decoder further includes an image texture prediction branch; wherein, the step of processing the reconstructed feature map through the encoder to obtain a first watermarked feature map includes: The image texture prediction branch analyzes the image regions in the original video frame samples to generate an image texture prediction feature map. The encoder determines an adaptive watermark embedding strategy based on the image texture prediction feature map. The encoder processes the reshaped feature map according to the adaptive embedding strategy to generate a first watermarked feature map.

[0010] According to a video watermarking method provided by the present invention, the initial deep learning watermarking model further includes an adversarial discriminator; wherein, determining the overall loss value based on the original video frame samples, the reconstructed feature map, the first watermarked feature map, and the target reconstructed feature map includes: The carrier image is determined based on the first watermark feature; The adversarial discriminator calculates the adversarial loss value based on the original video frame samples and the carrier image. The image loss value is determined based on the reconstructed feature map and the first watermarked feature map, and the watermark loss value is determined based on the reconstructed feature map and the target reconstructed feature map. The overall loss value is determined based on the image loss value, the watermark loss value, and the adversarial loss value.

[0011] According to a video watermarking method provided by the present invention, after embedding the watermarked feature map into the target video frame to generate a watermarked video, the method further includes: Obtain the watermarked video to be traced; The symmetrical watermark template state of the video frame to be traced is detected, and based on the symmetrical watermark template state, the watermarked video to be traced is restored to the target traceable video. Extract the target watermarked region from the target video frame to be traced in the target source-tracing video; The deep learning model extracts the target source watermark information from the target watermark area, and uses the target source watermark information to trace the source of video leakage.

[0012] According to a video watermarking method provided by the present invention, the step of detecting the symmetrical watermark template state of the watermarked video to be traced, and restoring the watermarked video to be traced into the target traceable video based on the symmetrical watermark template state, includes: Extract the source-traceable chroma component from the source-traceable watermarked video frames; Wiener filtering is performed on the chromaticity components to be traced to obtain filtered chromaticity data; The spatial symmetry of the filtered chromaticity data is calculated using a self-convolution function; Based on the adaptive threshold, a symmetric peak feature map is calculated according to the spatial symmetry. The state of the symmetric watermark template in the symmetric peak feature map is detected by Hough transform, and the geometric attack parameters are determined based on the state of the symmetric watermark template. Based on the geometric attack parameters, the watermarked video to be traced is restored to the target video for tracing.

[0013] The present invention also provides a video watermarking processing device, comprising the following modules: The acquisition module is used to acquire the chroma components of the original video frames; A generation module is used to embed a symmetrical watermark template into the chroma component to generate a target video frame; The generation module is also used to process the source watermark information through a deep learning watermark model to obtain a watermark feature map. The generation module is further configured to embed the watermarked feature map into the target video frame to generate a watermarked video.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the video watermarking processing method as described above.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video watermarking processing method as described above.

[0016] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the video watermarking processing method as described above.

[0017] This invention provides a video watermarking processing method, apparatus, device, storage medium, and product. The method involves: acquiring the chroma components of an original video frame; embedding a symmetrical watermark template into the chroma components to generate a target video frame; processing the source watermark information using a deep learning watermark model to obtain a watermarked feature map; and embedding the watermarked feature map into the target video frame to generate a watermarked video. This solves the technical problem that source information embedded in videos using existing watermarking technologies is easily destroyed, making effective source tracing impossible. Compared to existing technologies, this application, by dual embedding a symmetrical watermark template and a source watermark feature map generated by deep learning, not only effectively resists geometric attacks and ensures stable location and extraction of source information at the decoding end, but also enables the encoded video data to have source tracing capabilities while maintaining high visual quality and low bitrate overhead. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 This is one of the flowcharts illustrating the video watermarking method provided by the present invention.

[0020] Figure 2 This is a structural diagram of the deep learning watermarking model for the video watermarking processing method provided by this invention.

[0021] Figure 3 This is a schematic diagram of the symmetrical watermarking unit of the video watermarking processing method provided by the present invention.

[0022] Figure 4 This is a schematic diagram illustrating the symmetry of watermark flipping in the video watermarking processing method provided by the present invention.

[0023] Figure 5 This is a symmetrical peak feature map of the video watermarking processing method provided by the present invention.

[0024] Figure 6 This is the second flowchart of the video watermarking method provided by the present invention.

[0025] Figure 7 This is a schematic diagram of the video watermarking processing device provided by the present invention.

[0026] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] The following is combined with Figure 1 and Figure 6 The present invention describes a video watermarking processing method applicable to any video watermarking process. The subject executing this method can be an electronic device or a video watermarking processing device installed in the electronic device. The video watermarking processing device can be implemented by software, hardware, or a combination of both.

[0029] Figure 1 This is one of the flowcharts illustrating the video watermarking method provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 101: Obtain the chroma components of the original video frame; It should be noted that the original video frame refers to the video frame to which the watermark is to be embedded; the chromaticity component refers to the U component in the YUV components of the original video frame. Since the U component mainly carries the blue color difference information of the image, its pixel value changes are far less sensitive to the human visual system than the luminance component (Y). This characteristic makes it an ideal carrier for watermark embedding, because small perturbations or structural modifications applied to this component (such as embedding a specific template) are unlikely to cause significant subjective visual distortion, which is beneficial to maintaining the visual quality of the video after watermark embedding. At the same time, compared with the other chromaticity component (V, red color difference information), the U component is usually subjected to stronger downsampling and quantization in common video compression standards. If the watermark embedded here can survive under these conditions, it means that it has a natural tolerance to subsequent compression processing, thus achieving a good starting point for a balance between concealment and robustness. Therefore, choosing the U component as the embedding object is an intentional design that combines human visual characteristics and video processing characteristics, aiming to prioritize the invisibility of the watermark, and then enhance its survivability through subsequent template design.

[0030] Step 102: Embed the symmetrical watermark template into the chroma component to generate the target video frame; It should be noted that after obtaining the symmetrical watermark template, it can be embedded into the chroma component to generate the target video frame. The symmetrical watermark template refers to the centrally symmetrical watermark template. The side length of the symmetrical watermark template needs to be greater than or equal to half the height of the original video frame. This ensures that the watermark template has a sufficiently large physical size and coverage area in space. This not only provides a fundamental guarantee against severe center cropping attacks, but also ensures that the embedded features have stronger robustness to conventional processing such as video compression, filtering, and noise interference.

[0031] Step 103: Process the source watermark information using a deep learning watermark model to obtain a watermark feature map; It should be noted that after obtaining the source watermark information, it can be processed using a deep learning watermark model to obtain a watermark feature map. The source watermark information includes device ID, account, time, and other information, and can be a 128-bit binary string. Embedding the source watermark information in the video can quickly locate the leaker and the time of video leakage when the video is leaked.

[0032] It should be noted that, as Figure 2As shown, the deep learning watermarking model includes a watermark processor, an encoder, a combined noise layer, a decoder, and an adversarial discriminator. The decoder includes a watermark extraction branch and an image texture prediction branch. Here, the deep learning watermarking model refers to a trained deep learning model that can reconstruct source watermark information into a watermarked feature map using the watermark processor and encoder within the deep learning watermarking model.

[0033] Step 104: Embed the watermarked feature map into the target video frame to generate a watermarked video.

[0034] It should be noted that the watermarked feature map output by the deep learning watermarking model can be embedded into the central region of the target video frame. (The central region of a video frame is the least likely to be completely removed or severely distorted during various conventional processing methods (such as scaling, cropping, and transcoding). Therefore, embedding the watermarked feature map into the central region can maximize the preservation of watermark information after these conventional operations.) With the synergistic effect of the two watermarking schemes, the generated watermarked video exhibits good watermark concealment and high robustness.

[0035] This invention obtains the chroma components of the original video frame; embeds a symmetrical watermark template into the chroma components to generate a target video frame; processes the source watermark information using a deep learning watermark model to obtain a watermarked feature map; and embeds the watermarked feature map into the target video frame to generate a watermarked video. This solves the technical problem that source information embedded in videos using existing watermarking technologies is easily corrupted, making effective source tracing impossible. Compared to existing technologies, this application, by dual embedding a symmetrical watermark template and a source watermark feature map generated by deep learning, not only effectively resists geometric attacks and ensures stable location and extraction of source information at the decoding end, but also enables the encoded video data to have source tracing capabilities while maintaining high visual quality and low bitrate overhead.

[0036] In any of the above embodiments, before step 102, the method further includes: Retrieve preset synchronization messages; The preset synchronization message is converted into a two-dimensional matrix, and the two-dimensional matrix is ​​spread into a symmetric watermark unit using a spreading matrix; The symmetric watermark unit is scrambled using a scrambling matrix to obtain the target symmetric watermark unit. The target symmetrical watermark unit is flipped to obtain a symmetrical watermark template.

[0037] It should be noted that the preset synchronization message can be set in advance according to the actual situation; the preset synchronization message can be set to 64-bit data; such as Figure 3As shown, after converting the 64-bit preset synchronization message into a two-dimensional matrix, a spreading matrix is ​​used to spread the synchronization message into 16*16 symmetric watermark units w. u To improve watermark security and eliminate internal symmetry within the watermark unit, a scrambled matrix symmetric watermark unit w can also be used. u After scrambling, the target symmetric watermark unit is obtained, such as... Figure 4 As shown, the watermark unit is then repeatedly flipped to finally construct a centrally symmetrical watermark template W.

[0038] The video watermarking method provided in this invention significantly enhances the concealment and security of the watermark by spreading, scrambling, and constructing a centrally symmetrical watermark template from the synchronous message, enabling it to effectively resist malicious analysis and content attacks. At the same time, the symmetrical structure provides a stable synchronization benchmark for subsequent detection, greatly improving robustness against geometric attacks such as rotation and flipping, and ensuring that the watermark can still be accurately and efficiently extracted and identified under complex processing or attacks.

[0039] Based on any of the above embodiments, step 104 includes: Extract the image region to be embedded from the original video frame; The source watermark information is processed by the watermark processor of the deep learning watermark model to obtain a reconstructed watermark feature map. The decoder of the deep learning watermarking model determines the target embedding strategy based on the original video frames. The encoder of the deep learning watermarking model processes the image region to be embedded and the reconstructed watermark feature map according to the target embedding strategy to obtain a watermarked feature map.

[0040] In the specific implementation, the central region of the original video frame (which can be a central 256*256 region) is first extracted as the image region to be embedded. Then, the image region to be embedded and the source watermark information are input into the deep learning watermark model. The source watermark information is processed by the watermark processor to obtain the reshaped watermark feature map (the image region to be embedded and the reshaped watermark feature map are the same size). Then, the image texture prediction branch of the decoder determines the target embedding strategy (such as embedding strength, embedding position and embedding mode) based on the original video frame. Finally, the encoder processes the image region to be embedded and the reshaped watermark feature map according to the embedding strategy to obtain the watermarked feature map.

[0041] In its implementation, the watermark processor can use a fully connected layer and upsampling operations to diffuse the source tracing watermark information to a length of 65536 bits, and then reshape it into a 256*256 source tracing watermark feature. The encoder consists of a Transformer block and convolutional layers. The Transformer block uses a self-attention mechanism to model the long-distance dependencies between different regions of the image, effectively improving the spatial distribution and coordination of watermark information. The convolutional layers are used to extract local texture information, enhancing the model's ability to understand image details.

[0042] The video watermarking method provided in this invention effectively solves the classic problem of balancing watermark embedding strength and visual quality by introducing a deep fusion watermarking model and an adaptive embedding strategy. The Transformer block and convolutional layer in the model work together to deeply weave watermark information into the texture of video frames from two levels: global correlation modeling and local texture fusion. This significantly improves the information capacity of the watermark and its robustness to subsequent processing (such as compression and noise) while ensuring the watermark remains highly invisible. Simultaneously, the embedding strategy based on dynamic prediction of image content allows the watermark embedding position and strength to adapt to the complexity of the video content, further enhancing the watermark's concealment and universality in various video scenarios.

[0043] Based on any of the above embodiments, after step 104, the method further includes: In one feasible implementation, after embedding the watermarked feature map into the target video frame to generate a watermarked video, the method further includes: acquiring the watermarked video to be traced; detecting the symmetrical watermark template state of the video frame to be traced, and restoring the watermarked video to be traced into the target traceable video based on the symmetrical watermark template state; cropping the target watermarked region from the target video frame to be traced in the target traceable video; and extracting the target traceable watermark information from the target watermarked region using the deep learning model, so as to perform video leakage tracing based on the target traceable watermark information.

[0044] It should be noted that after a video is leaked, the relevant watermark information can be extracted from the leaked video (i.e., the watermarked video to be traced). First, it is necessary to analyze whether the video has undergone geometric transformation by detecting the symmetrical watermark template state of the watermarked video to be traced. If it has undergone geometric transformation, the video can be restored to its original spatial position based on the geometric transformation parameters. Then, the source tracing watermark information in the video can be extracted by the decoder in the deep learning watermark model. The device ID, account, time and other information in the source tracing watermark information can be analyzed to locate the leaker and the time of video leakage.

[0045] It's important to note that symmetrical watermark templates can play a geometrical correction role in the source tracing process. Because symmetrical watermark templates have a centrally symmetrical structure and a sufficiently large physical size, even after undergoing geometric attacks such as rotation, scaling, and cropping along with the video frame of the watermarked video to be traced, their overall or most key symmetry patterns will still be preserved in the attacked video frame, exhibiting deformations with a definite mathematical relationship to the original state. By using specialized detection algorithms (such as correlation-based template matching or symmetry axis analysis) to obtain and analyze the actual position, orientation, scale, and integrity of the symmetrical watermark template in the current video frame, the specific geometric transformation parameters (such as rotation angle, scaling factor, and cropping boundary) experienced by the current video frame can be accurately calculated. Based on these parameters, a precise inverse geometric transformation can be performed on the current video frame, thereby correcting its spatial state back to a spatial coordinate system consistent with the watermark embedding, generating a geometrically normalized target source tracing video. This correction process is a crucial prerequisite for the subsequent high-precision extraction of watermark information.

[0046] The video watermarking method provided in this invention can effectively eliminate the interference of geometric distortion on the extraction performance of deep learning decoders, ensure the geometric alignment between the input region received by the decoder and the embedded region during training, thereby ensuring the integrity and accuracy of the finally extracted target source tracing watermark information, and laying a solid foundation for reliable leakage tracing.

[0047] Based on any of the above embodiments, in this method, the step of detecting the symmetrical watermark template state of the watermarked video to be traced, and restoring the watermarked video to be traced into the target traceable video based on the symmetrical watermark template state, includes: Extract the source-traceable chroma component from the source-traceable watermarked video frames; Wiener filtering is applied to the chromaticity components to be traced to obtain filtered chromaticity data; the spatial symmetry of the filtered chromaticity data is calculated using a self-convolution function. Based on the adaptive threshold, a symmetric peak feature map is calculated according to the spatial symmetry; the symmetric watermark template state in the symmetric peak feature map is detected by Hough transform, and the geometric attack parameters are determined according to the symmetric watermark template state. Based on the geometric attack parameters, the watermarked video to be traced is restored to the target video for tracing.

[0048] It should be noted that the chroma component to be traced can be extracted from the watermarked video frames to be traced. Since the watermark template is initially embedded in the U component of the watermarked video frame, this ensures the directness and specificity of the detection. Wiener filtering is then performed to suppress noise and compression artifacts that may be introduced during video transmission, thereby highlighting the structural features of the watermark template. Next, spatial symmetry is calculated using a self-convolution function. Self-convolution operations can effectively enhance the response of regions with periodic or symmetrical characteristics in the image. Because the embedded template is a carefully designed centrally symmetric pattern, although this symmetry may be deformed after a geometric attack, its mathematical essence (the relationship with respect to a certain center point or axis) still exists. The self-convolution operation can quantify and highlight this residual symmetry pattern, thereby generating a feature map representing the original symmetry distribution (i.e., generating a symmetry peak feature map based on spatial symmetry using an adaptive threshold, which can transform continuous symmetry measures into discrete, significant candidate feature points (peaks). These peaks correspond to possible symmetry center or symmetry axis positions, filtering out interference responses caused by background image content and greatly improving the signal-to-noise ratio of the detection). Then, using the classic geometric shape detection method Hough transform, straight lines, circles, or other pre-defined symmetric structures are systematically identified from these symmetry peak feature maps, thereby accurately determining the state of the symmetric watermark template in the watermarked video frame to be traced (such as rotation angle, scaling ratio, cropping boundary), and calculating the precise geometric attack parameters accordingly. Finally, based on these geometric attack parameters, an inverse transformation is performed to restore the spatial coordinate system of the watermarked video frame to be traced to the same state as when the watermark was initially embedded, generating a geometrically normalized target traceability video.

[0049] In practice, the process of extracting the source watermark information is as follows: (1) Decode the watermarked video to be traced into YUV components, extract the U* component (i.e., the chroma component to be traced) from the YUV components, and perform Wiener filtering: in, This represents the local mean of U*. Let U* be the variance.

[0050] (2) Calculate the spatial symmetry of the filtered chromaticity data V obtained after Wiener filtering using the self-convolution function: in, V represents the downsampling function. p This indicates that the original matrix V is zero-padded to twice its original size. FFT and IFFT represent the Fast Fourier Transform and its corresponding inverse transform, respectively.

[0051] (3) Use an adaptive threshold to calculate the symmetric peak feature map M based on spatial symmetry, such as Figure 5 As shown: in, Let M be the local mean. Let M be the variance and β be the empirical coefficient.

[0052] (4) Use Hough transform to detect straight lines in the peak feature map and calculate the distance between different symmetrical watermark units after geometric deformation. By analyzing the angle of the straight lines detected by Hough transform, the geometric attack parameters can be determined: scaling ratio, rotation angle and cropping boundary. Finally, the target source tracing video is restored based on the geometric attack parameters.

[0053] (5) Extract the center 256*256 area of ​​the target source video to obtain the area image containing watermark information.

[0054] (6) Input the image of this area into the deep learning watermarking model, extract the target source watermark information through the watermark extraction branch in the decoder of the deep learning watermarking model, and then analyze the device ID, account, time and other information in the information, and finally locate the leaker and the time of video leakage.

[0055] The video watermarking method provided in this invention, through a symmetry-enhanced detection process and adaptive geometric attack analysis, can accurately locate and restore the symmetrical watermark template after the video has been subjected to attacks such as rotation and scaling, thereby effectively resisting geometric distortion and ensuring high-quality restoration of the source video and reliable extraction of watermark information.

[0056] Figure 6 This is the second flowchart of the video watermarking method provided by the present invention, as shown below. Figure 6 As shown, steps 1031 to 1032 are included before step 103: Step 1031: Obtain watermarked video samples and an initial deep learning watermark model, wherein the watermarked video samples include original video frame samples and source watermark information samples, and the initial deep learning watermark model includes a watermark processor, an encoder, a combined noise layer, and a decoder. It should be noted that the encoder includes a Transformer block and convolutions. The Transformer block utilizes a self-attention mechanism to model long-distance dependencies between different regions of the image, effectively improving the spatial distribution and coordination of watermark information. The convolutional layers are used to extract local texture information, enhancing the model's understanding of image details. A combined noise layer is also placed between the encoder and decoder to simulate various common attack types, such as JPEG compression, Gaussian blur, and Gaussian noise. This combined noise layer attacks the first watermarked feature map to obtain a watermarked noise feature map (i.e., the second watermarked feature map), thus promoting the model to learn more robust features. The decoder includes a watermark extraction branch and an image texture complexity prediction branch. The watermark extraction branch consists of multiple convolutional blocks and downsampling operations, converting the input watermarked noise feature map into a single-channel tensor. Finally, a fully connected layer is used to extract the source watermark information.

[0057] Step 1032: Based on the watermarked video samples, train the initial deep learning watermark model to obtain a deep learning watermark model.

[0058] It's important to note that a large number of watermarked video samples containing original video frames and their corresponding source watermark information can be used to drive the initial model to learn and master two core capabilities: first, how to adaptively and covertly embed watermark information into video frames based on the visual characteristics of the video content (especially texture complexity); and second, how to ensure that the embedded information can still be extracted with high accuracy and reliability after various simulated attacks (implemented by a combined noise layer). Specifically, the training process involves a loop of forward and backward propagation. In each iteration, the model receives watermarked video samples and performs forward computation through its internal watermark processor, encoder (guided by the image texture prediction branch), combined noise layer, and decoder (containing the watermark extraction branch), ultimately generating the predicted source information. The watermark extraction loss between the predicted and original information, the image quality loss between the watermarked frame and the original frame, and the adversarial loss generated by the adversarial discriminator are calculated and fused into a total loss value. Subsequently, through the backpropagation algorithm, this total loss value is used to simultaneously update all learnable parameters in the encoder, decoder, and discriminator.

[0059] This application obtains watermarked video samples and an initial deep learning watermarking model, wherein the watermarked video samples include original video frame samples and source watermark information samples, and the initial deep learning watermarking model includes a watermark processor, an encoder, a combined noise layer, and a decoder; based on the watermarked video samples, the initial deep learning watermarking model is trained to obtain a deep learning watermarking model.

[0060] Based on any of the above embodiments, the decoder includes a watermark extraction branch, and step 1032 includes: The watermarked video sample is input into the initial deep learning watermarking model, and the source tracing watermark information sample is processed by the watermark processor to obtain a reconstructed feature map; The encoder processes the reconstructed feature map to obtain a first watermarked feature map; the combined noise layer attacks the first watermarked feature map to obtain a second watermarked feature map. The second watermarked feature map is processed by the watermark extraction branch of the decoder to obtain the target reconstructed feature map; based on the original video frame samples, the reconstructed feature map, the first watermarked feature map, and the target reconstructed feature map, the overall loss value is determined, and the initial deep learning watermarking model is optimized according to the overall loss value to obtain the trained deep learning watermarking model.

[0061] It should be noted that the encoder can fully integrate the reconstructed feature map (i.e., the watermark features obtained after the watermark processor processes the source watermark information) with the image region to be embedded extracted from the original video frame samples to obtain the first watermarked feature map. To minimize the difference between the reconstructed feature map and the original video frame samples, the image loss function L is set. E (That is, determining the image loss based on the reconstructed feature map and original video frame samples): In the formula, θ E Here are the learnable parameters of the encoder, where E represents the encoder and I... co represents the first watermarked feature map, and wm represents the reconstructed feature map.

[0062] It should be noted that a combined noise layer is placed between the encoder and decoder to simulate various common attack types, such as JPEG compression, Gaussian blur, and Gaussian noise. The watermarked noise feature map is obtained by attacking the first watermarked feature map through the noise layer. (i.e., the second watermarked feature map) promotes the model to learn more robust features.

[0063] It should be noted that the decoder includes a watermark extraction branch, which consists of multiple convolutional blocks and downsampling operations. This branch converts the second watermark-containing noise feature map into a single-channel tensor, and finally uses a fully connected layer to extract the source watermark information (i.e., the target reconstructed feature map). To minimize the difference between the extracted target reconstructed feature map and the original watermark information (i.e., the reconstructed feature map), a watermark loss function L is set. D (That is, the watermark loss is determined based on the reconstructed feature map and the target reconstructed feature map): Where, θ S Here, represents the learnable parameters of the encoder, and D represents the watermark extraction branch of the encoder. This represents the second watermark feature image.

[0064] It should be noted that the overall loss value can be determined based on the image loss and the watermark loss, and then the initial deep learning watermark image can be optimized based on the overall loss value.

[0065] Based on any of the above embodiments, in this method, the decoder further includes an image texture prediction branch; wherein, processing the reconstructed feature map through the encoder to obtain the first watermarked feature map includes: The image texture prediction branch analyzes the image regions in the original video frame samples to generate an image texture prediction feature map. The encoder determines an adaptive watermark embedding strategy based on the image texture prediction feature map. The encoder processes the reshaped feature map according to the adaptive embedding strategy to generate a first watermarked feature map.

[0066] It should be noted that the image texture prediction branch consists of convolutional blocks and linear layers, and is mainly used to extract texture information from the original video frame samples, thereby outputting the image texture prediction feature map T. p A higher median value in the image texture prediction feature map indicates richer texture in that region, while a lower value indicates a smoother region. The image texture prediction feature map output by the image texture prediction branch is mainly used in the encoder to achieve end-to-end learning of image texture features. This allows the encoder to adaptively adjust the watermark embedding strategy (such as embedding strength, embedding position, and embedding mode) for the source information. The embedding strength is increased in texture-rich regions and decreased in smooth regions to avoid introducing significant distortion and reduce the additional bitrate overhead caused by watermark embedding. Specifically, the encoder can use the image texture prediction feature map T... p The spatial distribution dynamically determines the watermark embedding strength, preferred embedding position, and modulation mode in different regions. For example, the embedding strength is moderately increased in regions with rich texture and dense details, taking advantage of the relative insensitivity of the human visual system to changes in such regions to more stably integrate the watermark information into the background. In contrast, the embedding strength is significantly reduced or even avoided in regions with smooth and uniform textures, thereby effectively preventing visual defects such as block effects and contour distortion that may be caused by the introduction of watermarks, and minimizing the additional bitrate overhead caused by embedding operations. Ultimately, this significantly improves the imperceptibility of watermarks and their robustness to various processing attacks while ensuring visual quality and coding efficiency.

[0067] The video watermarking method provided in this invention can dynamically embed the watermark into texture regions that are less perceptible to the human eye by using an image texture complexity prediction branch based on the characteristics of video content. This reduces redundant bits and further increases the invisibility of the watermark.

[0068] In any of the above embodiments, the initial deep learning watermarking model further includes an adversarial discriminator; wherein determining the overall loss value based on the original video frame samples, the reconstructed feature map, the first watermarked feature map, and the target reconstructed feature map includes: The carrier image is determined based on the first watermark feature; The adversarial discriminator calculates the adversarial loss value based on the original video frame samples and the carrier image. The image loss value is determined based on the reconstructed feature map and the first watermarked feature map, and the watermark loss value is determined based on the reconstructed feature map and the target reconstructed feature map. The overall loss value is determined based on the image loss value, the watermark loss value, and the adversarial loss value.

[0069] It should be noted that the encoder can fully fuse the reconstructed feature map (i.e., the watermark features obtained after the watermark processor processes the source watermark information) with the image region to be embedded extracted from the original video frame samples to obtain the first watermarked feature map I. co To minimize the difference between the reconstructed feature map and the original video frame samples, the image loss function L is set. E (That is, determining the image loss based on the reconstructed feature map and original video frame samples): In the formula, θ E Here are the learnable parameters of the encoder, where E represents the encoder and I... co represents the first watermarked feature map, and wm represents the reconstructed feature map.

[0070] It should be noted that a combined noise layer is placed between the encoder and decoder to simulate various common attack types, such as JPEG compression, Gaussian blur, and Gaussian noise. The watermarked noise feature map is obtained by attacking the first watermarked feature map through the noise layer. (i.e., the second watermarked feature map) promotes the model to learn more robust features.

[0071] It should be noted that the decoder includes a watermark extraction branch, which consists of multiple convolutional blocks and downsampling operations. This branch converts the second watermark-containing noise feature map into a single-channel tensor, and finally uses a fully connected layer to extract the source watermark information (i.e., the target reconstructed feature map). To minimize the difference between the extracted target reconstructed feature map and the original watermark information (i.e., the reconstructed feature map), a watermark loss function L is set. D (That is, the watermark loss is determined based on the reconstructed feature map and the target reconstructed feature map): Where, θ D Here, represents the learnable parameters of the encoder, and D represents the watermark extraction branch of the encoder. This represents the second watermark feature image.

[0072] It should be noted that the adversarial discriminator consists of multiple convolutional blocks and pooling layers. It can minimize the visual difference between the original image (i.e., the original video frame sample) and the watermarked image (i.e., the carrier image) and maximize the fusion between the watermark and the carrier image by updating the learnable parameters in the deep learning watermarking model. Specifically, the adversarial loss function L corresponding to the adversarial discriminator... Ad L f for: Among them, L Ad L represents the first instance of combat loss. f Indicates the second instance of combat loss; It should be noted that the overall loss value can be calculated based on the first adversarial loss, the second adversarial loss, the image loss, and the watermarking loss. The specific calculation method is as follows: The video watermarking method provided in this invention introduces an adversarial discriminator and constructs a multi-objective loss function that integrates image quality, watermark integrity, and adversarial concealment for joint training. This enables the deep learning watermarking model to generate watermarked videos with higher visual quality and stronger concealment. While effectively ensuring the reliable embedding and extraction of watermark information, it significantly improves the watermark's resistance to visual perception and malicious detection.

[0073] The video watermarking processing apparatus provided by the present invention is described below. The video watermarking processing apparatus described below can be referred to in correspondence with the video watermarking processing method described above, such as... Figure 7 As shown, the video watermarking processing device includes: The acquisition module 10 is used to acquire the chroma components of the original video frames; The generation module 20 is used to embed the symmetrical watermark template into the chroma component to generate the target video frame; The generation module is also used to process the source watermark information through a deep learning watermark model to obtain a watermark feature map. The generation module is further configured to embed the watermarked feature map into the target video frame to generate a watermarked video.

[0074] Optionally, the generation module is further configured to: Retrieve preset synchronization messages; The preset synchronization message is converted into a two-dimensional matrix, and the two-dimensional matrix is ​​spread into a symmetric watermark unit using a spreading matrix; The symmetric watermark unit is scrambled using a scrambling matrix to obtain the target symmetric watermark unit. The target symmetrical watermark unit is flipped to obtain a symmetrical watermark template.

[0075] Optionally, the generation module is further configured to: Extract the image region to be embedded from the original video frame; The source watermark information is processed by the watermark processor of the deep learning watermark model to obtain a reconstructed watermark feature map. The decoder of the deep learning watermarking model determines the target embedding strategy based on the original video frames. The encoder of the deep learning watermarking model processes the image region to be embedded and the reconstructed watermark feature map according to the target embedding strategy to obtain a watermarked feature map.

[0076] Optionally, the generation module is further configured to: The process involves acquiring watermarked video samples and an initial deep learning watermark model. The watermarked video samples include original video frame samples and source watermark information samples. The initial deep learning watermark model includes a watermark processor, an encoder, a combined noise layer, and a decoder. Based on the watermarked video samples, the initial deep learning watermark model is trained to obtain a deep learning watermark model.

[0077] Optionally, the generation module is further configured to: The watermarked video sample is input into the initial deep learning watermarking model, and the source tracing watermark information sample is processed by the watermark processor to obtain a reconstructed feature map; The encoder processes the reconstructed feature map to obtain a first watermarked feature map; The first watermarked feature map is attacked by the combined noise layer to obtain the second watermarked feature map. The second watermarked feature map is processed by the watermark extraction branch of the decoder to obtain the target reconstructed feature map; Based on the original video frame samples, the reconstructed feature map, the first watermarked feature map, and the target reconstructed feature map, the overall loss value is determined, and the initial deep learning watermarking model is optimized according to the overall loss value to obtain a trained deep learning watermarking model.

[0078] Optionally, the generation module is further configured to: The image texture prediction branch analyzes the image regions in the original video frame samples to generate an image texture prediction feature map. The encoder determines an adaptive watermark embedding strategy based on the image texture prediction feature map. The encoder processes the reshaped feature map according to the adaptive embedding strategy to generate a first watermarked feature map.

[0079] Optionally, the generation module is further configured to: The carrier image is determined based on the first watermark feature; The adversarial discriminator calculates the adversarial loss value based on the original video frame samples and the carrier image. The image loss value is determined based on the reconstructed feature map and the first watermarked feature map, and the watermark loss value is determined based on the reconstructed feature map and the target reconstructed feature map. The overall loss value is determined based on the image loss value, the watermark loss value, and the adversarial loss value.

[0080] Optionally, the generation module is further configured to: Obtain the watermarked video to be traced; The symmetrical watermark template state of the video frame to be traced is detected, and based on the symmetrical watermark template state, the watermarked video to be traced is restored to the target traceable video. Extract the target watermarked region from the target video frame to be traced in the target source-tracing video; The deep learning model extracts the target source watermark information from the target watermark area, and uses the target source watermark information to trace the source of video leakage.

[0081] Optionally, the generation module is further configured to: Extract the source-traceable chroma component from the source-traceable watermarked video frames; Wiener filtering is performed on the chromaticity components to be traced to obtain filtered chromaticity data; The spatial symmetry of the filtered chromaticity data is calculated using a self-convolution function; Based on the adaptive threshold, a symmetric peak feature map is calculated according to the spatial symmetry. The state of the symmetric watermark template in the symmetric peak feature map is detected by Hough transform, and the geometric attack parameters are determined based on the state of the symmetric watermark template. Based on the geometric attack parameters, the watermarked video to be traced is restored to the target video for tracing.

[0082] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a video watermarking processing method, which includes: obtaining the chroma components of the original video frame; embedding a symmetrical watermark template into the chroma components to generate a target video frame; processing the source watermark information using a deep learning watermarking model to obtain a watermarked feature map; and embedding the watermarked feature map into the target video frame to generate a watermarked video.

[0083] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0084] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the video watermarking processing method provided by the above methods. The method includes: obtaining the chroma components of the original video frame; embedding a symmetrical watermark template into the chroma components to generate a target video frame; processing the source watermark information through a deep learning watermark model to obtain a watermarked feature map; and embedding the watermarked feature map into the target video frame to generate a watermarked video.

[0085] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the video watermarking processing method provided by the above methods. The method includes: obtaining the chroma components of an original video frame; embedding a symmetrical watermark template into the chroma components to generate a target video frame; processing the source watermark information using a deep learning watermark model to obtain a watermarked feature map; and embedding the watermarked feature map into the target video frame to generate a watermarked video.

[0086] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video watermarking method, characterized in that, include: Obtain the chroma components of the original video frames; Embed the symmetrical watermark template into the chroma components to generate the target video frame; The source watermark information is processed by a deep learning watermark model to obtain a watermark feature map; The watermarked feature map is embedded into the target video frame to generate a watermarked video.

2. The video watermarking method according to claim 1, characterized in that, Before embedding the symmetrical watermark template into the chroma component to generate the target video frame, the method further includes: Retrieve preset synchronization messages; The preset synchronization message is converted into a two-dimensional matrix, and the two-dimensional matrix is ​​spread into a symmetric watermark unit using a spreading matrix; The symmetric watermark unit is scrambled using a scrambling matrix to obtain the target symmetric watermark unit. The target symmetrical watermark unit is flipped to obtain a symmetrical watermark template.

3. The video watermarking method according to claim 1, characterized in that, The process of processing the source-tracing watermark information using a deep learning watermarking model to obtain a watermarked feature map includes: Extract the image region to be embedded from the original video frame; The source watermark information is processed by the watermark processor of the deep learning watermark model to obtain a reconstructed watermark feature map. The decoder of the deep learning watermarking model determines the target embedding strategy based on the original video frames. The encoder of the deep learning watermarking model processes the image region to be embedded and the reconstructed watermark feature map according to the target embedding strategy to obtain a watermarked feature map.

4. The video watermarking method according to claim 1, characterized in that, Before processing the source-tracing watermark information using a deep learning watermark model to obtain the watermarked feature map, the process also includes: The process involves obtaining watermarked video samples and an initial deep learning watermark model. The watermarked video samples include original video frame samples and source watermark information samples. The initial deep learning watermark model includes a watermark processor, an encoder, a combined noise layer, and a decoder. Based on the watermarked video samples, the initial deep learning watermark model is trained to obtain a deep learning watermark model.

5. The video watermarking method according to claim 4, characterized in that, The decoder includes a watermark extraction branch; the process of training the initial deep learning watermark model based on the watermarked video samples to obtain a deep learning watermark model includes: The watermarked video sample is input into the initial deep learning watermarking model, and the source tracing watermark information sample is processed by the watermark processor to obtain a reconstructed feature map; The encoder processes the reconstructed feature map to obtain a first watermarked feature map; The first watermarked feature map is attacked by the combined noise layer to obtain the second watermarked feature map. The second watermarked feature map is processed by the watermark extraction branch of the decoder to obtain the target reconstructed feature map; Based on the original video frame samples, the reconstructed feature map, the first watermarked feature map, and the target reconstructed feature map, the overall loss value is determined, and the initial deep learning watermarking model is optimized according to the overall loss value to obtain a trained deep learning watermarking model.

6. The video watermarking method according to claim 5, characterized in that, The decoder further includes an image texture prediction branch; wherein, the step of processing the reconstructed feature map through the encoder to obtain the first watermarked feature map includes: The image texture prediction branch analyzes the image regions in the original video frame samples to generate an image texture prediction feature map. The encoder determines an adaptive watermark embedding strategy based on the image texture prediction feature map. The encoder processes the reshaped feature map according to the adaptive embedding strategy to generate a first watermarked feature map.

7. The video watermarking method according to claim 5, characterized in that, The initial deep learning watermarking model further includes an adversarial discriminator; wherein, determining the overall loss value based on the original video frame samples, the reconstructed feature map, the first watermarked feature map, and the target reconstructed feature map includes: The carrier image is determined based on the first watermark feature; The adversarial discriminator calculates the adversarial loss value based on the original video frame samples and the carrier image. The image loss value is determined based on the reconstructed feature map and the first watermarked feature map, and the watermark loss value is determined based on the reconstructed feature map and the target reconstructed feature map. The overall loss value is determined based on the image loss value, the watermark loss value, and the adversarial loss value.

8. The video watermarking method according to claim 1, characterized in that, After embedding the watermarked feature map into the target video frame to generate the watermarked video, the process further includes: Obtain the watermarked video to be traced; The symmetrical watermark template state of the video frame to be traced is detected, and based on the symmetrical watermark template state, the watermarked video to be traced is restored to the target traceable video. Extract the target watermarked region from the target video frame to be traced in the target source-tracing video; The deep learning model extracts the target source watermark information from the target watermark area, and uses the target source watermark information to trace the source of video leakage.

9. The video watermarking method according to claim 8, characterized in that, The step of detecting the symmetrical watermark template state of the watermarked video to be traced, and restoring the watermarked video to be traced into the target traceable video based on the symmetrical watermark template state, includes: Extract the source-traceable chroma component from the source-traceable watermarked video frames; Wiener filtering is performed on the chromaticity components to be traced to obtain filtered chromaticity data; The spatial symmetry of the filtered chromaticity data is calculated using a self-convolution function; Based on the adaptive threshold, a symmetric peak feature map is calculated according to the spatial symmetry. The state of the symmetric watermark template in the symmetric peak feature map is detected by Hough transform, and the geometric attack parameters are determined based on the state of the symmetric watermark template. Based on the geometric attack parameters, the watermarked video to be traced is restored to the target video for tracing.

10. A video watermarking processing device, characterized in that, include: The acquisition module is used to acquire the chroma components of the original video frames; A generation module is used to embed a symmetrical watermark template into the chroma component to generate a target video frame; The generation module is also used to process the source watermark information through a deep learning watermark model to obtain a watermark feature map. The generation module is further configured to embed the watermarked feature map into the target video frame to generate a watermarked video.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the video watermarking processing method as described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the video watermarking method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the video watermarking method as described in any one of claims 1 to 9.

Images