Anti-photograph monitoring video watermarking method combining deep robust watermarking and template synchronization

Abstract

The present application proposes an anti-photographing monitoring video watermarking method combining deep robust watermarking and template synchronization, which ingeniously uses the feature that part of the background content in the monitoring video is basically unchanged, and selects part of the background image as the watermark carrier; the deep robust watermarking network used can embed a robust watermark that is invisible to the naked eye and resistant to photographing in the watermark carrier image; meanwhile, by increasing the watermark embedding cost of the simple texture area, the network is guided to embed the watermark information in the complex texture area as much as possible, which improves the shortcoming that the StegaStamp embedding trace is more obvious in the smooth area of the image, and significantly improves the visual quality of the watermark.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multimedia content security, in particular, to a shooting traceability technology for monitoring video; more particularly, to an anti-shooting monitoring video watermarking method combining deep robust watermarking and template synchronization. BACKGROUND

[0002] Most of the existing monitoring traceability is in the form of visible watermark displayed on the monitoring screen, such as time information displayed in the upper left corner of the screen, by directly covering the plaintext of the watermark information with pure white or other color fonts directly onto the monitoring video content. When the camera and other devices are used to shoot the display content on the monitoring screen, the watermark information will also be shot down, and thus the monitoring traceability can be performed according to the watermark information in the shooting video. Although this watermark is easy to generate, it undoubtedly affects the visual effect of the entire monitoring video; on the other hand, since the watermark information is visible to the naked eye, the watermark information can be easily destroyed by the stalker using post-processing and other means, resulting in difficulty in video traceability.

[0003] The Chinese invention patent with publication date of 2016.12.07: A robust video watermarking method against geometric attacks based on SIFT provides a video robust watermarking scheme; although the existing robust video watermarking method can resist some slight geometric attacks to some extent, in the process of shooting and shooting, the shooting angle and the video content size are arbitrary, and severe angle changes are easy to occur. In this scenario, the above method is prone to synchronization position error and synchronization omission when synchronizing the watermark, so these existing technologies do not effectively solve the problem of watermark synchronization, and still have certain limitations.

[0004] In the technical field of image watermarking, there is an anti-printing and shooting blind watermarking method based on deep learning: StegaStamp, but this method does not have adaptive selection of the embedding position of the watermark, resulting in obvious traces when the model embeds information in the smooth area of the image, seriously affecting the visual effect of the watermark image; in addition, this method can only process image watermarking and cannot be applied to video watermarking. SUMMARY

[0005] In view of the limitations of the prior art, the present application provides an anti-shooting monitoring video watermarking method combining deep robust watermarking and template synchronization, and the technical scheme adopted by the present application is:

[0006] An anti-shooting monitoring video watermarking method combining deep robust watermarking and template synchronization, which embeds watermark information in the monitoring video through the following steps:

[0007] S11, selecting a rectangular region with relatively fixed background image content from a background region in the monitoring video as a template region for watermark embedding;

[0008] S12, extracting a video frame of the monitoring video in a stream buffer, and obtaining a cropped image as a carrier image from the video frame according to the template region;

[0009] S13, obtaining device number information and a current timestamp of the monitoring video, and encoding them into a binary bit sequence; splicing the binary bit sequence with a corresponding CRC check code and a BCH error correction code to generate watermark information;

[0010] S14, inputting the carrier image and the watermark information into a preset deep robust watermark network, and generating a watermark image through an encoder in the deep robust watermark network; wherein the deep robust watermark network is a deep learning framework trained based on StegaStamp, combined with an image block perception similarity loss function and a YUV space difference loss function based on an image texture template;

[0011] S15, replacing the image in the template region of the video frame with the watermark image.

[0012] Compared with the prior art, the application ingeniously utilizes the feature that part of the background content in the monitoring video is basically unchanged, selects part of the background image as a watermark carrier, and uses a deep robust watermark network to embed a robust watermark that is invisible to the naked eye and resistant to shooting in the watermark carrier image. At the same time, by increasing the watermark embedding cost of the simple texture region, the network is guided to embed the watermark information in the complex texture region as much as possible, which improves the shortcoming that the StegaStamp embedding trace is obvious in the smooth region of the image, and significantly improves the visual quality of the watermark.

[0013] As a preferred scheme, the image block perception similarity loss function is represented by the following formula:

[0014] L P =LPIPS(I o ,I w );

[0015] Wherein, I o represents the carrier image; I w represents the watermark image.

[0016] As a preferred scheme, the YUV space difference loss function based on the image texture template is represented by the following formula:

[0017]

[0018] Wherein, Y, U, V represent the carrier image Io and a watermark image I w The component converted to YUV space, I(i,j,c) represents the pixel value size of the image in the c channel at the horizontal coordinate i and the vertical coordinate j position.

[0019] As a preferred solution, the overall loss function of the deep learning framework is represented by the following formula:

[0020] L = λ P L P + λ T L T + λ C L C + λ M L M ;

[0021] Wherein, λ represents the loss weight; L P represents the image block perceptual similarity loss function; L T represents the YUV space difference loss function based on image texture template; L C represents the discriminator loss function; L M represents the cross-entropy loss function.

[0022] Further, the discriminator loss function L C is represented by the following formula:

[0023] L C = D(I o )-D(I w );

[0024] Wherein, D(·) represents the discriminator network; I o represents the carrier image; I w represents the watermark image;

[0025] The cross-entropy loss function L M is represented by the following formula:

[0026]

[0027] Wherein, B i (i = 1, 2,..., 64) represents the bit sequence input into the network during the training process; M i (i = 1, 2,..., 64) represents the bit sequence output by the network during the training process.

[0028] As a preferred solution, the deep learning framework introduces distortion transformations including perspective transformation and / or blur and / or noise and / or color change and / or illumination and / or JPEG compression caused distortion during training at the distortion layer.

[0029] As a preferred solution, for a rephotographed video obtained by rephotographing a monitored video in which the watermark information is embedded, the watermark information is extracted from the rephotographed video by the following steps:

[0030] S21, by extracting video frames of the rephotographed video, calculating sift feature points of the rephotographed video;

[0031] S22, obtaining sift feature points of the carrier image, and matching the sift feature points of the carrier image with the sift feature points of the rephotographed video;

[0032] S23, solving a homography matrix H from the matching result of step S22, and performing perspective transformation on the video frames of the rephotographed video according to the homography matrix H;

[0033] S24, cropping the perspective transformation result of step S23 to the same size as the carrier image;

[0034] S25, inputting the cropped result of step S24 into the deep robust watermark network, and obtaining watermark information through a decoder in the deep robust watermark network.

[0035] The application also provides the following content:

[0036] A shooting-resistant monitored video watermark system combining deep robust watermark and template synchronization, comprising a watermark information embedding module for embedding watermark information in a monitored video, the watermark information embedding module comprising a template region selection unit, a carrier image extraction unit, a watermark information generation unit, a watermark image generation unit, and a video image replacement unit, wherein:

[0037] The template region selection unit is configured to select a rectangular region with relatively fixed background image content from a background region in the monitored video as a template region for watermark embedding;

[0038] The carrier image extraction unit is configured to extract a video frame of the monitored video in a stream buffer, and obtain a cropped image from the video frame as a carrier image according to the template region;

[0039] The watermark information generation unit is configured to obtain device number information and a current timestamp of the monitored video, encode them into a binary bit sequence, splice the binary bit sequence with a corresponding CRC check code and a BCH error correction code to generate watermark information;

[0040] The watermark image generation unit is configured to input the carrier image and watermark information into a preset deep robust watermark network, and generate a watermark image through an encoder in the deep robust watermark network; wherein the deep robust watermark network is a deep learning framework trained based on StegaStamp, combined with an image block perception similarity loss function and a YUV space difference loss function based on an image texture template.

[0041] The video image replacement unit is configured to replace the image of the video frame in the template region with the watermark image.

[0042] A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the anti-photographing monitoring video watermarking method combining deep robust watermarking and template synchronization.

[0043] A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the anti-photographing monitoring video watermarking method combining deep robust watermarking and template synchronization. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 A flowchart of a process of embedding watermark information in a monitoring video according to the anti-photographing monitoring video watermarking method combining deep robust watermarking and template synchronization of embodiment 1 of the present application;

[0045] Figure 2 An example of watermark information in embodiment 1 of the present application;

[0046] Figure 3 An operation example of the watermarking method of embodiment 1 of the present application;

[0047] Figure 4 A schematic diagram of a deep robust watermark network structure provided in embodiment 1 of the present application;

[0048] Figure 5 A schematic diagram of an encoder structure of the deep robust watermark network provided in embodiment 1 of the present application;

[0049] Figure 6 A schematic diagram of a decoder structure of the deep robust watermark network provided in embodiment 1 of the present application;

[0050] Figure 7 A flowchart of a process of extracting watermark information from the rephotographed video according to the anti-photographing monitoring video watermarking method combining deep robust watermarking and template synchronization of embodiment 1 of the present application;

[0051] Figure 8The anti-shooting monitoring video watermarking system schematic diagram provided by the embodiment 2 of the present application combines deep robust watermarking and template synchronization. DETAILED DESCRIPTION

[0052] The accompanying drawings are only used for illustrative description and cannot be understood as a limitation to the present application;

[0053] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0054] The terms used in the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein means and includes any or all possible combinations of one or more associated listed items.

[0055] The following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present application, as detailed in the appended claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not necessarily describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0056] In addition, in the description of the present application, "multiple" means two or more, unless otherwise specified. The association between the associated objects is described, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents a "or" relationship between the associated objects. The present application is further described below in conjunction with the drawings and embodiments.

[0057] In order to solve the limitations of the prior art, the present embodiment provides a technical solution, which is further described below in conjunction with the drawings and embodiments.

[0058] Embodiment 1

[0059] Please refer to Figure 1The application discloses a shooting-resistant monitoring video watermarking method combining deep robust watermarking and template synchronization, and embeds watermark information in a monitoring video through the following steps.

[0060] S11, a rectangular region with relatively fixed background image content is selected from a background region in the monitoring video as a template region for watermark embedding.

[0061] S12, a video frame in a streaming buffer of the monitoring video is extracted, and a cropped image is obtained from the video frame as a carrier image according to the template region.

[0062] S13, device number information and a current timestamp of the monitoring video are obtained and encoded into a binary bit sequence; the binary bit sequence is spliced with a corresponding CRC check code and a BCH error correction code to generate watermark information.

[0063] S14, the carrier image and the watermark information are input into a preset deep robust watermarking network, and a watermark image is generated by an encoder in the deep robust watermarking network; wherein the deep robust watermarking network is a deep learning framework trained based on StegaStamp, combined with an image block perception similarity loss function and a YUV space difference loss function based on an image texture template.

[0064] S15, the image in the template region of the video frame is replaced by the watermark image.

[0065] Compared with the prior art, the application ingeniously utilizes the feature that part of the background content in the monitoring video is basically unchanged, selects part of the background image as a watermark carrier, and uses a deep robust watermarking network which can embed a robust watermark invisible to the naked eye and resistant to shooting into the watermark carrier image; meanwhile, by increasing the watermark embedding cost of a simple texture region, the network is guided to embed the watermark information in a complex texture region as much as possible, thereby improving the shortcoming that the StegaStamp embedding trace is obvious in a smooth region of an image, and significantly improving the visual quality of the watermark.

[0066] Specifically, in the field, a video watermarking method refers to a processing method of video watermarking, which can include a process of embedding watermark information into a video, or a process of extracting watermark information from a video.

[0067] The scheme provided in the embodiment can generate a shooting-resistant, transcoding-resistant and invisible-to-the-naked-eye watermark image when embedding watermark information into a video.

[0068] It can be considered that the scheme provided by the embodiment mainly lies in: first, since the watermark is embedded in the video, it will be displayed on the monitoring screen, and the process of camera retake will make the watermark suffer different degrees of distortion, and the anti-photograph watermark based on the deep learning framework is adopted, and the robustness of the static image watermark is effectively migrated to the video. Secondly, considering that in the monitoring video scene, part of the background image content is almost unchanged, the part of the background image is used as a watermark synchronization template to fix the position of the watermark embedding. Finally, in order to improve the visual quality of the watermark, the embodiment is based on StegaStamp, extracts the texture complex region of the image as a template, reduces the embedding cost of the texture complex region while increases the embedding cost of the smooth region, and guides the network to embed the watermark information in the texture complex region as much as possible to improve the visual quality of the watermark video.

[0069] More specifically, as a preferred embodiment, please refer to Figure 2 The watermark information generated in step S13 can be in the form of a 64-bit binary watermark information sequence. The first 32 bits are used to store effective information, including timestamp information and device number information, wherein the year, month, day, hour, and minute each occupy 6, 4, 5, 5, and 6 bits, respectively, and the device number occupies 6 bits. The middle 8 bits store the CRC check code obtained according to the effective information. The last 24 bits are supervision information, which store a 24-bit BCH error correction code obtained according to the effective information. The BCH error correction code can correct up to 3 error bits.

[0070] In order to facilitate the deep robust watermark network to process the carrier image, as a preferred embodiment, the carrier image can be scaled to a size suitable for network processing before being input into the deep robust watermark network. Correspondingly, if the size of the watermark image is different from the original size of the carrier image, it can also be scaled to the same size as the original size of the carrier image before step S15.

[0071] For specific examples, please refer to Figure 3In the example, a rectangular region with almost constant content is first selected from the background image of a high-definition monitoring video as a watermark carrier, for example, the fixed coordinates (0, 0) of the video frame are used as the upper left corner of the rectangular region, and a 3-channel RGB image with a size of 300*300 is intercepted; the carrier image is scaled to a size of 512*512 suitable for the network, and then input into the encoder of the deep robust watermark network together with a 64-bit binary watermark information sequence. After the convolutional neural network calculation, a watermark image containing watermark information is output. The watermark image is scaled to the size of 300*300 of the original rectangular region, and then the reduced watermark image is used to replace the template region image in the original video frame, so that a video frame containing watermark information is obtained. The same embedding operation is performed on each frame in the monitoring video buffer, and finally the watermark video frame is displayed on the monitoring screen through the buffer, so that a continuous and watermark-containing monitoring video stream is obtained.

[0072] Further, the deep robust watermark network mainly includes an encoder, a decoder and a discriminator, and the network structure can be referred to Figure 4 ; wherein the input of the encoder is a 512*512 3-channel RGB carrier image and a 64-bit binary information sequence, and the output is a residual image with the same size as the input. The residual image is the difference between the watermark image and the carrier image, which represents the amplitude that needs to be modified on the corresponding pixels of the carrier image to embed the watermark information. The residual image and the carrier image are added to obtain the watermark image. After obtaining the watermark image, in order to make the watermark image be displayed on the monitoring screen and then be accurately decoded after being photographed, a distortion layer simulating the process is designed. The watermark image is input into the decoder after being affected by the simulation distortion function. The output of the decoder is a 64-bit binary information sequence. In another branch, the carrier image and the watermark image are input into the discriminator as positive samples and negative samples respectively, and the discrimination loss is used to guide the encoder to generate a more authentic watermark image.

[0073] The specific network structure of the encoder can be referred to Figure 5, which adopts a U-Net-like structure, takes a 512x512 RGB image and a 64-bit binary information sequence as input, and outputs a 512x512 residual image. First, in order to change the one-dimensional 64-bit sequence into two-dimensional, the 64-bit binary sequence is expanded into a vector with a length of 12288 through a fully connected layer, and then reshaped into a 64x64 three-channel image through vector folding, and up-sampled to the same size as the input image using nearest neighbor interpolation, and spliced with the input image as the input of the main body of the encoder network. The main structure of the network includes two parts of down-sampling and up-sampling. The network first continuously down-samples the feature map using 3x3 convolution, which reduces the size while continuously increasing the number of channels of the feature map, to extract the abstract features of the image. Then a series of nearest neighbor interpolation up-sampling operations are performed to expand the feature map size, and after each up-sampling, the feature map of the same size in the down-sampling process is spliced, and then a convolution operation with a step size of 1 is performed on the spliced feature map to output a feature map with the same size. In this process, the feature map size is continuously increased and the number of channels is continuously reduced, until a residual image with the same size as the original image is output. Finally, the residual image and the original image are added to obtain a watermark image containing the watermark information.

[0074] The specific network structure of the decoder is shown in Figure 6 , which takes a 512x512 three-channel RGB watermark image containing watermark information as input, and continuously down-samples it using a convolution kernel with a step size of 2, while continuously increasing the number of channels of the feature map to extract the features of the image. Finally, a fully connected layer is used to output a 64-bit binary information sequence. The structure of the discriminator is similar to that of the decoder, and a convolution kernel with a step size of 2 is also used for down-sampling, while the number of channels of the feature map is continuously increased to extract the features of the image. However, the final output is a vector with a length of 2, representing the probability of predicting true and false, respectively.

[0075] In particular, the image will cause different degrees of distortion during display on the monitor screen and then being photographed, and the purpose of the network is to train the decoder to decode the correct information from the image with a certain degree of distortion. Therefore, a function needs to be used to simulate this distortion during training, and therefore six derivable distortion transformations, including perspective transformation, blur, noise, color change, illumination, and JPEG compression, are introduced in the distortion layer.

[0076] Therefore, as a preferred embodiment, the deep learning framework introduces distortion transformations caused by perspective transformation and / or blur and / or noise and / or color change and / or illumination and / or JPEG compression in the distortion layer during training.

[0077] The main goal of the entire network architecture is to make the embedded watermark imperceptible to the naked eye, and in addition, to ensure that the decoding accuracy is high and robust to image distortion.

[0078] In order to make the watermark imperceptible to the naked eye, two loss functions are used to limit the difference between the watermark image and the carrier image in the embodiment: the first one is the learned perceptual image patch similarity (LPIPS) loss as a loss function, which is an image similarity evaluation function based on network learning. The more similar two images are in vision, the smaller the loss is.

[0079] As a preferred embodiment, the learned perceptual image patch similarity loss function is represented by the following formula:

[0080] L P = LPIPS(I o , I w );

[0081] Wherein, I o represents the carrier image; I w represents the watermark image.

[0082] The second one is the YUV space difference loss function based on the image texture template. If the watermark information is embedded in the smooth area of the image, it will make the area have obvious watermark traces, so it is necessary to give greater embedding cost to the smooth area of the image. For this purpose, the network first extracts the edge image by using the Canny operator when processing the image, and then performs morphological dilation operation on the edge image, so that the texture template I t with high brightness in complex areas and almost zero brightness in smooth areas is obtained, and it is normalized to 0 to 1. The carrier image I o and the watermark image I w are converted from the RGB space to the YUV space, and the calculation formula of the loss L T is the sum of pixel loss; as a preferred embodiment, the YUV space difference loss function based on the image texture template is represented by the following formula:

[0083]

[0084] Wherein, Y, U, V represent the components of the carrier image I o and the watermark image I w converted to the YUV space, and I(i,j,c) represents the pixel value size of the image in the c channel at the horizontal coordinate i and the vertical coordinate j position.

[0085] In order to make the decoding accuracy high, the error decoding result is punished, so cross entropy can also be used as a loss function, the higher the error rate, the greater the loss, and the purpose of training is to minimize the loss function. In addition, there is a discriminator loss, which can specifically adopt the loss function form of Wasserstein GAN

[0086] Therefore, as a preferred embodiment, the overall loss function of the deep learning framework is represented by the following formula:

[0087] L = λ P L P + λ T L T + λ C L C + λ M L M ;

[0088] Wherein, λ represents the loss weight; L P represents the image block perception similarity loss function; L T represents the YUV space difference loss function based on image texture template; L C represents the discriminator loss function; L M represents the cross entropy loss function.

[0089] Further, the discriminator loss function L C is represented by the following formula:

[0090] L C = D(I o )-D(I w );

[0091] Wherein, D(·) represents the discriminator network; I o represents the carrier image; I w represents the watermark image;

[0092] The cross entropy loss function L M is represented by the following formula:

[0093]

[0094] Wherein, B i (i = 1, 2,..., 64) represents the bit sequence input into the network during training; M i (i = 1, 2,..., 64) represents the bit sequence output by the network during training.

[0095] In particular, the MIRFLICKR natural image dataset containing 25000 natural images is used to train the network, and the model converges after 200000 iterations of training on a single graphics card. With a 2080ti graphics card, the encoding speed of a single image is 10 milliseconds, which meets the real-time encoding requirement.

[0096] As a preferred embodiment, please refer to Figure 7 For the reprinted video obtained by reprinting the monitoring video embedded with the watermark information, the watermark information is extracted from the reprinted video through the following steps:

[0097] S21, the sift feature points of the reprinted video are calculated by extracting the video frames of the reprinted video;

[0098] S22, the sift feature points of the carrier image are obtained, and the sift feature points of the reprinted video are matched;

[0099] S23, the homography matrix H is solved according to the matching result of step S22, and the video frames of the reprinted video are perspective transformed according to the homography matrix H;

[0100] S24, the perspective transformation result of step S23 is cropped to the same size as the carrier image;

[0101] S25, the cropped result of step S24 is input into the deep robust watermark network, and the watermark information is obtained through the decoder in the deep robust watermark network.

[0102] Specifically, when the monitoring video is reprinted, the image content of the watermark template is almost unchanged, and the sift feature matching is used to perform image registration between the reprinted frame and the template (i.e. the carrier image) used when embedding the watermark, which can realize fast and accurate watermark synchronization.

[0103] Specific examples can also be referred to Figure 3 After the watermark information is extracted from the reprinted video, the BCH supervision bits are used to correct the extracted watermark information, and the validity of the watermark information is verified according to the CRC check code; if valid, the timestamp and device number information can be successfully extracted, thereby realizing the tracing of the monitoring video.

[0104] Embodiment 2

[0105] A kind of anti-photograph monitoring video watermark system combined with deep robust watermark and template synchronization, please refer to Figure 8, comprising a watermark information embedding module 1 for embedding watermark information in a monitoring video, the watermark information embedding module 1 comprising a template region selection unit 11, a carrier image extraction unit 12, a watermark information generation unit 13, a watermark image generation unit 14, and a video image replacement unit 15, wherein:

[0106] The template region selection unit 11 is configured to select a rectangular region with relatively fixed background image content from a background region in the monitoring video as a template region for watermark embedding.

[0107] The carrier image extraction unit 12 is configured to extract a video frame of the monitoring video in a stream buffer, and obtain a cropped image as a carrier image from the video frame according to the template region.

[0108] The watermark information generation unit 13 is configured to obtain device number information and a current timestamp of the monitoring video, and encode them into a binary bit sequence; and concatenate the binary bit sequence with a corresponding CRC check code and a BCH error correction code to generate watermark information.

[0109] The watermark image generation unit 14 is configured to input the carrier image and the watermark information into a preset deep robust watermark network, and generate a watermark image through an encoder in the deep robust watermark network; wherein the deep robust watermark network is a deep learning framework trained based on StegaStamp, combined with an image block perception similarity loss function and a YUV space difference loss function based on an image texture template.

[0110] The video image replacement unit 15 is configured to replace the image in the template region of the video frame with the watermark image.

[0111] As a preferred embodiment, it further comprises a watermark information extraction module 2 for extracting watermark information from a rephotographed video; the rephotographed video is obtained by rephotographing the monitoring video embedded with the watermark information; the watermark information extraction module 2 comprises a feature point calculation unit 21, a feature point matching unit 22, a perspective transformation unit 23, a cropping unit 24, and a decoding unit 25; wherein:

[0112] The feature point calculation unit 21 is configured to calculate sift feature points of the rephotographed video by extracting video frames of the rephotographed video.

[0113] The feature point matching unit 22 is configured to obtain sift feature points of the carrier image, and match them with sift feature points of the rephotographed video.

[0114] The perspective transformation unit 23 is configured to solve a homography matrix H according to the matching result of the landmark matching unit 22, and perform perspective transformation on the video frames of the retaken video according to the homography matrix H.

[0115] The cropping unit 24 is configured to crop the perspective transformation result of the perspective transformation unit 23 to the same size as the carrier image.

[0116] The decoding unit 25 is configured to input the cropping result of the cropping unit 24 into the deep robust watermarking network, and obtain watermark information through a decoder in the deep robust watermarking network.

[0117] Embodiment 3

[0118] A storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the anti-photographing surveillance video watermarking method combined with deep robust watermarking and template synchronization in Embodiment 1.

[0119] Embodiment 4

[0120] A computer device, comprising a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the anti-photographing surveillance video watermarking method combined with deep robust watermarking and template synchronization in Embodiment 1.

[0121] Obviously, the above embodiments of the present application are merely exemplary and are not intended to limit the embodiments of the present application. Based on the above description, other different forms of changes or modifications can be made by those skilled in the art. It is not necessary or possible to enumerate all the embodiments. Any modification, equivalent replacement, and improvement within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.

Claims

1. A method for anti-surveillance video watermarking that combines deep robust watermarking and template synchronization, characterized in that, Embed watermark information in surveillance video using the following steps: S11. Select a rectangular area with relatively fixed background image content from the background area in the surveillance video as the template area for watermark embedding. S12, extract the video frames of the monitoring video in the stream buffer, and obtain the cropped image from the video frame as the carrier image according to the template area; S13, obtain the device number information and current timestamp of the monitoring video, and encode them into a binary bit sequence; concatenate the binary bit sequence with the corresponding CRC check code and BCH error correction code to generate watermark information; S14, the carrier image and watermark information are input into a preset deep robust watermarking network, and the watermark image is generated by the encoder in the deep robust watermarking network; wherein, the deep robust watermarking network is trained by a deep learning framework based on StegaStamp, which combines an image patch-aware similarity loss function and a YUV spatial difference loss function based on image texture template. The deep robust watermarking network mainly consists of three parts: an encoder, a decoder, and a discriminator. The encoder takes the carrier image and watermark information as input and outputs a residual image of the same size as the input. The residual image is added to the carrier image to obtain the watermark image. The watermark image is then input to the decoder to output new watermark information. In another branch, the carrier image and the watermark image are used as positive and negative samples, respectively, and input to the discriminator. The discriminant loss guides the encoder to generate a more realistic watermark image. S15, replace the image of the video frame in the template area with the more realistic watermark image.

2. The anti-shooting surveillance video watermarking method combining deep robust watermarking and template synchronization according to claim 1, characterized in that, The image patch perceptual similarity loss function is expressed by the following formula: ； in, Represents a carrier image; This represents a watermarked image.

3. The anti-shooting surveillance video watermarking method combining deep robust watermarking and template synchronization according to claim 1, characterized in that, The YUV spatial difference loss function based on image texture template is expressed by the following formula: ； Where Y, U, and V represent carrier images. and watermarked images Components converted to YUV space, This represents the pixel value in the image at position i (horizontal coordinate) and j (vertical coordinate) in the c channel.

4. The anti-shooting surveillance video watermarking method combining deep robust watermarking and template synchronization according to claim 1, characterized in that, The overall loss function of the deep learning framework is expressed by the following formula: ； in, Indicates the loss weight; This represents the image patch perceptual similarity loss function; This represents the YUV spatial difference loss function based on the image texture template; This represents the discriminator loss function; This represents the cross-entropy loss function.

5. The anti-shooting surveillance video watermarking method combining deep robust watermarking and template synchronization according to claim 4, characterized in that, The discriminator loss function Expressed as follows: ； in, This represents the discriminator network; Represents a carrier image; Represents a watermarked image; The cross-entropy loss function Expressed as follows: ； in, This represents the bit sequence input to the network during training; This represents the bit sequence output by the network during training.

6. The anti-shooting surveillance video watermarking method combining deep robust watermarking and template synchronization according to claim 1, characterized in that, The deep learning framework introduces distortion transformations into the distortion layer during training, including perspective transformation and / or blurring and / or noise and / or color changes and / or lighting and / or JPEG compression.

7. The anti-shooting surveillance video watermarking method combining deep robust watermarking and template synchronization according to claim 1, characterized in that, For a re-enacted video obtained by re-enacting a surveillance video embedded with the watermark information, the watermark information is extracted from the re-enacted video through the following steps: S21, by extracting the video frames of the re-enacted video, calculate the SIFT feature points of the re-enacted video; S22, obtain the SIFT feature points of the carrier image and match them with the SIFT feature points of the re-enacted video; S23, Solve for the homography matrix H using the matching result of step S22, and perform perspective transformation on the video frames of the re-enacted video based on the homography matrix H; S24, crop the perspective transformation result of step S23 to the same size as the carrier image; S25, input the cropping result of step S24 into the deep robust watermarking network, and obtain the watermark information through the decoder in the deep robust watermarking network.

8. A watermarking system for anti-shooting surveillance videos that combines deep robust watermarking and template synchronization, characterized in that, The method for implementing the method as described in any one of claims 1 to 7 includes a watermark information embedding module (1) for embedding watermark information in a surveillance video. The watermark information embedding module (1) includes a template region selection unit (11), a carrier image cropping unit (12), a watermark information generation unit (13), a watermark image generation unit (14), and a video image replacement unit (15), wherein: The template area selection unit (11) is used to select a rectangular area with relatively fixed background image content from the background area in the monitoring video as the template area for watermark embedding. The carrier image cropping unit (12) is used to extract video frames of the monitoring video in the stream buffer, and to obtain cropped images from the video frames as carrier images according to the template region; The watermark information generation unit (13) is used to obtain the device number information and current timestamp of the monitoring video, and encode them into a binary bit sequence; the binary bit sequence is concatenated with the corresponding CRC check code and BCH error correction code to generate watermark information; The watermark image generation unit (14) is used to input the carrier image and watermark information into a preset deep robust watermarking network and generate a watermark image through the encoder in the deep robust watermarking network; wherein, the deep robust watermarking network is trained by a deep learning framework based on StegaStamp, which combines an image patch perception similarity loss function and a YUV spatial difference loss function based on image texture template. The video image replacement unit (15) is used to replace the image of the video frame in the template area with the watermark image.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the anti-shooting surveillance video watermarking method that combines deep robust watermarking and template synchronization as described in any one of claims 1 to 7.

10. A computer device, characterized in that: The device includes a storage medium, a processor, and a computer program stored in the storage medium and executable by the processor, wherein the computer program, when executed by the processor, implements the steps of the anti-shooting surveillance video watermarking method combining depth robust watermarking and template synchronization as described in any one of claims 1 to 7.

Images