Watermark embedding method, watermark extracting method, electronic equipment and readable storage medium

Abstract

The invention discloses a watermark embedding method, a watermark extracting method, electronic equipment and a readable storage medium, and belongs to the technical field of multimedia information security. The method comprises the following steps: acquiring an input target video, and determining a target video frame image to be embedded with a watermark according to video parameters of the target video and / or image parameters of each video frame image in the target video; extracting features of the target video frame image, and determining a watermark embedding area of the target video frame image; obtaining a first singular value matrix of the watermark embedding area; segmenting a preset watermark image according to the number of the target video frame images and the first singular value matrix to obtain a plurality of local watermark images; and correspondingly embedding the plurality of local watermark images into the watermark embedding area of the target video frame image to obtain a watermark video frame image. The watermark is embedded in the way, so that the watermark embedding capacity can be improved, the difficulty of extracting a complete watermark image from a video frame image by an attacker is increased, and the safety of the watermark image is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of multimedia information security technology, and in particular to a watermark embedding method, an extraction method, an electronic device, and a readable storage medium. Background Art

[0002] With the increasing power of network multimedia tools and the popularization of digital video capture terminal devices, the storage, dissemination and communication of multimedia information such as images and videos have become more convenient. At the same time, the authenticity and copyright issues of digital video content have attracted widespread attention.

[0003] Related technologies typically embed watermarks into multimedia information, such as images and videos, to verify the authenticity of multimedia information. Most watermarks are embedded using visible watermarks, which overlay the watermark onto the video screen. This approach not only affects the user's viewing experience but is also easily removed using multimedia tools, failing to effectively protect the watermark information carried within the video screen. Invisible watermarks, on the other hand, typically embed the watermark image into shallow features of the video / image (such as the video name, video creation time, video resolution, or video QR code). This approach has a smaller watermark embedding capacity and is easily unstable, impacting the accuracy of video authentication. Summary of the Invention

[0004] The embodiments of the present application provide a watermark embedding method, an extraction method, an electronic device, and a readable storage medium, which are used to at least solve the problem that the related art embeds the watermark image into the shallow features of the video / image and the watermark embedding capacity is small.

[0005] In the first aspect, an embodiment of the present application provides a watermark embedding method, comprising: obtaining an input target video, and determining a target video frame image to be embedded with a watermark based on video parameters of the target video and / or image parameters of each video frame image in the target video; extracting features of the target video frame image, and determining a watermark embedding area of the target video frame image; obtaining a first singular value matrix of the watermark embedding area; segmenting a preset watermark image based on the number of target video frame images and the first singular value matrix to obtain multiple local watermark maps; and embedding the multiple local watermark maps into the watermark embedding area of the target video frame image to obtain a watermarked video frame image.

[0006] In the second aspect, an embodiment of the present application provides a watermark extraction method, including: obtaining a watermark video frame image; extracting features of the watermark video frame image and determining a watermark embedding area of the watermark video frame image; obtaining a second singular value matrix of the watermark embedding area; extracting a local watermark map corresponding to the watermark video frame image based on the second singular value matrix and a pre-stored first singular value matrix; and extracting a target watermark image based on the local watermark map corresponding to the watermark video frame image.

[0007] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first or second aspect above are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first or second aspect above are implemented.

[0009] In an embodiment of the present application, an input target video is obtained, and a target video frame image to be watermarked is determined based on the video parameters of the target video and / or the image parameters of each video frame image in the target video; features of the target video frame image are extracted to determine the watermark embedding region of the target video frame image; a first singular value matrix of the watermark embedding region is obtained; a preset watermark image is segmented based on the number of target video frame images and the first singular value matrix to obtain multiple local watermark images; and the multiple local watermark images are correspondingly embedded into the watermark embedding region of the target video frame image to obtain a watermarked video frame image. In this way, the watermark embedding capacity can be increased, the difficulty for an attacker to extract the complete watermark image from the video frame image can be increased, and the security of the watermark image can be improved.

[0010] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0012] Figure 1 A schematic diagram of the process of a watermark embedding method provided in an embodiment of the present application is shown;

[0013] Figure 2 The following is a flow chart of a method for generating a watermark image according to an embodiment of the present application;

[0014] Figure 3A schematic diagram of a process of extracting watermarks provided in an embodiment of the present application is shown;

[0015] Figure 4 Another schematic diagram of the process of extracting watermarks provided in an embodiment of the present application is shown;

[0016] Figure 5 The following is a schematic diagram showing the structure of a video anti-counterfeiting identification system provided by an embodiment of the present application;

[0017] Figure 6 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0018] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0019] In the field of multimedia information security, embedding watermarks in multimedia information, such as images and videos, can facilitate authenticity verification and copyright protection. Related technologies typically embed watermark images into shallow / low-level features of the video / image (such as the video title, video creation time, video resolution, and video QR code). This approach results in a small watermark embedding capacity and is easily compromised by geometric attacks such as scaling, translation, rotation, and shearing, resulting in low-accuracy video authentication.

[0020] In response to the problems existing in the above-mentioned multimedia information anti-counterfeiting identification process, an embodiment of the present application provides a watermark embedding method, which obtains a target video frame image of a target video to determine the watermark embedding area of the target video frame image; divides the preset watermark image according to the number of target video frame images and the first singular value matrix of the watermark embedding area to obtain multiple local watermark maps; and embeds the multiple local watermark maps into the watermark embedding area accordingly, which can improve the watermark embedding capacity, increase the difficulty for attackers to extract the complete watermark image from the video frame image, enhance the security of the watermark image, and thus improve the accuracy of video authenticity verification.

[0021] See also Figure 1 , Figure 1The flowchart of the watermark embedding method provided by the embodiment of the present application is shown. The execution subject of the method can be a terminal device or a server, wherein the terminal device can be a device such as a personal computer, or a mobile terminal device such as a mobile phone or a tablet computer, and the terminal device can be a terminal device used by a user. The server can be an independent server or a server cluster composed of multiple servers. Moreover, the server can be a background server of a certain business, or a background server of a certain website or application (such as online cloud video anti-counterfeiting identification software or watermark embedding / extraction application, etc.). In the embodiment of the present application, the execution subject is taken as an example to illustrate. For the case of the terminal device, it can be processed according to the following relevant content, which will not be repeated here. As shown in the figure, the watermark embedding method 100 may include the following operations.

[0022] Operation S101: obtaining an input target video, and determining a target video frame image to be embedded with a watermark according to video parameters of the target video and / or image parameters of each video frame image in the target video.

[0023] The target video can be a real-time captured video or a pre-stored video. The target video can include multiple consecutive video frames. The video parameters of the target video can be any parameters representing video information, such as video duration or video size. The image parameters can be any parameters representing image information, such as image grayscale, image brightness, image contrast, image saturation, image color, etc.

[0024] In an exemplary embodiment, when a user enters a valid account and password on the user interface to log in to the online cloud video anti-counterfeiting identification software, its backend server reads the input target video and determines the target video frame image to be embedded with the watermark based on the video parameters of the target video and / or the image parameters of each video frame image in the target video. Taking the determination of the target video frame image based on the image grayscale as an example, the average grayscale value of a single video frame image F in the target video is obtained as g(k), and the grayscale average value of k consecutive video frame images is g avg (k), where

[0025] Select a single video frame image whose average grayscale value g(k) is greater than the grayscale mean value g of k consecutive video frames avg (k) n target video frame images that need to be embedded with watermarks The formula is as follows:

[0026]

[0027] Among them, k represents the number of continuous video frame images included in the target video, and n∈[1,k) represents the target video frame image that needs to be embedded with the watermark. The number of .

[0028] In one possible implementation, if n ≥ log2 k, the number of target video frames that need to be watermarked is excessive. While this improves the security of the watermark embedding method and increases verification accuracy, it also increases computational complexity. In this case, the selection of target video frames can be based on a comprehensive consideration of one or more of the following features: for example, video length, video size, brightness, contrast, color, and other characteristic information of the video frames, to select an appropriate number of target video frames for watermarking.

[0029] S102: Extract features of the target video frame image and determine a watermark embedding area of the target video frame image.

[0030] In an exemplary embodiment, the server determines n target video frame images to be embedded with watermarks. For each target video frame image Extract the target video frame image Features, determine the target video frame image Watermark embedding area Among them, the extracted features can be high-level features used to characterize the important feature information areas that users pay attention to. Since the pixel positions of the important feature information areas that users pay attention to are relatively stable, determining the watermark embedding area based on this feature can reduce distortion and suppress image noise, and then embedding the watermark image into the watermark embedding area of the target video frame image, which can enhance the stability of the watermark image.

[0031] S103: Obtain a first singular value matrix of the watermark embedding area.

[0032] In an exemplary embodiment, the watermark embedding area may be Perform singular value decomposition (SVD) to obtain the left singular vector matrix U, the right singular vector matrix V and the first singular value matrix S, which are as follows:

[0033]

[0034] S104: Segment the preset watermark image according to the number of the target video frame images and the first singular value matrix to obtain a plurality of local watermark images.

[0035] In the embodiment of the present application, the watermark image W is obtained. fusedThe watermark image can be a pre-stored watermark, or a fusion image of watermarks extracted from multiple consecutive video frame images, or any one of multiple video frame images. The number of the first singular value matrix S for the watermark image W fused Segmentation is performed to obtain multiple local watermark images. Here, the watermark image W fused Segment into target video frame images The number of local watermark maps is the same as that of the first singular value matrix Match them so that each local watermark image can be embedded into the corresponding watermark embedding area.

[0036] S105: Embed the multiple local watermark images into the watermark embedding areas of the target video frame image to obtain a watermarked video frame image.

[0037] In the embodiment of the present application, the multiple local watermark images obtained in the above step S104 are correspondingly embedded into the target video frame image. Watermark embedding area The watermarked video frame image is obtained from the image, and then the video embedded with the watermarked image is output. Since the complete watermarked image is divided into multiple local watermark images and embedded in the target video frame images in a dispersed manner, it is more difficult for the attacker to obtain the complete watermarked image. At the same time, the larger watermarked image is divided into multiple smaller local watermark images and embedded in different target video frame images respectively, which can achieve the purpose of embedding a larger watermarked image, improve the watermark embedding capacity, and ensure the picture quality of the target video.

[0038] In a possible implementation, in the above step S101, after determining the target video frame image to be watermarked, the following steps may be further included:

[0039] The number of marked frames of the target video is determined according to the position information of the target video frame image in the target video; and the number of marked frames is saved in a target file in a preset file format.

[0040] In an exemplary embodiment, after determining the target video frame image in the target video, the server determines the marked frame number of the target video frame according to the position information of each target video frame image in the target video; determines a preset file format, which includes a text format (such as CSV, JSON, etc.), a binary format (such as Protobuf, Msgpack, etc.) and an XML format, etc.; and then converts the marked frame number data into a preset file format and writes it into the target file as a basis for selecting the target video frame image during watermark extraction.

[0041] In a possible implementation, in step S102, extracting features of the target video frame image and determining the watermark embedding area of the target video frame image may include:

[0042] Step 1021: extracting high-level features of the target video frame image using a preset feature extraction network;

[0043] Step 1023: Determine a first weight matrix based on the high-level features, and use the first weight matrix to perform weighted processing on the target video frame image to obtain a watermark embedding area of the target video frame image.

[0044] In an exemplary embodiment, the target video frame image The size of each is M×N, and it is input to the spatial selection unit of the attention mechanism module for processing to obtain the features of the target video frame image. After the average pooling operation, the first eigenvector is obtained The target video frame image After the maximum pooling operation, the second eigenvector is obtained The first eigenvector and the second eigenvector Perform fusion to obtain the first weight matrix Specifically, the two feature vectors can be fused in the following way:

[0045]

[0046] Among them, f represents the convolution operation and sigmoid() is the activation function.

[0047] The first weight matrix and target video frame image Perform matrix multiplication operation to obtain the target video frame image Watermark embedding area The formula is as follows:

[0048]

[0049] in, Represents vector-matrix multiplication.

[0050] In a possible implementation, in step S104, the preset watermark image is segmented according to the number of the target video frame images and the first singular value matrix to obtain a plurality of local watermark images, including:

[0051] Step 1041: determining the number of divisions of the watermark image according to the number of target video frame images, wherein the number of divisions is less than or equal to the number of target video frame images;

[0052] Step 1042: Segment the watermark image according to the number of segments to obtain multiple candidate local watermark images;

[0053] Step 1043: pre-process the multiple candidate local watermark images according to the dimension of the first singular value matrix to obtain multiple local watermark images.

[0054] In an exemplary embodiment, in order to improve the watermark embedding capacity, the preset watermark image W fused The watermark images are divided into multiple local watermark images, and the multiple local watermark images are correspondingly embedded into the watermark embedding area of the target video frame image. The number of watermark images W is determined by fused In order to ensure the integrity of the embedded watermark image, the watermark image W fused The number of segmentations can be equal to the target video frame image The number can also be smaller than the target video frame image The number of watermark images W fused According to the preset rules, it is divided into multiple candidate local watermark images According to the dimension of the singular value matrix, multiple candidate local watermark images are Perform preprocessing to make the candidate local watermark image The dimension of the first singular value matrix S is the same as that of the first singular value matrix S, so that the preprocessed candidate local watermark image Determined as a local watermark image

[0055] The method of preprocessing the plurality of candidate local watermark images according to the dimension of the singular value matrix to obtain a plurality of local watermark images includes:

[0056] If the dimension of a target candidate local watermark map among the multiple candidate local watermark maps is smaller than the dimension of the first singular value matrix, pixel value filling is performed on the target candidate local watermark map to obtain a local watermark map; or

[0057] If the dimension of the target candidate local watermark map among the multiple candidate local watermark maps is greater than the dimension of the first singular value matrix, pixel values of the target candidate local watermark map are selected to obtain a local watermark map.

[0058] In an exemplary embodiment, the candidate local watermark image The preprocessing rules are as follows:

[0059] If the target candidate local watermark image The dimension of the candidate local watermark image is smaller than the dimension of the singular value matrix S. Fill the pixel value to get the local watermark image Specifically, the target candidate local watermark image can be filled in a reverse loop data manner. Perform preprocessing and convert the candidate local watermark image Convert it into a one-dimensional sequence, expand it from the last pixel value, and loop it based on the one-dimensional sequence until the size of the one-dimensional sequence is 1×(M / 8×N / 8), and then convert the one-dimensional sequence into a local watermark image of M / 8×N / 8.

[0060] If the target candidate local watermark image The dimension of the candidate local watermark image is greater than the dimension of the singular value matrix S. Select pixel values to obtain local watermark image Specifically, the local watermark image will be selected Convert it into a one-dimensional sequence, remove the same number of pixel values at the front and back positions, so that the size of the remaining sequence is 1×(M / 8×N / 8), and then convert the one-dimensional sequence into a local watermark image of M / 8×N / 8

[0061] In a possible implementation, the step S105 of embedding the multiple local watermark images into the watermark embedding areas of the target video frame image to obtain the watermarked video frame image includes:

[0062] Step 1051: Superimpose each weighted local watermark image with the first singular value matrix of the corresponding watermark embedding area to obtain a second singular value matrix after embedding the local watermark image;

[0063] Step 1052: Determine watermark image features according to the second singular value matrix;

[0064] Step 1053: Update the image features of the watermark embedding area of the target video frame image to the watermark image features to obtain the watermark video frame image.

[0065] In an exemplary embodiment, the local watermark image is embedded into Embedded in the watermark embedding area In the first singular value matrix S of After weighting, it is superimposed with the first singular value matrix S of the corresponding watermark embedding area to obtain the second singular value matrix S' after embedding the local watermark image. The formula is as follows:

[0066]

[0067] Among them, α adapRepresents the embedding strength factor of the local watermark.

[0068] right Perform SVD decomposition to obtain two orthogonal matrices U', V' and a second singular value matrix S' containing the watermark image. According to the watermark embedding area in step S103 above, The left singular vector matrix U and the right singular vector matrix V are obtained when performing singular value decomposition, and the second singular value matrix S' is subjected to SVD inverse transformation to obtain the watermark image features, and the target video frame image is converted to The image features of the watermark embedding area are updated to the watermark image features to obtain the watermark video frame image

[0069] Since the watermark embedding area is trained selectively through the attention mechanism, it can effectively suppress image noise, and the singular value matrix of the watermark embedding area has good stability, the local watermark image is Embedding into the singular value matrix of the watermark embedding area can improve the robustness of the watermark image.

[0070] In a possible implementation, the method further includes: generating the preset watermark image; the preset watermark image can be obtained specifically by:

[0071] Acquire a video frame image in the target video;

[0072] Extracting fully connected layer features of the video frame image;

[0073] Extracting high-frequency sub-band features of the video frame image;

[0074] The fully connected layer features and the high-frequency sub-band coefficient features are subjected to feature fusion processing to obtain the preset watermark image.

[0075] In an exemplary embodiment, Figure 2 As shown, the watermark image can be generated by the following steps:

[0076] Step 201: Decompose the target video into multiple video frame images;

[0077] Step 202: Input the video frame image F, size M×N, into a pre-trained network model (such as AlexNet network) to extract the fully connected layer feature F fc6 、F fc7 After dimensionality reduction using the principal component analysis (PCA) method, the size is M / 4×N / 4.

[0078] Step 203: Perform frequency domain transformation on the video frame image F. Specifically, the video frame image F undergoes a 2-level lifting wavelet transform (LWT) to obtain 7 coefficient matrices, and extracts the high-frequency subband coefficient features LH2 of the second layer, each of which has a size of M / 4×N / 4.

[0079] Step 204: F fc6 and F fc7 The high-level features of the layer are fused with the high-frequency sub-band coefficient features LH2 to obtain the preset watermark image W fused , the size is M / 4×N / 4. The fusion method can be shown as follows:

[0080] W fused =[w6*F fc6 ,w7*F fc7 ,w lwt *LH2];

[0081] Among them, w6, w7, w lwt Represent the fusion weights of the three features respectively. The specific fusion method is as follows: fc6 、F fc7 , LH2 are binarized respectively, and then the two fully connected layer two-dimensional features are XORed Select the position coordinates of the high frequency sub-band LH2 in turn (2 t-1 ,2 t-1 ) Get the corresponding pixel value, and set the pixel values of other positions to 1. Related to the size of the high frequency subband, and then to F fc Perform bitwise AND operation to obtain fusion features, and encrypt the fusion features to obtain the watermark image W fused , the size is M / 4×N / 4.

[0082] Step 205: embed the watermark image into the target video according to a preset embedding rule to obtain a video with the watermark image embedded therein.

[0083] The generation of the watermark image combines the high-dimensional fully connected layer features of the video frame image and the high-frequency subband features in the frequency domain. The high-dimensional abstract features are more robust under geometric attacks such as scaling, translation, rotation, and shearing. At the same time, the high-frequency subband features in the frequency domain of the video frame image can reflect edge detail information and maintain the integrity of the original video frame image information itself, thereby improving the security of the watermark image and its robustness against geometric attacks.

[0084] See also Figure 3 , Figure 3A schematic flow chart of a watermark extraction method provided by an embodiment of the present application is shown. This watermark extraction method employs semi-blind extraction when extracting a watermark image. The extraction position and embedding strength of the watermark image are synchronized with the embedding process. As shown in the figure, the watermark extraction method 300 may include the following steps:

[0085] S301: Obtain a watermarked video frame image.

[0086] In an exemplary embodiment, when a user enters a valid account and password on the user interface to log in to the online cloud video anti-counterfeiting identification software, the video to be verified is read in, and the watermarked video frame image in the video to be verified is obtained. In order to verify the watermark video frame image The robustness of the watermark image can be used to determine the watermarked video frame image Perform geometric attacks such as scaling, rotation, shearing, or other attacks or no attacks to obtain the watermarked video frame image after the attack The size is M×N.

[0087] S302: Extract features of the watermarked video frame image and determine a watermark extraction area of the watermarked video frame image.

[0088] Among them, the spatial attention feature is used to represent the feature representation of the spatial position that the user pays attention to in the target video frame image.

[0089] In an exemplary embodiment, the watermarked video frame image is extracted The spatial attention features of the watermarked video frame image are determined Watermark extraction area

[0090] S303: Obtain a second singular value matrix of the watermark extraction area.

[0091] In the embodiment of the present application, the watermark extraction area can be Perform SVD decomposition to obtain the second singular value matrix S′ containing the watermark image Att .

[0092] S304: Extracting a local watermark image corresponding to the watermarked video frame image according to the second singular value matrix and the pre-stored first singular value matrix.

[0093] In an exemplary embodiment, the second singular value matrix S′ Att Perform SVD inverse transformation on the orthogonal matrices U', V' obtained in step S105 of the watermark embedding process to obtain according to And the pre-stored first singular value matrix S, extract the watermark video frame image Corresponding local watermark image Its size is M / 8×N / 8. Specifically, the local watermark image Extraction can be done in the following ways:

[0094]

[0095] S305: Extracting a target watermark image according to the local watermark image corresponding to the watermarked video frame image.

[0096] In an exemplary embodiment, according to the watermark video frame image Corresponding local watermark image Generate target watermark image. For example, you can use multiple watermark video frame images Corresponding local watermark image The target watermark image is obtained by splicing according to the preset rules.

[0097] In a possible implementation, in step S301, obtaining a watermarked video frame image includes:

[0098] Step 3011: Obtain the video to be verified;

[0099] Step 3012: Determine the position information of the watermarked video frame image in the target video according to the preset number of marked frames in the target file;

[0100] Step 3013: Acquire the watermarked video frame image according to the position information.

[0101] In an exemplary embodiment, the marked frame number can be obtained from the target file stored corresponding to the original target video. The marked frame number refers to the frame number of the target video containing the watermarked video frame image. For example, if the watermarked video frame image appears in the 2nd frame, the 4th frame, and the 7th frame, the marked frame number can be [2, 4, 7]. According to the marked frame number, the position information of the watermarked video frame image in the video to be verified is determined. For example, if the 2nd, 4th, and 7th frames in the target video are watermarked video frame images embedded with the watermark, the video frame images of the 2nd, 4th, and 7th frames in the video to be verified can be obtained and used as the watermarked video frame images to be verified.

[0102] In a possible implementation, in step S302, extracting features of the watermarked video frame image and determining a watermark extraction area of the watermarked video frame image includes:

[0103] Step 3021: extracting high-level features of the watermarked video frame image using a preset feature extraction network;

[0104] Step 3022: Determine a second weight matrix based on the high-level features, and use the second weight matrix to perform weighted processing on the watermarked video frame image to obtain a watermark embedded area of the watermarked video frame image.

[0105] In an exemplary embodiment, Figure 4 As shown, the method for extracting a watermark image may include the following steps:

[0106] Step 401: decomposing the attacked video with embedded watermark image into watermark video frame images;

[0107] Step 402: Convert the video frame containing the watermark into Input to the spatial selection unit of the attention mechanism module, the video frame image is The feature vector obtained by convolution of the average pooling layer is The video frame image is The feature vector obtained by convolution of the maximum pooling layer is Then the two obtained eigenvectors are fused into the second weight matrix Finally, the second weight matrix Video frame image with watermark Perform matrix multiplication operation to obtain the watermark extraction area The size of the watermark extraction area after processing is M / 8×N / 8;

[0108] Step 403: performing SVD decomposition on the watermark extraction area and extracting a local watermark image according to a preset extraction criterion;

[0109] Step 404: splicing the local watermark images extracted from the watermarked video frame images to obtain a watermark image.

[0110] In a possible implementation, in step S305, extracting a target watermark image according to the local watermark image corresponding to the watermarked video frame image includes:

[0111] Step 3051: When the dimension of the local watermark image is smaller than the dimension of the pre-stored candidate local watermark image, pixel value filling is performed on the local watermark image to obtain a pre-processed local watermark image; or when the dimension of the local watermark image is larger than the dimension of the candidate local watermark image, pixel value selection is performed on the local watermark image to obtain the pre-processed local watermark image.

[0112] Step 3052: splicing the pre-processed local watermark images corresponding to the watermarked video frame images to obtain the target watermark image.

[0113] In an exemplary embodiment, according to the pre-stored candidate local watermark image The dimension of the local watermark image Preprocessing, if the local watermark image The dimension is smaller than the pre-stored candidate local watermark image The dimension of the local watermark image Fill the pixel value to obtain the preprocessed local watermark image If the local watermark image The dimension of the candidate local watermark image is larger than The dimension of the local watermark image Select pixel values to obtain pre-processed local watermark image

[0114] n watermarked video frame images The corresponding n pre-processed local watermark images Splice and get the target watermark image W″ ext , its size is M / 4×N / 4, and the specific formula is as follows:

[0115]

[0116] In a possible implementation, after step S305, the following steps are further included:

[0117] S306: Determine a normalized correlation coefficient between the target watermark image and a preset watermark image; and determine an authenticity verification result of the video to be verified based on the normalized correlation coefficient.

[0118] In an exemplary embodiment, the normalized correlation coefficient (NC) between the target watermark image and the preset watermark image obtained in step S305 can be used to measure the similarity between the target watermark image and the preset watermark image. The calculation formula of the NC value is as follows:

[0119]

[0120] Among them, W fused is the preset watermark image, W″ ext The target watermark image extracted from the watermarked video frame image. The NC value ranges from 0 to 1. The larger the NC value, the more relevant the extracted target watermark image is to the original preset watermark image, and the stronger the robustness of the watermark image against attacks.

[0121] Furthermore, based on the normalized correlation coefficient NC, the authenticity verification result of the input video to be verified can be determined. For example, when the normalized correlation coefficient NC is greater than the preset threshold, the input video to be verified is true; when the normalized correlation coefficient NC is not greater than the preset threshold, the input video to be verified is false.

[0122] The above watermark embedding method and watermark extraction method can be implemented by software programs such as Matlab or OpenCV, such as Figure 5 As shown, the embodiment of the present application further provides a video anti-counterfeiting identification system, which may include:

[0123] The video reading module 510 is used to read the original target video or the video embedded with the watermark image to be verified, and decompose the video into continuous video frame images;

[0124] The anti-counterfeiting watermark generation module 520 is used to generate a watermark image;

[0125] The watermark embedding module 530 is used to divide the watermark image into local watermark images and embed them into discontinuous target video frame images to generate a video containing anti-counterfeiting identification watermark information;

[0126] It is also used to save the frame number of the selected target video frame image as a marked data file in the system, which is used as the basis for selecting the target video frame image when extracting the watermark; at the same time, the watermark image and the partial watermark image are saved in the system as data files, which is convenient for comparing and retrieving the extracted watermark with the preset watermark;

[0127] Attack module 540 is used to simulate attacks that videos may be subjected to during actual network transmission. For example, attacks such as mosaic, cropping, filtering, rotation, scaling, sharpening, and beautification can be added, or no attacks can be added (in an ideal state). In this way, the ability of the watermark method to resist attacks can be verified, thereby improving the accuracy of identity authentication.

[0128] The watermark extraction module 550 is used to extract the attacked watermark image from the video containing the anti-counterfeiting watermark information;

[0129] The result display module 560 is used to compare the extracted watermark image with the preset original watermark image, and use NC to evaluate the watermark's ability to resist attacks, that is, its robustness, and thus verify the authenticity of the video.

[0130] The video anti-counterfeiting identification system provided by the embodiment of the present application can be an online cloud system software or an application installed in a smart phone terminal. After logging into the system with a valid account and password, the video to be verified is read in, and the extracted watermark image is compared with the watermark image already existing in the database to verify the authenticity of the video. For the user, first log in to the anti-counterfeiting identification detection system of the present invention, read in the local original video, generate a watermark image as anti-counterfeiting identification information and save it in the database in a data file format as a basis for tracing back copyright protection or anti-counterfeiting. If a suspected video is found to be circulated on the Internet, it is downloaded and saved locally. The watermark image of the verification video is extracted by the anti-counterfeiting system. The extracted watermark information is retrieved and compared with the watermark information already in the database. The evaluation standard for the authenticity of the video is the normalized correlation coefficient NC (NC is used to identify the robustness of the watermark). In response to the user's touch operation of "Run" on the user interface, the video authenticity identification result and the NC coefficient are displayed to the user to verify the authenticity and copyright of the video.

[0131] Figure 6 A schematic diagram of the hardware structure of an electronic device that implements an embodiment of the present application is shown. Referring to the figure, at the hardware level, the electronic device 600 includes a processor 610, and optionally, an internal bus 620, a network interface 630, and a memory 640. Among them, the memory 640 may include a memory 641, such as a high-speed random access memory (RAM), and may also include a non-volatile memory 642 (non-volatile memory), such as at least one disk storage device. Of course, the electronic device may also include hardware required for other services.

[0132] The processor 610, the network interface 630, and the memory can be interconnected via an internal bus 620. The internal bus 620 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Such buses can be classified as address buses, data buses, control buses, and the like. For ease of illustration, only one bidirectional arrow is used in this figure, but this does not imply that there is only one bus or only one type of bus.

[0133] The memory 640 stores programs. Specifically, the programs may include program codes, which include computer operating instructions. The memory 640 may include a memory 641 and a non-volatile memory 642, and provides instructions and data to the processor 610.

[0134] The processor 610 reads the corresponding computer program from the non-volatile memory 642 into the memory and then runs it, forming a device for locating the target user at the logical level. The processor 610 executes the program stored in the memory and specifically performs the following: Figure 1 or Figure 3 The methods disclosed in the illustrated embodiments implement the functions and beneficial effects of the various methods described in the foregoing method embodiments, which will not be described in detail here.

[0135] The above application Figure 1 or Figure 3 The methods disclosed in the illustrated embodiments can be applied to or implemented by processor 610. Processor 610 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in processor 610. The above-mentioned processor 610 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0136] The computer device can also execute the methods described in the above method embodiments and realize the functions and beneficial effects of the methods described in the above method embodiments, which will not be repeated here.

[0137] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0138] The embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, which, when executed by an electronic device including multiple application programs, enables the electronic device to execute Figure 1 or Figure 3 The methods disclosed in the illustrated embodiments implement the functions and beneficial effects of the various methods described in the foregoing method embodiments, which will not be described in detail here.

[0139] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0140] Furthermore, an embodiment of the present application provides a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the following process is implemented: Figure 1 or Figure 3 The methods disclosed in the illustrated embodiments implement the functions and beneficial effects of the various methods described in the foregoing method embodiments, which will not be described in detail here.

[0141] The embodiments of the present application can be applied to various electronic device collaboration or interconnection scenarios, including: collaboration and interconnection between mobile phones and laptops / tablets; collaboration and interconnection between mobile terminals and smart TVs / displays; collaboration and interconnection between mobile phones or tablets and in-car entertainment systems; collaboration and interconnection between mobile terminals and smart conference systems, etc., thereby meeting the diverse needs of users in scenarios such as smart homes, smart offices, and smart travel.

[0142] In short, the above description is only a preferred embodiment of the present application and does not limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0143] The systems, devices, modules, or units described in the above embodiments may be implemented by computer chips or entities, or by products having certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0144] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0145] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0146] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

Claims

1. A watermark embedding method, characterized in that: include: Obtaining an input target video, and determining a target video frame image to be embedded with a watermark according to video parameters of the target video and / or image parameters of each video frame image in the target video; Extracting features of the target video frame image and determining a watermark embedding area of the target video frame image; Obtaining a first singular value matrix of the watermark embedding area; Segmenting the preset watermark image according to the number of the target video frame images and the first singular value matrix to obtain a plurality of local watermark images; The multiple local watermark images are correspondingly embedded into the watermark embedding area of the target video frame image to obtain a watermarked video frame image.

2. The method according to claim 1, characterized in that After determining the target video frame image to be embedded with the watermark, the method further includes: Determining the number of marked frames of the target video according to position information of the target video frame image in the target video; The marked frame number is saved in a target file in a preset file format.

3. The method according to claim 1, characterized in that The extracting the features of the target video frame image and determining the watermark embedding area of the target video frame image includes: Extracting high-level features of the target video frame image using a preset feature extraction network; A first weight matrix is determined according to the high-level features, and the target video frame image is weightedly processed using the first weight matrix to obtain a watermark embedding area of the target video frame image.

4. The method according to claim 1, wherein The step of segmenting the preset watermark image according to the number of the target video frame images and the first singular value matrix to obtain a plurality of local watermark images includes: Determining the number of divisions of the watermark image according to the number of target video frame images, wherein the number of divisions is less than or equal to the number of target video frame images; Segmenting the watermark image according to the number of segments to obtain a plurality of candidate local watermark images; According to the dimension of the first singular value matrix, the multiple candidate local watermark images are preprocessed to obtain multiple local watermark images; wherein the dimension of the local watermark image is the same as the dimension of the first singular value matrix.

5. The method according to claim 4, characterized in that The preprocessing of the plurality of candidate local watermark images according to the dimension of the first singular value matrix to obtain a plurality of local watermark images comprises: When it is determined that the dimension of the target candidate local watermark map among the multiple candidate local watermark maps is smaller than the dimension of the first singular value matrix, performing pixel value filling on the target candidate local watermark map to obtain a local watermark map; or In the case that the dimension of the target candidate local watermark map among the multiple candidate local watermark maps is greater than the dimension of the first singular value matrix, pixel values of the target candidate local watermark map are selected to obtain a local watermark map.

6. The method according to claim 1, characterized in that The step of embedding the multiple local watermark images into the watermark embedding areas of the target video frame image to obtain the watermarked video frame image includes: Superimposing each weighted local watermark image with the first singular value matrix of the corresponding watermark embedding area to obtain a second singular value matrix after embedding the local watermark image; determining watermark image features according to the second singular value matrix; The image features of the watermark embedding area of the target video frame image are updated to the watermark image features to obtain the watermark video frame image.

7. The method according to any one of claims 1 to 6, characterized in that The method further includes: generating the preset watermark image by: Acquire a video frame image in the target video; Extracting fully connected layer features of the video frame image; Extracting high-frequency sub-band features of the video frame image; The fully connected layer features and the high-frequency sub-band coefficient features are subjected to feature fusion processing to generate the preset watermark image.

8. A watermark extraction method, characterized in that: include: Get the watermarked video frame image; Extracting features of the watermarked video frame image and determining a watermark extraction area of the watermarked video frame image; Obtaining a second singular value matrix of the watermark extraction area; Extracting a local watermark image corresponding to the watermarked video frame image according to the second singular value matrix and the pre-stored first singular value matrix; A target watermark image is extracted based on the local watermark image corresponding to the watermarked video frame image.

9. The method according to claim 8, characterized in that The step of obtaining a watermarked video frame image includes: Get the video to be verified; Determining the position information of the watermarked video frame image in the video to be verified according to the number of marked frames in the preset target file; The watermarked video frame image is obtained according to the position information.

10. The method according to claim 8, characterized in that The extracting the features of the watermarked video frame image and determining the watermark extraction area of the watermarked video frame image includes: Extracting high-level features of the watermarked video frame image using a preset feature extraction network; A second weight matrix is determined according to the high-level features, and the watermarked video frame image is weightedly processed using the second weight matrix to obtain a watermark extraction area of the watermarked video frame image.

11. The method according to claim 8, characterized in that The step of extracting a target watermark image based on a local watermark image corresponding to the watermarked video frame image includes: When the dimension of the local watermark image is smaller than the dimension of the pre-stored candidate local watermark image, performing pixel value padding on the local watermark image to obtain a pre-processed local watermark image; or when the dimension of the local watermark image is greater than the dimension of the candidate local watermark image, performing pixel value selection on the local watermark image to obtain the preprocessed local watermark image; The pre-processed local watermark images corresponding to the watermarked video frame images are spliced to obtain the target watermark image.

12. The method according to claim 8, characterized in that After extracting the target watermark image according to the local watermark image corresponding to the watermarked video frame image, the method further includes: Determining a normalized correlation coefficient between the target watermark image and a preset watermark image; An authenticity verification result of the video to be verified is determined according to the normalized correlation coefficient.

13. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 12 are implemented.

14. A readable storage medium, characterized in that The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images