Video tampering detection method and device based on content correlation

Abstract

The invention discloses a video tampering detection method and device based on content correlation. The method comprises the following steps: embedding watermark information and auxiliary information into each video sub-sequence of a video file; extracting first watermark information of each video sub-sequence in the video file containing the watermark to obtain a recovered video file, and generating second watermark information based on the recovered video file; and comparing the first watermark information with the second watermark information, if the first watermark information and the second watermark information are all the same, judging that the video file containing the watermark is not tampered, otherwise, judging that the video file is tampered. The device comprises a watermark embedding module, a watermark extraction module and a video tampering detection module. According to the method, reversible and robust embedding of the video watermark information is realized, the accuracy and practicability of video tampering detection are improved, accurate positioning of a tampered area is realized, and the false alarm rate and the missing report rate in various complex video processing scenes are reduced.

Description

Technical Field

[0001] The present invention relates to video image processing technology, and in particular to a video tampering detection method and device based on content correlation. Background Art

[0002] In the process of video information dissemination, the receivers of video information all hope that the received video information is original and accurate, that is, video information that has not been tampered with by a third party. In the prior art, double watermarking is often used for anti-tampering detection. The tampering detection performance of this method will be weakened by video post-processing operations (such as blurring, transcoding, etc.), which is not conducive to reducing the false alarm rate and missed alarm rate of tampering detection in complex video processing scenarios. Summary of the invention

[0003] In order to overcome the above-mentioned shortcomings, the present invention provides a video tampering detection method and device based on content correlation, which can perform tampering detection based on watermark information generated by content correlation characteristics between video subsequences, thereby improving the robustness of video tampering detection and reducing the false alarm rate and missed alarm rate in various complex video processing scenarios of video tampering detection.

[0004] In order to solve the above technical problems, the technical solution of the present invention provides a video tampering detection method based on content relevance in a first aspect, comprising:

[0005] Decompose the video file into several video subsequences according to the set time step, obtain the watermark information of the corresponding video subsequence based on the structural similarity index of two adjacent frames in each video subsequence, and use the error information caused by embedding as the auxiliary information for video recovery, embed the corresponding watermark information and auxiliary information in each video subsequence to obtain a video file containing a watermark;

[0006] Extracting the first watermark information and the auxiliary information of each video subsequence in the watermarked video file, obtaining the restored video file with the aid of the auxiliary information, and forming the second watermark information of each video subsequence based on the restored video file;

[0007] The first watermark information and the second watermark information are compared one by one. If they are all the same, the video file containing the watermark has not been tampered with, otherwise it has been tampered with.

[0008] Preferably, the video tampering detection method further includes:

[0009] The position of the tampered video subsequence is determined based on the difference between the first watermark information and the second watermark information.

[0010] Preferably, the step of obtaining the watermark information of the corresponding video subsequence based on the structural similarity index of two adjacent frames of each video subsequence includes:

[0011] Perform background separation on each frame image of each video subsequence to obtain background information of each frame image;

[0012] Calculating the structural similarity index of two consecutive adjacent frames of images in each video subsequence based on the background information of each frame of image, and obtaining a correlation quantitative index of each video subsequence;

[0013] The correlation quantification index of each video subsequence is converted into watermark information.

[0014] Preferably, embedding corresponding watermark information and auxiliary information in each video subsequence to obtain a video file containing a watermark includes:

[0015] A frame image other than the first frame image and the last frame image is randomly selected in each video subsequence according to normal distribution, and the frame image is combined with the first frame image and the last frame image of the corresponding video subsequence as the watermark embedding frame of the video subsequence;

[0016] The watermark information and auxiliary information of each video subsequence are embedded in the watermark embedded frame of the corresponding video subsequence, and the watermark embedded frame and other frame images of the video subsequence are merged to form a video subsequence containing watermarks, and then the video subsequences containing watermarks are merged to form a video file containing watermarks.

[0017] Preferably, the step of embedding the watermark information and auxiliary information of each video subsequence into a watermark-embedded frame of the corresponding video subsequence, and merging the watermark-embedded frame and other frame images of the video subsequence to form a watermarked video subsequence includes:

[0018] Convert the watermark embedded frame from RGB color space to YUV color space to obtain its brightness information and chromaticity information, extract U channel chromaticity information from it, and divide its U channel into 8×8 blocks;

[0019] Perform DCT transformation on the divided video subsequences and divide them into smooth blocks and complex blocks according to their complexity;

[0020] Convert the correlation quantization index into a binary sequence as watermark information and embed it into the DCT intermediate frequency coefficients of the complex block, and embed the auxiliary information into the DCT intermediate frequency coefficients of the smooth block;

[0021] According to the positions of the complex block and the flat slider in the U channel coefficient matrix, the coefficient sequence after the watermark information and the auxiliary information are embedded is reintegrated into the U channel coefficient matrix, and then the inverse DCT transform is performed to obtain the U channel information after the watermark is embedded, and the watermark embedded frame after the watermark is embedded is obtained based on the U channel information after the watermark is embedded;

[0022] The watermarked frame is converted from the YUV color space to the RGB color space, and is merged with other frame images of the corresponding video subsequence to obtain a video subsequence containing the watermark.

[0023] Preferably, performing DCT transformation on the divided video subsequences to divide them into smooth blocks and complex blocks according to their complexity includes:

[0024] Perform DCT transformation on each sub-block in the U channel information of the divided 8×8 blocks to obtain a U channel coefficient matrix;

[0025] Each block of coefficient matrix is ​​scanned in a zigzag path and then arranged into a one-dimensional coefficient sequence, with the intermediate frequency coefficient located in the middle of each sequence;

[0026] The variance of the intermediate frequency coefficients of each sub-block is calculated block by block. The block with higher variance is the complex block, and the block with lower variance is the smooth block.

[0027] Preferably, the converting the correlation quantization index into a binary sequence as watermark information and embedding it into the DCT intermediate frequency coefficients of the complex block comprises:

[0028] Arrange the one-dimensional coefficient sequence from large to small according to the size of the required intermediate frequency coefficient variance, and record the position of the sub-block corresponding to each sequence in the U channel coefficient matrix;

[0029] The intermediate frequency coefficients of the first n sequences are extracted as the embedding area of ​​the watermark, and the intermediate frequency coefficients of the last n sequences are extracted as the embedding area of ​​the auxiliary information; where n is the number of bits of the binary watermark information, that is, the watermark information to be embedded in each complex coefficient block selected is one bit;

[0030] The watermark information is embedded bit by bit into the intermediate frequency coefficients of the corresponding sequence.

[0031] Preferably, embedding the auxiliary information into the DCT intermediate frequency coefficients of the smooth block comprises:

[0032] Construct auxiliary information for video restoration, that is, embed the watermark into the distortion information of the U channel coefficient as auxiliary information for restoring the video;

[0033] Apply Huffman coding to losslessly compress the auxiliary information to obtain the encoded auxiliary information;

[0034] The encoded auxiliary information is embedded into the DCT intermediate frequency coefficients of the smooth block.

[0035] Preferably, the step of extracting the first watermark information and the auxiliary information of each video subsequence in the watermarked video file, and obtaining the restored video file with the aid of the auxiliary information, comprises:

[0036] Convert the video subsequence containing the watermark from RGB color space to YUV color space, extract the U channel chrominance information, and divide it into non-overlapping sub-blocks of 8×8 size;

[0037] Perform DCT transformation on each sub-block of the U channel information of the divided 8×8 blocks to obtain the U channel coefficient matrix, and then scan each block of the coefficient matrix in a zigzag shape and arrange them into a one-dimensional sequence;

[0038] The variance of the intermediate frequency coefficients of each sub-block is calculated block by block, and they are arranged in order of variance. The following formula is performed on the last n smooth blocks with smaller variance to obtain the compressed auxiliary information γ ′ :

[0039]

[0040] Among them, k i ′ is the intermediate frequency coefficient of the smoothing block, T z is the threshold for controlling the embedding capacity of reversible auxiliary information in the histogram shift algorithm;

[0041] The compressed auxiliary information is decompressed into γ = {s1, s2, ..., s n}, and smooth the sub-block coefficient k i ′ Perform the following recovery:

[0042]

[0043] Among them, k i is the intermediate frequency coefficient of the smoothing block after restoration;

[0044] For the first n complex blocks with large variance, the watermark information w is extracted according to the following formula:

[0045]

[0046] Among them, m ′ i is the intermediate frequency coefficient of the complex block;

[0047] Calculate all modified complex block intermediate frequency coefficients m ′ i If the sum is greater than or equal to 0, the watermark information is 1; if the sum is less than 0, the watermark information is 0;

[0048] Based on the auxiliary information γ={s1,s2,…,s n} and the extracted watermark information w to restore the intermediate frequency coefficient m of the complex block ′ i :

[0049]

[0050] In the formula, m i is the intermediate frequency coefficient of the restored complex block;

[0051] The restored coefficient sequence is integrated into the corresponding U channel coefficient matrix, and then the inverse DCT transform is performed to obtain the U channel information of the restored video subsequence; the video subsequence is then converted from the YUV color space back to the RGB color space, and all video subsequences are merged to obtain the restored video file.

[0052] Preferably, the background separation of each frame image of each video subsequence uses a Gaussian mixture model to separate the foreground and the background to obtain the background information of each frame image, which includes:

[0053] Determine the optimal number K of Gaussian distributions and the threshold T of the background model;

[0054] Randomly initialize the mean μ of the kth Gaussian distribution k,t , covariance matrix Σ k,t and the mixing coefficient ω k,t , where the covariance matrix is ​​simplified as follows:

[0055]

[0056] Traverse each pixel in each frame image, compare the current pixel value with each Gaussian distribution, and find a matching Gaussian distribution; if a matching Gaussian distribution is found, update the parameters of the Gaussian distribution (μ t ,∑ t ,ω t ) and mark the pixel as background. If no matching Gaussian distribution is found, a new Gaussian distribution is created with the current pixel value, replacing the Gaussian distribution with the lowest mixing coefficient and marking the pixel as foreground.

[0057] The parameters of the Gaussian distribution (μ t ,∑ t ,ω t )The update formula is as follows:

[0058] μ t =(1-α)μ t-1 +αx t

[0059]

[0060] ω k，t =(1-α)ω k，t-1 +α(M k，t )

[0061] Where α is the learning rate. When the pixel matches a Gaussian distribution, α(M k，t ) is 1, otherwise it is 0;

[0062] For each pixel, all Gaussian distribution mixing coefficients ω t Sort in order from largest to smallest;

[0063] Determine the background model, accumulate the mixing coefficients of the B Gaussian distributions with the largest mixing coefficients until the cumulative sum is greater than the threshold T, and the above Gaussian distributions constitute the background model, and the rest are considered to be the foreground; the value of B can be calculated by the following formula:

[0064]

[0065] According to the background model, the foreground / background mark of each pixel is updated to obtain the background information of each frame of the image;

[0066] Here, finding a matching Gaussian distribution refers to finding a Gaussian distribution with the smallest difference from the current pixel value.

[0067] The second aspect of the technical solution of the present invention further provides a video tampering detection device based on content relevance, comprising:

[0068] A watermark embedding module is used to decompose a video file into several video subsequences according to a set time step, obtain the watermark information of the corresponding video subsequence based on the structural similarity index of two adjacent frames of each video subsequence, and embed the corresponding watermark information and auxiliary information in each video subsequence based on the error information caused by embedding as auxiliary information for video recovery to obtain a video file containing a watermark;

[0069] A watermark extraction module is used to extract the first watermark information and auxiliary information of each video subsequence in the video file containing the watermark, obtain the restored video file with the help of the auxiliary information, and generate the second watermark information of each video subsequence based on the restored video file;

[0070] The video tampering detection module is used to compare the first watermark information and the second watermark information one by one. If they are all the same, the video file containing the watermark has not been tampered with, otherwise it has been tampered with.

[0071] The third aspect of the technical solution of the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned video tampering detection method based on content relevance is implemented.

[0072] The fourth aspect of the technical solution of the present invention also provides a computer-storable medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the above-mentioned video tampering detection method based on content relevance is implemented.

[0073] Compared with the prior art, the present invention obtains the watermark information of the corresponding video subsequence through the structural similarity index of two adjacent frame images of each video subsequence, and embeds it into each video subsequence, and compares the first watermark information extracted from the video to be detected with the second watermark information generated by the restored video to see if they are exactly the same to determine whether the video to be detected has been tampered with. This method improves the robustness of video tampering detection and reduces the false alarm rate and missed alarm rate in various complex video processing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 Flow chart of a video tampering detection method based on content relevance in an embodiment of the present invention;

[0075] Figure 2 A schematic diagram of generating watermark information in an embodiment of the present invention;

[0076] Figure 3 This is a flow chart of watermark information embedding in an embodiment of the present invention;

[0077] Figure 4 This is a flow chart of watermark extraction and video recovery in an embodiment of the present invention. DETAILED DESCRIPTION

[0078] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0079] like Figure 1 As shown, an embodiment of the present invention provides a video tampering detection method based on content relevance, comprising:

[0080] S1. Decompose the video file into several video subsequences according to the set time step, obtain the watermark information of the corresponding video subsequence based on the structural similarity index of two adjacent frames in each video subsequence, and embed the corresponding watermark information and auxiliary information in each video subsequence based on the error information caused by embedding as the auxiliary information for video recovery to obtain a video file containing a watermark;

[0081] S2, extracting the first watermark information and auxiliary information of each video subsequence in the watermarked video file, obtaining a restored video file with the aid of the auxiliary information, and forming the second watermark information of each video subsequence based on the restored video file;

[0082] S3. Compare the first watermark information and the second watermark information one by one. If they are all the same, the video file containing the watermark has not been tampered with, otherwise it has been tampered with.

[0083] It is easy to understand that the time step can be set to 1 second or multiple seconds, which can be determined specifically according to the video time. Generally, the longer the video time is, the larger the time step can be used, and vice versa, the smaller the time step can be used, which can ultimately achieve a higher watermark embedding efficiency. In a feasible embodiment, a video file can be decomposed into video subsequences A1, A2, A3..., taking the time step of 1 second as an example, A1 represents the video subsequence of the video file at the 1st second, A2 represents the video subsequence of the video file at the 2nd second, and so on.

[0084] Since each video subsequence is composed of multiple frames of images, and this embodiment forms watermark information through the structural similarity index of two adjacent frames of images in each video subsequence, and since any modification to the video content will cause the content structure of the video file to change, the above-mentioned watermark information embedding can facilitate the comparison between the first watermark information and the second watermark information to determine whether the video file has been tampered with.

[0085] The present invention obtains the watermark information of the corresponding video subsequence through the structural similarity index of two adjacent frame images of each video subsequence, and embeds it into each video subsequence, and compares the first watermark information extracted from the video to be detected with the second watermark information generated by the restored video to see if they are exactly the same to determine whether the video to be detected has been tampered with. This method improves the robustness of video tampering detection and reduces the false alarm rate and missed alarm rate in various complex video processing scenarios.

[0086] In a preferred embodiment, the watermark information of the corresponding video subsequence is obtained based on the structural similarity index of two adjacent frames of each video subsequence, including: performing background separation on each frame of each video subsequence to obtain background information of each frame; calculating the structural similarity index of two consecutive adjacent frames in each video subsequence based on the background information of each frame to obtain the correlation quantification index of each video subsequence; converting the correlation quantification index of each video subsequence into watermark information. In this embodiment, the background separation of each frame of each video subsequence can be performed using a Gaussian mixture model to separate the foreground and background to obtain the background information of each frame, and the structural similarity index is calculated by the background information, thereby forming watermark information that reflects the content correlation of the video subsequence.

[0087] For ease of description, this embodiment exemplarily expresses the background information in the following manner, that is, the background information of several video subsequences is: Taking the time step of 1 second as an example, B 11Indicates the background information of the first frame of the video subsequence at the first second, B 12 Indicates the background information of the second frame image in the first second video subsequence, B 21 Indicates the background information of the first frame of the video subsequence at the second second, B 22 represents the background information of the second frame image in the second second video subsequence, and so on. tk It represents the background information corresponding to the k-th frame image in the video subsequence at the t-th moment.

[0088] In a preferred embodiment, the background separation of each frame image of each video subsequence uses a Gaussian mixture model to separate the foreground and the background to obtain the background information of each frame image, which includes:

[0089] (1) Determine the optimal number K of Gaussian distributions and the threshold T of the background model;

[0090] Exemplarily, the optimal number K of Gaussian distributions and the threshold T of the background model may be determined by the following steps:

[0091] First, initialize the number of Gaussian distributions K and the threshold T of the background model, and set the initial value range of K to between 3 and 5, and the initial value range of T to between 0.6 and 0.9; read each frame image in all video subsequences, traverse all possible sets of K and T values, and apply the Gaussian mixture model to separate the foreground and background according to the current K and T values;

[0092] Secondly, the clustering evaluation indicators Silhouette Coefficient and Calinski-Harabasz Index are used to evaluate the separation effect of foreground and background. First, the Silhouette Coefficient and Calinski-Harabasz Index are standardized to the same range (such as 0 to 1) to facilitate weighted combination. Then the weight of the Silhouette Coefficient is assigned to α, and the weight of the Calinski-Harabasz Index is (1-α). Next, the Silhouette Coefficient and Calinski-Harabasz Index are calculated, and the two evaluation indicators are standardized and weighted combined. Finally, the cross-validation method is used to try different weight combinations and select the weight with the best effect.

[0093] The silhouette coefficient is defined as: Where i represents the i-th sample, a(i) represents the average distance from sample i to other samples in the same cluster, and b(i) represents the average distance from sample i to the samples in the nearest cluster (but not including the cluster where the sample is located). The silhouette coefficient of the entire data set is the average of the silhouette coefficients of all samples: When s(i) is close to 1, it means that sample i is well clustered, close to other samples in the same cluster and far away from samples in other clusters. When s(i) is close to 0, it means that sample i is on the boundary of two clusters. When s(i) is close to -1, it means that sample i may be clustered incorrectly.

[0094] The calculation formula of the Calinski-Harabasz index is: Where k represents the total number of clusters contained in the sample set, N is the total number of samples contained in the sample set, and SS B is the inter-cluster divergence, which represents the sum of squares of the deviations between the cluster center and the global center, SS W It is the intra-cluster divergence, which represents the sum of squared deviations between samples within each cluster and the cluster center.

[0095] Finally, according to the results of the evaluation index, the K and T values ​​are adaptively adjusted, and the combination of K and T values ​​that maximizes the silhouette coefficient and Calinski-Harabasz index weighted value is selected, which is the target K and T value.

[0096] (2) Randomly initialize the mean μ of the kth Gaussian distribution of the video subsequence at time t k，t , covariance matrix∑ k，t and the mixing coefficient ω k，t , where the covariance matrix is ​​simplified as follows:

[0097]

[0098] in, is the variance of the kth Gaussian distribution of the video subsequence at time t, and I is the identity matrix.

[0099] (3) Traverse each pixel point in each frame image, compare the current pixel value with each Gaussian distribution, and find a matching Gaussian distribution; if a matching Gaussian distribution is found, update the parameters of the Gaussian distribution (μ t ,∑ t ,ω t ) and mark the pixel as background. If no matching Gaussian distribution is found, the current pixel value x is used. t Create a new Gaussian distribution, replace the Gaussian distribution with the lowest mixing coefficient and mark the pixel as foreground; it is worth noting that in this embodiment, finding a matching Gaussian distribution means finding a Gaussian distribution that matches the current pixel value x t Gaussian distribution with minimal variance.

[0100] The parameters of the Gaussian distribution (μ t ,∑ t ,ω t )The update formula is as follows:

[0101] μt =(1-α)μ t-1 +αx t

[0102]

[0103] ω k，t =(1-α)ω k，t-1 +α(M k，t )

[0104] Where α is the learning rate, μ t , ∑ t are the mean, variance and covariance matrix of the Gaussian distribution of the video subsequence at time t, μ t-1 , are the mean and variance of the Gaussian distribution of the video subsequence at time t-1, ω k，t is the mixing coefficient of the kth Gaussian distribution of the video subsequence at time t, ω k，t-1 is the mixing coefficient of the kth Gaussian distribution of the video subsequence at time t-1, and I is the unit matrix. k，t is an indicator function used to indicate whether the current pixel value matches the kth Gaussian distribution. If it matches, then M k,t =1, otherwise 0;

[0105] (4) For all Gaussian distribution mixing coefficients ω of each pixel t Sort in order from largest to smallest;

[0106] (5) Determine the background model, accumulate the mixing coefficients of the B Gaussian distributions with the largest mixing coefficients until the cumulative sum is greater than the threshold T, and the above Gaussian distributions constitute the background model, and the rest are considered to be the foreground; the value of B can be calculated by the following formula:

[0107]

[0108] B represents the number of Gaussian distributions selected in the background model. That is, the number of Gaussian distributions that meet the conditions, which is the minimum b value when the sum of the mixing coefficients of the first b Gaussian distributions exceeds the threshold T. argmin b Is a mathematical symbol that means finding the minimum value of b that makes the conditions in the brackets true. k is the mixing coefficient of the kth Gaussian distribution. T is the threshold of the background model.

[0109] (6) Based on the background model, update the foreground / background label of each pixel to obtain the background information of each frame image.

[0110] It should be pointed out that if the background information in some video subsequences is always in a state of slight change, resulting in the difference between the pixel value and its Gaussian distribution model exceeding the preset threshold, and finally being misidentified as the foreground, then the video subsequence after the Gaussian mixture model is established is subjected to the opening operation in the morphological operation, and the separation result is effectively refined by first performing the erosion operation and then the dilation operation.

[0111] Exemplarily, the specific steps and formulas of the Gaussian mixture model algorithm are as follows:

[0112] (1) Probability density function. For a pixel point x, the probability density function of the pixel value sequence {x|x1,x2,…} in all frames can be expressed as the weighted sum of K Gaussian distributions. The pixel value x at the current time t is t The resulting probability density function is described by the following formula:

[0113]

[0114] Among them, K is the number of Gaussian distributions required to represent the grayscale value of the pixel in a scene, ω i,t is the mixing coefficient of the i-th Gaussian distribution at time t, satisfying And 0≤ω i,t ≤1, is the probability density function of the i-th Gaussian distribution at time t, and its mean is μ i,t , the covariance matrix is ​​Σ i,t .

[0115] (2) The probability density function of a single Gaussian distribution can be calculated as follows:

[0116]

[0117] Among them, |Σ i,t | is the determinant of the covariance matrix, Σ -1 is the inverse matrix of the covariance matrix, d is x t Dimension.

[0118] In some preferred embodiments, the step of calculating the structural similarity index of two consecutive adjacent frames of images in each video subsequence based on the background information of each frame of image to obtain a correlation quantification index of each video subsequence; and converting the correlation quantification index of each video subsequence into watermark information comprises:

[0119] (1) Obtaining the background information of each frame image in the corresponding video subsequence, calculating the background information structural similarity index SSIM of two adjacent frames in the video subsequence, and obtaining a quantitative index C1, C2, C3, etc. reflecting the relevance of the video content;

[0120] The value range of the structural similarity index SSIM is (0,1), where 1 means that the background parts of the two adjacent frames in the video subsequence are exactly the same, and 0 means that the background parts of the two adjacent frames in the video subsequence are completely different. By calculating the SSIM value for the background information of each pair of consecutive frames in the video subsequence, a quantitative index C1, C2, C3... for measuring the relevance of the video content can be obtained, where C1 represents the SSIM value quantitative index of the first and second frames in the video subsequence, C2 represents the SSIM value quantitative index of the second and third frames in the video subsequence, and so on.

[0121] The structural similarity index SSIM can be calculated using the following formula:

[0122]

[0123] In the formula, B t It represents the background information corresponding to the frame image at the tth moment, B t+1 It represents the background information corresponding to the frame image at the t+1th moment, SSIM(B t ,B t+1 ) indicates background information B t With background information B t+1 The structural similarity index between and are the background information B in the video subsequence respectively. t and background information B t+1 The mean of and Respectively represent the background information B in the video subsequence t and background information B t+1 The variance of It means the background information B in the video subsequence t and background information B t+1 The covariance of, c1 and c2 are constants. Among them, c1=(k1L) 2 , c2=(k2L) 2 , k1, k2<<1, L is the grayscale level of the frame image (for example, for an 8-bit grayscale image, L=255). The mean is used as an estimate of brightness, the standard deviation is used as an estimate of contrast, and the covariance is used as a measure of structural similarity. In this embodiment, k1 and k2 are stable constants used to prevent the denominator from being zero. Their specific function is to avoid the situation where the denominator in the formula is close to zero when the image contrast or brightness difference is very small, resulting in unstable calculation results. Exemplarily, k1=0.01, k2=0.03.

[0124] In some preferred implementations, to ensure that the embedded watermark information has reasonable accuracy, the calculated original SSIM value is retained to two decimal places.

[0125] (2) converting the SSIM value of the background information of each two adjacent frames of the above correlation quantification index into binary, and concatenating the multiple binary numbers finally converted to form a long binary sequence, which is used as watermark information;

[0126] In order to convert the correlation quantitative index into binary form for embedding watermark, it can be magnified 100 times and converted into a real number between 0 and 100. This step not only avoids the precision loss and infinite repeating decimal problem that may occur in the direct conversion from the original SSIM value to the binary number, but also makes the expression of the watermark information more intuitive.

[0127] The SSIM value after amplification 100 times is converted into a binary number. Since the data in the computer is usually stored in bytes, each byte contains 8 binary bits, the SSIM value is converted into an 8-bit binary number, and the watermark information that is less than 8 bits is padded with zeros in front to ensure the consistency of the number of bits.

[0128] After converting the SSIM values ​​of the background information of each two adjacent frames of images into binary numbers, these binary numbers are connected in series to form a long binary sequence as the final watermark information.

[0129] For example, taking the Foreman video whose first second video subsequence is 30 frames of images as an example, the steps for generating the first second watermark information are as follows: Figure 2 As shown in the figure, after separating the background of 30 frames in the 1st second video subsequence, the SSIM value between two adjacent frames is calculated, and the SSIM value between the 1st and 2nd frames is 0.84, the SSIM value between the 2nd and 3rd frames is 0.85, ... the SSIM value between the 29th and 30th frames is 0.91; then it is magnified 100 times and converted into the corresponding binary number, such as the binary number corresponding to 0.84 after magnification is 01011001, and the binary number corresponding to 0.91 is 01011011, and then these binary numbers are connected in series in sequence, and the watermark information to be embedded in the first second of the final foreman video is 01011001010101010101010110 ... 01011011; the above generation operation is performed on each video subsequence of the Foreman video, and the obtained binary sequences represent the correlation information of the background content of each second of the video, and then embedded into the video as watermark information.

[0130] In one of the preferred embodiments, the method of embedding corresponding watermark information and auxiliary information in each video subsequence to obtain a watermarked video file includes: randomly selecting a frame image other than the first frame image and the last frame image in each video subsequence according to a normal distribution, and combining it with the first frame image and the last frame image of the corresponding video subsequence as the watermark embedded frame of the video subsequence; embedding the watermark information and auxiliary information of each video subsequence into the watermark embedded frame of the corresponding video subsequence, merging the watermark embedded frame after the watermark is embedded and other frame images of the video subsequence to form a video subsequence containing a watermark, and then merging the video subsequences containing watermarks to form a video file containing a watermark.

[0131] Exemplarily, in order to generate a random frame position sequence that follows a normal distribution, this embodiment uses a random number generator to generate a series of uniformly distributed random numbers in the range of 0 to 1, and then uses a Box-Muller transformation to transform the uniformly distributed random numbers into normally distributed random numbers. The Box-Muller transformation includes the following specific steps:

[0132] (1) Generate two random numbers x and y from a uniform distribution;

[0133] (2) Convert them into random numbers Z1 and Z2 in a standard normal distribution using the Box-Muller transformation formula; wherein the Box-Muller transformation formula is:

[0134]

[0135]

[0136] in, It is a transformation based on y, which generates a random variable that conforms to the Rayleigh distribution. The Rayleigh distribution is a continuous probability distribution that is often used to describe the magnitude of a vector with two orthogonal normal components. cos(2πx) and sin(2πx) map x to two coordinates on the unit circle. This is done to ensure the independence of the two generated normal variables Z1 and Z2.

[0137] In particular, although the same expected value and standard deviation are input, the sequence generated each time is unique due to the different randomly generated initial input values. In order to obtain a consistent embedding position sequence under given expected value and standard deviation conditions, this embodiment fixes the randomly generated initial input value. Then, by adjusting the expected value and standard deviation, different frame position sequences can be generated, thereby enhancing the confidentiality of the watermark information, so that potential attackers cannot determine which frame images in the video contain the watermark information.

[0138] (3) Scale and translate Z1 and Z2 by the input mean and standard deviation to obtain normally distributed random numbers that meet the required parameters.

[0139] like Figure 3 As shown, in some preferred embodiments, the watermark information and auxiliary information of each video subsequence are embedded into the watermark embedded frame of the corresponding video subsequence, and the watermark embedded frame after the watermark is embedded and other frame images of the video subsequence are combined to form a video subsequence containing a watermark, including:

[0140] (1) The watermark embedded frame is converted from the RGB color space to the YUV color space to obtain its brightness information and chromaticity information, and the U channel chromaticity information is extracted therefrom, and its U channel is divided into 8×8 blocks.

[0141] For example, YUV is a color space that represents brightness (Y) and chrominance (U, V) separately, and the conversion formula is as follows:

[0142] Y=0.299*R+0.587*G+0.114*B

[0143] U=-0.147*R-0.289*G+0.436*B

[0144] V=0.615*R-0.289*G+0.436*B

[0145] After the above color space conversion, the U channel chromaticity information is extracted and processed in 8×8 blocks to divide the U channel data into 8×8 two-dimensional matrices.

[0146] (2) Performing DCT transformation on the divided video subsequences and dividing them into smooth blocks and complex blocks according to their complexity;

[0147] This embodiment exemplarily provides a preferred implementation method, in which each sub-block in the U channel information of the divided 8×8 blocks is subjected to DCT transformation to obtain a U channel coefficient matrix; each block of the coefficient matrix is ​​scanned along a zigzag path, and then arranged into a one-dimensional coefficient sequence, and the intermediate frequency coefficient is located in the middle part of each sequence, usually located from the 16th coefficient to the 40th coefficient of the sequence; the variance of the intermediate frequency coefficient of each sub-block is calculated block by block, and the block with a higher variance is a complex block, and the block with a lower variance is a smooth block.

[0148] (3) Converting the correlation quantization index into a binary sequence as watermark information and embedding it into the DCT intermediate frequency coefficients of the complex block, and embedding the auxiliary information into the DCT intermediate frequency coefficients of the smooth block.

[0149] In some feasible embodiments, the conversion of the correlation quantization index into a binary sequence as watermark information and embedding it into the DCT intermediate frequency coefficients of the complex block includes: arranging the one-dimensional coefficient sequence from large to small according to the size of the required intermediate frequency coefficient variance, and recording the position of the sub-block corresponding to each sequence in the U channel coefficient matrix; extracting the intermediate frequency coefficients of the first n sequences as the embedding area of ​​the watermark, and the intermediate frequency coefficients of the last n sequences as the embedding area of ​​the auxiliary information; wherein n is the number of bits of the binary watermark information, that is, the watermark information required to be embedded in each selected complex coefficient block is one bit; and embedding the watermark information bit by bit into the intermediate frequency coefficients of the corresponding sequence.

[0150] Exemplarily, the watermark information after SSIM quantization is converted into a binary sequence D1, D2, D3... and embedded into the DCT intermediate frequency coefficients of the complex block. The specific embedding formula is as follows:

[0151]

[0152] Wherein, w is the watermark information, indicating each bit of the watermark data to be embedded. For example, when w=1, it means that the watermark bit to be embedded is "1", m i Represents the intermediate frequency coefficient value of the sequence corresponding to the complex block, s n Represents the amount of modification required to the intermediate frequency coefficients in the nth complex block coefficient sequence in order to embed the nth watermark bit. i represents the intermediate frequency coefficient value of the complex block before modification (that is, the intermediate frequency coefficient of the complex block after restoration), m ′ i Represents the intermediate frequency coefficient value of the modified complex block. Since the intermediate frequency coefficients are usually located from the 16th to the 40th coefficients in the sequence, the total number of coefficients required to be modified for each sequence to embed 1 bit of watermark information is 25. n The value of is obtained by calculating the average value of the background model threshold T minus the total value of the intermediate frequency coefficient of each sequence, and rounding it down, as shown in the following formula:

[0153]

[0154] If the watermark information is 1, all the intermediate frequency coefficients of the corresponding sequence are added with s n If it is 0, subtract s n .

[0155] In some other feasible embodiments, the embedding of the auxiliary information into the DCT intermediate frequency coefficients of the smooth block includes: constructing auxiliary information for video recovery, i.e., embedding the watermark into the distortion information of the U channel coefficients as auxiliary information for restoring the video; applying Huffman coding lossless compression to the auxiliary information to obtain the encoded auxiliary information; and embedding the encoded auxiliary information into the DCT intermediate frequency coefficients of the smooth block.

[0156] For example, since the embedding of watermark information in this paper will also affect the SSIM value of the background, in order to prevent the embedding of watermark information from affecting the result of tampering detection, the distortion information of the U channel coefficient caused by the watermark is embedded as the auxiliary information γ of the restored video, γ={s1,s2,…,s n}.

[0157] In order to save space, the auxiliary information needs to be losslessly compressed using Huffman coding. The specific steps are as follows:

[0158] ①Construct a symbol set. Since γ={s1,s2,…,s n} is a series of error information, and these error information are calculated according to the following formula:

[0159]

[0160] In the formula, s n Indicates the nth error information, s n The range of values ​​is limited, so different s in γ can be n The value is considered as a set of Huffman-coded symbols.

[0161] ②For each symbol (i.e. each different s n Then generate a set of symbols and their corresponding frequencies, for example: {(s1, f1), (s2, f2), ..., (s i , f i )}, where s i represents the i-th symbol, f i Indicates its frequency of occurrence.

[0162] ③ All symbols are divided according to their frequency f i Sort from small to large. Then merge the two symbols with the smallest frequency in turn to generate a new node whose frequency is the sum of the two symbol frequencies. Finally, insert the new node into the sorting queue and repeat this process until all symbols are merged into the root node of the same Huffman tree.

[0163] ④ Starting from the root node, assign code 0 to each left child node and code 1 to the right child node. In this way, each symbol s i Each is assigned a unique Huffman code h i .

[0164] ⑤ According to the Huffman tree constructed in steps 3) and 4), the auxiliary information γ = {s1, s2, ..., s n Each symbol s in} iEncode to generate a bit stream γ′, which represents the auxiliary information after lossless compression, γ′∈[0,1]. Thus, Huffman coding lossless compression of the auxiliary information is achieved.

[0165] The encoded auxiliary information γ′ is embedded into the DCT intermediate frequency coefficients of the smooth block. In order to avoid mutual interference between the watermark information and the auxiliary information, the coefficients are adjusted based on the principle of the histogram shift algorithm. Its basic idea is to embed the encoded information into the DCT intermediate frequency coefficients by moving the values ​​of the intermediate frequency coefficients on the histogram. When the original value of the i-th intermediate frequency coefficient in the smooth block is within a certain range, the value of the intermediate frequency coefficient is selectively moved according to the auxiliary information, the value of γ′ (0 or 1), to achieve the purpose of embedding information. The specific steps are as follows:

[0166] ① Select the specific position of the intermediate frequency coefficient in the smooth block (16th to 40th) as the embedding point. Then count the frequency of the values ​​of these intermediate frequency coefficients, construct a histogram, and set a threshold T z , the coefficient value with frequency greater than the threshold is regarded as the "peak point" of the histogram. z It is the threshold value for controlling the embedding capacity of reversible auxiliary information in the histogram shift algorithm. Since this variable has little influence on the video subsequence, it can be set to a constant value T. z =1.

[0167] ② In order to normalize the intermediate frequency coefficients exceeding the threshold without being affected by the embedded information, when k i >T z or k i <-T z When the coefficient k is directly i Shift right or left T z +1. That is, when k i >T z When k i Shift to the right, that is, k i The value of T increases z +1; when k i <T z When k i Shift to the left, that is, k i The value of T is reduced z +1. When γ′=1, it means that the information to be embedded is bit “1”. i ∈(0,T z ] the intermediate frequency coefficient, k i Shift to the right, that is, k i The value of increases by γ′×(T z +1). i ∈[-T z , 0) of the intermediate frequency coefficient, k i Shift to the left, that is, k iThe value of z +1). When γ′=0, it means that the information to be embedded is bit “0”. i ∈(0,T z ] and k i ∈[-T z , 0), no translation operation is performed and the coefficient remains unchanged. j represents the modified intermediate frequency coefficient value. The specific embedding process can be expressed by the following formula:

[0168]

[0169] ③ Regenerate the frame blocks with the DCT coefficients after embedding the information, perform inverse DCT transform, and restore the video subsequence.

[0170] (4) According to the positions of the complex blocks and the flat sliders in the U channel coefficient matrix, the coefficient sequence after the watermark information and the auxiliary information are embedded is reintegrated into the U channel coefficient matrix, and then an inverse DCT transform is performed to obtain the U channel information after the watermark is embedded, and the watermark embedded frame after the watermark is embedded is obtained based on the U channel information after the watermark is embedded.

[0171] (5) Convert the watermarked frame from the YUV color space to the RGB color space, and merge it with other frame images of the corresponding video subsequence to obtain a video subsequence containing the watermark.

[0172] like Figure 4 As shown, in some preferred embodiments, the step of extracting the first watermark information of each video subsequence in the watermarked video file to obtain the restored video file includes:

[0173] (1) Convert the watermarked video subsequence from RGB color space to YUV color space, extract the U channel chrominance information, and divide it into non-overlapping sub-blocks of 8×8 size;

[0174] (2) Perform DCT transformation on each sub-block of the U channel information of the divided 8×8 blocks to obtain the U channel coefficient matrix, and then scan each block of the coefficient matrix in a zigzag pattern and arrange them into a one-dimensional sequence;

[0175] (3) The variance of the intermediate frequency coefficients of each sub-block is calculated block by block, and the blocks are arranged in order of variance. The following formula is performed on the last n smooth blocks with smaller variance to obtain the compressed auxiliary information γ′:

[0176]

[0177] Among them, k′ i is the intermediate frequency coefficient of the smoothing block, T zis the threshold value for controlling the embedding capacity of reversible auxiliary information in the histogram shift algorithm; From the above formula, we can see that if the intermediate frequency coefficient of the smoothing block is within (-2×(T z -1), -T z ) or (T z , 2×(T z +1)), the auxiliary information is 1; if the intermediate frequency coefficient of the smoothing block is in the range of [-T z , T z ] and is not 0, the auxiliary information is 0.

[0178] (4) Decompress the compressed auxiliary information into γ = {s1, s2, ..., s n}, and smooth the sub-block coefficient k′ i Perform the following recovery:

[0179]

[0180] Among them, k i is the intermediate frequency coefficient of the smoothing block after restoration;

[0181] (5) For the first n complex blocks with large variance, extract the watermark information w according to the following formula:

[0182]

[0183] Among them, m′ j is the modified intermediate frequency coefficient value of the complex block; by calculating all the modified intermediate frequency coefficient values ​​m′ of the complex block i If the sum is greater than or equal to 0, the watermark information is 1; if the sum is less than 0, the watermark information is 0. This step determines the embedded watermark information based on the positive or negative value of the modification amount.

[0184] (6) Calculate the intermediate frequency coefficient values ​​m′ of all modified complex blocks i If the sum is greater than or equal to 0, the watermark information is 1; if the sum is less than 0, the watermark information is 0;

[0185] Based on the auxiliary information γ = {s1, s2, ..., s n} and the extracted watermark information w to restore the intermediate frequency coefficient m′ of the complex block i , thus obtaining the restored complex fast intermediate frequency coefficient m i , the specific formula is as follows:

[0186]

[0187] In the formula, m i is the intermediate frequency coefficient of the restored complex block;

[0188] (7) Integrate the restored coefficient sequence into the corresponding U channel coefficient matrix, and then perform inverse DCT transform to obtain the U channel information of the restored video subsequence; then convert the video subsequence from the YUV color space back to the RGB color space, and merge all the video subsequences to obtain the restored video file.

[0189] For example, in some preferred embodiments, the present embodiment extracts watermark information in segments during the watermark extraction process to obtain watermark information w, and uses the same method to generate watermark information w′ for the restored video file. Because in the process of generating watermark information, the background information SSIM value of each two adjacent frames is converted into 8-bit binary watermark information, and zeros are added to the front of the watermark information that is less than 8 bits to ensure the consistency of the number of bits. Therefore, w and w′ are divided into 8-bit segments respectively, and w={w1, w2, …, w′} is obtained. n}, w′={w′1, w′2,…,w′ n In actual tampering detection, w1 and w′1, w2 and w′2, …, w n and w′ n Perform bit-by-bit XOR. If all the obtained values ​​are 1, it is determined that no frame tampering operation has occurred in the video subsequence of the video file; if the final obtained value includes 0, it means that tampering operation has occurred in the video subsequence of the video file.

[0190] Moreover, in some feasible embodiments, the video tampering detection method further includes: determining the position of the tampered video subsequence based on the difference between the first watermark information and the second watermark information. Specifically, let the frame image in the video subsequence of the video file be f i , the value range of i is (1, n+1). Use the following formula to locate and record the location of the tampered frame image:

[0191]

[0192] This embodiment can accurately determine the position of the tampered video subsequence through the above method.

[0193] In some embodiments, this embodiment further provides a video tampering detection device based on content relevance, including:

[0194] A watermark embedding module is used to decompose a video file into several video subsequences according to a set time step, obtain the watermark information of the corresponding video subsequence based on the structural similarity index of two adjacent frames of each video subsequence, and embed the corresponding watermark information and auxiliary information in each video subsequence based on the error information caused by embedding as auxiliary information for video recovery to obtain a video file containing a watermark;

[0195] a watermark extraction module, used to extract the first watermark information and auxiliary information of each video subsequence in the watermarked video file, obtain the restored video file with the help of the auxiliary information, and form the second watermark information of each video subsequence based on the restored video file; and

[0196] The video tampering detection module is used to compare the first watermark information and the second watermark information one by one. If they are all the same, the video file containing the watermark has not been tampered with, otherwise it has been tampered with.

[0197] Since the execution mode of the video tampering detection device based on content relevance in this embodiment is substantially the same as the above-mentioned video tampering detection method based on content relevance, it will not be described in detail.

[0198] The electronic device of this embodiment provides computing services, which generally refers to a computer with high computing power and provided to multiple consumers through a network. The electronic device of this embodiment includes: a memory, a processor, and a system bus, and the memory includes executable programs stored thereon. Those skilled in the art can understand that the terminal device structure of this embodiment does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0199] The memory can be used to store software programs and modules. The processor executes various functional applications and data processing of the terminal by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the terminal (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0200] An executable program of a video tampering detection method based on content relevance is included in a memory, and the executable program can be divided into one or more modules / units. The one or more modules / units are stored in the memory and executed by a processor to complete the information acquisition and implementation process. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program in the processor.

[0201] The processor is the control center of the electronic device. It uses various interfaces and lines to connect the various parts of the entire terminal device. It executes various functions of the terminal and processes data by running or executing software programs and / or modules stored in the memory, and calling data stored in the memory, so as to monitor the terminal as a whole. Optionally, the processor may include one or more processing units; preferably, the processor may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor.

[0202] The system bus is used to connect the various functional components inside the computer. It can transmit data information, address information, and control information. Its types can be, for example, PCI bus, ISA bus, VESA bus, etc. The processor's instructions are transmitted to the memory through the bus, and the memory feeds back data to the processor. The system bus is responsible for the data and instruction exchange between the processor and the memory. Of course, the system bus can also be connected to other devices, such as network interfaces, display devices, etc.

[0203] The processor should at least include a CPU, a chipset, a memory, a disk system, etc. Other components will not be described in detail here.

[0204] In an embodiment of the present invention, the executable program executed by the processor included in the electronic device is specifically: a video tampering detection method based on content relevance, which includes the following steps:

[0205] Decompose the video file into several video subsequences according to the set time step, obtain the watermark information of the corresponding video subsequence based on the structural similarity index of two adjacent frames of each video subsequence, and use the error information caused by embedding as the auxiliary information for video recovery, embed the corresponding watermark information and auxiliary information in each video subsequence to obtain a video file containing a watermark;

[0206] Extracting the first watermark information of each video subsequence in the watermarked video file to obtain a restored video file, and forming the second watermark information of each video subsequence based on the restored video file;

[0207] The first watermark information and the second watermark information are compared one by one. If they are all the same, the video file containing the watermark has not been tampered with, otherwise it has been tampered with.

[0208] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0209] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0210] Those of ordinary skill in the art will appreciate that the modules, units and / or method steps of the various embodiments described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0211] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0212] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A video tampering detection method based on content relevance, characterized in that: include: Decompose the video file into several video subsequences according to the set time step, obtain the watermark information of the corresponding video subsequence based on the structural similarity index of two adjacent frames in each video subsequence, and use the error information caused by embedding as the auxiliary information for video recovery, embed the corresponding watermark information and auxiliary information in each video subsequence to obtain a video file containing a watermark; Extracting the first watermark information and the auxiliary information of each video subsequence in the watermarked video file, obtaining the restored video file with the aid of the auxiliary information, and generating the second watermark information of each video subsequence based on the restored video file; The first watermark information and the second watermark information are compared one by one. If they are all the same, the video file containing the watermark has not been tampered with, otherwise it has been tampered with.

2. The video tampering detection method based on content correlation according to claim 1 is characterized in that: The video tampering detection method also includes: The position of the tampered video subsequence is determined based on the difference between the first watermark information and the second watermark information.

3. The video tampering detection method based on content correlation according to claim 1 is characterized in that: The method of obtaining the watermark information of the corresponding video subsequence based on the structural similarity index of two adjacent frames of each video subsequence includes: Perform background separation on each frame image of each video subsequence to obtain background information of each frame image; Calculating the structural similarity index of two consecutive adjacent frames of images in each video subsequence based on the background information of each frame of image, and obtaining a correlation quantitative index of each video subsequence; Convert the correlation quantification index of each video subsequence into watermark information; The method of embedding corresponding watermark information and auxiliary information in each video subsequence to obtain a video file containing a watermark includes: A frame image other than the first frame image and the last frame image is randomly selected in each video subsequence according to normal distribution, and the frame image is combined with the first frame image and the last frame image of the corresponding video subsequence as the watermark embedding frame of the video subsequence; The watermark information and auxiliary information of each video subsequence are embedded in the watermark embedded frame of the corresponding video subsequence, and the watermark embedded frame and other frame images of the video subsequence are merged to form a video subsequence containing watermarks, and then the video subsequences containing watermarks are merged to form a video file containing watermarks.

4. The video tampering detection method based on content correlation according to claim 3 is characterized in that: The method of embedding the watermark information and auxiliary information of each video subsequence into a watermark-embedded frame of the corresponding video subsequence, and merging the watermark-embedded frame and other frame images of the video subsequence to form a watermarked video subsequence includes: Convert the watermark embedded frame from RGB color space to YUV color space to obtain its brightness information and chromaticity information, extract U channel chromaticity information from it, and divide its U channel into 8×8 blocks; Perform DCT transformation on the divided video subsequences and divide them into smooth blocks and complex blocks according to their complexity; Convert the correlation quantization index into a binary sequence as watermark information and embed it into the DCT intermediate frequency coefficients of the complex block, and embed the auxiliary information into the DCT intermediate frequency coefficients of the smooth block; According to the positions of the complex block and the flat slider in the U channel coefficient matrix, the coefficient sequence after the watermark information and the auxiliary information are embedded is reintegrated into the U channel coefficient matrix, and then the inverse DCT transform is performed to obtain the U channel information after the watermark is embedded, and the watermark embedded frame after the watermark is embedded is obtained based on the U channel information after the watermark is embedded; The watermarked frame is converted from the YUV color space to the RGB color space, and is merged with other frame images of the corresponding video subsequence to obtain a video subsequence containing the watermark.

5. The video tampering detection method based on content correlation according to claim 4 is characterized in that: The DCT transformation is performed on the divided video subsequences, and the subsequences are divided into smooth blocks and complex blocks according to the complexity, including: Perform DCT transformation on each sub-block in the U channel information of the divided 8×8 blocks to obtain a U channel coefficient matrix; Each block of coefficient matrix is ​​scanned in a zigzag path and then arranged into a one-dimensional coefficient sequence, with the intermediate frequency coefficient located in the middle of each sequence; The variance of the intermediate frequency coefficients of each sub-block is calculated block by block. The block with higher variance is the complex block, and the block with lower variance is the smooth block. The step of converting the correlation quantization indicator into a binary sequence as watermark information and embedding the watermark information into a DCT intermediate frequency coefficient of a complex block comprises: Arrange the one-dimensional coefficient sequence from large to small according to the size of the required intermediate frequency coefficient variance, and record the position of the sub-block corresponding to each sequence in the U channel coefficient matrix; The intermediate frequency coefficients of the first n sequences are extracted as the embedding area of ​​the watermark, and the intermediate frequency coefficients of the last n sequences are extracted as the embedding area of ​​the auxiliary information; where n is the number of bits of the binary watermark information, that is, the watermark information to be embedded in each complex coefficient block selected is one bit; Embed the watermark information bit by bit into the intermediate frequency coefficients of the corresponding sequence; The embedding of the auxiliary information into the DCT intermediate frequency coefficients of the smooth block comprises: Construct auxiliary information for video restoration, that is, embed the watermark into the distortion information of the U channel coefficient as auxiliary information for restoring the video; Apply Huffman coding to losslessly compress the auxiliary information to obtain the encoded auxiliary information; The encoded auxiliary information is embedded into the DCT intermediate frequency coefficients of the smooth block.

6. The video tampering detection method based on content correlation according to claim 5 is characterized in that: The method of extracting the first watermark information and the auxiliary information of each video subsequence in the watermarked video file and obtaining the restored video file with the aid of the auxiliary information comprises: Convert the video subsequence containing the watermark from RGB color space to YUV color space, extract the U channel chrominance information, and divide it into non-overlapping sub-blocks of 8×8 size; Perform DCT transformation on each sub-block of the U channel information of the divided 8×8 blocks to obtain the U channel coefficient matrix, and then scan each block of the coefficient matrix in a zigzag shape and arrange them into a one-dimensional sequence; The variance of the intermediate frequency coefficients of each sub-block is calculated block by block, and they are arranged in order of variance. The following formula is performed on the last n smooth blocks with smaller variance to obtain the compressed auxiliary information γ ′ : Among them, k i ′ is the intermediate frequency coefficient of the smoothing block, T z is the threshold for controlling the embedding capacity of reversible auxiliary information in the histogram shift algorithm; The compressed auxiliary information is decompressed into γ = {s1, s2, ..., s n }, and smooth the sub-block coefficient k i ′ Perform the following recovery: Among them, s n Indicates the nth error information, k i is the intermediate frequency coefficient of the smoothing block after restoration; For the first n complex blocks with large variance, the watermark information w is extracted according to the following formula: Among them, m ′ i is the intermediate frequency coefficient of the complex block; Calculate all modified complex block intermediate frequency coefficients m ′ i If the sum is greater than or equal to 0, the watermark information is 1; if the sum is less than 0, the watermark information is 0; Based on the auxiliary information γ={s1,s2,…,s n } and the extracted watermark information w to restore the intermediate frequency coefficient m of the complex block ′ i : In the formula, m i is the intermediate frequency coefficient of the restored complex block; The restored coefficient sequence is integrated into the corresponding U channel coefficient matrix, and then the inverse DCT transform is performed to obtain the U channel information of the restored video subsequence; the video subsequence is then converted from the YUV color space back to the RGB color space, and all video subsequences are merged to obtain the restored video file.

7. The video tampering detection method based on content correlation according to claim 3 is characterized in that: The background separation of each frame image of each video subsequence uses a Gaussian mixture model to separate the foreground and the background to obtain the background information of each frame image, which includes: Determine the optimal number K of Gaussian distributions and the threshold T of the background model; Randomly initialize the mean μ of the kth Gaussian distribution k,t , covariance matrix Σ k,t and the mixing coefficient ω k,t , where the covariance matrix is ​​simplified as follows: in, is the variance of the kth Gaussian distribution of the video subsequence at time t, and I is the identity matrix; Traverse each pixel in each frame image, compare the current pixel value with each Gaussian distribution, and find a matching Gaussian distribution; if a matching Gaussian distribution is found, update the parameters of the Gaussian distribution (μ t ,Σ t ,ω t ) and mark the pixel as background. If no matching Gaussian distribution is found, a new Gaussian distribution is created with the current pixel value, replacing the Gaussian distribution with the lowest mixing coefficient and marking the pixel as foreground. The parameters of the Gaussian distribution (μ t ,Σ t ,ω t )The update formula is as follows: m t =(1-a)m t-1 +αx t oh k,t =(1-a)ω k,t-1 +α(M k,t ) Where α is the learning rate, x t is the current pixel value, μ t , Σ t are the mean, variance and covariance matrix of the Gaussian distribution of the video subsequence at time t, μ t-1 , are the mean and variance of the Gaussian distribution of the video subsequence at time t-1, ω k,t is the mixing coefficient of the kth Gaussian distribution of the video subsequence at time t, ω k,t-1 is the mixing coefficient of the kth Gaussian distribution of the video subsequence at time t-1, I is the identity matrix, M k,t is an indicator function used to indicate whether the current pixel value matches the kth Gaussian distribution. If it matches, M k,t =1, otherwise 0; For each pixel, all Gaussian distribution mixing coefficients ω t Sort in order from largest to smallest; Determine the background model, accumulate the mixing coefficients of the B Gaussian distributions with the largest mixing coefficients until the cumulative sum is greater than the threshold T, and the above Gaussian distributions constitute the background model, and the rest are considered to be the foreground; the value of B can be calculated by the following formula: B represents the number of Gaussian distributions selected in the background model, that is, the number of Gaussian distributions that meet the conditions. It is the minimum b value when the sum of the mixing coefficients of the first b Gaussian distributions exceeds the threshold T; argmin b Is a mathematical symbol, which means finding the minimum b value that makes the conditions in the brackets true; ω k is the mixing coefficient of the kth Gaussian distribution; T is the threshold of the background model; According to the background model, the foreground / background mark of each pixel is updated to obtain the background information of each frame of the image; Here, finding a matching Gaussian distribution refers to finding a Gaussian distribution with the smallest difference from the current pixel value.

8. A video tampering detection device based on content relevance, characterized in that: include A watermark embedding module is used to decompose a video file into several video subsequences according to a set time step, obtain watermark information of the corresponding video subsequence based on the structural similarity index of two adjacent frames in each video subsequence, and embed the corresponding watermark information and auxiliary information in each video subsequence based on the error information caused by embedding as auxiliary information for video recovery to obtain a video file containing a watermark; A watermark extraction module is used to extract the first watermark information and auxiliary information of each video subsequence in the video file containing the watermark, obtain the restored video file with the help of the auxiliary information, and generate the second watermark information of each video subsequence based on the restored video file; and The video tampering detection module is used to compare the first watermark information and the second watermark information one by one. If they are all the same, the video file containing the watermark has not been tampered with, otherwise it has been tampered with.

9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the video tampering detection method based on content relevance described in any one of claims 1 to 7 is implemented.

10. A computer storable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video tampering detection method based on content relevance described in any one of claims 1 to 7 is implemented.

Images