A monitoring video tampering detection method and system based on feature data feedback

Abstract

The application relates to the technical field of video tampering detection, and discloses a monitoring video tampering detection method and system based on feature data feedback, and a monitoring video tampering detection method based on feature data feedback, which comprises the following steps: step S101, calculating a normalized brightness frame; step S102, generating a block boundary activation field; step S103, generating a multi-scale grid energy score; step S104, calculating a multi-scale spectral kurtosis; step S105, screening a hidden dominant candidate set; and step S106, judging the classification label of non-malicious error concealment and malicious frame deletion. The application firstly performs normalization processing on the brightness component of a video frame, reduces the interference of non-tampering factors such as exposure changes, then captures the structural traces of error concealment through a time derivative field and a block boundary activation field, strengthens feature differences through frequency domain transformation and multi-scale energy analysis, and finally completes classification in combination with a presentation timestamp to construct a decision index.

Description

Technical Field

[0001] This invention relates to the field of video tampering detection technology, and more specifically, to a method and system for detecting tampering in surveillance videos based on feature data feedback. Background Technology

[0002] Surveillance video plays an irreplaceable role in security, evidence collection, and judicial fields, and its data integrity is directly related to tracing the truth of events and determining responsibility. In practical applications, surveillance video often faces two types of frame loss issues. One type is non-malicious frame loss during transmission and storage due to network fluctuations or equipment failures, which the decoding end fills in the missing content according to the encoded blocks through error hiding mechanisms. The other type is malicious frame deletion / editing to cover up key information. Both types of situations manifest as discontinuities in the timeline between frames, forming similar-looking isomorphic phenomena, posing a significant challenge to accurately distinguishing the type of tampering.

[0003] Existing video tampering detection technologies have significant limitations. Most methods can only detect missing frames or data integrity corruption, failing to effectively distinguish between non-malicious error concealment and malicious frame deletion. This problem stems from multiple factors. Error concealment, performed by filling in coded blocks, leaves structural traces, while malicious frame deletion lacks such characteristics. However, both manifest as discontinuities in inter-frame data, making it difficult for traditional methods to capture these subtle differences. Furthermore, surveillance videos are often affected by non-tampering factors such as exposure changes, automatic gain adjustment, and natural textures. These interferences can mask the structural features of error concealment, leading to misjudgments by traditional detection methods. In addition, existing technologies often rely on inter-frame consistency checks or complex feature stacking, which not only lacks robustness but also struggles to adapt to the block structure differences of different coding standards such as H.264 and HEVC. This further exacerbates the difficulty of distinguishing between the two types of missing frames, failing to meet the need for accurate tampering type determination in practical applications and impacting the validity and reliability of surveillance videos as evidence. Summary of the Invention

[0004] This invention provides a method and system for detecting tampering in surveillance videos based on feature data feedback, thereby solving the technical problems mentioned in the background.

[0005] This invention provides a method for detecting tampering in surveillance videos based on feature data feedback, comprising the following steps:

[0006] Step S101: Decode the video stream, extract the luminance component of each frame, calculate the intra-frame mean and standard deviation, and perform mean removal and normalization operations on the luminance component accordingly to obtain a normalized luminance frame.

[0007] Step S102: Perform differential calculation on adjacent normalized luminance frames to obtain the temporal derivative field, calculate the absolute value of the difference in the horizontal and vertical dimensions of the temporal derivative field respectively and superimpose them to generate the block boundary activation field.

[0008] Step S103: Transform the block boundary activation field to the two-dimensional frequency domain, extract the grid harmonic energy ratio of the corresponding frequency band according to the preset coding block scale, and generate the multi-scale grid energy fraction corresponding to each block scale accordingly.

[0009] Step S104: Calculate the distribution kurtosis value of the multi-scale raster energy fraction, and multiply the distribution kurtosis value by the maximum value in the multi-scale raster energy fraction to obtain the multi-scale spectral kurtosis.

[0010] Step S105: Calculate the median and median absolute deviation of the multi-scale spectral kurtosis of the entire sequence, and calculate the standardized anomaly based on this. Select video frames with anomalies exceeding a preset threshold to form a hidden dominant candidate set.

[0011] Step S106: Extract the video presentation timestamp sequence, calculate the normalized time gap intensity, construct a decision index by combining multi-scale spectral kurtosis, and obtain the classification labels of non-malicious error hiding and malicious frame deletion in the hidden dominant candidate set based on the decision index.

[0012] This invention provides a surveillance video tampering detection system based on feature data feedback, comprising:

[0013] The normalized luminance frame module decodes the video stream, extracts the luminance component of each frame, calculates the intra-frame mean and standard deviation, and performs mean removal and normalization operations on the luminance components accordingly to obtain a normalized luminance frame.

[0014] The block boundary activation field generation module performs differential calculations on adjacent normalized luminance frames to obtain a temporal derivative field. It then calculates the absolute values ​​of the differences in the temporal derivative field in the horizontal and vertical dimensions and superimposes them to generate the block boundary activation field.

[0015] The multi-scale grid energy fraction generation module transforms the block boundary activation field to the two-dimensional frequency domain, extracts the grid harmonic energy ratio of the corresponding frequency band according to the preset coding block scale, and generates the multi-scale grid energy fraction corresponding to each block scale accordingly.

[0016] The multi-scale spectral kurtosis calculation module calculates the distribution kurtosis value of the multi-scale raster energy fraction, and multiplies the distribution kurtosis value by the maximum value in the multi-scale raster energy fraction to obtain the multi-scale spectral kurtosis.

[0017] The hidden dominant candidate set screening module calculates the median and median absolute deviation of the multi-scale spectral kurtosis of the entire sequence, and then calculates the standardized outlier based on this. Video frames with an outlier exceeding a preset threshold are selected to form the hidden dominant candidate set.

[0018] The video tampering judgment module extracts the video presentation timestamp sequence to calculate the normalized time gap intensity, and constructs a judgment index by combining multi-scale spectral kurtosis. Based on the judgment index, it obtains the classification labels of non-malicious error hiding and malicious frame deletion in the hidden dominant candidate set.

[0019] The beneficial effects of this invention are as follows: First, it normalizes the luminance components of video frames to reduce interference from non-tampering factors such as exposure changes. Then, it captures structural traces hidden by errors through a time derivative field and block boundary activation field. Frequency domain transformation and multi-scale energy analysis enhance feature differences. Finally, it constructs a decision index based on the presentation timestamp to complete the classification. This invention adapts to the block structure differences of mainstream video coding standards, exhibits strong robustness, and provides clear tampering type determination results for security, evidence collection, and judicial fields. This makes the data integrity verification of surveillance videos more targeted, ensuring their usability and credibility as evidence, thereby meeting the refined needs of video tampering detection in practical applications. Attached Figure Description

[0020] Figure 1 This is a flowchart of a surveillance video tampering detection method based on feature data feedback according to the present invention;

[0021] Figure 2 This is a schematic diagram of a surveillance video tampering detection system based on feature data feedback according to the present invention.

[0022] In the figure: Normalized brightness frame module 201, block boundary activation field generation module 202, multi-scale raster energy score generation module 203, multi-scale spectral kurtosis calculation module 204, hidden dominant candidate set screening module 205, and video tampering judgment module 206. Detailed Implementation

[0023] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0024] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in one or more embodiments of the present invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" indicate that the element or object preceding the term encompasses the elements or objects listed following the term and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0025] like Figures 1-2 As shown, a method for detecting tampering in surveillance videos based on feature data feedback includes the following steps:

[0026] Step S101: Decode the video stream, extract the luminance component of each frame, calculate the intra-frame mean and standard deviation, and perform mean removal and normalization operations on the luminance component accordingly to obtain a normalized luminance frame.

[0027] Step S102: Perform differential calculation on adjacent normalized luminance frames to obtain the temporal derivative field, calculate the absolute value of the difference in the horizontal and vertical dimensions of the temporal derivative field respectively and superimpose them to generate the block boundary activation field.

[0028] Step S103: Transform the block boundary activation field to the two-dimensional frequency domain, extract the grid harmonic energy ratio of the corresponding frequency band according to the preset coding block scale, and generate the multi-scale grid energy fraction corresponding to each block scale accordingly.

[0029] Step S104: Calculate the distribution kurtosis value of the multi-scale raster energy fraction, and multiply the distribution kurtosis value by the maximum value in the multi-scale raster energy fraction to obtain the multi-scale spectral kurtosis.

[0030] Step S105: Calculate the median and median absolute deviation of the multi-scale spectral kurtosis of the entire sequence, and calculate the standardized anomaly based on this. Select video frames with anomalies exceeding a preset threshold to form a hidden dominant candidate set.

[0031] Step S106: Extract the video presentation timestamp sequence, calculate the normalized time gap intensity, construct a decision index by combining multi-scale spectral kurtosis, and obtain the classification labels of non-malicious error hiding and malicious frame deletion in the hidden dominant candidate set based on the decision index.

[0032] In one embodiment of the present invention, the luminance component of each frame is extracted from the decoded video stream, the intra-frame mean and standard deviation are calculated, and mean removal and normalization operations are performed on the luminance component accordingly to obtain a normalized luminance frame, including:

[0033] Decode the video stream frame by frame to extract the brightness values ​​of each pixel in a single frame. ,in The frame number, Horizontal pixel coordinates Vertical pixel coordinates For frame width, For frame height;

[0034] The intra-frame mean is obtained by summing the brightness values ​​of each pixel in the single frame and dividing by the total number of pixels. The calculation formula is: ,in This represents the total number of pixels.

[0035] The standard deviation of the frame is obtained by summing the squares of the differences between the brightness values ​​of each pixel in the single frame and the mean value within the frame, dividing the sum by the total number of pixels, and then taking the square root. The calculation formula is:

[0036] ;

[0037] Define a non-zero minimum constant The normalized luminance frame is generated by subtracting the intra-frame mean from the luminance value of each pixel in the single-frame image, and then dividing by the sum of the intra-frame standard deviation and the non-zero minimum constant. The calculation formula is:

[0038] .

[0039] It should be noted that the brightness value of each pixel in a single frame image is the quantized brightness value corresponding to each pixel position in the single frame image, reflecting the brightness level of that pixel. This can be achieved by decoding the frames in the video stream using decoding algorithms conforming to mainstream video coding standards such as H.264 or H.265, extracting the Y component data in the YUV (luminance / chrominance) space from the decoded image data, i.e., the brightness value of each pixel. The total number of pixels is the total number of pixels contained in a single frame image, reflecting the pixel scale of the single frame image. This can be obtained by reading the resolution parameters of the decoded single frame image of the video stream, i.e., the frame width and frame height, and multiplying the frame width and frame height. The frame width and frame height can be obtained by parsing from the Sequence Parameter Set (SPS) of the video coding. The intra-frame mean reflects the overall average brightness level of a single frame image. The intra-frame standard deviation is a statistical measure of the deviation of the brightness value of each pixel in a single frame image from the intra-frame mean, reflecting the discrete distribution of the brightness values ​​in a single frame image. The non-zero minimum constant is a tiny positive number set to avoid the denominator being zero during normalization calculations; its preferred value is 10 to the power of negative 4. The normalized luminance frame is image data obtained after the luminance values ​​of each pixel in a single frame have been processed by removing the mean and normalizing. It reflects the intra-frame luminance distribution after eliminating overall brightness differences and luminance scale differences.

[0040] It should be noted that generating normalized brightness frames can eliminate the overall brightness and brightness scale differences between different frames caused by non-tampering factors such as exposure changes and automatic gain adjustment. This reduces the interference of these irrelevant factors on feature extraction in subsequent steps, making the subsequently calculated features such as the temporal derivative field and block boundary activation field more focused on the structural changes caused by video tampering or erroneous hiding. This provides stable and comparable basic data for the entire surveillance video tampering detection method.

[0041] In one embodiment of the present invention, a temporal derivative field is obtained by performing differential calculation on adjacent normalized luminance frames. The absolute values ​​of the differences in the temporal derivative field are calculated in the horizontal and vertical dimensions respectively and then superimposed to generate a block boundary activation field, including:

[0042] Extract the current normalized luminance frame Compared with the previous normalized luminance frame The temporal derivative field is obtained by subtracting the value of the pixel at the same coordinate position in the previous normalized brightness frame from the value of each pixel in the current normalized brightness frame. The calculation formula is:

[0043] ,in The frame number, Let these be pixel coordinates, and their values ​​should satisfy the following range: and , and These are the frame width and frame height, respectively.

[0044] Calculate the absolute value of the difference between the value of each pixel in the time derivative field and the value of its horizontally adjacent pixel to obtain the absolute value of the horizontal difference. The calculation formula is:

[0045] ,in The range of values ​​is ;

[0046] Calculate the absolute value of the difference between the value of each pixel in the time derivative field and the value of its vertically adjacent pixel to obtain the absolute value of the vertical difference. The calculation formula is:

[0047] ,in The range of values ​​is ;

[0048] Add the absolute value of the horizontal difference to the absolute value of the vertical difference to generate the block boundary activation field. The calculation formula is:

[0049] ,in This represents the absolute value of the horizontal difference. This represents the absolute value of the vertical difference.

[0050] It should be noted that the temporal derivative field is the set of differences between the pixel values ​​at the same coordinates in the current normalized luminance frame and the previous normalized luminance frame, reflecting the intensity of the temporal change in luminance between two adjacent frames. The absolute value of the horizontal difference is the set of absolute values ​​of the differences between each pixel in the temporal derivative field and its horizontally adjacent pixel values, reflecting the intensity of pixel changes in the horizontal direction of the temporal derivative field. The absolute value of the vertical difference is the set of absolute values ​​of the differences between each pixel in the temporal derivative field and its vertically adjacent pixel values, reflecting the intensity of pixel changes in the vertical direction of the temporal derivative field. The block boundary activation field is the result of superimposing the absolute values ​​of the horizontal and vertical differences, reflecting the activation degree of block boundaries in the temporal derivative field.

[0051] It should be noted that video coding block boundaries may exhibit both horizontal and vertical pixel abrupt changes. Single-directional differencing cannot fully capture these two-dimensional boundary features. Superposition allows for simultaneous response to both horizontal and vertical block boundary abrupt changes, making the block-by-block filling traces caused by error concealment more prominent in the activation field. For pixels located at the right boundary of the frame (x-coordinate equal to frame width minus 1), their horizontal difference absolute value is 0; for pixels located at the bottom boundary of the frame (y-coordinate equal to frame height minus 1), their vertical difference absolute value is 0. The block boundary activation field value at these boundary pixels is equal to the absolute value of the difference in the effective direction (i.e., only the effective absolute value of the difference in the vertical or horizontal direction is retained). For example, if a pixel is located at the right boundary of the frame with a y-coordinate of 5, its horizontal difference absolute value is 0, and its vertical difference absolute value is calculated as 2, then the block boundary activation field value for this pixel is 2. If a pixel is located at both the right and bottom boundaries, its horizontal and vertical difference absolute values ​​are both 0, and the block boundary activation field value is 0.

[0052] It should be noted that by obtaining the temporal derivative field reflecting the brightness changes between frames through the difference between adjacent frames, and then capturing the spatial abrupt changes in this field by calculating the absolute values ​​of the differences in the horizontal and vertical directions, the block boundary activation field is finally generated by superposition. This method can suppress the interference of regular static textures and steady-state coded block effects, because these features change gradually in adjacent frames and are weakened in the temporal derivative field and subsequent difference operations, while the transient abrupt changes in block boundaries caused by error hiding are amplified. This provides targeted input for the block raster resonance detection in the subsequent steps, making the subsequent analysis more focused on the structural features of error hiding, and providing key feature support for the entire method to distinguish between non-malicious error hiding and malicious frame deletion.

[0053] In one embodiment of the present invention, the block boundary activation field is transformed to a two-dimensional frequency domain, and the grid harmonic energy ratio of the corresponding frequency band is extracted according to a preset coding block scale. Based on this, a multi-scale grid energy fraction corresponding to each block scale is generated, including:

[0054] Activation field for block boundary Perform a two-dimensional discrete Fourier transform to obtain the two-dimensional frequency domain amplitude. The calculation formula is:

[0055] ,in For frame width, For frame height, For horizontal frequency domain coordinates, Vertical frequency domain coordinates The imaginary unit;

[0056] Define a set of coding block scales containing the values ​​8, 16, 32, and 64. ,Right now ;

[0057] For any coded block scale in this set of coded block scales Calculate the quotient of frame width to the size of the coding block and the quotient of frame height to the size of the coding block, respectively, and multiply these two quotients by the first, second, and third harmonic orders, respectively. Then, rounding to the nearest integer, we obtain the set of frequency points in the horizontal direction. frequency point set in the vertical direction The calculation formula is:

[0058] ;

[0059] ;

[0060] in This is the rounding function;

[0061] The frequency domain coordinates belonging to the set of frequency points in the horizontal direction or the set of frequency points in the vertical direction are determined as the corresponding frequency bands. Defined as:

[0062] ;

[0063] Define a non-zero minimum constant Calculate the sum of squares of the two-dimensional frequency domain amplitude within the corresponding frequency band, and use it as the numerator. Calculate the sum of squares of the two-dimensional frequency domain amplitude across the entire frequency domain and the sum of the non-zero minimum constant, and use this sum as the denominator. Divide the numerator by the denominator to obtain the grid harmonic energy ratio. The calculation formula is:

[0064] ;

[0065] By combining the harmonic energy proportions of the raster corresponding to each coding block scale, a multi-scale raster energy fraction is generated. , is represented as: .

[0066] It should be noted that the 2D Discrete Fourier Transform (DFT) is an algorithm that converts spatial domain data to the frequency domain, used to convert the block boundary activation field into a 2D frequency domain amplitude. The 2D frequency domain amplitude is the frequency domain value obtained after the block boundary activation field undergoes the DFT, reflecting the energy intensity at different frequency points. The coding block scale set contains four values: 8, 16, 32, and 64, reflecting the typical block granularity of mainstream video coding. The coding block scale is any value in the coding block scale set, reflecting the spatial size of the coding block. The quotient of frame width and coding block scale is the value obtained by dividing the frame width by the coding block scale, reflecting the ratio of the frame's horizontal direction to the coding block scale. The quotient of frame height and coding block scale is the value obtained by dividing the frame height by the coding block scale, reflecting the ratio of the frame's vertical direction to the coding block scale. The harmonic orders are three fixed values: 1, 2, and 3, reflecting the order of the raster harmonics.

[0067] It should be noted that the horizontal frequency set is the set of frequency points obtained by multiplying the quotient of the frame width and the coding block scale by the harmonic order and rounding it to the nearest integer, reflecting the frequency domain position of the raster harmonics in the horizontal direction. The vertical frequency set is the set of frequency points obtained by multiplying the quotient of the frame height and the coding block scale by the harmonic order and rounding it to the nearest integer, reflecting the frequency domain position of the raster harmonics in the vertical direction. The corresponding frequency band is the region containing the frequency domain coordinates corresponding to either the horizontal or vertical frequency set, reflecting the frequency domain range of the raster harmonics that matches the coding block scale. The non-zero minimum constant is a fixed small value used to avoid division by zero errors, reflecting the protection threshold to prevent calculation anomalies. The sum of squares of the two-dimensional frequency domain amplitudes within the corresponding frequency band is the cumulative value of the squares of the two-dimensional frequency domain amplitudes of all frequency points in the corresponding frequency band, reflecting the total energy of that frequency band. The sum of squares of the two-dimensional frequency domain amplitudes across the entire frequency domain is the cumulative value of the squares of the two-dimensional frequency domain amplitudes of all frequency points in the entire frequency domain, reflecting the total energy of the frequency domain. The grid harmonic energy percentage is the ratio of the energy in the corresponding frequency band to the sum of the energy in the entire frequency domain plus a non-zero minimum constant, reflecting the proportion of grid harmonic energy at a specific coding block scale. The multi-scale grid energy fraction is the set of grid harmonic energy percentages corresponding to each coding block scale, reflecting the distribution of grid harmonic energy at different coding block scales.

[0068] It should be noted that surveillance videos commonly use H.264 and HEVC encoding. H.264 has a macroblock size of 16 and a subblock size of 8, while HEVC has a CTU and large-size encoding blocks of 32 and 64. Selecting these four scales comprehensively covers the block structure of mainstream encoding methods. When error hiding is performed on an encoding block basis, a grid structure with a periodicity around the encoding block scale is formed. Its fundamental frequency in the frequency domain is the quotient of the frame size and the block size. The 1st, 2nd, and 3rd harmonics can completely cover the energy accumulation characteristics of this grid structure, avoiding the omission of key information by a single harmonic. For example, a surveillance video using H.264 encoding has an encoding block size of 16, a frame width of 1920, and a fundamental frequency of 1920 divided by 16 equals 120. The 1st, 2nd, and 3rd harmonic frequencies are 120, 240, and 360, respectively. The energy accumulation near these frequencies precisely reflects the error hiding grid structure. Calculating the energy proportion of this frequency band can accurately capture this feature. In contrast, the energy distribution of natural textures is dispersed and does not form significant accumulations in these specific frequency bands.

[0069] It should be noted that the specific implementation of the 2D Discrete Fourier Transform (DFT) is explicitly stated as follows: The block boundary activation field is zero-padded to the nearest power of 2. If the frame width or frame height is already a power of 2, no padding is required. The zero-padded values ​​do not change the frequency domain characteristics of the original data. The handling of frequency points exceeding the range is explicitly stated as follows: the maximum value of the frequency domain coordinates is the size after the 2D DFT minus 1. When the calculated frequency point is greater than or equal to this maximum value, the maximum value is used; when the frequency point is less than 0, 0 is used. For example, if the frame width is 1920, after zero padding to 2048, the maximum horizontal coordinate in the frequency domain is 2047. Given a coding block size of 8 and harmonic order 3, the calculated value is 1920 divided by 8 and multiplied by 3, which equals 720. This is within the acceptable range, so we take 720. If the frame width is 500, after padding to 512, the maximum horizontal coordinate in the frequency domain is 511. Given a coding block size of 8 and harmonic order 3, 500 divided by 8 and multiplied by 3 equals 187.5, which rounds to 188. This is within the acceptable range, so we take 188. If the frame width is 200, after padding to 256, the maximum horizontal coordinate in the frequency domain is 255. Given a coding block size of 8 and harmonic order 3, 200 divided by 8 and multiplied by 3 equals 75. This is within the acceptable range, so we take 75.

[0070] It should be noted that this invention converts the block-structure filling traces caused by error hiding into quantifiable frequency domain indicators, adapting to the block structure features of different coding standards, while suppressing the interference of natural texture and steady-state coding block effects. This is because the energy of such interference is dispersed in the frequency domain and will not form significant clusters in specific grid harmonic frequency bands, providing an accurate frequency domain feature basis for subsequent steps to distinguish between non-malicious error hiding and malicious frame deletion through multi-scale analysis.

[0071] In one embodiment of the present invention, the distribution kurtosis value of the multi-scale raster energy fraction is calculated, and the distribution kurtosis value is multiplied by the maximum value among the multi-scale raster energy fractions to obtain the multi-scale spectral kurtosis, including:

[0072] The cumulative multi-scale grid energy fraction includes the proportion of harmonic energy of each grid. Divide the accumulated result by the number of elements in the coded block scale set. to obtain the mean The calculation formula is:

[0073] ,in For the coding block scale, The frame number;

[0074] Calculate the square of the difference between the harmonic energy percentage of each grid and the mean, sum all the squared results and divide by the number of elements to obtain the second-order central moment. The calculation formula is:

[0075] ;

[0076] Calculate the fourth power of the difference between the harmonic energy percentage of each grid and the mean, sum all the fourth power results and divide by the number of elements to obtain the fourth-order central moment. The calculation formula is:

[0077] ;

[0078] Define a non-zero minimum constant Using the fourth central moment as the numerator and the sum of the square of the second central moment and the non-zero minimum constant as the denominator, calculate the quotient of the numerator divided by the denominator to obtain the distribution kurtosis value. The calculation formula is:

[0079] ;

[0080] The grid harmonic energy percentage with the largest value among the multi-scale grid energy fractions is selected as the maximum fraction. The calculation formula is:

[0081] ;

[0082] Calculate the product of the distribution kurtosis value and the maximum fraction to generate the multi-scale spectral kurtosis. The calculation formula is:

[0083] .

[0084] It should be noted that the second-order central moment is the average of the squares of the differences between the energy proportions of each grid harmonic and the mean, reflecting the dispersion of each fraction relative to the mean. The fourth-order central moment is the average of the fourth power of the differences between the energy proportions of each grid harmonic and the mean, reflecting the peak correlation characteristics of each fraction relative to the mean. The non-zero minimum constant is a fixed small value used to avoid division-by-zero errors, reflecting a protection threshold to prevent calculation anomalies, preferably 10 to the power of -4. The distribution kurtosis is the ratio of the fourth-order central moment to the sum of the squares of the second-order central moment and the non-zero minimum constant, reflecting the peak distribution of the multi-scale grid energy fractions. The maximum fraction is the grid harmonic energy proportion with the largest value among the multi-scale grid energy fractions, reflecting the intensity of the strongest grid harmonic energy proportion. The multi-scale spectral kurtosis is the product of the distribution kurtosis and the maximum fraction, reflecting the composite characteristics of the fused distribution pattern and the strongest resonance intensity.

[0085] It should be noted that the block-by-block execution characteristic of error concealment leads to a significant prominence of raster harmonic energy at a certain coding block scale, exhibiting a single-scale spike distribution. The distribution kurtosis value can capture this spike characteristic, and the maximum score can reflect the strongest resonance intensity. Multiplying the two can amplify the feature signal of error concealment. For example, error concealment results in a raster harmonic energy proportion of 0.8 at a certain coding block scale, while other scales are around 0.1, with a mean of 0.275, a second-order central moment of approximately 0.08, a fourth-order central moment of approximately 0.12, a distribution kurtosis value of approximately 17.5, a maximum score of 0.8, and a multi-scale spectral kurtosis of 14. In contrast, under natural texture interference, the energy proportions at each scale are evenly distributed between 0.2 and 0.3, with a mean of 0.25, a second-order central moment of approximately 0.001, a fourth-order central moment of approximately 0.00001, a distribution kurtosis value of approximately 4, a maximum score of 0.3, and a multi-scale spectral kurtosis of 1.2, significantly lower than in the error concealment scenario, thus achieving feature differentiation.

[0086] In one embodiment of the present invention, the median and median absolute deviation of the multi-scale spectral kurtosis of the entire sequence are statistically analyzed, and the standardized outlier is calculated accordingly. Video frames with outliers exceeding a preset threshold are selected to form a hidden dominant candidate set, including:

[0087] Extracting multi-scale spectral kurtosis sequences covering the entire video segment ,in The frame number;

[0088] Calculate the median of this multi-scale spectral kurtosis sequence. The calculation formula is:

[0089] ;

[0090] Calculate the absolute value of the difference between each multi-scale spectral kurtosis value in the multi-scale spectral kurtosis sequence and the median to obtain the absolute deviation sequence. Calculate the median of this absolute deviation sequence as the median absolute deviation. The calculation formula is:

[0091] ;

[0092] Define a non-zero minimum constant With preset threshold ;

[0093] Subtract the median from each multiscale spectral kurtosis value in the multiscale spectral kurtosis sequence to obtain the numerator; use the sum of the absolute deviation of the median and the non-zero minimum constant as the denominator; calculate the quotient of the numerator divided by the denominator to obtain the standardized outlier. The calculation formula is:

[0094] ;

[0095] Select video frames with a normalized anomaly score greater than the preset threshold to establish a hidden dominant candidate set. , is represented as:

[0096] .

[0097] It should be noted that the multi-scale spectral kurtosis sequence is a chronological sequence of all multi-scale spectral kurtosis values ​​covering the entire video, reflecting the temporal distribution characteristics of the multi-scale spectral kurtosis across the entire video. The absolute deviation sequence is a chronological sequence of the absolute values ​​of the differences between each multi-scale spectral kurtosis value and the median, reflecting the degree of deviation of each frame's multi-scale spectral kurtosis from the baseline level. The normalized anomaly score is the result of robustly normalizing the deviation of the multi-scale spectral kurtosis values ​​from the median, reflecting the degree of anomaly in a single frame's multi-scale spectral kurtosis. The preset threshold is a fixed value used to filter out anomalous frames. The hidden dominance candidate set is a set of video frames with a normalized anomaly score greater than the preset threshold, reflecting the set of event frames suspected of being dominated by erroneous hidden elements.

[0098] It should be noted that surveillance videos often contain extreme interference such as sudden changes in lighting and intense movement. Traditional normalization based on the mean and standard deviation is easily affected by extreme values, leading to distortion in anomaly detection. This invention uses the median as the benchmark level because it is not affected by extreme values ​​at both ends of the sequence. The absolute deviation of the median is used as a fluctuation scale, which also has robustness. The standardized anomaly degree calculated by combining the two can truly reflect the anomaly attributes of a single frame. The specific value range of the preset threshold is 2 to 3, which can be adjusted according to the total number of video frames. When the total number of video frames is greater than 10,000, a value of 3 is used; when the total number of video frames is between 5,000 and 10,000, a value of 2.5 is used; and when the total number of video frames is less than 5,000, a value of 2 is used to ensure a balance between missed detections and false detections in videos of different lengths. The length of the multi-scale spectral kurtosis sequence is defined as covering all frames of the video from the first frame to the last frame, without additional cropping. The frame index corresponds one-to-one with the original frame index of the video. For example, if the video has a total of 8,000 frames, the multi-scale spectral kurtosis sequence contains 8,000 values, corresponding to frames 1 to 8,000 respectively.

[0099] In one embodiment of the present invention, the normalized time gap intensity is calculated by extracting the video presentation timestamp sequence, and a decision index is constructed by combining multi-scale spectral kurtosis. Based on the decision index, classification labels for non-malicious error hiding and malicious frame deletion in the dominant candidate set of hidden elements are determined, including:

[0100] Read and present timestamp sequences from video data ,in For time steps; calculate the median of the difference between two adjacent presentation timestamps in the presentation timestamp sequence to obtain the typical frame interval. The calculation formula is:

[0101] ;

[0102] Calculate the current presentation timestamp Compared to the previous presentation timestamp The difference is divided by the typical frame interval, and then a value of one is subtracted from the result. The larger of the result after subtracting one and the value of zero is selected to obtain the normalized time gap intensity. The calculation formula is:

[0103] ;

[0104] Define a non-zero minimum constant Obtain the multi-scale spectral kurtosis corresponding to the current presentation timestamp. Using the multiscale spectral kurtosis as the numerator, and the sum of the multiscale spectral kurtosis, the normalized time gap intensity, and the non-zero minimum constant as the denominator, the quotient of the numerator divided by the denominator is calculated to generate the decision index. The calculation formula is:

[0105] ;

[0106] Targeting hidden dominant candidate set The video frames included are compared with the value 0.5. If the judgment index is greater than or equal to the value 0.5, it is marked as a non-malicious error hiding. If the judgment index is less than the value 0.5, it is marked as malicious frame deletion.

[0107] It should be noted that the presentation timestamp sequence is a set of presentation timestamps read from the video data and arranged in frame time steps, reflecting the expected display time of each frame. The typical frame interval is the median difference between two adjacent presentation timestamps, reflecting the baseline frame interval during normal video playback. The normalized time gap strength is the excess proportion of the difference between the current and previous presentation timestamps relative to the typical frame interval, reflecting the relative strength of the time gap. The decision index is the ratio of multi-scale spectral kurtosis to the sum of multi-scale spectral kurtosis, normalized time gap strength, and a non-zero minimum constant, reflecting the relative proportion of the two types of features. The classification label is the determination result of the type of video frame tampering; non-malicious error hiding reflects frame loss caused by non-malicious frame dropping, while malicious frame deletion reflects frame loss caused by malicious editing.

[0108] It should be noted that non-malicious error hiding is accompanied by block-based filling structural features, resulting in a larger multi-scale spectral kurtosis value and a relatively smaller timeline gap intensity. The decision index will be biased towards 1, satisfying the condition of being greater than or equal to 0.5. Malicious frame deletion usually lacks structural filling of error hiding, has a smaller multi-scale spectral kurtosis value, and a larger timeline gap intensity due to the direct deletion of the frame. The decision index will be biased towards 0, satisfying the condition of being less than 0.5. For example, if a frame is non-maliciously error-hidden, with a multi-scale spectral kurtosis of 14, a normalized timeline gap intensity of 0.5, and a non-zero minimum constant of 10 to the power of -6, the decision index is 14 ÷ (14 + 0.5 + 10 to the power of -6) ≈ 0.967 ≥ 0.5, and is marked as non-malicious error hiding. If a frame is maliciously deleted, with a multi-scale spectral kurtosis of 1.2, a normalized timeline gap intensity of 3, and a decision index of 1.2 ÷ (1.2 + 3 + 10 to the power of -6) ≈ 0.286 < 0.5, it is marked as maliciously deleted.

[0109] It should be noted that the preferred value for the non-zero minimum constant is 10 to the power of -4. This value is small enough that it will not affect the calculation accuracy of the decision exponent, and at the same time, it can completely avoid the abnormal situation of the denominator being zero. The source of the presentation timestamp sequence is clearly the presentation timestamp field of the video container. For example, MP4 containers are read from the tkhd (track header) or mdhd (media header) atom, and HLS (HTTP live stream) is extracted from the timestamp information corresponding to the EXTINF tag, ensuring the accuracy and consistency of the timestamp data.

[0110] In one embodiment of the present invention, such as Figure 2 As shown, a surveillance video tampering detection system based on feature data feedback includes:

[0111] The normalized luminance frame module 201 decodes the video stream, extracts the luminance component of each frame, calculates the intra-frame mean and standard deviation, and performs mean removal and normalization operations on the luminance component accordingly to obtain a normalized luminance frame.

[0112] The block boundary activation field generation module 202 performs differential calculation on adjacent normalized luminance frames to obtain a temporal derivative field, calculates the absolute value of the difference in the horizontal and vertical dimensions of the temporal derivative field respectively, and superimposes them to generate a block boundary activation field.

[0113] The multi-scale grid energy fraction generation module 203 transforms the block boundary activation field to the two-dimensional frequency domain, extracts the grid harmonic energy ratio of the corresponding frequency band according to the preset coding block scale, and generates the multi-scale grid energy fraction corresponding to each block scale accordingly.

[0114] The multi-scale spectral kurtosis calculation module 204 calculates the distribution kurtosis value of the multi-scale raster energy fraction, and multiplies the distribution kurtosis value by the maximum value in the multi-scale raster energy fraction to obtain the multi-scale spectral kurtosis.

[0115] The hidden dominant candidate set screening module 205 calculates the median and median absolute deviation of the multi-scale spectral kurtosis of the entire sequence, and then calculates the standardized outlier based on this. Video frames with outliers exceeding a preset threshold are selected to form the hidden dominant candidate set.

[0116] The video tampering judgment module 206 extracts the video presentation timestamp sequence, calculates the normalized time gap intensity, constructs a judgment index by combining multi-scale spectral kurtosis, and obtains the classification labels of non-malicious error hiding and malicious frame deletion in the hidden dominant candidate set based on the judgment index.

[0117] It should be noted that the interval and threshold sizes are set for ease of comparison. The size of the threshold depends on the amount of sample data and the base number set by those skilled in the art for each set of sample data, as long as it does not affect the proportional relationship between the parameter and the quantized value. Furthermore, the above formulas are all dimensionless calculations, and the formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0118] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A method for detecting tampering in surveillance videos based on feature data feedback, characterized in that, Includes the following steps: Step S101: Decode the video stream, extract the luminance component of each frame, calculate the intra-frame mean and standard deviation, and perform mean removal and normalization operations on the luminance component accordingly to obtain a normalized luminance frame. Step S102: Perform differential calculation on adjacent normalized luminance frames to obtain the temporal derivative field, calculate the absolute value of the difference in the horizontal and vertical dimensions of the temporal derivative field respectively and superimpose them to generate the block boundary activation field. Step S103: Transform the block boundary activation field to the two-dimensional frequency domain, extract the grid harmonic energy ratio of the corresponding frequency band according to the preset coding block scale, and generate the multi-scale grid energy fraction corresponding to each block scale accordingly. Step S104: Calculate the distribution kurtosis value of the multi-scale raster energy fraction, and multiply the distribution kurtosis value by the maximum value in the multi-scale raster energy fraction to obtain the multi-scale spectral kurtosis. Step S105: Calculate the median and median absolute deviation of the multi-scale spectral kurtosis of the entire sequence, and calculate the standardized anomaly based on this. Select video frames with anomalies exceeding a preset threshold to form a hidden dominant candidate set. Step S106: Extract the video presentation timestamp sequence, calculate the normalized time gap intensity, construct a decision index by combining multi-scale spectral kurtosis, and obtain the classification labels of non-malicious error hiding and malicious frame deletion in the hidden dominant candidate set based on the decision index. Step S103 specifically includes: A two-dimensional discrete Fourier transform is performed on the block boundary activation field to obtain the two-dimensional frequency domain amplitude; a set of coding block scales including values ​​of eight, sixteen, thirty-two and sixty-four is set. For any coding block scale in the coding block scale set, calculate the quotient of frame width to coding block scale and the quotient of frame height to coding block scale respectively. Multiply these two quotients by the first, second and third harmonic orders respectively and round them to the nearest integer to obtain the horizontal frequency point set and the vertical frequency point set. The frequency domain coordinates belonging to the set of frequency points in the horizontal direction or the set of frequency points in the vertical direction are determined as the corresponding frequency bands; Set a non-zero minimum constant, calculate the sum of squares of the amplitude of the two-dimensional frequency domain within the corresponding frequency band, and use it as the numerator. Calculate the sum of squares of the amplitude of the two-dimensional frequency domain within the entire frequency domain and the non-zero minimum constant, and use it as the denominator. Divide the numerator by the denominator to obtain the grid harmonic energy ratio. Combine the grid harmonic energy ratios corresponding to each coding block scale to generate a multi-scale grid energy fraction. Step S106 specifically includes: Read the presentation timestamp sequence from the video data, calculate the median of the difference between two adjacent presentation timestamps in the presentation timestamp sequence, and obtain the typical frame interval; Calculate the difference between the current presentation timestamp and the previous presentation timestamp, divide the difference by the typical frame interval, subtract one from the result, and select the larger value between the result after subtracting one and zero to obtain the normalized time gap intensity. Set a non-zero minimum constant, obtain the multi-scale spectral kurtosis corresponding to the current presentation timestamp, use the multi-scale spectral kurtosis as the numerator, and use the sum of the multi-scale spectral kurtosis, the normalized time gap intensity, and the non-zero minimum constant as the denominator. Calculate the quotient of the numerator divided by the denominator to generate the decision index. For video frames contained in the hidden dominant candidate set, if the decision index is greater than or equal to 0.5, they are marked as non-malicious erroneous hiding; if the decision index is less than 0.5, they are marked as malicious frame deletion.

2. The method for detecting tampering in surveillance videos based on feature data feedback according to claim 1, characterized in that, Decode the video stream frame by frame to extract the brightness values ​​of each pixel in a single frame image; The brightness values ​​of each pixel in a single frame are summed and divided by the total number of pixels to obtain the intra-frame mean. The standard deviation of the frame is obtained by summing the squares of the differences between the brightness values ​​of each pixel in the single frame and the mean value within the frame, dividing the sum by the total number of pixels and taking the square root. A non-zero minimum constant is set, and the brightness value of each pixel in the single frame image is subtracted from the mean value within the frame, and then divided by the sum of the standard deviation within the frame and the non-zero minimum constant to generate a normalized brightness frame.

3. The method for detecting tampering in surveillance videos based on feature data feedback according to claim 1, characterized in that, Extract the current normalized luminance frame and the previous normalized luminance frame, and subtract the value of the pixel at the same coordinate position in the previous normalized luminance frame from the value of each pixel in the current normalized luminance frame to obtain the time derivative field. Calculate the absolute value of the difference between the value of each pixel in the time derivative field and the value of its horizontally adjacent pixel to obtain the absolute value of the horizontal difference; Calculate the absolute value of the difference between the value of each pixel in the time derivative field and the value of its vertically adjacent pixel to obtain the absolute value of the vertical difference; The absolute value of the horizontal difference is added to the absolute value of the vertical difference to generate the block boundary activation field.

4. The method for detecting tampering in surveillance videos based on feature data feedback according to claim 1, characterized in that, The average value is obtained by summing the proportion of harmonic energy of each grid in the multi-scale grid energy fraction, dividing the summed result by the number of elements in the coding block scale set; Calculate the square of the difference between the harmonic energy percentage of each grid and the mean, sum all the squared results and divide by the number of elements to obtain the second-order central moment; Calculate the fourth power of the difference between the harmonic energy percentage of each grid and the mean value, sum all the fourth power results and divide by the number of elements to obtain the fourth central moment.

5. The method for detecting tampering in surveillance videos based on feature data feedback according to claim 4, characterized in that, Define a non-zero minimum constant, take the fourth central moment as the numerator, and take the sum of the square of the second central moment and the non-zero minimum constant as the denominator. Calculate the quotient of the numerator divided by the denominator to obtain the distribution kurtosis value. The grid harmonic energy percentage with the largest value among the multi-scale grid energy fractions is selected as the maximum fraction. Calculate the product of the distribution kurtosis value and the maximum fraction to generate the multi-scale spectral kurtosis.

6. The method for detecting tampering in surveillance videos based on feature data feedback according to claim 1, characterized in that, Extract the multi-scale spectral kurtosis sequence covering the entire video segment; calculate the median of the multi-scale spectral kurtosis sequence; calculate the absolute value of the difference between each multi-scale spectral kurtosis value in the multi-scale spectral kurtosis sequence and the median to obtain the absolute deviation sequence; Calculate the median of this absolute deviation sequence, and use it as the median absolute deviation; Set a non-zero minimum constant and a preset threshold; Subtract the median from each multiscale spectral kurtosis value in the multiscale spectral kurtosis sequence to obtain the numerator; The sum of the absolute deviation of the median and the non-zero minimum constant is used as the denominator; the quotient of the numerator divided by the denominator is calculated to obtain the standardized outlier; video frames with a standardized outlier greater than the preset threshold are selected to establish a hidden dominant candidate set.

7. A surveillance video tampering detection system based on feature data feedback, characterized in that, Performing a surveillance video tampering detection method based on feature data feedback as described in any one of claims 1 to 6, comprising: The normalized luminance frame module decodes the video stream, extracts the luminance component of each frame, calculates the intra-frame mean and standard deviation, and performs mean removal and normalization operations on the luminance components accordingly to obtain a normalized luminance frame. The block boundary activation field generation module performs differential calculations on adjacent normalized luminance frames to obtain a temporal derivative field. It then calculates the absolute values ​​of the differences in the temporal derivative field in the horizontal and vertical dimensions and superimposes them to generate the block boundary activation field. The multi-scale grid energy fraction generation module transforms the block boundary activation field to the two-dimensional frequency domain, extracts the grid harmonic energy ratio of the corresponding frequency band according to the preset coding block scale, and generates the multi-scale grid energy fraction corresponding to each block scale accordingly. The multi-scale spectral kurtosis calculation module calculates the distribution kurtosis value of the multi-scale raster energy fraction, and multiplies the distribution kurtosis value by the maximum value in the multi-scale raster energy fraction to obtain the multi-scale spectral kurtosis. The hidden dominant candidate set screening module calculates the median and median absolute deviation of the multi-scale spectral kurtosis of the entire sequence, and then calculates the standardized outlier based on this. Video frames with an outlier exceeding a preset threshold are selected to form the hidden dominant candidate set. The video tampering judgment module extracts the video presentation timestamp sequence to calculate the normalized time gap intensity, and constructs a judgment index by combining multi-scale spectral kurtosis. Based on the judgment index, it obtains the classification labels of non-malicious error hiding and malicious frame deletion in the hidden dominant candidate set.

Images