A video surveillance object forgery detection system

Abstract

The present invention discloses a video surveillance object forgery detection system, comprising a sequentially connected spatiotemporal feature extraction module, an attention-based CNN module, and an attention-based decoder RNN module. The present invention utilizes TSF features to reduce computational costs and data redundancy, thereby achieving more efficient CNN training; the present invention's embedded self-attention mechanism processes important information between network layers; the present invention uses an encoder-attention-decoder RNN model for feature classification, which focuses on time series characteristics, such as the differences and correlations between real frames and real frames, forged frames and real frames, and forged frames and forged frames, and can mine and learn high-discrimination information to better identify forged frames.

Description

Technical Field

[0001] The invention relates to the field of video surveillance security, and in particular to a video surveillance object forgery detection system. Background Art

[0002] The large-scale deployment of video surveillance equipment plays an important role in social security. What is worrying is that video surveillance resources have become a potential target for high-tech criminals. If these high-tech criminals illegally tamper with the content of surveillance videos, the tampered video content may affect judicial justice and even threaten social stability. Therefore, it is urgent to determine the authenticity of video surveillance sources.

[0003] The video surveillance object forgery problem is similar to the video copy-paste forgery, which is tampered by copying and pasting an object from one frame to another frame in the same video. However, this type of forgery cannot be simply classified as video copy-paste forgery. For example, it is known that surveillance videos are composed of at least 25 consecutive frames per second. Even if the copied source frame and the pasted target frame are only one minute apart in the time series, they are at least 1500 frames apart. In actual video surveillance forgery, the tamperer will make a longer interval between the copied frame and the pasted frame to better cover up the traces of forgery. However, video copy-paste forgery detection algorithms usually select short video clips with relatively few frames, rather than the entire video of infinite length for video forensics. Therefore, in the short video detection clip, the copied frame and the pasted frame usually do not appear at the same time with high probability. Therefore, video surveillance forgery is usually implemented in longer videos, which makes the existing video copy-paste forgery detection methods invalid. Because the existing video copy-paste forgery detection methods need to find the copied frame and the pasted frame at the same time for matching calculation.

[0004] The actual video object surveillance forgery has the characteristics of video copy-paste forgery, but it cannot be detected using the method, which brings about a new topic, namely video surveillance (object) forgery detection (VSOFD). The existing video surveillance object forgery detection has the following shortcomings:

[0005] (1) Lack of specific combined features to deeply mine video attributes. Existing technologies focus on achieving feature extraction in specific fields, such as time domain, spatial domain, and spatiotemporal domain. However, it is well known that video is a combination of time and space domains, where a video frame is a time series with spatial interdependence. In addition, video frames are also composed of energy spectra. The insertion or removal of forged objects will greatly change the frame energy distribution, that is, change the frequency. However, few technologies consider the specific combined time-spacial-frequency domain (TSF) attributes for feature extraction;

[0006] (2) There is a lack of effective CNN networks for feature processing. The features extracted from video frames are in the temporal, spatial and frequency domains, and a network suitable for extracting such features needs to be designed to extract effective video frame features.

[0007] (3) The RNN architecture lacks an embedded attention mechanism and cannot deeply mine frame correlations in video time series. Generally speaking, CNNs with spatial attention can process spatial domain features but cannot process continuous temporal features. Recurrent neural networks (RNNs) have cyclic features and can process time series, but lack attention mechanisms and deep networks to handle complex redundant information. Currently, the new Transformer architecture combines the features of CNN and RNN (including embedded attention mechanisms); however, it loses sequence position information and fails to capture local temporal features (such as correlation information between frames). In addition, the massive amount of video data leads to inefficient feature extraction and subsequent processing. Summary of the invention

[0008] In order to overcome the shortcomings of the prior art, the present invention proposes a video surveillance object forgery detection system. The technical solution of the present invention is implemented as follows: a video surveillance object forgery detection system comprises a spatiotemporal frequency feature extraction module, an attention-based CNN module and an attention-based decoder RNN module connected in sequence, wherein:

[0009] The time-space frequency feature extraction module extracts the time-space frequency features of a specific combination in three steps: time feature extraction, time-space feature extraction and time-space frequency feature extraction;

[0010] The attention-based CNN module adopts the convolutional neural network CondenseNet as the backbone network, introduces self-learning grouped convolution to combine with full dense connection and increase reuse parameters, and introduces a self-attention model embedded between each BN-Relu-Conv block and pooling layer of the backbone network;

[0011] The attention-based decoder RNN module uses a bidirectional LSTM embedded with an Attention mechanism to analyze the correlation between the current frame and the previous and next frames before passing information to the linear layer, and uses a cross-entropy loss function to reversely adjust the parameters of the encoder-attention-decoder model during the training of the decoder network.

[0012] Furthermore, the time-space-frequency feature extraction module extracts the time feature of a specific combination, including the following steps:

[0013] The video clip is divided into independent frames, and then the foreground-background subtraction method is used to obtain the residual of the motion area of ​​the video frame. For the feature extraction of the residual, the weight allocation strategy followed is: use a multi-order residual kernel to model the correlation between the central frame and adjacent frames in the time direction. If the distances between the previous and next frames and the central frame i are equal, their corresponding weights are equivalent, and the weight of the central frame is equal to the sum of the weights of all frames.

[0014] Furthermore, the weight calculation formula of the center frame is:

[0015]

[0016] in represents the rounding down operation, || represents the absolute value operation, λ j is the weight of the adjacent frame j in the time direction.

[0017] Furthermore, the spatiotemporal feature extraction module extracts spatiotemporal features of a specific combination, including the following steps: defining a Gaussian kernel Gs with a size of (2k+1)×(2k+1) x,y It is used to implement filtering on each time residual map to obtain the corresponding space-time residual map. The calculation formula is:

[0018]

[0019] Where σ is the standard deviation, (2k+1)×(2k+1) represents the kernel size of the Gaussian filter, Gs x,y is the weight of the Gaussian kernel position (x,y), where 1≤x,y≤(2k+1).

[0020] Furthermore, the spatiotemporal frequency feature extraction module extracts the spatiotemporal frequency features of a specific combination by introducing the feature extraction method DCT-216 in image steganalysis, extracting TSF features from the spatiotemporal residual feature map, that is, extracting TSF features from the two-dimensional residual map Extract the one-dimensional TSF (feature) vector

[0021] Furthermore, the backbone network CondenseNet of the attention-based CNN module inputs a two-dimensional time series feature matrix into the CNN module. Here, f = 216 dimensions, t = 2Ln+1 = 19, the last FC-1000 and Soft-Max layers are removed, and five BN-Relu-Conv-MaxPool blocks and two FC-4096 layers are retained as the baseline backbone.

[0022] Furthermore, the backbone network CondenseNet of the attention-based CNN module embeds a self-attention mechanism, and the self-attention formula is in m=1, 2, 3, 4 are the scaling factors of the four corresponding BN-Relu-Conv blocks.

[0023] Furthermore, embedding the Attention mechanism in the decoder includes the following steps:

[0024] S1, a feature matrix output by the backbone network Input an attention-based encoder-decoder RNN module, where the size of l is 4096, t = 2Ln + 1 represents the number of frames, and the bidirectional LSTM converts the forward and backward output states and Connect them together to obtain the continuous encoding of the previous frame, the current frame i and the next frame;

[0025] S2, using the Attention mechanism to input Calculate the important content, for a query vector q j , through a scoring function to calculate the query vector q j And for each input y e i The correlation between them is used to obtain a score, and the score is normalized using softmax. The normalized result is the query vector q j In each input y e i The attention distribution on a=[a 1 ,a 2 ,a 3 ,……,a 2Ln+1 ], where each value and the original input One-to-one correspondence, a i The calculation formula is:

[0026]

[0027] in, The parameters W, U, and v in the formula are learnable parameter matrices or vectors;

[0028] S3, weighted sum of input information according to attention distribution, the final result c j This reflects what the model is currently focusing on:

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] (1) The present invention extracts and analyzes specific combined features of video frames, decomposes the video sequence into a series of video frames, extracts motion residual features from the video frames, and converts the two-dimensional residual information matrix of each video frame into Convert to a one-dimensional TSF vector The TSF vector is the TSF feature of the corresponding frame. By using dimensionality reduction technology, the two-dimensional time-spacial-frequency matrix (TSFM) Able to accurately represent continuous frame information in the video The use of TSF features can also help reduce computational costs and data redundancy, thereby achieving more efficient CNN training.

[0031] (2) The present invention uses TSF features to process an effective CNN network backbone model. In the feature processing stage, the input TSFM is regarded as a special image without considering its time series. The proposed network model regards each column (TSF vector) of this special image as an independent unit and processes it independently. In addition, the embedded self-attention mechanism can process important information between network layers;

[0032] (3) The present invention uses an encoder-attention-decoder RNN model for feature classification, which focuses on time series characteristics, such as the differences and correlations between real frames and real frames, forged frames and real frames, and forged frames and forged frames, and can mine and learn high-discrimination information to better identify forged frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a structural schematic diagram of a video surveillance object forgery detection system of the present invention;

[0034] Figure 2 It is a work flow chart of a spatiotemporal frequency feature extraction module in a video surveillance object forgery detection system of the present invention;

[0035] Figure 3 It is a flow chart of the method for extracting the time-space frequency features of a specific combination in the present invention;

[0036] Figure 4 Schematic diagram of the encoder-attention-decoder RNN module structure in the present invention;

[0037] Figure 5 It is a schematic diagram of the bidirectional LSTM unit structure in the present invention;

[0038] Figure 6 It is a schematic diagram of the Encode structure in the decoder RNN module based on the attention mechanism in the present invention;

[0039] Figure 7 It is a schematic diagram of the structure of the Attention module in the decoder RNN module based on the attention mechanism in the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] See also Figure 1 The present invention discloses a video surveillance object forgery detection system, comprising a spatiotemporal-frequency feature extraction module, an attention-based CNN module and an attention-based decoder RNN module connected in sequence.

[0042] The work flow of the spatiotemporal frequency feature extraction module of the video surveillance object forgery detection system of the present invention is as follows: Figure 2 As shown in the figure, the input of the spatiotemporal-frequency feature extraction module is the video clip to be detected, and the output feature matrix is ​​used as the input of the attention-based CNN module.

[0043] Please see attached Figure 3 ,The time-space-frequency feature extraction module extracts the time-space-frequency features of a specific ,combination in three steps: (1) time feature extraction, (2) time-space feature extraction, (3) time-space-frequency feature extraction.

[0044] (1) Temporal feature extraction

[0045] First, the present invention divides the video clip into independent frames, and then uses the foreground-background subtraction method to obtain the residual of the motion area of ​​the video frame, see formula (1):

[0046]

[0047] RF i =|F i (x,y)-CF(x,y)| (1-b)

[0048] Where Fi(x,y) represents the content of the i-th frame, while x and y are the pixel coordinates of the i-th frame, and Col represents the frame combination from the i-Lp-th frame to the i+Lp-th frame in a time window centered on the i-th frame (Lp is the number of left / right neighboring frames of the i-th frame). In addition, CF(x,y) is the median of the pixel values ​​corresponding to the pixel coordinates (x,y) in this frame combination. For example, the median of the pixel values ​​with coordinates (x=2, y=2) is searched in all frames of the frame combination to obtain CF(2,2). Finally, RF i is the center frame F i The absolute value of the difference with CF. Extracting the residual from a continuous sequence of frames (frame groups) instead of using isolated frames can better identify the difference between forged frames and real frames.

[0049] Based on the residual feature extraction algorithm, this paper proposes a set of reasonable weight allocation strategies to obtain better residual maps from the frame group:

[0050] Strategy (i): Use multi-order residual kernels to model the correlation between the central frame and adjacent frames in the temporal direction, because the farther the distance between adjacent frames and the central frame, the weaker the correlation (residual weight) between them, and vice versa.

[0051] Strategy (ii): If the distances between the previous and next frames and the center frame i (center of the frame group) are equal, then their corresponding weights are equivalent.

[0052] Strategy (iii): The weight of the center frame is equal to the sum of the weights of all frames.

[0053] Based on strategy (i) and (ii), the present invention uses standard normal distribution to determine the number of frames and weight distribution. The probability density function of normal distribution is defined as follows:

[0054]

[0055] When mean μ = 0 and variance σ = 1, formula (2) represents the standard normal distribution. However, this distribution cannot be directly used to define the weight of each frame in the frame group because it does not satisfy the principle strategy (iii). In order to satisfy the three strategies at the same time, a normal distribution with mean μ = 0 and variance σ = 0.8 is suitable for determining the multi-order residual kernel (weight) of the frames in the frame group. Because the weight is close to zero when 2Lp+1 = 9, that is, the number of neighboring frames Lp = 4, a relatively small number of consecutive frames 2Lp+1 = 3, 5 or 7 are suitable for constructing a frame group, setting Lp = 7. Formula (3) is used to calculate the weight distribution.

[0056]

[0057] in represents the rounding down operation, || represents the absolute value operation, λj is the weight of the adjacent frame j in the time direction.

[0058] (2) Spatiotemporal feature extraction

[0059] The weight (coefficient) of the Gaussian filter can better reflect the correlation strength between the target pixel and its neighboring pixels, rather than using the mean and median filters. At the same time, the Gaussian convolution kernel can be used as an effective filter to remove noise interference. The present invention defines a Gaussian kernel Gs with a size of (2k+1)×(2k+1) x,y It is used to implement filtering on each time residual map to obtain the corresponding space-time residual map. As shown in formula (4):

[0060]

[0061] Where σ is the standard deviation, (2k+1)×(2k+1) represents the kernel size of the Gaussian filter, for example, k=1,3, Gs x,y is the weight of the Gaussian kernel position (x, y), where 1≤x, y≤(2k+1). To determine the value of σ, we refer to the scale-invariant feature transform (SIFT) basic construction method. In addition, the value of σ is set to 1.4, referring to the normalized Gaussian convolution kernel in the SIFT method, with a size of 3×3, as shown in formula (5).

[0062]

[0063] (3) Temporal, spatial and frequency feature extraction

[0064] The spatiotemporal feature map is the same size as the video frame and also contains a large amount of information, which brings huge information volume and information complexity problems to the subsequent deep neural network (DNN) processing. For example, a video frame with a size of 320×320 pixels needs to process 102,000 pixels, which can easily lead to low efficiency. Secondly, its huge amount of information and lack of attention to specific information may cause the discrimination details to be ignored. In order to solve these problems, the present invention introduces the feature extraction method DCT-216 (216 is the DCT feature dimension) in image steganalysis, which can extract TSF features from the spatiotemporal residual feature map, that is, from the two-dimensional residual map Extract the one-dimensional TSF (feature) vector In addition, it can further focus on the coherence and differences of key features between adjacent frames. This special combination of time, space and frequency features is more suitable for efficient and effective VSOFD feature extraction and subsequent required processing.

[0065] The attention-based CNN module in the present invention is used to process the input TSF features of VSOFD. In order to effectively extract deep features, three main factors need to be considered when constructing an attention-based convolutional neural network: (1) the structural characteristics of the TSF of the input frame group; (2) the structural characteristics of the CNN; and (3) the embedded attention mechanism.

[0066] Through the above-mentioned steganalysis feature extraction, each video frame will be extracted from the two-dimensional matrix Compress to one-dimensional TSF (feature) vector Where x and y represent the pixel coordinates, and f represents the TSF vector. Therefore, the content of consecutive frames can be obtained from the three-dimensional matrix Compress to 2D TSF matrix Where t represents the number of consecutive frames. The TSF vector i is composed of Ln frame TSF vectors on both sides to form a time-space frequency matrix TSFM i , where TSF vector i is the TSFM i = the central column of {i-Ln,…,i-1,i,i+1,…,i+Ln}, t = 2Ln+1. In addition, TSFM i Should contain the components of TSF i The corresponding frame, that is, Ln ≥ Lp (the construction details of the TSF vector are described in detail in the previous (I)), where Lp = 7, so Ln = 11. The input TSFM is a two-dimensional time series, which can be regarded as an image with time series characteristics, so some natural language processing (NLP) or image processing models can be used as a reference for VSOFD.

[0067] Based on the specific TSF structural characteristics that the attention-based CNN module backbone neural network design relies on, we propose two design principles:

[0068] Principle (a), using column convolution (k row ,k column )=(k r ,k c )=(k,1) to maintain the independence of each TSF vector (frame information) in the entire CNN feature processing;

[0069] Principle (b), Atrous convolution with a large dilation rate can expand the receptive field to include more information without downsampling.

[0070] The larger the convolution kernel, the more dense information is concerned, and the higher the computational cost. For the dilation rate of Atrous convolution, the larger the dilation rate, the more global information is concerned, but at the same time the less local information is concerned, and the computational cost is reduced. The present invention chooses to use the lowest dimensional DCT-216 feature as the basis. The present invention uses a small column Atrous convolution kernel (kh ×k v ,r) to process TSF features based on DCT-216, where (k h =k r =3)×(k v =k c =1) is a small convolution kernel, and r = 0 is the dilation rate. In addition, considering that the Atrous convolution with dilation rate r can expand the receptive field, the step coefficient s in each pooling layer is associated with r, where s = 2r.

[0071] The present invention adopts a convolutional neural network CondenseNet as the backbone network. CondenseNet introduces a self-learning group convolution, combined with full dense connection and increased reuse parameters. The training of CondenseNet includes two stages: concentration and optimization.

[0072] The concentration phase uses filter pruning techniques to remove less important dense connections, while the optimization phase mainly trains sparse models. The specific implementation is as follows: Input a two-dimensional time series feature matrix into the CNN module Here, f = 216 (dimensions), t = 2Ln+1 = 19, the last FC-1000 and Soft-Max layers are removed, and five BN-Relu-Conv-MaxPool blocks and two FC-4096 layers are retained as the baseline backbone.

[0073] Therefore, the five blocks in the network model, including five pooling layers, mean that the length of the TSF vector will be reduced by five times. The growth rate g means that the current layer is (1+g) deeper than the previous layer. When g=0.5, the output network depth will be condensed to half of its input value. At the same time, in each block, the network model follows two principles (a) and (b) for feature processing.

[0074] Table 1 details the CONDENSENET network model, including the dilation rate r of the convolutional layer, the step coefficient s of the pooling layer, and the network depth d.

[0075] Table 1 Network parameters of CONDENSENET processing TSF characteristics of DCT-216

[0076]

[0077] The present invention introduces a self-attention model embedded between each BN-Relu-Conv block and pooling layer of the backbone network. In view of the characteristics of TSFM, that is, the independence of each column, this self-attention does not consider the temporal characteristics. In addition, it is meaningless to use self-attention alone to process spatial features. Therefore, the embedded self-attention mechanism only focuses on the network layer (depth). Self-attention is shown in formula (6). The self-attention module uses W Q ,W K and W V matrices to get the query (q), key (k), and value (v).

[0078]

[0079] in, m = 1, 2, 3, 4 are the scaling factors of the four corresponding BN-Relu-Conv blocks. More specifically, ω m,k is the processed feature map ω m,t (Size is ) is the kth processed feature vector of the self-attention input, with a size of In this way, each independent feature column is regarded as a feature of a word in NLP. m As a scaling factor, it is used to prevent the accumulation of values ​​from becoming too large, thereby keeping the variance in the range of 0 to 1, where s m Depends on the network depth d. For example, referring to Table 1, when CondenseNet processes the TSF features of DCT-216 in the first block, the network depth d is 16, which means s m The value is also 16. Each s m are all different. Similarly, W Qm ,W Km and W Vm Finally, self-attention learns important features to obtain m,t (Size is ) attention masks of the same size.

[0080] like Figure 4 As shown, the present invention proposes an attention-based decoder RNN module.

[0081] Before passing the information to the linear layer, an LSTM model can be utilized to analyze frame correlations, while the attention mechanism embedded in the encoder-decoder architecture can better capture global and local connections.

[0082] First, a feature matrix output by the backbone network Input an attention-based encoder-decoder RNN module, where the size of l is 4096 and t = 2Ln+1 represents the number of frames. Bidirectional LSTM (Bi-LSTM) Figure 5 As shown, The forward and backward output states and Connect them as shown in formula (7), so as to obtain the encoding of the previous frame, current frame i and next frame as follows: Figure 6 As shown in

[0083]

[0084] This invention uses the Attention mechanism to input To calculate the important content, first for a query vector q j , through a scoring function to calculate the query vector q j The correlation between each input hi is used to obtain a score score. Then, these scores are normalized using softmax, and the normalized result is the query vector q j In each input y e i The attention distribution on a=[a 1 ,a 2 ,a 3 ,……,a 2Ln+1 ], where each value and the original input One to one correspondence. i The calculation formulas (8) and (9) are as follows:

[0085]

[0086] in,

[0087]

[0088] The parameters W, U, and v in the above formula are all learnable parameter matrices or vectors.

[0089] Finally, the input information is weighted and summed according to these attention distributions, and information can be selectively extracted from the input. The final result c j (Formula (10)) reflects the current focus of the model, such as Figure 6 As shown in:

[0090]

[0091] In the decoder network, the present invention uses forward LSTM to convert q k-1 ,c k-1 , and y kCombined, q k-1 is the hidden layer output of the previous LSTM, c k-1 is the output of the Attention module, is the true label of the current frame, such as Figure 7 shown.

[0092] Each c k and q k Together with the 512×1 vector and the corresponding true label (GT), we input it into the forward LSTM network. Finally, we use the fully connected layer as the classification layer to further obtain the output y of the (2Ln+1)×2 vector. t .

[0093] During training, the cross entropy loss function is used to reverse the parameters of the attention-based decoder RNN model. After training, y t to identify real and fake frames.

[0094] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A video surveillance object forgery detection system, characterized in that: It includes a sequentially connected spatiotemporal feature extraction module, an attention-based CNN module, and an attention-based decoder RNN module, wherein: The time-space frequency feature extraction module extracts the time-space frequency features of a specific combination in three steps: time feature extraction, time-space feature extraction and time-space frequency feature extraction; The temporal feature extraction module for the specific combination of temporal features includes the following steps: dividing the video clip into independent frames, and then using the foreground-background subtraction method to obtain the residual of the motion region of the video frame. For the feature extraction of the residual, the weight allocation strategy followed is: using a multi-order residual kernel to model the correlation between the central frame and the adjacent frames in the time direction. If the distances between the previous and next frames and the central frame i are equal, their corresponding weights are equivalent, and the weight of the central frame is equal to the sum of the weights of all frames. The weight calculation formula of the central frame is: in represents the rounding down operation, || represents the absolute value operation, λ j is the weight of the adjacent frame j in the time direction; The spatiotemporal feature extraction module extracts spatiotemporal features of a specific combination by: defining a Gaussian kernel Gs with a size of (2k+1)×(2k+1) x,y It is used to implement filtering on each time residual map to obtain the corresponding space-time residual map. The calculation formula is: Where σ is the standard deviation, (2k+1)×(2k+1) represents the kernel size of the Gaussian filter, Gs x,y is the weight of the Gaussian kernel position (x, y), where 1≤x, y≤(2k+1); the spatiotemporal frequency feature extraction module further comprises the steps of extracting the spatiotemporal frequency features of a specific combination by introducing the feature extraction method DCT-216 in image steganalysis, extracting TSF features from the spatiotemporal residual feature map, that is, extracting the TSF features from the two-dimensional residual map Extract the one-dimensional TSF feature vector Embedding the Attention mechanism in the decoder includes the following steps: S1, a feature matrix output by the backbone network Input an attention-based encoder-decoder RNN module, where the size of l is 4096, t = 2Ln + 1 represents the number of frames, and the bidirectional LSTM (Bi-LSTM) converts the forward and backward output states and Connect to form y e i , the calculation formula is: Thus, the encoding of the previous frame, the current i frame and the next frame is obtained continuously; S2, using the Attention mechanism to input Calculate the important content, for a query vector q j , through a scoring function to calculate the query vector q j And for each input y e i The correlation between them is used to obtain a score, and the score is normalized using softmax. The normalized result is the query vector q j In each input y e i The attention distribution on a=[a1,a2,a3,……,a 2Ln+1 ], where each value and the original input One-to-one correspondence, a i The calculation formula is: in, The parameters W, U, and v in the formula are learnable parameter matrices or vectors; S3, weighted sum of input information according to attention distribution, the final result c j This reflects what the model is currently focusing on: The attention-based CNN module adopts the convolutional neural network CondenseNet as the backbone network, introduces self-learning group convolution, combines it with full dense connection and increases reuse parameters, and introduces a self-attention model embedded between each BN-Relu-Conv block and pooling layer of the backbone network; The attention-based decoder RNN module uses a bidirectional LSTM embedded with an Attention mechanism to analyze the correlation between the current frame and the previous and next frames before passing information to the linear layer, and uses a cross-entropy loss function to reversely adjust the parameters of the attention-based decoder RNN model during the training of the decoder network.

2. The video surveillance object forgery detection system according to claim 1, characterized in that: The backbone network CondenseNet of the attention-based CNN module inputs a two-dimensional time series feature matrix into the CNN module Among them, f = 216 dimensions, t = 2Ln+1 = 19, the last FC-1000 and Soft-Max layers of CondenseNet are removed, and five BN-Relu-Conv-MaxPool blocks and two FC-4096 layers are retained as the baseline backbone.

3. The video surveillance object forgery detection system according to claim 1, characterized in that: The backbone network CondenseNet of the attention-based CNN module is embedded with a self-attention mechanism, and the self-attention formula is: in are the scaling factors of the four corresponding BN-Relu-Conv blocks.

Images