A video anomaly detection method based on foreground information enhancement

Abstract

The invention discloses a video anomaly detection method based on foreground information enhancement, comprising: preprocessing a video frame to segment the background and the foreground; building a generative network and a discriminant network model; inputting the preprocessed video into the generative network to generate a predicted video frame for the next frame, constructing a loss function of the generative network, adding RGB three-color intensity constraints and gradient constraints into the loss function to ensure the continuity of the video frame in space, adding motion constraints to ensure the continuity of the video frame in time, and adding adversarial constraints to confront the discriminant network; constructing the loss function of the discriminant network, using a discriminator to distinguish between the predicted video frame and the corresponding actual video frame; training the generative network and the discriminant network with multiple groups of normal behavior video frames in a training set; inputting the video frame data to be judged into the trained generative network and the discriminant network, and obtaining a result based on an adaptive peak signal-to-noise ratio to determine whether abnormal behavior occurs.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, to deep learning and video anomaly detection technology, and in particular to a video anomaly detection method based on foreground information enhancement. Background Art

[0002] With the widespread deployment of video acquisition networking devices, the volume of videos and images has grown rapidly, and long and short video data has accounted for the majority of Internet traffic. Therefore, how to efficiently and quickly analyze video data, extract useful information, and assist artificial intelligence decision-making is a major issue that needs to be solved in the current computer vision field.

[0003] As an important field of computer vision application, the security field relies on fast and reliable video data analysis performance. In particular, in surveillance videos, the most common requirement is to automatically identify abnormal events in video streams, which is the task of video anomaly detection. Among them, the technology combining computer vision with deep learning is an effective way to solve various complex video image processing problems. Computer vision is a simulation of biological vision using computers and related equipment. It is an important part of the field of artificial intelligence. Its main task is to obtain information about the corresponding scene by processing the collected pictures or video data.

[0004] In traditional computer vision systems, the main goal is to extract features from images, including subtasks such as edge detection, corner detection, and color-based segmentation. Traditional computer vision systems need to predetermine image features and use some feature extraction algorithms (such as scale-invariant feature transform matching algorithms, accelerated robust feature algorithms, etc.). The improvement of detection algorithm performance can be solved by manual fine-tuning, and it needs to be hard-coded for specific applications, which poses a great obstacle to the realization of high-quality computer vision. Computer vision systems based on deep learning are neural networks trained within deep learning systems, which can avoid searching for specific features through complex programming algorithms. With the increase in computer computing power, computer vision based on deep learning is expected to automatically train and improve models to improve video data analysis performance.

[0005] The current video anomaly detection models based on deep learning mainly include video anomaly detection models based on future frame prediction, probability-based video anomaly detection models, distance-based video anomaly detection models, reconstruction-based video anomaly detection models, domain-based video anomaly detection, etc. However, abnormal events are usually unpredictable. The current video anomaly detection method does not consider separating the foreground and background during data preprocessing, so that the background information will also enter the deep learning neural network, wasting some resources on studying the extraction of background features, while the background feature data has limited effect in video anomaly detection. Secondly, the detection of abnormal events relies on judging the difference between the reconstructed image and the actual image. Since most algorithms compare the peak signal-to-noise ratio (PSNR) of the entire image, it is difficult to detect abnormal frames in a small range, which reduces the accuracy of anomaly detection. Summary of the invention

[0006] The purpose of the present invention is to provide a video anomaly detection method based on foreground information enhancement, which can judge and detect abnormal situations according to the needs of video intelligent decision-making, and improve the accuracy of video anomaly detection and processing efficiency.

[0007] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0008] A video anomaly detection method based on foreground information enhancement comprises the following steps:

[0009] Step 1, pre-process the background and foreground segmentation of the continuous t+1 frames of video in the training set, and retain the foreground area of ​​the video image;

[0010] Step 2: Build a generation network and a discriminant network model; the generation network includes a generator and an optical flow network;

[0011] The preprocessed continuous t frames of video frame I 1 ,I 2 ,…,I t As the actual video frame enters the generation network, generating I t The next frame prediction video frame Construct the loss function of the generative network, add RGB three-color intensity constraints and gradient constraints to the loss function to ensure the continuity of the video frames in space, add motion constraints to ensure the continuity of the video frames in time, and add adversarial constraints to compete with the discriminant network; construct the loss function of the discriminant network, and use the discriminator to distinguish the predicted video frames from the corresponding actual video frames;

[0012] Step 3, training the generative network and the discriminative network using multiple groups of normal behavior video frames in the training set;

[0013] Step 4: Input the video frame data to be judged into the trained generation network and the discrimination network, and determine whether abnormal behavior occurs based on the adaptive peak signal-to-noise ratio, wherein the adaptive peak signal-to-noise ratio is as follows:

[0014]

[0015] To predict video frames The maximum pixel value in the image, Row is the number of pixels in the image row, and Col is the number of pixels in the image column. I i,j Represents the predicted video frame The pixel value of the pixel point at coordinate (i, j) in the actual video frame I corresponding to the predicted video frame; G i,j To predict video frames and the mask value of the foreground mask of the recognition area of ​​the actual video frame I at the pixel point with coordinate (i, j); ‖G‖ 1 To predict video frames And the norm of the foreground mask matrix G of the identified area of ​​the actual video frame I.

[0016] Furthermore, the loss function of the generative network is expressed as follows:

[0017]

[0018] st:λ int +λ gd +λ op +λ adv =1

[0019] Among them, λ int , gd , op , adv is the strength loss L int , gradient loss L gd , optical flow loss L op and fighting losses The weight of the loss function.

[0020] Furthermore, the addition of RGB three-color intensity constraints and gradient constraints is used to ensure the continuity of the video frame in space, which is specifically expressed as follows:

[0021]

[0022] Where L int is the strength loss, represents the pixel data of the generated predicted video frame, and I represents the pixel data of the actual video frame corresponding to the predicted video frame;

[0023]

[0024] Where L gd is the gradient loss, and the subscripts i and j represent the video frames The position of the pixel in I;

[0025] The motion constraint is added to ensure the temporal continuity of the video frames, which is specifically expressed as follows:

[0026]

[0027] Among them, L op is the optical flow loss, f(I t+1 ,I t ) indicates that the optical flow network is used to calculate two video frames I t+1 ,I t The optical flow information of the optical flow network is as follows: the input of the optical flow network is the predicted video frame generated by the generator, the corresponding actual video frame, and the previous frame of the actual video frame;

[0028] The added adversarial constraint is used to compete with the discriminant network, which is expressed as:

[0029]

[0030] in, To combat losses, Indicates the judgment network judges the video frame Is it a generated "fake" image or a real "true" image? Express judgment The probability that the label of pixel (i, j) is true,

[0031] Furthermore, the loss function of the discriminator is It is expressed as:

[0032]

[0033] Among them, D(I) i,j It represents the probability of judging that the label of pixel (i, j) in video frame I is true.

[0034] Furthermore, the training of the generative network and the discriminative network using multiple groups of normal behavior video frames in the training set includes:

[0035] A set of normal behavior video frames in the training set are preprocessed. 1 ,I 2 ,…,I t Feed into the generative network to generate predicted video frames And I t , The corresponding actual video frame I t+1 Calculate the loss function of the generator and the discriminator; adjust the loss function of the generator and the discriminator according to the calculated loss function;

[0036] Use the next set of normal behavior video frames for training, and use all the data in the training set to train the generation network and the discriminant network. One iteration is called an iteration; iterate the training multiple times until the loss function of the generator reaches a preset value or the preset number of iterations, at which time the training ends.

[0037] Furthermore, the predicted video frame and the actual video frame I t+1 The identification area foreground mask matrix G is expressed as follows:

[0038]

[0039] in, and K is the predicted video frame The actual video frame I corresponding to it t+1 The binary mask matrix of i,j It is expressed as follows:

[0040]

[0041] Furthermore, the method further comprises:

[0042] Evaluation results of adaptive peak signal-to-noise ratio The standardization process is performed to obtain a scoring function, which is defined as follows:

[0043]

[0044] is the maximum value of the adaptive peak signal-to-noise ratios obtained from multiple groups of video frames of the same scene in the video frame data to be determined; is the minimum value of the adaptive peak signal-to-noise ratio obtained from multiple groups of video frames of the same scene in the video frame data to be judged;

[0045] The score function Score(t) can be used to determine whether a video frame has abnormal behavior: the closer the score function Score(t) is to 1, the more normal the video frame is, otherwise, there is abnormal behavior.

[0046] Furthermore, the video frames in the training set are video frames with a continuous time sequence and a same size, and the video frames only contain normal behaviors.

[0047] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the video anomaly detection method based on foreground information enhancement are implemented.

[0048] A computer-readable storage medium stores a computer program, which implements the steps of the video anomaly detection method based on foreground information enhancement when executed by a processor.

[0049] Compared with the prior art, the present invention has the following technical features:

[0050] 1. The information required for video anomaly detection is concentrated on the foreground information, so a preprocessing scheme that strengthens the foreground information and weakens the background information is proposed for video anomaly detection. The video data is preprocessed before being sent to the video anomaly detection model to enhance the foreground information of the video and weaken the background information, so that the video anomaly detection model can perform detection efficiently.

[0051] 2. Aiming at the low response of abnormal behavior in small frames, an evaluation of video anomaly detection based on adaptive peak signal-to-noise ratio criterion is proposed. During preprocessing, the area of ​​foreground information is identified, so that the video anomaly detection model only detects video anomalies through a small frame of the identified area instead of the full frame, so as to improve the response of abnormal behavior in small frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a schematic flow diagram of steps 1 to 3 in the method of the present invention;

[0053] Figure 2 is a schematic flow diagram of step 4 in the method of the present invention;

[0054] Figure 3 It is the structural diagram of the U-Net model;

[0055] Figure 4 It is a schematic diagram of the framework of the method of the present invention. DETAILED DESCRIPTION

[0056] The video anomaly detection model based on future frame prediction is a suitable video anomaly detection model. It is worth noting that the focus of video anomaly detection is not on the background but on the foreground. The present invention proposes a video anomaly detection method based on foreground information enhancement. The innovation of the present invention is that, considering the importance of foreground information and the negligibility of background information, before the video image enters the deep learning network, the foreground information is strengthened and the background information is weakened before data processing. Video anomaly detection is performed based on the foreground information combined with the video anomaly detection model based on future frame prediction to improve the accuracy and efficiency of video anomaly detection.

[0057] See attached Figure 1 and 2 The video anomaly detection method based on foreground information enhancement of the present invention comprises the following steps:

[0058] First, the video frame data in the dataset needs to be processed, the video stream needs to be decomposed and processed into video frames with continuous time series and the same size, and the dataset needs to be divided into a training set and a test set. The training set only contains video frames with normal behavior, and a certain frame in the test set may or may not have abnormal behavior.

[0059] In the present invention, the continuous t frames are converted into 256*256 pixel size. A group of video frames is taken as an example for explanation.

[0060] Step 1: Preprocess the background and foreground segmentation of the continuous t+1 frames of video in the training set, and retain the foreground area of ​​the video image.

[0061] Input t+1 consecutive video frames, use background segmentation (such as neighbor algorithm, mixed Gaussian algorithm) to segment foreground and background; set all the pixel values ​​of the background part identified in the video frame to 0, at this time most of the information of the video frame is foreground information, and record the t+1 consecutive video frames at this time as I 1 ,I 2 ,…,I t+1 .

[0062] Step 2: Build the generative network and discriminative network models

[0063] The goal of the generation network in the present invention is to generate a predicted video frame with a small difference from the actual video frame of normal behavior; the goal of the discrimination network is to improve the generation effect of the generation network by discriminating whether the predicted video frame generated by the generation network is an actual video frame.

[0064] Step 2.1, the generation network includes a generator and an optical flow network (in this scheme, U-Net is used as the generator ( Figure 3 ), FlowNet is an optical flow network for illustration); the discriminant network includes a discriminator (this scheme uses a Markov discriminator (PatchGAN) as the discriminator for illustration).

[0065] The preprocessed continuous t frames of video frame I 1 ,I 2 ,…,I t As the actual video frame enters the generation network, generating I t The next frame prediction video frame The actual video frame corresponding to the predicted video frame is I t+1 .

[0066] Step 2.2, establish the loss function L of the generated network G

[0067] The U-Net model is a frame prediction model with the same input and output resolution. The generator is trained to make the objective function L G As small as possible, so that the generated predicted video frame As close as possible to the corresponding actual video frame I t+1 ; The framework schematic diagram of the present invention is as follows Figure 4 .

[0068] The loss function L of the generative network G It is obtained by weighted accumulation of four loss functions, namely, intensity loss L int , gradient loss L gd , optical flow loss L op and fighting losses λ int , gd , op , adv is the strength loss L int , gradient loss L gd , optical flow loss L op and fighting losses The weight of the total loss function.

[0069]

[0070] st:λ int +λ gd +λ op +λ adv =1

[0071] (1) Add RGB three-color intensity constraints and gradient constraints to ensure the spatial continuity of video frames.

[0072] The actual video frames after preprocessing are sent to the generative network U-Net as training data to train the generative network to generate predicted video frames; the intensity loss and gradient loss are used to ensure the spatial continuity of the generated predicted video frames.

[0073] Strength loss L int , which is used to constrain the content difference between the predicted video frame and the actual video frame, that is, to keep the generated predicted video frame as similar as possible to the actual video frame in the RGB space; represents the pixel data of the generated predicted video frame, and I represents the pixel data of the actual video frame corresponding to the predicted video frame.

[0074]

[0075] Gradient loss L gd , which is used to constrain the edge information between the predicted video frame and the actual video frame, that is, to constrain the edge shape of the generated image, sharpen the generated image, and highlight the edge information; the serial numbers i and j in the function represent the video frame The position of the pixel in .

[0076]

[0077] (2) Add motion constraints to ensure the temporal continuity of video frames

[0078] The predicted video frame generated by the generator, the corresponding actual video frame, and the previous frame of the actual video frame are input into the pre-trained optical flow network FlowNet, and motion constraints are added to ensure the temporal continuity of the video frames. Optical flow is the instantaneous speed of the pixel movement of a moving object in space on the observation imaging plane. It is a method that uses the change of pixels in the time domain in the image sequence and the correlation between adjacent frames to find the correspondence between the previous frame and the current frame, thereby calculating the motion information of the object between adjacent frames. Generally speaking, optical flow is caused by the movement of the foreground target itself in the scene, the movement of the camera, or the joint movement of the two. Therefore, obtaining the optical flow information of the image can obtain the motion information of the object on the image.

[0079] The optical flow network FlowNet is used to obtain motion information in the video and calculate the optical flow change using the loss of L1 distance, f(I t+1 ,I t ) indicates that the FlowNet optical flow network is used to calculate the two frames I t+1 ,I t The optical flow information of , so the optical flow loss can be expressed as:

[0080]

[0081] (3) Adding adversarial constraints to compete with the discriminant network

[0082] The generated predicted video frames need to be trained to be as similar as possible to the actual video frames; the generated network obtains video frames of normal behavior to generate predicted video frames that are as similar as possible to the actual video frames; during training, 0 and 1 represent "false" labels and "true" labels respectively.

[0083] Optimizing the generative network requires training the discriminative network to distinguish between actual video frames and generated predicted video frames, so that the generator and discriminator are optimized through adversarial learning.

[0084] When training the generator, the weight of the discriminator is fixed. The task of the generative network is to make the generated image "fool" the discriminator so that the discriminator judges that the image generated by the generative network is "real", that is, the label value is 1. Therefore, the adversarial loss of the generative network is defined as the generator mean square error loss expressed as:

[0085]

[0086] in, Indicates the judgment network judges the video frame Is it a generated "fake" image or a real "true" image? Express judgment The probability that the label of pixel (i, j) is true,

[0087] Step 2.3, establish the loss function of the discriminator

[0088] The discriminant network only contains the discriminator, so the loss function of the discriminant network is the loss function of the discriminator.

[0089] When training the discriminator, the weights of the generator are fixed. The goal of training the discriminator is to try to distinguish the actual video frames from the predicted video frames, correctly judge the images generated by the generation network as "fake", that is, the label value is 0, and judge the real images as "true", that is, the label value is 1, so the adversarial loss of the discriminator is expressed as;

[0090]

[0091] Among them, D(I) i,j It represents the probability of judging that the label of pixel (i, j) in video frame I is true.

[0092] Step 3: Train the generative network and the discriminative network using multiple groups of normal behavior video frames in the training set.

[0093] Step 3.1: After a set of normal behavior video frames are preprocessed in step 1, a continuous t+1 frame normal behavior video frame I is obtained, in which most of the information is foreground information. 1 ,I 2 ,…,I t+1 .

[0094] Step 3.2, I 1 ,I 2 ,…,I t Feed into the generative network to generate predicted video frames And I t , The corresponding actual video frame It+1 Calculate the loss function for the generator and discriminator.

[0095] Step 3.3, adjust the internal parameters of the generator and discriminator according to the loss function calculated in step 3.2 to reduce the value of the loss function.

[0096] Step 3.4, use the next set of normal behavior video frames for training and repeat steps 3.1 to 3.3, use all the data in the training set to train the generator network and the discriminator network once as one iteration; iterate the training multiple times until the loss function of the generator reaches a smaller preset value or a certain preset number of iterations, at which time the training ends.

[0097] Step 4: Input the video frame data to be judged into the trained generation network and the discrimination network to determine whether abnormal behavior occurs based on the adaptive peak signal-to-noise ratio.

[0098] Step 4.1: Use the data in the test set (or the image data actually collected) as the video frame data to be judged, and take the preprocessed video frames I of the continuous t+1 frames in the test set. 1 ,I 2 ,…,I t ,I t+1 Input the trained generative network and generate predicted video frames Get predicted video frames using background segmenter The actual video frame I corresponding to it t+1 The binary mask matrix With K.

[0099] Take frame I to illustrate the foreground mask matrix K: The background segmentor determines whether each pixel of the image is the foreground. If the pixel is the foreground, the value of the mask corresponding to the pixel is set to 1. If it is the background, the value of the mask corresponding to the pixel is set to 0, thereby obtaining the binary mask matrix K:

[0100]

[0101] Get predicted video frame and the actual video frame I t+1 The identification area foreground mask matrix G:

[0102]

[0103] In step 4.2, an adaptive peak signal-to-noise ratio is calculated to evaluate the difference between the predicted video frame and the actual video frame image.

[0104] Because the training network is trained using video data of normal behavior, in theory, the future frames generated by the network can only represent normal behavior, and abnormal behavior cannot be predicted. Assuming that the future frames of normal behavior videos can be well predicted, the difference between the predicted video frames and the corresponding actual video frames should be small; on the contrary, the future frames of abnormal behavior videos cannot be predicted, so the difference between the predicted video frames and the corresponding actual video frames should be large. The peak signal-to-noise ratio can well measure the difference between two pictures. The peak signal-to-noise ratio is defined as follows:

[0105]

[0106] To predict video frames The maximum pixel value in the video frame, N represents the number of pixels in the video frame image. Represents the predicted video frame The pixel value of the i-th pixel in i Represents the pixel value of the i-th pixel in the actual video frame I.

[0107] If the similarity between the predicted video frame and the actual video frame is higher, the peak signal-to-noise ratio is larger, indicating that the probability that the video frame is a normal frame is greater; because the video frame cannot be predicted if it is an abnormal frame, the similarity between the generated predicted video frame and the actual video frame will be very low, and the peak signal-to-noise ratio will also be very small.

[0108] According to this evaluation method, the difference of the entire image can be evaluated. However, when the area of ​​the abnormal event is too small, the corresponding peak signal-to-noise ratio (PSNR) change will also become smaller, resulting in a low detection rate of abnormal events in small images. In order to better respond to abnormal events in small images, an adaptive peak signal-to-noise ratio evaluation method is proposed, and the adaptive peak signal-to-noise ratio is defined as follows:

[0109]

[0110] To predict video frames The maximum pixel value in the image, Row is the number of pixels in the image row, Col is the number of pixels in the image column (the size of the image is Row*Col, such as the image size in the present invention is 256*256 pixels, so Row=256, Col=256), Represents the predicted video frame The pixel value of the pixel at the coordinate (i, j), I i,j Represents the pixel value of the pixel at coordinate (i, j) in the actual video frame I. i,j To predict video frames and the mask value of the foreground mask of the recognition area of ​​the actual video frame I at the pixel point with coordinate (i, j); ‖G‖1 is the norm of the foreground mask matrix G, and the number of pixels belonging to the foreground area in the identified area is obtained.

[0111] Adaptive peak signal-to-noise ratio can be used to evaluate the image difference only in the foreground area of ​​the predicted video frame and its corresponding actual video frame, thereby improving the response rate to abnormal events in small frames.

[0112] Step 4.3, evaluation results of adaptive peak signal-to-noise ratio The standardization process is performed to obtain a scoring function, which is defined as follows:

[0113]

[0114] It is the maximum value of the adaptive peak signal-to-noise ratio obtained from multiple groups of video frames of the same scene in the test set. It is the minimum value of the adaptive peak signal-to-noise ratio obtained from multiple groups of video frames of the same scene in the test set.

[0115] In step 4.3, the score function Score(t) can express the result of adaptive peak signal-to-noise ratio normalization of each frame of the test video in the range [0,1]. The score function Score(t) can be used to determine whether the video frame has abnormal behavior. The closer the score function Score(t) is to 1, the more normal the video frame is, and vice versa, abnormal behavior exists.

[0116] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A video anomaly detection method based on foreground information enhancement, characterized in that: The following steps are involved: Step 1, pre-process the background and foreground segmentation of the continuous t+1 frames of video in the training set, and retain the foreground area of ​​the video image; Step 2: Build a generative network and a discriminative network model; the generative network includes a generator and an optical flow network; The preprocessed continuous t frames of video frames I1, I2, ..., I t As the actual video frame enters the generation network, generating I t The next frame prediction video frame Construct the loss function of the generative network, add RGB three-color intensity constraints and gradient constraints to the loss function to ensure the continuity of the video frames in space, add motion constraints to ensure the continuity of the video frames in time, and add adversarial constraints to compete with the discriminant network; construct the loss function of the discriminant network, and use the discriminator to distinguish the predicted video frames from the corresponding actual video frames; Step 3, training the generative network and the discriminative network using multiple groups of normal behavior video frames in the training set; Step 4: Input the video frame data to be judged into the trained generation network and the discrimination network, and determine whether abnormal behavior occurs based on the adaptive peak signal-to-noise ratio, wherein the adaptive peak signal-to-noise ratio is as follows: To predict video frames The maximum pixel value in the image, Row is the number of pixels in the image row, and Col is the number of pixels in the image column. I i,j Represents the predicted video frame The pixel value of the pixel point at coordinate (i, j) in the actual video frame I corresponding to the predicted video frame; G i,j To predict video frames and the mask value of the foreground mask of the recognition area of ​​the actual video frame I at the pixel point with coordinate (i, j); ‖G‖1 is the predicted video frame And the norm of the foreground mask matrix G of the identified area of ​​the actual video frame I.

2. The video anomaly detection method based on foreground information enhancement according to claim 1 is characterized in that: The loss function of the generative network is expressed as follows: Among them, λ int , gd , op , adv is the strength loss L int , gradient loss L gd , optical flow loss L op and fighting losses The weight of the loss function.

3. The video anomaly detection method based on foreground information enhancement according to claim 1 is characterized in that: The addition of RGB three-color intensity constraints and gradient constraints is used to ensure the continuity of the video frame in space, which is specifically expressed as follows: Where L int is the strength loss, represents the pixel data of the generated predicted video frame, and I represents the pixel data of the actual video frame corresponding to the predicted video frame; Where L gd is the gradient loss, and the subscripts i and j represent the video frames The position of the pixel in I; The motion constraint is added to ensure the temporal continuity of the video frames, which is specifically expressed as follows: Among them, L op is the optical flow loss, f(I t+1 ,I t ) indicates that the optical flow network is used to calculate two video frames I t+1 ,I t The optical flow information of the optical flow network is as follows: the input of the optical flow network is the predicted video frame generated by the generator, the corresponding actual video frame, and the previous frame of the actual video frame; The added adversarial constraint is used to compete with the discriminant network, which is expressed as: in, To combat losses, Indicates the judgment network judges the video frame Is it a generated "fake" image or a real "true" image? Express judgment The probability that the label of pixel (i, j) is true, 4. The video anomaly detection method based on foreground information enhancement according to claim 1 is characterized in that: The loss function of the discriminator It is expressed as: Among them, D(I) i,j It represents the probability of judging that the label of pixel (i, j) in video frame I is true.

5. The video anomaly detection method based on foreground information enhancement according to claim 1 is characterized in that: The training of the generative network and the discriminative network using multiple groups of normal behavior video frames in the training set includes: A set of normal behavior video frames in the training set are preprocessed I1,I2,…,I t Feed into the generative network to generate predicted video frames And I t , The corresponding actual video frame I t+1 Calculate the loss function of the generator and the discriminator; adjust the loss function of the generator and the discriminator according to the calculated loss function; Use the next set of normal behavior video frames for training, and use all the data in the training set to train the generation network and the discriminant network. One iteration is called an iteration; iterate the training multiple times until the loss function of the generator reaches a preset value or the preset number of iterations, at which time the training ends.

6. The video anomaly detection method based on foreground information enhancement according to claim 1 is characterized in that: The predicted video frame and the actual video frame I t+1 The identification area foreground mask matrix G is expressed as follows: in, and K is the predicted video frame The actual video frame I corresponding to it t+1 The binary mask matrix of i,j It is expressed as follows:

7. The video anomaly detection method based on foreground information enhancement according to claim 1 is characterized in that: The method further comprises: Evaluation results of adaptive peak signal-to-noise ratio The standardization process is performed to obtain a scoring function, which is defined as follows: is the maximum value of the adaptive peak signal-to-noise ratios obtained from multiple groups of video frames of the same scene in the video frame data to be determined; is the minimum value of the adaptive peak signal-to-noise ratio obtained from multiple groups of video frames of the same scene in the video frame data to be judged; The score function Score(t) can be used to determine whether a video frame has abnormal behavior: the closer the score function Score(t) is to 1, the more normal the video frame is, otherwise, there is abnormal behavior.

8. The video anomaly detection method based on foreground information enhancement according to claim 1 is characterized in that: The video frames in the training set are video frames with a continuous time sequence and the same size, and the video frames only contain normal behaviors.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the video anomaly detection method based on foreground information enhancement according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the video anomaly detection method based on foreground information enhancement according to any one of claims 1 to 8 are implemented.

Images