A video anomaly detection method based on object meta

Abstract

The application relates to the field of video anomaly monitoring in computer vision, and particularly relates to a video anomaly detection method based on an object element. An object element is generated by performing optical flow calculation and object detection on normal video images of an arbitrary scene. Next, the object element is input into a multi-level memory search guided autoencoder for training to obtain an image reconstruction model. Then, the image reconstruction model is applied to a to-be-detected video of the same scene to perform image reconstruction on the objects in the to-be-detected video. Finally, the reconstructed objects are mapped back to the original video frames, and anomaly detection is realized according to the differences between the images. The application accurately extracts object pixels in the video frames through optical flow calculation, object detection and instance segmentation, and performs dimensional fusion of the object pixels with calculated type information, position information and an optical flow map to generate a new type of data, namely the object element. Then, the object element is input into the multi-level memory search guided autoencoder for training and reconstruction, and video anomaly detection is realized from multiple dimensions.

Description

TECHNICAL FIELD

[0001] The application relates to the field of video anomaly monitoring in computer vision, and is a video anomaly detection method based on object elements. BACKGROUND

[0002] Video anomaly detection refers to detecting abnormal behavior events in a given video, and the task has high complexity because the types of abnormal events are extremely numerous and all abnormal types cannot be exhausted, so an unsupervised model is mostly used for video anomaly detection. At present, in the field of video anomaly detection, a deep learning method is widely used, and an automatic encoder Autoencoder is a quite popular detection method. The automatic encoder is trained on normal video frames, and a current frame is reconstructed through the trained model. Since the difference of the abnormal frame reconstruction is larger than that of the normal frame reconstruction, the abnormal frame in the video can be detected through the method. However, such a method only depends on pixel image reconstruction, which makes the neural network only learn pixel-level information and lack other key information, so it is difficult to effectively detect in a complex video situation. SUMMARY

[0003] The application provides a video anomaly detection method based on object elements. Object elements are generated by performing optical flow calculation and object detection on normal video images of an arbitrary scene. Next, the object elements are input into a multi-level memory search guided automatic encoder for training to obtain an image reconstruction model. Then, the image reconstruction model is applied to a to-be-detected video of the same scene to reconstruct images of objects. Finally, the reconstructed images are mapped back to the original video frames to realize anomaly detection according to the difference between the images.

[0004] To achieve the above object, the application provides the following technical scheme: a video anomaly detection method based on object elements, the method comprising:

[0005] S1, performing optical flow calculation and object detection on normal video images of an arbitrary scene to generate object elements;

[0006] S2, inputting the object elements into a multi-level memory search guided automatic encoder for training to obtain an image reconstruction model;

[0007] S3, applying the image reconstruction model to a to-be-detected video of the same scene to reconstruct images of objects;

[0008] S4, mapping the reconstructed images back to the original video frames to realize anomaly detection according to the difference between the images.

[0009] Further, the step S1 is specifically:

[0010] S11, select a normal video of an arbitrary scene, and extract video frames at the same interval to obtain a set of video frame images (Fr0, Fr1, Fr2, …, Fr t-1 ) with the same time interval, and the number of video frames is t;

[0011] S12, calculate the optical flow between adjacent frames in the video frame images with the same time interval using the FlowNet network to obtain optical flow maps (Fl0, Fl1, Fl2, …, Fl t-1 ) with the number of t-1;

[0012] S13, use Mask RCNN to perform object detection and instance positioning on the first t-1 video frame images with the same time interval to obtain the type information and location mask information of each object;

[0013] S14, encode the type information and location mask information, and convert them into Numpy arrays with uniform size to obtain object type meta and object location meta

[0014] S15, according to the location mask information, extract the same position optical flow from each of the t-1 optical flow maps, and convert the extracted optical flow into Numpy arrays with uniform size to obtain object optical flow meta

[0015] S16, according to the location mask information, extract the object pixels from each of the first t-1 video frame images with the same time interval, and convert the extracted object pixels into Numpy arrays with uniform size to obtain object pixel meta

[0016] S17, combine the object type meta , object location meta , object optical flow meta and object pixel meta of each object in dimension to obtain object meta

[0017] Further, the step S2 is specifically:

[0018] S21, construct a multi-level memory search guided autoencoder;

[0019] S22, the structure of the multi-level memory search guided autoencoder is first-level encoder, first-level memory module, second-level encoder, second-level memory module, third-level encoder, third-level memory module, third-level decoder, second-level decoder and first-level decoder in turn; wherein the encoder is responsible for down-sampling encoding of the input, the decoder is responsible for up-sampling decoding of the input, and the memory module is responsible for storing the encoding output by the encoder; the essence of the memory module is an NxC matrix, N is the maximum number of records, and C is the dimension of the recorded encoding; during training, the memory module opens the storage function, changes the internal matrix value through matrix operation, thereby realizing storage; during detection, the memory module closes the storage function, only provides query and reconstruction functions, and finds and outputs the record closest to the input encoding through encoding similarity;

[0020] S23, the structure of the encoding part: first-level encoder, second-level encoder is Conv layer, BN layer and LeakReLU layer in turn, and the structure of the third-level encoder is Conv layer, BN layer, LeakReLU layer, Conv layer, BN layer and LeakReLU layer in turn;

[0021] S24, the structure of the decoding part: third-level decoder is deConv layer, BN layer, LeakReLU layer, deConv layer, BN layer and LeakReLU layer in turn, the structure of the second-level decoder is deConv layer, BN layer and LeakReLU layer in turn, and the structure of the third-level decoder is a single deConv layer;

[0022] S25, after the object meta to be trained is input into the multi-level memory search guided autoencoder, the image reconstruction model is obtained through training.

[0023] It is further explained that the step S3 is specifically:

[0024] S31, the same scene video to be detected is real-time frame extracted according to equal time interval;

[0025] S32, each pair of adjacent video frames is input into the FlowNet network to calculate the optical flow map;

[0026] S33, Mask RCNN is used for object detection and instance segmentation of the previous frame in each pair of adjacent video frames, all object metas therein are constructed, and a position list is used to record the position coordinates of the object represented by each object meta in the original image;

[0027] S34, the image reconstruction model trained in S2 is used to initialize the multi-level memory search guided autoencoder, so that the memory module loads the memory matrix, and the memory module initialization is completed;

[0028] S35. Input all object elements constructed from each pair of adjacent video frames into a multi-level memory search-guided autoencoder. After being converted into codes by the encoder, the codes are input into the memory module. The memory module searches the memory matrix to obtain the most similar code that was memorized during training and outputs it. After multi-level encoding and multi-level memory search, the deepest code is obtained. Then, it is upsampled and reconstructed by a three-level decoder to obtain the reconstructed object elements.

[0029] S36. Calculate the total loss function L when a multi-level memory search-guided autoencoder reconstructs an image of the input object, which consists of the reconstruction loss L. rec and the entropy loss L of the memory module ent Composition; Assuming the input of the model is object element M, its reconstructed output is Then, during the encoding and decoding process, the reconstruction loss of this object element is to minimize the L2 distance:

[0030]

[0031] In the memory module, regularization strategies are added to improve... The sparsity of this property will be minimized. The entropy of the i-th level memory module is such that it results in an entropy loss in each memory module. Therefore, for the i-th level memory module, the entropy loss is:

[0032]

[0033] in, The probability of matching the input of this memory module with the nth row of the corresponding memory matrix, according to L rec and L ent The total loss function L of the model can be obtained:

[0034] L=αL rec +βL ent #(3)

[0035] Where α and β are equilibrium parameters.

[0036] To further explain, step S4 specifically involves:

[0037] S41. Map each reconstructed object element back to the original image according to the record in the position list. After completing all mappings, the reconstructed images of all video frame images except the last video frame image are obtained.

[0038] S42. Calculate the AUC score based on the difference between the reconstructed image and the original image;

[0039] S43, visualizing the AUC score, sequentially labeling all extracted video frames in a segment of video frames to be detected, and calculating the AUC score of all video frames except the last video frame, and drawing a line chart by taking the label set as the x-axis and the AUC score set as the y-axis, the smoother the line chart, the better the detection effect;

[0040] S44, visualizing and labeling the abnormality in the video frame: subtracting the reconstructed image from the original image, and determining that it is an abnormal object when the number of noise points in the subtraction process is greater than the set threshold of the pixel number of the object itself, and labeling it.

[0041] The beneficial effects of the present application are:

[0042] 1. The present application constructs a new data form of object element carrying multiple key abnormal information;

[0043] 2. The present application can realize video anomaly detection based on object element;

[0044] 3. The present application can be applied to the monitoring field and the early warning field, such as real-time monitoring of abnormal behaviors of road vehicles and pedestrians BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 Flowchart of the present application;

[0046] Figure 2 Object element generation process diagram of the present application;

[0047] Figure 3 Multi-level memory search guided autoencoder construction diagram of the present application;

[0048] Figure 4 AUC score line chart of the UCSDped2 data set after image reconstruction;

[0049] Figure 5 Visualization result chart of abnormality detection of multiple data sets. DETAILED DESCRIPTION

[0050] The present application will be described in detail below in conjunction with the embodiments shown in the drawings. However, these embodiments do not limit the present application, and the structural, method, or functional changes made by those skilled in the art based on these embodiments are included in the protection scope of the present application.

[0051] As shown in Figure 1 , the present application is a video anomaly detection method based on object element, including the following steps:

[0052] S1, optical flow calculation and object detection are performed on normal video images of an arbitrary scene to generate object elements;

[0053] S2, the object elements are input into a multi-level memory search guided autoencoder for training to obtain an image reconstruction model;

[0054] S3, the image reconstruction model is applied to a to-be-detected video of the same scene to perform image reconstruction on the objects therein;

[0055] S4, the reconstructed images are mapped back to the original video frames to realize anomaly detection according to the differences between the images.

[0056] As a preferred embodiment of the present application, step S1 comprises:

[0057] Optical flow calculation and object detection are performed on normal video images of an arbitrary scene to generate object elements, as shown in Figure 2 The construction process of the object elements. Each object element is composed of four parts: type code, position code, optical flow code and pixel code.

[0058] S11, a normal video of an arbitrary scene is selected, and video frames are extracted therefrom at the same interval to obtain a group of video frame images (Fr0, Fr1, Fr2,..., Fr t-1 ) at equal time intervals, and the number of video frames is t;

[0059] S12, a FlowNet network is introduced, which is used to calculate the displacement direction and displacement size of a pixel point in X in the adjacent two frames (X, Y) of the video stream in Y, and map the calculation results to an optical flow graph, wherein the color of each point in the optical flow graph represents the displacement direction of the pixel at the point, and the depth of each point represents the displacement size of the pixel at the point; here, the FlowNet network is used to calculate the optical flow between adjacent frames in the video frame images at equal time intervals to obtain optical flow graphs (Fl0, Fl1, Fl2,..., Fl t-1 ) with a number of t-1;

[0060] S13, Mask RCNN is introduced, which is essentially a two-stage framework, the first stage scans the image and generates proposals (i.e. regions that may contain targets), and the second stage classifies the proposals and generates bounding boxes and masks, thereby realizing classification detection and accurate position acquisition of objects; here, Mask RCNN is used to perform object detection and instance positioning on the first t-1 video frame images at equal time intervals to obtain type information and position mask information of each object;

[0061] S14, the type information and position mask information are encoded and converted into Numpy arrays of uniform size to obtain object type elements and object position elements

[0062] S15, according to the position mask information, same position optical flow extraction is carried out on each of the t-1 optical flow maps, and the extracted optical flow is converted into a uniform size Numpy array, and object optical flow elements are obtained

[0063] S16, according to the position mask information, object pixel extraction is carried out on each of the previous t-1 time interval video frame images, and the extracted object pixels are converted into a uniform size Numpy array, and object pixel elements are obtained

[0064] S17, the object type element corresponding to each object object position element object optical flow element and object pixel element are dimensionally combined to obtain the object element corresponding to each object

[0065] As a preferred embodiment of the present application, as shown in the specific construction diagram of the multi-level memory search guided autoencoder, the autoencoder is composed of three-level encoder, three-level memory module and three-level decoder. Step S2 includes the following contents: Figure 3

[0066] S21, a multi-level memory search guided autoencoder is constructed;

[0067] S22, the structure of the multi-level memory search guided autoencoder is a first-level encoder, a first-level memory module, a second-level encoder, a second-level memory module, a third-level encoder, a third-level memory module, a third-level decoder, a second-level decoder and a first-level decoder in turn; wherein the encoder is responsible for down-sampling encoding of the input, the decoder is responsible for up-sampling decoding of the input, and the memory module is responsible for storing the encoding output by the encoder; the essence of the memory module is an NxC matrix, N is the maximum number of records, and C is the dimension of the recorded encoding; during training, the memory module opens the storage function, changes the internal matrix value through matrix operation, thereby realizing storage; during detection, the memory module closes the storage function, only provides query and reconstruction functions, and finds the record closest to the input encoding through encoding similarity and outputs;

[0068] S23, further, in the encoding part, the structure of the first-level encoder is a Conv layer, a BN layer and a LeakReLU layer in turn, the Conv layer adopts grouped convolution with a group number of 4, the input channel number is 12, and the output channel number is 96, the memory depth of the first-level memory module is 2000, and the feature depth is 96;

[0069] ​The structure of the second-level encoder is sequentially a Conv layer, a BN layer and a LeakReLU layer, the Conv layer adopts group convolution with a group number of 4, an input channel number of 96 and an output channel number of 128, the memory depth of the second-level memory module is 2000, and the feature depth is 128;

[0070] The structure of the third-level encoder is sequentially a Conv layer, a BN layer, a LeakReLU layer, a Conv layer, a BN layer and a LeakReLU layer, wherein the first Conv layer adopts group convolution with a group number of 4, an input channel number of 128 and an output channel number of 256, the second Conv layer adopts group convolution with a group number of 4, an input channel number of 256 and an output channel number of 256, the memory depth of the third-level memory module is 2000, and the feature depth is 256;

[0071] S24, further, in the decoding part, the structure of the third-level decoder is sequentially a deConv layer, a BN layer, a LeakReLU layer, a deConv layer, a BN layer and a LeakReLU layer, wherein the first deConv layer adopts group convolution with a group number of 4, an input channel number of 256 and an output channel number of 256, the second deConv layer adopts group convolution with a group number of 4, an input channel number of 256 and an output channel number of 128;

[0072] The structure of the second-level decoder is sequentially a deConv layer, a BN layer and a LeakReLU layer, wherein the deConv layer adopts group convolution with a group number of 4, an input channel number of 128 and an output channel number of 96;

[0073] The structure of the third-level decoder is a single deConv layer, which adopts group convolution with a group number of 4, an input channel number of 96 and an output channel number of 12;

[0074] S25, the object element to be trained is input into the multi-level memory search guided autoencoder for training, and an image reconstruction model is obtained.

[0075] As a preferred embodiment of the application, the object element to be trained is input into the multi-level memory search guided autoencoder for 150 times of training.

[0076] As a preferred embodiment of the application, the step S3 is specifically:

[0077] The image reconstruction model is applied to the to-be-detected video of the same scene to reconstruct the images of the objects in the to-be-detected video;

[0078] S31, the to-be-detected video of the same scene is real-time frame extracted according to the same time interval as in S1;

[0079] S32, input each pair of adjacent video frames extracted from the video to be detected into the FlowNet network, and calculate the optical flow map;

[0080] S33, use Mask RCNN to perform object detection and instance segmentation on the previous frame in each pair of adjacent video frames, and construct all object elements therein, while using a position list to record the position coordinates of each object element in the original image;

[0081] S34, initialize the multi-level memory search guided autoencoder using the image reconstruction model trained in S2, load the memory matrix into the memory module, and complete the memory module initialization;

[0082] S35, input all object elements constructed from each pair of adjacent video frames into the multi-level memory search guided autoencoder, convert them into codes through the encoder, and input them into the memory module. The memory module finds the most similar codes in the memory matrix during the training process and outputs them. After multi-level encoding and multi-level memory search, the deepest level code is obtained, and then it is reconstructed through three-level decoder upsampling, so as to obtain the reconstructed object element;

[0083] S36, calculate the total loss function L of the multi-level memory search guided autoencoder when reconstructing the input object element, which is composed of reconstruction loss L rec and entropy loss L ent of the memory module; assuming that the input of the model is object element M, and the reconstruction output is The reconstruction loss of the object element in the encoding and decoding process is the minimum l2 distance:

[0084]

[0085] In the memory module, since the regularization strategy is added to improve the sparsity of , it will minimize the entropy of , so that the entropy loss is generated in each memory module. Therefore, for the i-th memory module, the entropy loss is:

[0086]

[0087] wherein, is the probability of matching the nth row of the corresponding memory matrix of the input of the memory module. According to L rec and L ent , the total loss function L of the model can be obtained:

[0088] L = aL rec + bL ent #(3)

[0089] wherein, a and b are balance parameters;

[0090] As a preferred embodiment of the present application, the step S4 is specifically:

[0091] mapping the reconstructed objects back to the original video frames to achieve anomaly detection according to the difference between images;

[0092] S41, mapping each reconstructed object element according to the record in the position list back to the original image, and obtaining the reconstructed images of all video frame images except the last video frame image after completing all the mapping;

[0093] S42, calculating the AUC score according to the difference between the reconstructed image and the original image, which is essentially the area under the ROC curve, and the actual significance is the probability that the positive example is ranked in front of the negative example, so the AUC is a commonly used evaluation index to measure the pros and cons of the binary classification model, and the higher the score means the better the detection effect;

[0094] The specific steps of AUC score calculation are as follows: first, unify the reconstructed image and the original image into Numpy array format, and subtract each bit of the two arrays to construct a difference Numpy array;

[0095] Secondly, read the annotation result file of the data set, and compare the real difference array in the annotation result file with the difference Numpy array obtained by subtraction to calculate the proportion of negative samples judged as positive FPR and the proportion of positive samples judged as positive TPR;

[0096] Then, draw the ROC curve with FPR as the horizontal axis and TPR as the vertical axis, and each point on the curve corresponds to a threshold;

[0097] Finally, calculate the area under the ROC curve, which is the AUC score;

[0098] S43, visualizing the AUC score, labeling all extracted video frames in a segment of video frames in order, and calculating the AUC score of all video frames except the last video frame, and drawing a line chart with the label set as the x-axis and the AUC score set as the y-axis, the smoother the line chart, the better the detection effect, the AUC score line chart and the comparison line chart refer to the AUC score line chart and the comparison line chart of Figure 4 , the AUC score line chart of the 4th scene of the UCSDped2 data set from left to right is (a) based on pixel reconstruction, (b) based on optical flow reconstruction, (c) based on pixel and optical flow combined reconstruction, and (d) based on object element reconstruction.

[0099] S44, visualizing and labeling the anomaly in the video frame: subtract the reconstructed image from the original image, when the number of noise points in the subtraction process is greater than the set threshold of the number of pixels of the object itself, it is determined to be an abnormal object, and is labeled;

[0100] Since only AUC score can only indicate whether the video frame is abnormal, it cannot accurately indicate the abnormal area, therefore, the anomaly in the video frame needs to be visualized and labeled; since there is loss in the reconstruction process, a large number of noise points will be generated when the reconstructed image is subtracted from the original image, each noise point indicates that the pixel at this position in the reconstructed image is different from the original image; due to the randomness of the reconstruction loss, the noise points generated by the reconstruction of the normal object are basically not concentrated, and the amount is small, therefore, a threshold is set, when the number of reconstruction noise points is greater than the set threshold of the number of pixels of the object itself, it is determined to be an abnormal object, and is labeled, as a preferred embodiment of the present application, the threshold is preferably 50%. The detection result graph is shown in Figure 5 Abnormality detection result graph on multiple data set segments. From top to bottom, (a) UCSDped2 data set, (b) StreetScene data set and (c) Avenue data set.

[0101] It should be noted that the present specification is explained according to the embodiments, but not every specific execution step is an independent technical solution, and this explanation mode is used to make the reader more understand the design steps and various methods used in the present application. Related personnel need to take the whole specification as a whole, and various technical solutions can be appropriately combined to form other embodiments that related personnel can understand.

[0102] The series of detailed descriptions listed above are only specific descriptions of the feasible embodiments of the present application, and they are not used to limit the protection scope of the present application, and equivalent embodiments or changes made without departing from the spirit of the present application should be included in the protection scope of the present application.

Claims

1. A method for video anomaly detection based on object meta, characterized in that, It comprises the following steps: S1: optical flow calculation and object detection are performed on a video image of an arbitrary scene to generate an object element; The specific process of the step S1 is as follows: S11: select a video of an arbitrary scene, extract video frames at the same interval to obtain a set of video frame images (Fr0, Fr1, Fr2, …, Fr t-1 ) with the same time interval, and the number of video frames is t; S12: calculate the optical flow between adjacent frames in the equal time interval video frame images using the FlowNet network, and obtain the optical flow map (Fl0, Fl1, Fl2, …, Fl t-1 ), the number of which is t-1; S13: object detection and instance positioning are performed on the previous t-1 time interval video frame images using Mask RCNN to obtain type information and location mask information of each object; S14: Encode the type information and position mask information and convert to a uniform size Numpy array, resulting in the object type meta f i t and the object position meta S15: According to the position mask information, same position optical flow extraction is performed on each of the t-1 optical flow maps, and the extracted optical flow is converted into a Numpy array of uniform size, to obtain an object optical flow element S16: According to the position mask information, object pixel extraction is performed on each of the first t-1 equally time-interval video frame images, and the extracted object pixels are converted into a Numpy array of uniform size, to obtain an object pixel matrix X S17: merging the object type meta f corresponding to each object i t , object position meta object optical flow meta and object pixel meta S18: merging the object meta corresponding to each object S2: the object element is input into a multi-level memory search guided auto-encoder for training to obtain an image reconstruction model; S3: the image reconstruction model is applied to a to-be-detected video of the same scene to perform image reconstruction on the objects of the to-be-detected video; S4: the reconstructed image is mapped back to the original video frame to realize anomaly detection according to the difference between the images.

2. The object meta-based video anomaly detection method of claim 1, wherein, The specific process of the step S2 is as follows: S21: a multi-level memory search guided auto-encoder is constructed; S22: the structure of the multi-level memory search guided auto-encoder is a first-level encoder, a first-level memory module, a second-level encoder, a second-level memory module, a third-level encoder, a third-level memory module, a third-level decoder, a second-level decoder and a first-level decoder in sequence; wherein the encoder is responsible for down-sampling coding of the input, the decoder is responsible for up-sampling decoding of the input, and the memory module is responsible for storing the coding output by the encoder; The essence of the memory module is an NxC matrix, N is the maximum number of records, and C is the dimension of the recorded coding; during training, the memory module opens the storage function, changes the internal matrix value through matrix operation, and thus realizes storage; during detection, the memory module closes the storage function, only provides query and reconstruction functions, and finds and outputs the record closest to the input coding through coding similarity search; S23: the coding part: the structure of the first-level encoder and the second-level encoder is a Conv layer, a BN layer and a LeakReLU layer in sequence, and the structure of the third-level encoder is a Conv layer, a BN layer, a LeakReLU layer, a Conv layer, a BN layer and a LeakReLU layer in sequence; S24: the decoding part: the structure of the third-level decoder is a deConv layer, a BN layer, a LeakReLU layer, a deConv layer, a BN layer and a LeakReLU layer in sequence, the structure of the second-level decoder is a deConv layer, a BN layer and a LeakReLU layer in sequence, and the structure of the third-level decoder is a single deConv layer; S25: the object element to be trained is input into the multi-level memory search guided auto-encoder for training to obtain the image reconstruction model. 3.The video anomaly detection method based on object meta according to claim 1, wherein, The specific process of the step S3 is as follows: S31: real-time frame extraction is performed on the to-be-detected video of the same scene according to equal time intervals; S32: the optical flow map is calculated by inputting each pair of adjacent video frames into a FlowNet network; S33: object detection and instance segmentation are performed on the previous frame in each pair of adjacent video frames using Mask RCNN to construct all object elements therein, and a position list is used to record the position coordinates of the object represented by each object element in the original image; S34: the multi-level memory search guided auto-encoder is initialized using the image reconstruction model trained in S2, the memory module is loaded with a memory matrix, and the memory module initialization is completed; S35: Input all object elements constructed from each pair of adjacent video frames into the multi-level memory search-guided autoencoder. After being converted into codes by the encoder, the codes are input into the memory module. The memory module searches the memory matrix to obtain the most similar code that was memorized during training and outputs it. After multi-level encoding and multi-level memory search, the deepest code is obtained. Then, it is upsampled and reconstructed by a three-level decoder to obtain the reconstructed object elements. S36: calculate the total loss function L of the object meta input in the multi-level memory search guide auto-encoder image reconstruction, the total loss function L is composed of reconstruction loss L rec and the entropy loss L of the memory module ent ; the input of the assumed model is the object meta M, and the reconstruction output is The reconstruction loss of the object meta in the coding and decoding process is to minimize the l2 distance: In the memory module, since the regularization strategy is added to improve the sparsity of , it will minimize the entropy of , thus generating an entropy loss in each memory module, so for the i-th memory module, its entropy loss is: wherein, is the probability of matching the input of the memory module to the nth row of the corresponding memory matrix, according to L rec and L ent The total loss function L of the model can be obtained: L = aL + βL rec L = aL + βL ent (3) Where α and β are equilibrium parameters.

4. The object meta-based video anomaly detection method of claim 1, wherein, The specific process of step S4 is as follows: S41: Map each reconstructed object element back to the original image according to the record in the position list. After completing all mappings, the reconstructed images of all video frames except the last video frame are obtained. S42: Calculate the AUC score based on the difference between the reconstructed image and the original image; S43: Visualize the AUC scores. All extracted video frames in a segment of video frames to be detected are numbered sequentially, and the AUC scores of all video frames except the last video frame are calculated. When drawing the visualization, the set of numbers is used as the x-axis and the set of AUC scores is used as the y-axis to draw a line graph. The smoother the line graph, the better the detection effect. S44: Visualize and label anomalies in video frames: Subtract the original image from the reconstructed image. If the number of noise points during the subtraction exceeds a set threshold for the number of pixels of the object itself, it is determined to be an abnormal object and labeled accordingly.

Images