Open world video anomaly detection method based on mixed supervision

Abstract

The invention belongs to the technical field of video anomaly detection, and particularly relates to an open world video anomaly detection method based on mixed supervision. The method comprises the following steps: acquiring a to-be-detected video, inputting the to-be-detected video into a trained video anomaly detection model for processing, and outputting a video anomaly score; the training process of the video anomaly detection model comprises the following steps: acquiring a training video, performing supervised training on the abnormal region preliminary positioning module according to the training video, and generating a predicted abnormal region; respectively extracting features from the predicted abnormal region features and the corresponding panoramic image features by adopting a feature fusion module, and carrying out feature fusion to obtain fused features; the fusion features are divided into normal features and abnormal features according to frame label types to which the fusion features belong, and unsupervised training is carried out on a feature reconstruction module by taking the normal features as a training set; the detection accuracy can be improved, and the generalization ability of the model can be improved more remarkably.

Description

Technical Field

[0001] The present invention belongs to the technical field of video anomaly detection, and in particular relates to an open-world video anomaly detection method based on hybrid supervision. Background Art

[0002] Current video anomaly detection algorithms can be categorized into three main types based on the strength of the supervisory signal. The first type is unsupervised, meaning that anomalies are not annotated, but rather indirectly detected by learning the distribution of features from normal video frames. During training, only normal data is learned. During testing, any input from a model that is not trained on the normal data is classified as anomaly. This essentially treats anomaly detection as an "out of distribution" problem. In practice, these algorithms are primarily based on self-supervised proxy tasks such as reconstruction and video prediction. For example, if a video prediction network is trained using only normal video sequences during training, ideally, during testing, when presented with anomaly video sequences not seen during training, the network will struggle to output correct predictions. Therefore, the degree of anomaly in the input video can be indirectly determined by the quality of the predictions. The second type is weakly supervised, meaning that annotation is performed at the video level. The model inputs multiple video clips and outputs anomaly scores for each video clip. The model is learned using a multi-instance ranking loss. Essentially, this treats anomaly detection as a regression problem. The third is the video anomaly detection algorithm based on full supervision. Generally, it uses the stronger supervision signal of abnormal area labeling to make the model feature extraction focus more on the area where the anomaly occurs rather than other irrelevant background areas. The extracted features are then input into the classifier for detection. Its essence is still a classification problem.

[0003] Existing research primarily suffers from the limitations of a single supervised paradigm. While unsupervised video anomaly detection can address the unbounded nature of anomaly categories, most approaches employ self-supervised proxy tasks such as reconstruction and prediction, which inherently have limitations. On the one hand, neural networks inherently have strong generalization capabilities, allowing them to reconstruct or predict anomalies that were "unseen" during training but have relatively simple semantic information, resulting in "normalization of abnormal data." On the other hand, proxy tasks such as reconstruction and prediction are inherently challenging. Even after sufficient training, models may not be able to reconstruct or predict normal data with complex semantic information, resulting in "normalization of normal data." In short, the ability to reconstruct target data effectively depends not solely on whether the data was "seen" during training, but also on factors such as the complexity of the target data's semantic information. While weakly supervised and fully supervised video anomaly detection methods label anomalies themselves, enabling the model to learn their characteristics, this can alleviate the "normalization of abnormal data" problem to a certain extent. However, it does not take into account the unbounded nature of anomaly categories: since the categories of anomalies are not exhaustive and different types of anomalies vary greatly, limited labels cannot cover all types of anomalies. Anomalies that have not been "seen" during the training phase are still difficult to detect. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention proposes an open-world video anomaly detection method based on hybrid supervision, which includes: obtaining a video to be detected and inputting it into a trained video anomaly detection model for processing, and outputting a video anomaly score;

[0005] The training process of the video anomaly detection model includes:

[0006] S1: Obtain training videos, perform supervised training on the abnormal area preliminary positioning module based on the training videos and generate predicted abnormal areas;

[0007] S2: A feature fusion module is used to extract features from the predicted abnormal area features and their corresponding panoramic image features, and then perform feature fusion to obtain fused features;

[0008] S3: The fused features are divided into normal features and abnormal features according to the frame label type. The normal features are used as the training set to perform unsupervised training on the feature reconstruction module to generate reconstructed video frame features.

[0009] Preferably, the process of performing supervised training on the abnormal area preliminary positioning module includes:

[0010] S11: Annotate abnormal objects in training videos at the pixel level to generate fine-grained labels;

[0011] S12: Generate coarse-grained rectangular region labels based on fine-grained labels;

[0012] S13: Based on the coarse-grained rectangular region labels, the target detection model is used for supervised training. The smoothing loss is calculated and the parameters are continuously adjusted according to the smoothing loss to obtain a trained abnormal region preliminary positioning module.

[0013] Preferably, in step S2, the formula for feature fusion is:

[0014]

[0015] Among them, f concat represents the fusion feature, f region Represents the predicted abnormal area characteristics, f global Represents the global context feature, and concat represents the concatenation operation.

[0016] Preferably, the process of performing unsupervised training on the feature reconstruction module includes:

[0017] S31: Encode the normal features using an encoder to obtain a query vector;

[0018] S32: The memory module generates a memory item vector and calculates a memory value of the query vector based on the memory item vector and the query vector;

[0019] S33: concatenate the query vector and the memory value and input the concatenation result into the decoder for processing to obtain the reconstructed video frame features;

[0020] S34: Update the memory item vector and calculate the reconstruction loss. Perform backpropagation based on the reconstruction loss and continuously adjust the feature reconstruction module parameters to obtain a trained feature reconstruction module.

[0021] Furthermore, the formula for calculating the matching degree based on the memory item vector and the query vector is:

[0022]

[0023] in, represents the memory value of the query vector k, w k,m Indicates the matching degree between the query vector k and the memory item vector m, q k represents the query vector k, M represents the number of memory item vectors, and p m Represents the memory item vector m.

[0024] Furthermore, the formula for updating the memory item vector is:

[0025]

[0026]

[0027] Among them, p ＇ m Indicates p m Updated memory item vector, p m Represents the memory item vector m, v k,m represents the matching similarity matrix between the query vector k and the memory item vector m, v ′ k,m represents the matching similarity matrix between the normalized query vector k and the memory item vector m, q k represents the query vector k; u m represents the query vector index set corresponding to the memory vector m. All query vectors in the set use the memory item vector m as the most similar memory item vector to themselves. K represents the number of query vectors.

[0028] Furthermore, the reconstruction loss is the weighted sum of feature reconstruction loss, query vector compactness loss, and memory item feature separation loss.

[0029] Furthermore, the query vector compactness loss is expressed as:

[0030]

[0031] Among them, L compact represents the query vector compactness loss, N represents the number of video frame sequence samples, K represents the number of query vectors, represents the query vector j in sample i, p p Representation and query vectors The memory vector with the closest distance, ‖·‖2 represents the L2 norm.

[0032] Furthermore, the loss of memory feature separation is expressed as:

[0033]

[0034] Among them, L seperate represents the loss of memory feature separation, N represents the number of video frame sequence samples, K represents the number of query vectors, represents the query vector j in sample i, p p Representation and query vectors The nearest memory vector, p n Representation and query vectors The second closest memory vector, ‖·‖2 represents the L2 norm.

[0035] Preferably, the calculation formula for the video anomaly score is:

[0036] S t =λ(1-g(PSNR(f output,f input )))+(1-λ)g(DS(g,p))

[0037]

[0038] Among them, S t represents the video anomaly score, λ represents the balance hyperparameter, g(·) is the normalization function, PSNR represents the peak signal-to-noise ratio, and f output represents the reconstructed video frame features output by the feature reconstruction module, f input represents the normal features of the input feature reconstruction module, DS represents the DS function, K represents the number of query vectors, q j Denotes the query vector j, p p Representation and query vector q j The memory vector with the closest distance, ‖·‖2 represents the L2 norm.

[0039] The present invention achieves excellent discrimination performance on the two public datasets, UBnormal and ShanghaiTech. The proposed model learns the characteristics of anomalies through supervision on an unsupervised basis. This allows for the detection of anomalies not seen during training and for better discrimination of known anomalies. This hybrid approach leverages the strengths of the two supervised paradigms, achieving superior generalization performance compared to a single supervised paradigm. Compared to existing technologies, the present invention improves detection accuracy and significantly enhances the model's generalization capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a diagram of the video anomaly detection model training framework in the present invention;

[0041] Figure 2 This is a framework diagram of the unsupervised feature reconstruction module in the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] The present invention proposes a hybrid supervision-based open-world video anomaly detection method, which includes the following contents:

[0044] Obtain the video to be detected and input it into the trained video anomaly detection model for processing, and output the video anomaly score.

[0045] like Figure 1 As shown in Figure 2, the training process of the video anomaly detection model includes:

[0046] S1: Obtain training videos, perform supervised training on the abnormal area preliminary positioning module based on the training videos, and generate predicted abnormal areas.

[0047] To obtain training videos, the Ubnormal dataset can be used. First, the fine-grained pixel-level annotations in the dataset are converted into coarse-grained abnormal region-level annotations. The initial abnormal region localization (IARL) module is trained through abnormal region supervision to obtain preliminary abnormal region localization. The specific process is as follows:

[0048] S11: Annotate abnormal objects in training videos at the pixel level to generate fine-grained labels.

[0049] The Ubnormal dataset uses masks to annotate abnormal objects at the pixel level. Normal objects and backgrounds have a mask pixel value of 0, and the mask pixel value of abnormal objects is the number of the abnormal object. Let the abnormal object number be Q k (k=1,2,…n), the mask M can be expressed as:

[0050]

[0051] The original pixel-level annotations of the dataset are binarized, and all non-zero pixels are converted to 255, thus forming a binary mask B:

[0052]

[0053] S12: Generate coarse-grained rectangular region labels based on fine-grained labels.

[0054] Based on the above fine-grained labels, coarse-grained rectangular region labels are further generated; specifically: connected region analysis is performed on the binary mask B to identify all non-zero pixel regions and obtain the bounding box R of each region. k For each region, the bounding box is defined as: R k ={(x min ,y min ,x max ,y max )|

[0055] For all (i, j) of the kth region, B(i, j) = 255} is satisfied. Then all abnormal regions in a frame are merged into a final rectangular region as the final region-level label. Specifically, if there are multiple bounding boxes The merge operation is defined as:

[0056] R merged=(x min ,y min ,x max ,y max )

[0057] Where x min ,y min , x max ,y max is defined as follows:

[0058]

[0059] S13: Based on the coarse-grained rectangular region labels, the target detection model is used for supervised training. The smoothing loss is calculated and the parameters are continuously adjusted according to the smoothing loss to obtain a trained abnormal region preliminary positioning module.

[0060] The target detection model (Faster R-CNN) is used to perform preliminary positioning of abnormal areas. The loss function in the training process is a smoothing loss, which is defined as:

[0061] L loc =smoothL1(R merged ,R predicted )

[0062] Where smooth_L1 is the commonly used smooth L1 loss function; R predicted It is the initial predicted bounding box of the abnormal area; smooth_L1 is defined as follows:

[0063]

[0064] S2: A feature fusion module is used to extract features from the predicted abnormal area features and their corresponding panoramic image features, and then perform feature fusion to obtain fused features.

[0065] After locating the abnormal area, the obtained rectangular area R predicted =(x min ,y min ,x max ,y max ) Separate the abnormal area I from the panoramic image I region :

[0066] i region =I[y min :y max ,x min :x max ]

[0067] For the separated abnormal area I region Perform feature extraction. Preferably, use the pre-trained Backbone network f to extract the feature map fregion :

[0068] f region =f(I region )

[0069] In addition, in order to obtain global context features, feature extraction is also applied to the original panoramic image I:

[0070] f glo b al =f(I)

[0071] Next, the abnormal features are fused with the global context features. Specifically, the abnormal region features f region With the global context feature f global Perform the connection operation to obtain the fusion f concat .

[0072]

[0073] Among them, concat(.) means to connect in the feature dimension to generate a fused feature vector for subsequent processing. If the initial abnormal area locator does not locate the abnormal area, then f global Repeat the connection twice to facilitate better comparison of positive and abnormal features.

[0074] S3: The fused features are divided into normal features and abnormal features according to the frame label type. The normal features are used as the training set to perform unsupervised training on the feature reconstruction module to generate reconstructed video frame features.

[0075] The fusion features are divided into normal features and abnormal features according to the frame label type. Specifically:

[0076] Assume that the video frame feature vector set Each frame sequence feature vector f i ∈

[0077] R B×C×T×H×W That is, the fusion feature f of video frame i concat , which corresponds to the frame label l i ∈{0,1}, where N represents the number of frame sequence samples, B represents the batch size, C represents the number of feature channels, F represents the number of frames in the original frame sequence corresponding to the feature, H and W represent the height and width of the feature respectively, l i =0 indicates normal frame label, l i =1 indicates abnormal frame label. In order to achieve abnormal detection, the feature set F is divided into i Divided into normal feature set F norm and abnormal feature set F abnorm .

[0078] The unsupervised feature reconstruction module is mainly divided into three parts: encoder E, memory module M and decoder D. During the training phase, only the features F corresponding to the normal video frames are used. norm To train the reconstruction model, ideally, in the test phase, the feature F corresponding to the input abnormal video frame is abnorm , will get a poor reconstruction effect. The role of the memory module is to appropriately suppress F by memorizing typical normal features. abnorm The reconstruction effect of F abnorm cannot be well reconstructed to avoid abnormal normalization. Therefore, for the model input feature F input :

[0079]

[0080] like Figure 2 As shown in Figure 2, the process of unsupervised training of the feature reconstruction module using normal features as training sets includes:

[0081] S31: Use an encoder to encode the normal features to obtain a query vector.

[0082] F input Input into the encoder E to obtain the query vector for the memory module Among them C F , H F , W F Represent the number of channels, width and height of the feature map output by encoder E respectively, and then flatten the width and height dimensions of the feature vector q to obtain the query vector set corresponding to the feature map Where K = H F ×

[0083] W F .

[0084] Q=Flatten(E(F inpit ))

[0085] S32: The memory module generates a memory item vector, and calculates a memory value of the query vector according to the memory item vector and the query vector.

[0086] The memory module generates M memory items During training, the memory module primarily reads and updates memory items. The main purpose of the read operation is to ensure that the input to decoder D is as relevant as possible to normal features. This ensures that even if the feature reconstruction module receives abnormal features, the decoder may still decode normal features, resulting in a large feature reconstruction error, which enables the model to better distinguish between normal and abnormal conditions. The main purpose of the update operation is to update the memory items in the memory module based on the input features and certain calculation rules. This ensures that the memory items in the memory module are highly linearly independent, memorizing the more typical and representative normal features, thereby enhancing the model's discriminability.

[0087] Memory module reading process:

[0088] The first step is to calculate each query vector q k and memory term vector p m The cosine similarity between them is used to calculate a two-dimensional correlation matrix of size M×K, and then the softmax function is used to calculate q in the vertical direction of the matrix. k With p m The matching degree between k,m :

[0089]

[0090] For each query vector q k , based on w k,m As a weight, for each memory item vector p in the memory module m Calculate the weighted average to get the memory value

[0091]

[0092] in, represents the memory value of the query vector k, w k,m Indicates the matching degree between the query vector k and the memory item vector m, q k represents the query vector k, M represents the number of memory item vectors, and p m Represents the memory item vector m.

[0093] S33: Concatenate the query vector and the memory value and input the concatenation result into the decoder for processing to obtain the reconstructed video frame features.

[0094] The original query vector q is converted into k It becomes a linear combination of the typical normal feature vectors stored in the memory module, so that the subsequent decoder can only reconstruct the input features based on the normal mode, and appropriately reduce the model's ability to reconstruct abnormal features. Finally, we get q k and the corresponding Connect in the channel dimension to get a size of 2C F ×H F ×W F The feature vector is input into the decoder for feature reconstruction to generate reconstructed video frame features.

[0095] S34: Update the memory item vector and calculate the reconstruction loss. Perform backpropagation based on the reconstruction loss and continuously adjust the feature reconstruction module parameters to obtain a trained feature reconstruction module.

[0096] Memory module update process:

[0097] For each memory item vector, find the query vector that takes the memory item vector as the most similar memory item vector. In this way, each memory vector will find multiple related query vectors, and then use these related query vectors to update the memory vector. m Represents the query vector index set corresponding to the mth memory vector in the memory module, through U m Update the memory vector with the query vector corresponding to the index:

[0098]

[0099] In the formula, f(·) is the L2 norm. By performing weighted summation on the relevant query vectors instead of direct summation, it is ensured that when updating the memory vector, more emphasis is placed on the query vectors that are more similar to it. Finally, p is calculated. m and q k The matching similarity matrix between them is of size M×K, and then the similarity is normalized using the softmax function in the vertical direction of the matrix:

[0100]

[0101] Finally, U m v involved in the collection k,m Perform secondary normalization:

[0102]

[0103] During the training process, the memory item vector is updated in each iteration (if the memory vector does not have any query vector as the most similar memory vector, it will not be updated).

[0104] Calculating the reconstruction loss: This paper uses the weighted sum of feature reconstruction loss, query vector compactness loss, and memory item feature separation loss as the reconstruction loss; specifically:

[0105] Feature reconstruction loss L rec It is mainly used to constrain the model so that the feature map f decoded by the decoder D outputWith the input feature map f input can be closer. The main constraint is to minimize f output With f input Q2 distance:

[0106]

[0107] Query vector compactness loss Q compact The main purpose is to force the query vector q to be closer to its corresponding closest memory item vector p, and the L2 norm is also used to constrain it:

[0108]

[0109] in, represents the query vector j in sample i, p p Representation and query vectors The memory vector with the closest distance, ‖·‖2 represents the L2 norm. p The subscript p is the same as the query vector The index of the nearest memory vector:

[0110] Since only normal video features are learned during the training phase, this compact loss can force the memory module to memorize more normal phenomena. seperate The main purpose is to force the memory vectors stored in the memory module to be highly linearly independent, so that the most typical normal conditions can be memorized while ensuring that the memory information covers more normal phenomena.

[0111]

[0112] Among them, p n Representation and query vectors The second closest memory vector, p n is the query vector The second closest memory vector:

[0113] In some preferred embodiments of the present invention, the formula for calculating the reconstruction loss L is expressed as:

[0114] L=l rec +λ c L compact +λ s L seperate

[0115] Among them, λ c and λ s L compact and L seperate The corresponding weight hyperparameters.

[0116] Back propagation is performed based on the above reconstruction loss and the parameters of the feature reconstruction module are continuously adjusted to obtain a trained feature reconstruction module.

[0117] After the video anomaly detection model is trained, the video to be detected is obtained and input into the trained video anomaly detection model for processing, and the video anomaly score is output.

[0118] The input data first passes through the supervised module (abnormal region preliminary location module), which uses the model to enhance the features of the prior knowledge of known abnormalities. After fusing the predicted abnormal region features with the global context features, it is finally input into the unsupervised module (feature reconstruction module). The final abnormality level is determined based on the output of the unsupervised module. The process of video anomaly score includes:

[0119] The first step is to calculate the unsupervised module feature reconstruction error. This is mainly to calculate the PSNR index between the input and output features of the unsupervised module to measure the difference between the two:

[0120]

[0121] Obviously, for abnormal video frame features, the reconstruction error is likely to be large, resulting in a lower PSNR value. Conversely, for normal video frame features, the reconstruction error is likely to be small, resulting in a higher PSNR value. In addition to the feature reconstruction error, the distance between the query vector obtained by encoder D and the most similar memory item is also an important reference for the anomaly score:

[0122]

[0123] For abnormal sample frame features, since they are outside the normal frame feature distribution, they may obtain a higher DS value. Conversely, for normal sample frame features, since they are within the normal frame feature distribution, they may obtain a lower DS value. The final anomaly score calculation takes into account both PSNR and DS factors, and the calculation method is as follows:

[0124] S t =λ(1-g(PSNR(f output ,f input )))+(1-λ)g(DS(q,p))

[0125] Among them, S t represents the video anomaly score, λ represents the balancing hyperparameter; g(·) is the normalization function used to convert PSNR(f output ,f input ) and DS(q,p) are limited to [0,1]; PSNR represents the peak signal-to-noise ratio, f outputrepresents the reconstructed video frame features output by the feature reconstruction module, f input represents the normal features of the input feature reconstruction module, and DS represents the DS function.

[0126] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation plans of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A hybrid supervision-based anomaly detection method for open-world videos, characterized in that include: Obtain the video to be detected and input it into the trained video anomaly detection model for processing, and output the video anomaly score; The training process of the video anomaly detection model includes: S1: Obtain training videos, perform supervised training on the abnormal area preliminary positioning module based on the training videos and generate predicted abnormal areas; S2: A feature fusion module is used to extract features from the predicted abnormal area features and their corresponding panoramic image features, and then perform feature fusion to obtain fused features; S3: The fused features are divided into normal features and abnormal features according to the frame label type. The normal features are used as the training set to perform unsupervised training on the feature reconstruction module to generate reconstructed video frame features.

2. The hybrid supervised open-world video anomaly detection method according to claim 1, characterized in that The process of supervised training of the abnormal area preliminary positioning module includes: S11: Annotate abnormal objects in training videos at the pixel level to generate fine-grained labels; S12: Generate coarse-grained rectangular region labels based on fine-grained labels; S13: Based on the coarse-grained rectangular region labels, the target detection model is used for supervised training. The smoothing loss is calculated and the parameters are continuously adjusted according to the smoothing loss to obtain a trained abnormal region preliminary positioning module.

3. The hybrid supervised open-world video anomaly detection method according to claim 1, characterized in that In step S2, the formula for feature fusion is: Among them, f concat represents the fusion feature, f region Represents the predicted abnormal area characteristics, f global Represents the global context feature, and concat represents the concatenation operation.

4. The hybrid supervised open-world video anomaly detection method according to claim 1, wherein: The process of unsupervised training of the feature reconstruction module includes: S31: Encode the normal features using an encoder to obtain a query vector; S32: The memory module generates a memory item vector and calculates a memory value of the query vector based on the memory item vector and the query vector; S33: concatenate the query vector and the memory value and input the concatenation result into the decoder for processing to obtain the reconstructed video frame features; S34: Update the memory item vector and calculate the reconstruction loss. Perform backpropagation based on the reconstruction loss and continuously adjust the feature reconstruction module parameters to obtain a trained feature reconstruction module.

5. The hybrid-supervised open-world video anomaly detection method according to claim 4, characterized in that: The formula for calculating the matching degree based on the memory item vector and the query vector is: in, represents the memory value of the query vector k, w k,m Indicates the matching degree between the query vector k and the memory item vector m, q k represents the query vector k, M represents the number of memory item vectors, and p m Represents the memory item vector m.

6. The hybrid supervised open-world video anomaly detection method according to claim 4, characterized in that: The formula for updating the memory item vector is: Among them, p ＇ m Indicates p m Updated memory item vector, p m Represents the memory item vector m, v k,m Represents the matching similarity matrix between the query vector k and the memory item vector m, v′ k,m represents the matching similarity matrix between the normalized query vector k and the memory item vector m, q k represents the query vector k; U m represents the query vector index set corresponding to the memory vector m. All query vectors in the set use the memory item vector m as the most similar memory item vector to themselves. K represents the number of query vectors.

7. The open-world video anomaly detection method based on hybrid supervision according to claim 4, wherein The characteristic is that the reconstruction loss is the weighted sum of feature reconstruction loss, query vector compactness loss and memory item feature separation loss.

8. The hybrid-supervised open-world video anomaly detection method according to claim 7, characterized in that: The query vector compactness loss is expressed as: Among them, L comPact represents the query vector compactness loss, N represents the number of video frame sequence samples, k represents the number of query vectors, represents the query vector j in sample i, p p Representation and query vectors The memory vector with the closest distance, ‖·‖2 represents the L2 norm.

9. The hybrid-supervised open-world video anomaly detection method according to claim 7, wherein: The loss of memory feature separation is expressed as: Among them, L seperate represents the loss of separation of memory item features, n represents the number of video frame sequence samples, k represents the number of query vectors, represents the query vector j in sample i, p p Representation and query vectors The nearest memory vector, p n Representation and query vectors The second closest memory vector, ‖·‖2 represents the L2 norm.

10. The hybrid-supervised open-world video anomaly detection method according to claim 1, wherein: The calculation formula for the video anomaly score is: S t =λ(1-g(PSNR(f output ,f input )))+(1-λ)g(DS(q,p)) Among them, S t represents the video anomaly score, λ represents the balance hyperparameter, g(·) is the normalization function, PSNR represents the peak signal-to-noise ratio, and f output represents the reconstructed video frame features output by the feature reconstruction module, f input represents the normal features of the input feature reconstruction module, DS represents the DS function, K represents the number of query vectors, q j Denotes the query vector j, p p Representation and query vector q j The memory vector with the closest distance, ‖·‖2 represents the L2 norm.

Images