A weakly supervised video anomaly detection method and system based on denoising and debiasing

Abstract

The application relates to the technical field of video understanding, and particularly discloses a weakly supervised video anomaly detection method and system based on denoising and deviation removal, which comprises the following steps: calculating anomaly scores of positive and negative package instances, screening top-k instances and constructing instance pairs; calculating loss values of the instance pairs, and dividing the instance pairs into a low-loss reserved set and a high-loss candidate noise set; performing secondary evaluation of the instance pairs in the high-loss candidate noise set in the semantic level by using a pre-trained visual language model, identifying and recalling abnormal samples that are wrongly filtered, and obtaining a recall set; and jointly training the low-loss reserved set and the recall set to obtain an anomaly detection model used for video frame-level anomaly event positioning. Through the dynamic denoising stage and the deviation removal stage based on the visual language model, the application effectively reduces the influence of noise segments on the training process, recovers difficult abnormal samples with discriminative value, and improves the stability and robustness of anomaly positioning.

Description

Technical Field

[0001] This invention relates to the field of video understanding technology, and in particular to a weakly supervised video anomaly detection method and system based on denoising and bias removal. Background Technology

[0002] Video anomaly detection is a technical task that automatically identifies anomalous behaviors or events from long-term video sequences. Its goal is to distinguish between anomalous and normal behavior in complex dynamic scenes without pre-defining specific anomaly categories. This technology has significant value in applications such as public safety monitoring, traffic management, industrial production monitoring, smart cities, and drone inspection. In practical applications, anomalous events typically occur infrequently, have variable durations, and are diverse in type. Accurate anomaly labeling of video data frame-by-frame or time-segment by segment requires substantial manual intervention, resulting in high labeling costs. For these reasons, fully supervised video anomaly detection methods are limited in large-scale practical deployments. Weakly supervised video anomaly detection methods rely solely on video-level labels for training; that is, they only label whether the video contains anomalous events without providing the specific time and location of the anomaly. This significantly reduces labeling costs while maintaining detection performance and is gradually becoming an important research direction in this field.

[0003] Existing weakly supervised video anomaly detection methods typically employ a multi-instance learning framework, dividing a video into multiple time segments and treating the entire video as a single instance package. Within this framework, the model infers which segments might contain anomalous behavior by scoring them as anomalies. Multi-instance learning assumes that anomalous videos contain at least one anomalous segment, while all segments in normal videos are considered normal. This assumption provides a feasible modeling approach for anomaly localization under weak supervision. In the multi-instance learning framework, the selection of anomalous segments usually depends on the model's predicted anomaly score ranking. However, in the early stages of training, the model has not yet developed stable discriminative abilities, and the estimation of the segment's anomalousness level is highly uncertain. In videos containing anomalies, many normal segments may be incorrectly assigned high anomaly scores by the model, thus being selected as anomalous samples for model training. These incorrectly labeled normal segments introduce inaccurate supervision signals during training, forming noise samples that interfere with the model's learning of true anomalous patterns. As training progresses, these noise samples may repeatedly participate in model optimization, causing the model to gradually favor learning significant appearance or motion features unrelated to anomalies, weakening its ability to discriminate true anomalous behavior.

[0004] Furthermore, under weak supervision, some real-world anomalous behaviors are subtle or visually highly similar to normal behaviors. These anomalous segments often exhibit high prediction uncertainty during training, and their distribution may highly overlap with noisy samples. Existing methods, by simply filtering samples based on anomalous scores or loss values, easily discard these discriminative anomalous segments. This leads to the model focusing excessively on easily distinguishable simple samples during training, neglecting the learning of complex anomalous behaviors, thus limiting the model's generalization ability and robustness in complex scenarios.

[0005] Therefore, existing weakly supervised video anomaly detection methods face technical challenges under multi-instance learning frameworks, such as severe interference from noisy samples and difficulty in distinguishing noisy samples from difficult anomaly samples, which restricts further improvement in anomaly detection performance. How to effectively suppress the influence of noisy supervision under weak supervision conditions while retaining sample information that is of significant value for anomaly discrimination remains a pressing technical challenge. Summary of the Invention

[0006] The present invention aims to solve the above-mentioned problems. To this end, the present invention provides a weakly supervised video anomaly detection method and system based on denoising and bias removal. By dynamically denoising and bias removal based on a visual language model, the method effectively reduces the impact of noisy segments on the training process and recovers difficult anomaly samples with discriminative value, thereby improving the stability and robustness of anomaly localization.

[0007] This invention provides a weakly supervised video anomaly detection method based on denoising and bias removal, the technical solution of which includes: S1: Acquire video data and construct a multi-instance learning positive and negative packet; S2: Calculate the anomaly score of the instances of the positive and negative packets, filter the top-k instances based on the anomaly score, and construct instance pairs; S3: Denoising stage: Calculate the loss value of the instance pair, and divide the instance pair into a low-loss retention set and a high-loss candidate noise set based on the loss value; S4: Debiasing stage: The pre-trained visual language model is used to perform a second semantic evaluation on the instance pairs in the high-loss candidate noise set, identify and recall the abnormal samples that were incorrectly filtered, and obtain the recall set. S5: Jointly train the model using the low-loss retention set and the recall set, calculate the final total loss, and iteratively update the anomaly detection model parameters through backpropagation until the model converges, thereby obtaining an anomaly detection model for video frame-level anomaly event localization.

[0008] Furthermore, during each training round, the instance pairs are sorted according to their loss values, and a high-loss candidate noise set and a low-loss retention set are selected from the instance pairs based on the dropout rate that dynamically changes with the training iteration. For the high-loss candidate noise set, a visual language model is used for secondary evaluation, and a recall set and a dropout set are selected based on the dynamically changing recall rate, wherein the dropout rate and recall rate are gradually adjusted with the training iteration.

[0009] Furthermore, in step S3, the formula for calculating the discard rate is: in, For the discard rate, The total number of iterations within a training round. The preset maximum discard rate, This represents the number of iterations.

[0010] Furthermore, in step S4, a pre-trained visual language model is used to identify the incorrectly filtered abnormal samples, and a portion of the incorrectly filtered abnormal samples are added to the recall set according to the recall rate.

[0011] Furthermore, recall rate The calculation formula is: in, For the preheating period, To achieve the maximum recall rate, The total number of iterations within a training round. This represents the number of iterations.

[0012] Furthermore, the process of performing a semantic-level secondary evaluation of instance pairs in the high-loss candidate noise set using a pre-trained visual language model includes: Extract keyframes of instance pairs from the high-loss candidate noise set; By comparing the degree of anomaly between two keyframes using a pre-trained visual language model, the anomaly inference results of the keyframes are obtained. Identify the incorrectly filtered anomalous samples based on the keyframe anomaly inference results.

[0013] Furthermore, the keyframes of an instance pair include keyframes for normal instances and keyframes for abnormal instances; Keyframe anomaly inference results include: keyframes of anomaly instances are more likely to contain anomalous events, keyframes of normal instances are more likely to contain anomalous events, and keyframes that are difficult to distinguish are more likely to contain anomalous events. When the visual language model determines that the keyframes of an abnormal instance are more likely to contain abnormal events, the instance pair is determined to be an abnormal sample that has been incorrectly filtered; if the visual language model determines that the keyframes of a normal instance are more likely to contain abnormal events or it is difficult to distinguish the keyframes that are more likely to contain abnormal events, the instance pair is determined to be a noise sample.

[0014] Furthermore, for each instance pair in the high-loss candidate noise set, an intermediate frame is extracted from the corresponding video segment as a keyframe.

[0015] Furthermore, in step S2, anomaly scores for all instances of the positive and negative packets are calculated based on the anomaly detection model; For each positive and negative sample pack, select the top-k instances with the highest outlier scores; A positive sample bag and a negative sample bag are paired up, with each instance in the positive sample bag corresponding one-to-one with the instance in the negative sample bag, forming an instance pair; Anomaly detection models include MGFN, RTFM, TEVAD, Sultani et al., or UR-DMU.

[0016] This invention also provides a weakly supervised video anomaly detection system based on denoising and bias removal, the technical solution of which includes: The preprocessing module is used to acquire video data and construct multi-instance learning positive and negative packets; The instance pair construction module is used to calculate the anomaly score of the instances of the positive and negative packets, filter the top-k instances based on the anomaly score, and construct instance pairs; A denoising module is used to calculate the loss value of the instance pair and divide the instance pair into a low-loss retention set and a high-loss candidate noise set based on the loss value; The bias removal module is used to perform a semantic-level secondary evaluation of instance pairs in the high-loss candidate noise set using a pre-trained visual language model, identify and recall anomalous samples that have been incorrectly filtered, and obtain a recall set. The training module is used to jointly train the model using the low-loss retention set and the recall set, calculate the final total loss, and iteratively update the anomaly detection model parameters through backpropagation until the model converges, thereby obtaining an anomaly detection model for video frame-level anomaly event localization.

[0017] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects: 1. This invention combines the denoising stage with the bias removal stage. During the training process, suspected noise samples are first screened out based on the sample loss value, and then the screened samples are evaluated a second time. This reduces noise interference while retaining real abnormal samples that may be mistakenly filtered out, thus alleviating the supervision noise problem caused by inaccurate selection of pseudo-abnormal samples in the multi-instance learning framework.

[0018] 2. This invention introduces a joint adjustment mechanism for dynamic dropout rate and recall rate during the training process. As training iterations progress, the dropout rate gradually increases to enhance the filtering ability of noisy samples, while the recall rate gradually improves to recover difficult anomaly samples, thereby achieving a balance between suppressing noise sample interference and preserving abnormal information. In the bias correction stage, this invention utilizes a pre-trained visual language model to perform semantic-level anomaly judgment on candidate samples, thereby identifying and recalling abnormal segments that were misjudged as noise, improving the accuracy of difficult anomaly sample identification.

[0019] 3. This invention can be directly integrated into existing weakly supervised video anomaly detection baseline methods based on multi-instance learning models. Without changing the core architecture of the original benchmark model, it can stably improve the prediction accuracy of the model in various complex monitoring scenarios.

[0020] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the method flow provided by the present invention.

[0023] Figure 2 This is a schematic diagram of the framework architecture provided by the present invention.

[0024] Figure 3 This is a quantitative comparison of the results of this invention and the RTFM method on four test samples in the UCF-Crime and MSAD datasets for weakly supervised video anomaly detection tasks. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. The following embodiments are used to illustrate this invention but should not be used to limit the scope of this invention.

[0026] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0027] The following is combined with Figures 1 to 3 The present invention will be further described in detail below, providing a weakly supervised video anomaly detection method and system based on denoising and bias removal: In this embodiment, as Figure 1 and Figure 2 As shown, a weakly supervised video anomaly detection method based on denoising and bias removal is provided, including the following steps: S1: Acquire video data, preprocess the video data, and construct a multi-instance learning positive and negative package.

[0028] The input video data (including normal and abnormal videos) is preprocessed to extract data features. Each video is divided into multiple non-overlapping time segments, and the video-level labels (normal or abnormal) are used to construct a training sample structure for positive and negative packets in multi-instance learning. The positive and negative packets include negative sample packets and positive sample packets.

[0029] Specifically, the following steps are included: S11: Explanation of video-level weak supervision annotation.

[0030] Obtain an anomaly detection video dataset, where each video sample has only a video-level binary label: in, For video-level tags, This indicates a normal video. This indicates an abnormal video, for which the dataset does not contain any frame-level annotation information.

[0031] S12: Video feature extraction.

[0032] For each video sample, a pre-trained I3D model is used to extract spatiotemporal feature representations, resulting in a set of feature sequences. One feature sequence is extracted for each video sample.

[0033] S13: Video segment division.

[0034] Each video sample For a given feature sequence, divide it into Video clips that do not overlap in time: in, This is the first instance in a multi-instance learning process. This is the second instance in the multi-instance learning process. Let m be the m-th instance in the multi-instance learning. Each video segment is considered an instance in the multi-instance learning, and a feature sequence is considered a bag in the multi-instance learning.

[0035] S14: Building a multi-instance package.

[0036] Construct negative sample packets from normal video samples. Its package label is This indicates that all instances contained in the package are functioning correctly. The labels for the negative sample pack.

[0037] Construct a positive sample package from abnormal video samples. Its package label is This indicates that the package contains at least one exception instance. The label is for the positive sample package.

[0038] S2: Calculate the anomaly score of the positive and negative packet instances based on the anomaly detection model, filter the top-k instances based on the anomaly score, and construct instance pairs.

[0039] An anomaly scoring function based on the anomaly detection model is used to calculate segment-level anomaly scores for each video segment. The top-k video segments with the highest confidence in the positive and negative packets are selected as pseudo-label candidates for subsequent model training and sample selection.

[0040] Specifically, the following steps are included: S21: Anomaly scoring function.

[0041] Based on the multi-instance learning anomaly detection model (multi-instance learning baseline model), construct the corresponding anomaly scoring function. This is used to output anomaly scores for each video segment instance. Common anomaly detection models include the Sultani et al. method and the UR-DMU method.

[0042] The input consists of negative and positive sample packets of batch size b. An anomaly score is calculated using the anomaly detection model. in, For the first in a batch The outlier scores of all instances within a positive sample bag. For the first in a batch The anomaly scores of all instances within a negative sample bag. For the first One positive sample package, For the first One negative sample bag, for The anomaly score of the m-th instance. for The anomaly score of the m-th instance. For batch size.

[0043] S22: Filter instances based on abnormal scores.

[0044] For each positive and negative sample packet, sort the instances according to their anomaly scores from highest to lowest, and select the top-k instances with the highest anomaly scores. One example.

[0045] In this embodiment, Therefore, the selected instance is: in, for The instance with the highest internal anomaly score, for The instance with the highest internal anomaly score, for The maximum value of the anomaly score for all instances within the instance. for The maximum value of the anomaly score for all instances within the instance.

[0046] S23: Construct instance pairs based on the selected instances.

[0047] A positive sample bag and a negative sample bag are paired up, with each instance in the positive sample bag corresponding to an instance in the negative sample bag. These pairs are used for subsequent loss value calculation.

[0048] S3: Denoising stage based on dynamic threshold: Calculate the loss value of the instance pair, and divide the instance pair into a low-loss retention set and a high-loss candidate noise set based on the loss value.

[0049] In each training round, denoising is performed based on the current loss distribution. Based on the observation that noisy samples have higher and more volatile loss values ​​in the early stages of training, a dynamic dropout rate mechanism is introduced to adaptively filter out suspected noisy samples with high loss, ultimately resulting in a low-loss retention set and a high-loss candidate noise set. The dynamic dropout rate is the proportion of instance pairs assigned to the high-loss candidate noise set. As the number of iterations progresses within the training rounds, the dropout rate gradually increases, causing the number of retained samples to gradually decrease, while the number of samples in the high-loss candidate noise set increases accordingly.

[0050] Specifically, the following steps are included: S31: Loss value calculation.

[0051] The multi-instance learning loss function is computed based on instance pairs. Training is performed by comparing the loss functions to encourage anomalous instances to give higher anomalous scores. in, For contrastive loss function in multi-instance learning, To ensure the truncation operation is performed, the loss value is non-negative.

[0052] The loss value for each instance pair is calculated based on the contrastive loss function learned from multiple instances.

[0053] S32: Loss sequence sorting.

[0054] A training batch The loss values ​​of all instance pairs within the sequence constitute a loss sequence, which is then sorted in ascending order to obtain the sorted loss sequence.

[0055] S33: Dynamic discard rate calculation.

[0056] Introducing the number of iterations within the current training round Dynamically adjusted dropout rate The instance pairs are divided based on the discard rate, and the calculation formula is as follows: in, The total number of iterations within a training round. This is the preset maximum discard rate. The discard rate increases with the number of iterations.

[0057] This embodiment uses the Sultani et al. model as an example. Take 30, The threshold is set to 0.5, and the discard ratio increases from 0 to 0.5 with each iteration. This strategy allows the model to utilize more samples for stable learning in the early stages of each training epoch, while gradually enhancing its noise filtering capability in later stages.

[0058] S34: Sample screening.

[0059] Based on the aforementioned dropout rate, the sorted loss sequence is... Divided into low-loss retention sets and high-loss candidate noise set : in, From Starting position to number Slicing operations on the nth element, excluding the nth element. One element; For from the first element to The final slicing operation. A high-loss candidate noise set. As input for the subsequent deviation correction stage.

[0060] S4: Debiasing stage based on visual language model: Use a pre-trained visual language model to perform a semantic-level secondary evaluation on the instance pairs in the high-loss candidate noise set, identify and recall the abnormal samples that were incorrectly filtered, and obtain the recall set.

[0061] In each training round, a semantic-level secondary evaluation is performed on the high-loss candidate noise set obtained in step S3 (denoising stage). Specifically, a pre-trained visual language model is used to identify anomalies in the candidate noise samples, identifying and retrieving incorrectly filtered anomalous samples (instance pairs), and adding a portion of the difficult anomalous samples back into the recall set according to a preset recall rate. The recall rate gradually increases with the number of training iterations in each round, thereby recalling only a small number of high-confidence anomalous samples in the early stages of each training round and gradually expanding the recall range in later stages to reduce the selection bias caused by screening solely based on loss values.

[0062] Specifically, the following steps are included: S41: Keyframe extraction.

[0063] For high-loss candidate noise set For each candidate noise instance pair, the corresponding intermediate frame of the video segment is extracted from the original video dataset as a keyframe. As a representative frame of visual semantic features, among which, For the first Keyframes of a normal instance, and For the first Keyframes for each abnormal instance.

[0064] S42: Zero-shot anomaly inference for visual language models.

[0065] A conservative cue word-guided pre-trained visual language model with frozen parameters is constructed to carefully compare two keyframes to determine which is more likely to contain an anomalous event. This involves comparing the anomalousness of the two keyframes to obtain keyframe anomaly inference results. These results include: keyframes containing anomalous instances are more likely to contain anomalous events; keyframes containing normal instances are more likely to contain anomalous events; and keyframes that are difficult to distinguish are more likely to contain anomalous events.

[0066] Example prompt: You are a law enforcement analyst specializing in anomaly detection in surveillance video. You are given two keyframe images (Image 1 and Image 2). Your task is to determine which image is more likely to show an anomalous event. Return 0 if Image 1 is anomalous; return 1 if Image 2 is anomalous; return 2 if there is no high confidence level to make a judgment.

[0067] S43: Recall of abnormal samples in course learning.

[0068] When a visual language model determines that keyframes of an abnormal instance are more likely to contain abnormal events (abnormal frames) When the keyframes of a normal instance are more likely to contain anomalous events (or when it is difficult to distinguish keyframes that are more likely to contain anomalous events), they are judged as anomalous samples that have been incorrectly filtered; conversely, if the visual language model determines that keyframes of normal instances are more likely to contain anomalous events (normal frames), then the keyframes are judged to be anomalous samples that have been incorrectly filtered. Comparison of abnormal frames If the abnormality is higher or it is difficult to distinguish between high and low abnormality, it is judged as a noise sample and discarded.

[0069] A course-based learning strategy is adopted, based on the number of iterations within the current training round. Calculate recall rate The calculation formula is: in, For the preheating period, This represents the maximum recall rate.

[0070] Based on the recall rate, select the anomalous samples that were incorrectly filtered out and add them to the recall set. The number of instance pairs in the recall set is the product of the recall rate and the total number of outlier samples that were incorrectly filtered out.

[0071] This embodiment uses the Sultani et al. model as an example, with a preheating cycle. Set to 5, representing the total number of iterations within a training round. Take 30, the preset maximum recall rate Set the recall rate to 0.1. This means that the recall rate gradually increases from 0 to 0.1 in each training round, progressively identifying and recalling difficult samples that are judged as true anomalies by the visual language model.

[0072] S5: Joint Model Training: Joint training is performed using the low-loss retention set and the recall set. The total loss function is calculated, and the network parameters of the anomaly detection model learned by multiple instances are updated through backpropagation. After training, an anomaly detection model for video frame-level anomaly event localization is obtained.

[0073] The low-loss samples retained in the denoising stage are jointly trained with the hard samples recalled in the bias removal stage. The final total loss function is calculated, and the network parameters of the anomaly detection model are updated through backpropagation. After multiple rounds of training, the final converged anomaly detection model for video frame-level anomaly event localization is obtained.

[0074] Specifically, retain the low-loss set. With recall collection Merging is used to calculate the final training loss; the parameters of the anomaly detection model, such as UR-DMU and Sultani et al. anomaly detection models, are updated through backpropagation. In each iteration of a training round, steps S2-S5 are repeated: all instance pairs obtained in step S2 are processed through step S3 to obtain... and ; Obtained through step S4 ;use and Joint training and parameter updates of the anomaly detection model network are performed. After multiple rounds of training and optimization, a converged anomaly detection model for video frame-level anomaly event localization is obtained. Video data is input into the anomaly detection model for video frame-level anomaly event localization, and the video frames containing anomalies are calculated.

[0075] In this embodiment, the anomaly detection model is MGFN, RTFM, TEVAD, Sultani et al., or UR-DMU.

[0076] This method was systematically validated through comparative experiments on three widely used public weakly supervised video anomaly detection datasets: ShanghaiTech, UCF-Crime, and MSAD. These datasets cover a variety of typical monitoring scenarios, including indoor and outdoor environments, different viewpoints, and different anomaly types, comprehensively verifying the applicability of the method in real-world applications.

[0077] In the experiments, our proposed method was integrated into various baseline models for video anomaly detection based on multi-instance learning frameworks to verify its universality under different model structures and feature representations. Without changing the original baseline model's network structure and core parameter settings, only the proposed denoising and bias removal stages were introduced to optimize the model training process. All experimental results used the area under the frame-level receiver operating characteristic curve (AUC), a common metric in weakly supervised video anomaly detection, to quantitatively evaluate the model's detection performance in frame-level anomaly localization tasks. The experimental results are shown in Tables 1, 2, and 3. On three public datasets, integrating the proposed denoising and bias removal framework into different multi-instance learning baseline models resulted in a stable improvement in frame-level anomaly detection performance for each baseline model.

[0078] Table 1 Results of the ShanghaiTech Dataset

[0079] In Table 1, the baseline models selected are RTFM, TEVAD, MGFN, and UR-DMU, which are integrated with this method (D 2 After MIL, AUC improved to varying degrees.

[0080] Table 2 Results of the UCF-Crime Dataset

[0081] In Table 2, the baseline models selected are MGFN, RTFM, TEVAD, Sultani et al., and UR-DMU, which are integrated with this method (D 2 After MIL, AUC improved to varying degrees.

[0082] Table 3 Results of the MSAD dataset

[0083] In Table 3, the baseline models selected are RTFM, TEVAD, MGFN, and UR-DMU, which are integrated with this method (D 2 After MIL, AUC improved to varying degrees.

[0084] Figure 3 The experiment shown uses several typical test videos for case analysis, comparing the differences in detection results between the original anomaly detection model and the method introduced in this study. Figure 3The dataset includes one fire scene, one traffic accident scene, and two normal scenes. Each scene displays two typical frames (normal frame and abnormal frame), and the frame positions are marked on a graph. The horizontal axis of the graph represents the video frame number, and the vertical axis represents the anomaly score. Experimental results show that, for the baseline model RTFM, in abnormal videos, this method RTFM+D... 2 MIL can more accurately provide concentrated high anomaly scores within the actual anomaly occurrence time period, effectively suppressing false alarm responses in normal segments and improving the temporal consistency of anomaly localization. In normal video, this method can significantly reduce false anomaly responses, making the anomaly score curve more stable, thereby reducing false alarms.

[0085] In summary, quantitative comparative experiments and qualitative case studies on multiple publicly available weakly supervised video anomaly detection datasets confirm that this method achieves ideal results in addressing the noise supervision and training bias issues in weakly supervised video anomaly detection under a multi-instance learning framework, demonstrating good practical application value.

[0086] This embodiment also provides a weakly supervised video anomaly detection system based on denoising and bias removal, the technical solution of which is as follows: including: The preprocessing module is used to acquire video data and construct multi-instance learning positive and negative packets; The instance pair construction module is used to calculate the anomaly score of the instances of the positive and negative packets, filter the top-k instances based on the anomaly score, and construct instance pairs; A denoising module is used to calculate the loss value of the instance pair and divide the instance pair into a low-loss retention set and a high-loss candidate noise set based on the loss value; The bias removal module is used to perform a semantic-level secondary evaluation of instance pairs in the high-loss candidate noise set using a pre-trained visual language model, identify and recall anomalous samples that have been incorrectly filtered, and obtain a recall set. The training module is used to jointly train the model using the low-loss retention set and the recall set, calculate the final total loss, and iteratively update the anomaly detection model parameters through backpropagation until the model converges, thereby obtaining an anomaly detection model for video frame-level anomaly event localization.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A weakly supervised video anomaly detection method based on denoising debiasing, characterized in that, include: S1: Acquire video data and construct a multi-instance learning positive and negative packet; S2: Calculate the anomaly score of the instances of the positive and negative packets, filter the top-k instances based on the anomaly score, and construct instance pairs; S3: Calculate the loss value of the instance pair, and divide the instance pair into a low-loss retention set and a high-loss candidate noise set based on the loss value; S4: Use a pre-trained visual language model to perform a semantic-level secondary evaluation on the instance pairs in the high-loss candidate noise set, identify and recall the abnormal samples that were incorrectly filtered, and obtain the recall set. S5: Jointly train the model using the low-loss retention set and the recall set, calculate the final total loss, and iteratively update the anomaly detection model parameters through backpropagation until the model converges, thereby obtaining an anomaly detection model for video frame-level anomaly event localization.

2. The weakly supervised video anomaly detection method based on denoising and bias removal as described in claim 1, characterized in that, In each training round, the instance pairs are sorted according to their loss values, and a high-loss candidate noise set and a low-loss retention set are selected from the instance pairs based on the dropout rate that changes dynamically with the training iteration. For the high-loss candidate noise set, a visual language model is used for secondary evaluation, and a recall set and a dropout set are selected based on the dynamically changing recall rate, wherein the dropout rate and recall rate are gradually adjusted with the training iteration.

3. The weakly supervised video anomaly detection method based on de-noising and de-biasing of claim 2, wherein, In step S3, the formula for calculating the discard rate is: wherein, is a drop rate, is the total number of iterations within one training round, is a preset maximum drop rate, is the number of iterations.

4. The weakly supervised video anomaly detection method based on de-noising and de-biasing of claim 2, wherein, In step S4, a pre-trained visual language model is used to identify the incorrectly filtered abnormal samples, and a portion of the incorrectly filtered abnormal samples are added to the recall set according to the recall rate.

5. The weakly supervised video anomaly detection method based on denoising and bias removal as described in claim 4, characterized in that, Recall rate The calculation formula is: wherein, is a preheat period, is a maximum recall rate, is a total number of iterations within one training round, is an iteration number.

6. The weakly supervised video anomaly detection method based on de-noising and de-biasing of claim 1, wherein, The process of performing a semantic-level secondary evaluation of instances in a high-loss candidate noise set using a pre-trained visual language model includes: Extract keyframes of instance pairs from the high-loss candidate noise set; By comparing the degree of anomaly between two keyframes using a pre-trained visual language model, the anomaly inference results of the keyframes are obtained. Identify the incorrectly filtered anomalous samples based on the keyframe anomaly inference results.

7. The weakly supervised video anomaly detection method based on de-noising and de-biasing of claim 6, wherein, Keyframes for instance pairs include keyframes for normal instances and keyframes for abnormal instances; Keyframe anomaly inference results include: keyframes of anomaly instances are more likely to contain anomalous events, keyframes of normal instances are more likely to contain anomalous events, and keyframes that are difficult to distinguish are more likely to contain anomalous events. When the visual language model determines that the keyframes of an abnormal instance are more likely to contain abnormal events, the instance pair is determined to be an abnormal sample that has been incorrectly filtered; if the visual language model determines that the keyframes of a normal instance are more likely to contain abnormal events or it is difficult to distinguish the keyframes that are more likely to contain abnormal events, the instance pair is determined to be a noise sample.

8. The weakly supervised video anomaly detection method based on de-noising and de-biasing of claim 6, wherein, For each instance pair in the set of high-loss candidate noise, an intermediate frame is extracted from the corresponding video segment as a keyframe.

9. The weakly supervised video anomaly detection method based on de-noising and de-biasing of claim 1, wherein, In step S2, anomaly scores are calculated for all instances of the positive and negative packets based on the anomaly detection model; For each positive and negative sample pack, select the top-k instances with the highest outlier scores; A positive sample bag and a negative sample bag are paired up, with each instance in the positive sample bag corresponding one-to-one with the instance in the negative sample bag, forming an instance pair; The anomaly detection models are MGFN, RTFM, TEVAD, Sultani et al., or UR-DMU.

10. A weakly supervised video anomaly detection system based on denoising debiasing, characterized in that, A weakly supervised video anomaly detection method based on denoising and bias removal as described in any one of claims 1 to 9, comprising: The preprocessing module is used to acquire video data and construct multi-instance learning positive and negative packets; The instance pair construction module is used to calculate the anomaly score of the instances of the positive and negative packets, filter the top-k instances based on the anomaly score, and construct instance pairs; A denoising module is used to calculate the loss value of the instance pair and divide the instance pair into a low-loss retention set and a high-loss candidate noise set based on the loss value; The bias removal module is used to perform a semantic-level secondary evaluation of instance pairs in the high-loss candidate noise set using a pre-trained visual language model, identify and recall anomalous samples that have been incorrectly filtered, and obtain a recall set. The training module is used to jointly train the model using the low-loss retention set and the recall set, calculate the final total loss, and iteratively update the anomaly detection model parameters through backpropagation until the model converges, thereby obtaining an anomaly detection model for video frame-level anomaly event localization.

Images