An Unsupervised Domain Adaptive Anomaly Detection Algorithm

Abstract

This invention discloses an unsupervised domain adaptive anomaly detection algorithm, comprising the following steps: Step 1) Experimental verification on the UCSD dataset and the ShanghaiTech Campus dataset; Step 2) Constructing an unsupervised domain adaptive anomaly detection algorithm: the model includes a pre-training module and a domain adaptation module; Step 3) Iteratively training the unsupervised domain adaptive anomaly detection algorithm model; Step 4) Obtaining the detection results of the unsupervised domain adaptive anomaly detection model. This unsupervised domain adaptive anomaly detection algorithm performs supervised pre-training on the source domain dataset and proposes a perceptual contrast loss, which can promote the reconstruction of normal samples by the video frame reconstruction network and inhibit the reconstruction of abnormal samples by the video frame reconstruction network, clearly defining the discrimination boundary between normal and abnormal events in the source domain and introducing prior knowledge from the source domain that can well define normal and abnormal events into the target domain.

Description

Technical Field

[0001] This invention relates to anomaly event detection technology in surveillance videos, and more particularly to an anomaly event detection algorithm based on unsupervised domain adaptation. Background Technology

[0002] The continuous development and widespread application of anomaly detection technology in surveillance videos plays a crucial role in promoting the development of smart surveillance. In recent years, deep learning-based anomaly detection algorithms in surveillance videos have made significant progress in improving model detection performance. Supervised anomaly detection algorithms typically rely on large amounts of labeled data, requiring manual annotation of each frame of the video before training the network model; moreover, the collected data often suffers from class imbalance, leading to poor anomaly detection results. Unsupervised anomaly detection algorithms, on the other hand, do not require data annotation. By learning the feature representations of a large number of normal events, they identify data that does not conform to the distribution patterns of normal event features as anomalies, saving significant manual annotation costs. However, because unsupervised learning-based anomaly detection algorithms are trained using only normal samples, they lack prior information about anomalies, resulting in unclear boundaries between normal and anomalies and a high risk of false positives. Furthermore, existing anomaly detection algorithms in surveillance videos lack scenario applicability; they perform well in one scenario but poorly in others.

[0003] Methods for detecting abnormal events in surveillance videos based on computer vision can be divided into traditional methods and deep learning methods. Deep learning, due to its powerful learning capabilities, is widely used in various fields. Researchers often use deep learning methods to solve many challenging problems in the field of abnormal event detection in surveillance videos. Compared with traditional algorithms, deep learning-based algorithms for detecting abnormal events in surveillance videos have superior performance and are the mainstream algorithms in this research direction.

[0004] Based on whether and how the data is labeled, anomaly detection algorithms in surveillance videos can be categorized into fully supervised learning algorithms, weakly supervised learning algorithms, and unsupervised learning algorithms. In real-world video surveillance scenarios, anomaly events are relatively few, and the imbalance between normal and anomaly event data can lead to a decline in algorithm model detection performance. Furthermore, anomaly events are diverse and inexhaustible. Unsupervised learning algorithms do not require data labeling; they learn feature representations from a large number of normal events and classify data that does not conform to the distribution patterns of normal event features as anomalies. Existing unsupervised anomaly detection algorithms are trained using only normal event data, lacking prior information about anomalies. This results in unclear boundaries between normal and anomaly events, easily leading to false positives.

[0005] To improve the ability of network models to distinguish between normal and abnormal events, Park H, Noh J, Ham B, and others proposed learning memory-guided normal and abnormal detection at the IEEE Computer Vision and Pattern Recognition Conference. By introducing a memory module, they proposed feature compactness loss and feature separation loss to train the memory module, making the normal event features in the memory module more diverse, thereby improving the discriminative ability of the network model.

[0006] However, existing exception event algorithms have the following two main drawbacks:

[0007] (1) Supervised anomaly detection algorithms usually rely on a large amount of labeled data. Before training the network model, each frame of the video needs to be manually labeled, and the collected data often has the problem of class imbalance.

[0008] (2) The abnormal event detection algorithm based on unsupervised learning only uses normal samples for training and lacks prior information on abnormal events, which leads to the problem of unclear distinction between normal and abnormal events. Summary of the Invention

[0009] The technical problem to be solved by this invention is to provide an unsupervised domain adaptive anomaly detection algorithm. In order to solve the problem that the boundary between normal and abnormal events is unclear and false detection is easy to occur because the unsupervised anomaly detection algorithm lacks prior information about anomalies, the algorithm can introduce prior knowledge in the source domain that can well define normal and abnormal events into the target domain.

[0010] To solve the above-mentioned technical problems, the present invention is implemented through the following technical solution: an anomaly event detection algorithm based on unsupervised domain adaptation, comprising the following steps:

[0011] Step 1) Conduct experimental verification on the UCSD dataset and the ShanghaiTech Campus dataset; use the ShanghaiTech Campus dataset as the source domain data, and use the UCSD Ped1 dataset and the UCSD Ped2 dataset as the target domain data respectively for the experiment; the source domain data is data with different scene or environmental conditions than the target domain data, and the training set of the source domain dataset contains both normal event video frames and abnormal event video frames, and is labeled, while the training set of the target domain dataset only contains normal event video frames and is unlabeled;

[0012] Step 2) Construct an anomaly detection algorithm based on unsupervised domain adaptation: including a pre-training module and a domain adaptation module;

[0013] The pre-training module performs supervised training on the source domain data. When reconstructing the input video frames, only normal samples are used to train the reconstruction network. In the pre-training stage, a supervised learning approach is adopted to input the reconstructed video frames, normal samples, and abnormal samples into the feature extraction network to extract corresponding features. By narrowing the distance between the features of the reconstructed video frames and the features of normal samples, and widening the distance between the features of the reconstructed video frames and the features of abnormal samples, the video frame reconstruction network is promoted to reconstruct normal samples, and the video frame reconstruction network is inhibited to reconstruct abnormal samples, thus clarifying the discrimination boundary between normal and abnormal events in the source domain.

[0014] The unsupervised domain adaptive anomaly detection algorithm uses the VGG19 pre-trained model in PyTorch as the feature extraction network. Reconstructed video frames, normal samples, and anomaly samples are input into this network, and features at different scales in layers 4, 9, 14, 23, and 32 are extracted respectively. A perceptual contrast loss is employed to reduce the distance between the features of the reconstructed video frames and the features of normal samples at corresponding scales, and to increase the distance between the features of the reconstructed video frames and the features of anomaly samples at corresponding scales. This promotes the reconstruction of normal samples by the video frame reconstruction network and inhibits the reconstruction of anomaly samples by the video frame reconstruction network, thus clearly defining the discrimination boundary between normal and anomaly events in the source domain. The formula for the perceptual contrast loss function is shown below:

[0015] ,

[0016] in, , , They represent the first The number of channels, height, and width of the feature map corresponding to the item. This indicates a normal sample. Indicates an abnormal sample. This indicates the reconstruction of video frames. This indicates the result obtained by inputting video frames into the feature extraction network. Item feature map;

[0017] In the pre-training module, the total loss function formula is as follows:

[0018] ,

[0019] in, , These represent the weight coefficients of the MSE loss function and the perceptual contrast loss function, respectively. During the pre-training process, the parameters of the feature extraction network remain unchanged, while the parameters of the reconstruction network are continuously updated and optimized.

[0020] The target domain is trained using unsupervised learning. Normal samples from the target domain and the source domain are input into the pre-trained reconstruction network for training. The entire training process is based on adversarial learning. The video frame reconstruction network and the domain discriminator are trained alternately to continuously optimize their parameters. This allows the video frame reconstruction network to learn domain-invariant features that perform well in both domains, thereby aligning the data distributions of the two domains and enabling the domain discriminator to determine whether the input video frame comes from the source domain or the target domain.

[0021] Step 3) The pre-training module uses a loss function Backpropagation is performed to update the weight parameters of the reconstruction network. The training process for the target domain is as follows: when training the domain discriminator, the video frame reconstruction network is fixed so that the domain discriminator can distinguish whether the data comes from the source domain or the target domain; when training the video frame reconstruction network, the domain discriminator is fixed so that the video frame reconstruction network generates results that the domain discriminator cannot distinguish. Through backpropagation, the parameters of the domain discriminator and the video frame reconstruction network are continuously updated and optimized, and finally, the data distribution of the target domain is aligned with the data distribution of the source domain.

[0022] Step 4) Obtain the detection results of the unsupervised domain adaptive anomaly detection model: Use the test sample set as input to the trained unsupervised domain adaptive anomaly detection model for forward inference to obtain the detection results for each test sample.

[0023] Furthermore, the aforementioned , These represent the weight coefficients of the MSE loss function and the perceptual contrast loss function, respectively. The loss function replaces the MSE loss function to minimize the difference between the predicted video frames and the actual video frames.

[0024] Furthermore,

[0025] The loss function for training the domain discriminator is shown in the following formula.

[0026] ,

[0027] The loss function for training the video frame reconstruction network is shown in the formula below. ,

[0028] in, Represents source domain data, Represents target domain data. This indicates a video frame reconstruction network. Discriminator for the domain, The label representing the source domain data is defined as 0. The label representing the target domain data is defined as 1. The binary cross-entropy loss function is represented by the following formula:

[0029] ,

[0030] in, Indicates the tag value. This represents the actual predicted value. This represents the weighting coefficient.

[0031] Furthermore, in the target domain, an unsupervised learning approach is adopted through an unsupervised domain adaptive module. This approach only requires training on normal samples in the target domain. The prior knowledge from the pre-training module that can define normal and abnormal events is introduced into the target domain, clarifying the discrimination boundary between normal and abnormal events in the target domain. Moreover, through adversarial training, the distribution of source domain data and target domain data is aligned, reducing domain shift.

[0032] Compared with existing technologies, the advantages of this invention are as follows: This unsupervised domain adaptive anomaly detection algorithm first performs supervised pre-training on the source domain dataset and proposes a perceptual contrast loss. By narrowing the distance between the reconstructed video frame features and the normal sample features, and widening the distance between the reconstructed video frame features and the anomaly sample features, it promotes the reconstruction of normal events by the video frame reconstruction network and inhibits the reconstruction of anomalies by the video frame reconstruction network, thus clarifying the discrimination boundary between normal and anomalies in the source domain. Then, in the target domain, unsupervised learning is adopted. Through adversarial learning-based unsupervised domain adaptation, the prior knowledge that can well define normal and anomalies in the pre-training stage is introduced into the target domain, clarifying the discrimination boundary between normal and anomalies in the target domain and improving the anomaly detection performance of the algorithm model in the target domain. Finally, through adversarial training, the distribution of source domain data and target domain data is aligned, reducing domain offset and making the algorithm model more applicable to different scenarios. The prior knowledge that can well define normal and anomalies is introduced into the target domain. Attached Figure Description

[0033] The invention will be further described below with reference to the accompanying drawings.

[0034] Figure 1 This is a schematic diagram of the abnormal event detection algorithm based on unsupervised domain adaptation according to the present invention. Detailed Implementation

[0035] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0036] An anomaly detection algorithm based on unsupervised domain adaptation, such as Figure 1 As shown, it includes the following steps:

[0037] Step 1) Conduct experimental verification on the UCSD dataset and the ShanghaiTech Campus dataset; use the ShanghaiTech Campus dataset as the source domain data, and use the UCSD Ped1 dataset and the UCSD Ped2 dataset as the target domain data respectively for the experiment; the source domain data is data with different scene or environmental conditions than the target domain data, and the training set of the source domain dataset contains both normal event video frames and abnormal event video frames, and is labeled; while the training set of the target domain dataset only contains normal event video frames and is unlabeled;

[0038] Step 2) Construct an anomaly detection algorithm based on unsupervised domain adaptation: This model includes a pre-training module and a domain adaptation module;

[0039] The pre-training module performs supervised training on the source domain data. When reconstructing the input video frames, only normal samples are used to train the reconstruction network. In the pre-training stage, a supervised learning approach is adopted to input the reconstructed video frames, normal samples, and abnormal samples into the feature extraction network to extract corresponding features. By narrowing the distance between the features of the reconstructed video frames and the features of normal samples, and widening the distance between the features of the reconstructed video frames and the features of abnormal samples, the reconstruction of normal samples by the video frame reconstruction network can be promoted, while the reconstruction of abnormal samples by the video frame reconstruction network can be suppressed, thus clarifying the discrimination boundary between normal and abnormal events in the source domain.

[0040] The unsupervised domain adaptive anomaly detection algorithm uses the VGG19 pre-trained model in PyTorch as the feature extraction network. Reconstructed video frames, normal samples, and anomaly samples are input into this network, and features at different scales in layers 4, 9, 14, 23, and 32 are extracted respectively. A perceptual contrast loss is proposed, which reduces the distance between the features of the reconstructed video frames and the features of normal samples at corresponding scales, and increases the distance between the features of the reconstructed video frames and the features of anomaly samples at corresponding scales. This promotes the reconstruction of normal samples by the video frame reconstruction network and inhibits the reconstruction of anomaly samples by the video frame reconstruction network, thus clarifying the discrimination boundary between normal and anomaly events in the source domain. The formula for the perceptual contrast loss function is shown below:

[0041]

[0042] in, , , They represent the first The number of channels, height, and width of the feature map corresponding to the item. This indicates a normal sample. Indicates an abnormal sample. This indicates the reconstruction of video frames. This indicates the result obtained by inputting video frames into the feature extraction network. The feature map is constrained by the perceptual contrast loss, so that the reconstructed video frame is more similar to the normal sample in terms of both texture details and semantic information, and is more different from the abnormal sample.

[0043] Therefore, in the pre-training module, the formula for the total loss function is as follows:

[0044]

[0045] in, , These represent the weight coefficients of the MSE loss function and the perceptual contrast loss function, respectively, or using... The loss function replaces the MSE loss function to minimize the difference between the predicted video frame and the real video frame. During the pre-training process, the parameters of the feature extraction network remain unchanged, while the reconstruction network continuously updates and optimizes its parameters. This promotes the reconstruction of normal samples by the video frame reconstruction network, inhibits the reconstruction of abnormal samples by the video frame reconstruction network, and clarifies the discrimination boundary between normal and abnormal events in the source domain.

[0046] The abnormal event detection algorithm of this invention is based on the generative adversarial approach for unsupervised domain adaptation. By alternating training of the video frame reconstruction network and the domain discriminator, the parameters of the video frame reconstruction network and the domain discriminator are continuously optimized, so that the video frame reconstruction network can learn domain-invariant features that perform well in both domains, thereby aligning the data distribution of the two domains, and enabling the domain discriminator to distinguish as much as possible whether the input video frame comes from the source domain or the target domain.

[0047] Unsupervised learning is employed in the target domain, with normal samples from both the target and source domains input into a pre-trained reconstruction network for joint training. The entire training process is based on adversarial learning, with the video frame reconstruction network and the domain discriminator trained alternately to continuously optimize them and align the data distributions between the source and target domains. Specifically, when training the domain discriminator, the video frame reconstruction network is fixed to enable it to distinguish as much as possible whether the data comes from the source or target domain; when training the video frame reconstruction network, the domain discriminator is fixed to enable it to generate results that the domain discriminator cannot distinguish.

[0048] The loss function for training the domain discriminator is shown in the following formula.

[0049]

[0050] The loss function for training the video frame reconstruction network is shown in the formula below. Through backpropagation, the parameters of the domain discriminator and the video frame reconstruction network are continuously updated and optimized, ultimately aligning the target domain data distribution with the source domain data distribution.

[0051]

[0052] in, Represents source domain data, Represents target domain data. This indicates a video frame reconstruction network. Discriminator for the domain, The label representing the source domain data is defined as 0. The label representing the target domain data is defined as 1. The binary cross-entropy loss function is represented by the following formula:

[0053]

[0054] in, Indicates the tag value. This represents the actual predicted value. Indicates the weighting coefficient;

[0055] By using the unsupervised domain adaptive module, unsupervised learning is adopted in the target domain. Only normal samples in the target domain need to be trained. The prior knowledge in the pre-training module that can well define normal and abnormal events can be introduced into the target domain, clarifying the discrimination boundary between normal and abnormal events in the target domain, improving the abnormal event detection performance of the algorithm model in the target domain, and through adversarial training, the distribution of source domain data and target domain data can be aligned, reducing domain offset and making the algorithm model more applicable to different scenarios.

[0056] Step 3) Iteratively train the anomaly detection algorithm model based on unsupervised domain adaptation: A loss function is used in the pre-training module. Backpropagation is performed to update the weight parameters of the reconstruction network; the video frame reconstruction network in the target domain uses a loss function. Perform backpropagation to update the parameters of the domain discriminator and the video frame reconstruction network;

[0057] Step 4) Obtain the detection results of the unsupervised domain adaptive anomaly detection model: Use the test sample set as input to the trained unsupervised domain adaptive anomaly detection model for forward inference to obtain the detection results for each test sample.

[0058] This unsupervised domain-adaptive anomaly detection algorithm first performs supervised pre-training on the source domain dataset and proposes a perceptual contrastive loss. By narrowing the distance between the reconstructed video frame features and the normal sample features, and widening the distance between the reconstructed video frame features and the anomaly sample features, it promotes the reconstruction of normal events by the video frame reconstruction network and inhibits the reconstruction of anomalies by the video frame reconstruction network, thus clarifying the discrimination boundary between normal and anomalies in the source domain. Then, in the target domain, unsupervised learning is adopted. Through adversarial learning-based unsupervised domain adaptation, the prior knowledge that can well define normal and anomalies in the pre-training stage is introduced into the target domain, clarifying the discrimination boundary between normal and anomalies in the target domain and improving the anomaly detection performance of the algorithm model in the target domain. Finally, through adversarial training, the distribution of source and target domain data is aligned, reducing domain shift and making the algorithm model more applicable to different scenarios. The prior knowledge that can well define normal and anomalies is introduced into the target domain.

[0059] It should be emphasized that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. An anomaly detection algorithm based on unsupervised domain adaptation, characterized in that, Includes the following steps: Step 1) Conduct experimental verification on the UCSD dataset and the ShanghaiTech Campus dataset; use the ShanghaiTech Campus dataset as the source domain data, and use the UCSD Ped1 dataset and the UCSD Ped2 dataset as the target domain data respectively for the experiment; the source domain data is data with different scene or environmental conditions than the target domain data, and the training set of the source domain dataset contains both normal event video frames and abnormal event video frames, and is labeled, while the training set of the target domain dataset only contains normal event video frames and is unlabeled; Step 2) Construct an anomaly detection algorithm based on unsupervised domain adaptation: including a pre-training module and a domain adaptation module; The pre-training module performs supervised training on the source domain data. When reconstructing the input video frames, only normal samples are used to train the reconstruction network. In the pre-training stage, a supervised learning approach is adopted to input the reconstructed video frames, normal samples, and abnormal samples into the feature extraction network to extract corresponding features. By narrowing the distance between the features of the reconstructed video frames and the features of normal samples, and widening the distance between the features of the reconstructed video frames and the features of abnormal samples, the video frame reconstruction network is promoted to reconstruct normal samples, and the video frame reconstruction network is inhibited to reconstruct abnormal samples, thus clarifying the discrimination boundary between normal and abnormal events in the source domain. The unsupervised domain adaptive anomaly detection algorithm uses the VGG19 pre-trained model in PyTorch as the feature extraction network. Reconstructed video frames, normal samples, and anomaly samples are input into this network, and features at different scales in layers 4, 9, 14, 23, and 32 are extracted respectively. A perceptual contrast loss is employed to reduce the distance between the features of the reconstructed video frames and the features of normal samples at corresponding scales, and to increase the distance between the features of the reconstructed video frames and the features of anomaly samples at corresponding scales. This promotes the reconstruction of normal samples by the video frame reconstruction network and inhibits the reconstruction of anomaly samples by the video frame reconstruction network, thus clearly defining the discrimination boundary between normal and anomaly events in the source domain. The formula for the perceptual contrast loss function is shown below: ， in, , , They represent the first The number of channels, height, and width of the feature map corresponding to the item. This indicates a normal sample. Indicates an abnormal sample. This indicates the reconstruction of video frames. This indicates the result obtained by inputting video frames into the feature extraction network. Item feature map; In the pre-training module, the total loss function formula is as follows: ， in, , These represent the weight coefficients of the MSE loss function and the perceptual contrast loss function, respectively. During the pre-training process, the parameters of the feature extraction network remain unchanged, while the parameters of the reconstruction network are continuously updated and optimized. The target domain is trained using unsupervised learning. Normal samples from the target domain and the source domain are input into the pre-trained reconstruction network for training. The entire training process is based on adversarial learning. The video frame reconstruction network and the domain discriminator are trained alternately to continuously optimize their parameters. This allows the video frame reconstruction network to learn domain-invariant features that perform well in both domains, thereby aligning the data distributions of the two domains and enabling the domain discriminator to determine whether the input video frame comes from the source domain or the target domain. Step 3) The pre-training module uses a loss function Backpropagation is performed to update the weight parameters of the reconstruction network. The training process for the target domain is as follows: when training the domain discriminator, the video frame reconstruction network is fixed so that the domain discriminator can distinguish whether the data comes from the source domain or the target domain; when training the video frame reconstruction network, the domain discriminator is fixed so that the video frame reconstruction network generates results that the domain discriminator cannot distinguish. Through backpropagation, the parameters of the domain discriminator and the video frame reconstruction network are continuously updated and optimized, and finally, the data distribution of the target domain is aligned with the data distribution of the source domain. Step 4) Obtain the detection results of the unsupervised domain adaptive anomaly detection model: Use the test sample set as input to the trained unsupervised domain adaptive anomaly detection model for forward inference to obtain the detection results for each test sample.

2. The anomaly detection algorithm based on unsupervised domain adaptation according to claim 1, characterized in that, The , These represent the weight coefficients of the MSE loss function and the perceptual contrast loss function, respectively. The loss function replaces the MSE loss function to minimize the difference between the predicted video frames and the actual video frames.

3. The anomaly detection algorithm based on unsupervised domain adaptation according to claim 1, characterized in that, The loss function for training the domain discriminator is shown in the following formula. ， The loss function for training the video frame reconstruction network is shown in the formula below. , in, Represents source domain data, Represents target domain data. This indicates a video frame reconstruction network. Discriminator for the domain, The label representing the source domain data is defined as 0. The label representing the target domain data is defined as 1. The binary cross-entropy loss function is represented by the following formula: ， in, Indicates the tag value. This represents the actual predicted value. This represents the weighting coefficient.

4. The anomaly detection algorithm based on unsupervised domain adaptation according to claim 1, characterized in that, The target domain adopts an unsupervised learning approach through an unsupervised domain adaptation module. It only needs to train on normal samples in the target domain, introduces the prior knowledge in the pre-training module that can define normal and abnormal events into the target domain, clarifies the discrimination boundary between normal and abnormal events in the target domain, and aligns the distribution of source domain data and target domain data through adversarial training to reduce domain shift.

Images