Weakly supervised video anomaly detection method based on feature decoupling

Abstract

This invention discloses a weakly supervised video anomaly detection method based on feature decoupling. The method first collects normal and abnormal videos to construct a training dataset, and then uses a pre-trained visual-language model to obtain the embedding representations of video and text labels. Next, by constructing a local-global temporal module, video features are modeled from both local and global temporal dependencies to better capture temporal information in the video. Furthermore, the feature decoupling module effectively decouples shared features between video features and text labels, and integrates shared features of text labels into video features to enhance the representational power of video features. Finally, by constructing a classification network and training the entire neural network using a multi-instance learning method, accurate detection of video anomalies is achieved. This invention effectively captures temporal dependencies in video data through the design of the local-global temporal module, effectively decouples video features from text label features through the feature decoupling module, and improves the representational power of video features by effectively fusing shared features of video features and text labels.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and pattern recognition, specifically, it relates to a weakly supervised video anomaly detection method based on feature decoupling. Background Technology

[0002] In the field of video surveillance, anomaly detection technology is widely used in security monitoring, traffic management, and industrial production. Traditional video anomaly detection methods typically require a large amount of labeled data for model training, but obtaining large-scale labeled data in real-world scenarios is a time-consuming and labor-intensive task, especially for labeling anomalous events, which is both difficult and expensive. To address this issue, weakly supervised video anomaly detection technology has emerged.

[0003] In recent years, advanced visual models, such as C3D, I3D, and ViT, have been widely adopted in the field of weakly supervised video anomaly detection to extract features at the video frame level. These pre-trained models can capture rich visual information in videos. Subsequently, these features are trained using a binary classifier based on multiple instance learning (MIL) to achieve anomaly detection in videos.

[0004] Joo et al. (Joo HK, Vo K, Yamazaki K, et al. Clip-tsa: Clip-assisted temporal self-attention for weakly-supervised video anomaly detection[C] / / 2023 IEEE International Conference on Image Processing(ICIP).IEEE,2023:3230-3234.) only used visual features extracted by the image encoder of the CLIP model to train the network. However, this method did not utilize the semantic information in the visual features and ignored the generalization ability of the CLIP model between vision and language.

[0005] P et al. (Wu P, Zhou X, Pang G, et al. Vadclip: Adapting vision-language models for weakly supervised video anomaly detection[C] / / Proceedings of the AAAIConference on Artificial Intelligence.2024,38(6):6074-6082.) utilized the traditional classification paradigm of weakly supervised video anomaly detection and the vision-language alignment function provided by CLIP, achieving higher performance under a network optimized by both coarse-grained and fine-grained branches. This method also proposed a local-global adapter to capture the temporal dependencies between video features. However, this method did not fully explore the relationship between visual and linguistic features in the CLIP model. Furthermore, the receptive field of the local adapter in the local-global adapter was fixed, and the global adapter in the local-global adapter ignored the importance of the feature's own location.

[0006] In summary, although some progress has been made in weakly supervised video anomaly detection methods based on vision-language models, a weakly supervised video anomaly detection method with stronger temporal modeling capabilities and the ability to fully consider the relationship between visual and linguistic features is needed to achieve higher performance. Summary of the Invention

[0007] The purpose of this invention is to provide a weakly supervised video anomaly detection method based on feature decoupling. This invention effectively captures the temporal dependencies in video data through the design of a local-global temporal module, and effectively decouples video features from text label features through a feature decoupling module. By effectively fusing shared features between video features and text labels, the representational power of video features is improved. Ultimately, this enhances the accuracy and robustness of the model in detecting anomalies.

[0008] To achieve the above-mentioned invention, the present invention specifically includes the following steps:

[0009] Step 1: Collect normal and abnormal videos to build a training dataset;

[0010] Step 2: Use a pre-trained vision-language model to obtain the embedding representations of the video and video text tags;

[0011] Step 3: Construct the local-global time module;

[0012] Step 4: Construct a feature decoupling module and integrate the shared features of text tags into the video segment features;

[0013] Step 5: Construct a classification network;

[0014] Step 6: Train the entire deep neural network using multi-instance learning;

[0015] Step 7: Input the video into a deep neural network to determine if any abnormal events have occurred in the video.

[0016] Furthermore, the aforementioned weakly supervised video anomaly detection method based on feature decoupling is characterized in that, in step one, the acquisition of normal and anomalous videos to construct the training dataset refers to acquiring a series of video data with a resolution of 640×480 and a frame rate of 24fps. This video data includes both normal and anomalous cases. Each video is labeled as normal or anomalous and has a corresponding text label. If a video is labeled as normal, its text label is "normal"; if a video is labeled as anomalous, its text label will indicate the specific anomalous event, such as "stealing," "abuse," "arrest," "fighting," "shooting," etc.

[0017] Furthermore, the aforementioned weakly supervised video anomaly detection method based on feature decoupling is characterized in that, in step two, obtaining the embedding representations of the video and video text tags using a pre-trained visual-language model involves first dividing each video into several segments, each segment containing n frames, and then inputting each segment into a pre-trained visual-language model CLIP (Contrastive Language-Image Pre-training) to obtain the image embedding representations. These embedding representations are averaged to obtain segment-level features used to represent the visual information of the video. Simultaneously, the text tags corresponding to the video are input into the CLIP's frozen text tag encoder to obtain the text tag embedding representations, forming a text tag feature matrix.

[0018] Furthermore, the aforementioned weakly supervised video anomaly detection method based on feature decoupling is characterized in that, in step three, the construction of the local-global temporal module refers to the video features first generating four local features at different scales through the local temporal module, and then concatenating these features together to capture information of the video at different temporal scales. Next, the global temporal module utilizes a graph convolutional neural network to capture the global temporal dependencies of the video features.

[0019] Furthermore, the aforementioned weakly supervised video anomaly detection method based on feature decoupling is characterized in that, in step four, constructing a feature decoupling module and integrating shared features of text tags into video segment features, involves first inputting segment-level features and text tag features into a shared feature encoder to obtain shared feature representations. Then, the segment-level features are input into a private feature decoder to obtain the segment's private features; similarly, the text tag features are also input into the private feature decoder to obtain the text tag's private features. Next, the private features and shared features are input into the decoder respectively to obtain reconstructed video features and text tag features. To achieve feature decoupling, reconstruction constraints, aggregation constraints of shared features, and separation constraints of private features are introduced to ensure that the decoupled features can accurately reconstruct the original features, and that there is a certain difference between the shared features and the private features, thereby improving the feature representation capability. Finally, the shared text tag features and segment-level features are concatenated to obtain fused features.

[0020] Furthermore, the aforementioned weakly supervised video anomaly detection method based on feature decoupling is characterized in that, in step five, constructing the classification network refers to constructing a three-layer fully connected neural network to obtain the binary classification result. First, the input features are mapped through a first linear layer to a hidden layer containing 512 neurons. Then, the output of the first layer serves as the input to the second layer, which maps it to another hidden layer containing 128 neurons. Next, the third linear layer receives the output of the second layer, maps it to a single neuron, and is responsible for generating the final binary classification task output. This output value is activated by the Sigmoid function, mapping it to a value between 0 and 1.

[0021] Furthermore, the aforementioned weakly supervised video anomaly detection method based on feature decoupling is characterized in that, in step six, training the entire deep neural network using multi-instance learning refers to representing abnormal videos as positive packets, where different time periods form positive instances, and representing normal videos as negative packets, where different time periods form negative instances. During training, a positive packet and a negative packet are input into the classification network as a batch. For each instance in the positive and negative packets, the model outputs an anomaly probability. Then, the top k instances with the highest anomaly probabilities from each packet are selected, and their average score is calculated. Using the binary cross-entropy loss function, the average scores of the positive and negative packets are compared with their corresponding labels to obtain the classification loss. The total loss of the entire deep neural network consists of a weighted sum of the classification loss and the loss of the feature decoupling module.

[0022] Furthermore, the aforementioned weakly supervised video anomaly detection method based on feature decoupling is characterized in that, in step seven, inputting the video into a deep neural network to determine whether an abnormal event has occurred in the video refers to inputting the video into the trained deep neural network to obtain the anomaly classification probability of each segment, setting an anomaly threshold T, and if the anomaly probability of a video segment is greater than T, then the video segment is considered to have experienced an abnormal event; if the anomaly probability of a video segment is less than T, then the video segment is considered to be normal. Attached Figure Description

[0023] Figure 1 The main steps of a weakly supervised video anomaly detection method based on feature decoupling, as provided in this invention, are as follows.

[0024] Figure 2 This is a network structure diagram of an example of the present invention.

[0025] Figure 3 This is a structural diagram of the local-global time module as described in an example of the present invention.

[0026] Figure 4 This is a structural diagram of the feature decoupling module described in an example of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0028] The application principle of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0029] like Figure 1 As shown, the present invention implements a weakly supervised video anomaly detection based on feature decoupling, comprising the following steps:

[0030] S101, Collect normal and abnormal videos to build a training dataset;

[0031] S102, using a pre-trained vision-language model to obtain embedded representations of video and video text tags;

[0032] S103, Construct the local-global time module;

[0033] S104, Construct a feature decoupling module and integrate the shared features of text labels into the video segment features;

[0034] S105, Construct a classification network;

[0035] S106 uses multi-instance learning to train the entire deep neural network;

[0036] S107, Input the video into the deep neural network to determine if any abnormal events have occurred in the video.

[0037] The step S101, which involves collecting normal and abnormal videos to construct the training dataset, refers to collecting a series of video data with a resolution of 640×480 and a frame rate of 24fps. This video data includes both normal and abnormal cases. Each video is labeled as normal or abnormal and has a corresponding text label. If a video is labeled as normal, its text label is "normal"; if a video is labeled as abnormal, its text label will indicate the specific abnormal event, such as "stealing," "abuse," "arrest," "fighting," "shooting," etc.

[0038] Step S102, which describes obtaining the embedding representations of videos and video text tags using a pre-trained visual-language model, involves first dividing each video into several segments, each segment containing n frames. Then, each segment is input into the pre-trained visual-language model CLIP (Contrastive Language-Image Pre-training) to obtain the image embedding representations. These embedding representations are averaged to obtain segment-level features, which are used to represent the visual information of the video. Simultaneously, the text tags corresponding to the video are input into the CLIP's frozen text tag encoder to obtain the text tag embedding representations, forming a text tag feature matrix.

[0039] The construction of the local-global temporal module described in step S103 refers to the process whereby the video features are first generated into four local features at different scales through the local temporal module, and then these features are concatenated together to capture information about the video at different temporal scales. Next, the global temporal module uses a graph convolutional neural network to capture the global temporal dependencies of the video features.

[0040] Step S104, which involves constructing a feature decoupling module and integrating the shared features of text tags into video segment features, refers to first inputting segment-level features and text tag features into a shared feature encoder to obtain a shared feature representation. Then, the segment-level features are input into a private feature decoder to obtain the segment's private features; similarly, the text tag features are also input into the private feature decoder to obtain the text tag's private features. Next, the private features and shared features are input into the decoder respectively to obtain the reconstructed video features and text tag features. To achieve feature decoupling, reconstruction constraints, aggregation constraints of shared features, and separation constraints of private features are introduced to ensure that the decoupled features can accurately reconstruct the original features, and that there is a certain difference between the shared features and the private features, thereby improving the feature representation capability. Finally, the shared text tag features and segment-level features are concatenated to obtain the fused features.

[0041] The construction of the classification network described in step S105 refers to constructing a three-layer fully connected neural network to obtain the binary classification result. First, the input features are mapped through a first linear layer to a hidden layer containing 512 neurons. Then, the output of the first layer serves as the input to the second layer, which maps it to another hidden layer containing 128 neurons. Next, the third linear layer receives the output of the second layer, maps it to a single neuron, and is responsible for generating the final binary classification task output. This output value is activated by the sigmoid function, mapping it to a value between 0 and 1.

[0042] Step S106, which describes training the entire deep neural network using multi-instance learning, refers to representing abnormal videos as positive packets, where different time periods form positive instances, and representing normal videos as negative packets, where different time periods form negative instances. During training, a positive packet and a negative packet are input into the classification network as a batch. For each instance in the positive and negative packets, the model outputs an anomaly probability. Then, the top k instances with the highest anomaly probabilities from each packet are selected, and their average score is calculated. Using the binary cross-entropy loss function, the average scores of the positive and negative packets are compared with their corresponding labels to obtain the classification loss. The total loss of the entire deep neural network consists of a weighted sum of the classification loss and the loss from the feature decoupling module.

[0043] The step S107, which involves inputting the video into a deep neural network to determine whether an abnormal event has occurred in the video, refers to inputting the video into the trained deep neural network to obtain the abnormal classification probability of each segment, setting an abnormal threshold T, and considering that an abnormal event has occurred in the video segment if the abnormal probability of the video segment is greater than T, and that the video segment is normal if the abnormal probability of the video segment is less than T.

[0044] Specific embodiments of the present invention:

[0045] The main body of the method of this invention comprises seven parts: 1) collecting normal and abnormal videos to construct a training dataset; 2) using a pre-trained visual-language model to obtain the embedding representations of videos and video text tags; 3) constructing a local-global temporal module; 4) constructing a feature decoupling module and integrating the shared features of text tags into video segment features; 5) constructing a classification network; 6) using multi-instance learning to train the entire deep neural network; 7) inputting the video into the deep neural network to determine whether any abnormal events have occurred in the video;

[0046] 1. Collect normal and abnormal videos to construct a training dataset.

[0047] Collect video data at 640×480 resolution and 24fps. The training set contains M videos, denoted as... Each training video has a corresponding label, denoted as _____. Define y i =0 indicates that the video v i It's a normal video, y i =1 indicates that the video v i These are abnormal videos. Each video also has a specific abnormal text tag; there are a total of Q text tags, denoted as Q. If it is a normal video, the corresponding text label is "normal". If it is an abnormal video, the corresponding text label is the corresponding abnormal event, such as "stealing", "abuse", "arrest", "fighting", "shooting", etc.

[0048] 2. Use a pre-trained vision-language model to obtain the embedding representations of video and video text tags.

[0049] For each video v i Divide it into m segments, each segment has n frames, and denote the segment as c. i,j Where the subscripts i and j represent the j-th segment in the i-th video, and c i,j Features are obtained by inputting the frozen image encoder into the pre-trained visual-language model CLIP (Contrastive Language-Image Pre-training). Where n is the frame length of the video segment, d = 512 is the dimension size, and the CLIP model used is ViT-B / 16. The average of X is used to obtain the segment-level features. Record video features The text label feature matrix is ​​obtained by inputting Q text labels into the pre-trained CLIP frozen text label encoder.

[0050] 3. Construct a local-global time module

[0051] like Figure 3 The diagram shows the structure of the local-global event module. The local-global time module includes a local time module and a global time module. The local time module uses four 1D convolutional layers with different dilation rates to obtain four local features at different scales.

[0052] Specifically, the first 1D convolutional layer has 512 input channels, 128 output channels, a kernel size of 1, a stride of 1, and a dilation rate of 1. It pads the input sequence with one zero element on each side. The second 1D convolutional layer has 512 input channels, 128 output channels, a kernel size of 3, a stride of 1, and a dilation rate of 2. It pads the input sequence with two zero elements on each side. The third 1D convolutional layer has 512 input channels, 128 output channels, a kernel size of 3, a stride of 1, and a dilation rate of 3. It pads the input sequence with three zero elements on each side. The fourth 1D convolutional layer has 512 input channels, 128 output channels, a kernel size of 3, a stride of 1, and a dilation rate of 4. It pads the input sequence with four zero elements on each side.

[0053] The feature F of a video is input into the local temporal module to obtain four local features. The four local features are concatenated to obtain the video feature F with local temporal dependence. l The aforementioned global temporal module refers to the use of a Graph Convolutional Network (GCN) to capture the global temporal dependencies of video features. Specifically, the input of the global temporal module is the output F of the local temporal module. l First, the video features F are obtained according to formula (1). l The similarity adjacency matrix H between fragment features sim .

[0054]

[0055] Among them, I N Let be the identity matrix, and α = 0.5 be a hyperparameter used to control the sensitivity to self-similarity. Then, the video features F are obtained according to formula (2). l The distance adjacency matrix H between fragment features dis .

[0056]

[0057] The distance between the i-th segment and the j-th segment is defined by relative distance, and β = 0.5 is a hyperparameter used to control the sensitivity to its own distance. Finally, the video features F with global time dependence are obtained according to formula (3).lg .

[0058] F lg =Relu(Concat(Softmax(H) sim )F l W sim ,Softmax(H dis )F l W dis (3)

[0059] Where ReLU is the activation function, Concat(,) is the concatenation operation, and W... sim and W dis These are the learning parameters for a graph convolutional network.

[0060] 4. Construct a feature decoupling module and integrate the shared features of text labels into the video segment features.

[0061] like Figure 4 The feature decoupling module structure diagram is shown below. First, the fragment-level feature f is obtained according to formula (4). lg The similarity probability with each text label feature.

[0062]

[0063] Where τ is a temperature coefficient used to adjust the magnitude of the probability.

[0064] Selecting fragment-level features f lg The most similar text label features are denoted as

[0065] The feature decoupling module includes a shared feature encoder E. com Two private feature encoders and Two decoders D V D T , to fragment-level features f lg and text label features f t Input to shared encoder E com Shared features were obtained separately. and Fragment-level features f lg Input to private feature decoder Obtain the private features of the fragment Text label features f t Enter into the private feature decoder Obtain the private features of the text tags Next, the private features of the fragment and the private features of the text label are concatenated and then input into the decoder D. Vand D T Video features reconstructed from and reconstructed text label features To decouple frame features and text label features, the reconstruction constraints are obtained according to formula (5).

[0066]

[0067] The aggregation constraints of shared features are obtained according to formula (6).

[0068]

[0069] The separation constraints of private features are obtained according to formula (7).

[0070]

[0071] The total constraints of the feature decoupling module are obtained according to formula (8).

[0072]

[0073] Finally, the feature decoupling module obtains the text label shared features. With fragment-level features f lg The splicing results in fusion features

[0074] 5. Construct a classification network

[0075] A three-layer fully connected architecture is used for binary classification. First, the input feature vector passes through a first linear layer, which maps the input features to a hidden layer containing 512 neurons. Then, the output of the first layer serves as the input to the second linear layer, which maps it to another hidden layer containing 128 neurons. Next, the third linear layer receives the output of the second layer and maps it to a single neuron, which generates the final binary classification output. Finally, the entire output is activated by a sigmoid function, mapping the output value to between 0 and 1, representing the probability of an anomaly.

[0076] 6. Use multi-instance learning to train the entire deep neural network.

[0077] like Figure 2 This is a network structure diagram for an example of the present invention, where an abnormal video is represented as a positive packet. Different time periods form positive instances in the package. Representing a normal video as a negative packet Negative instances within the package are formed at different time periods. During training, a positive packet and a negative packet are input into the classification network as a batch, and each instance of the positive packet is assigned an anomaly probability. Each instance of the negative packet receives an anomaly probability. Take P a The top k highest probability scores Take P n The top k highest probability scores The classification loss is obtained using formula (9).

[0078]

[0079] Where BCE(·) represents the binary cross-entropy loss function. S represents a The mean, S represents a The mean of the label y a =1, y n =1.

[0080] The total loss of the entire deep neural network is obtained according to formula (10).

[0081]

[0082] Where λ is a hyperparameter used to control the weights of the loss of the feature decoupling module.

[0083] 7. Input the video into a deep neural network to determine if any abnormal events have occurred in the video.

[0084] The video is input into the trained deep neural network to obtain the anomaly classification probability of each segment. An anomaly threshold T is set. If the anomaly probability of a video segment is greater than T, it is considered that an abnormal event has occurred in the video segment. If the anomaly probability of a video segment is less than T, it is considered that the video segment is normal.

[0085] Compared with the prior art, the present invention has the following beneficial effects:

[0086] (1) This invention provides a local-global temporal module to enhance the temporal dependencies between video segment features. The local temporal module uses one-dimensional convolutional layers with different dilation rates to extract local features at different scales, thereby capturing local information of the video at different temporal scales. Secondly, the global temporal module uses a graph convolutional neural network to perform global temporal modeling of video features. By adjusting its own focus, it can better understand the global temporal dependencies between video features. This method of combining local and global temporal modeling enables the model to fully utilize the temporal information between video segments.

[0087] (2) This invention provides a feature decoupling module to fully exploit the semantic features of visual information. This module can effectively decouple video features from text tag features. At the same time, by effectively fusing video features and text shared tag features, it improves the representational power of video features. This representation includes the semantic information of both video and text tags, thereby enhancing the robustness of video features.

Claims

1. A weakly supervised video anomaly detection method based on feature decoupling, characterized in that, Step 1: Collect normal and abnormal videos to construct a training dataset: Collect a series of video data with a resolution of 640×480 and a frame rate of 24fps, including normal and abnormal cases, with each video having a corresponding text label; Step 2: Use a pre-trained visual-language model to obtain the embedding representations of the videos and video text labels: Use the pre-trained visual-language model CLIP to obtain the image features and text label embedding representations of the video segments; Step 3: Construct a local-global temporal module, the input of which is a sequence of segment-level video features arranged in chronological order. First, the input feature sequence F is fed into the local time module, which consists of four one-dimensional convolutional layers with different dilation rates. These layers extract local features at different time scales, and F is obtained through these four convolutional layers. c1 F c2 F c3 F c4 Four local feature representations are generated, and then these four features are concatenated along the channel dimension to obtain the local feature Fl. Further, the global temporal module is input to construct the adjacency matrix of the graph, and graph convolution operations are performed on the graph structure. The output of the global temporal module is the global feature representation. Step four: Construct a feature decoupling module and integrate the shared features of text labels into the video segment features. Step five: Construct a classification network: Use a three-layer fully connected neural network and output using the Sigmoid function. Step six: Train the entire deep neural network using multi-instance learning: Construct positive and negative packets for abnormal and normal videos respectively, with different time periods forming instances in each packet. During training, the network is trained using... Positive and negative packets are input into the classification network in batches, and the network outputs the anomaly probability of each instance. The top k instances with the highest anomaly probability in each packet are selected, and their average score is calculated. The average score of the packets is compared with the label using the binary cross-entropy loss function to obtain the classification loss. The total loss of the entire deep neural network consists of the weighted sum of the classification loss and the loss of the feature decoupling module. Step 7: Input the video into the deep neural network to determine whether there are any abnormal events in the video: Input the video into the trained deep neural network to obtain the anomaly classification probability of each segment. Set an anomaly threshold T. If the anomaly probability of a segment is greater than T, it is judged as an abnormal event; if it is less than T, it is judged as normal.

2. The weakly supervised video anomaly detection method based on feature decoupling according to claim 1, characterized in that... Step 3, constructing the local-global temporal module, refers to the local-global temporal module comprising a local temporal module and a global temporal module. The local temporal module uses four 1D convolutional layers with different dilation rates to obtain four local features at different scales. Specifically, the first 1D convolutional layer has 512 input channels, 128 output channels, a kernel size of 1, a stride of 1, and a dilation rate of 1, with one zero element padded on each side of the input sequence. The second 1D convolutional layer has 512 input channels, 128 output channels, and a kernel size of 3. The first layer has a stride of 1 and a dilation rate of 2, with 2 zero elements padded on each side of the input sequence. The second layer is a one-dimensional convolutional layer with 512 input channels, 128 output channels, a kernel size of 3, a stride of 1, and a dilation rate of 3, with 3 zero elements padded on each side of the input sequence. The third layer is a one-dimensional convolutional layer with 512 input channels, 128 output channels, a kernel size of 3, a stride of 1, and a dilation rate of 4, with 4 zero elements padded on each side of the input sequence. The features F of a video are input into the local temporal module to obtain four local features. The four local features are concatenated to obtain video features Fl with local temporal dependencies. The global temporal module refers to the use of a graph convolutional neural network (GCN) to capture the global temporal dependencies of the video features. Specifically, the input of the global temporal module is the output Fl of the local temporal module. l First, the video features F are obtained according to formula (1). l The similarity adjacency matrix H between fragment features sim , Among them, I N Let be the identity matrix, and α = 0.5 be a hyperparameter used to control the sensitivity to self-similarity. Then, the video features F are obtained according to formula (2). l The distance adjacency matrix H between fragment features dis , The distance between the i-th segment and the j-th segment is defined by relative distance, and β = 0.5 is a hyperparameter used to control the sensitivity to its own distance. Finally, the video features F with global time dependence are obtained according to formula (3). lg , F lg =Relu(Concat(Softmax(H sim )F l W sim ,Softmax(H dis )F l W dis )) (3) Where ReLU is the activation function, Concat(,) is the concatenation operation, and W... sim and W dis These are the learning parameters for GCN.

3. The weakly supervised video anomaly detection method based on feature decoupling according to claim 1, characterized in that... Step four, which involves constructing a feature decoupling module and integrating the shared features of text tags into video segment features, refers to first obtaining the segment-level features f according to formula (4). lg The similarity probability with each text label feature. Where τ is a temperature coefficient used to adjust the probability magnitude, selected in relation to the fragment-level feature f. lg The most similar text label features are denoted as The feature decoupling module includes a shared feature encoder E. com Two private feature encoders and Two decoders D V D T , to fragment-level features f lg and text label features f t Input to shared encoder E com Shared features were obtained separately. and Fragment-level features f lg Input to private feature decoder Obtain the private features of the fragment Text label features f t Enter into the private feature decoder Obtain the private features of the text tags Next, the private features of the fragment and the private features of the text label are concatenated and then input into the decoder D. V and D T Video features reconstructed from and reconstructed text label features To decouple frame features and text label features, the reconstruction constraints are obtained according to formula (5). The aggregation constraints for shared features are obtained according to formula (6). The separation constraints of the private features are obtained according to formula (7). The total constraints of the feature decoupling module are obtained according to formula (8). Finally, the feature decoupling module obtains the text label shared features. With fragment-level features f lg The splicing results in fusion features

Images