A video anomaly behavior recognition method based on knowledge distillation of relaxed divergence function

Abstract

This invention discloses a video anomaly behavior recognition method based on knowledge distillation using a relaxed divergence function. This method allows for a more relaxed matching of the outputs of the teacher network and the student network, thereby improving the knowledge distillation effect. The specific implementation method is as follows: First, consecutive video images are randomly extracted from the entire dataset and preprocessed as training samples. Then, these samples are fed into the proposed teacher network for training to obtain the optimal parameters. Next, the optimal parameters of the teacher network are used to distill and train the student network. Due to the significant capacity difference between the teacher network and the student network, the proposed relaxed divergence function is used to match the output confidence of the two networks. Finally, soft loss and hard loss are used as a joint loss function to backpropagate the error of the student network, thereby optimizing the parameters of the student network to achieve optimal results.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and knowledge distillation, specifically to a video abnormal behavior recognition method based on knowledge distillation using relaxed divergence functions. Background Technology

[0002] Existing knowledge distillation methods focus on baseline settings, where the teacher model and training strategy are not very robust. To address these issues, [B. Heo, J. Kim, S. Yun, H. Park, N. Kwak, and J.Y. Choi. A comprehensive overhaul of feature distillation. In Proceedings of the IEEE / CVF International Conference on Computer Vision, [J] pages 1921–1930, 2019.] proposes extracting information through multiple intermediate layers, but this requires additional convolutions for feature alignment, increasing model complexity. [Tian Y, Krishnan D, Isola P. Contrastive Representation Distillation: arXiv, 10.48550 / arXiv.1910.10699 [P]. 2019.] introduces a contrastive loss that transfers pairwise relationships, but it requires storing image features in a memory bank, incurring additional computational costs. [Mirzadeh SI, Farajtabar M, Li...] A, et al. Improved Knowledge Distillation via Teacher Assistant [C] / / 2020:5191-5198.》 suggests reducing the difference between teachers and students by adding a teaching assistant of medium model size, but cannot use a stronger model as the teacher model.

[0003] This invention proposes a video anomaly behavior recognition method based on knowledge distillation using a relaxed divergence function, which can better extract experience from a stronger teacher. Experience shows that the prediction discrepancy between a student network and a strong teacher network can be quite significant. Therefore, exact matching of KL divergence predictions can interfere with training, causing existing methods to perform poorly. In the method presented in this patent, we simply preserve the relationship between the predictions of the teacher and student networks and propose a correlation-based relaxed divergence function to explicitly capture this relationship. Experimental results on the Tobacco factory and HMDB51 datasets demonstrate that the proposed method significantly improves the model's recognition accuracy, proving its effectiveness compared to other mainstream methods. Summary of the Invention

[0004] The present invention mainly overcomes the shortcomings of the prior art. The purpose of the present invention is to provide a method based on deep learning theory and to efficiently deploy a simplified neural network model on edge devices with limited computing power.

[0005] To achieve the above technical objectives, the present invention adopts the following technical solution:

[0006] 1. A video anomaly behavior recognition method based on knowledge distillation of relaxed divergence function, characterized by the following steps:

[0007] Step 1: Create training samples X. The specific steps are as follows:

[0008] (1) Randomly select n video samples X′ with f frames from the dataset, f∈(1,64]; X′ is c×f a×b matrices, a∈(224,1080); b∈(224,1080); c is the number of channels of the video image, c∈{1,3};

[0009] (2) Resize the n video samples X′ with f frames to obtain the resized video samples X″, where X″ is c×f w×h matrices; w and h are fixed values ​​of 224; then perform maximum and minimum value normalization on X″ to obtain the normalized video samples X″′, where X″′ is c×f w×h matrices; the calculation formula is as follows:

[0010]

[0011] In the formula x″′ wh Represents a single element in matrix X″′, x″′ wh ∈(-1, 1); x″ wh Represents a single element in matrix X″′, x″ wh ∈(0, 255); x min x represents the minimum value of normalization.min =-1; x max x represents the maximum value of the normalized value. max =1;

[0012] (3) Finally, the n normalized video samples X″′ are combined into a set X as training samples, X={X″′1,X″′2,X″′3,…,X″′ n The training samples X consist of n×c×f w×h matrices; n is the number of normalized video samples X″′ in the training samples X, n∈N;

[0013] Step 2: Perform data augmentation on the training sample X to obtain the augmented sample. The specific steps are as follows:

[0014] (1) When the random probability q of the i-th training round i Greater than the trigger probability p i At that time, q i =random(0,1); trigger probability p i The calculation formula is:

[0015]

[0016] In the formula p i Let p be the trigger probability of the data augmentation algorithm in the i-th round. i ∈(0,1]; e m e represents the maximum number of training rounds. m ∈N; e i Represents the current i-th training round, e i ∈(1, e m The algorithm performs data augmentation on n normalized video samples X″′ to obtain augmented video samples. There are c×f w×h matrices; The calculation formula is as follows:

[0017]

[0018] In the formula, f is the number of frames in the normalized video sample X″′, and Z i "″′; i = {1, 2, ..., f}, representing the i-th frame image in the normalized video sample X″′, Z i "′ represents c w×h matrices; M i i = {1, 2, ..., f}, representing the i-th frame image Z in the normalized video sample X″′. i "′ Multiplication of the mask, M i There are c w×h matrices;

[0019] (2) For n enhanced video samples Combined to obtain enhanced samples The calculation formula for the enhancement algorithm is:

[0020]

[0021] In the formula, q i Let q be the random probability of the i-th training round. i =random(0,1); p i To enhance the trigger probability of the algorithm in the i-th training round; X is a training sample, and the training sample X consists of n×c×f w×h matrices; Representative of enhanced video samples, enhanced video samples There are c×f w×h matrices. Representative augmented sample, For n enhanced video samples The set that is formed There are n×c×f w×h matrices. Step 3: Construct the teacher network MSFP-NET and augment the samples. The model parameters P that achieve the highest accuracy Acc in MSFP-NET were obtained by training the model on the MSFP-NET network. t The specific steps are as follows:

[0022] (1) Construct the video abnormal behavior recognition model MSFP-NET. MSFP-NET consists of MSAI, MSFP and 3D convolution. The calculation formula is as follows:

[0023] MSFP-NET(X)=MSAI(MSFP(MSAI(Conv 3×3×3 (Conv 1×1×1 (Conv 7×7×7 (X))))))

[0024] In the formula, Conv 1×1×1 (·) represents a 1×1×1 convolution, Conv 3×3×3 (·) represents a 3×3×3 convolution, Conv 7×7×7 (·) represents a 7×7×7 convolution. MSFP stands for Multi-Scale Fusion Block, which is composed of multi-scale average pooling and 1×1×1 convolution. The calculation formula is as follows:

[0025]

[0026] In the formula, GAP(·) represents global pooling, and Conv 1×1×1 (·) represents a convolution with a 1×1×1 kernel, N is the number of scales in an MSFP, N=3; AP k×k (·) represents average pooling with a kernel of k×k×k, where k∈{3,5,7};

[0027] MSAI stands for Multi-Scale Attention Feature Block. Inception consists of 3×3 convolutions and 1×1 convolutions, and the calculation formula is as follows:

[0028] MSAI(X) = 3DInception(X + Conv) 1×1×1 (MSA(Conv 1×1×1 (X))))

[0029] In the formula, 3DInception represents a three-dimensional multi-scale feature extraction block, and 3DInception is composed of 3×3×3 convolutions. 3×3×3 (·) and 1×1×1 convolution Conv 1×1×1 The (·) component is calculated using the following formula:

[0030] 3DInception = Concat(Conv) 1×1×1 (X), Conv 3×3×3 (Conv 1×1×1 (X)), Conv 3×3×3 (Conv 1×1×1 (X)))

[0031] In the formula, Concat(X1, X2, X3) represents arranging the matrix features of features X1, X2, and X3 in order to obtain X. Cat X1, X2, and X3 are n×c×f w×h matrices; X Cat There are n×c×3×f w×h matrices;

[0032] MSA stands for Multi-Scale Attention Module, which is composed of multi-scale average pooling and convolution, and computes...

[0033] The formula is as follows:

[0034]

[0035] In the formula, DW_Conv 5×5×5 This represents a depthwise separable convolution with a kernel size of 5×5×5. i The size of the convolution kernel at the i-th scale is calculated using the following formula:

[0036]

[0037] (2) Enhance the sample The data is input into the MSFP-NET video abnormal behavior recognition model via a queue, and the model prediction matrix y is obtained by the MSFP-NET model. (t) The calculation formula is as follows:

[0038]

[0039] In the formula, the prediction matrix y (t) It is an n×s matrix, where s represents the augmented sample. Tag types; Conv 1×1×1 (·) represents a 1×1×1 convolution, Conv 3×3×3 (·) represents a 3×3×3 convolution, Conv 7×7×7 (·) represents a 7×7×7 convolution;

[0040] (3) Backpropagation of error is used, and the cross-entropy loss function is used to measure the true value. With the model prediction matrix y (t) The distance, and calculate the training loss L. t L t ∈(0, +∞), loss function formula:

[0041]

[0042] By enhancing samples For each training iteration, the stochastic gradient descent function SGD is used to make the loss value L t Backpropagation in the MSFP-NET network model allows the model parameters P in the i-th training round of MSFP-NET to be adjusted. i The formula for updating model parameters using SGD, by randomly varying the gradient in the direction of the negative gradient, is as follows:

[0043] P i =P i-1 -η▽L(P i-1 )

[0044] In the formula P i P represents the model parameters in the i-th round of training of AFG-NET, where P i-1 Let represent the model parameters in the (i-1)th round of training of AFG-NET, and η represent the optimization step size of SGD, η∈(0,1); ▽L(P i-1 The training loss L represents the training loss L on the model parameters P during the (i-1)th round of training of AFG-NET. i-1 Perform differentiation;

[0045] (5) Calculate the current training round e i Accuracy of AFG-NET model in Chinese i Acc i ∈(0,1), and store the training round e. i AFG-NET model P i e i Represents the current i-th training round, e i ∈(1, e m ]; e mFor the maximum number of training rounds, e m ∈N;

[0046]

[0047] In the formula, TP is the number of positive samples correctly predicted by the model, TN is the number of negative samples correctly predicted by the model, FP is the number of positive samples incorrectly predicted by the model, and FN is the number of negative samples incorrectly predicted by the model.

[0048] Step 4: Construct the student network AFG-NET, feed the training samples X into the student network AFG-NET for training, and use knowledge distillation to obtain the network parameters P that optimize the recognition accuracy Acc of the AFG-NET model across all training rounds. s The specific steps are as follows:

[0049] (1) Construct the video abnormal behavior recognition model AFG-NET. AFG-NET consists of AFG, miniMSFP, Inception and two-dimensional convolution. The calculation formula is as follows:

[0050] AFG-NET(X)=Inception(miniMSFP(Inception(Conv 3×3 (Conv 1×1 (Conv 7×7 (AFG(X)))))))

[0051] In the formula, Conv 1×1 (·) represents a 1×1 convolution, Conv 3×3 (·) represents a 3×3 convolution, Conv 7×7 (·) represents a 7×7 convolution, AFG represents an attention feature gate layer, and AFG consists of a global pooling GAP(·) and a multi-channel feature sparsification function sp(·), calculated as follows:

[0052] AFG(X) = GAP(X) × sp(X)

[0053] miniMSFP is a small multi-scale fusion block. It is composed of multi-scale average pooling and 1×1 convolution, and the calculation formula is as follows:

[0054]

[0055] In the formula, GAP(·) represents global pooling, and Conv 1×1 (·) represents a 1×1 kernel convolution, N is the number of scales in a miniMSFP, N=3; AP k×k (·) represents average pooling with a kernel of k×k, where k∈{3, 5, 7};

[0056] Inception represents a multi-scale feature extraction block. Inception consists of a 3×3 convolutional Conv... 3×3 (·) and 1×1 convolution Conv 1×1 The (·) component is calculated using the following formula:

[0057] Inception(X) = Concat(Conv) 1×1 (X), Conv 3×3 (Conv 1×1 (X)), Conv 3×3 (Conv 1×1 (X)))

[0058] In the formula, Concat(X1, X2, X3) represents arranging the matrix features of features X1, X2, and X3 in order to obtain X. Cat X1, X2, and X3 are n×c×f w×h matrices; X Cat There are n×c×3×f w×h matrices;

[0059] (2) Input the training sample X into the AFG-NET network model;

[0060] (3) The AFG layer in the AFG-NET network model performs global feature calculation on the training sample X to obtain the global pooling feature X. GAP Global pooling feature X GAP Let X be an n×c matrix. GAP ∈(-1, 1); the calculation formula is as follows:

[0061] X GAP =GAP(X)=X n×c ·mean(X f×w×h )

[0062] In the formula, GAP(·) represents global pooling of the f, w, and h dimensions of the training sample X, where X represents the training sample. n×c This is an n×c matrix formed by the n and c dimensions of the training samples X; mean(·) represents taking the mean, X f×w×h f w×h matrices representing the f, w, and h dimensions of the training sample X;

[0063] (4) The input layer AFG layer in the AFG-NET network model performs multi-channel feature sparsification on the training sample X to obtain the sparsified features X. SP , sparsification feature X SP It is a set of n×c w×h matrices; the calculation formula is as follows:

[0064]

[0065] In the formula, X represents the training sample, sp(·) is the multi-channel feature sparsification function, and x i It is the feature matrix of the training sample X expanded along the third dimension for f channels, and the feature matrix x is... i There are n×c w×h matrices; a i Represents the characteristic matrix x i The inhibitory factor, a i The calculation formula is as follows:

[0066]

[0067] In the formula, λ is the channel suppression ratio, λ∈{10} n}, n∈{-3,-4,-5};γ i γ is the weight scaling ratio of the BN layer of the i-th channel in the multi-channel feature sparsification function sp(·). i ∈(0,1); γ i The calculation formula is as follows:

[0068]

[0069] Conv 1×1 (x i ) represents the input of the i-th channel BN layer in the multi-channel feature sparsification function sp(·), Conv 1×1 (x i ) represents n×c w×h matrices; BN(x i ) represents the output of the i-th channel BN layer in the function sp(·), BN(x) i ) represents n×c w×h matrices; γ i The scaling weight of the BN layer for the i-th channel in the representation function F(X), γ i ∈(0,1); β i The bias weight of the i-th channel BN layer in the representative function sp(·) is β. i ∈(0,1);

[0070] (5) Using global pooling feature X GAP The n×c elements and the sparsity feature X SP The output matrix of the AFG layer is obtained by multiplying the n×c w×h matrices respectively. Output matrix There are n×c w×h matrices; the calculation formula is as follows:

[0071]

[0072] The special symbol ⊙ indicates that the elements of the previous matrix are multiplied by the corresponding matrices in the next set;

[0073] (6) Output of AFG The data is fed into AFG-NET for feature extraction, and finally the AFG-NET prediction matrix X is obtained. sout Prediction matrix X sout Let X be an n×s matrix, where s represents the number of label categories for the training sample X;

[0074] (7) The model parameter P with the highest MSFP-NET accuracy Acc in step 2. t Load it into the teacher model MSFP-NET, and use model parameters P t The MSFP-NET predicts the training samples X, obtaining the prediction matrix X of the teacher network MSFP-NET. tout X tout Let X be an n×s matrix, where s represents the number of label categories for the training samples X;

[0075] (8) Set the knowledge distillation temperature parameter τ, τ∈(0,1), and set the prediction matrix X of the teacher network MSFP-NET. tout Divide by τ and feed the result into the Softmax function to obtain the soft label Y. (t) The prediction matrix X of the student network AFG-NET sout Divide by τ and feed the result into the Softmax function to obtain the soft prediction Y. (s) The calculation formula is as follows:

[0076]

[0077] (9) The output characteristics X of the student network AFG-NET sout Divide by τ and feed the result into the Softmax function to obtain the hard prediction Y;

[0078] Y = Softmax(X) sout (10) Calculate the Koubek-Leibler divergence loss L KD The calculation formula is as follows:

[0079]

[0080] In the formula L KD For the Koubek-Leibler divergence loss, Y (t) For soft tags, Y (s) This is a soft prediction;

[0081] Find λ inter The formula for calculating the internal sparsity factor is as follows:

[0082]

[0083] In the formula, τ represents the knowledge distillation temperature parameter, n is the number of video samples X′ in the training sample X, n∈N; d p (·) represents the Pearson similarity function, and s represents the number of label categories for the training samples X.

[0084] The internal sparsity factor λ inter and Koubek-Leibler divergence loss L KD Multiplication yields the soft loss L soft The calculation formula is as follows:

[0085] L soft =λ inter *L KD

[0086] (11) The cross-entropy loss function is used as the cost function, and the L1 norm is used to sparsify the parameter γ of AFG. Similarly, the hard labels are obtained. The distance L between the hard prediction Y and the hard prediction Y hard L hard Represents hard loss, hard loss L hard The calculation formula is as follows:

[0087]

[0088] In the formula Let λg(γ) be the cross-entropy loss function, and let λg(γ) be the sparsity penalty for the scaling factor γ of the BN layer in the multi-channel feature sparsification function sp(·). λ is the scaling factor of the penalty term used to balance these two losses, and λ∈{10} n}, n∈{-3,-4,-5}; g(·) is the L1 norm penalty formula, calculated as follows:

[0089]

[0090] In the formula, f represents the number of channels in the multi-channel feature sparsification function sp(·), and γ i The scaling weight of the BN layer for the i-th channel in the function sp(·) is used; the distillation loss L is calculated using a weighted sum of hard and soft losses, as shown in the following formula:

[0091] L=χL soft +δL hard

[0092] χ=1-δ

[0093] In the formula, χ is the soft loss balance parameter, χ∈(0,1); δ is the hard loss balance parameter, δ∈(0,1); in each iteration of the dataset, gradient descent is used to adjust the weights between neurons in the student network to reduce the cost function and thus optimize the network.

[0094] (12) Through each training of training sample X, the stochastic gradient descent function SGD is used to make the distillation loss L backpropagate in the AFG-NET network model, so that the model parameters P of the i-th round of training of AFG-NET are improved. i Randomly varying the model parameters in the direction of the negative gradient to optimize the network, the formula for updating the model parameters using SGD is as follows:

[0095] P i =P i-1 -η▽L(P i-1 )

[0096] In the formula P i P represents the model parameters in the i-th round of training of AFG-NET, where P i-1 Let represent the model parameters in the (i-1)th round of training of AFG-NET, and η represent the optimization step size of SGD, η∈(0,1); ▽L(P i-1 The training loss L represents the training loss L on the model parameters P during the (i-1)th round of training of AFG-NET. i-1 Perform differentiation;

[0097] (13) Compare the model recognition accuracy Acc in each training round. i And save Acc i The highest model parameter P i And used as deployment parameter P s The calculation is as follows:

[0098]

[0099] Step 5: Use the deployment parameters P saved in Step 4 s As the network parameters of the AFG-NET model, and with the network parameters set as P s The AFG-NET model is deployed on the recognition terminal. At the same time, the terminal performs video abnormal behavior recognition on the input video and finally obtains the video abnormal behavior recognition result y, where y is a positive integer representing the abnormal behavior category of the video input, y∈[1,s].

[0100] The innovation of this invention is reflected in:

[0101] (1) Compared with the Network Slimming method, the above methods can only prune the neural network after the model is trained. The method of reducing network complexity in this invention is feature dimensionality reduction, which can be carried out in real time during network training.

[0102] (2) Compared with other knowledge distillation methods, this invention proposes an internal sparsity factor and uses the internal sparsity factor to adjust the Koubeck-Leibler divergence loss L. KD Weighting is performed to achieve sparsity L KD The purpose.

[0103] Beneficial effects:

[0104] Compared with the prior art, the present invention has the following beneficial effects:

[0105] 1. By combining the strong training strategy proposed in this invention with the strong teacher model, knowledge distillation provides stronger guidance to student networks, resulting in a better improvement in the recognition accuracy of student networks.

[0106] 2. This invention proposes a relaxed divergence function, which makes the direct instruction between the teacher network and the student network more gentle. Since the complexity of the teacher network and the student network differs by nearly 20 times in this invention, the instruction from the teacher network may be too complex for the student network. We use the relaxed divergence function for matching to improve this problem. Attached Figure Description

[0107] Figure 1 The flowchart of the SHIFTMASK data augmentation algorithm used in this invention is as follows: the main process is to establish training samples, then calculate the trigger probability, the algorithm generates a mask, the image and the mask are synthesized to obtain augmented samples, and finally the model is trained.

[0108] Figure 2 This is a structural diagram of the MSFP-NET network model proposed in this invention. The MSFP-NET network consists of MSAI blocks, 3D inception blocks, and MSFP.

[0109] Figure 3 This is a structural diagram of the MSAI block in the MSFP-NET proposed in this invention. The MSAI block consists of 3D inception, 3D convolution, MSA, and batch normalization.

[0110] Figure 4 This is a structural diagram of the MSA in the MSAI block proposed in this invention. The MSA is composed of pooling layers and convolutional layers of different sizes.

[0111] Figure 5 This is a structural diagram of the MSFP block proposed in this invention. The MSFP block is composed of pooling layers and convolutional layers of different sizes.

[0112] Figure 6 The diagram shows the structure of AFG-NET used in this invention. The ATG-NET network consists of AFG, inception blocks, and miniMSFP, and its computational complexity is only 1.66 GFLOPs, which is tens of times lower than that of other three-dimensional networks.

[0113] Figure 7This is a structural diagram of the AFG layer in the AFG-NET used in this invention. The ATG layer is composed of multiple channels connected in parallel, and the channels with small scaling sparsity are suppressed in the final output stage.

[0114] Figure 8 This is a structural diagram of the knowledge distillation method of the present invention. The knowledge distillation method of the present invention consists of MSFP-NET and AFG-NET, and its training loss consists of soft loss and hard loss. Detailed Implementation

[0115] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0116] Example:

[0117] A video anomaly behavior recognition method based on knowledge distillation of relaxed divergence function is proposed, and its implementation steps are as follows:

[0118] Step 1: Create training samples X. The specific steps are as follows:

[0119] (1) Randomly select n video samples X′ with f frames from the dataset, f∈(1,64]; X′ is c×f a×b matrices, a∈(224,1080); b∈(224,1080); c is the number of channels of the video image, c∈{1,3};

[0120] (2) Resize the n video samples X′ with f frames to obtain the resized video samples X″, where X″ is c×f w×h matrices; w and h are fixed values ​​of 224; then perform maximum and minimum value normalization on X″ to obtain the normalized video samples X″′, where X″′ is c×f w×h matrices; the calculation formula is as follows:

[0121]

[0122] In the formula x″′ wh Represents a single element in matrix X″′, x″′ wh ∈(-1, 1); x″ wh Represents a single element in matrix X″′, x″ wh ∈(0, 255); x min x represents the minimum value of normalization. min =-1; x max x represents the maximum value of the normalized value. max =1;

[0123] (3) Finally, the n normalized video samples X″′ are combined into a set X as training samples, X={X″′1,X″′2,X″′3,…,X″′ n The training samples X consist of n×c×f w×h matrices; n is the number of normalized video samples X″′ in the training samples X, n∈N;

[0124] Step 2: Perform data augmentation on the training sample X to obtain the augmented sample. The specific steps are as follows:

[0125] (1) When the random probability q of the i-th training round i Greater than the trigger probability p i At that time, q i =random(0,1); trigger probability p i The calculation formula is:

[0126]

[0127] In the formula p i Let p be the trigger probability of the data augmentation algorithm in the i-th round. i ∈(0,1]; e m e represents the maximum number of training rounds. m ∈N; e i Represents the current i-th training round, e i ∈(1, e m The algorithm performs data augmentation on n normalized video samples X″′ to obtain augmented video samples. There are c×f w×h matrices; The calculation formula is as follows:

[0128]

[0129] In the formula, f is the number of frames in the normalized video sample X″′, and Z i "″′; i = {1, 2, ..., f}, representing the i-th frame image in the normalized video sample X″′, Z i "′ represents c w×h matrices; M i i = {1, 2, ..., f}, representing the i-th frame image Z in the normalized video sample X″′. i "′ Multiplication of the mask, M i There are c w×h matrices;

[0130] (2) For n enhanced video samples Combined to obtain enhanced samples The calculation formula for the enhancement algorithm is:

[0131]

[0132] In the formula, q i Let q be the random probability of the i-th training round. i =random(0,1); p i To enhance the trigger probability of the algorithm in the i-th training round; X is a training sample, and the training sample X consists of n×c×f w×h matrices;

[0133] Representative of enhanced video samples, enhanced video samples There are c×f w×h matrices. X represents the augmented samples, where X is the number of augmented video samples. The set that is formed There are n×c×f w×h matrices.

[0134] Step 3: Construct the teacher network MSFP-NET, which will enhance the sample The model parameters P that achieve the highest accuracy Acc in MSFP-NET were obtained by training the model on the MSFP-NET network. t The specific steps are as follows:

[0135] (1) Construct the video abnormal behavior recognition model MSFP-NET. MSFP-NET consists of MSAI, MSFP and 3D convolution. The calculation formula is as follows:

[0136] MSFP-NET(X)=MSAI(MSFP(MSAI(Conv 3×3×3 (Conv 1×1×1 (Conv 7×7×7 (X))))))

[0137] In the formula, Conv 1×1×1 (·) represents a 1×1×1 convolution, Conv 3×3×3 (·) represents a 3×3×3 convolution, Conv 7×7×7 (·) represents a 7×7×7 convolution. MSFP stands for Multi-Scale Fusion Block, which is composed of multi-scale average pooling and 1×1×1 convolution. The calculation formula is as follows:

[0138]

[0139] In the formula, GAP(·) represents global pooling, and Conv 1×1×1 (·) represents a convolution with a 1×1×1 kernel, N is the number of scales in an MSFP, N=3; AP k×k (·) represents average pooling with a kernel of k×k×k, where k∈{3,5,7};

[0140] MSAI stands for Multi-Scale Attention Feature Block. Inception consists of 3×3 convolutions and 1×1 convolutions, and the calculation formula is as follows:

[0141] MSAI(X) = 3DInception(X + Conv) 1×1×1 (MSA(Conv 1×1×1 (X))))

[0142] In the formula, 3DInception represents a three-dimensional multi-scale feature extraction block, and 3DInception is composed of 3×3×3 convolutions. 3×3×3 (·) and 1×1×1 convolution Conv 1×1×1 The (·) component is calculated using the following formula:

[0143] 3DInception = Concat(Conv) 1×1×1 (X), Conv 3×3×3 (Conv 1×1×1 (X)), Conv 3×3×3 (Conv 1×1×1 (X)))

[0144] In the formula, Concat(X1, X2, X3) represents arranging the matrix features of features X1, X2, and X3 in order to obtain X. Cat X1, X2, and X3 are n×c×f w×h matrices; X Cat There are n×c×3×f w×h matrices;

[0145] MSA stands for Multi-Scale Attention Module, which is composed of multi-scale average pooling and convolution. The calculation formula is as follows:

[0146]

[0147] In the formula, DW_Conv 5×5×5 This represents a depthwise separable convolution with a kernel size of 5×5×5. i The size of the convolution kernel at the i-th scale is calculated using the following formula:

[0148]

[0149] (2) Enhance the sample The data is input into the MSFP-NET video abnormal behavior recognition model via a queue, and the model prediction matrix y is obtained by the MSFP-NET model. (t) The calculation formula is as follows:

[0150]

[0151] In the formula, the prediction matrix y(t) It is an n×s matrix, where s represents the augmented sample. Tag types; Conv 1×1×1 (·) represents a 1×1×1 convolution, Conv 3×3×3 (·) represents a 3×3×3 convolution, Conv 7×7×7 (·) represents a 7×7×7 convolution;

[0152] (3) Backpropagation of error is used, and the cross-entropy loss function is used to measure the true value. With the model prediction matrix y (t) The distance, and calculate the training loss L. t L t ∈(0, +∞), loss function formula:

[0153]

[0154] By enhancing samples For each training iteration, the stochastic gradient descent function SGD is used to make the loss value L t Backpropagation in the MSFP-NET network model allows the model parameters P in the i-th training round of MSFP-NET to be adjusted. i The formula for updating model parameters using SGD, by randomly varying the gradient in the direction of the negative gradient, is as follows:

[0155] P i =P i-1 -η▽L(P i-1 )

[0156] In the formula P i P represents the model parameters in the i-th round of training of AFG-NET, where P i-1 Let represent the model parameters in the (i-1)th round of training of AFG-NET, and η represent the optimization step size of SGD, η∈(0,1); ▽L(P i-1 The training loss L represents the training loss L on the model parameters P during the (i-1)th round of training of AFG-NET. i-1 Perform differentiation;

[0157] (5) Calculate the current training round e i Accuracy of AFG-NET model in Chinese i Acc i ∈(0,1), and store the training round e. i AFG-NET model P i e i Represents the current i-th training round, e i ∈(1, e m ]; e m For the maximum number of training rounds, e m ∈N;

[0158]

[0159] In the formula, TP is the number of positive samples correctly predicted by the model, TN is the number of negative samples correctly predicted by the model, FP is the number of positive samples incorrectly predicted by the model, and FN is the number of negative samples incorrectly predicted by the model.

[0160] Step 4: Construct the student network AFG-NET, feed the training samples X into the student network AFG-NET for training, and use knowledge distillation to obtain the network parameters P that optimize the recognition accuracy Acc of the AFG-NET model across all training rounds. s The specific steps are as follows:

[0161] (1) Construct the video abnormal behavior recognition model AFG-NET. AFG-NET consists of AFG, miniMSFP, Inception and two-dimensional convolution. The calculation formula is as follows:

[0162] AFG-NET(X)=Inception(miniMSFP(Inception(Conv 3×3 (Conv 1×1 (Conv 7×7 (AFG(X)))))))

[0163] In the formula, Conv 1×1 (·) represents a 1×1 convolution, Conv 3×3 (·) represents a 3×3 convolution, Conv 7×7 (·) represents a 7×7 convolution, AFG represents an attention feature gate layer, and AFG consists of a global pooling GAP(·) and a multi-channel feature sparsification function sp(·), calculated as follows:

[0164] AFG(X) = GAP(X) × sp(X)

[0165] miniMSFP is a small multi-scale fusion block. It is composed of multi-scale average pooling and 1×1 convolution, and the calculation formula is as follows:

[0166]

[0167] In the formula, GAP(·) represents global pooling, and Conv 1×1 (·) represents a 1×1 kernel convolution, N is the number of scales in a miniMSFP, N=3; AP k×k (·) represents average pooling with a kernel of k×k, where k∈{3, 5, 7};

[0168] Inception represents a multi-scale feature extraction block. Inception consists of a 3×3 convolutional Conv... 3×3 (·) and 1×1 convolution Conv 1×1 The (·) component is calculated using the following formula:

[0169] Inception(X) = Concat(Conv) 1×1 (X), Conv 3×3 (Conv 1×1 (X)), Conv 3×3 (Conv 1×1 (X)))

[0170] In the formula, Concat(X1, X2, X3) represents arranging the matrix features of features X1, X2, and X3 in order to obtain X. Cat X1, X2, and X3 are n×c×f w×h matrices; X Cat There are n×c×3×f w×h matrices;

[0171] (2) Input the training sample X into the AFG-NET network model;

[0172] (3) The AFG layer in the AFG-NET network model performs global feature calculation on the training sample X to obtain the global pooling feature X. GAP Global pooling feature X GAP Let X be an n×c matrix. GAP ∈(-1, 1); the calculation formula is as follows:

[0173] X GAP =GAP(X)=X n×c ·mean(X f×w×h )

[0174] In the formula, GAP(·) represents global pooling of the f, w, and h dimensions of the training sample X, where X represents the training sample. n×c This is an n×c matrix formed by the n and c dimensions of the training samples X; mean(·) represents taking the mean, X f×w×h f w×h matrices representing the f, w, and h dimensions of the training sample X;

[0175] (4) The input layer AFG layer in the AFG-NET network model performs multi-channel feature sparsification on the training sample X to obtain the sparsified features X. SP , sparsification feature X SP It is a set of n×c w×h matrices; the calculation formula is as follows:

[0176]

[0177] In the formula, X represents the training sample, sp(·) is the multi-channel feature sparsification function, and x i It is the feature matrix of the training sample X expanded along the third dimension for f channels, and the feature matrix x is... i There are n×c w×h matrices; a i Represents the characteristic matrix x i The inhibitory factor, a i The calculation formula is as follows:

[0178]

[0179] In the formula, λ is the channel suppression ratio, λ∈{10} n}, n∈{-3,-4,-5};γ i γ is the weight scaling ratio of the BN layer of the i-th channel in the multi-channel feature sparsification function sp(·). i ∈(0,1); γ i The calculation formula is as follows:

[0180]

[0181] Conv 1×1 (x i ) represents the input of the i-th channel BN layer in the multi-channel feature sparsification function sp(·), Conv 1×1 (x i ) represents n×c w×h matrices; BN(x i ) represents the output of the i-th channel BN layer in the function sp(·), BN(x) i ) represents n×c w×h matrices; γ i The scaling weight of the BN layer for the i-th channel in the representation function F(X), γ i ∈(0,1); β i The bias weight of the i-th channel BN layer in the representative function sp(·) is β. i ∈(0,1);

[0182] (5) Using global pooling feature X GAP The n×c elements and the sparsity feature X SP The output matrix of the AFG layer is obtained by multiplying the n×c w×h matrices respectively. Output matrix There are n×c w×h matrices; the calculation formula is as follows:

[0183]

[0184] The special symbol ⊙ indicates that the elements of the previous matrix are multiplied by the corresponding matrices in the next set;

[0185] (6) Output of AFG The data is fed into AFG-NET for feature extraction, and finally the AFG-NET prediction matrix X is obtained. sout Prediction matrix X sout Let X be an n×s matrix, where s represents the number of label categories for the training sample X;

[0186] (7) The model parameter P with the highest MSFP-NET accuracy Acc in step 2. t Load it into the teacher model MSFP-NET, and use model parameters P t The MSFP-NET predicts the training samples X, obtaining the prediction matrix X of the teacher network MSFP-NET. tout X tout Let X be an n×s matrix, where s represents the number of label categories for the training samples X;

[0187] (8) Set the knowledge distillation temperature parameter τ, τ∈(0,1), and set the prediction matrix X of the teacher network MSFP-NET. tout Divide by τ and feed the result into the Softmax function to obtain the soft label Y. (t) The prediction matrix X of the student network AFG-NET sout Divide by τ and feed the result into the Softmax function to obtain the soft prediction Y. (s) The calculation formula is as follows:

[0188]

[0189]

[0190] (9) The output characteristics X of the student network AFG-NET sout Divide by τ and feed the result into the Softmax function to obtain the hard prediction Y;

[0191] Y = Softmax(X) sout (10) Calculate the Koubek-Leibler divergence loss L KD The calculation formula is as follows:

[0192]

[0193] In the formula L KD For the Koubek-Leibler divergence loss, Y (t) For soft tags, Y (t) For prediction;

[0194] Find λ inter The formula for calculating the internal sparsity factor is as follows:

[0195]

[0196] In the formula, τ represents the knowledge distillation temperature parameter, n is the number of video samples X′ in the training sample X, n∈N; d p (·) represents the Pearson similarity function, and s represents the number of label categories for the training samples X.

[0197] The internal sparsity factor λ inter and Koubek-Leibler divergence loss L KD Multiplication yields the soft loss L soft The calculation formula is as follows:

[0198] L soft =λ inter *L KD

[0199] (11) The cross-entropy loss function is used as the cost function, and the L1 norm is used to sparsify the parameter γ of AFG. Similarly, the hard labels are obtained. The distance L between the hard prediction Y and the hard prediction Y hard L hard Represents hard loss, hard loss L hard The calculation formula is as follows:

[0200]

[0201] In the formula Let λg(γ) be the cross-entropy loss function, and let λg(γ) be the sparsity penalty for the scaling factor γ of the BN layer in the multi-channel feature sparsification function sp(·). λ is the scaling factor of the penalty term used to balance these two losses, and λ∈{10} n}, n∈{-3,-4,-5}; g(·) is the L1 norm penalty formula, calculated as follows:

[0202]

[0203] In the formula, f represents the number of channels in the multi-channel feature sparsification function sp(·), and γ i The scaling weight of the BN layer for the i-th channel in the function sp(·) is used; the distillation loss L is calculated using a weighted sum of hard and soft losses, as shown in the following formula:

[0204] L=χL soft +δL hard

[0205] χ=1-δ

[0206] In the formula, χ is the soft loss balance parameter, χ∈(0,1); δ is the hard loss balance parameter, δ∈(0,1); in each iteration of the dataset, gradient descent is used to adjust the weights between neurons in the student network to reduce the cost function and thus optimize the network.

[0207] (12) Through each training of training sample X, the stochastic gradient descent function SGD is used to make the distillation loss L backpropagate in the AFG-NET network model, so that the model parameters P of the i-th round of training of AFG-NET are improved. i Randomly varying the model parameters in the direction of the negative gradient to optimize the network, the formula for updating the model parameters using SGD is as follows:

[0208] P i =P i-1 -η▽L(P i-1 )

[0209] In the formula P i P represents the model parameters in the i-th round of training of AFG-NET, where P i-1 Let represent the model parameters in the (i-1)th round of training of AFG-NET, and η represent the optimization step size of SGD, η∈(0,1); ▽L(P i-1 The training loss L represents the training loss L on the model parameters P during the (i-1)th round of training of AFG-NET. i-1 Perform differentiation;

[0210] (13) Compare the model recognition accuracy Acc in each training round. i And save Acc i The highest model parameter P i And used as deployment parameter P s The calculation is as follows:

[0211]

[0212] Step 5: Use the deployment parameters P saved in Step 4 s As the network parameters of the AFG-NET model, and with the network parameters set as P s The AFG-NET model is deployed on the recognition terminal. At the same time, the terminal performs video abnormal behavior recognition on the input video and finally obtains the video abnormal behavior recognition result y, where y is a positive integer representing the abnormal behavior category of the video input, y∈[1,s].

Claims

1. A video abnormal behavior recognition method based on relaxation divergence function knowledge distillation, characterized by The method comprises the following steps: Step 1, making a training sample X, the specific steps are as follows: (1) randomly selecting n video samples X' with f frames from the data set, f e (1, 64]; X' is a cxf a x b matrix, a e (224, 1080); b e (224, 1080); c is the number of channels of the video image, c e {1, 3}; (2) performing size transformation on the n video samples X' with f frames to obtain a transformed size video sample X'', X'' is cxf w x h; w and h are fixed values 224; then performing maximum and minimum value normalization on X'' to obtain a normalized video sample X''', X''' is cxf w x h matrix; the calculation formula is as follows: where x'" represents a single element in the matrix X'", x'" ∈ (0, 255); x wh where x'" represents a single element in the matrix X'", x'" ∈ (0, 255); x wh ∈(-1, 1); x" wh where x" represents a single element in the matrix X", x" ∈ (0, 255); x wh ∈(-1, 1); x" min where x represents a single element in the matrix X, x ∈ (0, 255); x min = -1; x max = 1; x max = 1; x (3) Finally, the n normalized video samples X″′ are combined into a set X as training samples, X={X″′1,X″′2,X″′3,…,X″′ n The training samples X consist of n×c×f w×h matrices; n is the number of normalized video samples X″′ in the training samples X, n∈N; Step 2, data augmentation is performed on the training sample X to obtain an augmented sample The specific steps are as follows: (1) When the random probability q of the i-th training round i is greater than the trigger probability p i , q i = random(0, 1); the calculation formula of the trigger probability p i is: where p i is the probability of triggering the i-th round of the data augmentation algorithm, p i ∈ (0, 1]; e m represents the maximum number of training rounds, e m ∈ N; e i representing the current i-th training round, e i ∈(1, e m ]; the algorithm performs data augmentation on the n normalized video samples X′′ to obtain augmented video samples is a cxf matrix; The calculation formula is as follows: where f is the number of frames of the normalized video sample X" ', Z i ; i = {1, 2,..., f} represents the i-th frame image in the normalized video sample X" ', Z i "'is a c w x h matrix; M i ; i = {1, 2,..., f} represents the i-th frame image in the normalized video sample X" ', Z i mask multiplied with, M i is a c w x h matrix; (2) n enhancement video samples are combined to obtain an enhancement sample (2) n enhancement video samples are combined to obtain an enhancement sample The calculation formula of the enhancement algorithm is: where q i is the random probability of the i-th training round, q i = random(0, 1); p i is the trigger probability of the i-th training round of the enhancement algorithm; X is the training sample, and the training sample X is an n x c x f w x h matrix; represents an enhanced video sample, and the enhanced video sample is a c x f w x h matrix, represents an enhanced sample, is a set composed of n enhanced video samples , and is an n x c x f w x h matrix. Step 3: Construct the teacher network MSFP-NET, which will enhance the sample The model parameters P that achieve the highest accuracy Acc in MSFP-NET were obtained by training the model on the MSFP-NET network. t The specific steps are as follows: (1) constructing a video abnormal behavior recognition model MSFP-NET, MSFP-NET is composed of MSAI, MSFP and three-dimensional convolution, the calculation formula is as follows: MSFP-NET(X) = MSAl(MSFP(MSAI(Conv 3×3×3 (Conv 1×1×1 (Conv 7×7×7 (X)))))) where Conv 1×1×1 (·) represents 1x1x1 convolution, Conv 3×3×3 (·) represents 3x3x3 convolution, Conv 7×7×7 (·) represents 7x7x7 convolution, MSFP is a multi-scale fusion block, MSFP is composed of a multi-scale average pooling and a 1x1x1 convolution, and the calculation formula is as follows: where GAP(·) represents global pooling, Conv 1×1×1 (·) represents a convolution with a kernel of 1 x 1 x 1, the number of scales of N generations of MSFP, N = 3; AP k×k (·) represents average pooling with a kernel of k x k x k, k e {3, 5, 7}; MSAI represents a multi-scale attention feature block, Inception is composed of 3x3 convolution and 1x1 convolution, the calculation formula is as follows: MSAI(X) = 3D Inception(X + Conv 1×1×1 (MSA(Conv 1×1×1 (X)))) In the formula, 3DInception represents a three-dimensional multi-scale feature extraction block, and the 3DInception is composed of a 3x3x3 convolution Conv 3×3×3 (·) and a 1x1x1 convolution Conv 1×1×1 (·), and the calculation formula is as follows: 3DInception = Concat(Conv 1×1×1 (X), Conv 3×3×3 (Conv 1×1×1 (X)), Conv 3×3×3 (Conv 1×1×1 (X))) In the formula, Concat (X1, X2, X3) represents sequentially arranging the matrix features of characteristics X1, X2, and X3 to obtain X Cat , X1, X2, and X3 are n×c×f w×h matrices; X Cat is an n×c×3×f w×h matrix; MSA represents a multi-scale attention module, which is composed of multi-scale average pooling and convolution, the calculation formula is as follows: In the formula, DW_Conv 5×5×5 represents a depth separable convolution with a kernel size of 5x5x5, Scale i represents the size of the convolution kernel of the i-th meso-scale, and the calculation formula is as follows: (2) the enhanced sample The model prediction matrix y is obtained by inputting the queue into the video abnormal behavior recognition model MSFP-NET and calculating the MSFP-NET model (t) The calculation formula is as follows: where the prediction matrix y (t) is an n x s matrix, s representing the label classes of the augmented samples Conv 1×1×1 (·) represents a 1 x 1 x 1 convolution, Conv 3×3×3 (·) represents a 3 x 3 x 3 convolution, Conv 7×7×7 (·) represents a 7 x 7 x 7 convolution; (3) Error back propagation is adopted, and cross-entropy loss function is used to measure the real value The distance between the model prediction matrix y (t) and the real value matrix y t , and the training loss L t is calculated, L t ∈(0, +∞), loss function formula: By enhancing the sample of each training, using the random gradient descent function SGD makes the loss value L t In the MSFP-NET network model, the model parameters P i of the i-th round of training of MSFP-NET are updated in the direction of the negative gradient, and the calculation formula of the model parameters updated by SGD is as follows: P i = P i-1 - η▽L(P i-1 ) where P i represents the model parameters of the i-th round of training of AFG-NET, where P i-1 represents the model parameters of the i-1-th round of training of AFG-NET, η represents an optimization step of SGD, and η ∈ (0, 1); represents derivation of the training loss L with respect to the model parameters P i-1 of the i-1-th round of training of AFG-NET; and represents derivation of the training loss L with respect to the model parameters P i-1 of the i-th round of training of AFG-NET. (5) Calculate the current training round e i The AFG-NET model recognition accuracy Acc i , Acc i ∈(0, 1), and save the training round e i The AFG-NET model P i , e i represents the current i-th training round, e i ∈(1, e m ]; e m is the maximum training round, e m ∈N; In the formula, TP is the number of correct predictions of positive samples by the model, TN is the number of correct predictions of negative samples by the model, FP is the number of incorrect predictions of positive samples by the model, and FN is the number of incorrect predictions of negative samples by the model; Step 4, constructing a student network AFG-NET, sending the training sample X into the student network AFG-NET for training, and using a knowledge distillation technology to obtain network parameters P with the optimal AFG-NET model recognition accuracy Acc in all training rounds s The specific steps are as follows: (1) constructing a video abnormal behavior recognition model AFG-NET, AFG-NET is composed of AFG, miniMSFP, Inception and two-dimensional convolution, the calculation formula is as follows: AFG-NET(X) = Inception(miniMSFP(Inception(Conv 3×3 (Conv 1×1 (Conv 7×7 (AFG(X))))))) where Conv 1×1 (·) represents 1x1 convolution, Conv 3×3 (·) represents 3x3 convolution, Conv 7×7 (·) represents 7x7 convolution, AFG represents attention feature gating layer, AFG is composed of global pooling GAP (·) and multi-channel feature sparsification function sp (·), the calculation formula is as follows: AFG(X)=GAP(X)×sp(X) miniMSFP is a small multi-scale fusion block, which is composed of multi-scale average pooling and 1x1 convolution, the calculation formula is as follows: where GAP(·) represents global pooling, Conv 1×1 (·) represents a convolution with a kernel of 1 x 1, N is the number of scales for the miniMSFP, N = 3; AP k×k (·) represents average pooling with a kernel of k x k, k e {3, 5, 7}; Inception represents a multi-scale feature extraction block, Inception is composed of 3x3 convolution Conv 3×3 (·) and 1x1 convolution Conv 1×1 (·), the calculation formula is as follows: Inception(X) = Concat(Conv 1×1 (X), Conv 3×3 (Conv 1×1 (X)), Conv 3×3 (Conv 1×1 (X))) In the formula, Concat (X1, X2, X3) represents sequentially arranging the matrix features of characteristics X1, X2, and X3 to obtain X Cat , X1, X2, and X3 are n×c×f w×h matrices; X Cat is an n×c×3×f w×h matrix; (2) inputting the training sample X into the AFG-NET network model; (3) The AFG layer in the AFG-NET network model performs global feature calculation on the training sample X to obtain global pooling features X GAP The global pooling features X GAP are an n x c matrix, X GAP ∈(-1, 1); the calculation formula is as follows: X GAP = GAP(X) = X n×c · mean(X f×w×h ) where GAP(·) represents global pooling on the f, w, h dimensions of the training sample X, X represents the training sample, X n×c is an n x c matrix formed from the n, c dimensions of the training sample X; mean(·) represents taking the mean, X f×w×h represents f w x h matrices formed from the f, w, h dimensions of the training sample X. (4) The input layer AFG layer in the AFG-NET network model performs multi-channel feature sparsification on the training sample X to obtain sparse features X SP , which is a set of n x c w x h matrices; the calculation formula is as follows: SP ​ where X represents the training sample, sp(·) is a multi-channel feature sparsification function, x i is a feature matrix of f channels of the training sample X unfolded in the third dimension, the feature matrix x i is an n x c matrix of w x h matrices; a i represents a suppression factor of the feature matrix x i , a i is calculated as follows: where λ is a channel inhibition ratio, λ ∈ {10 n}, n ∈ {-3, -4, -5}; γ i is a weight scaling ratio of the i-th channel BN layer in the multi-channel feature sparsification function sp(·), γ i ∈ (0, 1); γ i The calculation formula is as follows: Conv 1×1 (x i ) represents the input of the i-th channel BN layer in the multi-channel feature sparsification function sp(·), Conv 1×1 (x i ) is an n x c w x h matrix; BN(x i ) represents the output of the i-th channel BN layer in the function sp(·), BN(x i ) is an n x c w x h matrix; γ i represents the i-th channel BN layer scaling weight in the function F(X), γ i ∈(0, 1); β i represents the i-th channel BN layer bias weight in the function sp(·), β i ∈(0, 1). (5) using global-pooling features X GAP with n x c elements, multiplied by n x c w x h matrices in the sparse features X SP output matrix of n x c w x h matrices; the calculation formula is as follows:​ The special symbol represents that the elements of the former matrix are multiplied with the matrices in the latter set; (6) The output of the AFG is sent into the AFG-NET for feature extraction, and finally the AFG-NET prediction matrix X is obtained sout The prediction matrix X sout is an n x s matrix, and s represents the label category of the training sample X;​ (7) The model parameter P with the highest MSFP-NET accuracy Acc in step 2. t Load it into the teacher model MSFP-NET, and use model parameters P t The MSFP-NET predicts the training samples X, obtaining the prediction matrix X of the teacher network MSFP-NET. tout X tout Let X be an n×s matrix, where s represents the number of label categories for the training samples X; (8) Set the knowledge distillation temperature parameter τ, τ ∈ (0, 1) to divide the prediction matrix X of the teacher network MSFP-NET tout After dividing by τ, it is sent into the Softmax function to obtain the soft label Y (t) , the prediction matrix X of the student network AFG-NET sout After dividing by τ, it is sent into the Softmax function to obtain the soft prediction Y (s) , the calculation formula is as follows: (9) The output features X of the student network AFG-NET are divided by τ and sent into the Softmax function to obtain the hard prediction Y. sout After dividing by τ, the hard prediction Y is obtained by sending it into the Softmax function. Y = Softmax(X sout ) (10) Kubelka-Munk dispersion loss L is calculated KD The formula is as follows: where L KD is the Kubelka-Munk dispersion loss, Y (t) is the soft label, Y (s) is the soft prediction; Finding λ inter As an internal sparse factor, the formula is as follows: In the formula, τ represents a knowledge distillation temperature parameter, n is the number of video samples X' in the training sample X, n∈N; d p (·) is a Pearson similarity function, s represents the number of label categories of the training sample X, The internal sparse factor λ inter and the Kubelka-Munk divergence loss L KD are multiplied to obtain the soft loss L soft , the calculation formula is as follows: L soft = λ inter * L KD (11) Using cross-entropy loss function as cost function, while using L1 norm to sparsify the parameter γ of AFG, and the same to get hard label and the distance L of hard prediction Y hard , L hard represents the hard loss, and the calculation formula of hard loss L hard is as follows: wherein is the cross-entropy loss function, λg(γ) is the sparsity penalty on the scale factor γ of the BN layer in the multi-channel feature sparsification function sp(·), λ is the penalty term scaling factor to balance the two loss terms, λ ∈ {10 n}, n ∈ {-3, -4, -5}; g(·) is the L1 norm penalty formula, which is calculated as follows: where f denotes the number of channels of the multi-channel feature sparsification function sp(·), and γ i represents the i-th channel BN layer scaling weight in the function sp(·); the weighted sum of the hard loss and the soft loss is used as the distillation loss L, and the calculation formula is as follows: L = χL soft + δL hard χ=1-δ In the formula, χ is a soft loss balance parameter, χ e (0, 1); δ is a hard loss balance parameter, δ e (0, 1); in each iteration of the data set, the gradient descent is used to adjust the weights between the student network neurons to reduce the cost function, and then the network is optimized; (12) Through each training of the training sample X, the distillation loss L is made to be back-propagated in the AFG-NET network model by using a stochastic gradient descent function SGD, so that the model parameter P of the i-th round of training of the AFG-NET i The random change in the direction of the negative gradient is used to optimize the network, and the calculation formula of the model parameter updated by using the SGD is as follows: P i = P i-1 - η▽L(P i-1 ) where P i represents the model parameters of the i-th round of training of AFG-NET, where P i-1 represents the model parameters of the i-th round of training of AFG-NET, where P i-1 represents the model parameters of the i-th round of training of AFG-NET, where P i-1 represents the model parameters of the i-th round of training of AFG-NET, where P (13) compare the model recognition accuracy Acc of each training round i and save Acc i the highest model parameter P i and as the deployment parameter P s , calculated as follows: Step 5, using the deployment parameters P saved in step 4 s As the network parameters of the AFG-NET model, and the network parameters are P s The AFG-NET model of the application is deployed on the identification terminal, and the terminal performs video abnormal behavior identification on the input video, and finally obtains a video abnormal behavior identification result y, y is a positive integer representing the abnormal behavior category of the video input, y∈[1,s].

Images