Abnormal behavior detection method, device and storage medium

Abstract

The present invention discloses an abnormal behavior detection method, device and storage medium. Based on the use of deep learning, it gives full play to the advantages of high response and low latency of edge servers, and proposes an abnormal behavior detection method based on data-free knowledge transfer. The present invention can ensure that a synthetic image data set for training is provided for the edge model without accessing the original data set and without using any natural images and labeled data, and the differences between the cloud model and the edge model are used to improve the diversity of the synthetic images. Experiments have proved that the feasibility of the present invention is high, the trained model can run on the edge device, and the accuracy rate is high when detecting abnormal behavior, and it has the advantages of low latency and fast response, which effectively overcomes the disadvantage that large and complex cloud models cannot run on edge devices.

Description

Technical Field

[0001] The present invention relates to abnormal behavior detection technology, and in particular to an abnormal behavior detection method, device and storage medium. Background Art

[0002] With the continuous development of computer intelligence technology, abnormal behavior detection of crowds is becoming more and more common in the fields of smart security, smart cities, etc. Most of the current abnormal behavior detection uses cloud service technology, but in practical applications, it often requires calculating huge amounts of data and obtaining instant feedback, which will occupy a large amount of network bandwidth and bring huge pressure to the communication network.

[0003] Edge computing is a supplement and extension of cloud computing. It distributes large services to edge devices for processing, making data processing more timely and transmission more secure. Models deployed in the cloud have high requirements for hardware computing power and memory resources. Therefore, to implement edge computing, in order to enable the model to run on edge devices, large and complex models in the cloud must first be compressed. [1] Most current solutions for this type of model compression task are based on the concept of knowledge distillation. [2] The concept is to use a trained larger model to train a new smaller model so that its output matches the output of the original model. This method requires the original dataset during training. [3] . However, in abnormal behavior detection applications, cloud models are often trained on datasets with millions or even billions of images. These datasets are not only very large, but also difficult to store, transmit and manage. Another issue that cannot be ignored is data privacy. Due to user privacy, security or patent issues, it is difficult to share training datasets during knowledge distillation. There are currently several methods to visualize the knowledge contained in trained neural networks, such as transforming the input image and using the gradient ascent method to visualize the features of each layer of the network. [5] ; or update the random input through objective function gradient iteration to minimize the difference between the input and the original image [7] However, the images synthesized by these methods are often not realistic and are quite different from natural images, so they cannot be used for knowledge distillation.

[0004] In addition, generative adversarial networks (GANs) can be used to generate high-fidelity synthetic images. [8] , this method trains two adversarial networks (generator and discriminator) to generate synthetic images. Although the resolution of the synthetic images generated by this method is improved, training the GAN generator still requires access to the original data.

[0005] Therefore, how to ensure that the training data can be restored from the trained cloud model without accessing the original data set, and how to transfer the abnormal behavior detection knowledge in the cloud model to the model that can run on the edge device, while maintaining a high accuracy in abnormal behavior detection, is a technical problem that needs to be solved urgently. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide an abnormal behavior detection method, device and storage medium to improve the accuracy of abnormal behavior detection without accessing the original data in view of the shortcomings of the existing technology.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: an abnormal behavior detection method, comprising the following steps:

[0008] S1. Obtain a pre-trained cloud model, and input a random noise image into the cloud model to obtain the output distribution of the cloud model;

[0009] S2. Generate a feature distribution regularization term using the mean expected value and variance expected value of all convolutional layer feature maps obtained from the cloud model, and obtain a loss function using the feature distribution regularization term to obtain a synthetic image;

[0010] S3. Input the synthesized image into the cloud model and the edge model, perform distribution matching based on the softmax values ​​output by the cloud model and the edge model, train the edge model, and obtain an abnormal behavior detection model.

[0011] The present invention uses a pre-trained cloud model to obtain synthetic images and train an edge model. Without accessing the original data, the training data can be restored from the cloud model to obtain a training set composed of synthetic images for training the edge model. This solves the problem in traditional abnormal behavior detection applications that the original data set is often too large to access, resulting in the inability to achieve abnormal behavior detection. This eliminates the need to consume a large amount of computing power and memory resources, thereby improving the accuracy of abnormal behavior detection.

[0012] In order to further improve the accuracy of abnormal behavior detection, the present invention utilizes the differences between the cloud model and the edge model to expand the coverage of the training set composed of synthetic images, thereby improving the accuracy of the edge model. The method of the present invention also includes:

[0013] S4, inputting the random noise image into the trained edge model to obtain an output distribution of the edge model;

[0014] S5, calculating the Wasserstein distance between the output distribution of the cloud model and the output distribution of the edge model, and updating the synthetic image using the Wasserstein distance;

[0015] S6. Return to step S2 and repeat steps S2 to S5 until the number of iterations is reached to obtain the final abnormal behavior detection model.

[0016] In step S1, the ResNet-50 model pre-trained on the training set ImageNet is used as the cloud model. The pre-trained model trained with ImageNet has richer prior knowledge, and the ResNet-50 model has better performance, has stricter requirements on device operating conditions, and is suitable for deployment in the cloud.

[0017] In step S2, the following loss function is used as the optimization target, and back propagation is performed to use the gradient ascent method to optimize the input random noise image: Update and change the RGB values ​​of the random noise image to obtain a synthetic image:

[0018]

[0019] in is the classification loss; is the input of the cloud model; y is the true value of the target label; are the penalized total variance and norm, , ; ; and is the scaling factor; Represents the cloud model convolutional layers; and Respectively The mean and variance of the feature maps corresponding to the convolutional layers; represents the expected value; X represents the training set for training the cloud model, Represents the input of the cloud model during training, and They represent the number of times the cloud model is trained. The mean and variance of the feature maps corresponding to the convolutional layers; Express request Norm operation.

[0020] The present invention introduces an image regularization term and The quality of synthesized images is improved, and the feature similarity between synthesized images and cloud model training images can be effectively enhanced.

[0021] In step S3, ResNet-18 is used as the edge model. The ResNet-18 model meets the operating conditions of edge devices.

[0022] In step S5, the loss function is updated using the Wasserstein distance to obtain an updated composite image:

[0023]

[0024] in, is the scaling factor; , , inf represents the lower bound of the function; express and The joint distribution of Indicates modulo length operation; Represents the joint distribution The expected value of the Wasserstein distance between the cloud model output distribution and the edge model output distribution; and They are the output distribution of the cloud model and the output distribution of the edge model respectively.

[0025] The present invention introduces image generation loss Penalize the similarity of the output distribution of the cloud model and the edge model, encourage synthetic images to cause divergence between the cloud model and the edge model, expand the coverage of the training set composed of synthetic images, and promote knowledge transfer from the cloud model to the edge model.

[0026] , Experiments have shown that the synthetic images generated by this scaling factor have higher quality and are more suitable as training sets for edge models.

[0027] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the steps of the method of the present invention.

[0028] The present invention also provides a computer program product, comprising a computer program / instruction; when the computer program / instruction is executed by a processor, the steps of the method of the present invention are implemented.

[0029] The present invention also provides a computer-readable storage medium on which a computer program / instruction is stored; when the computer program / instruction is executed by a processor, the steps of the method of the present invention are implemented.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. The present invention proposes a data-free knowledge transfer method, which can recover training data from a cloud model without accessing the original data, and obtain a training set consisting of synthetic images for training edge models. This solves the problem that in traditional abnormal behavior detection applications, the original data set is often too large to access, resulting in the inability to implement anomaly detection, without consuming a large amount of computing power and memory resources;

[0032] 2. The present invention uses the gradient ascent method to visualize the features of each layer of the neural network, and combines the mean, expected value and expected variance of the convolutional layer feature map stored in the BN layer to generate realistic and high-resolution training images, which greatly improves the accuracy of anomaly detection and solves the problem that traditional synthetic images are not realistic and cannot be used for model compression;

[0033] 3. The present invention utilizes the differences between the cloud model and the edge model to expand the coverage of the training set composed of synthetic images, thereby improving the accuracy of the edge model;

[0034] 4. Based on the use of deep learning, the present invention gives full play to the advantages of high response and low latency of edge servers, and can provide synthetic image datasets for training for edge models without accessing the original datasets, using any natural images and labeled data, and using the differences between cloud models and edge models to improve the diversity of synthetic images. Experiments have shown that the present invention is highly feasible, and the trained model can run on edge devices, and has a high accuracy rate in abnormal behavior detection, and has the advantages of low latency and fast response, effectively overcoming the disadvantage that large and complex cloud models cannot run on edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a schematic diagram of the method of an embodiment of the present invention;

[0036] Figure 2 (a) Noisy image input to the cloud model; Figure 2 (b) using DeepDream [5] The image synthesized by the method; Figure 2 (c) is the image synthesized after adding the feature distribution regularization term; Figure 2 (d) is the image synthesized after adding the image generation loss regularization term. DETAILED DESCRIPTION

[0037] Based on the advantages of low latency and fast response of edge networks in abnormal behavior detection, the present invention transfers knowledge from the cloud model to the edge device and deploys it to the edge device. No natural images and label data are required in the process of training the edge model.

[0038] The architecture of the present invention mainly consists of three parts: (1) Synthesizing training images. Using a cloud model to synthesize images for an edge model that can be deployed on an edge device, generating realistic images with high resolution. (2) Improving the diversity of synthesized images. The present invention is a method for enhancing synthesized images based on iterative competition between an edge model and an edge model during the image generation process. The main idea is to improve the diversity of synthesized images by utilizing the differences between the cloud model and the edge model. (3) Performing knowledge distillation. Using a training set composed of synthetic images to train the edge model.

[0039] Here are the steps:

[0040] Step 1: Use ResNet-50 pre-trained on the training set ImageNet [4] As a cloud model, ImageNet is a large visual database for visual object recognition software research, containing more than 20,000 categories. Input a random noise image to the cloud model. The image uses random value impulse noise, and its noise grayscale value is any grayscale value between 0-255. Output the probability distribution of the image, and calculate the loss function of the probability distribution and any target label. The loss function is obtained by calculating the cross entropy:

[0041]

[0042] Where n represents the n categories of probability distribution, Represents the target label The probability of the category, Represents the probability distribution of the cloud model output The probability of the category.

[0043] The expected mean and variance of all convolutional layer feature maps are obtained from the BN layer [6] of the cloud model to generate the feature distribution regularization term. This feature distribution regularization can greatly improve the quality of the generated image. The feature distribution regularization term is:

[0044] ;

[0045] in It is the input of the cloud model; The representative model convolutional layers; and Time The mean and variance of the feature map corresponding to the convolutional layer; represents the expected value; X represents the training set for training the cloud model, Represents the input of the cloud model during training, and Indicates the number of times the cloud model is trained. The mean and variance of the feature map corresponding to the convolutional layer can be obtained through the BN layer; Express request Norm operations; Indicates that the operation results of all convolutional layers will be summed.

[0046] Step 2: Update the pixel values ​​of the random noise image input to the cloud model in the first step during back propagation by optimizing the following , and get the composite image:

[0047]

[0048] in is the classification loss; is the input of the cloud model; y is the true value of the target label; Penalize the total variance and norm, , ; is the feature distribution regularization term obtained in the first step; and is the corresponding scaling factor, , .

[0049] Step 3: Perform knowledge distillation and convert ResNet-18 [4] As the edge model, use the cloud model in the first step and the synthetic image training set in the second step to preliminarily train the edge model. The training process is to input synthetic images to the cloud model and the edge model, and perform distribution matching based on the softmax values ​​output by the cloud model and the edge model, and then train the edge model. [2] During training, synthetic images are input to the cloud model and the edge model, and two probability distributions are obtained and softmax calculations are performed separately to generate softened probability distributions. The softmax calculation formula is as follows:

[0050]

[0051] in represents the exponential function operation with e as the base, Indicates The output probability of the class, and Indicates The input probability of the class and the jth class, T is the temperature coefficient, which is used to control the softening degree of the probability distribution.

[0052] Use cross entropy to find the loss function of the above two probability distributions, and use this loss function to train the marginal model.

[0053] Step 4: Input the same random noise image as the cloud model to the edge model, and obtain the output distributions of the cloud model and edge model respectively. and .

[0054] Step 5: Calculate the Wasserstein distance between the two output distributions as an additional image generation loss to penalize the similarity of the output distributions. The image generation loss regularization term is:

[0055]

[0056] in ;

[0057] Where inf represents the lower bound of the function; express and The joint distribution of Indicates modulo length operation; Represents the joint distribution The expected value of the sample distance under the condition of . The formula of the second step is modified as follows:

[0058]

[0059] in is the classification loss, obtained by calculating the cross entropy; is the input of the cloud model; y is the true value of the target label; Penalize the total variance and norm; is the feature distribution regularization term obtained in the first step; Generate loss regularization term for the image; and is the corresponding scaling factor, , , , .

[0060] Step 6: Perform knowledge distillation, continue training the edge model using the training set consisting of the cloud model and synthetic images, and repeat steps 4 to 6 100 times.

[0061] We use ResNet-50 as the fixed cloud model and ResNet-18 as the edge model. The cloud model has been pre-trained on the training set ImageNet and set , , , , to generate a 224*224 synthetic image. The noise image input to the cloud model is shown in Figure 2 (a). The input target labels are fish, hamburger, plant, and volcano. [5] The images synthesized by the method are shown in Figure 2 (b). It can be seen that these images are not realistic and are quite different from natural images. After adding the feature distribution regularization term of the present invention, the synthesized images are shown in Figure 2 (c). The realism and resolution of the images are greatly improved, and they can be used for knowledge distillation of training edge models. After adding the image generation loss regularization term of the present invention, the synthesized images are shown in Figure 2 (d), which improves the diversity of the synthesized images.

[0062] References

[0063] [1] P. Molchanov, A. Mallya, S. Tyree, I. Frosio, and J. Kautz. "Importance estimation for neural network pruning". In CVPR, 2019.

[0064] [2] G. Hinton, O. Vinyals, and J. Dean. "Distilling the knowledge in aneural network". arXiv preprint arXiv:1503.02531, 2015.

[0065] [3]T. Chen, I. Goodfellow, and J. Shlens. “Net2net: Acceleratinglearning via knowledge transfer”. In ICLR, 2016.

[0066] [4]K. He, X. Zhang, S. Ren, and J. Sun. “Deep residual learning for image recognition”. In CVPR, 2016.

[0067] [5] A. Mordvintsev, C. Olah, and M. Tyka. "Inceptionism: Going deeper into neural networks", 2015.

[0068] [6]S. Ioffe and C. Szegedy. “Batch normalization: Accelerating deepnetwork training by reducing internal covariate shift”. arXiv preprint arXiv:1502.03167, 2015.

[0069] [7] A. Mahendran and A. Vedaldi. “Understanding deep imagerepresentations by inverting them”. In CVPR, 2015.

[0070] [8]A. Brock, J. Donahue, and K. Simonyan. “Large scale GAN trainingfor high fidelity natural image synthesis”. In ICLR, 2019.

Claims

1. A method for detecting abnormal behavior, characterized in that: The following steps are involved: S1. Obtain a pre-trained cloud model, and input a random noise image into the cloud model to obtain the output distribution of the cloud model; S2. Generate a feature distribution regularization term using the mean expected value and variance expected value of all convolutional layer feature maps obtained from the cloud model, and obtain a loss function using the feature distribution regularization term to obtain a synthetic image; S3, inputting the synthesized image into the cloud model and the edge model, performing distribution matching according to the softmax values ​​output by the cloud model and the edge model, training the edge model, and obtaining an abnormal behavior detection model; S4, inputting the random noise image into the trained edge model to obtain an output distribution of the edge model; S5, calculating the Wasserstein distance between the output distribution of the cloud model and the output distribution of the edge model, and updating the synthetic image using the Wasserstein distance; S6, return to step S2, repeat steps S2 to S5, until the number of iterations is reached, and obtain the final abnormal behavior detection model; In step S2, the following loss function is used as the optimization target, and back propagation is performed to use the gradient ascent method to optimize the input random noise image: Update and change the RGB values ​​of the random noise image to obtain a synthetic image: in is the classification loss; y is the true value of the target label; R TV (·), R l2 (·) penalize the total variance and l2 norm respectively, α tv , and α f is the scaling factor; l represents the lth convolutional layer of the cloud model; and are the mean and variance of the feature map corresponding to the lth convolutional layer; E(·) represents the expected value; X represents the training set for training the cloud model, x represents the input of the cloud model during training, and E(μ l (x)|X) and They represent the mean and variance of the feature map corresponding to the lth convolutional layer of the cloud model during training; ||·||2 represents the l2 norm operation; In step S5, the loss function is updated using the Wasserstein distance to obtain an updated composite image: Among them, α w is the scaling factor; inf represents the lower bound of the function; express and The joint distribution of ||·|| represents the modulus length operation; E(x, y)~γ[||xy||] represents the expected value of the Wasserstein distance between the cloud model output distribution and the edge model output distribution under the joint distribution γ; and They are the output distribution of the cloud model and the output distribution of the edge model respectively.

2. The abnormal behavior detection method according to claim 1, characterized in that: In step S1, the ResNet-50 model pre-trained on the training set ImageNet is used as the cloud model.

3. The abnormal behavior detection method according to claim 2, characterized in that: In step S3, ResNet-18 is used as the edge model.

4. The abnormal behavior detection method according to claim 1, characterized in that: α tv ＝1*10 -4 ， α f ＝5*10 -2 ，α w ＝1.0。 5. A computer device comprising a memory, a processor and a computer program stored in the memory; characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.

6. A computer program product comprising a computer program / instructions; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program / instruction stored thereon; characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images