A portable abnormal behavior intelligent analysis system

Abstract

The present invention provides a portable abnormal behavior intelligent analysis system, which mainly includes an edge device backpack, a cloud server and a mobile platform. The edge device backpack includes a lithium battery and an intelligent inverter, a surveillance camera, a tripod, an intelligent behavior recognition system, and an edge computing device; the intelligent abnormal behavior recognition uses the edge device as a carrier to realize real-time alarm of abnormal behavior, display the results on the mobile terminal and provide alarm processing function; the cloud server is used for training the intelligent behavior recognition model and updating the parameters. The purpose of this system is to build a portable video surveillance alarm platform, deploy cameras, add edge computing rules, realize abnormal behavior alarm function, process alarm information in real time, and reduce alarm delay.

Description

Technical Field

[0001] The present invention relates to the technical fields of image processing, artificial intelligence, edge computing, cloud computing, and the like, and in particular to a portable abnormal behavior intelligent analysis system. Background Art

[0002] With the development of economy and the progress of the times, a large number of scenarios require the use of behavior analysis algorithms, such as construction sites, large-scale sports events and campus risk monitoring. Traditional behavior analysis relies on manual analysis of the images of manual surveillance cameras, which is costly and inefficient. With the development of computer vision and deep learning, abnormal behavior recognition can be analyzed and processed using artificial intelligence algorithms, which reduces costs and improves recognition accuracy. However, artificial intelligence algorithms require server devices with powerful computing power to achieve recognition effects and accuracy. The disadvantage is that artificial intelligence server devices are relatively expensive, large in size and weight, and require a lot of electricity, so they cannot be deployed portable.

[0003] In order to solve the above problems. We designed an edge device backpack, which contains all the equipment required for edge computing. When users need to temporarily build an edge computing terminal outdoors, they only need to open the backpack and assemble various devices. The edge computing terminal is a distributed computing that processes the collected data nearby without uploading a large amount of data to the core management platform of the cloud server. Compared with cloud server computing, edge computing uses the existing data and computing power on the cloud server to greatly shorten the response time of the recognition result. When the recognition result is returned, the feature information of the video will be sent to the cloud server as a new training set. Edge computing has the advantages of low latency, high agility, high reliability, high scalability, and reduced pressure on network transmission. The edge device backpack is small in size, light in weight, and low in power consumption. It can achieve rapid deployment and solve the disadvantage of traditional servers being not portable enough.

[0004] In order to allow edge devices to run for a long time outdoors without power supply, we use lithium batteries and smart inverters as edge energy sources and integrate edge devices into a backpack. This allows you to build an intelligent behavior recognition system anytime and anywhere without being restricted by the weight of the equipment, and the cost is only one-fifth of that of traditional servers. Summary of the invention

[0005] The embodiment of the present application proposes a portable abnormal behavior intelligent analysis system, which mainly includes an edge device backpack, a cloud server and a mobile terminal. The purpose is to build a portable video surveillance alarm platform, deploy cameras, add edge computing rules, realize abnormal behavior alarm function, and reduce alarm delay.

[0006] To achieve the above objectives, this application adopts the following technical solutions:

[0007] Step 1: A portable abnormal behavior intelligent analysis system, characterized in that it is composed of an edge device backpack, a cloud server and a mobile terminal, wherein the edge device backpack contains a lithium battery and an intelligent inverter, a tripod, an edge terminal and an intelligent behavior analysis system. The camera is used to obtain real-time video human movement information; the lithium battery and the intelligent inverter are used to provide long-term power supply to the edge terminal; the intelligent abnormal behavior recognition system uses the edge computing device as a carrier, and the built-in artificial intelligence algorithm is used to realize the recognition of abnormal behavior and the sending of abnormal information, and the abnormal information is sent to the cloud server; the cloud server is used for abnormal information processing and the training and parameter update of the intelligent abnormal behavior recognition system model. The mobile terminal is used to display alarm information and remotely replace the edge algorithm model. The overall system architecture is shown in Figure (1), and the portable edge backpack is shown in Figure (2).

[0008] Step 2: A portable intelligent behavior analysis system based on edge devices, characterized in that the device comprises three modules in total, a remote edge device related module, a cloud server related module, and a mobile terminal module. The functions of each module are shown in Figure (3).

[0009] Step 3: The intelligent abnormal behavior recognition process based on the edge device is shown in Figure (4), and the neural network architecture is shown in Figure (5), which is characterized by including the following steps:

[0010] After the edge camera acquires the video data, it transmits the data to the edge computing device, where the deployed data preprocessing model will preprocess the video information, including converting the video information into image information, image denoising, and image enhancement. Data enhancement can enrich the data set and increase the robustness of the deep learning network.

[0011] The enhanced image data is input into the D-YOLOv5 deep learning network built into the intelligent behavior recognition system, and the network resizes all images to 640*640. The feature extraction part uses a combination of a deformable convolution module and a bottleneck CSP module. The deformable convolution module contains a 3*3 deformable convolution kernel, batch normalization, and a leaky Relu activation function. Traditional convolution kernels are of fixed size and have insufficient generalization capabilities. Deformable convolution realizes the size adjustment of the convolution kernel on the basis of traditional convolution, and adds the direction vector of the convolution kernel to make the convolution kernel closer to the feature object. The role of the bottleneck CSP network is that CSPNet solves the problem of gradient information duplication in network optimization in other large convolutional neural network framework skeleton networks, and integrates the gradient changes from beginning to end into the feature map, thereby reducing the number of model parameters and FLOPS values, which not only ensures the inference speed and accuracy, but also reduces the model size. After four times of the above operation, feature maps of various scales are generated, including 160*160, 80*80, 40*40, and 20*20. The SPP module uses the maximum pooling operation to fuse feature maps of different sizes to form a new feature map. FPN transfers the strong semantic features of high-level feature maps to enhance the entire feature pyramid. PAN is a bottom-up pyramid after FPN, which transfers the strong positioning information of the bottom layer. D-YOLOv5 uses the intersection-over-union loss as the loss function of the bounding box. Non-maximum suppression is used for multiple target boxes generated by regression, and the highest score prediction box is selected.

[0012] Step 4: The function of the cloud server module is to realize the training of the intelligent early warning system model and the update of parameters. It is characterized in that the edge server uploads the video information summarized by the edge device and the video of the alarm information to the cloud server. The server uses the relevant video to further train the corresponding model, update the parameters of the model, and improve the accuracy of the model. The server also includes a data analysis and processing module to process alarm information, a resource management module, and stores the video information and algorithm model returned by the edge.

[0013] Step 5: Replace the anomaly detection algorithm model on the mobile terminal.

[0014] The algorithm modules integrated in the edge end can realize different abnormal behavior detection functions by using different models. The mobile end displays all functional modules such as detecting the wearing of safety helmets and detecting the dress code. Click one of the modules, and the mobile end will send the selected abnormal behavior identification information to the cloud server. The server will parse the information and send the relevant functional module model files to the edge end. The edge box will call the updated model files to identify new abnormal behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1This is the system architecture diagram;

[0016] Figure 2 This is the structure diagram of the edge device backpack;

[0017] Figure 3 It is the system function module diagram;

[0018] Figure 4 Identify flow charts for abnormal behavior;

[0019] Figure 5 This is a neural network architecture diagram DETAILED DESCRIPTION

[0020] The implementation example of this application has solved the current technical problems of high cost of deep learning equipment, large model parameters, poor portability, low detection accuracy and high transmission delay by providing a portable abnormal behavior intelligent analysis system.

[0021] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0022] Step 1: The edge uses a camera to obtain video data, and a lithium battery and smart inverter are used to provide power to the edge computing server. The built-in artificial intelligence algorithm of the edge computing server first uses padding, random detection, image denoising and other methods to enhance the video data. The deep learning network uses the D-CSPDarknet50 backbone network to extract the features of the video frame, and uses 3 convolution kernels of size 3*3 and a sliding step of 2 for downsampling. Padding is 1. The dimension of the feature map becomes 512*20*20. The bottleneck CSP module uses a residual module to fuse the feature map information. The spatial pyramid module uses a standard convolution module to halve the input channels, and then uses the maximum pooling operation with kernel-size 5, 9 and 13 to reduce the number of parameters. The feature map is merged with the results of the three maximum pooling operations and the data without pooling operations, and the final number of channels is twice the original.

[0023] Channel Block Attention Module (CBAM) is a hybrid domain attention mechanism that evaluates and scores channel attention and spatial attention at the same time. CBAM contains two sub-modules, Channel Attention Module and SpartialAttention Module, which implement channel and spatial attention respectively. The FPN layer conveys strong semantic features from top to bottom, while the feature pyramid conveys strong positioning features from bottom to top, and aggregates parameters of different detection layers from different backbone layers. The bounding box prediction part has three output heads with strides of 8, 16, and 32 respectively. The large output feature map is used to detect small objects, and the small output feature map is used to detect large objects. In the post-processing process of target detection, the NMS operation is used to screen many target boxes, and the prediction results are finally output.

[0024] Step 2: The edge sends the identified abnormal information and the 15-second video clips before and after the abnormal video to the cloud server. The server parses the information and determines whether an alarm is needed. Information that meets the alarm requirements will be sent to the mobile terminal. The server uses the relevant abnormal videos to further train the corresponding algorithm model, update the model parameters, and improve the recognition accuracy of the model.

[0025] Step 3: When the mobile terminal needs to remotely replace the model file being used by the edge terminal, the mobile terminal selects the model information, and the selected model type will be sent to the cloud server. After the server parses it, the updated model file data will be sent to the edge terminal, and the edge box at the edge terminal uses the new model to analyze the live video.

Claims

1. A portable abnormal behavior intelligent analysis system, comprising: It consists of edge device backpack, cloud server and mobile terminal; The edge device backpack includes a lithium battery and a smart inverter, a surveillance camera, a tripod, an intelligent behavior recognition system, and an edge computing device; the lithium battery and the smart inverter are used to provide power to the edge computing device; the surveillance camera is used to obtain real-time video images of the surveillance area; The tripod is used to support the camera; the intelligent behavior recognition system uses the edge computing device as a carrier to analyze the video information obtained by the surveillance camera using a deep learning algorithm to determine whether there is abnormal behavior; The abnormal behavior data is sent to the cloud server for data analysis; the cloud server is used to process and analyze edge data, use deep learning algorithms to train abnormal behavior recognition models, and update parameters, store data, and manage resources; the mobile terminal is used to notify, display and process alarm information, and provides the function of remotely replacing abnormal behavior recognition models.

2. A portable abnormal behavior intelligent analysis system according to claim 1, comprising the following steps: S21 is used to implement the steps of abnormal behavior identification: The camera uploads the acquired human behavior video information to the edge computing device. The intelligent behavior recognition system deployed in advance in the device performs deep learning reasoning on the video information to obtain the corresponding behavior category and confidence level, and uploads the alarm information with a confidence level higher than the preset value to the cloud server. S22 is used to implement the steps of data analysis and model updating: The cloud server receives the abnormal information sent by the edge device, analyzes the information and saves the alarm screen video, and sends the alarm information to the mobile terminal; the video information sent by the edge device is divided into positive samples and negative samples, and the relevant videos are used to conduct a new round of deep learning algorithm training on the corresponding abnormal behavior recognition model, update the model parameters, and improve the model recognition accuracy; S23 is used to implement the steps of replacing multiple abnormal behavior recognition models: The mobile terminal has multiple abnormal behavior recognition modules to choose from. The selected abnormal behavior recognition module information is sent to the cloud server. The server forwards its model file to the edge computing device. The deep learning algorithm in the intelligent behavior recognition system uses the new model file to perform abnormal behavior recognition. S24 Portable Edge Device Portfolio: The edge device backpack consists of a lithium battery and smart inverter, a surveillance camera, a tripod, an intelligent behavior recognition system, and an edge computing device. The entire edge device is integrated into a backpack with high device integration, small size, and light weight, making it convenient for field deployment.

3. According to the portable abnormal behavior intelligent analysis system of claim 2, step S21 is used to realize abnormal behavior identification, characterized in that: Collect various types of human behavior video data at construction sites, divide the data into positive and negative samples, and mark abnormal behaviors; convert video information into images and perform image enhancement. The enhancement methods include: image occlusion, adaptive Gaussian filter denoising and affine transformation; pass the enhanced images into the D-YOLOv5 deep learning algorithm; The input of the algorithm performs adaptive anchor frame calculation and adaptive image scaling on the image; the image is input into the D-Darkent50 feature extraction network for feature extraction. The feature extraction network has a total of 50 layers, and each layer uses 3 3*3 deformable convolutions for feature extraction, which reduces the model parameters and improves the feature recognition accuracy; the extracted feature map is passed to the Neck network and the FPANet network is used for feature fusion. The feature maps of different periods are fused to enhance the model to detect objects of different scales; the FPANet network is based on the FPN feature fusion network and the PAN feature fusion network. At the same time, we add a channel attention mechanism to the feature fusion to simulate humans and give different weights to different channel information; the feature extractor of this network adopts a new FPN structure that enhances the bottom-up path, which improves the propagation of low-level features; the model uses the Head network for final detection; in D-YOLOV5, the Leaky Relu activation function is used in the middle and hidden layers; The formula is: where a i is a fixed parameter in the interval (1, +∞); The final detection layer uses the Sigmoid activation function, and its formula is as follows:

4. According to the portable abnormal behavior intelligent analysis system of claim 2, step S22 is used to implement data analysis and model update, characterized in that: The cloud server includes a data analysis module, an algorithm training module, an information storage module, and an abnormal information alarm and forwarding module; the data analysis module is used to classify and analyze the information received by the system and count the frequency of each type of alarm information sent; the algorithm training module is used to perform deep learning training in the cloud, update the parameters of the model, and improve the accuracy of model recognition; the information storage module is used to store alarm information and alarm-related videos, regularly delete data that has not been used for a long time, and improve the efficiency of storage space use; the abnormal information alarm and forwarding module is used to process the abnormal alarm data sent by the edge server and determine whether the requirements for generating alarm information are met; If the requirements are met, various alarm information will be generated and the alarm data will be forwarded to the mobile terminal.

5. According to the portable abnormal behavior intelligent analysis system of claim 2, step S23 is used to implement the replacement of multiple abnormal behavior recognition models, characterized in that: The mobile function selection page displays a variety of optional abnormal behavior recognition modules, and sends the abnormal behavior recognition model information selected by the user to the cloud server. After parsing the data, the server sends the abnormal behavior recognition model file stored in the cloud to the edge server. Its built-in deep learning algorithm uses the new model file to perform new abnormal behavior recognition, enabling one algorithm on the edge to run multiple types of behavior recognition modules, and the mobile terminal can remotely change the edge algorithm model with one click.

6. According to the portable abnormal behavior intelligent analysis system of claim 2, the portable edge device combination of step S24 is characterized in that: The edge device backpack contains all the devices at the edge end, providing a backpack-style portable smart device. A two-way invisible zipper is set on the opening of the backpack body, and symmetrical straps are set on one side of the backpack body. The backpack has four built-in storage compartments; Lithium batteries and smart inverters, cameras, tripods, and edge computing devices are stored separately; the lithium batteries and smart inverters are used to provide power to the edge computing devices; the cameras are used to capture real-time video images of the monitoring area; The tripod is used to support the camera; The edge computing device is used to carry an intelligent behavior recognition system; the edge device backpack is lightweight, small in size, and has waterproof and sun-proof functions. When used outdoors, the edge platform can be quickly built.

Images