A method, system, device, and readable storage medium for monitoring and detecting violent behavior.

Abstract

This invention discloses a method, system, device, and readable storage medium for detecting violent behavior, belonging to the field of computer vision technology. The method includes the following steps: Step 1, constructing a violent behavior video dataset; Step 2, constructing a three-dimensional convolutional neural network to extract features from the violent behavior video data; Step 3, using a multilayer perceptron to classify the feature data. This method, system, device, and readable storage medium replaces time-consuming and labor-intensive manual detection methods. It employs a densely connected network combining 3D convolutions to extract features from the violent behavior video data and uses a multilayer perceptron algorithm to classify the features extracted by the network. By replacing the 2D convolutional kernels in the densely connected network with 3D convolutional kernels, the convolutional neural network gains the ability to extract video features. The algorithm classifies violent behavior videos, and the computer analyzes the video content captured by the camera, saving manpower and preventing violent incidents.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a method, system, device, and readable storage medium for detecting violent behavior. Background Technology

[0002] With the development of neural network algorithms and the improvement of computer performance, neural network algorithms have been widely used in various fields. Surveillance cameras have penetrated into every corner of the city, curbing disorderly conduct and maintaining social security. However, how to quickly and efficiently detect violent behavior by surveillance cameras is a challenge. The amount of data from existing surveillance cameras is enormous, making it impossible to complete the task manually. Therefore, it is necessary to develop a method for detecting violent behavior by surveillance cameras to solve the existing problems. Summary of the Invention

[0003] The purpose of this invention is to provide a method, system, device, and readable storage medium for detecting violent behavior in surveillance cameras, so as to solve the problem of low efficiency in detecting violent behavior in surveillance cameras.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a method for detecting and monitoring violent behavior, comprising the following steps:

[0005] Step 1: Construct a dataset of videos depicting violent behavior;

[0006] Step 2: Construct a three-dimensional convolutional neural network to extract features from violent behavior video data;

[0007] Step 3: Use a multilayer perceptron to classify the feature data.

[0008] Preferably, the method for constructing the three-dimensional convolutional neural network is dense connection and 3D convolution.

[0009] Preferably, the construction of the dense connections and 3D convolutions includes the following steps:

[0010] Step 21: Connect each convolutional layer in the network to build a densely connected network; modify the convolutional kernels of the densely connected network, replacing all 2D convolutional kernels in the densely connected network with 3D convolutional kernels.

[0011] Step 22: Train the neural network using the dataset constructed in Step 1, so that the 3D convolutional neural network can extract image features.

[0012] The method of replacing all 2D convolutional kernels in a densely connected network with 3D convolutional kernels includes: generalizing 2D convolutional kernels to three dimensions using 3D convolutional kernels, for the third... The first 3D convolutional layer In the layer feature map Values ​​in coordinates The calculation formula is shown in Formula 1:

[0013]

[0014] in, For input sampling points, It is the current 3D convolution kernel in the th position. In the layer feature map Values ​​in coordinates For activation function, For bias, For the ( A collection of feature map indices for each layer. and The length and width of the convolution kernel, It is the size of the convolution kernel in the temporal direction; These are the input sampling points. The sampling points around the perimeter are obtained according to the definition of convolution. The weights are the sampling points.

[0015] Preferably, constructing the violent behavior video dataset includes collecting online datasets, retrieving surveillance videos, and editing violent behavior segments from the collected videos.

[0016] Preferably, in step 3, the multilayer perceptron is trained using the output of the trained network to enable it to perform classification, wherein the classification method used by the multilayer perceptron is binary classification.

[0017] Preferably, the 3D convolution includes a densely connected layer 1 with six 1×1×1 3D convolutional kernels and six 3×3×3 3D convolutional kernels, a transformation layer 1 with one 1×1×1 3D convolutional kernel and one 3×3×3 3D average pooling, a densely connected layer 2 with twelve 1×1×1 3D convolutional kernels and twelve 3×3×3 3D convolutional kernels, a transformation layer 2 with one 1×1×1 3D convolutional kernel and one 3×3×3 3D average pooling, a densely connected layer 3 with twenty-four 1×1×1 3D convolutional kernels and twenty-four 3×3×3 3D convolutional kernels, a transformation layer 3 with one 2×7×7 3D global max pooling, and a fully connected layer; wherein...

[0018] After transition layer 3, the activation function is shown in Equation 2; where, n For the dimensions of the input data, For dimension Input value, For dimension The output probability;

[0019] (2)

[0020] The output of the activation function is fed into a multilayer perceptron for classification.

[0021] The present invention also provides a system for monitoring and detecting violent behavior, the system comprising:

[0022] A module for building a violent behavior video dataset is used to construct such datasets.

[0023] A module for building 3D convolutional neural networks is provided for constructing 3D convolutional neural networks.

[0024] The data feature extraction module is used to extract features from video data of violent behavior;

[0025] The data classification module is used to classify feature data using a multilayer perceptron;

[0026] The dense connection and 3D convolution building module is used to connect each convolutional layer in the network to build a dense connection network; the convolution kernels of the dense connection network are modified to replace all 2D convolution kernels in the dense connection network with 3D convolution kernels; and the neural network is trained on the Hockey Fights dataset and the Violent-Flows dataset to extract image features.

[0027] The present invention also provides a device for monitoring and detecting violent behavior, comprising:

[0028] Memory for storing non-transitory computer-readable instructions; and

[0029] A processor for executing the computer-readable instructions such that when the computer-readable instructions are executed by the processor, the method for monitoring and detecting violent behavior is implemented.

[0030] The present invention also provides a computer-readable storage medium for storing non-transitory computer-readable instructions, which, when executed by a computer, cause the computer to perform the method for monitoring and detecting violent behavior.

[0031] The technical effects and advantages of this invention are as follows: This method, system, device, and readable storage medium for detecting violent behavior replaces the time-consuming and laborious manual detection method. It uses a densely connected network that combines 3D convolution to extract features from violent behavior video data and uses a multilayer perceptron algorithm to classify the features extracted from the violent behavior video data by the network.

[0032] By replacing the 2D convolutional kernels in a densely connected network with 3D convolutional kernels, the convolutional neural network is able to extract video features.

[0033] By classifying violent videos using algorithms and analyzing video content captured by cameras using computers, we can save manpower, reduce labor costs, and prevent violent incidents from occurring. Attached Figure Description

[0034] Figure 1 This is a flowchart of the present invention;

[0035] Figure 2 This is a flowchart illustrating the construction of dense connections and 3D convolutions in this invention;

[0036] Figure 3 This is a flowchart illustrating the framework of the present invention.

[0037] Figure 4 This is a structural diagram of the densely connected layer of the present invention;

[0038] Figure 5 This is a schematic diagram of 3D convolution in this invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] This invention provides, for example Figure 1 , Figure 3 The method for detecting and monitoring violent behavior shown includes the following steps:

[0041] Step 1: Construct a violent behavior video dataset. In this embodiment, videos monitoring violent behavior are collected in various forms, including online datasets and actual monitoring video retrieval. Violent behavior segments are then edited from the collected videos.

[0042] Step 2: An improved CNN is constructed using dense connections and 3D convolution to extract features from violent behavior video data; in this embodiment, the CNN is a convolutional neural network.

[0043] The feature extraction process for violent video data specifically includes the following steps: Figure 2 As shown,

[0044] Step 21: Construct a feature extraction CNN using a densely connected network; modify the network structure of the densely connected network; for example... Figure 4 As shown, the feature maps in each dense module are connected to fuse high- and low-level features, enabling the network model to better extract high- and low-level semantic features of the video.

[0045] like Figure 5 As shown, 3D convolution will be for the first The first 3D convolutional layer In the layer feature map Values ​​in coordinates The calculation formula is shown in Formula 1:

[0046]

[0047] in, For input sampling points, It is the current 3D convolution kernel's position in the th... In the layer feature map Values ​​in coordinates For activation function, For bias, For the ( A collection of feature map indices for each layer. and The length and width of the convolution kernel, It is the size of the convolution kernel in the temporal direction; These are the input sampling points. The sampling points obtained from the surrounding area according to the definition of convolution, The weights of the sampling points;

[0048] Step 22: Train the modified densely connected network using the dataset constructed in S21 to extract image features;

[0049] Step 3: Use the output of the trained network to train the multilayer perceptron to enable it to perform classification.

[0050] In this embodiment, the 3D-CNN is based on a densely connected network, including a densely connected layer 1 with six 1×1×1 and 3×3×3 3D convolutional kernels, a transformation layer 1 with one 1×1×1 and 3×3×3 3D average pooling kernel, a densely connected layer 2 with twelve 1×1×1 and 3×3×3 3D convolutional kernels, a transformation layer 2 with one 1×1×1 and 3×3×3 3D average pooling kernel, a densely connected layer 3 with twenty-four 1×1×1 and 3×3×3 3D convolutional kernels, a transformation layer 3 with one 2×7×7 3D global max pooling kernel, and a fully connected layer, i.e., a multilayer perceptron; wherein:

[0051] After transition layer 3, the activation function is:

[0052] (2)

[0053] The data can then be sent to a multilayer perceptron for classification.

[0054] The present invention also provides a system for monitoring and detecting violent behavior, the system comprising:

[0055] A module for building a violent behavior video dataset is used to construct such datasets.

[0056] A module for building 3D convolutional neural networks is provided for constructing 3D convolutional neural networks.

[0057] The data feature extraction module is used to extract features from video data of violent behavior;

[0058] The data classification module is used to classify feature data using a multilayer perceptron;

[0059] The dense connection and 3D convolution building module is used to connect each convolutional layer in the network to build a dense connection network; the convolution kernels of the dense connection network are modified to replace all 2D convolution kernels in the dense connection network with 3D convolution kernels; and the neural network is trained on the Hockey Fights dataset and the Violent-Flows dataset to extract image features.

[0060] The present invention also provides a device for monitoring and detecting violent behavior, comprising:

[0061] Memory for storing non-transitory computer-readable instructions; and

[0062] A processor for executing the computer-readable instructions such that when the computer-readable instructions are executed by the processor, the method for monitoring and detecting violent behavior is implemented.

[0063] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0064] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting violent behavior, characterized in that: Includes the following steps: Step 1: Construct a dataset of videos depicting violent behavior; Step 2: Construct a three-dimensional convolutional neural network to extract features from violent behavior video data; Step 3: Use a multilayer perceptron to classify the feature data; The method for constructing a three-dimensional convolutional neural network is dense connection and 3D convolution; the construction of dense connection and 3D convolution includes the following steps: Step 21: Connect each convolutional layer in the network to construct a densely connected network; modify the convolutional kernels of the densely connected network, replacing all 2D convolutional kernels in the densely connected network with 3D convolutional kernels; the 3D convolutions include a densely connected layer 1 with 6 1×1×1 3D convolutional kernels and 6 3×3×3 3D convolutional kernels, a transition layer 1 with 1 1×1×1 3D convolutional kernel and 1 3×3×3 3D average pooling, a densely connected layer 2 with 12 1×1×1 3D convolutional kernels and 12 3×3×3 3D convolutional kernels, a transition layer 2 with 1 1×1×1 3D convolutional kernel and 1 3×3×3 3D average pooling, a densely connected layer 3 with 24 1×1×1 3D convolutional kernels and 24 3×3×3 3D convolutional kernels, a transition layer 3 with 2×7×7 3D global max pooling, and a fully connected layer; wherein, after the transition layer 3, the activation function is as shown in Formula 2; wherein, n For the dimensions of the input data, For dimension Input value, For dimension The output probability; (2) The output of the activation function is sent to the multilayer perceptron for classification.

2. The detection method for monitoring violent behavior according to claim 1, characterized in that: The construction of the dense connections and 3D convolutions also includes the following steps: Step 22, training the neural network with the dataset constructed in Step 1, so that the three-dimensional convolutional neural network can extract image features.

3. The detection method for monitoring violent behavior according to claim 1, characterized in that: The method of replacing all 2D convolutional kernels in a densely connected network with 3D convolutional kernels includes: generalizing 2D convolutional kernels to three dimensions using 3D convolutional kernels, for the third... The first 3D convolutional layer In the layer feature map Values ​​in coordinates The calculation formula is shown in Formula 1: in, For input sampling points, It is the current 3D convolution kernel's position in the th... In the layer feature map Values ​​in coordinates For activation function, For bias, For the ( A collection of feature map indices for each layer. and The length and width of the convolution kernel, It is the size of the convolution kernel in the temporal direction; These are the input sampling points. The sampling points obtained from the surrounding area according to the definition of convolution, W The weights are the sampling points.

4. The detection method for monitoring violent behavior according to claim 1, characterized in that: The construction of the violent behavior video dataset includes collecting online datasets, retrieving surveillance videos, and editing violent behavior segments from the collected videos.

5. The detection method for monitoring violent behavior according to claim 1, characterized in that: In step 3, the multilayer perceptron is trained using the output of the trained network to enable it to perform classification. The multilayer perceptron uses binary classification.

6. A system for implementing the behavior detection method for monitoring violence according to any one of claims 1-5, the system comprising: A module for building a violent behavior video dataset is used to construct such datasets. A 3D convolutional neural network building module, used to build 3D convolutional neural networks; The data feature extraction module is used to extract features from video data of violent behavior; The data classification module is used to classify feature data using a multilayer perceptron; The dense connection and 3D convolution building module is used to connect each convolutional layer in the network to build a dense connection network; modify the convolutional kernels of the dense connection network to replace all the 2D convolutional kernels in the dense connection network with 3D convolutional kernels; and train the neural network with the constructed dataset so that the 3D convolutional neural network can extract image features.

7. A device for monitoring and detecting violent behavior, comprising: Memory is used to store non-transitory computer-readable instructions; as well as A processor for executing the computer-readable instructions such that, when executed by the processor, the computer-readable instructions implement the detection method for monitoring violent behavior according to any one of claims 1 to 5.

8. A computer-readable storage medium for storing non-transitory computer-readable instructions, which, when executed by a computer, cause the computer to perform the detection method for monitoring violent behavior as described in any one of claims 1 to 5.

Images