Personnel abnormal behavior monitoring identification method and early warning system

Abstract

The invention relates to the field of personnel abnormal behavior monitoring, in particular to a personnel abnormal behavior monitoring identification method and early warning system.The method comprises the steps that a convolutional neural network model is constructed, and the convolutional neural network model comprises a first convolutional layer, a channel multi-scale difference attention layer and a second convolutional layer; in the channel multi-scale difference attention layer, convolution is carried out on each first feature map output by the first convolution layer through a convolution kernel to obtain two second feature maps, pooling is carried out on each second feature map to obtain feature matrixes, the Frobenius norms of the two feature matrixes are calculated, and the Frobenius norms serve as the weights of the first feature maps; performing weighted connection on the plurality of first feature maps and inputting the first feature maps to a second convolutional layer; and training the convolutional neural network model to obtain a detection model, inputting the acquired personnel image into the detection model to detect the abnormal behavior of the personnel, and if the abnormal behavior of the personnel is detected, giving out an early warning. According to the invention, the accuracy of abnormal behavior recognition of the neural network model is improved.

Description

Technical Field

[0001] The present invention relates to the field of monitoring abnormal behavior of personnel, and in particular to a monitoring and identifying method and an early warning system for abnormal behavior of personnel. Background Art

[0002] Traditional public area monitoring usually involves security personnel patrolling and checking the corresponding areas, as well as checking pedestrians, vehicles, and items carried by them to ensure the safety of public areas. When there is a large flow of people, a large amount of human resources is required. With the development of Internet technology, video surveillance technology has been applied to monitoring abnormal behavior of personnel. By detecting the behavior, expression, or whether a person is carrying dangerous items, it is determined whether the corresponding person has abnormal behavior so that timely measures can be taken to prevent dangerous behavior.

[0003] A Chinese patent application document with publication number CN110826522A discloses a method, system, storage medium and monitoring equipment for monitoring abnormal human behavior. The method includes: obtaining a video data set uploaded in a human behavior database; updating a preset I3D model using octave convolution to obtain a target I3D model; training and testing the video data set through the target I3D model to obtain a behavior prediction model; inputting video information of personnel flow in a monitoring area into the behavior prediction model to obtain abnormal information of personnel with abnormal behavior in the monitoring area, and generating early warning and warning information according to the type of abnormal behavior.

[0004] However, in the actual detection process, due to the small size of dangerous objects, such as knives, sticks, etc., there is a large gap between dangerous objects and the human body. Dangerous objects occupy fewer pixels in the image and their features are relatively fuzzy. Existing image recognition models tend to identify dangerous objects and human bodies as human bodies, thereby identifying abnormal behaviors as normal behaviors, resulting in low accuracy in the model's identification of abnormal behaviors. Summary of the invention

[0005] In order to solve the problem that dangerous objects are small in size and difficult to identify, the present invention provides a method for monitoring and identifying abnormal behavior of personnel and an early warning system.

[0006] In a first aspect, the present invention provides a method for monitoring and identifying abnormal behavior of personnel, which adopts the following technical solution: Construct a convolutional neural network model, which includes a convolutional layer 1, a channel multi-scale difference attention layer, and a convolutional layer 2; In the channel multi-scale difference attention layer, each first feature map output by the convolution layer 1 is convolved with a convolution kernel to obtain two second feature maps, each second feature map is pooled to obtain a feature matrix, the Frobenius norm of the two feature matrices is calculated, and the Frobenius norm is used as the weight of the first feature map; multiple first feature maps are weightedly connected and input to the convolution layer 2; The convolutional neural network model is trained to obtain a detection model, and the acquired personnel images are input into the detection model to detect abnormal personnel behavior. If abnormal personnel behavior is detected, an early warning is issued.

[0007] By assigning corresponding weights to the first feature map through the channel multi-scale difference attention layer, different first feature maps can be distinguished, the model's attention to channels with large multi-scale differences is improved, and the model's abnormal behavior recognition accuracy is improved.

[0008] Preferably, the convolutional neural network model also includes an input layer and a multi-scale convolution layer. The input layer inputs the three-channel image of the personnel image into the multi-scale convolution layer, uses multiple convolution kernels to convolve the three-channel image to obtain multiple initial feature maps, calculates the receptive field of the initial feature map in the personnel area in the personnel image, calculates the weight of the corresponding initial feature map according to the ratio of the receptive field to the personnel area, and inputs the multiple initial feature maps into the convolution layer one through weighted connection.

[0009] The weight of each initial feature map is calculated through the receptive field corresponding to the initial feature map. The larger the weight, the more important the corresponding feature map is. Compared with the traditional fixed weight connection method, the accuracy of identifying distinguishing features (dangerous goods) is improved.

[0010] Preferably, the weight expression of the initial feature map is: ; In the formula, represents the weight of the initial feature map, avg represents the average function, represents the receptive field of the initial feature map on the person image, It is the person area with abnormal behavior on the person image.

[0011] By calculating the weight of the corresponding initial feature map through the receptive field, it is easy to understand the importance of the initial feature map, thereby improving the accuracy of identifying dangerous objects.

[0012] Preferably, the method for training the convolutional neural network model is: Obtain a training set, which includes multiple images of people, and set a corresponding label for each image of the person, including normal and abnormal labels; Use cross entropy loss as the loss function and input the training set into the convolutional neural network model for training; The training is stopped in response to the number of training times reaching a set maximum number of training times or the loss of the convolutional neural network model being less than a preset loss threshold.

[0013] By training the convolutional neural network model, a detection model is obtained, which can be used to identify abnormal behaviors of personnel.

[0014] Preferably, the loss function is expressed as: ; In the formula, represents the prediction loss of the i-th person image, r represents the label of the i-th person image, whose value is 0 or 1, and p represents the probability of the convolutional neural network model predicting it to be abnormal.

[0015] By determining the loss function, the accuracy of training the convolutional neural network model to identify abnormal behaviors is improved.

[0016] Preferably, global average pooling or global maximum pooling is performed on each second feature map.

[0017] Preferably, the convolutional neural network model also includes a global attention layer, a fully connected layer and an output layer.

[0018] In a second aspect, the present invention provides a warning system for abnormal behavior of personnel, which adopts the following technical solution: A personnel abnormal behavior early warning system, a processor and a memory, the memory storing computer program instructions, when the computer program instructions are executed by the processor, an abnormal behavior monitoring and identification method for personnel is implemented.

[0019] The beneficial effect is: the above-mentioned method for monitoring and identifying abnormal behavior of personnel is generated into a computer program and stored in a memory so as to be loaded and executed by a processor, thereby making a system based on the memory and the processor for easy use.

[0020] The present invention has the following technical effects: By assigning corresponding weights to the first feature maps through the channel multi-scale difference attention layer, different first feature maps can be distinguished, the model's attention to channels with large multi-scale differences can be improved, and greater weights can be given to the first feature maps that are beneficial to dividing small objects and human bodies. The trained model can maintain a certain attention on human body recognition while improving its attention to small objects, thereby avoiding the phenomenon of identifying dangerous objects and human bodies as human bodies, and improving the accuracy of the model's abnormal behavior recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0022] Figure 1 The present invention is a flowchart of a method for monitoring and identifying abnormal behavior of personnel according to an embodiment of the present invention.

[0023] Figure 2 It is a flowchart of the operation of the multi-scale convolutional layer in an embodiment of the present invention.

[0024] Figure 3 It is a flow chart of the operation of the channel multi-scale difference attention layer in an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0026] It should be understood that when the terms "first", "second", etc. are used in the claims, descriptions, and drawings of the present invention, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the description and claims of the present invention indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their collections.

[0027] The embodiment of the present invention discloses a method for monitoring and identifying abnormal behavior of personnel, referring to Figure 1 , including the following steps, as follows: S1: Build a convolutional neural network model.

[0028] The convolutional neural network model includes input layer, multi-scale convolution layer, convolution layer one, channel multi-scale difference attention layer and convolution layer two, global attention layer, fully connected layer and output layer.

[0029] S11: The input layer inputs the three channel images corresponding to the person image to the multi-scale convolution layer, where the three channels are R, G, and B channels, that is, the R channel corresponds to one image, the G channel corresponds to one image, and the B channel corresponds to one image. The image input to the input layer can be regarded as an input tensor. For example, an input tensor is: , Represents the input tensor, H, W, and C represent the height, width, and number of channels of the input tensor respectively.

[0030] S12: The multi-scale convolution layer uses multiple convolution kernels to convolve the three-channel image to obtain multiple initial feature maps, calculates the receptive field of the initial feature map in the person area in the person image, calculates the weight of the corresponding initial feature map according to the ratio of the receptive field to the person area, and weightedly connects the multiple initial feature maps and inputs them into the convolution layer one.

[0031] Exemplary, combined Figure 2 As shown, the size of the three-channel image is 32×32. A 3×3 convolution kernel is used to convolve the input three-channel image. The convolution step is 1, and the padding is 1, 2, and 3 respectively (the purpose of padding is to ensure that the size of the output initial feature map is the same as the size of the input three-channel image). The convolution obtains an initial feature map of size 32×32. Similarly, using The convolution kernel convolves the input tensor to obtain an initial feature map of size 32×32 ,use The convolution kernel convolves the input tensor to obtain an initial feature map of size 32×32 , the corresponding tensor for: .

[0032] For the initial feature map , calculate the receptive field of the initial feature map in the personnel area in the personnel image, and calculate the ratio of the receptive field to the personnel area. The expression is: , represents the receptive field of the initial feature map on the person image, It is the person area with abnormal behavior on the person image.

[0033] In the images of people with abnormal behavior in history, each person image corresponds to a ratio, and the average of multiple ratios is used as the initial feature map The weight of the initial feature map is expressed as: ; In the formula, represents the weight of the initial feature map, avg represents the average function, represents the receptive field of the initial feature map on the person image, is the region of the person with abnormal behavior on the person image. It can be seen that the initial feature map , and Each corresponds to a weight, and the weight is normalized to obtain , and .

[0034] Multiple initial feature maps are weightedly connected to obtain a weighted initial feature map. The expression of weighted connection is: , is the weighted feature map after weighted connection, is the initial feature map The corresponding weight, is the initial feature map The corresponding weight, is the initial feature map Corresponding weights. The weight of each initial feature map is calculated through the receptive field corresponding to the initial feature map. The larger the weight, the more important the information contained in the corresponding feature map. Compared with the traditional fixed weight connection method, the accuracy of the convolutional neural network model in identifying distinguishing features (dangerous goods) is improved.

[0035] S13: The convolution layer performs convolution processing on the weighted feature map output by the multi-scale convolution layer to obtain a first feature map.

[0036] Exemplarily, the convolution layer 1 contains 32 convolution kernels of size 3×3, a step size of 1, a padding of 1, and the convolution layer 1 outputs 32 first feature maps of size 32×32.

[0037] S14: The channel multi-scale difference attention layer uses the convolution kernel to convolve each first feature map output by the convolution layer 1 to obtain two second feature maps, pools each second feature map to obtain a feature matrix, calculates the Frobenius norm of the two feature matrices, and uses the Frobenius norm as the weight of the first feature map; multiple first feature maps are weighted and connected and input to the convolution layer 2. The pooling method uses global average pooling or global maximum pooling.

[0038] Exemplary, combined Figure 3 As shown in the figure, the first feature map g is convolved with a 3×3 convolution kernel to obtain a second feature map p, and the first feature map g is convolved with a 7×7 convolution kernel to obtain a second feature map q. The second feature map p is pooled to obtain a feature matrix A, and the second feature map q is pooled to obtain a feature matrix B. The Frobenius norm between the feature matrix A and the feature matrix B is calculated. The expression of the Frobenius norm is: ; Where D represents the Frobenius norm between the feature matrix A and the feature matrix B, represents the element in the xth row and yth column of the feature matrix A, represents the element of the xth row and yth column in the feature matrix B, and m and n represent the number of rows and columns of the feature matrix respectively. The Frobenius norm is used as the weight of the first feature map b. The weighted first feature map is obtained by weighted connection of multiple first feature maps. The weighted connection method of the first feature map is the same as that of the initial feature map, which will not be repeated here.

[0039] Due to the large volume difference between dangerous objects (such as knives and sharp objects) and human bodies in abnormal behaviors of people, it is specifically manifested as follows: the characteristics of dangerous objects are mainly reflected in small-scale characteristics, and the characteristics of people are mainly reflected in large-scale characteristics; for dangerous objects, the smaller the scale difference between dangerous objects and people, the easier it is for the model to identify dangerous objects and human bodies as one, and to mistakenly identify abnormal behaviors as normal behaviors. The larger the scale difference between dangerous objects and people, the easier it is for the model to accurately divide and identify dangerous objects and human bodies. Therefore, by assigning corresponding weights to the first feature maps through the channel multi-scale difference attention layer, different first feature maps can be distinguished, the model's attention to channels with large multi-scale differences can be improved, and the first feature maps that are conducive to dividing small objects and human bodies can be given greater weights, thereby distinguishing dangerous objects from people.

[0040] S15: The convolution layer 2 performs convolution processing on the weighted first feature map output by the channel multi-scale difference attention layer to obtain a third feature map.

[0041] For example, the convolutional layer 2 contains 64 pixels of size The convolution kernel has a stride of 1 and a padding of 1. The convolution layer 2 outputs 64 third feature maps with a size of 32×32.

[0042] S16: The third feature map is input into the global attention layer. The global attention layer convolves the third feature map input by the convolution layer 2 to obtain query vectors and key vectors at different positions, and calculates the similarity between any two positions in the same third feature map.

[0043] The expression of similarity is: ; In the formula, represents the similarity between position i and position j in the third feature graph, f represents the dimension of the query vector or key vector, The query vector representing position i is l The value on the dimension, The key vector representing position j is l The softmax function is used to normalize the similarities of all positions to obtain the attention weight matrix, the attention weight matrix is ​​multiplied by the third feature map to obtain the weighted third feature map, and the weighted third feature map is connected to obtain the output feature map.

[0044] Exemplarily, the number of output feature maps is 64 and the size is 32×32.

[0045] S17: Input the output feature map to the fully connected layer, and output the behavior category of the person in the person image through the output layer, and the behavior category is normal or abnormal.

[0046] Exemplarily, 64 feature maps are input into a fully connected layer containing 128 output neurons, the output size of the fully connected layer is 1×1×128, and the output layer outputs the behavior category of the person in the person image as abnormal.

[0047] S2: Train the convolutional neural network model to obtain a detection model.

[0048] The method for training the convolutional neural network model is: obtaining a training set, the training set includes multiple images of people, setting corresponding labels for each image of a person, the labels include normal and abnormal; using cross entropy loss as the loss function, the expression of the loss function is: ; In the formula, represents the prediction loss of the i-th person image, r represents the label of the i-th person image, and its value is 0 or 1, 1 represents abnormality, 0 represents normality, and p represents the probability of the convolutional neural network model predicting abnormality. The training set is input into the convolutional neural network model for training; the training is stopped in response to the maximum number of training times set or the loss of the convolutional neural network model is less than the preset loss threshold.

[0049] S3: Input the acquired personnel image into the detection model to detect abnormal personnel behavior. If abnormal personnel behavior is detected, an early warning is issued.

[0050] The images of people are acquired in real time and input into the detection model to detect abnormal behavior of people. If abnormal behavior of people is detected, an early warning is issued to prompt the corresponding security personnel to pay attention to the abnormal people in time to prevent the abnormal people from causing harm to the surrounding people.

[0051] An embodiment of the present invention further discloses a personnel abnormal behavior warning system, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a personnel abnormal behavior monitoring and identification method according to the present invention is implemented.

[0052] The above system also includes other components well known to those skilled in the art, such as a communication bus and a communication interface, and their configuration and functions are known in the art, so they will not be described in detail here.

[0053] In the present invention, the aforementioned memory may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus or device. For example, a computer-readable storage medium may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high bandwidth memory HBM (High Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc., or any other medium that can be used to store the required information and can be accessed by an application, a module or both. Any such computer storage medium may be part of a device or accessible or connectable to a device.

[0054] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

[0055] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for monitoring and identifying abnormal behavior of personnel, characterized in that: Includes steps: Construct a convolutional neural network model, which includes a convolutional layer 1, a channel multi-scale difference attention layer, and a convolutional layer 2; In the channel multi-scale difference attention layer, each first feature map output by the convolution layer 1 is convolved with a convolution kernel to obtain two second feature maps, each second feature map is pooled to obtain a feature matrix, the Frobenius norm of the two feature matrices is calculated, and the Frobenius norm is used as the weight of the first feature map; multiple first feature maps are weightedly connected and input to the convolution layer 2; The convolutional neural network model is trained to obtain a detection model, and the acquired personnel images are input into the detection model to detect abnormal personnel behavior. If abnormal personnel behavior is detected, an early warning is issued.

2. A method for monitoring and identifying abnormal behavior of personnel according to claim 1, characterized in that: The convolutional neural network model also includes an input layer and a multi-scale convolution layer. The input layer inputs the three-channel image of the person image into the multi-scale convolution layer, uses multiple convolution kernels to convolve the three-channel image to obtain multiple initial feature maps, calculates the receptive field of the initial feature map in the person area in the person image, calculates the weight of the corresponding initial feature map according to the ratio of the receptive field to the person area, and inputs the multiple initial feature maps into the convolution layer one through weighted connection.

3. A method for monitoring and identifying abnormal behavior of personnel according to claim 2, characterized in that: The weight expression of the initial feature map is: ; In the formula, represents the weight of the initial feature map, avg represents the average function, represents the receptive field of the initial feature map on the person image, It is the person area with abnormal behavior on the person image.

4. A method for monitoring and identifying abnormal behavior of personnel according to claim 1, characterized in that: The method for training the convolutional neural network model is: Obtain a training set, which includes multiple images of people, and set a corresponding label for each image of the person, including normal and abnormal labels; Use cross entropy loss as the loss function and input the training set into the convolutional neural network model for training; The training is stopped in response to the number of training times reaching a set maximum number of training times or the loss of the convolutional neural network model being less than a preset loss threshold.

5. A method for monitoring and identifying abnormal behavior of personnel according to claim 4, characterized in that: The expression of the loss function is: ; In the formula, represents the prediction loss of the i-th person image, r represents the label of the i-th person image, whose value is 0 or 1, and p represents the probability of the convolutional neural network model predicting it to be abnormal.

6. A method for monitoring and identifying abnormal behavior of personnel according to claim 1, characterized in that: Perform global average pooling or global maximum pooling on each second feature map.

7. A method for monitoring and identifying abnormal behavior of personnel according to claim 1, characterized in that: The convolutional neural network model also includes a global attention layer, a fully connected layer, and an output layer.

8. A warning system for abnormal behavior of personnel, characterized in that: include: A processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a method for monitoring and identifying abnormal behavior of personnel according to any one of claims 1 to 7 is implemented.

Images