A real-time monitoring and identification method for abnormal action

Abstract

The application provides a real-time monitoring and identification method for abnormal action, expands sample data, and performs gray-scale processing on the sample; performs local model operation processing on the training sample; constructs a composite neural network model; in a real-time video monitoring process, if the output of the composite neural network is Y=0, it indicates that the image monitored at this moment is a normal action image, and the alarm system does not work; if the output of the composite neural network is Y=1, it indicates that the image monitored at this moment is an abnormal action image, representing the occurrence of abnormal action, and the alarm system issues an alarm. The application improves the real-time identification rate and accuracy of the monitoring system for abnormal action.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of intelligent monitoring, and particularly relates to a real-time monitoring and identification method for abnormal actions. BACKGROUND

[0002] Due to the rapid economic development, people's life rhythm is getting faster and faster, and the time for taking care of and understanding the family situation is getting less and less. The application of the intelligent monitoring system in the family realizes the remote care of children, the elderly and other needs. The development of the family monitoring network camera plays an important role in the application degree of the intelligent monitoring system.

[0003] However, the existing monitoring system has slow identification speed, low identification accuracy, frequent false alarms, no pre-alarm or pre-alarm delay when abnormal actions occur. The user experience of the intelligent monitoring is poor, and the use rate is low. Therefore, improving the identification speed and accuracy of the monitoring system is a problem to be solved in the field of intelligent monitoring at present. SUMMARY

[0004] The application proposes a real-time monitoring and identification method for abnormal actions, aiming to improve the real-time identification speed and accuracy of the monitoring system for abnormal actions.

[0005] The real-time monitoring and identification method for abnormal actions comprises the following steps:

[0006] S10, expanding sample data and performing gray scale processing on the sample;

[0007] S20, performing local model operation processing on the training sample;

[0008] S30, constructing a composite neural network model;

[0009] S40, in the real-time video monitoring process, if the output of the composite neural network is Y=0, it indicates that the image monitored at this moment is a normal action image, and the alarm system does not work; if the output of the composite neural network is Y=1, it indicates that the image monitored at this moment is an abnormal action image, representing the occurrence of abnormal action, and the alarm system issues an alarm.

[0010] Further, the step S10 comprises:

[0011] Collect historical monitoring image data as an original image, divide the original sample image into sub-images with a size of g*b, then take the corresponding pixels of each sub-image at the corresponding position of each sub-image, and calculate the average value of the pixels of all position sub-images corresponding to the sample image: x g*b represents the pixel value corresponding to each sub-image, represents the average value of the g*b sub-image pixels of the sample image.

[0012] The original image is resampled in the form of pixel All original image samples are expanded by using the above sample expansion method to generate total experimental samples, and the total experimental sample set is divided into training samples and test samples.

[0013] Further, the training sample local model operation processing method is:

[0014] I * (i, j) represents the pixel of the i-th row and j-th column of the sample image, I * (i, j) represents the pixel of the i-th row and j-th column of the image obtained after the local model operation of I

[0015]

[0016]

[0017]

[0018] Finally, the

[0019] Wherein U, V represent the window size of the local model operation, u = [-U, U], v = [-V, V], mu(i, j) represents the mean of the i-th row and j-th column of the sample image, sigma(i, j) represents the variance of the i-th row and j-th column of the sample image, and a is a constant.

[0020] The present application has at least the following beneficial effects:

[0021] 1. The sample increasing method used in the present application can expand one sample into multiple samples, thereby quickly increasing the number of samples and balancing the number of various images. Moreover, the calculation is simple and rapid, which improves the recognition accuracy and generalization of images, thereby reducing the occurrence rate of false alarms.

[0022] 2. The local model operation processing of the sample used in the present application greatly reduces the data amount in the operation process, avoids the influence of errors generated in the large data environment on data operation, and greatly improves the data operation speed, effectively solving the influence of the picture size problem on the subsequent experiment.

[0023] 3. The composite neural network of the present application adds a data compression layer after the convolution layer to perform data compression processing on the feature map formed after the convolution layer, which can solve the problem of sudden change of intermediate layer data distribution in the training process of the neural network, better prevent the disappearance or explosion of gradient, accelerate convergence, and improve precision. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The abnormal action real-time monitoring and identification step diagram of the present application;

[0025] Figure 2 The composite neural network structure diagram of the present application. DETAILED DESCRIPTION

[0026] In order to more clearly illustrate the present application, the following will be combined with the drawings of the specification and specific embodiments to make detailed description.

[0027] Reference Figure 1 The present application proposes an abnormal action real-time monitoring and identification method, comprising the following steps:

[0028] S10 expands sample data and performs gray scale processing on the sample;

[0029] S101 collects historical monitoring image data as an original image, divides the original sample image into sub-images with a size of g*b, then takes the corresponding pixels of each sub-image at the corresponding position of each sub-image, and calculates the average value of the sub-image pixels of all positions corresponding to the sample image: x g*b represents the pixel value corresponding to each sub-image, represents the average value of the g*b sub-image pixels of the sample image.

[0030] The original image is resampled in the form of pixel combination. The above sample expansion method is used to expand all original image samples to generate total experimental samples, and the total experimental sample set is divided into training samples and test samples.

[0031] The sample increasing method used in the present application can expand one sample into multiple samples, thereby quickly increasing the number of samples and balancing the number of various images. Moreover, the calculation is simple and rapid, which improves the recognition accuracy and generalization of the image, thereby reducing the occurrence rate of false alarms.

[0032] S102 pre-processes the training sample image data set, converts the colored image into a gray image, and the gray scale processing technology is prior art, which will not be elaborated here.

[0033] S20 performs local canonical operation processing on the training sample;

[0034] I * (i, j) represents the pixel of the i-th row and j-th column of the sample image, represents the pixel of the i-th row and j-th column of the image obtained after the local canonical operation of I ( i, j).

[0035] The following formula is used for calculation.

[0036]

[0037]

[0038]

[0039] Finally, the following formula is used for calculation.

[0040] Wherein U, V represent the window size of local modal operation, u = [-U, U], v = [-V, V], μ(i, j) represents the mean value of the i-th row and j-th column pixel of the sample image, σ(i, j) represents the variance of the i-th row and j-th column pixel of the sample image, and a is a constant.

[0041] The local modal operation processing used in the application greatly reduces the data amount in the operation process, avoids the influence of errors generated in a large data environment on data operation, and greatly improves the data operation speed, effectively solving the influence of the picture size problem on subsequent experiments.

[0042] The image after the local modal operation is taken as the training sample of the composite neural network model constructed by the application.

[0043] S30 refers to Figure 2 The composite neural network model is constructed, and the input of the composite neural network constructed by the application is the pixel of the image sample The output is the classification of two kinds of images of normal action and abnormal action, the normal action Y = 0, and the abnormal action Y = 1. The composite neural network model includes an input layer, a convolution layer, a data compression layer, a pooling layer, a full connection layer and an output layer.

[0044] S301 input layer

[0045] The input of the input layer is The output of the input layer is equal to the input, that is X(i, j) represents the pixel value of the i-th row and j-th column of the output image of the input layer.

[0046] S302 the number of neurons of the convolution layer is S, and any neuron is represented by s.

[0047] The convolution layer has S convolution kernels, and any convolution kernel is represented by s, s = {1, 2, 3,..., S}, K s *K s represents the size of the convolution kernel, l s is the step length of the convolution kernel.

[0048] The size of the feature map after convolution is This refers to the number of rings added around the original image. H s-1 H is the height of the image before convolution. s B is the height of the convolved image; s-1 B is the width of the image before convolution. s It is the width of the image after convolution.

[0049] The pixels in the i-th row and j-th column of the s-th feature map of the convolutional layer output image It is calculated using the following formula: f is the activation function, and ρ represents the extended constant of the function. Then... Represents the i+il-th feature map of the s-th feature map of the input image. s line j+jl s Column pixel values, b s denoted as the learning weights and biases of the s-th feature map, respectively.

[0050] The convolutional layer design of this invention is lightweight with fewer parameters, thus improving computational speed. The activation function used in the convolutional layer has stronger non-linear expressive power compared to those used in previous convolutional neural networks, resulting in better performance for mapping and extracting feature maps.

[0051] The number of neurons in the S303 data compression layer is Q, and any neuron is represented by q.

[0052] use Let represent the mean value of the pixels in row i and column j of the s feature maps output by the convolutional layer. This represents the variance of the pixels in row i and column j of the s feature maps output by the convolutional layer.

[0053] The mean and variance are obtained using the following formulas:

[0054]

[0055] Will The formula for data compression is:

[0056]

[0057] The q-th feature map in row i and column j represents the pixel in column j of the output of the data compression layer, where q = {1, 2, 3, ..., Q}, γ q and β q ...

[0058] The composite neural network of the present invention adds a data compression layer after the convolutional layer to compress the feature map formed after the convolutional layer. This can solve the problem of sudden changes in the data distribution of intermediate layers during the training process of the neural network, better prevent gradient vanishing or exploding, speed up convergence, and improve accuracy.

[0059] The number of neurons in the S304 pooling layer is C, and any neuron is represented by c.

[0060] The feature size of the image after pooling is: H s It is the height of the image before pooling. B is the height of the image after pooling; s It is the width of the image before pooling. It is the width of the image after pooling, l s The step size.

[0061] The pixels in the i-th row and j-th column of the c-th feature map of the pooling layer output image The value is obtained by calculating using the following formula:

[0062]

[0063]

[0064]

[0065] r c Let be the learning weights and biases of the c-th feature map output by the pooling layer, where c = {1, 2, 3, ..., C}.

[0066] The present invention uses pooling operations to prevent overfitting of convolutional neural networks, thereby improving the accuracy and robustness of the composite neural network model constructed by the present invention. This improves the accuracy of real-time monitoring of abnormal actions in real-time video surveillance, thus reducing the occurrence rate of false alarms.

[0067] The number of neurons in the fully connected layer S305 is Z.

[0068] use The output vector of the fully connected layer is calculated using the following formula.

[0069]

[0070] in For the input vector, Let be the transpose of the weights of the fully connected layer. This is the bias matrix of the fully connected layer.

[0071] The output layer has two neurons, Y=1 representing abnormal action and Y=0 representing normal action.

[0072] Prediction function

[0073] The composite neural network model constructed by the application is actually a binary classification problem for real-time monitoring and identification of abnormal action.

[0074] The F(R) value ranges from 0 to 1, and a threshold θ is set, when F(R) ≥ θ, it indicates that the input belongs to abnormal action Y=1, otherwise it is normal action Y=0, Y represents the output of the prediction classification obtained by processing the composite neural network.

[0075] The composite neural network model described in the application uses Adam optimizer to optimize parameters, which is a prior art and will not be described in detail here.

[0076] S307 processes the test sample as in steps S103 and S20, tests the composite neural network, and optimizes the threshold.

[0077] Y represents the actual classification of the input sample, and when the test sample is actually an abnormal action sample When the test sample is actually a normal action sample If If , it indicates that the composite neural network correctly identifies the input sample. Then the correct identification rate of the composite neural network constructed by the application for abnormal action is obtained

[0078] J accuracy = the number of correct classifications of test samples / the total number of test samples.

[0079] According to the need, set the threshold τ of the correct identification rate, when J accuracy ≥ τ, the training of the composite neural network described in the application is completed.

[0080] S40, in the process of real-time video monitoring, if the output of the composite neural network is Y=0, it indicates that the image monitored at this moment is a normal action image, and the alarm system does not work; if the output of the composite neural network is Y=1, it indicates that the image monitored at this moment is an abnormal action image, representing the occurrence of abnormal action, and the alarm system sends an alarm.

[0081] In summary, the abnormal action real-time monitoring and identification method described in the application is realized.

[0082] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 one or more flow or blocks.

[0083] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks. Figure 1 one or more flow or blocks.

[0084] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those skilled in the art once they learn of the basic inventive concepts. Therefore, the appended claims are intended to cover all such modifications and variations as fall within the true spirit and scope of the application.

[0085] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for real-time monitoring and identification of abnormal motion, characterized in that, The method comprises the following steps: S10, expanding sample data and performing gray scale processing on the sample; S20, performing local canonical operation processing on the training sample; the local canonical operation processing method is specifically as follows: with denotes the pixel of the i-th row and j-th column of the sample image, denotes the pixel of the i-th row and j-th column of the image obtained after the local normalization operation; ， ， ， Finally, we obtain , where U, V represent the window size of the local modulo operation, , , represents the mean value of the i-th row j-th column pixel of the sample image, represents the variance of the i-th row j-th column pixel of the sample image, is a constant; S30, constructing a composite neural network model, wherein the composite neural network model comprises an input layer, a convolution layer, a data compression layer, a pooling layer, a full connection layer and an output layer; The convolution layer has S convolution kernels, and each convolution kernel is denoted by s, s = {1, 2, 3,..., S}, denotes the size of the convolution kernel, is the step length of the convolution kernel; The feature map size after convolution is ; wherein, is the number of padding circles around the original image; is the height of the image before convolution; is the height of the image after convolution; is the width of the image before convolution; is the width of the image after convolution; the s-th feature map of the output image of the convolutional layer the pixel in the j-th row and the i-th column is calculated by the following equation: ； wherein, is an activation function, and ; denotes an extended constant of the function; then ; denotes the pixel value of the s-th feature map of the input image at the i-th row and j-th column; row column; , are the learning weight and bias of the s-th feature map, respectively. The number of neurons of the data compression layer is Q, and any neuron is represented by q; with denotes the mean of the i row j column pixel of the s feature maps of the convolutional layer output, denotes the variance of the i row j column pixel of the s feature maps of the convolutional layer output; The mean value and the variance are obtained through the following formulas respectively: ， The data compression calculation formula is: The data compression calculation formula is: ， represents the i-th row j-th column pixel of the q-th feature map output by the data compression layer, q = {1, 2, 3,..., Q}, and is a pair of learning parameters for the q-th feature map; S40, in the real-time video monitoring process, if the output of the composite neural network is Y=0, it indicates that the image monitored at this moment is a normal action image, and the alarm system does not work; if the output of the composite neural network is Y=1, it indicates that the image monitored at this moment is an abnormal action image, representing the occurrence of an abnormal action, and the alarm system issues an alarm.

2. The method of claim 1, wherein the method further comprises: The step S10 comprises: Collect historical monitoring image data as original image, divide the original sample image into sub-images with size of g*b, then take the corresponding pixels of each sub-image at the corresponding position of each sub-image, calculate the average value of all position sub-image pixels corresponding to the sample image: ; represents the pixel value corresponding to each sub-image, represents the average value of g*b sub-image pixels of the sample image. The original image is resampled in the form of pixel combination; all original image samples are expanded using the above sample expansion method to generate total experimental samples, and the total experimental sample set is divided into training samples and test samples.

Images