Real-time monitoring and identifying method for abnormal action

Abstract

The invention provides a real-time monitoring and identifying method for abnormal actions. The real-time monitoring and identifying method comprises the following steps: expanding sample data and carrying out graying processing on a sample; carrying out local model operation processing on the training sample; constructing a composite neural network model; in the real-time video monitoring process, if the output Y of the composite neural network is equal to 0, the monitored image at the moment is a normal action image, and the alarm system does not work; and if the output Y of the composite neural network is equal to 1, the image monitored at the moment is an abnormal action image and represents the occurrence of an abnormal action, and the alarm system gives an alarm. According to the invention, the real-time identification rate and accuracy of the monitoring system on abnormal actions are improved.

Description

Technical Field

[0001] The invention belongs to the field of intelligent monitoring technology, and in particular relates to a real-time monitoring and identification method for abnormal actions. Background Art

[0002] Due to the rapid economic development, people's pace of life is getting faster and faster, and the time to take care of and understand the family situation is getting less and less. The application of intelligent monitoring systems in the home has realized the needs of remote care of children and the elderly. The development of home monitoring network cameras has an important impact on the application of intelligent monitoring systems.

[0003] However, the existing monitoring system has a slow recognition speed and low recognition accuracy for abnormal actions, often giving false warnings, and not giving warnings or delayed warnings when abnormal actions occur. This results in poor user experience with intelligent monitoring and low usage. Therefore, improving the recognition rate and accuracy of monitoring systems is an urgent problem to be solved in the current intelligent monitoring field. Summary of the invention

[0004] The present invention proposes a method for real-time monitoring and identification of abnormal actions, the purpose of which is to improve the real-time identification rate and accuracy of the monitoring system for abnormal actions.

[0005] The method for real-time monitoring and identifying abnormal actions described in the present invention comprises the following steps:

[0006] S10 expands the sample data and performs grayscale processing on the sample;

[0007] S20 performs local modeling operation processing on the training samples;

[0008] S30 builds a composite neural network model;

[0009] S40 During real-time video monitoring, if the output of the composite neural network is Y=0, it means that the image monitored at this moment is a normal action image and the alarm system does not work; if the output of the composite neural network is Y=1, it means that the image monitored at this moment is an abnormal action image, indicating the occurrence of abnormal action, and the alarm system sounds an alarm.

[0010] Further, the step S10 includes:

[0011] Collect historical monitoring image data as the original image, divide the original sample image into sub-images of size g*b, then take the corresponding pixels of each sub-image at the corresponding position of each sub-image, and calculate the average value of the sub-image pixels at all positions corresponding to the sample image: x g*b Represents the pixel value corresponding to each sub-image, Represents the mean value of the pixels of g*b sub-images of the sample image;

[0012] The original image is pixel-by-pixel Resample in the form of combination; use the above sample expansion method to expand all original image samples to generate total experimental samples, and divide the total experimental sample set into training samples and test samples.

[0013] Furthermore, the local modeling operation processing method of the training sample is:

[0014] Use I * (i, j) represents the pixel in the i-th row and j-th column of the sample image. Indicates I * (i, j) the pixel in row i and column j of the image obtained after local modeling operation;

[0015]

[0016]

[0017]

[0018] Finally get

[0019] Where U and V represent the window sizes of the local modeling operation, u = [-U, U], v = [-V, V], μ(i, j) represents the mean of the pixels in the i-th row and j-th column of the sample image, σ(i, j) represents the variance of the pixels in the i-th row and j-th column of the sample image, and a is a constant.

[0020] The present invention has at least the following beneficial effects:

[0021] 1. The sample increase method used in the present invention can expand one sample into multiple samples, thereby quickly increasing the number of samples and balancing the number of various images. Moreover, the calculation is simple and fast, which improves the recognition accuracy and generalization of the image, thereby reducing the incidence of false alarms.

[0022] 2. The local modeling operation processing of samples used in the present invention greatly reduces the amount of data in the operation process, avoids the influence of errors generated by the big data environment on data operation, greatly improves the data operation speed, and effectively solves the influence of image size problems on subsequent experiments.

[0023] 3. The composite neural network of the present invention adds a data compression layer after the convolution layer to perform data compression processing on the feature map formed after the convolution layer, which can solve the problem of sudden changes in the data distribution of the intermediate layer during the training process of the neural network, better prevent the disappearance or explosion of the gradient, accelerate convergence, and improve accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A diagram of the steps for real-time monitoring and identification of abnormal actions described in the present invention;

[0025] Figure 2 A structural diagram of the composite neural network described in the present invention. DETAILED DESCRIPTION

[0026] In order to explain the present invention more clearly, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0027] refer to Figure 1 The present invention proposes a method for real-time monitoring and identification of abnormal actions, comprising the following steps:

[0028] S10 expands the sample data and performs grayscale processing on the sample;

[0029] S101 collects historical monitoring image data as the original image, divides the original sample image into sub-images of size g*b, then takes the pixels corresponding to each sub-image at the corresponding position of each sub-image, and calculates the average value of the pixels of the sub-images at all positions corresponding to the sample image: x g*b Represents the pixel value corresponding to each sub-image, Represents the mean value of the pixels of g*b sub-images of the sample image.

[0030] The original image is pixel-by-pixel The above sample expansion method is used to expand all original image samples, generate total experimental samples, and divide the total experimental sample set into training samples and test samples.

[0031] The sample increase method used in the present invention can expand one sample into multiple samples, thereby quickly increasing the number of samples and balancing the number of various images. In addition, the calculation is simple and fast, which improves the recognition accuracy and generalization of the image, thereby reducing the incidence of false alarms.

[0032] S102 pre-processes the training sample image data set to convert the color image into a gray image. The grayscale processing technology is an existing technology and will not be elaborated in detail here.

[0033] S20 performs local modeling operation processing on the training samples;

[0034] Use I * (i, j) represents the pixel in the i-th row and j-th column of the sample image. I* ( i, j) is the pixel in the i-th row and j-th column of the image obtained after the local modeling operation.

[0035] Calculated by the following formula.

[0036]

[0037]

[0038]

[0039] Finally get

[0040] Where U and V represent the window sizes of the local modeling operation, u = [-U, U], v = [-V, V], μ(i, j) represents the mean of the pixels in the i-th row and j-th column of the sample image, σ(i, j) represents the variance of the pixels in the i-th row and j-th column of the sample image, and a is a constant.

[0041] The local modeling operation processing of samples used in the present invention greatly reduces the amount of data in the operation process, avoids the influence of errors generated in the big data environment on data operation, greatly improves the data operation speed, and effectively solves the influence of image size problems on subsequent experiments.

[0042] The image after the local modeling operation is used as a training sample of the composite neural network model constructed by the present invention.

[0043] S30 Reference Figure 2 Construct a composite neural network model, the input of the composite neural network constructed by the present invention is the pixels of the image sample The output is two types of image classification: normal action and abnormal action. Normal action is Y = 0, and abnormal action is Y = 1. It includes input layer, convolution layer, data compression layer, pooling layer, fully connected layer, and output layer.

[0044] S301 Input Layer

[0045] The input layer has the following input: The output of the input layer is equal to the input, then X(i, j) represents the pixel value of the i-th row and j-th column of the input layer output image.

[0046] The number of neurons in the S302 convolutional layer is S, and any neuron is represented by s.

[0047] The convolution layer has S convolution kernels, and any convolution kernel is represented by s, s = {1, 2, 3, ..., S}, K s *K s represents the size of the convolution kernel, l s is the stride of the convolution kernel.

[0048] The size of the feature map after convolution is It refers to the number of circles filled around the original image. s-1 is the height of the image before convolution, H s is the height of the image after convolution; B s-1 is the width of the image before convolution, B s is the width of the image after convolution.

[0049] The pixel in the i-th row and j-th column of the s-th feature map of the convolutional layer output image Calculated by the following formula: f is the activation function, and ρ represents the expansion constant of the function. Then The i+ilth feature map of the sth feature map of the input image s rowj+jl s The pixel value of the column, b s are the learning weights and biases of the s-th feature map, respectively.

[0050] The design model of the convolution layer of the present invention is relatively light, and the model parameters are also less, which improves the calculation speed. The excitation function used in the convolution layer has stronger nonlinear expression ability than the excitation function used in the previous convolutional neural network, and is better for mapping and extracting feature maps.

[0051] The number of neurons in the S303 data compression layer is Q, and any neuron is represented by q.

[0052] use Represents the mean value of the pixels in row i and column j of the s feature maps output by the convolutional layer, Represents the variance of the pixels in row i and column j of the s feature maps output by the convolutional layer.

[0053] The mean and variance are obtained by the following formulas:

[0054]

[0055] Will The calculation formula for data compression is:

[0056]

[0057] represents the pixel in row i and column j of the qth feature map output by the data compression layer, q = {1, 2, 3, ..., Q}, γ q and β q is a pair of learning parameters for the qth feature map.

[0058] The composite neural network of the present invention adds a data compression layer after the convolution layer to perform data compression processing on the feature map formed after the convolution layer, which can solve the problem of sudden changes in the data distribution of the middle layer during the training process of the neural network, better prevent the disappearance or explosion of the gradient, accelerate convergence, and improve accuracy.

[0059] The number of neurons in the S304 pooling layer is C, and any neuron is represented by c.

[0060] The feature size of the image after pooling is: H s is the height of the image before pooling, is the height of the pooled image; B s is the width of the image before pooling, is the width of the pooled image, l s is the step length.

[0061] The pixel in the i-th row and j-th column of the c-th feature map of the pooling layer output image The value of is obtained by the following formula:

[0062]

[0063]

[0064]

[0065] r c are the learning weights and biases of the c-th feature map output by the pooling layer, c = {1, , 2, 3, …, C}.

[0066] The pooling operation of the present invention is conducive to preventing overfitting of the convolutional neural network, improving the accuracy and robustness of the composite neural network model constructed by the present invention, thereby improving the accuracy of real-time monitoring of abnormal actions in real-time video monitoring, thereby reducing the incidence of false alarms.

[0067] The number of neurons in the S305 fully connected layer is Z.

[0068] use Represents the output vector of the fully connected layer, which is calculated by the following formula

[0069]

[0070] in is the input vector, is the transposed matrix of the weights of the fully connected layer, is the bias matrix of the fully connected layer.

[0071] The output layer S306 has two neurons, Y=1 representing abnormal action and Y=0 representing normal action.

[0072] Prediction Function

[0073] The composite neural network model constructed by the present invention is actually a binary classification problem for real-time monitoring and identification of abnormal actions.

[0074] The range of F(R) value is within (0, 1), and the threshold θ is set. When F(R)≥θ, it means that the input belongs to abnormal action Y=1, otherwise it is normal action Y=0. Y represents the output of the predicted classification obtained by the composite neural network processing.

[0075] The composite neural network model described in the present invention uses the Adam optimizer to optimize parameters. This technology is prior art and will not be elaborated in detail here.

[0076] S307 processes the test sample as in steps S103 and S20, tests the composite neural network, and optimizes the threshold.

[0077] use Indicates the actual classification of the input sample, when the test sample is actually an abnormal action sample When the test sample is actually a normal action sample like It means that the composite neural network correctly identifies the input sample. Then the recognition accuracy of the composite neural network constructed by the present invention for abnormal actions is obtained.

[0078] J accuracy = Number of test samples correctly classified / total number of test samples.

[0079] Set the recognition accuracy threshold τ as needed. accuracy When ≥τ, the composite neural network training of the present invention is completed.

[0080] S40 During real-time video monitoring, if the output of the composite neural network is Y=0, it means that the image monitored at this moment is a normal action image and the alarm system does not work; if the output of the composite neural network is Y=1, it means that the image monitored at this moment is an abnormal action image, indicating the occurrence of abnormal action, and the alarm system sounds an alarm.

[0081] In summary, the method for real-time monitoring and identification of abnormal actions described in the present invention is realized.

[0082] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0084] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0085] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A real-time monitoring and identification method for abnormal actions, characterized in that: The following steps are involved: S10 expands the sample data and performs grayscale processing on the sample; S20 performs local modeling operation processing on the training samples; S30 builds a composite neural network model; S40 During real-time video monitoring, if the output of the composite neural network is Y=0, it means that the image monitored at this moment is a normal action image and the alarm system does not work; if the output of the composite neural network is Y=1, it means that the image monitored at this moment is an abnormal action image, indicating the occurrence of abnormal action, and the alarm system sounds an alarm.

2. The method for real-time monitoring and identification of abnormal actions according to claim 1, characterized in that: The step S10 comprises: Collect historical monitoring image data as the original image, divide the original sample image into sub-images of size g*b, then take the corresponding pixels of each sub-image at the corresponding position of each sub-image, and calculate the average value of the sub-image pixels at all positions corresponding to the sample image: x g*b Represents the pixel value corresponding to each sub-image, Represents the mean value of the pixels of g*b sub-images of the sample image; The original image is pixel-by-pixel Resample in the form of combination; use the above sample expansion method to expand all original image samples to generate total experimental samples, and divide the total experimental sample set into training samples and test samples.

3. The method for real-time monitoring and identification of abnormal actions according to claim 1, characterized in that: The local modeling operation processing method of the training sample is: Use I * (i, j) represents the pixel in the i-th row and j-th column of the sample image. Indicates I * (i, j) the pixel in row i and column j of the image obtained after local modeling operation; Finally get Where U and V represent the window sizes of the local modeling operation, u = [-U, U], v = [-V, V], μ(i, j) represents the mean of the pixels in the i-th row and j-th column of the sample image, σ(i, j) represents the variance of the pixels in the i-th row and j-th column of the sample image, and a is a constant.

Images