Abnormal behavior recognition monitoring method based on incremental spatiotemporal learning

Abstract

To provide an abnormal behavior recognition monitoring method based on incremental spatiotemporal learning.SOLUTION: The method includes establishing a spatiotemporal model, collecting a first video monitor screen in a first preset time zone of the monitor, inputting it into the spatiotemporal model, training the spatiotemporal model, identifying a second video monitor screen having abnormal behavior in a second preset time zone of the monitor by the trained spatiotemporal model, and transmitting it to a human for verification. When the abnormal behavior passes a manual verification, the abnormal behavior is determined to be normal, the abnormal behavior in the second video monitoring screen is constructed as a second normal behavior by the fuzzy aggregation method, the second normal behavior is input to the spatiotemporal model, the spatiotemporal model is retrained, and the above steps are repeated.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present application relates to the technical field of abnormal behavior monitoring, in particular to a method for detecting abnormal behavior based on incremental spatio-temporal learning. This invention relates to a method for monitoring and recognizing normal behavior. [Background technology]

[0002] Human behavior recognition algorithms are used to evaluate the movement techniques of athletes in sports, Virtual character motion control in medicine, evaluation of patient movement ability in security It is widely used in many fields, such as human behavior recognition. Human behavior recognition algorithms use sensor data and machine learning techniques to recognize human movements and Automatically analyze and identify behavior. Compared with traditional surveillance videos, human behavior recognition algorithms Enables real-time, accurate unmanned monitoring. Real-time monitoring and analysis can detect potential This allows the system to identify and respond to important safety risks in a timely manner, ensuring the safety and stability of substations. Sexuality improves. Current recent developments in artificial intelligence for anomaly detection in video surveillance are part of the challenge It only addresses the nature of anomalous behavior over time and does not address the real-time There are limited developments in anomaly detection and localization for real-world video surveillance. Summary of the Invention

[0003] This application relates to an artificial intelligence (AI) system used for anomaly detection in video surveillance that detects abnormal behavior over time. Ignoring the nature of motion, develop anomaly detection and localization for real-time video surveillance To address the issue of limitations in the above, we propose an anomaly detection method based on incremental spatio-temporal learning. An activity recognition monitoring method is provided, the method comprising: Establishing a space-time model; Acquiring a first video surveillance screen of a monitor during a first preset time period; The behavior in the first video surveillance screen is defined as a first normal behavior, and the first normal behavior is defined as the first normal behavior in the time-space model. training said spatio-temporal model; The trained spatio-temporal model detects abnormal behavior during a second preset time period of the monitor. identifying a second video surveillance screen; transmitting the second video surveillance image for manual verification; If the abnormal behavior in the second video surveillance image passes manual verification, the abnormal behavior is determined to be positive. and a fuzzy aggregation method is used to identify the abnormal behavior in the second video surveillance image. Constructing it as normal behavior The second normal behavior is input to the spatio-temporal model, and the spatio-temporal model is retrained. The abnormal behavior is repeatedly performing the steps of identifying a second video monitoring screen having a If the abnormal behavior does not pass manual verification, the abnormal behavior is determined to be abnormal and recorded. This includes: In a possible embodiment, the first preset time and the second preset time are consecutive. death, The first preset time period starts at a first time point and ends at a second time point, and the second preset time period The interval begins at time point 2 and ends at time point 3. In a possible embodiment, the spatio-temporal model includes an input data layer, a convolutional layer, The input data layer may include the first video surveillance screen and / or the second video surveillance screen. used to pre-process and enhance the learning ability of the spatio-temporal model; The convolution layer separates the first video monitoring screen and / or the second video monitoring screen. It is used to analyze and learn from data. In a possible embodiment, the input data layer is a first video surveillance screen and / or a front The step of pre-processing the second video surveillance image includes: A sliding window of length T is used to image the first video surveillance screen and / or the second video surveillance screen. Extracting video surveillance screens, The extracted first video surveillance image and / or the extracted second video surveillance image are displayed in succession as frames. The successive frames are converted to grayscale downscaling and 224×22 4 and normalize the pixel values ​​by scaling them from 0 to 1. Chemical processing, stacking said successive frames of length T to form an input time rectangle. In a possible embodiment, the spatio-temporal model further comprises a ConvLSTM layer; The ConvLSTM layer is used to capture spatio-temporal features from the successive frames; The model of the ConvLSTM layer is shown as follows: JPEG2025035989000091.jpg9154JPEG2025035989000092.jpg9159JPEG2025035989000093.jpg6118JPEG2025035989000094.jpg9159JPEG2025035989000095.jpg861In the formula, " JPEG2025035989000096.jpg33" indicates a convolution operation, and " "JPEG2025035989000097.jpg34" indicates the Hadamard product operation, JPEG2025035989000098.jpg78 indicates the input, JPEG2025035989000099.jpg937 indicates the cell state, JPEG2025035989000100.jpg938 shows the hidden state, JPEG2025035989000101.jpg818Japanese JPEG2025035989000102.jpg67 is a 3D tensor, " JPEG2025035989000103.jpg45" shows the sigmoid function, JPEG2025035989000104.jpg713 and JPEG2025035989000105.jpg713 is the 2D convolution kernel in ConvLSTM. In a possible embodiment, the spatio-temporal model is adapted to determine the normal and abnormal behaviors by an abnormality threshold. Distinguishing between anomalous behaviors, the anomaly threshold being manually selected; The lower the anomaly threshold, the greater the sensitivity to detect anomalous behavior during monitoring of the spatio-temporal model. The degree is higher, and the number of detections of the second video monitoring screen with abnormal behavior is higher; The higher the anomaly threshold, the greater the sensitivity to detect anomalous behavior while monitoring the spatio-temporal model. Therefore, the degree of abnormal behavior is reduced, and the number of detections of the second video monitoring screen having abnormal behavior is reduced. In a possible embodiment, the manual verification comprises: determining whether the abnormal behavior is acceptable; The reconstruction error is expressed as a fraction of each of the input time rectangles of an anomaly localization, is the localization of a specific region in a video frame where an anomaly occurs, and the formula for calculating the reconstruction error is is shown in equations (6) and (7): JPEG2025035989000106.jpg1394JPEG2025035989000107.jpg17107where, JPEG2025035989000108.jpg5123 is the time window, JPEG2025035989000109.jpg854 is the height of the video frame. In a possible embodiment, the trained spatio-temporal model is used to generate a second preset for the monitor. The step of identifying a second video surveillance image having abnormal behavior during the time period includes: The spatio-temporal model detects that the reconstruction error of the input time rectangle is greater than the anomaly threshold. If so, classify the input time rectangle as anomalous and switch the monitoring screen to the second video monitoring screen. This includes identifying As can be seen from the above, the present application provides an anomaly detection method based on incremental spatio-temporal learning. The behavior recognition monitoring method includes establishing a spatio-temporal model, Collecting a first video surveillance image; determining a behavior in the first video surveillance image as a first normal behavior; inputting the first normal behavior into the spatio-temporal model and training the spatio-temporal model; The spatiotemporal model of the training has abnormal behavior during a second preset time period of the monitor. identifying a second video surveillance screen; and transmitting the second video surveillance screen for manual verification. If the abnormal behavior in the second video surveillance screen passes manual verification, the abnormal behavior is The abnormal behavior in the second video surveillance screen is determined to be normal by a fuzzy aggregation method. constructing a second normal behavior as a second normal behavior; inputting the second normal behavior into the spatiotemporal model; Repeating the steps of retraining the spatial model and identifying a second video surveillance scene having abnormal activity. and executing the abnormal behavior again if the abnormal behavior does not pass manual verification. The present application includes detecting normal behavior in a time series and integrating the normal behavior into a spatiotemporal model. By inputting the time series into the database, the spatiotemporal model is continuously trained and used to detect abnormal behavior. This makes it possible to improve the accuracy of detecting abnormal behavior. [Brief description of the drawings]

[0004] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present invention and are not to be construed as limiting the present invention. The embodiments are illustrated and, together with the specification, are used to explain the principles of the embodiments of the present invention. Apparently, the accompanying drawings described below are merely some embodiments of the present invention, and those skilled in the art can easily understand the present invention. If so, obtain other drawings based on these attached drawings without creative labor. can be done. [Figure 1] 1 is a schematic flowchart of an abnormal behavior recognition monitoring method based on incremental spatio-temporal learning provided by an embodiment of the present application; [Diagram 2] 1 is a schematic flowchart of an input data layer provided by an embodiment of the present application for preprocessing the first video monitoring screen and / or the second video monitoring screen; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] Exemplary embodiments are described more fully below with reference to the accompanying drawings, in which: The present invention may be embodied in various forms and is not limited to the examples given herein, but rather is provided to illustrate the preferred embodiments. The present invention will be more comprehensive and complete in the present invention, and those skilled in the art will be able to easily understand the present invention. The described features, structures, or characteristics may be used interchangeably. The present invention may be combined in any suitable manner into one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the present invention. Human activity recognition algorithms use sensor data and machine learning techniques to recognize human movements and It can automatically analyze and recognize human behavior. Compared with traditional surveillance videos, Cognitive algorithms enable real-time, accurate, unattended monitoring of substation operation. Monitors the operational behavior of maintenance personnel, detects potential operational errors and abnormalities, and issues warnings in a timely manner. Furthermore, the human behavior recognition algorithm can detect unauthorized Detect abnormal behavior within the substation, such as personnel entering the substation area, vandalized equipment, or malicious operations. Through real-time monitoring and analysis, potential security risks can be identified. Improve the safety and stability of substations by identifying risks and taking timely action. Recent artificial intelligence developments for anomaly detection in video surveillance are part of the challenge. has not addressed the issue and has largely ignored the nature of abnormal behavior over time. In real-world video surveillance environments, active learning can achieve anomaly detection in dynamically changing environments. The training spatio-temporal model is based on the first acceptable They are trained to identify normal behaviors. However, they may encounter unexpected new normal behaviors or behaviors that may be considered abnormal. In a dynamic environment, where existing behaviors are changing into normal behaviors, the detection system It is important that our ability to detect these new scenarios evolves with the times. Using the G-aggregation method, we include unknown / new normal behaviors specific to the corresponding monitoring context. Continuously train the spatiotemporal model. As shown in FIG. 1, the present application provides an anomaly recognition monitoring system based on incremental spatio-temporal learning. The method includes the following steps. S100: Establish a space-time model. The spatio-temporal model is the ISTL model, which is composed of a spatio-temporal autoencoder and a video input. Learn appearance and motion representations from the forces. Spatiotemporal autoencoders are unsupervised learning algorithms. It is a method to minimize the reconstruction error using backpropagation to make the target value equal to the input value. S200: Collect a first video monitoring screen of a monitor during a first preset time period. The first video surveillance image comprises video frames indicative of normal behavior within a given camera viewing angle. Training Video Stream The training video stream consists of a sequence of frames of height h and width w. R, JPEG2025035989000111.jpg934, JPEG2025035989000112.jpg66 shows all the video frames of the camera view in the real world. S300: The behavior in the first video surveillance screen is regarded as a first normal behavior, and the first normal behavior is modeled as a spatiotemporal model. The input is then used to train a spatiotemporal model. After the first normal behavior is input into the time-space model, the time-space model undergoes training and some In an embodiment, the spatio-temporal model may be trained and then tested. During testing, a test video stream is M We use JPEG2025035989000113.jpg914, where JPEG2025035989000114.jpg931, JPEG2025035989000115.jpg914 contains video frames of normal and abnormal behavior. The purpose is to make the spatiotemporal model After learning the normal behavior representation from JPEG2025035989000116.jpg917 The goal is to verify it with JPEG2025035989000117.jpg914, distinguish abnormal behavior, and determine that training of the spatio-temporal model is complete. S400: Detect abnormal behavior during the second preset time period of the monitor using the trained spatiotemporal model. Identify a second video surveillance screen to Based on the training results, the spatio-temporal model detects behaviors that are different from the first normal behavior during the second preset time period. and determine whether the behavior is abnormal. S500: Send the second video surveillance screen for manual verification. The spatio-temporal model is trained based only on the first video surveillance image, so the recognized abnormal behavior However, in some cases, the recognition may not be accurate, and in such cases, a manual re-verification is required to verify the accuracy of the recognized abnormal behavior. It is necessary to. S600: If the abnormal behavior in the second video surveillance screen passes manual verification, the abnormal behavior is normal. Then, the abnormal behavior in the second video surveillance image is determined as the second normal behavior by the fuzzy aggregation method. and build it. Specifically, the ISTL model in this application is first trained on previously recognized normal behaviors in a monitored environment. The purpose of the manual validation and feedback is to improve the spatio-temporal model The goal is to proactively provide dynamically evolving normal behaviors for the detected anomalies. If the behavior is a false positive, the video frame in the second video surveillance screen having the abnormal behavior is The frame tag is manually judged as “normal” to obtain a second normal behavior, which is then used for the ongoing learning phase. do. After manual feedback, the video frames that were judged to be normal were used as the continuation of the ISTL model. It is used for periodic training to update its knowledge of normal concepts. The fuzzy aggregation of video frames enhances the continuous learning of the ISTL model and reduces the learning iterations. During the detection phase, all evaluated video frames are Based on fuzzy scale Labeled with JPEG2025035989000118.jpg79, The images are then grouped into a finite set (n) based on the JPEG2025035989000119.jpg79. Then, in the continuous learning phase, The fuzzy aggregation algorithm is JPEG2025035989000120.jpg79 Highest JPEG2025035989000121.jpg Select 75 video frame rectangles to train the ISTL model. Parameters JPEG2025035989000122.jpg75 and JPEG2025035989000123.jpg55 is defined based on the duration of the video surveillance stream used for continuous learning during initialization. The scenario selection for continuous training is defined as (8), where: JPEG2025035989000124.jpg723, where: JPEG2025035989000125.jpg1158 and JPEG2025035989000126.jpg76 are the indices of the selected time rectangles, which are included in the continuous training dataset. R, JPEG2025035989000127.jpg1952 The dataset of successive training iterations was subjected to manual verification of false positive detection and fuzzy aggregation. It is constructed from time rectangles selected from normal behavior. The detection model is continuously trained to update Maintaining the stability of previously known normal behaviors while updating the ability to capture normal behaviors This fuzzy aggregation method is useful for IoT stream mining, text mining, To maintain stability and plasticity in continuous learning for video stream mining and video processing. They have been successful in doing so. S700: Input the second normal behavior into the time-space model, retrain the time-space model, and detect the abnormal behavior. The step of identifying a second video monitoring screen having the same is repeated. After scenario selection, the ISTL model continues based on the representation selected from the input video data. The expected and acceptable behaviors are then updated from the monitored domain. ,The updated ISTL model is reused for anomaly detection. S800: If the abnormal behavior does not pass manual verification, the abnormal behavior is determined to be abnormal and recorded. can be. In addition, if the manual verification determines that the behavior is abnormal, it is necessary to input the behavior into the training of the spatio-temporal model. do not have. In some embodiments of the present application, the first preset time period is a first time point JPEG2025035989000128.jpg85 to the second point JPEG2025035989000129.jpg86, and the second preset time period is JPEG2025035989000130.jpg86 JPEG2025035989000131.jpg86 and are continuous within the first preset time. All input monitoring screens of the time-space model It is continuous and avoids duplication and omission. In some embodiments of the present application, the spatio-temporal model includes an input data layer and a convolutional layer. The input data layer preprocesses the first video surveillance image and / or the second video surveillance image, and It is used to enhance the learning ability of the inter-model. As shown in FIG. 2, the specific pre-processing steps include: S001: A first video monitoring screen and / or a second video monitoring screen are displayed by a sliding window of length T. Extract the video surveillance screen. S002: Continuously frame the extracted first video surveillance screen and / or the extracted second video surveillance screen. The frames are converted to grayscale downscaling and 224x224 The pixel values ​​are then normalized by scaling from 0 to 1. S003: Stack consecutive frames of length T to form an input time rectangle. Note that the time window It should be appreciated that longer actions can be included by increasing the length of the DoT. The convolutional layer analyzes and learns from the first video surveillance image and / or the second video surveillance image. It is used to The convolutional layer (CNN) is inspired by biological processes similar to the organization of animal visual cortex. The connectivity of neurons in the convolutional layer is such that individual cortical neurons can The visual system is similar to that of animals, responding only to stimuli in a specific area (i.e., receptive field). In video analysis, convolutional layers are designed in a way that the values ​​are learned during training. Preserves spatial relationships in the input frames by learning feature representations using a filter. It is possible.

[0006] In some embodiments of the present application, the spatio-temporal model further includes a ConvLSTM layer, The nvLSTM layer is used to capture spatio-temporal features from successive frames. By using the RNN internal memory to process the input sequence, Designed to capture dynamic temporal behavior, LSTM units are a common feature of RNNs. The LSTM unit is a modification of the traditional building block. It consists of a force gate, a forget gate, and a cell. An input gate is a gate that is used when an input value enters a cell. Define the range. The forget gate controls the range for which the value of the previous time step is retained in the cell. ,The output gate controls the extent to which the current input value is used in the cell's activation calculation. It remembers values ​​for any time interval. LSTM is primarily used to model long-term temporal correlations. Since spatial information is not encoded in the state transitions, it has a disadvantage in dealing with spatial data. However, there are methods to learn temporal patterns while preserving the spatial structure of surveillance video streams. It is extremely important to realize this, especially in anomaly detection. An extension of M, i.e. both input-to-state and state-to-state transitions are convolutional structures. We use ConvLSTM with the following structure. The ConvLSTM layer has input, hidden state, gate, We overcome this drawback by designing the cell output as a 3D tensor. The matrix operations in the ConvL algorithm have been replaced with convolution operations. The STM layer can capture spatiotemporal features from the input frame sequence. The model with the ConvLSTM layer looks like this: JPEG2025035989000132.jpg9154JPEG2025035989000133.jpg9159JPEG2025035989000134.jpg6118JPEG2025035989000135.jpg9159JPEG2025035989000136.jpg861In the formula, " JPEG2025035989000137.jpg33" indicates a convolution operation, and " "JPEG2025035989000138.jpg34" indicates the Hadamard product operation, JPEG2025035989000139.jpg78 indicates the input, JPEG2025035989000140.jpg937 shows the cell state, JPEG2025035989000141.jpg938 shows the hidden state, JPEG2025035989000142.jpg818 and JPEG2025035989000143.jpg67 is a 3D tensor, " JPEG2025035989000144.jpg45" shows the sigmoid function, JPEG2025035989000145.jpg713 and JPEG2025035989000146.jpg713 is the 2D convolution kernel in ConvLSTM. According to the above embodiment, the space-time autoencoder configuration adopted in this application is shown in Table 1. Table 1 Space-time autoencoder configuration In some embodiments of the present application, the spatio-temporal model distinguishes between normal and abnormal behaviors using an anomaly threshold. The anomaly threshold is manually selected, and the lower the anomaly threshold, the greater the anomaly during monitoring of the spatio-temporal model. The detection sensitivity for normal behavior is increased, and the number of detections of the second video surveillance screen with abnormal behavior is The higher the anomaly threshold, the greater the sensitivity of the spatio-temporal model to detect abnormal behavior during monitoring. , and the number of detections of the second video monitoring screen having abnormal behavior is reduced. In this application, we define a reconstruction error threshold to distinguish between normal and abnormal behaviors, and an abnormality threshold The image was named JPEG2025035989000148.jpg76. In a real video surveillance application, the Sensitivity is Select the value JPEG2025035989000149.jpg76. JPEG2025035989000150.jpg76When the value is lower, the sensitivity of the monitored area is higher and the number of alarms is higher. JPEG2025035989000151.jpg76When the value is high, the sensitivity is low and there is a possibility that sensitive abnormalities in the monitored area will be missed. Furthermore, the present invention provides a time threshold JPEG2025035989000152.jpg76 was introduced. JPEG2025035989000153.jpg76 is defined as the number of video frames higher than that which would cause an event to be recognized as anomalous. JPEG2025035989000154.jpg76 is used to reduce false positive anomaly alarms caused by sudden changes in surveillance video streams. Here, sudden changes in the surveillance video stream are caused by occlusion, motion blur, and This can be caused by bright light and high brightness lighting conditions. In some embodiments of the present application, the manual verification may include detecting the presence of a second video monitor image due to a reconstruction error. Judge whether the abnormal behavior is pass or not, and the reconstruction error is calculated for each input time for abnormal localization. Expressed as a fraction of a rectangle, anomaly localization is the specific area in a video frame where an anomaly occurs. The calculation formula for the reconstruction error is shown in Equation (6) and Equation (7): JPEG2025035989000155.jpg 1394 JPEG2025035989000156.jpg 17107 Where, JPEG2025035989000157.jpg5123 is the time window, JPEG2025035989000158.jpg854 is the height of the video frame. Anomaly localization involves identifying the specific area of ​​a video frame where an anomaly occurs. After detecting anomalies in the occlusion clip, we calculate the reconstruction error over non-overlapping spatiotemporal local rectangular windows. The reconstruction error of the local rectangle is calculated using Equation (7). In some embodiments of the present application, the trained spatio-temporal model is used to select a second preset for the monitor. The step of identifying a second video surveillance scene having abnormal behavior in a time period includes: If it is detected that the reconstruction error of the input time rectangle is greater than the abnormality threshold, the input time rectangle is reconstructed as an abnormal rectangle. and identifying a second video surveillance screen within the surveillance screen. The ISTL model is first trained using previously recognized normal behaviors in a surveillance environment. Used for anomaly detection. Video frame anomalies are detected, i.e. the reconstruction of the input time rectangle If the error is greater than the anomaly threshold, the input time rectangle is classified as anomalous. Send the frame for manual verification. As can be seen from the above, the present application relates to an abnormal behavior recognition system based on incremental spatio-temporal learning. A method for monitoring a person's identity is provided, the method comprising: establishing a spatiotemporal model; collecting a first video surveillance image during a certain period of time; The first normal behavior is input to the spatio-temporal model as a normal behavior, and the spatio-temporal model is trained. Things to do , lesson According to the spatiotemporal model of the training, an abnormality was detected during the second preset time period of the monitor. identifying a second video surveillance screen having the behavior; and subjecting the second video surveillance screen to manual inspection. and sending, if the abnormal behavior in the second video monitoring screen passes manual verification, The abnormal behavior is determined to be normal, and the abnormal behavior is detected by a fuzzy aggregation method. constructing the abnormal behavior as a second normal behavior, and inputting the second normal behavior into the spatio-temporal model; and retraining the spatio-temporal model to identify a second video surveillance scene having abnormal behavior. and repeatedly executing the steps of: The present application includes detecting normal behaviors by time series detection and recording the abnormal behavior. By inputting the above information into the spatio-temporal model, the spatio-temporal model is continuously trained to detect abnormal behavior. This can be utilized to detect abnormal behavior, thereby improving the accuracy of detecting abnormal behavior. In the examples of this application, the words "comprise", "include" and other variations mean a non-exclusive inclusion. A structure, article, or device of a series of elements includes not only those elements but also those explicitly listed. It may also cover other elements not included in the specification or inherent in those structures, articles or devices. Unless further limited, the term "includes" is intended to An element is intended to exclude the presence of another identical element in the structure, article or device in which it is contained. isn't it. Those skilled in the art will readily envision other embodiments of the present disclosure in light of the specification and embodiments. This application includes any variations, uses or adaptations of this disclosure. , uses or adaptations may be made in accordance with the general principles of this disclosure and may be made without departing from the spirit or scope of the present invention. The present specification and examples are intended to include those well known in the art or commonly used technical means. It is intended that the present disclosure be practiced as a method and apparatus for the preparation of such compositions, with the true scope and spirit of the present disclosure being indicated by the following claims. do.

Claims

1. Establishing a space-time model; acquiring a first video surveillance image of the monitor during a first preset time period; The behavior in the first video surveillance screen is defined as a first normal behavior, and the first normal behavior is defined as a first normal behavior in the time-space model. training said spatio-temporal model; The trained spatio-temporal model detects abnormal behavior during a second preset time period of the monitor. identifying a second video surveillance screen; transmitting said second video surveillance image for manual review; If the abnormal behavior in the second video surveillance image passes manual verification, the abnormal behavior is determined to be positive. and a fuzzy aggregation method is used to identify the abnormal behavior in the second video surveillance screen. Constructing it as normal behavior The second normal behavior is input to the spatio-temporal model, and the spatio-temporal model is retrained. repeatedly performing the steps of identifying a second video surveillance screen having a If the abnormal behavior does not pass manual verification, the abnormal behavior is determined to be abnormal and recorded. The method for detecting abnormal behavior based on incremental spatio-temporal learning includes: How to see.

2. the first preset time and the second preset time are consecutive, The first preset time period starts at a first time point and ends at a second time point, and the second preset time period 2. The ink of claim 1, wherein the interval begins at a second time point and ends at a third time point. An abnormal behavior recognition monitoring method based on real-time spatiotemporal learning.

3. The spatio-temporal model includes an input data layer and a convolution layer; The input data layer may include the first video surveillance screen and / or the second video surveillance screen. used to pre-process and enhance the learning ability of the spatio-temporal model; The convolution layer separates the first video monitoring screen and / or the second video monitoring screen.

2. The incremental learning method according to claim 1, wherein the incremental learning method is used to analyze and learn from the An abnormal behavior recognition monitoring method based on space-time learning.

4. The input data layer may include a first video surveillance screen and / or a second video surveillance screen. The processing step includes: A sliding window of length T is used to image the first video surveillance screen and / or the second video surveillance screen. Extracting video surveillance screens, The extracted first video surveillance image and / or the extracted second video surveillance image are displayed in succession as frames. The successive frames are converted to grayscale downscaling and 224×22 4 pixel value and normalize the pixel value by scaling it from 0 to 1. Chemical processing, and stacking the successive frames of length T to form an input time rectangle. The method for abnormal behavior recognition and monitoring based on incremental spatio-temporal learning according to claim 3 .

5. The spatio-temporal model further includes a ConvLSTM layer; The ConvLSTM layer is used to capture spatio-temporal features from the successive frames; The model of the ConvLSTM layer is expressed as follows: In the formula, " indicates the convolution operation, and " " denotes the Hadamard product operation, indicates the input, indicates the cell state, denotes the hidden state, and is a three-dimensional tensor, " indicates a sigmoid function, and is a two-dimensional convolution kernel in ConvLSTM. The abnormal behavior recognition monitoring method based on the incremental space-time learning described above.

6. The spatio-temporal model distinguishes between the normal behavior and the abnormal behavior by an abnormality threshold. The threshold is usually chosen manually. The lower the anomaly threshold, the greater the sensitivity to detect anomalous behavior during monitoring of the spatio-temporal model. The degree is higher, and the number of detections of the second video monitoring screen having abnormal behavior is higher; The higher the anomaly threshold, the greater the sensitivity to detect anomalous behavior while monitoring the spatio-temporal model. and the frequency of detection of the second video monitoring screen having abnormal behavior is reduced. The method for detecting abnormalities based on the incremental spatio-temporal learning according to any one of claims 1 to 5. Behavioral recognition monitoring method.

7. The manual verification determines whether the abnormal behavior in the second video surveillance image is acceptable due to a reconstruction error. Determine whether The reconstruction error is expressed as a fraction of each of the input time rectangles of an anomaly localization, is the localization of a specific region in a video frame where an anomaly occurs, and the formula for calculating the reconstruction error is is shown in equations (6) and (7): Where: is the time window, is the height of a video frame. An abnormal behavior recognition monitoring method based on spatial learning.

8. The trained spatio-temporal model detects abnormal behavior during a second preset time period of the monitor. The step of identifying a second video surveillance screen includes: The spatio-temporal model detects that the reconstruction error of the input time rectangle is greater than the anomaly threshold. If so, classify the input time rectangle as anomalous and switch the monitoring screen to the second video monitoring screen.

8. The incremental spatio-temporal learning method of claim 7, further comprising: An abnormal behavior recognition monitoring method based on.

Images