Embedded behavior anomaly detection system based on time sequence image analysis

Abstract

The invention relates to the technical field of computer vision, in particular to an embedded behavior anomaly detection system based on time sequence image analysis, which comprises an image acquisition module, a motion characterization module and a detection and early warning module, and is characterized in that the image acquisition module acquires video frames regularly and converts the video frames into an HSV color space for preprocessing; the motion characterization module comprises a behavior model acquisition sub-module, a motion trail characterization sub-module and a local feature extraction sub-module, a Frey inner frame is constructed by introducing a differential geometry theory, parameterized representation is performed on a motion trail, differential invariant features such as curvature and torsion are calculated, and anomaly judgment is performed in combination with a multi-scale anomaly measurement frame; the detection and early warning module analyzes the abnormal characteristics, generates a detection result and feeds back the detection result to update the model, and a self-adaptive model updating mechanism based on manifold learning is adopted, so that the system has view angle invariance, high detection sensitivity and self-adaptive learning capability, and can be deployed on low-power-consumption embedded equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to an embedded behavior anomaly detection system based on time series image analysis, which is applied to the fields of video monitoring, security monitoring, smart home, medical monitoring, etc. BACKGROUND

[0002] With the wide application of video monitoring systems, how to automatically detect abnormal behaviors from massive video data has become an important research topic. Traditional abnormal behavior detection methods are usually based on rule matching or simple motion feature analysis, which are difficult to adapt to complex and variable real environments, and often require a large amount of computing resources, which are not suitable for deployment on embedded devices.

[0003] The existing abnormal behavior detection technology mainly has the following problems: first, the traditional detection method based on position and speed features is sensitive to the change of camera position, and needs to retrain the model when the view angle changes; second, the existing method mainly relies on the trajectory representation in the absolute coordinate, which is difficult to capture the essential features of the behavior; third, the fixed model method lacks adaptive learning ability and is difficult to adapt to environmental changes; finally, the calculation of complex algorithms is large, which is not suitable for running on resource-limited embedded devices.

[0004] Therefore, there is an urgent need for an efficient, accurate, and adaptive behavior anomaly detection system that can be implemented on embedded devices. SUMMARY

[0005] The purpose of the present application is to provide an embedded behavior anomaly detection system based on time series image analysis, which accurately models the motion trajectory by introducing differential geometry theory, and realizes accurate analysis and anomaly detection of complex human behaviors.

[0006] The present application provides an embedded behavior anomaly detection system based on time series image analysis, which comprises:

[0007] An image acquisition module is used to acquire a sequence of monitoring video frames at regular intervals, pre-process each frame of image, convert the RGB color space to the HSV color space, crop the center region of the image, remove the background of the image, and obtain a color image containing a person;

[0008] A motion representation module is in communication connection with the image acquisition module, and is used to receive the color image containing a person transmitted by the image acquisition module, extract a motion trajectory, establish a differential geometry framework, generate differential invariant features, construct a behavior feature manifold, calculate a feature mapping distance, and judge a behavior anomaly; wherein the motion representation module comprises a behavior model acquisition submodule, a motion trajectory representation submodule, and a local feature extraction submodule.

[0009] The detection and early warning module is connected in communication with the motion characterization module, is configured to receive the abnormal features delivered by the motion characterization module, perform abnormal motion detection, generate an abnormal detection result, and feed back the abnormal detection result and scene information of a current video frame to a behavior model obtaining sub-module of the motion characterization module, so as to update a behavior baseline model.

[0010] Preferably, the behavior model obtaining sub-module is configured to receive the color image containing a person delivered by the image acquisition module, divide a data sequence into one sample every 9 frames, and initialize an empty sequence as a model behavior baseline; when there is a data sequence input, compare a first data value of the sequence with an average value in the empty sequence, and if the first data value is greater than a set threshold value such as 0.8, add a current data value to an end of the sequence, replace the average value with the current value, then shift the sequence forward by one bit, receive a new data value, compare the new data value with the average value in the sequence, and repeat the above operation; when all values in the sequence are less than 0.2, stop processing, take the sequence at this time as a motion behavior baseline, and deliver the motion behavior baseline to the motion trajectory characterization sub-module.

[0011] Preferably, the motion trajectory characterization sub-module is configured to receive the motion behavior baseline as a target trajectory vector, take a currently acquired video frame as an original image, analyze the original image by using a sparse optical flow algorithm, extract a motion feature, and take a difference between a current frame motion vector and the target trajectory vector as a difference value of the feature; repeat the extraction of the motion feature to obtain a difference distribution sequence, i.e., a motion trajectory; and the motion trajectory characterization sub-module constructs a differential geometry characterization of the trajectory based on a Fre'chet framework, includes calculation of a trajectory curvature feature and a torsion feature, and forms a differential invariant feature vector.

[0012] Preferably, the local feature extraction sub-module is configured to take each frame motion trajectory vector as one sample, perform convolution operation by using a convolutional neural network, finally obtain a local abnormal point, and according to a size of the feature difference value, mark a frame with a difference greater than an abnormal threshold value as abnormal; and the convolutional neural network performs time domain abnormal detection, and a network structure includes an input layer, a feature layer, a time domain sampling connection layer, a full connection layer, and an output layer.

[0013] Preferably, the detection and early warning module is configured to take an abnormal motion sample as input, perform abnormal motion detection by using a neural network, and a loss function of the neural network includes a number of training samples, an abnormal detection label, a network prediction value, and an abnormal detection accuracy rate index parameter; and feed back a detection result of the neural network and scene information of a current video frame to the behavior model obtaining sub-module, so as to update the behavior baseline model.

[0014] Preferably, the motion characterization module adopts a multi-scale abnormality measurement framework, including:

[0015] a local scale analysis unit for setting a sliding window size of 3 frames, calculating local statistical properties of the differential invariants within the window, and detecting transient abnormal behaviors;

[0016] a segment scale analysis unit for setting an analysis segment length of 9 frames, calculating statistical distribution of the differential invariants within the segment, and detecting short-time abnormal behaviors;

[0017] a global scale analysis unit for analyzing the complete behavior sequence, extracting global morphological features of the trajectory, and detecting long-time abnormal behaviors.

[0018] Preferably, the motion representation module further comprises a behavior model updating unit based on manifold learning, for:

[0019] combining the extracted differential invariants into a high-dimensional feature vector;

[0020] constructing a topological structure of the feature space;

[0021] initializing an empty feature set as the starting point of the manifold;

[0022] when receiving a new feature vector, calculating its mapping relationship with the manifold;

[0023] judging whether the feature belongs to a known behavior pattern according to the mapping error;

[0024] updating the manifold structure to contain the new normal behavior feature.

[0025] Preferably, the behavior model updating unit based on manifold learning is further used for:

[0026] periodically checking the integrity of the manifold structure;

[0027] removing feature points that are outdated or rarely appear;

[0028] merging highly similar feature regions to simplify the model;

[0029] enhancing the features in combination with scene information;

[0030] adjusting the abnormal threshold in specific scenarios;

[0031] considering the influence of scene changes on behavior patterns.

[0032] Preferably, the detection method of the system comprises:

[0033] extracting the motion speed of the current frame in the collected video frame sequence, and obtaining the motion trajectory of the current frame through the motion speeds of the current frame and the previous N frames;

[0034] constructing the motion trajectory of the current frame to obtain the original motion trajectory image at the current time;

[0035] The original motion trajectory image features are normalized and input into a convolutional neural network;

[0036] The motion trajectory of the current frame is subjected to convolution operation and pooling operation, so that the motion trajectory anomaly distribution probability of the current frame in the detection region is obtained.

[0037] The collected video frames are subjected to scene information extraction, the video frames are input into a convolutional neural network, the scene is detected, the current frame image features are input into a recurrent neural network for anomaly detection of the current frame, and the anomaly detection result is recorded;

[0038] The anomaly detection result and the scene detection result are input into a motion model acquisition submodule, and the scene anomaly detection result is updated.

[0039] As preferred, the system adopts an RPI3B+ processor, adopts 3.5W energy, and supports maximum 2-way USB camera input; the system further comprises:

[0040] A scene adaptability unit is used for learning scene features under normal user behavior, and generating self-adaptive feedback to changes in the scene;

[0041] A behavior feature extraction unit is used for directly extracting features from the video, and encoding continuous short-time images of each video into feature vectors through an intelligent coding network;

[0042] A behavior model unit is used for learning the relevance between features based on time sequence image information, learning the relevance between features through a recurrent neural network on time sequence image features extracted by the intelligent coding network, constructing a motion pattern, and recording the duration of each action.

[0043] The present application has the following beneficial effects:

[0044] 1. The motion trajectory is characterized by a differential geometry framework, realizing viewpoint invariance, so that the detection performance remains stable even if the camera position changes;

[0045] 2. A multi-scale anomaly measurement framework is adopted, which can capture small behavior anomalies and significantly improve detection sensitivity;

[0046] 3. A model updating mechanism based on manifold learning enables the system to have self-adaptive learning ability and does not need to be retrained after environmental changes;

[0047] 4. A lightweight algorithm design significantly reduces the amount of calculation and is suitable for running on resource-limited embedded devices;

[0048] 5. A low-power design controls the overall power consumption within 3.5W, which is suitable for long-time monitoring applications. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 The overall architecture diagram of the embedded behavior anomaly detection system based on time sequence image analysis of the present application;

[0050] Figure 2 The structural schematic diagram of the motion representation module of the present application;

[0051] Figure 3 The flowchart of the trajectory differential representation method based on the Frei and Chan framework of the present application;

[0052] Figure 4 The schematic diagram of the multi-scale anomaly measurement framework of the present application;

[0053] Figure 5 The behavior model updating mechanism based on manifold learning of the present application;

[0054] Figure 6 The working flowchart of the detection and early warning module of the present application. DETAILED DESCRIPTION

[0055] Reference Figures 1-6 The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0056] As Figure 1 shown, the embedded behavior anomaly detection system based on time sequence image analysis provided by the present application comprises an image acquisition module 1, a motion representation module 2 and a detection and early warning module 3. The image acquisition module 1 is connected with the motion representation module 2 through a communication interface, and the motion representation module 2 is connected with the detection and early warning module 3 through a communication interface, forming a complete data processing closed loop.

[0057] The image acquisition module 1 is used for collecting a sequence of monitoring video frames at a regular time interval and pre-processing each frame of image. Specifically, the image acquisition module 1 first collects video frames at a preset frequency (preferably 25 frames per second), and then converts the RGB color space to the HSV color space. In an embodiment of the present application, the HSV color space has better illumination invariance than the RGB color space, and can effectively cope with the case of environmental light change.

[0058] After conversion to the HSV color space, the image acquisition module 1 further crops the center region of the image to remove the background of the image. Preferably, the cropping range is the region of 80% of the center of the image, which can effectively remove the edge noise while retaining the main target information. Through the background removal processing, the final color image containing a person is obtained, providing high-quality input data for the subsequent motion feature extraction.

[0059] As Figure 2As shown, the motion representation module 2 is communicatively connected with the image acquisition module 1, and is configured to receive the color image containing a person transmitted by the image acquisition module 1, extract a motion trajectory, establish a differential geometry framework, generate a differential invariant feature, construct a behavior feature manifold, calculate a feature mapping distance, and determine a behavior anomaly. The motion representation module 2 includes a behavior model acquisition sub-module 21, a motion trajectory representation sub-module 22, and a local feature extraction sub-module 23.

[0060] The behavior model acquisition sub-module 21 is configured to receive the color image containing a person transmitted by the image acquisition module 1, divide the data sequence into a sample by every 9 frames, and initialize an empty sequence as a model behavior baseline. In a preferred embodiment of the present application, 9 frames are selected as the basic analysis unit, which is the best parameter verified by experiments, and can achieve a good balance between capturing sufficient timing information and maintaining computational efficiency.

[0061] When there is a data sequence input, the behavior model acquisition sub-module 21 compares the first data value of the sequence with the average value in the empty sequence. If it is greater than a set threshold value (such as 0.8), the current data value is added to the end of the sequence, the average value is replaced by the current value, and then the sequence is shifted forward by one bit to receive a new data value, which is compared with the average value in the sequence, and the above operation is repeated. The threshold value 0.8 is an empirical value obtained by analyzing a large amount of experimental data, which can effectively distinguish potential new behavior patterns.

[0062] When all values in the sequence are less than another set threshold value (such as 0.2), the processing is stopped, the sequence at this time is taken as a motion behavior baseline, and is transmitted to the motion trajectory representation sub-module 22. The threshold value 0.2 is also based on experimental statistics, which indicates that when all feature differences are small enough, the current behavior pattern has stabilized and can be used as a baseline model.

[0063] The motion trajectory representation sub-module 22 is configured to receive the motion behavior baseline as a target trajectory vector, use the sparse optical flow algorithm to analyze the current acquired video frame as a raw image, and extract motion features. The present application selects the sparse optical flow algorithm instead of the dense optical flow algorithm because the former achieves a good balance between computational efficiency and accuracy, and is suitable for running on embedded devices.

[0064] In a preferred embodiment, the parameters of the sparse optical flow algorithm are set as follows: the pyramid level is 3, the window size is 15x15 pixels, the maximum number of iterations is 30, and the minimum feature point quality is 0.01. These parameter settings can ensure that sufficient accurate motion features are obtained while maintaining computational efficiency.

[0065] The motion trajectory representation submodule 22 takes the difference between the current frame motion vector and the target trajectory vector as the difference value of the feature, and obtains the difference distribution sequence, i.e., the motion trajectory, by repeatedly extracting the motion feature. In particular, the motion trajectory representation submodule 22 constructs the differential geometry representation of the trajectory based on the Frenet frame, including calculating the curvature feature and torsion feature of the trajectory, to form the differential invariant feature vector.

[0066] As shown in Figure 3 , the trajectory differential representation method based on the Frenet frame includes the following steps:

[0067] First, the trajectory point sequence is represented parametrically. Assuming that the trajectory point sequence is , where is the time parameter, representing the value on the time axis; , , are the coordinate components in three-dimensional space, representing the position of the trajectory in three-dimensional space.

[0068] Then, the tangent vector of the trajectory is calculated:

[0069] ,

[0070] where is the unit tangent vector of the trajectory at time ; represents the first-order derivative of with respect to time , i.e., the velocity vector of the trajectory; represents the modulus of the vector , used for normalizing the tangent vector; , , are the first-order derivatives of , , with respect to time , representing the velocity components of the trajectory in each coordinate axis direction.

[0071] Next, the normal vector of the trajectory is calculated:

[0072] ,

[0073] where is the unit normal vector of the trajectory at time ; represents the first-order derivative of with respect to time ; represents the modulus of the vector , used for normalizing the normal vector; denotes the second derivative of time , i.e. the acceleration vector of the trajectory; denotes the dot product of vectors, the projection of the acceleration vector on the tangent vector; denotes the component of the acceleration vector in the tangent direction; denotes the component of the acceleration vector in the normal plane.

[0074] Then, the binormal vector B(t) of the trajectory is calculated:

[0075] ,

[0076] where B(t) is the unit binormal vector of the trajectory at time t; denotes the cross product of vectors, the cross product of the tangent and normal vectors, resulting in a vector perpendicular to both.

[0077] Based on the above three orthogonal vectors, the Frenet frame, i.e. the local coordinate system, at each point on the trajectory can be constructed.

[0078] Further, the curvature of the trajectory is calculated:

[0079] ,

[0080] where K(t) is the curvature of the trajectory at time t, indicating the degree of deviation of the trajectory from a straight line; denotes the modulus of the cross product of the velocity and acceleration vectors; denotes the cube of the modulus of the velocity vector. The curvature is an important feature of the shape of the trajectory, and the larger the value indicates the higher the degree of curvature of the trajectory.

[0081] Finally, the torsion of the trajectory is calculated:

[0082] ,

[0083] where T(t) is the torsion of the trajectory at time t, indicating the degree of deviation of the trajectory from a plane; denotes the third derivative of time , i.e. the jerk vector of the trajectory; the dot product of the cross product of the velocity and acceleration vectors and the jerk vector; The square of the modulus of the cross product of the velocity vector and the acceleration vector. The value of zero indicates that the trajectory is located in a plane.

[0084] By calculating the curvature and torsion of the trajectory, the differential invariant feature vector of the trajectory can be obtained, which is not affected by the observation angle and has good invariance.

[0085] The local feature extraction submodule 23 uses each frame motion trajectory vector as a sample, and uses a convolutional neural network for convolution operation, and finally obtains a local abnormal point. According to the size of the feature difference value, the frame whose difference is greater than the abnormal threshold is marked as abnormal.

[0086] In an embodiment of the present application, the convolutional neural network performs time domain anomaly detection, and the network structure includes an input layer, a feature layer, a time domain sampling connection layer, a full connection layer, and an output layer. Specifically, the input layer receives the motion trajectory vector, the feature layer extracts spatial features through convolution operation, the time domain sampling connection layer captures the time sequence relationship, the full connection layer comprehensively analyzes the features, and the output layer gives the abnormal probability.

[0087] Preferably, the convolutional layer uses a 3x3 convolution kernel, a step of 1, and a padding of 1 to keep the feature map size unchanged. The time domain sampling connection layer uses a long short-term memory (LSTM) unit, and the hidden layer dimension is 128, which can effectively capture the time sequence dependence.

[0088] The local feature extraction submodule 23 marks the frame whose difference is greater than the abnormal threshold as abnormal according to the size of the feature difference value. In a preferred embodiment, the abnormal threshold is set to 0.6, which is an empirical value based on a large amount of experimental data, and can achieve a good balance between reducing the false positive rate and maintaining the detection sensitivity.

[0089] The detection and warning module 3 is in communication connection with the motion representation module 2, and is used to receive the abnormal features transmitted by the motion representation module 2, perform abnormal motion detection, generate an abnormal detection result, and feed back the abnormal detection result and the scene information of the current video frame to the behavior model acquisition submodule 21 of the motion representation module 2, for updating the behavior baseline model.

[0090] As shown in Figure 6 , the detection and warning module 3 takes the abnormal motion sample as input and uses a neural network to detect abnormal motion. The loss function of the neural network includes the number of training samples, the abnormal detection label, the network prediction value, and the abnormal detection accuracy indicator parameter. Specifically, the loss function can be represented as:

[0091] ,

[0092] wherein, The loss function value represents the degree of difference between the model's prediction and the true label. The number of training samples represents the number of samples used for training. These are anomaly detection labels (0 for normal, 1 for abnormal), indicating the first... The true category of each sample; Network prediction ( , indicating that the model is for the th The probability that a sample is predicted to be abnormal; This is a parameter for anomaly detection accuracy, used to balance cross-entropy loss and absolute error loss, and to control the relative importance of the two loss terms; It is the natural logarithm function; This represents the absolute error between the predicted value and the true label.

[0093] Preferably, parameters Setting it to 0.3 is the optimal value obtained through extensive experimental comparisons, which can improve the model's generalization ability while maintaining detection accuracy.

[0094] The detection and early warning module 3 feeds back the neural network detection results and scene information of the current video frame to the behavior model acquisition submodule 21 to update the behavior baseline model. This closed-loop feedback mechanism enables the system to continuously learn and adapt to environmental changes, improving detection accuracy.

[0095] In a preferred embodiment of the present invention, such as Figure 4 As shown, the motion characterization module 2 adopts a multi-scale anomaly measurement framework, including local scale analysis units, segment scale analysis units, and global scale analysis units.

[0096] The local scale analysis unit is used to set the sliding window size to 3 frames, calculate the local statistical properties of differential invariants within the window, and detect transient anomalous behavior. Specifically, for the sequence of differential invariants within the window... Calculate its mean ,variance and skewness :

[0097] ,

[0098] ,

[0099] ,

[0100] in, It is the arithmetic mean of the differential invariants within the local window; The variance of the differential invariants within the local window represents the degree of dispersion of the data; skewness of the differential invariants within a local window, representing the asymmetry of the data distribution; denotes the value of the differential invariant of the frame, which can be the curvature , torsion , or their combination; denotes the summation operation from the frame to the frame.

[0101] By analyzing the changes of these statistical properties, instantaneous abnormal behaviors such as sudden acceleration or sudden stop can be detected.

[0102] The paragraph-scale analysis unit is used to set the analysis paragraph length to 9 frames, calculate the statistical distribution of the differential invariants within the paragraph, and detect short-time abnormal behaviors. Specifically, for the differential invariant sequence within the paragraph, in addition to calculating the mean, variance, and skewness, the kurtosis and autocorrelation coefficient are also calculated:

[0103] ,

[0104] ,

[0105] wherein, is the kurtosis of the differential invariants within the paragraph, representing the degree of spikiness of the data distribution; is the mean of the differential invariants within the paragraph; is the standard deviation of the differential invariants within the paragraph; is the autocorrelation coefficient of the lag steps, representing the correlation between the data sequence and its own lag steps; is the lag order, usually taking 1 to 3; denotes the summation operation from the frame to the frame; denotes the summation operation from the frame to the frame.

[0106] These statistical properties can help detect short-time abnormal behaviors such as unnatural turns or pauses.

[0107] The global-scale analysis unit is used to analyze the complete behavior sequence, extract the global morphological features of the trajectory, and detect long-time abnormal behaviors. Global analysis usually considers the trajectory morphology within 30 seconds or longer, extracting the overall geometric features of the trajectory such as the long-term statistical properties of the curvature distribution, torsion distribution, and speed distribution. These features can help detect long-time abnormal behaviors such as abnormal movement paths or long periods of inactivity.

[0108] like Figure 5 As shown, in another preferred embodiment of the present invention, the motion representation module 2 further includes a behavior model update unit based on manifold learning, which is used to combine the extracted differential invariants into high-dimensional feature vectors, construct the topological structure of the feature space, and perform manifold learning and model update.

[0109] Specifically, the behavior model update unit based on manifold learning first combines the differential invariants into a high-dimensional feature vector. ,in For feature vectors; The i-th differential invariant feature can be curvature, torsion, or other motion features; The feature dimension represents the length of the feature vector; the superscript... This represents the transpose of a vector, converting a row vector into a column vector.

[0110] Then, define the similarity measurement function between feature vectors:

[0111] ,

[0112] in, For feature vectors and The distance metric between two feature vectors indicates the degree of similarity between them. and These are the i-th and j-th feature vectors, respectively; The weight of the k-th feature is used to balance the importance of different features; This represents the k-th component of the i-th eigenvector; This represents the k-th component of the j-th eigenvector; This represents the square root of the sum of squares of the differences between the 1st and dth eigencomponents, i.e., the calculation of the Euclidean distance.

[0113] Based on similarity metrics, a topological structure for the feature space is constructed, and an empty feature set is initialized as the starting point of the manifold. When a new feature vector is received, its mapping to the manifold is calculated:

[0114] ,

[0115] in, For the new feature vector to manifold The mapping error represents the degree of difference between the new feature and the known behavioral pattern; For the new feature vector; The current manifold model contains a set of known behavioral feature points; Representing a manifold Feature points in ; representing finding the feature point with minimum distance in the manifold .

[0116] According to the mapping error, it is judged whether the feature belongs to the known behavior mode. If (wherein is a high threshold, and the default is 0.8), it is considered that represents a potential new behavior mode, and it is added to the manifold model; if (wherein is a low threshold, and the default is 0.2), it is considered that belongs to the known behavior mode, and the manifold model is not updated.

[0117] In addition, the behavior model updating unit based on manifold learning also regularly checks the manifold structure integrity, removes the feature points that are outdated or low-frequency, merges the highly similar feature regions, and simplifies the model. Preferably, the structure checking period is 100 frames, which is the best value based on experimental verification, and can balance between maintaining the model accuracy and reducing the computational overhead.

[0118] The behavior model updating unit based on manifold learning also combines scene information for feature enhancement, adjusts the abnormal threshold in a specific scene, and considers the influence of scene changes on the behavior mode. This context-aware mechanism enables the system to more accurately identify abnormal behavior in a specific scene.

[0119] As shown in Figures 1 to 6 , the detection method of the system of the present application comprises: extracting the motion speed of the current frame in the collected video frame sequence, obtaining the motion trajectory of the current frame through the motion speed of the current frame and the previous N frames; constructing the motion trajectory of the current frame to obtain the original motion trajectory image at the current time; inputting the normalized features of the original motion trajectory image to the convolutional neural network; performing convolution operation and pooling operation on the motion trajectory of the current frame to obtain the motion trajectory abnormal distribution probability of the current frame in the detection region; extracting the scene information of the collected video frame, inputting the video frame to the convolutional neural network, detecting the scene, inputting the current frame image features to the recurrent neural network for abnormal detection of the current frame, and recording the abnormal detection result; inputting the abnormal detection result and the scene detection result to the motion model acquisition submodule to update the scene abnormal detection result.

[0120] ​In an embodiment of the present application, the system adopts RPI3B+ processor, adopts 3.5W energy, supports maximum 2-way USB camera input. The system also includes a scene adaptability unit for learning scene features under normal user behavior, and generating adaptive feedback to changes in the scene; a behavior feature extraction unit for directly extracting features from video, encoding each continuous short image captured from the video into a feature vector through an intelligent coding network; a behavior model unit for learning the relevance between features based on time-series image information, constructing motion patterns through a recurrent neural network on the time-series image features extracted by the intelligent coding network, and recording the duration of each action.

[0121] In particular, in order to ensure the consistency of data processing, the system normalizes data from different sources. For image data, pixel values are normalized to the range [0, 1]; for motion features, feature values are mapped to the range [-1, 1] through the maximum and minimum value normalization method:

[0122] .

[0123] wherein, is the normalized feature value, ranging between [-1, 1]; is the original feature value; is the minimum value of the feature; is the maximum value of the feature; Mapping the feature value to the range [0, 1]; multiplying by 2 and then subtracting 1 adjusts the range to [-1, 1].

[0124] Through normalization processing, the scales of different source data can be unified, improving the stability and accuracy of the system.

[0125] The embedded behavior anomaly detection system based on time-series image analysis provided by the present application realizes accurate detection of abnormal behavior through the collaborative work of the image acquisition module, the motion representation module, and the detection and warning module. The system introduces differential geometry theory to accurately model the motion trajectory, adopts a multi-scale anomaly measurement framework and an adaptive model updating mechanism based on manifold learning, and has the advantages of viewpoint invariance, high detection sensitivity, and adaptive learning ability.

[0126] In practical applications, the present system can be deployed on embedded platforms such as RPI3B+, with a power consumption of only 3.5W, supporting maximum 2-way USB camera input, and is suitable for various scenes such as security monitoring, smart home, medical monitoring, etc. Compared with traditional methods, the present system has significant improvements in detection accuracy, adaptability, and computational efficiency, and has wide application prospects.

[0127] The above embodiments only express the specific implementation of the present application, which is described in more detail and in more detail, but cannot be understood as a limitation on the scope of the patent of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application.

Claims

1. An embedded behavioral anomaly detection system based on time-series image analysis, characterized in that, include: The image acquisition module is used to periodically acquire surveillance video frame sequences, preprocess each frame image, convert the RGB color space to the HSV color space, crop the central area of ​​the image, remove the image background, and obtain a color image containing people. The motion representation module is communicatively connected to the image acquisition module. It is used to receive a color image containing a person transmitted by the image acquisition module, extract the motion trajectory, establish a differential geometric framework, generate differential invariant features, construct a behavioral feature manifold, calculate the feature mapping distance, and determine behavioral anomalies. The motion representation module includes a behavior model acquisition submodule, a motion trajectory representation submodule, and a local feature extraction submodule. The detection and early warning module is communicatively connected to the motion representation module. It is used to receive abnormal features transmitted by the motion representation module, perform abnormal motion detection, generate abnormal detection results, and feed back the abnormal detection results and scene information of the current video frame to the behavior model acquisition submodule of the motion representation module for updating the behavior baseline model.

2. The embedded behavioral anomaly detection system based on time-series image analysis according to claim 1, characterized in that: The behavior model acquisition submodule is used to receive color images containing people transmitted by the image acquisition module, divide the data sequence into samples of 9 frames each, and initialize an empty sequence as the model behavior baseline; when a data sequence is input, the first data value of the sequence is compared with the average value in the empty sequence. If it is greater than a set threshold, such as 0.8, the current data value is added to the end of the sequence, replacing the average value with the current value. Then the sequence is shifted forward by one position, receiving new data values, comparing them with the average value in the sequence, and repeating the above operation; when all values ​​in the sequence are less than 0.2, processing stops, and the sequence at this time is used as the motion behavior baseline and transmitted to the motion trajectory representation submodule.

3. The embedded behavioral anomaly detection system based on time-series image analysis according to claim 2, characterized in that: The motion trajectory representation submodule is used to receive the motion behavior baseline as the target trajectory vector, take the currently acquired video frame as the original image, analyze it using the sparse optical flow algorithm, extract motion features, and take the difference between the current frame motion vector and the target trajectory vector as the feature difference value. The motion features are repeatedly extracted to obtain the differential distribution sequence, i.e., the motion trajectory; wherein, the motion trajectory representation submodule constructs the differential geometric representation of the trajectory based on the Frenet framework, including calculating the trajectory curvature features and torsion features to form a differential invariant feature vector.

4. The embedded behavioral anomaly detection system based on time-series image analysis according to claim 3, characterized in that: The local feature extraction submodule is used to treat the motion trajectory vector of each frame as a sample, perform convolution operation using a convolutional neural network, and finally obtain local anomaly points. Based on the magnitude of the feature difference value, frames with differences greater than the anomaly threshold are labeled as anomalies. The convolutional neural network performs temporal anomaly detection, and the network structure includes an input layer, a feature layer, a temporal sampling connection layer, a fully connected layer, and an output layer.

5. The embedded behavioral anomaly detection system based on time-series image analysis according to claim 1, characterized in that: The detection and early warning module is used to take abnormal motion samples as input and use a neural network to detect abnormal motion. The loss function of the neural network includes the number of training samples, abnormal detection labels, network prediction values, and abnormal detection accuracy index parameters. The detection results of the neural network and the scene information of the current video frame are fed back to the behavior model acquisition submodule to update the behavior baseline model.

6. The embedded behavioral anomaly detection system based on time-series image analysis according to claim 1, characterized in that: The motion characterization module employs a multi-scale anomaly measurement framework, including: The local scale analysis unit is used to set the sliding window size to 3 frames, calculate the local statistical properties of the differential invariants within the window, and detect transient abnormal behavior. The segment scale analysis unit is used to set the analysis segment length to 9 frames, calculate the statistical distribution of differential invariants within the segment, and detect short-term abnormal behavior; The global scale analysis unit is used to analyze complete behavior sequences, extract global trajectory morphological features, and detect long-term abnormal behavior.

7. The embedded behavioral anomaly detection system based on time-series image analysis according to claim 1, characterized in that: The motion representation module further includes a behavior model update unit based on manifold learning, used for: The extracted differential invariants are combined into a high-dimensional feature vector; Construct the topological structure of the feature space; Initialize an empty feature set as the starting point of the manifold; When a new eigenvector is received, its mapping relationship with the manifold is calculated; Determine whether a feature belongs to a known behavior pattern based on the mapping error; Update the manifold structure to include new normal behavioral characteristics.

8. The embedded behavioral anomaly detection system based on time-series image analysis according to claim 7, characterized in that: The manifold-based behavior model update unit is also used for: Regularly check the integrity of the manifold structure; Remove outdated or infrequently occurring feature points; Merge highly similar feature regions to simplify the model; Feature enhancement is performed by incorporating scene information; Adjust the anomaly threshold in specific scenarios; Consider the impact of changes in the scenario on behavioral patterns.

9. The embedded behavioral anomaly detection system based on time-series image analysis according to claim 1, characterized in that: The detection method of the system includes: The motion velocity of the current frame in the acquired video frame sequence is extracted, and the motion trajectory of the current frame is obtained by combining the motion velocities of the current frame and the previous N frames. Construct the motion trajectory of the current frame to obtain the original motion trajectory image at the current moment; The features of the original motion trajectory image are normalized and then input into the convolutional neural network; Perform convolution and pooling operations on the motion trajectory of the current frame to obtain the probability of abnormal distribution of the motion trajectory of the current frame within the detection area; Scene information is extracted from the acquired video frames. The video frames are input into a convolutional neural network to detect the scene. The image features of the current frame are input into a recurrent neural network to detect anomalies in the current frame and the anomaly detection results are recorded. The anomaly detection results and scene detection results are input together into the motion model acquisition submodule to update the scene anomaly detection results.

10. The embedded behavioral anomaly detection system based on time-series image analysis according to claim 9, characterized in that: The system uses an RPI3B+ processor, consumes 3.5W of power, and supports up to two USB camera inputs. The system also includes: Scene adaptation unit is used to learn scene characteristics under normal user behavior and generate adaptive feedback to changes in the scene; The behavioral feature extraction unit is used to extract features directly from the video. Through an intelligent coding network, it encodes the continuous short-time images captured from each video into feature vectors. The behavior model unit is used to learn the correlation between features based on temporal image information extracted by the intelligent coding network, construct motion patterns, and record the duration of each action through a recurrent neural network.

Images