Embedded behavior anomaly detection system based on time-series image analysis

Abstract

The present application relates to the technical field of computer vision, in particular to an embedded behavior anomaly detection system based on time sequence image analysis, comprising an image acquisition module, a motion representation module and a detection and early warning module, the image acquisition module acquires video frames regularly and converts them into HSV color space for preprocessing; the motion representation module comprises a behavior model acquisition submodule, a motion trajectory representation submodule and a local feature extraction submodule, a Frei Frame is constructed by introducing differential geometry theory to parameterize the motion trajectory, differential invariant features such as curvature and torsion are calculated, and abnormality is determined in combination with a multi-scale abnormality measurement framework; the detection and early warning module analyzes abnormal features, generates a detection result and feeds back to update the model, the present application adopts an adaptive model updating mechanism based on manifold learning, has view invariance, high detection sensitivity and adaptive learning ability, and can be deployed on a low-power embedded device.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an embedded behavior anomaly detection system based on time-series image analysis, which can be applied to video surveillance, security monitoring, smart homes, medical monitoring and other fields. Background Technology

[0002] With the widespread application of video surveillance systems, how to automatically detect abnormal behavior from massive amounts of video data has become an important research topic. Traditional abnormal behavior detection methods are usually based on rule matching or simple motion feature analysis, which are difficult to adapt to complex and ever-changing real-world environments and often require a large amount of computing resources, making them unsuitable for deployment on embedded devices.

[0003] Existing abnormal behavior detection technologies mainly suffer from the following problems: First, traditional detection methods based on position and velocity features are sensitive to changes in camera position, requiring model retraining when the viewpoint changes; second, existing methods mainly rely on trajectory representation in absolute coordinates, making it difficult to capture the essential features of behavior; third, fixed-model methods lack adaptive learning capabilities and are difficult to adapt to environmental changes; and finally, complex algorithms have high computational costs, making them unsuitable for running on resource-constrained embedded devices.

[0004] Therefore, there is an urgent need for a behavior anomaly detection system that can be implemented efficiently, accurately, and adaptively on embedded devices. Summary of the Invention

[0005] The purpose of this invention is to provide an embedded behavior anomaly detection system based on temporal image analysis. By introducing differential geometry theory to accurately model motion trajectories, it can achieve accurate analysis and anomaly detection of complex human behavior.

[0006] This invention proposes an embedded behavior anomaly detection system based on time-series image analysis, comprising:

[0007] The image acquisition module is used to periodically acquire surveillance video frame sequences, preprocess each frame image, convert the RGB color space to the HSV color space, crop the central area of ​​the image, remove the image background, and obtain a color image containing people.

[0008] The motion representation module is communicatively connected to the image acquisition module. It is used to receive a color image containing a person transmitted by the image acquisition module, extract the motion trajectory, establish a differential geometric framework, generate differential invariant features, construct a behavioral feature manifold, calculate the feature mapping distance, and determine behavioral anomalies. The motion representation module includes a behavior model acquisition submodule, a motion trajectory representation submodule, and a local feature extraction submodule.

[0009] The detection and early warning module is communicatively connected to the motion representation module. It is used to receive abnormal features transmitted by the motion representation module, perform abnormal motion detection, generate abnormal detection results, and feed back the abnormal detection results and scene information of the current video frame to the behavior model acquisition submodule of the motion representation module for updating the behavior baseline model.

[0010] Preferably, the behavior model acquisition submodule is used to receive a color image containing a person transmitted by the image acquisition module, divide the data sequence into samples of 9 frames each, and initialize it as an empty sequence as the model behavior baseline; when a data sequence is input, the first data value of the sequence is compared with the average value in the empty sequence. If it is greater than a set threshold, such as 0.8, the current data value is added to the end of the sequence, replacing the average value with the current value. Then the sequence is shifted forward by one position, receiving a new data value, comparing it with the average value in the sequence, and repeating the above operation; when all values ​​in the sequence are less than 0.2, processing stops, the sequence at this time is used as the motion behavior baseline, and transmitted to the motion trajectory representation submodule.

[0011] Preferably, the motion trajectory representation submodule is used to receive the motion behavior baseline as the target trajectory vector, use the currently acquired video frame as the original image, analyze it using a sparse optical flow algorithm, extract motion features, and use the difference between the current frame motion vector and the target trajectory vector as the feature difference value; repeatedly extract motion features to obtain a differential distribution sequence, i.e., the motion trajectory; wherein, the motion trajectory representation submodule constructs a differential geometric representation of the trajectory based on the Frenet framework, including calculating the trajectory curvature features and torsion features to form a differential invariant feature vector.

[0012] Preferably, the local feature extraction submodule is used to treat the motion trajectory vector of each frame as a sample, perform convolution operation using a convolutional neural network, and finally obtain local anomaly points. Based on the magnitude of the feature difference value, frames with differences greater than the anomaly threshold are labeled as anomalies. The convolutional neural network performs temporal anomaly detection, and the network structure includes an input layer, a feature layer, a temporal sampling connection layer, a fully connected layer, and an output layer.

[0013] Preferably, the detection and early warning module is used to take abnormal motion samples as input, and use a neural network to detect abnormal motion. The loss function of the neural network includes the number of training samples, abnormal detection labels, network prediction values, and abnormal detection accuracy index parameters. The detection results of the neural network and the scene information of the current video frame are fed back to the behavior model acquisition submodule to update the behavior baseline model.

[0014] Preferably, the motion characterization module employs a multi-scale anomaly measurement framework, including:

[0015] The local scale analysis unit is used to set the sliding window size to 3 frames, calculate the local statistical properties of the differential invariants within the window, and detect transient abnormal behavior.

[0016] The segment scale analysis unit is used to set the analysis segment length to 9 frames, calculate the statistical distribution of differential invariants within the segment, and detect short-term abnormal behavior;

[0017] The global scale analysis unit is used to analyze complete behavior sequences, extract global trajectory morphological features, and detect long-term abnormal behavior.

[0018] Preferably, the motion representation module further includes a behavior model update unit based on manifold learning, used for:

[0019] The extracted differential invariants are combined into a high-dimensional feature vector;

[0020] Construct the topological structure of the feature space;

[0021] Initialize an empty feature set as the starting point of the manifold;

[0022] When a new eigenvector is received, its mapping relationship with the manifold is calculated;

[0023] Determine whether a feature belongs to a known behavior pattern based on the mapping error;

[0024] Update the manifold structure to include new normal behavioral characteristics.

[0025] Preferably, the manifold-based behavior model update unit is further configured to:

[0026] Regularly check the integrity of the manifold structure;

[0027] Remove outdated or infrequently occurring feature points;

[0028] Merge highly similar feature regions to simplify the model;

[0029] Feature enhancement is performed by incorporating scene information;

[0030] Adjust the anomaly threshold in specific scenarios;

[0031] Consider the impact of changes in the scenario on behavioral patterns.

[0032] Preferably, the detection method of the system includes:

[0033] The motion velocity of the current frame in the acquired video frame sequence is extracted, and the motion trajectory of the current frame is obtained by combining the motion velocities of the current frame and the previous N frames.

[0034] Construct the motion trajectory of the current frame to obtain the original motion trajectory image at the current moment;

[0035] The features of the original motion trajectory image are normalized and then input into the convolutional neural network;

[0036] Perform convolution and pooling operations on the motion trajectory of the current frame to obtain the probability of abnormal distribution of the motion trajectory of the current frame within the detection area;

[0037] Scene information is extracted from the acquired video frames. The video frames are input into a convolutional neural network to detect the scene. The image features of the current frame are input into a recurrent neural network to detect anomalies in the current frame and the anomaly detection results are recorded.

[0038] The anomaly detection results and scene detection results are input together into the motion model acquisition submodule to update the scene anomaly detection results.

[0039] Preferably, the system uses an RPI3B+ processor, consumes 3.5W of power, and supports up to two USB camera inputs; the system also includes:

[0040] Scene adaptation unit is used to learn scene characteristics under normal user behavior and generate adaptive feedback to changes in the scene;

[0041] The behavioral feature extraction unit is used to extract features directly from the video. Through an intelligent coding network, it encodes the continuous short-time images captured from each video into feature vectors.

[0042] The behavior model unit is used to learn the correlation between features based on temporal image information extracted by the intelligent coding network, construct motion patterns, and record the duration of each action through a recurrent neural network.

[0043] The present invention has the following beneficial effects:

[0044] 1. By representing the motion trajectory through a differential geometry framework, viewpoint invariance is achieved, ensuring stable detection performance even when the camera position changes;

[0045] 2. Employing a multi-scale anomaly measurement framework, it can capture minute behavioral anomalies, significantly improving detection sensitivity;

[0046] 3. The manifold learning-based model update mechanism enables the system to have adaptive learning capabilities, eliminating the need for retraining after environmental changes;

[0047] 4. Lightweight algorithm design significantly reduces computational load, making it suitable for running on resource-constrained embedded devices;

[0048] 5. Low power consumption design, with overall power consumption controlled within 3.5W, suitable for long-term monitoring applications. Attached Figure Description

[0049] Figure 1 This is an overall architecture diagram of the embedded behavioral anomaly detection system based on time-series image analysis of the present invention;

[0050] Figure 2 This is a schematic diagram of the motion characterization module of the present invention;

[0051] Figure 3 This is a flowchart of the trajectory differential representation method based on the Frenet framework of the present invention;

[0052] Figure 4 This is a schematic diagram of the multi-scale anomaly measurement framework of the present invention;

[0053] Figure 5 This is a schematic diagram of the behavior model update mechanism based on manifold learning in this invention;

[0054] Figure 6 This is a flowchart of the detection and early warning module of the present invention. Detailed Implementation

[0055] Please refer to Figures 1-6 The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0056] like Figure 1 As shown, the embedded behavioral anomaly detection system based on time-series image analysis provided by this invention includes an image acquisition module 1, a motion representation module 2, and a detection and early warning module 3. The image acquisition module 1 is connected to the motion representation module 2 via a communication interface, and the motion representation module 2 is connected to the detection and early warning module 3 via a communication interface, forming a complete data processing closed loop.

[0057] Image acquisition module 1 is used to periodically acquire a sequence of monitoring video frames and preprocess each frame. Specifically, image acquisition module 1 first acquires video frames at a preset frequency (preferably 25 frames per second), and then converts the RGB color space to the HSV color space. In one embodiment of the present invention, the HSV color space has better illumination invariance than the RGB color space, and can effectively cope with changes in ambient light.

[0058] After conversion to the HSV color space, the image acquisition module 1 further crops the central region of the image to remove the background. Preferably, the cropping area is 80% of the central region of the image, which effectively removes edge noise while retaining the main target information. Through background removal processing, a color image containing the person is finally obtained, providing high-quality input data for subsequent motion feature extraction.

[0059] like Figure 2As shown, the motion representation module 2 is communicatively connected to the image acquisition module 1. It receives color images containing a person from the image acquisition module 1, extracts the motion trajectory, establishes a differential geometric framework, generates differential invariant features, constructs a behavioral feature manifold, calculates the feature mapping distance, and determines behavioral anomalies. The motion representation module 2 includes a behavioral model acquisition submodule 21, a motion trajectory representation submodule 22, and a local feature extraction submodule 23.

[0060] The behavior model acquisition submodule 21 receives color images containing people transmitted by the image acquisition module 1, divides the data sequence into samples of 9 frames each, and initializes it as an empty sequence as the baseline for model behavior. In a preferred embodiment of the present invention, selecting 9 frames as the basic analysis unit is an experimentally verified optimal parameter that achieves a good balance between capturing sufficient temporal information and maintaining computational efficiency.

[0061] When a data sequence is input, the behavior model acquisition submodule 21 compares the first data value of the sequence with the average value in the empty sequence. If it is greater than a set threshold (e.g., 0.8), the current data value is added to the end of the sequence, replacing the average value with the current value. Then, the sequence is shifted forward one position to receive new data values, which are compared with the average value in the sequence, and the above operation is repeated. The threshold of 0.8 is an empirical value derived from the analysis of a large amount of experimental data, which can effectively distinguish potential new behavioral patterns.

[0062] When all values ​​in the sequence are less than another set threshold (e.g., 0.2), processing stops, and the sequence at this point is used as the baseline for motion behavior and passed to the motion trajectory representation submodule 22. The threshold of 0.2 is also based on experimental statistics, indicating that when all feature differences are sufficiently small, the current behavior pattern has stabilized and can be used as a baseline model.

[0063] The motion trajectory representation submodule 22 receives the motion behavior baseline as the target trajectory vector, uses the currently acquired video frame as the original image, and analyzes it using a sparse optical flow algorithm to extract motion features. This invention chooses the sparse optical flow algorithm instead of the dense optical flow algorithm because the former achieves a good balance between computational efficiency and accuracy, making it suitable for running on embedded devices.

[0064] In a preferred embodiment, the parameters of the sparse optical flow algorithm are set as follows: pyramid level 3, window size 15×15 pixels, maximum number of iterations 30, and minimum feature point quality 0.01. These parameter settings ensure that sufficiently accurate motion features are obtained while maintaining computational efficiency.

[0065] The motion trajectory representation submodule 22 uses the difference between the current frame motion vector and the target trajectory vector as the feature difference value. By repeatedly extracting motion features, a difference distribution sequence is obtained, i.e., the motion trajectory. In particular, the motion trajectory representation submodule 22 constructs a differential geometric representation of the trajectory based on the Frenet framework, including calculating the trajectory curvature features and torsion features to form a differential invariant feature vector.

[0066] like Figure 3 As shown, the trajectory differential representation method based on the Frenet framework includes the following steps:

[0067] First, the trajectory point sequence is parameterized. Assume the trajectory point sequence is... ,in This is a time parameter, representing the value on the time axis; , , These are the coordinate components in three-dimensional space, representing the position of the trajectory in three-dimensional space.

[0068] Then, calculate the tangent vector of the trajectory. :

[0069] ,

[0070] in, For the trajectory in time The unit tangent vector at that point; express Regarding time The first derivative of , i.e., the velocity vector of the trajectory; Representing vectors The magnitude is used to normalize the tangent vector; , , They are respectively , , Regarding time The first derivative of represents the velocity components of the trajectory along each coordinate axis.

[0071] Next, calculate the normal vector of the trajectory. :

[0072] ,

[0073] in, For the trajectory in time The unit normal vector at that location; express Regarding time The first derivative; Representing vectors The magnitude is used to normalize the normal vector; express Regarding time The second derivative of the trajectory is the acceleration vector of the trajectory. This represents the dot product operation of vectors. Calculate the projection of the acceleration vector onto the tangent vector direction; This represents the component of the acceleration vector along the tangent vector direction; This represents the component of the acceleration vector in the normal plane.

[0074] Then, calculate the binormal vector B(t) of the trajectory:

[0075] ,

[0076] in, For the trajectory in time The unit binormal vector at that location; This represents the cross product operation of vectors. Calculate the cross product of the tangent vector and the normal vector to obtain a vector perpendicular to both.

[0077] Based on the three orthogonal vectors mentioned above, a Frenet frame, i.e., a local coordinate system, can be constructed for each point on the trajectory.

[0078] Furthermore, the curvature of the trajectory is calculated. :

[0079] ,

[0080] in, For the trajectory in time The curvature at a point indicates the degree to which the trajectory deviates from a straight line; Represents the magnitude of the cross product of the velocity vector and the acceleration vector; Represents the cube of the magnitude of the velocity vector. Curvature It is an important feature of the trajectory shape; the larger the value, the higher the degree of trajectory curvature.

[0081] Finally, the torsion of the trajectory is calculated. :

[0082] ,

[0083] in, For the trajectory in time The torsion at a point indicates the degree to which the trajectory deviates from the plane; express Regarding time The third derivative of the trajectory is the jerk vector. Calculate the cross product of the velocity vector and the acceleration vector, and the dot product of the jerk vector; Represents the square of the magnitude of the cross product of the velocity and acceleration vectors. Torsion. It is an important feature of three-dimensional trajectory, and a value of zero indicates that the trajectory lies in a plane.

[0084] By calculating the curvature and torsion of the trajectory, we can obtain the differential invariant eigenvectors of the trajectory. These features are unaffected by the observation perspective and have good invariance.

[0085] The local feature extraction submodule 23 is used to take the motion trajectory vector of each frame as a sample, perform convolution operation with a convolutional neural network, and finally obtain local anomaly points. Based on the magnitude of the feature difference value, frames with differences greater than the anomaly threshold are marked as anomalies.

[0086] In one embodiment of the present invention, a convolutional neural network is used for temporal anomaly detection. The network structure includes an input layer, a feature layer, a temporal sampling connection layer, a fully connected layer, and an output layer. Specifically, the input layer receives motion trajectory vectors, the feature layer extracts spatial features through convolution operations, the temporal sampling connection layer captures temporal relationships, the fully connected layer comprehensively analyzes the features, and the output layer provides the anomaly probability.

[0087] Preferably, the convolutional layers use 3×3 kernels with a stride of 1 and padding of 1 to maintain the feature map size. The temporal sampling connection layers use Long Short-Term Memory (LSTM) units, and the hidden layer dimension is 128, which can effectively capture temporal dependencies.

[0088] The local feature extraction submodule 23 identifies frames with differences greater than an anomaly threshold as anomalies based on the magnitude of the feature difference values. In a preferred embodiment, the anomaly threshold is set to 0.6, which is an empirical value derived from a large amount of experimental data and can achieve a good balance between reducing the false alarm rate and maintaining detection sensitivity.

[0089] The detection and early warning module 3 is communicatively connected to the motion representation module 2. It is used to receive abnormal features transmitted by the motion representation module 2, perform abnormal motion detection, generate abnormal detection results, and feed back the abnormal detection results and scene information of the current video frame to the behavior model acquisition submodule 21 of the motion representation module 2 for updating the behavior baseline model.

[0090] like Figure 6 As shown, the detection and early warning module 3 takes abnormal motion samples as input and uses a neural network to detect abnormal motion. The loss function of the neural network includes the number of training samples, the anomaly detection label, the network prediction value, and the anomaly detection accuracy index parameters. Specifically, the loss function can be expressed as:

[0091] ,

[0092] in, The loss function value represents the degree of difference between the model's prediction and the true label. The number of training samples represents the number of samples used for training. These are anomaly detection labels (0 for normal, 1 for abnormal), indicating the first... The true category of each sample; Network prediction ( , indicating that the model is for the th The probability that a sample is predicted to be abnormal; This is a parameter for anomaly detection accuracy, used to balance cross-entropy loss and absolute error loss, and to control the relative importance of the two loss terms; It is the natural logarithm function; This represents the absolute error between the predicted value and the true label.

[0093] Preferably, parameters Setting it to 0.3 is the optimal value obtained through extensive experimental comparisons, which can improve the model's generalization ability while maintaining detection accuracy.

[0094] The detection and early warning module 3 feeds back the neural network detection results and scene information of the current video frame to the behavior model acquisition submodule 21 to update the behavior baseline model. This closed-loop feedback mechanism enables the system to continuously learn and adapt to environmental changes, improving detection accuracy.

[0095] In a preferred embodiment of the present invention, such as Figure 4 As shown, the motion characterization module 2 adopts a multi-scale anomaly measurement framework, including local scale analysis units, segment scale analysis units, and global scale analysis units.

[0096] The local scale analysis unit is used to set the sliding window size to 3 frames, calculate the local statistical properties of differential invariants within the window, and detect transient anomalous behavior. Specifically, for the sequence of differential invariants within the window... Calculate its mean ,variance and skewness :

[0097] ,

[0098] ,

[0099] ,

[0100] in, It is the arithmetic mean of the differential invariants within the local window; The variance of the differential invariants within the local window represents the degree of dispersion of the data; The skewness of the differential invariant within a local window represents the asymmetry of the data distribution; Indicates the first The differential invariant value of a frame can be curvature. Torque or combinations thereof; Indicates the period from the first Frame to the The summation operation of frames.

[0101] By analyzing changes in these statistical characteristics, transient abnormal behaviors, such as sudden acceleration or abrupt stop, can be detected.

[0102] The segment-scale analysis unit is used to set the analysis segment length to 9 frames, calculate the statistical distribution of differential invariants within the segment, and detect short-term anomalous behavior. Specifically, for the sequence of differential invariants within the segment... In addition to calculating the mean, variance, and skewness, it also calculates the kurtosis. and autocorrelation coefficient :

[0103] ,

[0104] ,

[0105] in, Kurtosis is the derivative invariant within a segment, representing the degree of peak in the data distribution; Let be the mean of the differential invariants within the segment; The standard deviation of the differential invariant within the segment; Lagging The autocorrelation coefficient of a step represents the time lag between the data sequence and itself. The correlation of steps; This is the lag order, typically ranging from 1 to 3; Indicates the period from the first Frame to the Summation of frames; Indicates the period from the first Frame to the The summation operation of frames.

[0106] These statistical properties can help detect short-term anomalous behavior, such as unnatural turns or pauses.

[0107] The global scale analysis unit is used to analyze complete behavioral sequences, extract global trajectory morphological features, and detect long-term anomalous behavior. Global analysis typically considers trajectory morphology over a period of 30 seconds or longer, extracting overall geometric features of the trajectory, such as the long-term statistical properties of curvature, torsion, and velocity distributions. These features can help detect long-term anomalous behavior, such as abnormal movement paths or prolonged periods of stillness.

[0108] like Figure 5 As shown, in another preferred embodiment of the present invention, the motion representation module 2 further includes a behavior model update unit based on manifold learning, which is used to combine the extracted differential invariants into high-dimensional feature vectors, construct the topological structure of the feature space, and perform manifold learning and model update.

[0109] Specifically, the behavior model update unit based on manifold learning first combines the differential invariants into a high-dimensional feature vector. ,in For feature vectors; The i-th differential invariant feature can be curvature, torsion, or other motion features; The feature dimension represents the length of the feature vector; the superscript... This represents the transpose of a vector, converting a row vector into a column vector.

[0110] Then, define the similarity measurement function between feature vectors:

[0111] ,

[0112] in, For feature vectors and The distance metric between two feature vectors indicates the degree of similarity between them. and These are the i-th and j-th feature vectors, respectively; The weight of the k-th feature is used to balance the importance of different features; This represents the k-th component of the i-th eigenvector; This represents the k-th component of the j-th eigenvector; This represents the square root of the sum of squares of the differences between the 1st and dth eigencomponents, i.e., the calculation of the Euclidean distance.

[0113] Based on similarity metrics, a topological structure for the feature space is constructed, and an empty feature set is initialized as the starting point of the manifold. When a new feature vector is received, its mapping to the manifold is calculated:

[0114] ,

[0115] in, For the new feature vector to manifold The mapping error represents the degree of difference between the new feature and the known behavioral pattern; For the new feature vector; The current manifold model contains a set of known behavioral feature points; Representing a manifold Feature points in ; In manifold Searching for and The feature point with the smallest distance.

[0116] Determine whether a feature belongs to a known behavioral pattern based on the mapping error. If (in If the threshold is high (default is 0.8), then it is considered... Representing potential new behavioral patterns, add them to the manifold model; if (in If the threshold is low (default is 0.2), then it is considered... If the behavior pattern is known, the manifold model will not be updated.

[0117] Furthermore, the behavior model update unit based on manifold learning periodically checks the integrity of the manifold structure, removes outdated or infrequently occurring feature points, merges highly similar feature regions, and simplifies the model. Preferably, the structure check cycle is 100 frames, which is the optimal value based on experimental validation, achieving a balance between maintaining model accuracy and reducing computational overhead.

[0118] The behavior model update unit based on manifold learning also incorporates scene information for feature enhancement, adjusting the anomaly threshold in specific scenarios and considering the impact of scene changes on behavior patterns. This context-aware mechanism enables the system to more accurately identify abnormal behaviors in specific scenarios.

[0119] like Figures 1 to 6 As shown, the detection method of the system of the present invention includes: extracting the motion speed of the current frame in the acquired video frame sequence, and obtaining the motion trajectory of the current frame by combining the motion speed of the current frame with that of the previous N frames; constructing the motion trajectory of the current frame to obtain the original motion trajectory image at the current moment; normalizing the features of the original motion trajectory image and inputting it into a convolutional neural network; performing convolution and pooling operations on the motion trajectory of the current frame to obtain the probability of abnormal motion trajectory distribution of the current frame within the detection area; extracting scene information from the acquired video frames, inputting the video frames into the convolutional neural network to detect the scene, inputting the image features of the current frame into a recurrent neural network to detect anomalies in the current frame, and recording the anomaly detection results; inputting the anomaly detection results and the scene detection results together into the motion model acquisition submodule to update the scene anomaly detection results.

[0120] In one embodiment of the present invention, the system employs an RPI3B+ processor with 3.5W of power and supports up to two USB camera inputs. The system also includes a scene adaptation unit for learning scene characteristics under normal user behavior and generating adaptive feedback to scene changes; a behavior feature extraction unit for directly extracting features from the video and encoding consecutive short-time images captured from each video into feature vectors using an intelligent encoding network; and a behavior model unit for learning the correlation between features based on temporal image information and the temporal image features extracted by the intelligent encoding network, constructing motion patterns, and recording the duration of each action using a recurrent neural network.

[0121] Specifically, to ensure data processing consistency, the system normalizes data from different sources. For image data, pixel values ​​are normalized to the range [0,1]; for motion features, feature values ​​are mapped to the range [-1,1] using a maximum-minimum normalization method.

[0122] .

[0123] in, These are the normalized eigenvalues, ranging from [-1, 1]. These are the original eigenvalues; The minimum value of the feature; The maximum value of the feature; Map the eigenvalues ​​to the range [0,1]; multiply by 2 and then subtract 1 to adjust the range to [-1,1].

[0124] Normalization can unify the scale of data from different sources, thereby improving the stability and accuracy of the system.

[0125] The embedded behavioral anomaly detection system based on temporal image analysis provided by this invention achieves accurate detection of abnormal behavior through the collaborative work of an image acquisition module, a motion representation module, and a detection and early warning module. The system introduces differential geometry theory to accurately model motion trajectories, employs a multi-scale anomaly measurement framework and an adaptive model update mechanism based on manifold learning, and possesses advantages such as viewpoint invariance, high detection sensitivity, and adaptive learning capabilities.

[0126] In practical applications, this system can be deployed on embedded platforms such as the RPI3B+, consuming only 3.5W, and supporting up to two USB camera inputs. It is suitable for various scenarios including security monitoring, smart homes, and medical monitoring. Compared to traditional methods, this system offers significant improvements in detection accuracy, adaptability, and computational efficiency, demonstrating broad application prospects.

[0127] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. An embedded behavioral anomaly detection system based on temporal image analysis, characterized in that, include: The image acquisition module is used to periodically acquire surveillance video frame sequences, preprocess each frame image, convert the RGB color space to the HSV color space, crop the central area of ​​the image, remove the image background, and obtain a color image containing people. The motion representation module is communicatively connected to the image acquisition module. It is used to receive a color image containing a person transmitted by the image acquisition module, extract the motion trajectory, establish a differential geometric framework, generate differential invariant features, construct a behavioral feature manifold, calculate the feature mapping distance, and determine behavioral anomalies. The motion representation module includes a behavior model acquisition submodule, a motion trajectory representation submodule, and a local feature extraction submodule. The detection and early warning module is communicatively connected to the motion representation module. It is used to receive abnormal features transmitted by the motion representation module, perform abnormal motion detection, generate abnormal detection results, and feed back the abnormal detection results and scene information of the current video frame to the behavior model acquisition submodule of the motion representation module for updating the behavior baseline model. The behavior model acquisition submodule is used to receive color images containing people transmitted by the image acquisition module, divide the data sequence into samples of 9 frames each, and initialize an empty sequence as the model behavior baseline; when a data sequence is input, the first data value of the sequence is compared with the average value in the empty sequence. If it is greater than a set threshold, such as 0.8, the current data value is added to the end of the sequence, replacing the average value with the current value. Then the sequence is shifted forward by one position, receiving new data values, comparing them with the average value in the sequence, and repeating the above operation; when all values ​​in the sequence are less than 0.2, processing stops, and the sequence at this time is used as the motion behavior baseline and transmitted to the motion trajectory representation submodule. The motion trajectory representation submodule is used to receive the motion behavior baseline as the target trajectory vector, use the currently acquired video frame as the original image, analyze it using the sparse optical flow algorithm, extract motion features, and use the difference between the current frame motion vector and the target trajectory vector as the feature difference value; repeatedly extract motion features to obtain a difference distribution sequence, i.e., the motion trajectory; wherein, the motion trajectory representation submodule constructs a differential geometric representation of the trajectory based on the Frenet framework, including calculating the trajectory curvature features and torsion features to form a differential invariant feature vector; The local feature extraction submodule is used to treat the motion trajectory vector of each frame as a sample, perform convolution operation using a convolutional neural network, and finally obtain local anomaly points. Based on the magnitude of the feature difference value, frames with differences greater than the anomaly threshold are labeled as anomalies. The convolutional neural network performs temporal anomaly detection, and the network structure includes an input layer, a feature layer, a temporal sampling connection layer, a fully connected layer, and an output layer.

2. The embedded behavior anomaly detection system based on time-series image analysis of claim 1, wherein: The detection and early warning module is configured to take abnormal motion samples as input, perform abnormal motion detection using a neural network, and use a loss function of the neural network to contain a number of training samples, an abnormal detection label, a network prediction value, and an abnormal detection accuracy index parameter; feed the neural network detection result and scene information of a current video frame to the behavior model obtaining submodule to update the behavior baseline model.

3. The embedded behavior anomaly detection system based on time-series image analysis of claim 1, wherein: The motion representation module uses a multi-scale abnormality measurement framework, which includes: a local scale analysis unit configured to set a sliding window size to 3 frames, calculate local statistical properties of differential invariants in the window, and detect instantaneous abnormal behavior; a segment scale analysis unit configured to set an analysis segment length to 9 frames, calculate statistical distribution of differential invariants in the segment, and detect short-time abnormal behavior; a global scale analysis unit configured to analyze a complete behavior sequence, extract global morphological features of a trajectory, and detect long-time abnormal behavior.

4. The embedded behavior anomaly detection system based on time-series image analysis of claim 1, wherein: The motion representation module further includes a behavior model updating unit based on manifold learning, which is configured to: combine the extracted differential invariants into a high-dimensional feature vector; construct a topological structure of the feature space; initialize an empty feature set as a starting point of the manifold; when a new feature vector is received, calculate a mapping relationship between the new feature vector and the manifold; determine whether the feature belongs to a known behavior mode according to a mapping error; update the manifold structure to include a new normal behavior feature.

5. The embedded behavior anomaly detection system based on time-lapse image analysis of claim 4, wherein: The behavior model updating unit based on manifold learning is further configured to: periodically check the integrity of the manifold structure; remove feature points that are outdated or appear infrequently; merge highly similar feature regions to simplify the model; enhance features in combination with scene information; adjust the abnormal threshold in a specific scene; consider the influence of scene changes on behavior modes.

6. The embedded behavior anomaly detection system based on temporal image analysis of claim 1, wherein: The detection method of the system includes: extracting a motion speed of a current frame in a video frame sequence, obtaining a motion trajectory of the current frame from the motion speed of the current frame and N previous frames; constructing the motion trajectory of the current frame to obtain an original motion trajectory image at the current time; normalizing features of the original motion trajectory image and inputting the features to a convolutional neural network; performing convolution and pooling operations on the motion trajectory of the current frame to obtain a motion trajectory abnormality distribution probability of the current frame in a detection region; extracting scene information from the collected video frames, inputting the video frames to a convolutional neural network to detect the scene, inputting features of the current frame image to a recurrent neural network to perform abnormal detection on the current frame, and recording the abnormal detection result; inputting the abnormal detection result and the scene detection result to a motion model obtaining submodule to update the scene abnormal detection result.

7. The embedded behavior anomaly detection system based on time-lapse image analysis of claim 6, wherein: The system uses an RPI3B+ processor and supports a maximum of 2 USB camera inputs with a power of 3.5W; The system further includes: a scene adaptability unit configured to learn scene features under normal user behavior and produce self-adaptive feedback to changes in the scene; a behavior feature extraction unit configured to directly extract features from a video, encode continuous short-time images captured from each video into a feature vector through an intelligent coding network, and The behavior model unit is used for learning the correlation between the features through a recurrent neural network based on the time sequence image information and the time sequence image features extracted by the intelligent coding network, constructing a motion pattern, and recording the duration of each action.

Images