Face image-based abnormal person tracking method and system

Abstract

The invention relates to the technical field of image tracking, in particular to an abnormal person tracking method and system based on a face image, and the method comprises the steps: carrying out the static identity feature extraction and dynamic behavior feature extraction of a stable face feature basic image, and obtaining a static identity feature set and a dynamic behavior feature set; performing space-time alignment processing on the static identity feature set and the dynamic behavior feature set to obtain a space-time aligned feature set; performing feature decoupling processing based on the space-time alignment feature set to obtain independent identity features and independent behavior features; and carrying out abnormal mode identification on the independent behavior characteristics to obtain an abnormal behavior type classification result. According to the invention, cross-scene and cross-time-frame personnel tracking can be realized by combining the static identity features and the abnormal behavior classification result; an abnormal person moving track is drawn according to the timestamp and the space coordinate of the abnormal behavior, track coherence is improved based on continuity scores, and tracking interruption caused by view angle switching or time intervals is avoided.

Description

Technical Field

[0001] This invention relates to the field of image tracking technology, specifically to a method and system for tracking abnormal individuals based on facial images. Background Technology

[0002] Currently, traditional methods typically rely solely on static features (such as single face recognition) or dynamic behavioral features (such as simple action recognition), lacking multi-level comprehensive feature extraction. This single feature extraction approach is easily affected by complex environments or abnormal behaviors, failing to accurately distinguish between normal and abnormal behaviors, resulting in low recognition rates, especially in dynamic scenarios. Furthermore, traditional methods usually lack effective mechanisms to decouple identity features from behavioral features, which are often intertwined, leading to a decrease in the accuracy of the recognition system. For example, in complex scenarios, interference between identity features and behavioral features may cause tracking errors, affecting the stability and reliability of the system.

[0003] Furthermore, when facing people tracking across camera viewpoints or time frames, traditional methods typically rely on simple image matching or position tracking techniques, which are easily affected by issues such as viewpoint switching, time intervals, and occlusion, leading to tracking interruptions or errors. In addition, when identifying abnormal behavior, traditional methods often use simple behavioral pattern analysis or threshold judgment, which makes the system susceptible to noise or changes and difficult to accurately identify complex abnormal behaviors, such as aggressive tendencies or abnormal loitering. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: a method for tracking abnormal persons based on facial images, comprising: Acquire multi-source face images, perform dynamic environment adaptation processing on the multi-source face images, and obtain stable basic images of face features; Static identity feature extraction and dynamic behavior feature extraction are performed on the stable facial feature base image to obtain a static identity feature set and a dynamic behavior feature set. The static identity feature set includes facial texture detail features and contour structure features, and the dynamic behavior feature set includes expression change time sequence features and head movement trajectory features. The static identity feature set and the dynamic behavior feature set are spatiotemporally aligned to obtain a spatiotemporally aligned feature set. Based on the spatiotemporal aligned feature set, feature decoupling processing is performed to obtain independent identity features and independent behavioral features; wherein, the feature decoupling processing is used to eliminate the cross-mixing of identity information and behavioral information; Anomaly pattern recognition is performed on the independent behavioral features to obtain anomaly behavior type classification results; Based on the independent identity features and the abnormal behavior type classification results, the target personnel are tracked across frames and scenes to obtain the abnormal personnel tracking results.

[0005] Preferably, static identity feature extraction and dynamic behavior feature extraction are performed on the stable facial feature base image to obtain a static identity feature set and a dynamic behavior feature set, including: The stable facial feature base image is input into the input layer of the multi-task feature extraction network. The image features are divided into an identity feature branch and a behavior feature branch by the dual-branch routing module of the input layer. The identity feature branch is used to extract static identity features, and the behavior feature branch is used to extract dynamic behavior features. In the identity feature branch, the stable facial feature base image is subjected to multi-scale texture enhancement processing to obtain a texture feature map; the texture feature map is subjected to contour boundary enhancement processing to extract the geometric structure features of the facial contour, the gray-level co-occurrence matrix of key region pixels is calculated as texture detail features, and the geometric structure features and the texture detail features are fused to obtain a static identity feature set; In the behavioral feature branch, continuous frame difference operation is performed on the stable facial feature base image to obtain a dynamic difference image sequence; Optical flow field calculation is performed on the dynamic difference image sequence, pixel motion vectors are extracted as head motion trajectory features, and the temporal change rate of pixels in the expression region is calculated as expression change temporal features. The head motion trajectory features and the expression change temporal features are fused to obtain a dynamic behavior feature set.

[0006] Preferably, the stable facial feature base image undergoes multi-scale texture enhancement processing to obtain a texture feature map; the texture feature map undergoes contour boundary enhancement processing to extract the geometric structure features of the facial contour, the gray-level co-occurrence matrix of key region pixels is calculated as texture detail features, and the geometric structure features and the texture detail features are fused to obtain a static identity feature set, including: In the identity feature branch, a multi-scale convolution kernel set is initialized, wherein the multi-scale convolution kernel set includes Gaussian kernels and Laplacian kernels of different sizes; Based on each convolution kernel in the multi-scale convolution kernel set, the stable face feature base image is convolved to obtain a texture feature map, wherein all texture feature maps together constitute a multi-scale texture feature map set. The multi-scale texture feature map set is subjected to feature fusion processing to extract semantic texture information at different levels, resulting in a fused texture feature map. Edge detection processing is performed on the fused texture feature map to identify facial contour boundaries, and the curvature change parameters and boundary length parameters of the contour lines are extracted as geometric structural features. Calculate the gray-level co-occurrence matrix of pixels within the region enclosed by the contour line, extract contrast, correlation, energy, and homogeneity as texture detail features, and concatenate the geometric structure features and texture detail features by feature dimension to obtain a static identity feature set.

[0007] Preferably, optical flow field calculation is performed on the dynamic difference image sequence to extract pixel motion vectors as head motion trajectory features, and the temporal change rate of pixels in the expression region is calculated as expression change temporal features. The head motion trajectory features and the expression change temporal features are fused to obtain a dynamic behavior feature set, including: In the behavioral feature branch, the dynamic difference image sequence is Gaussian smoothed to suppress noise interference, resulting in a smoothed dynamic difference image sequence. The Lucas-Kanade optical flow algorithm is used to calculate the optical flow field of the smoothed dynamic difference image sequence to obtain the motion vector of each pixel in consecutive frames. The mean value of the motion vector and the rate of change of direction of the pixels in the head region are extracted as the head motion trajectory features. In the stable facial feature base image, the key expression regions are located, including the eyes, eyebrows and mouth. The pixel value changes of the key expression regions in the smoothed dynamic difference image sequence are extracted, and the average rate of pixel value change per unit time is calculated as the temporal feature of expression change. The head movement trajectory features and the temporal features of facial expression changes are divided into time windows to obtain behavioral feature segments. The behavioral feature segments are then spliced ​​together in chronological order to obtain a dynamic behavioral feature set.

[0008] Preferably, the static identity feature set and the dynamic behavior feature set are subjected to spatiotemporal alignment processing to obtain a spatiotemporally aligned feature set, including: The static identity feature set and the dynamic behavior feature set are input into the alignment calibration module of the multi-task feature extraction network; through timestamp synchronization and spatial coordinate mapping processing, a spatiotemporal alignment feature set is obtained.

[0009] Preferably, feature decoupling processing is performed based on the spatiotemporal aligned feature set to obtain independent identity features and independent behavioral features, including: Construct a cross-modal feature correlation matrix, wherein the feature correlation matrix is ​​used to represent the correlation strength between each feature component in the static identity feature set and the dynamic behavior feature set; The cross-modal feature correlation matrix is ​​subjected to singular value decomposition to obtain feature vectors and singular values. Strongly correlated feature pairs are then selected based on the magnitude of the singular values. Based on the strongly correlated feature pairs, a decoupled projection matrix is ​​constructed, and the static identity feature set and the dynamic behavior feature set are projected onto the identity feature subspace and the behavior feature subspace, respectively. In the identity feature subspace, residual behavioral feature components are removed by low-pass filtering, and residual identity feature components are removed by high-pass filtering in the behavioral feature subspace, resulting in initially decoupled independent identity features and independent behavioral features. Calculate the residual correlation between the independent identity features and the dynamic behavior feature set after initial decoupling. If the residual correlation is higher than a preset threshold, adjust the decoupling projection matrix and re-execute the projection and decoupling operations until the residual correlation is lower than the preset threshold. The independent identity features and independent behavioral features after final decoupling are normalized to ensure that the two types of features are within the same numerical range.

[0010] Preferably, abnormal pattern recognition is performed on the independent behavioral features to obtain abnormal behavior type classification results, including: The independent behavioral features are input into an abnormal behavior classifier, wherein the classifier includes an LSTM recurrent network and a temporal convolutional network; The temporal dependencies of behavioral features are extracted through the LSTM recurrent network, and the local mutation patterns of behavioral features are captured through the temporal convolutional network. The outputs of the LSTM network and the temporal convolutional network are fused, and the resulting data is input into a fully connected layer for classification. The outputs include the abnormal behavior type and its corresponding confidence level. The abnormal behavior type includes aggression, abnormal loitering, item theft, and camouflage behavior. The confidence level represents the reliability of the classification result. The abnormal behavior type classification results are post-processed to remove classification results with confidence levels below a preset value and retain abnormal behavior types with high confidence levels, thus obtaining the abnormal behavior type classification results.

[0011] Preferably, based on the independent identity features and the abnormal behavior type classification results, the target person is tracked across frames and scenes to obtain abnormal person tracking results, including: Based on the independent identity features, a personnel identity identifier is constructed, and an identity association relationship is established under different time frames and different camera scenarios; Based on the timestamps and spatial coordinates of the abnormal behavior type classification results, the movement trajectory of the abnormal person is drawn, wherein the trajectory includes the location information and time information of key nodes; Calculate the continuity score of the tracking trajectory, wherein the score reflects the coherence of the trajectory in time and space; associate and store the abnormal behavior type classification result, the tracking trajectory, and the continuity score to obtain the abnormal person tracking result, wherein the tracking result includes the abnormal person's identity identifier, main abnormal type, complete movement trajectory, trajectory continuity score, and highest confidence classification result.

[0012] An abnormal person tracking system based on facial images, applicable to the aforementioned abnormal person tracking method based on facial images, includes: An image acquisition unit is used to acquire multi-source face images and perform dynamic environmental adaptation processing on the multi-source face images to obtain a stable basic image of face features. The feature extraction unit is used to extract static identity features and dynamic behavior features from the stable facial feature base image to obtain a static identity feature set and a dynamic behavior feature set. The static identity feature set includes facial texture detail features and contour structure features, and the dynamic behavior feature set includes expression change time sequence features and head movement trajectory features. The feature processing unit is used to perform spatiotemporal alignment processing on the static identity feature set and the dynamic behavior feature set to obtain a spatiotemporally aligned feature set. The feature decoupling unit is used to perform feature decoupling processing based on the spatiotemporal aligned feature set to obtain independent identity features and independent behavioral features; wherein, the feature decoupling processing is used to eliminate the cross-mixing of identity information and behavioral information; An anomaly identification unit is used to perform anomaly pattern recognition on the independent behavioral features to obtain anomaly behavior type classification results; The result tracking unit is used to perform cross-frame and cross-scene tracking of target personnel based on the independent identity features and the abnormal behavior type classification results, and obtain abnormal personnel tracking results.

[0013] Compared with the prior art, the beneficial effects of the present invention are: This invention extracts static identity features and dynamic behavior features, and combines them with a multi-task feature extraction network. This method can more accurately distinguish between abnormal and normal behaviors from complex images. The extraction of comprehensive features enables the recognition system to cope with more changing environments and various types of abnormal behaviors. This invention effectively separates static identity features and dynamic behavioral features through feature decoupling, reducing the overlap and mixing between identity and behavioral information. By independently analyzing identity and behavioral features, it avoids tracking errors caused by information interference, thereby improving the stability and reliability of the tracking system. This invention combines static identity features with abnormal behavior classification results to achieve cross-scene and cross-time frame personnel tracking; the system can draw the movement trajectory of abnormal personnel based on the timestamp and spatial coordinates of abnormal behavior, and improve the continuity of the trajectory based on the continuity score to avoid tracking interruption caused by perspective switching or time intervals. This invention combines LSTM recurrent networks and temporal convolutional networks to deeply mine the temporal dependencies and local mutation patterns of behavioral features, thereby accurately identifying various abnormal behaviors such as attack tendencies, abnormal loitering, and item theft. The high-confidence output of the classification helps to improve the accuracy of the final decision and reduce the false alarm rate. Attached Figure Description

[0014] Figure 1 This is a schematic flowchart of the overall method in one embodiment of the present invention; Figure 2 This is a schematic diagram of the overall system architecture in one embodiment of the present invention.

[0015] In the diagram: 1. Image acquisition unit; 2. Feature extraction unit; 3. Feature processing unit; 4. Feature decoupling unit; 5. Anomaly recognition unit; 6. Result tracking unit. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Example 1, please refer to Figure 1 This invention provides a technical solution: a method for tracking abnormal persons based on facial images, comprising: S1. Acquire multi-source face images, perform dynamic environment adaptation processing on the multi-source face images, and obtain stable basic images of face features; S2. Static identity feature extraction and dynamic behavior feature extraction are performed on the stable facial feature base image to obtain a static identity feature set and a dynamic behavior feature set. The static identity feature set includes facial texture detail features and contour structure features, while the dynamic behavior feature set includes expression change time sequence features and head movement trajectory features. S3. Perform spatiotemporal alignment processing on the static identity feature set and the dynamic behavior feature set to obtain the spatiotemporal aligned feature set; S4. Based on the spatiotemporal aligned feature set, feature decoupling processing is performed to obtain independent identity features and independent behavioral features; among them, feature decoupling processing is used to eliminate the cross-mixing of identity information and behavioral information; S5. Perform abnormal pattern recognition on independent behavioral features to obtain abnormal behavior type classification results; S6. Based on the classification results of independent identity features and abnormal behavior types, target personnel are tracked across frames and scenes to obtain abnormal personnel tracking results.

[0018] It should be noted that the system needs to acquire facial images from different sources (e.g., different cameras, different environments, etc.). Since environmental changes can affect image quality (e.g., lighting, angle), these images need dynamic environmental adaptation processing to eliminate the impact of environmental changes on image quality, thereby obtaining a stable base image of facial features. This results in clear and consistent facial features within the image. After obtaining the stable facial image, the system will perform two different feature extractions: static identity features are used to describe facial features, such as facial texture details (e.g., skin texture, wrinkles, etc.) and facial contour structure, such as face shape, the relative positions of the eyes, nose, and mouth; these features... Static identity features represent a person's unique identity; dynamic behavioral features are related to a person's actions and facial expressions, including: temporal features of facial expressions, representing changes in facial expressions over time, such as the process of smiling, frowning, etc.; and head movement trajectory features, representing the trajectory of head movements in space, such as the path of turning the head, nodding, etc. Since static identity features and dynamic behavioral features usually come from different image regions and time points, they need to be spatiotemporally aligned. This means processing static and dynamic features uniformly according to time and space to ensure their correlation and consistency. For example, facial expressions and head movements must be processed synchronously with static identity features for subsequent analysis. The purpose of feature decoupling is to separate static identity features from dynamic behavioral features. In reality, static identity information (such as facial features) and dynamic behavioral information (such as facial expressions and head movements) may overlap and become mixed, affecting recognition accuracy. Through feature decoupling, the system can effectively extract independent identity features and independent behavioral features, eliminating interference between them and thus improving recognition accuracy. Analyzing independent behavioral features helps identify abnormal patterns. This typically involves detecting actions or changes that do not conform to normal behavioral patterns. For example, a person's behavioral pattern may suddenly change compared to other times, or they may exhibit unusual behavior, such as abnormal head movement trajectories or facial expression changes. These abnormal patterns may represent potential risks or abnormal behavior. By classifying the patterns of abnormal behavior, the system can identify the type of abnormal behavior, such as whether it is violent, an escape attempt, or other suspicious behavior. Finally, the system will use independent identity features and abnormal behavior type classification results to perform cross-frame (continuous images in time) and cross-scene (different camera perspectives) person tracking. With this information, the system can continuously track target persons, whether they appear in the same scene or in different scenes. The ultimate goal is to obtain abnormal person tracking results, that is, the system can identify specific persons and continuously monitor their activities when they exhibit abnormal behavior, and make corresponding security responses or alerts.

[0019] In an optional embodiment, static identity feature extraction and dynamic behavior feature extraction are performed on the stable facial feature base image to obtain a static identity feature set and a dynamic behavior feature set, including: A stable facial feature base image is input into the input layer of a multi-task feature extraction network. The image features are divided into an identity feature branch and a behavior feature branch by the dual-branch routing module of the input layer. The identity feature branch is used to extract static identity features, and the behavior feature branch is used to extract dynamic behavior features. In the identity feature branch, multi-scale texture enhancement processing is performed on the stable facial feature base image to obtain a texture feature map; contour boundary enhancement processing is performed on the texture feature map to extract the geometric structure features of the facial contour, and the gray-level co-occurrence matrix of the key region pixels is calculated as the texture detail features. The geometric structure features and texture detail features are fused to obtain a static identity feature set. In the behavioral feature branch, continuous frame difference operation is performed on the stable facial feature base image to obtain a dynamic difference image sequence. Optical flow field calculation is performed on the dynamic difference image sequence, and pixel motion vectors are extracted as head motion trajectory features. The temporal change rate of pixels in the expression region is calculated as the expression change temporal feature. The head motion trajectory feature and the expression change temporal feature are fused to obtain a dynamic behavior feature set.

[0020] It should be noted that a stable base image of facial features is input into the input layer of a multi-task feature extraction network. This network has two branches: an identity feature branch for extracting static identity features and a behavior feature branch for extracting dynamic behavior features. The network uses this dual-branch routing module to divide the input image features into two different branches for extracting static identity and dynamic behavior features respectively. In the identity feature branch, the goal is to extract features related to facial identity, especially facial texture details and geometric structures. The stable base image of facial features undergoes texture enhancement processing at different scales. By enhancing the texture information of the image at different scales, the system can better capture subtle facial texture features. For example, by magnifying local areas of the image, details such as skin texture and wrinkles can be better identified. Through texture enhancement, a texture feature map is obtained, which is an image containing rich texture information to help identify detailed facial features. The texture feature map undergoes contour boundary enhancement processing, which aims to highlight the contour structure features of the face, especially the boundaries of the face, such as the shapes of the eyes, mouth, and nose. After strengthening the contour boundaries, the geometric structural features of the facial contours are further extracted, including the spatial relationships between key facial features, such as the positions of the eyes and nose, and the facial contour lines. The gray-level co-occurrence matrix (GLCM) of key region pixels is calculated; this matrix describes the spatial characteristics of pixel gray-level changes in an image and is a commonly used texture feature extraction method. Through the GLCM, texture detail features are extracted to further enrich the facial detail information. Finally, the geometric structural features and texture detail features are fused to obtain a static identity feature set. These features can uniquely represent a person's identity, aiding in subsequent identity recognition. The behavioral feature branch aims to extract dynamic behavioral features, particularly changes in facial expressions and head movement trajectories. In the behavioral feature branch, firstly, continuous frame difference calculations are performed on the stable facial feature base image. This means the system compares each frame in the video sequence, calculating the difference between two frames to obtain a dynamic difference image sequence. These difference images show changes in facial expressions or postures. Optical flow field calculations are then performed on the dynamic difference image sequence. Optical flow is a technique for describing the motion of objects in an image; it can extract the motion vector of pixels by calculating the displacement of pixels over time, which can be used to describe the head movement trajectory. The system extracts head motion trajectory features, such as head rotation and nodding, through optical flow field analysis. These features reflect the head's motion patterns in space. Temporal change rate is calculated for pixels in the expression region. This temporal change rate describes the speed and amplitude of facial expression changes over time, reflecting the process of expression change, such as the amplitude of a smile or the speed of a frown. Finally, the head motion trajectory features and the temporal features of facial expression changes are fused to obtain a dynamic behavior feature set. These features reflect a person's behavioral performance, such as changes in facial expression and head movement.

[0021] In an optional embodiment, a multi-scale texture enhancement process is performed on a stable facial feature base image to obtain a texture feature map; the texture feature map is then subjected to contour boundary enhancement processing to extract the geometric structural features of the facial contour; the gray-level co-occurrence matrix of key region pixels is calculated as texture detail features; and the geometric structural features and texture detail features are fused to obtain a static identity feature set, including: In the identity feature branch, a multi-scale convolution kernel set is initialized, which includes Gaussian kernels and Laplacian kernels of different sizes; Based on each convolution kernel in the multi-scale convolution kernel set, convolution operation is performed on the stable facial feature base image to obtain texture feature maps. All texture feature maps together construct a multi-scale texture feature map set. Feature fusion processing is performed on the multi-scale texture feature map set to extract semantic texture information at different levels and obtain the fused texture feature map; Edge detection processing is performed on the fused texture feature map to identify facial contour boundaries, and the curvature change parameters and boundary length parameters of the contour lines are extracted as geometric structural features. Calculate the gray-level co-occurrence matrix of pixels within the region enclosed by the contour line, extract contrast, correlation, energy, and homogeneity as texture detail features, and concatenate the geometric structure features and texture detail features according to feature dimensions to obtain a static identity feature set.

[0022] It's important to note that the multi-scale convolutional kernel set contains convolutional kernels of different sizes, primarily of two types: Gaussian kernels, used to smooth images, capture low-frequency information, reduce noise interference, and highlight smooth facial textures; and Laplacian kernels, a high-pass filter used to detect edges and details in images, highlighting high-frequency information such as contours and details. These different sized kernels help extract features from images at different scales, capturing changes in detail and structure. Convolution operations are performed on stable facial feature base images based on Gaussian and Laplacian kernels. Convolution operations can extract local feature information from images, such as texture, edges, and smooth regions. Gaussian kernel convolution primarily highlights smooth areas and overall facial structure, while Laplacian kernel convolution highlights facial details and edges. Texture feature maps are the output images obtained after convolution operations. These images describe the texture information of different regions in the image, including changes in detail and structure. All texture feature maps together constitute a multi-scale texture feature map set. This set contains texture features at different scales, providing rich facial information. Feature fusion processing is performed on a multi-scale texture feature map set, merging texture features from different convolutional kernels and scales. The aim of this process is to extract more comprehensive facial texture information from multiple levels and scales, including both coarse and fine texture details. By fusing these features, the system can acquire multi-level, deep semantic texture information of the face. The fused texture feature map is the result of feature fusion, containing both global and local texture features of the face, providing rich information for subsequent face recognition. Edge detection processing is then performed on the fused texture feature map. Edge detection is a technique used to extract object boundaries in an image by detecting the boundaries of objects in the image. Regions with significant grayscale variations are used to identify facial contours. Facial contour boundaries are the main geometric structures of the face, such as the boundaries of the eyes, nose, and mouth, providing important information about facial shape and structure. After edge detection, curvature variation parameters and boundary length parameters of the contour lines are extracted: Curvature variation parameters describe the curve changes of the facial contour, such as the curvature of the eyes, bridge of the nose, and chin; this information helps the system better understand the geometry of the face; Boundary length parameters describe the boundary length of the facial contour, which helps analyze the overall shape of the face. These geometric features provide shape and structural information about the facial contour, helping to further distinguish different facial features. Within the region enclosed by the contour line, the gray-level co-occurrence matrix (GLCM) of the pixels is calculated. The GLCM is a statistical method describing the spatial relationship of pixel gray-level changes in an image, enabling the extraction of texture details. Contrast describes the degree of pixel value variation in an image, reflecting the coarseness of the texture. Correlation describes the degree of texture repetition, reflecting the relationship between pixels in the image. Energy reflects the uniformity of the texture; higher energy indicates a smoother texture. Homogeneity describes the consistency of the texture, characterizing whether the image has a relatively uniform texture. By calculating these texture detail features, the system can further extract detailed information from the image, such as skin roughness and texture consistency. Geometric structural features (such as curvature changes and boundary lengths) are concatenated with texture detail features (such as contrast and correlation) in terms of feature dimensions. This concatenation fuses different types of features into a high-dimensional feature vector. Finally, a static identity feature set is obtained, which includes facial texture details, geometric shapes, and other static features. This feature set is the foundation for identity recognition, helping the system uniquely identify a person's facial features.

[0023] In an optional embodiment, optical flow field calculation is performed on the dynamic difference image sequence, pixel motion vectors are extracted as head motion trajectory features, and the temporal change rate of pixels in the expression region is calculated as expression change temporal features. The head motion trajectory features and expression change temporal features are fused to obtain a dynamic behavior feature set, including: In the behavioral feature branch, Gaussian smoothing is applied to the dynamic difference image sequence to suppress noise interference, resulting in a smoothed dynamic difference image sequence. The Lucas-Kanade optical flow algorithm was used to calculate the optical flow field of the smoothed dynamic difference image sequence to obtain the motion vector of each pixel between consecutive frames. The mean value and direction change rate of the motion vector of the pixels in the head region were extracted as the head motion trajectory features. In a stable facial feature base image, key facial expression regions are located, including eyes, eyebrows and mouth. The pixel value changes of key facial expression regions in a smoothed dynamic difference image sequence are extracted, and the average rate of pixel value change per unit time is calculated as the temporal feature of facial expression change. The head movement trajectory features and facial expression change temporal features are divided into time windows to obtain behavioral feature segments. These behavioral feature segments are then spliced ​​together in chronological order to obtain a dynamic behavioral feature set.

[0024] It should be noted that dynamic difference image sequences may contain noise, such as camera shake or other interference during the shooting process, requiring Gaussian smoothing. Gaussian smoothing reduces noise in the image by using a Gaussian kernel convolution, preserving important dynamic information and making subsequent analysis more stable. After smoothing, the noise in the dynamic difference image sequence is suppressed, and the image is smoother, which helps to extract facial and head motion features more accurately. The Lucas-Kanade optical flow algorithm is used to calculate the smoothed dynamic difference image sequence. The optical flow algorithm is used to estimate pixel motion between consecutive image frames, that is, the displacement of each pixel in time. The optical flow field describes the motion vector of each pixel in the image between consecutive frames, that is, the displacement direction and velocity of the pixel. These motion vectors can help understand the dynamic motion of the face and head. Specifically, the motion vector of the head region can be used to extract the motion trajectory of the head for analyzing the overall motion pattern of the head. From the calculated optical flow field, the mean and rate of change of direction of the motion vector of the head region are extracted as the motion trajectory features of the head. These features can reflect the head's movement speed, direction change, and motion trend. Key facial expression regions include the eyes, eyebrows, and mouth, which are the main areas of facial expression change. Locating these key regions is crucial for understanding facial expression changes. In a stable facial feature base image, these key expression regions are first located, and their positions in the smoothed dynamic difference image sequence are obtained. Next, the pixel value changes of these key regions are extracted, i.e., the grayscale changes of these regions at different time points (e.g., whether the eyes are wide open, whether the mouth is open, etc.). These changes reflect the dynamic changes of facial expressions. The average rate of pixel value change per unit time is calculated, i.e., the speed of expression change; this indicator is the temporal feature of expression change, reflecting the frequency and intensity of facial expression changes. For the extracted head motion trajectory features and temporal features of expression change, time windows are divided. A time window divides the feature sequence into small segments according to time, each segment representing an expression change. Behavioral feature segments are used to capture the dynamic changes in behavioral characteristics over time. For example, the trajectory of head rotation may change rapidly within a short period, while facial expression changes may have a certain delay. Dividing the time window helps the system analyze these dynamic changes. Each behavioral feature segment is a segment of features obtained based on the time window division, which can be a specific time period of head movement or facial expression change. Each behavioral feature segment describes the specific dynamic behavior of the face and head within that time period. By splicing all the behavioral feature segments in chronological order, a complete set of dynamic behavioral features is obtained. This feature set contains the temporal information of head movement trajectory and facial expression changes. The resulting set of dynamic behavioral features can completely describe a person's dynamic behavior over different time periods, including head movement patterns and changes in facial expressions. These features are very helpful in understanding an individual's behavioral patterns and emotional expression.

[0025] In an optional embodiment, a spatiotemporal alignment process is performed on the static identity feature set and the dynamic behavior feature set to obtain a spatiotemporally aligned feature set, including: The static identity feature set and dynamic behavior feature set are input into the alignment calibration module of the multi-task feature extraction network; through timestamp synchronization and spatial coordinate mapping, the spatiotemporal alignment feature set is obtained.

[0026] It should be noted that the static identity feature set and the dynamic behavior feature set represent an individual's identity information and behavioral information, respectively. However, they are different types of features, capturing static (identity) and dynamic (behavioral) information. To process both types of information simultaneously, a multi-task learning framework is needed to extract and fuse these features. A multi-task feature extraction network refers to a neural network model that can handle multiple tasks simultaneously, typically used to jointly train multiple related tasks to improve the model's overall performance. These multiple tasks include extracting static identity features and dynamic behavior features. The alignment and calibration module aligns and calibrates features from different times or spaces, ensuring that static identity features and dynamic behavior features can be consistently fused across the spatiotemporal dimensions. This module's role is to "calibrate" these features from different sources so that they can... Effective fusion is achieved within the same spatiotemporal framework. Timestamp synchronization refers to the temporal alignment of static identity features and dynamic behavioral features. In multi-task learning, static identity features are typically extracted at a specific moment, while dynamic behavioral features are extracted over a period of time (e.g., temporal features of facial expression changes). To enable these two types of features to be meaningfully combined in time, they need to be synchronized to ensure that the features they represent correspond to the same point in time or time period. This is usually achieved through timestamps. Each feature point has a timestamp representing the specific moment it was extracted. During the alignment process, it is necessary to ensure that the timestamps of static features and dynamic features are aligned. For example, if dynamic behavioral features are extracted within a certain time period, then the static features extracted within that time period need to correspond one-to-one with these dynamic features. Spatial coordinate mapping refers to mapping and aligning the spatial positions of features from different sources (such as static and dynamic features). Static identity features (such as facial features) typically have fixed positions in two-dimensional or three-dimensional space, while dynamic behavioral features may shift in space due to head movements or changes in facial expressions. Spatial coordinate mapping ensures that these features are compared and fused within the same spatial framework. For example, in facial recognition, static identity features (such as facial feature points) and dynamic behavioral features (such as eye movement trajectories) need to be mapped to the same spatial coordinate system to ensure effective alignment within the same facial image or model. Spatiotemporally aligned feature sets refer to feature sets that are aligned in both time and space after the aforementioned timestamp synchronization and spatial coordinate mapping processes. Specifically, it includes a combination of static identity features and dynamic behavioral features, where static identity features typically represent an individual's identity information, while dynamic behavioral features represent changes in an individual's behavior at different times. Spatiotemporally aligned feature sets allow these features to be processed within a unified spatiotemporal framework, enabling the combination of static and dynamic features to better provide information for subsequent tasks (such as behavior analysis and emotion recognition).

[0027] In an optional embodiment, feature decoupling processing is performed based on a spatiotemporally aligned feature set to obtain independent identity features and independent behavioral features, including: Construct a cross-modal feature correlation matrix, whereby the feature correlation matrix is ​​used to represent the correlation strength between each feature component in the static identity feature set and the dynamic behavior feature set; Singular value decomposition is performed on the cross-modal feature correlation matrix to obtain feature vectors and singular values. Strongly correlated feature pairs are then selected based on the magnitude of the singular values. Based on strongly correlated feature pairs, a decoupled projection matrix is ​​constructed to project the static identity feature set and the dynamic behavior feature set onto the identity feature subspace and the behavior feature subspace, respectively. In the identity feature subspace, residual behavioral feature components are removed by low-pass filtering, and residual identity feature components are removed by high-pass filtering in the behavioral feature subspace, resulting in initially decoupled independent identity features and independent behavioral features. Calculate the residual correlation between the independent identity features and the dynamic behavior feature set after initial decoupling. If the residual correlation is higher than a preset threshold, adjust the decoupling projection matrix and re-execute the projection and decoupling operations until the residual correlation is lower than the preset threshold. The independent identity features and independent behavioral features after final decoupling are normalized to ensure that the two types of features are within the same numerical range.

[0028] It should be noted that the feature correlation matrix is ​​a two-dimensional matrix used to represent the correlation between static and dynamic features. For example, the (i,j)th element of the matrix represents the correlation between the i-th static identity feature and the j-th dynamic behavior feature. This correlation is typically obtained by calculating methods such as the Pearson correlation coefficient. Singular value decomposition (SVD) is a common linear algebra method for decomposing matrices. Strongly correlated feature pairs are selected based on the magnitude of the singular values. Large singular values ​​indicate strong correlations, while small singular values ​​indicate weak correlations; these small-valued feature pairs can usually be ignored. The decoupling projection matrix separates static and dynamic features, placing them into their respective subspaces, i.e., the subspaces of static identity features and dynamic behavior features. The process involves several steps: projecting static identity feature sets into an identity feature subspace; projecting dynamic behavior feature sets into a behavior feature subspace; constructing the decoupled projection matrix based on strongly correlated feature pairs obtained from SVD; ensuring that the projected feature subspaces are independent to allow for separate processing of static and dynamic information; and identifying residual feature information that may be incorrectly projected into the wrong subspace. In the identity feature subspace, low-pass filtering removes residual dynamic behavior features, filtering out high-frequency dynamic information and retaining only slowly changing identity features. In the behavior feature subspace, high-pass filtering removes residual identity feature components, filtering out low-frequency identity information and retaining only rapidly changing behavior features. After filtering, relatively independent identity and behavioral features are obtained, reducing mutual interference between them. Residual correlation refers to the correlation that still exists between the decoupled identity features and dynamic behavioral features. If the residual correlation is still high, that is, there is still a strong connection between the identity features and behavioral features, the decoupling projection matrix needs to be adjusted and the projection and filtering operations need to be performed again. A threshold is set. If the residual correlation is higher than the threshold, the decoupling is considered incomplete and needs to be optimized. By continuously adjusting the projection matrix, the residual correlation is reduced to below the preset threshold, thus achieving the ideal decoupling effect. Normalization is to map the decoupled identity and behavioral features to the same numerical range, which ensures that they are comparable in subsequent applications (such as machine learning, feature fusion, etc.). Usually, the feature values ​​are scaled to the range of 0 to 1, or standardized to a mean of 0 and a variance of 1.

[0029] In an optional embodiment, abnormal pattern recognition is performed on independent behavioral features to obtain abnormal behavior type classification results, including: Independent behavioral features are input into an abnormal behavior classifier, which includes an LSTM recurrent network and a temporal convolutional network. Temporal dependencies of behavioral features are extracted using an LSTM recurrent network, and local mutation patterns of behavioral features are captured using a temporal convolutional network. The outputs of the LSTM network and the temporal convolutional network are fused together and input into the fully connected layer for classification. The output is the type of abnormal behavior and the corresponding confidence score. The abnormal behavior types include aggression, abnormal loitering, item theft and disguise behavior. The confidence score represents the reliability of the classification result. The abnormal behavior type classification results are post-processed to remove classification results with confidence levels below the preset value and retain abnormal behavior types with high confidence levels, thus obtaining the abnormal behavior type classification results.

[0030] It should be noted that the independent behavioral features are obtained from the previous decoupling process, having removed the interference of static identity features and retained pure behavioral information. The classifier's task is to determine whether abnormal behavior exists based on the input behavioral features and to identify the type of abnormal behavior, such as aggressive tendencies or abnormal loitering. This classifier consists of two main parts: an LSTM recurrent neural network and a temporal convolutional network (TCN). LSTM is a special type of recurrent neural network (RNN) particularly suitable for processing time-series data. It can learn the temporal dependencies in the data, i.e., how to associate previous behavioral features with current behavioral features, capturing patterns of behavioral changes. For example, if someone exhibits a certain behavioral trend over a period of time, such as suddenly accelerating... LSTM can learn the temporal relationships of gait; in this scenario, LSTM's role is to identify dynamic patterns over time, such as determining whether there is a continuous abnormal behavior over a period of time, such as sustained attack actions or loitering. TCN is another network architecture specifically designed to process temporal data, and its main advantage is its ability to effectively capture local abrupt change patterns. It processes temporal data through convolutional layers to extract information about local behavioral changes. TCN can detect abrupt changes or shifts in the short term, such as a person suddenly changing from walking to stopping, or suddenly accelerating. Such abrupt changes are often markers of certain abnormal behaviors, such as rapid action during theft or the sudden occurrence of an attack. By capturing these local abrupt change patterns, TCN can effectively identify sudden changes in behavior. The outputs of LSTM and TCN are fused, merging their outputs into a unified feature representation. This can be achieved by concatenating their outputs, weighted averaging, or other fusion methods. LSTM captures global temporal dependencies, while TCN detects local behavioral abrupt changes. Feature fusion yields a more comprehensive representation of behavioral features. The fully connected layer, a standard layer in neural networks, integrates and weights the input features for classification decisions. In this step, the behavioral features processed by LSTM and TCN are fed into the fully connected layer for final classification. The output of the fully connected layer is the abnormal behavior type and the confidence score for each type. Types: These include behaviors such as aggression, loitering, item theft, and disguise; Confidence: This is the reliability of the classification result, usually expressed as a probability value; for example, the classification of aggression might have a 90% confidence level, while loitering might have an 80% confidence level; Post-processing steps are used to further optimize the classification results, ensuring that the final output is more accurate and meaningful; If the confidence level of a certain abnormal behavior type is lower than a preset threshold, such as 60%, the classification result is considered unreliable and is excluded; For classification results with high confidence levels, such as above 80%, they are considered to have high reliability and can be retained as the final classification result; A filtered abnormal behavior type is obtained, and each behavior type is accompanied by a confidence level, indicating the credibility of the classification result.

[0031] In an optional embodiment, based on independent identity features and abnormal behavior type classification results, target personnel are tracked across frames and scenes to obtain abnormal personnel tracking results, including: Based on independent identity features, construct personnel identity identifiers and establish identity association relationships in different time frames and different camera scenarios; Based on the timestamps and spatial coordinates of the abnormal behavior type classification results, the movement trajectory of the abnormal personnel is drawn, where the trajectory includes the location information and time information of key nodes; Calculate the continuity score of the tracking trajectory, where the score reflects the coherence of the trajectory in time and space; associate and store the abnormal behavior type classification results, tracking trajectory, and continuity score to obtain the abnormal person tracking results, where the tracking results include the abnormal person's identity identifier, main abnormal type, complete movement trajectory, trajectory continuity score, and highest confidence classification result.

[0032] It's important to note that unique identity features can create a unique identifier for each person. This identifier helps the system recognize the same person across different times and camera scenarios, regardless of their location or actions. For example, if a person walks past camera A and then past camera B, the system can link these two trajectories to the same person using the identifier. Behavioral features are time-series data, therefore, it's necessary to analyze a person's behavior based on different time periods (time frames). For instance, a person moving from one location to another might span multiple time frames. Different cameras will capture the same person's behavior at different times, so it's necessary to establish spatial associations of identity based on data from multiple cameras. Through these different times and cameras... The system can establish time-space correlations, identifying the same person's identity across different times and camera scenarios. Following the previous steps, the system identifies and categorizes abnormal behavior types based on behavioral characteristics, such as aggression and unusual loitering. Each classification result includes a timestamp (indicating the time the abnormal behavior occurred) and spatial coordinates (indicating the location of the abnormal behavior). Based on these timestamps and spatial coordinates, the system can map the movement trajectory of the abnormal individual. Each key point in the trajectory includes: location (spatial coordinates): indicating the abnormal individual's position at a specific moment; and time (timestamp): indicating the time corresponding to that location. In this way, the system can clearly display the abnormal individual's movement route and mark the locations where they exhibited abnormal behavior at specific times and spaces. The continuity score reflects the temporal and spatial coherence of an abnormal person's movement trajectory. In other words, the system determines whether the abnormal person's behavior conforms to a continuous movement pattern, whether there are sudden stops or jumps, and whether it follows a normal path. A higher score is given if the trajectory exhibits stable, coherent behavior, such as walking along a corridor without sudden stops or reversals; a lower score is given if the trajectory contains numerous jumps, such as sudden changes in direction or prolonged pauses. This score helps in further analyzing the regularity or suddenness of abnormal behavior, and whether the person exhibits unusual behavioral patterns within a short period. The abnormal person's behavior classification, trajectory information, and trajectory continuity score are stored in a linked manner to ensure all relevant information is recorded and analyzed simultaneously. This data includes: Abnormal Person Identifier: a unique identifier for the individual; Main Abnormal Types: based on the classifier's output, indicating the person's main abnormal behavior types, such as aggression or abnormal loitering; Complete Movement Trajectory: including the person's entire movement path, marked by time and spatial coordinates; Trajectory Continuity Score: reflecting the degree of temporal and spatial coherence of the trajectory; Highest Confidence Classification Result: indicating the reliability of the classification result, typically the category with the highest confidence in the classifier's output. Ultimately, the system generates a complete abnormal personnel tracking result, which includes: Abnormal Personnel Identifier: indicating the unique identity of the person; Main Abnormal Type: identifying the person's main abnormal behavior; Complete Movement Trajectory: recording the person's trajectory in detail, including all location and time information; Trajectory Continuity Score: reflecting the consistency of the trajectory; Highest Confidence Classification Result: the confidence value of the classification result. These results provide detailed temporal and spatial information about the abnormal behavior, which is helpful for further analysis, investigation, and response.

[0033] Example 2, please refer to Figure 2 This invention provides a technical solution: an abnormal person tracking system based on facial images, applicable to the aforementioned abnormal person tracking method based on facial images, comprising: Image acquisition unit 1 is used to acquire multi-source face images and perform dynamic environmental adaptation processing on the multi-source face images to obtain stable basic images of face features; Feature extraction unit 2 is used to extract static identity features and dynamic behavior features from the stable facial feature base image to obtain a static identity feature set and a dynamic behavior feature set. The static identity feature set includes facial texture detail features and contour structure features, while the dynamic behavior feature set includes expression change time sequence features and head movement trajectory features. Feature processing unit 3 is used to perform spatiotemporal alignment processing on static identity feature set and dynamic behavior feature set to obtain spatiotemporal aligned feature set; Feature decoupling unit 4 is used to perform feature decoupling processing based on spatiotemporally aligned feature sets to obtain independent identity features and independent behavioral features; wherein, feature decoupling processing is used to eliminate the cross-mixing of identity information and behavioral information; Anomaly identification unit 5 is used to identify abnormal patterns in independent behavioral features and obtain abnormal behavior type classification results; The result tracking unit 6 is used to track target personnel across frames and scenes based on independent identity features and abnormal behavior type classification results, and obtain abnormal personnel tracking results.

[0034] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method of tracking a suspicious person based on a face image, characterized by, The method comprises the following steps: obtaining a multi-source face image, and performing dynamic environment adaptation processing on the multi-source face image to obtain a stable face feature base image; performing static identity feature extraction and dynamic behavior feature extraction on the stable face feature base image respectively to obtain a static identity feature set and a dynamic behavior feature set, wherein the static identity feature set comprises face texture detail features and contour structure features, and the dynamic behavior feature set comprises expression change timing features and head motion trajectory features; performing space-time alignment processing on the static identity feature set and the dynamic behavior feature set to obtain a space-time alignment feature set; performing feature decoupling processing based on the space-time alignment feature set to obtain independent identity features and independent behavior features; wherein the feature decoupling processing is used to eliminate the cross-mixing of identity information and behavior information; performing abnormal mode recognition on the independent behavior features to obtain an abnormal behavior type classification result; performing cross-frame and cross-scene tracking on a target person based on the independent identity features and the abnormal behavior type classification result to obtain an abnormal person tracking result. 2.The face image based abnormal person tracking method of claim 1, wherein, The method comprises the following steps: inputting the stable face feature base image into the input layer of a multi-task feature extraction network, and dividing the image features into identity feature branches and behavior feature branches through a double-branch routing module of the input layer, wherein the identity feature branches are used for extracting static identity features, and the behavior feature branches are used for extracting dynamic behavior features; in the identity feature branches, performing multi-scale texture enhancement processing on the stable face feature base image to obtain a texture feature map; performing contour boundary strengthening processing on the texture feature map to extract the geometric structure features of the face contour, calculating the gray level co-occurrence matrix of the key region pixel points as the texture detail features, and fusing the geometric structure features and the texture detail features to obtain a static identity feature set; in the behavior feature branches, performing continuous frame difference operation on the stable face feature base image to obtain a dynamic difference image sequence; performing optical flow field calculation on the dynamic difference image sequence to extract the pixel motion vector as the head motion trajectory features, calculating the timing change rate of the expression region pixel points as the expression change timing features, and fusing the head motion trajectory features and the expression change timing features to obtain a dynamic behavior feature set. 3.The face image based abnormal person tracking method of claim 2, wherein, performing multi-scale texture enhancement processing on the stable face feature base image to obtain a texture feature map; performing contour boundary strengthening processing on the texture feature map to extract the geometric structure features of the face contour, calculating the gray level co-occurrence matrix of the key region pixel points as the texture detail features, and fusing the geometric structure features and the texture detail features to obtain a static identity feature set, comprising: in the identity feature branches, initializing a multi-scale convolution kernel set, wherein the multi-scale convolution kernel set comprises Gaussian kernels and Laplacian kernels of different sizes; Convolution is performed on the stable face feature basis image based on each convolution kernel in the set of multi-scale convolution kernels, to obtain a texture feature map, wherein all the texture feature maps are collectively constructed into a set of multi-scale texture feature maps; Feature fusion processing is performed on the set of multi-scale texture feature maps to extract semantic texture information at different levels, to obtain a fused texture feature map; Edge detection processing is performed on the fused texture feature map to identify a face contour boundary, and a curvature change parameter and a boundary length parameter of the contour line are extracted as geometric structure features; A gray level co-occurrence matrix of pixel points in a region surrounded by the contour line is calculated, and contrast, correlation, energy and homogeneity are extracted as texture detail features, and the geometric structure features and the texture detail features are spliced in feature dimension to obtain a static identity feature set. 4.The face image based abnormal person tracking method of claim 3, wherein, Pixel motion vectors are extracted as head motion trajectory features by performing optical flow field calculation on the dynamic difference image sequence, and a time sequence change rate of expression region pixel points is calculated as expression change time sequence features, and the head motion trajectory features and the expression change time sequence features are fused to obtain a dynamic behavior feature set, including: In the behavior feature branch, Gaussian smoothing processing is performed on the dynamic difference image sequence to suppress noise interference, to obtain a smoothed dynamic difference image sequence; Lucas-Kanade optical flow algorithm is used to perform optical flow field calculation on the smoothed dynamic difference image sequence to obtain a motion vector of each pixel point between consecutive frames, and a motion vector mean and a direction change rate of head region pixel points are extracted as head motion trajectory features; An expression key region is located in the stable face feature basis image, wherein the expression key region includes eyes, eyebrows and a mouth, a pixel value change amount of the expression key region in the smoothed dynamic difference image sequence is extracted, and an average rate of pixel value change per unit time is calculated as an expression change time sequence feature; The head motion trajectory features and the expression change time sequence features are divided into time windows to obtain behavior feature segments, and the behavior feature segments are spliced in time sequence to obtain a dynamic behavior feature set. 5.The face image based abnormal person tracking method of claim 4, characterized in that, Temporal and spatial alignment processing is performed on the static identity feature set and the dynamic behavior feature set to obtain a spatio-temporal alignment feature set, including: The static identity feature set and the dynamic behavior feature set are input into an alignment calibration module of the multi-task feature extraction network; and a spatio-temporal alignment feature set is obtained through timestamp synchronization and spatial coordinate mapping processing. 6.The face image based abnormal person tracking method of claim 5, wherein, Feature decoupling processing is performed based on the spatio-temporal alignment feature set to obtain independent identity features and independent behavior features, including: A cross-modal feature correlation matrix is constructed, wherein the feature correlation matrix is used to represent the correlation strength between feature components in the static identity feature set and the dynamic behavior feature set; Singular value decomposition processing is performed on the cross-modal feature correlation matrix to obtain eigenvectors and singular values, and strongly correlated feature pairs are selected based on the singular value size. Projecting the static identity feature set and the dynamic behavior feature set into identity feature subspace and behavior feature subspace respectively based on the strong correlation characteristics to construct a decoupling projection matrix; In the identity feature subspace, removing the residual behavior feature component by low-pass filtering, and in the behavior feature subspace, removing the residual identity feature component by high-pass filtering to obtain independent identity features and independent behavior features after preliminary decoupling; Calculating the residual correlation of the independent identity features after preliminary decoupling and the dynamic behavior feature set, if the residual correlation is higher than a preset threshold, adjusting the decoupling projection matrix and re-executing the projection and decoupling operations until the residual correlation is lower than the preset threshold; Normalizing the independent identity features and the independent behavior features after final decoupling to ensure that the two types of features are in the same numerical range. 7.The face image based abnormal person tracking method of claim 6, wherein, Abnormal pattern recognition is performed on the independent behavior features to obtain an abnormal behavior type classification result, including: Inputting the independent behavior features into an abnormal behavior classifier, wherein the classifier includes an LSTM recurrent network and a temporal convolution network; Extracting the time sequence dependency of the behavior features through the LSTM recurrent network and capturing the local mutation pattern of the behavior features through the temporal convolution network; Feature fusion is performed on the outputs of the LSTM network and the temporal convolution network, and a full connection layer is inputted to classify and output an abnormal behavior type and a corresponding confidence; the abnormal behavior type includes attack tendency, abnormal wandering, object stealing and camouflage behavior, and the confidence represents the reliability of the classification result; Post-processing is performed on the abnormal behavior type classification result to remove classification results with a confidence lower than a preset value and retain abnormal behavior types with high confidence to obtain an abnormal behavior type classification result. 8.The face image based abnormal person tracking method of claim 7, wherein, Based on the independent identity features and the abnormal behavior type classification result, cross-frame and cross-scene tracking is performed on target personnel to obtain an abnormal personnel tracking result, including: Constructing a personnel identity based on the independent identity features to establish an identity association relationship in different time frames and different camera scenes; Drawing a moving track of an abnormal personnel based on the timestamp and spatial coordinates of the abnormal behavior type classification result, wherein the track includes position information and time information of key nodes; Calculating a continuity score of the tracking track, wherein the score reflects the coherence of the track in time and space; associating and storing the abnormal behavior type classification result, the tracking track and the continuity score to obtain an abnormal personnel tracking result, wherein the tracking result includes an abnormal personnel identity, a main abnormal type, a complete moving track, a track continuity score and a highest confidence classification result.

9. A system for tracking abnormal persons based on face images, which is suitable for the method for tracking abnormal persons based on face images according to any one of claims 1-8, characterized in that, including: An image acquisition unit configured to acquire multi-source face images, perform dynamic environment adaptation processing on the multi-source face images, and obtain stable face feature base images; The feature extraction unit is configured to perform static identity feature extraction and dynamic behavior feature extraction on the stable face feature basis image respectively to obtain a static identity feature set and a dynamic behavior feature set, wherein the static identity feature set includes facial texture detail features and contour structure features, and the dynamic behavior feature set includes expression change timing features and head movement trajectory features. The feature processing unit is configured to perform space-time alignment processing on the static identity feature set and the dynamic behavior feature set to obtain a space-time alignment feature set. The feature decoupling unit is configured to perform feature decoupling processing based on the space-time alignment feature set to obtain independent identity features and independent behavior features, wherein the feature decoupling processing is configured to eliminate the cross-mixing of identity information and behavior information. The anomaly recognition unit is configured to perform anomaly pattern recognition on the independent behavior features to obtain an abnormal behavior type classification result. The result tracking unit is configured to perform cross-frame and cross-scene tracking on the target personnel based on the independent identity features and the abnormal behavior type classification result to obtain an abnormal personnel tracking result.

Images