An abnormal behavior detection system and method based on a posture optimization strategy

Abstract

The application relates to an abnormal behavior detection system and method based on a posture optimization strategy, which comprises a tracking device, an optimization device and a detection device. The detection device comprises a bidirectional interactive graph self-attention standardization flow model. The optimization device can be used by a user to cluster all the relevant joint nodes of pedestrian posture data into high-motion areas and low-motion areas, and also performs posture optimization on each joint node coordinate based on probability density adaptive scaling to obtain optimized posture data. The bidirectional interactive graph self-attention standardization flow model converts the optimized posture data into probability distribution data, and the detection device also obtains abnormal behavior detection results according to the comparison results of the probability distribution data and normal behavior detection data. Furthermore, a bidirectional interactive graph self-attention enhanced network combining high and low motion joint node information is formed, which can resist the influence of background noise, effectively avoids the problem of missed detection of part of the abnormal behaviors on the basis of reducing the calculation cost.

Description

Technical Field

[0001] This invention relates to the field of computer modeling and systems technology, and more specifically, to an abnormal behavior detection system and method based on a posture optimization strategy. Background Technology

[0002] Video anomaly detection refers to the use of various network models to detect abnormal behaviors in surveillance videos. These abnormal behaviors include unnatural running or gathering of people, theft, and fights. Video anomaly detection is challenging because the definition of anomaly is highly subjective, resulting in no fixed pattern of abnormal behavior and making it difficult to define anomalies universally. Therefore, existing technologies focus on using normal events as guidance to optimize the parameters of a pre-designed network model, thereby achieving video anomaly detection.

[0003] Existing video anomaly detection methods can be categorized into pixel-based and pose-based methods based on the different data modalities representing human behavior. Pixel-based methods detect anomalous events by analyzing pixel-level changes in video frames. However, these methods are significantly affected by background noise and have high computational costs.

[0004] Pose-based detection methods include reconstruction prediction methods and distribution modeling-based detection methods. Reconstruction prediction methods focus on reconstructing the current sequence and predicting the future sequence. These methods mainly reconstruct or predict pose sequences of normal behavior and rely on the constraints of reconstruction error, but often require greater computational cost and memory consumption, leading to a decrease in real-time performance.

[0005] In contrast, detection methods based on distribution modeling focus on acquiring the probability distribution characteristics of normal pose data and identifying samples that deviate from this probability distribution as anomalies. This method models the data distribution of normal samples and maps it to a latent space to calculate the anomaly probability value. However, the performance of the model obtained by this method is easily affected by the distribution of the input data. When the distinction between normal behavior and certain abnormal behavior in the input data is low, such as body swaying caused by dizziness being highly similar to normal behavior in dynamic features, the density estimation ability of the normalized flow will be limited, resulting in some abnormal behaviors being assigned a high probability by the model and thus missed. Summary of the Invention

[0006] The technical problem to be solved by this invention is how to overcome the technical defects of existing video anomaly detection technology, such as being greatly affected by background noise, having high computational costs, and missing some abnormal behaviors. To overcome these technical defects, this invention provides an anomaly behavior detection system and method based on a pose optimization strategy, specifically including an anomaly behavior detection system based on a pose optimization strategy and an anomaly behavior detection method based on a pose optimization strategy.

[0007] This invention provides an abnormal behavior detection system based on a pose optimization strategy, comprising the following components connected in series:

[0008] The tracking device is configured to extract pedestrian posture data of a specified pedestrian from a video of a pedestrian; the pedestrian posture data includes video frames and the coordinates of multiple body joints in the video frames.

[0009] The optimization device is configured to allow the user to cluster all joints involved in the pedestrian posture data into high motion regions and low motion regions, obtain the joint motion amplitude probability density of the high motion regions and low motion regions respectively using a kernel density estimation algorithm, and perform posture optimization based on probability density adaptive scaling on the coordinates of each joint to obtain optimized posture data.

[0010] The detection device is configured to convert the optimized pose data into probability distribution data by invoking its included bidirectional interactive graph self-attention normalized flow model, and then obtain abnormal behavior detection results based on the comparison results of the probability distribution data and normal behavior detection data.

[0011] The abnormal behavior detection system based on a posture optimization strategy disclosed in this invention comprises a tracking device, an optimization device, and a detection device. The detection device includes a bidirectional interactive graph self-attention normalized flow model. The optimization device allows the user to cluster all joints involved in pedestrian posture data into high-motion and low-motion regions. Furthermore, the optimization device utilizes a kernel density estimation algorithm to obtain the probability density of joint motion amplitudes in both high-motion and low-motion regions, and performs posture optimization based on probability density adaptive scaling on the coordinates of each joint to obtain optimized posture data. The bidirectional interactive graph self-attention normalized flow model transforms the optimized posture data into probability distribution data. The detection device also obtains the abnormal behavior detection result based on the comparison between the probability distribution data and normal behavior detection data. This forms a bidirectional interactive graph self-attention enhancement network that combines high and low motion joint information, which can resist the influence of background noise and focus more on key motion features and long-distance limb interactions in anomaly detection, outperforming most existing methods in experiments. Meanwhile, by optimizing the processing of posture data, the problem of low distinguishability between normal behavior and some abnormal behavior in the input data is effectively alleviated while reducing computational costs. This improves the model's ability to distinguish abnormal behavior and effectively avoids the problem of some abnormal behaviors being missed.

[0012] In one possible implementation, the tracking device comprises, in series:

[0013] The pose estimator is configured to extract human joint information from a pedestrian video and obtain the joint coordinates of each joint to obtain the pedestrian pose data.

[0014] The tracker is configured to maintain the identity tracking of the same pedestrian across consecutive video frames and assign a unique number to each pedestrian in order to obtain cross-frame identity trajectory information for a given pedestrian.

[0015] Pedestrian pose data can be extracted from videos and pedestrians can be tracked using pose estimators and trackers. Furthermore, the inter-frame motion changes of all joints can be statistically analyzed, providing a basis for pose optimization and enabling controllable pose optimization.

[0016] In one possible implementation, the optimization device includes:

[0017] The region segmentation module, which communicates with the tracker, is configured to allow the user to perform clustering of all key points involved in the pedestrian posture data into high-motion regions and low-motion regions.

[0018] The attitude optimization module, which communicates with both the region segmentation module and the detection device, is configured to use a kernel density estimation algorithm to obtain the probability density of joint motion amplitude in the high motion region and the low motion region, respectively, and to perform attitude optimization based on probability density adaptive scaling on the coordinates of each joint to obtain optimized attitude data.

[0019] The optimized device with the above structure and functions can not only meet the requirement of allowing users to divide human joints into high motion regions and low motion regions according to motion characteristics, but also can count the inter-frame motion change amplitude of all joints in the two regions, and use the kernel density estimation algorithm to model the normal motion amplitude, and then use the probability value to guide the scaling of the joints, which significantly enhances the distinguishability between normal samples and abnormal samples.

[0020] In one possible implementation, the attitude optimization module is configured to perform the following steps to obtain the optimized attitude data:

[0021] A1: Extract multiple consecutive video frames from the pedestrian posture data, and extract the coordinates of all joints of the specified pedestrian in the multiple consecutive video frames.

[0022] A2: Calculate the Euclidean distance between the coordinates of each joint in adjacent video frames, and use this Euclidean distance as the motion amplitude to obtain the motion amplitude of each joint in each video frame.

[0023] A3: Integrate the motion amplitude values ​​of all relevant nodes in the high motion region in the consecutive video frames into a high motion region motion amplitude set, and integrate the motion amplitude values ​​of all relevant nodes in the low motion region in the consecutive video frames into a low motion region motion amplitude set.

[0024] A4: The kernel density estimation algorithm is used to fit the probability density function of the motion amplitude set in the high motion region to obtain the probability density of the joint motion amplitude in the high motion region; and the kernel density estimation algorithm is used to fit the probability density function of the motion amplitude set in the low motion region to obtain the probability density of the joint motion amplitude in the low motion region.

[0025] A5: Perform a linear combination of the two joint motion amplitude probability densities obtained in step A4 to obtain a scaling function, and calculate the scaling function value of the motion amplitude of each joint in each video frame.

[0026] A6: Perform a scalar multiplication operation between the scaling function value of the motion amplitude of each joint point in each video frame and the joint point coordinates in that video frame, and use the result as the optimized coordinates of each joint point in each video frame.

[0027] A7: Adjust the image of the video frame according to the optimized coordinates of each joint point in each video frame to obtain the image adjustment results of the consecutive video frames, and use the image adjustment results of the consecutive video frames as the optimized posture data.

[0028] The posture optimization module, operating in the manner described above, statistically analyzes the inter-frame motion amplitude changes of joints in different regions of the body based on their motion characteristics. It then uses a kernel density estimation algorithm to model the motion amplitude and guides the adaptive scaling of joints with probability values, thereby achieving overall posture sequence optimization and further improving the distinguishability between normal and abnormal samples.

[0029] In one possible implementation, the bidirectional interactive graph self-attention normalized flow model is a network structure composed of multiple normalized flow models connected in series. Each normalized flow model is configured to transform its input data into its output data using a fusion algorithm of graph convolution and self-attention. The input data of the first normalized flow model is the optimized pose data, and the output data of the last normalized flow model is the probability distribution data. This allows the complex original distribution to be transformed into a simple target distribution, thereby better approximating the data distribution characteristics under the target distribution and ensuring the effective extraction of the probability distribution data.

[0030] In one possible implementation, the standardized flow model includes:

[0031] The normalization layer model is configured to perform affine normalization on the channel dimension of the input data of the normalized flow model to which it belongs in order to obtain normalized data.

[0032] The permutation layer model, which communicates with the normalization layer model, is configured to shuffle the channel order of the normalized data through a reversible convolution with a kernel size of 1×1 to obtain the permutation result data.

[0033] The bidirectional interactive graph self-attention module is configured to perform a fusion algorithm of graph convolution and self-attention;

[0034] The affine coupling model, which communicates with both the permutation layer model and the bidirectional interactive graph self-attention module, is configured to split the permutation result data into a first part and a second part by channel average; then, the bidirectional interactive graph self-attention module is invoked to transform the first part into a scaling factor and a translation amount, and the second part is multiplied by the scaling factor. The resulting product is then summed with the translation amount to obtain the second part transformation result; finally, the first part and the second part transformation results are channel-stitched to obtain the output data of the normalized flow model to which the affine coupling model belongs.

[0035] The standardized flow model, possessing the aforementioned structure and functionality, performs affine normalization on the input features along the channel dimension using a normalization layer model. Initial normalization is achieved through optimizable translation and scaling parameters. The permutation layer model enhances the model's information mixing capability by shuffling the channel order. The combination of a bidirectional interactive graph self-attention module and an affine coupling model effectively captures global dependencies while maintaining affine coupling. This further improves the standardized flow model's ability to model and express anomalous behavior.

[0036] In one possible implementation, the bidirectional interactive graph self-attention module is configured to perform the following steps:

[0037] B1: Generate a topological adjacency matrix representing the physical structure of the human body's internal connections based on the natural connectivity of the human skeleton. Construct a motion saliency adjacency matrix representing whether there is a connection and the strength of the connection between two joints based on the prior of joint motion saliency, and set an initial adaptive adjacency matrix.

[0038] B2: Perform self-attention mechanism mapping processing on the first part to obtain self-attention output;

[0039] B3: The self-attention output is sequentially processed by global average pooling, one-dimensional convolution, Gaussian error linear activation, two-dimensional convolution, and sigmoid activation function to obtain channel importance features;

[0040] B4: Multiply the channel importance feature by the initial adaptive adjacency matrix to obtain the adaptive adjacency matrix;

[0041] B5: Perform graph convolution operations based on the topological adjacency matrix, the motion saliency adjacency matrix, and the adaptive adjacency matrix on the first part respectively, and sum the results obtained from the graph convolution operations to obtain spatial features;

[0042] B6: The spatial features are sequentially processed by global average pooling, convolution, and sigmoid activation function to obtain spatial attention weights;

[0043] B7: Multiply the self-attention output by the spatial attention weights to obtain self-attention features;

[0044] B8: Summing the spatial features and the self-attention features yields the scaling factor and translation amount.

[0045] The model operates in the manner described above to adaptively adjust the adjacency matrix weights, thereby focusing on regions with high motion characteristics and enhancing the ability of the standardized flow model to model cross-joint motion patterns. Furthermore, three adjacency matrices are designed to enhance the modeling ability of graph convolution for local structural information. While maintaining the advantages of graph convolution in modeling local structure, a structure-aware self-attention mechanism is introduced to capture global dependencies. Moreover, a bidirectional information interaction mechanism between graph convolution and self-attention enables feature complementarity between the graph convolution and self-attention branches, thus endowing graph convolution with global perception capabilities and further improving the standardized flow model's ability to model and express abnormal behavior.

[0046] Another technical solution of the present invention is to provide an abnormal behavior detection method based on a posture optimization strategy, comprising the following steps:

[0047] S1: Extract pedestrian posture data of a specified pedestrian from the pedestrian video using a tracking device; the pedestrian posture data includes video frames and the coordinates of multiple body joints in the video frames;

[0048] S2: The user uses an optimization device to cluster all the key points involved in the pedestrian posture data into high motion regions and low motion regions;

[0049] S3: The optimization device uses a kernel density estimation algorithm to obtain the motion amplitude probability density of joints in the high motion region and the low motion region respectively, and performs attitude optimization based on probability density adaptive scaling on the coordinates of each joint to obtain optimized attitude data.

[0050] S4: The optimized posture data is converted into probability distribution data by the detection device, and then the abnormal behavior detection result is obtained based on the comparison result between the probability distribution data and the normal behavior detection data.

[0051] The abnormal behavior detection method based on posture optimization strategy disclosed in this invention first instructs the user to cluster all joints involved in pedestrian posture data into high-motion and low-motion regions on an optimization device. The optimization device then executes a kernel density estimation algorithm to obtain the probability density of joint motion amplitudes in the high-motion and low-motion regions, respectively. Posture optimization based on probability density adaptive scaling is then applied to the coordinates of each joint to obtain optimized posture data. Subsequently, an abnormal behavior detection result is obtained by comparing the probability distribution data with normal behavior detection data using a detection device. This forms a bidirectional interactive graph self-attention enhancement mechanism that combines high and low motion joint information, which can resist the influence of background noise and focus more on key motion features and long-distance limb interactions in anomaly detection, demonstrating excellent performance in highly challenging cross-dataset experiments. Simultaneously, by optimizing the posture data, the method effectively alleviates the problem of low discriminative power between normal behavior and some abnormal behavior in the input data while reducing computational costs, improving the model's ability to distinguish abnormal behavior and effectively avoiding the missed detection of some abnormal behaviors. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of an abnormal behavior detection system based on a posture optimization strategy disclosed in an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram illustrating the division of the high motion region and low motion region as disclosed in the embodiments of the present invention;

[0054] Figure 3 This is a schematic diagram of the operation flow of the optimization device disclosed in the embodiments of the present invention;

[0055] Figure 4 This is a flowchart illustrating the operation of the attitude optimization module disclosed in this embodiment of the invention.

[0056] Figure 5 This is a schematic diagram of the standardized flow model structure disclosed in the embodiments of the present invention;

[0057] Figure 6 This is a schematic diagram of the bidirectional interactive graph self-attention normalized flow model disclosed in the embodiments of the present invention;

[0058] Figure 7 This is a schematic diagram of the operation flow of the graph convolution and self-attention fusion algorithm disclosed in the embodiments of the present invention;

[0059] Figure 8 A flowchart of the method disclosed in the embodiments of the present invention. Detailed Implementation

[0060] First, those skilled in the art should understand that these embodiments are merely used to explain the technical principles of the embodiments of this application and are not intended to limit the scope of protection of the embodiments of this application. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0061] In the description of the embodiments of this application, it should be noted that, unless otherwise explicitly specified and limited, the term "forming a communication link structure" means that the multiple communication elements or modules involved form a network structure or network link structure through communication connection. Communication or communication connection means that there is information transmission between the first feature and the second feature. This information transmission can be unidirectional or bidirectional. The communication connection can be realized by electrical connection of wires, radio connection, electrical connection of electromagnetic media (such as optical fiber, semiconductor), communication realized by channel, etc.

[0062] In the embodiments of this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0063] The present application will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0064] See Figures 1 to 8 This application discloses an abnormal behavior detection system based on a posture optimization strategy. Figure 1 This is a schematic diagram of the abnormal behavior detection system. The abnormal behavior detection system includes a tracking device, an optimization device, and a detection device. The detection device includes a bidirectional interactive graph self-attention normalized flow model. The tracking device, optimization device, and detection device are connected in series to form a communication link structure.

[0065] See Figure 1 In this abnormal behavior detection system, the tracking device is configured to extract pedestrian posture data of a specified pedestrian from the pedestrian's video; the pedestrian posture data includes video frames and the coordinates of multiple body joints within the video frames; the video used can be acquired through a video capture device. For example... Figure 1As shown, in this embodiment, the tracking device includes an attitude estimator and a tracker, and the attitude estimator and the tracker are connected in series to form a communication link structure, and the tracker also communicates with the optimization device.

[0066] In the tracking device, the pose estimator is configured to extract human joint information from the pedestrian's video and obtain the coordinates of each joint to acquire pedestrian pose data. The tracker is configured to maintain the identity tracking of the same pedestrian in consecutive video frames and assign a unique number to each pedestrian to obtain cross-frame identity trajectory information for a given pedestrian. The purpose of obtaining cross-frame identity trajectory information is to confirm whether the tracked person (each pedestrian is represented by a number) is still in the corresponding video frame, and the pedestrian's relative position in the video frame, thereby providing a basis for subsequently selecting multiple consecutive video frames.

[0067] See Figure 1 , Figure 2 and Figure 3 In this abnormal behavior detection system, the optimization device is configured to allow the user to cluster all joints involved in the pedestrian posture data into high-motion regions and low-motion regions. The kernel density estimation (KDE) algorithm is used to obtain the probability density of joint motion amplitudes in the high-motion and low-motion regions, respectively. Then, posture optimization based on probability density adaptive scaling is performed on the coordinates of each joint to obtain optimized posture data. For example... Figure 1 As shown, in this embodiment, the optimization device includes a region division module and an attitude optimization module. The region division module communicates with the tracker, and the attitude optimization module communicates with both the region division module and the detection device.

[0068] See Figure 2 In the optimization device, the region segmentation module is configured to allow the user to cluster all joints involved in the pedestrian posture data into high-motion and low-motion regions. In this embodiment, the region segmentation module includes a display and an editor set on the display, allowing the user to divide the 17 joints of the human body into high-motion and low-motion regions based on the motion characteristics of the human joints, as shown in Figure 2. The joints in the high-motion region are represented by hollow dots and mainly include the hand, elbow, knee, and foot joints. These joints have greater degrees of freedom of movement and can produce larger amplitude movements during actions such as walking and running. High-motion regions are represented by... H express, H = {8, 9, 10, 11, 14, 15, 16, 17}, low motion region (joints are represented by solid dots) using L express, L = {1, 2, 3, 4, 5, 6, 7, 12, 13}, the numbers in the set represent Figure 2 The joint number defined in the code.

[0069] Observations of normal human walking patterns reveal that the joint motion amplitude in high-motion regions is significantly greater than that in low-motion regions. This characteristic makes joints in high-motion regions play a more crucial role in distinguishing movement patterns and extracting features. Therefore, subsequent optimized posture data modeling will fully utilize the different characteristics of high and low motion regions to enhance their contribution to motion features and improve the accuracy of anomaly detection.

[0070] See figure Figure 3 and Figure 4 In the optimization device, the attitude optimization module is configured to use a kernel density estimation algorithm to obtain the probability density of joint motion amplitude in the high motion region and the low motion region, and to perform attitude optimization based on probability density adaptive scaling on the coordinates of each joint to obtain optimized attitude data. Figure 3 To optimize the overall operation diagram of the device, Figure 4 The flowchart shows the operation of the attitude optimization module. Specifically, the attitude optimization module is configured to perform the following steps to obtain optimized attitude data:

[0071] A1: Extract multiple consecutive video frames from pedestrian pose data, and extract the coordinates of all joints of a specified pedestrian in these multiple consecutive video frames. In this embodiment, the first... The first joint point in the The coordinates in a video frame are represented as follows:

[0072] .

[0073] A2: Calculate the Euclidean distance between the coordinates of each joint in adjacent video frames, and use this Euclidean distance as the motion amplitude to obtain the motion amplitude of each joint in each video frame. The first joint point in the The motion amplitude in the nth video frame refers to the nth... The first joint point in the and The Euclidean motion changes between video frames are specifically expressed as follows:

[0074] ,

[0075] in, Indicates the first The first joint point in the Motion amplitude values ​​in each video frame.

[0076] A3: Integrate the motion amplitude values ​​of all relevant nodes in the high-motion region across multiple consecutive video frames into a high-motion region motion amplitude set, and integrate the motion amplitude values ​​of all relevant nodes in the low-motion region across multiple consecutive video frames into a low-motion region motion amplitude set. Step S3 is... Figure 3 In the discrete statistical process described in this embodiment, the set of motion amplitudes in the high-motion region is represented as:

[0077] ,

[0078] The set of motion amplitude values ​​in the low-motion region is represented as:

[0079] ,

[0080] in, This indicates the number of consecutive video frames.

[0081] A4: The kernel density estimation algorithm is used to fit the probability density function of the motion amplitude set in the high-motion region to obtain the probability density of the joint motion amplitude in the high-motion region; and the kernel density estimation algorithm is used to fit the probability density function of the motion amplitude set in the low-motion region to obtain the probability density of the joint motion amplitude in the low-motion region. In this embodiment, the probability density of the joint motion amplitude in the high-motion region is expressed as:

[0082] ,

[0083] in, Represents the amplitude of motion. For the kernel function, a Gaussian kernel function is chosen in this embodiment because the normal motion amplitude distribution exhibits characteristics of small variation, concentrated values, strong unimodality, and physical boundaries. For this highly concentrated unimodal distribution, Scott's rule is used to calculate the bandwidth, i.e., the bandwidth... ,in This represents the standard deviation of a sample (i.e., consecutive video frames). Representing the sample size, this format provides a moderate smoothness for the concentrated distribution, effectively balancing the bias and variance of the density estimate. The method for obtaining the probability density of joint motion amplitudes in low-motion regions is the same, using... This represents the probability density of joint motion amplitude in low-motion regions.

[0084] A5: Perform a linear combination of the motion amplitude probability densities of the two joints obtained in step A4 to obtain the scaling function, and calculate the scaling function value of the motion amplitude of each joint in each video frame. The specific formula for the linear combination is as follows:

[0085] ,

[0086] in, and These are hyperparameters used to control the impact of high-motion and low-motion regions on scaling. This is the scaling function value, also known as the attitude optimization coefficient. Therefore, the... The first joint point in the The scaling function value of motion amplitude in each video frame is: .

[0087] A6: Multiply the scaling function value of the motion amplitude of each joint point in each video frame by the joint point's coordinates in that video frame, and use the result as the optimized coordinates of each joint point in each video frame. The specific calculation formula is as follows:

[0088] ,

[0089] in, Representing the The first joint point in the Optimized coordinates in each video frame.

[0090] A7: Adjust the image of the video frame according to the optimized coordinates of each joint in each video frame to obtain the image adjustment results of multiple consecutive video frames, and use the image adjustment results of multiple consecutive video frames as the optimized posture data.

[0091] See Figure 5 , Figure 6 and Figure 7 In this abnormal behavior detection system, the detection device is configured to convert optimized pose data into probability distribution data by calling its included bidirectional interactive graph self-attention normalized flow model. The abnormal behavior detection result is then obtained based on a comparison between the probability distribution data and normal behavior detection data. In this embodiment, the bidirectional interactive graph self-attention normalized flow model is a network structure composed of multiple normalized flow models connected in series. The normalized flow model is configured to convert its input data into its output data using a fusion algorithm of graph convolution and self-attention. The input data of the first normalized flow model is the optimized pose data, and the output data of the last normalized flow model is the probability distribution data.

[0092] See Figure 5In this embodiment, the normalized flow model includes a normalization layer model (Actnorm), a permutation layer model (Permatition), a bidirectional interactive graph self-attention module, and an affine coupling model. The permutation layer model communicates with the normalization layer model, and the affine coupling model communicates with both the permutation layer model and the bidirectional interactive graph self-attention module. In the normalized flow model, the normalization layer model is configured to perform affine normalization on the channel dimension of the input data of its respective normalized flow model to obtain normalized data. The permutation layer model is configured to shuffle the channel order of the normalized data using a 1×1 reversible convolution to obtain the permutation result data. Figure 6 Chinese (Representation). The bidirectional interactive graph self-attention module is configured to perform a fusion algorithm of graph convolution and self-attention. The affine coupling model is configured to split the permutation result data into a first part based on channel average ( Figure 6 Chinese (representation) and the second part ( Figure 6 Chinese (This is followed by a call to the bidirectional interactive graph self-attention module to transform the first part into a scaling factor and a translation, multiplying the second part by the scaling factor, and summing the resulting product with the translation to obtain the transformation result of the second part; finally, the transformation results of the first and second parts are channel-stitched to obtain the output data of the normalized flow model to which the affine coupling model belongs.) Figure 6 middle,

[0093] ,

[0094] ,

[0095] in, Represents the scaling factor. This represents the amount of translation.

[0096] See Figure 7 In this embodiment, the bidirectional interactive graph self-attention module is configured to perform the following steps (i.e., the flow of the graph convolution and self-attention fusion algorithm):

[0097] B1: Generate a topological adjacency matrix representing the physical structure of the human body's internal connections based on the natural connectivity of the human skeleton. Figure 7 Chinese (Representing the topological adjacency matrix), a motion-significant adjacency matrix is ​​constructed based on the prior knowledge of joint motion significance, representing whether there is a connection between two joints and the strength of that connection. Figure 7 Chinese (Represents the adjacency matrix of motion saliency), and sets the initial adaptive adjacency matrix.

[0098] B2: Perform self-attention mechanism mapping processing on the first part to obtain self-attention output.

[0099] B3: The self-attention output is sequentially processed by global average pooling, one-dimensional convolution, Gaussian error linear activation, two-dimensional convolution, and sigmoid activation function to obtain channel importance features. Figure 7 Chinese (Indicates the importance characteristics of the channel).

[0100] B4: Multiply the channel importance feature by the initial adaptive adjacency matrix to obtain the adaptive adjacency matrix. Figure 7 Chinese (This represents an adaptive adjacency matrix).

[0101] B5: Perform graph convolution operations based on the adjacency matrix of the topological structure on the first part respectively. Figure 7 The general GCN represents graph convolution operations, graph convolution operations based on motion saliency adjacency matrices, and graph convolution operations based on adaptive adjacency matrices. The results of the graph convolution operations are summed to obtain spatial features. Figure 7 Chinese (Representing spatial features). In this embodiment, a 1×1 convolution kernel was further performed before summation. Figure 7 In this context, Conv1×1 represents a convolution process with a kernel size of 1×1.

[0102] B6: Spatial features are sequentially processed by global average pooling, convolution, and sigmoid activation function to obtain spatial attention weights.

[0103] B7: Left-multiply the self-attention output by the spatial attention weights to obtain the self-attention features. Figure 7 Chinese (Represents spatial characteristics).

[0104] B8: Summing the spatial features and self-attention features yields the scaling factor and translation.

[0105] See Figure 8 The following will further disclose the abnormal behavior detection method corresponding to the abnormal behavior detection system based on the posture optimization strategy in this embodiment. The method includes the following steps:

[0106] S1: Extract pedestrian posture data of a specified pedestrian from the pedestrian video using a tracking device; the pedestrian posture data includes video frames and the coordinates of multiple body joints in the video frames.

[0107] S2: The user uses an optimization device to cluster all key points involved in the pedestrian pose data into high-motion regions and low-motion regions, such as... Figure 2As shown.

[0108] S3: The optimization device uses a kernel density estimation algorithm to obtain the motion amplitude probability density of joints in the high motion region and the low motion region respectively, and performs attitude optimization based on probability density adaptive scaling on the coordinates of each joint to obtain optimized attitude data.

[0109] S4: The optimized posture data is converted into probability distribution data by the detection device, and then the abnormal behavior detection result is obtained by comparing the probability distribution data with the normal behavior detection data.

[0110] The technical performance of the abnormal behavior detection system based on pose optimization strategy in this embodiment will be described in detail below. This embodiment compares the abnormal behavior detection system with current mainstream models on four types of public datasets (ShanghaiTech, ShanghaiTech-HR, UBnormal, and UBnormal-HR).

[0111] The ShanghaiTech dataset covers 13 different surveillance scenarios from various perspectives, including 330 training videos and 107 test videos. Anomalies include fighting, chasing, cycling, driving, and skateboarding. The UBnormal dataset is built using virtual scenes, covering 29 different scenarios, with 268 training videos, 64 validation videos, and 211 test videos. Anomalies include fighting, falling, dancing, and running. The ShanghaiTech-HR dataset is a subset of the ShanghaiTech dataset, and the UBnormal-HR dataset is a subset of the UBnormal dataset. Both UBnormal and UBnormal-HR datasets contain only anomalies related to human activity.

[0112] Following previous methods, this embodiment uses the area under the ROC curve (AUC) as the evaluation metric to measure the ability of the binary classification model to distinguish between positive and negative samples. It is trained on an Nvidia GeForce RTX 4090 using the adamx optimizer and a learning rate of 5×10⁻⁶. -4 The training was performed for 8 epochs with a batch size of 256. On the ShanghaiTech dataset, this embodiment used a video frame sequence length of 24 frames. However, on the UBnormal dataset, considering the lower pose estimation and tracking accuracy, this embodiment sets the video frame sequence length to 16 frames.

[0113] Furthermore, this embodiment sets a prior distribution N(3, ..., for normal samples. IThe implementation uses K = 8 normalized flow models. To extract human skeleton information from the video, this embodiment employs AlphaPose as a pose estimator to detect the skeletal keypoints of each target. Simultaneously, YOLOX is used as a tracker to locate individuals in the video, and PoseFlow is combined for temporal tracking of multi-target poses to obtain a continuous and complete human skeleton sequence.

[0114] This embodiment compares the abnormal behavior detection system with state-of-the-art models, including RGB-based and pose-based models. RGB-based models include FPDM, MULDE, HSC, USTN-DSC, VADiffusion, SSMTL++, and Jigsaw, while pose-based models include STG-NF, MoPRL, PoseWatch, MoCoDAD, STGformer, HSTGCNN, GEPC, ​​GiCiSAD, TSGAD, TrajREC, COSKAD, MPED-RNN, and Bipoco.

[0115] Table 1 presents a systematic comparison of the proposed abnormal behavior detection system with several current advanced models in terms of AUC. Underlined data represents the relevant results of this system, bold data represents the best results, and "-" indicates that the data is missing. For some models, the results were reproduced under a unified configuration to ensure fairness in the evaluation. The results show that the proposed abnormal behavior detection system outperforms most mainstream models on four different types of datasets, especially in pose-based unsupervised anomaly detection tasks. Notably, even when some datasets contain abnormal events unrelated to human behavior, the proposed system still surpasses RGB-based detection models in most cases, further validating its strong robustness to complex scenes.

[0116] Table 1: Comparison results of the abnormal behavior detection system in this embodiment with the current state-of-the-art models.

[0117]

[0118] During the evaluation process, the abnormal behavior detection system of this embodiment was comprehensively compared with mainstream models, most of which were published within the last three years and represent the current research frontier in this field. As shown in Table 1, the abnormal behavior detection system of this embodiment achieved an AUC score of 86.3% on the ShanghaiTech dataset, outperforming the vast majority of models and second only to the MUDLE model (86.7%), with a difference of only 0.4%. On the ShanghaiTech-HR subset, the abnormal behavior detection system achieved an AUC of 87.9%, significantly outperforming all existing pose-based models and reaching the current state of best performance. On the UBnormal dataset, the abnormal behavior detection system of this embodiment achieved an AUC of 72.6%, second only to MUDLE (72.8%), but outperforming other pose-based or RGB-based models. On the UBnormal-HR subset, the abnormal behavior detection system of this embodiment further extended its performance advantage, significantly outperforming all current pose-based detection models (see Table 1 for specific values).

[0119] In summary, the abnormal behavior detection system based on a posture optimization strategy disclosed in this embodiment comprises a tracking device, an optimization device, and a detection device. The detection device includes a bidirectional interactive graph self-attention normalized flow model. The optimization device allows users to cluster all joints involved in pedestrian posture data into high-motion and low-motion regions. Furthermore, the optimization device utilizes a kernel density estimation algorithm to obtain the probability density of joint motion amplitudes in the high-motion and low-motion regions, and performs posture optimization based on probability density adaptive scaling on the coordinates of each joint to obtain optimized posture data. The bidirectional interactive graph self-attention normalized flow model transforms the optimized posture data into probability distribution data. The detection device also obtains the abnormal behavior detection result based on the comparison between the probability distribution data and normal behavior detection data. This forms a bidirectional interactive graph self-attention enhancement network that combines high and low motion joint information, which can resist the influence of background noise and focus more on key motion features and long-distance limb interactions in anomaly detection. In experiments, it outperforms most existing methods and demonstrates excellent performance in highly challenging cross-dataset experiments. Meanwhile, by optimizing the processing of posture data, the problem of low distinguishability between normal behavior and some abnormal behavior in the input data is effectively alleviated while reducing computational costs. This improves the model's ability to distinguish abnormal behavior and effectively avoids the problem of some abnormal behaviors being missed.

[0120] In the description of the embodiments of this application, it should be noted that the terms "inner" and "outer" and other terms indicating direction or positional relationship are based on the direction or positional relationship shown in the drawings. This is only for the convenience of description and does not indicate or imply that the device or component must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this application.

[0121] In the description of this application, the references to terms such as "an embodiment," "some embodiments," "in this embodiment," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0122] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An abnormal behavior detection system based on a posture optimization strategy, characterized in that, comprises, in sequence: a tracking device configured to extract pedestrian pose data of a specified pedestrian from a video of pedestrians; the pedestrian pose data comprises video frames and joint coordinates of multiple joints of a body in the video frames; an optimization device configured to allow a user to cluster all joints involved in the pedestrian pose data into high motion areas and low motion areas, to obtain joint motion amplitude probability densities of the high motion areas and the low motion areas respectively by using a kernel density estimation algorithm, and to perform pose optimization on each joint coordinate based on adaptive scaling of the probability densities to obtain optimized pose data; a detection device configured to convert the optimized pose data into probability distribution data by invoking a bidirectional interactive graph self-attention standardization flow model contained therein, and then to obtain an abnormal behavior detection result according to a comparison result of the probability distribution data and normal behavior detection data; the bidirectional interactive graph self-attention standardization flow model is a network structure formed by sequentially connecting multiple standardization flow models, and the standardization flow model is configured to convert input data thereof into output data thereof by using a fusion algorithm of graph convolution and self-attention; wherein the input data of the standardization flow model at the first position is the optimized pose data, and the output data of the standardization flow model at the last position is the probability distribution data; the standardization flow model comprises: a normalization layer model configured to perform affine normalization operation on input data of the standardization flow model to which the normalization layer model belongs in a channel dimension to obtain normalized data; a permutation layer model in communication with the normalization layer model and configured to perform channel order scrambling on the normalized data by using a reversible convolution method with a convolution kernel size of 1x1 to obtain permutation result data; a bidirectional interactive graph self-attention module configured to perform the fusion algorithm of graph convolution and self-attention; an affine coupling model in communication with both the permutation layer model and the bidirectional interactive graph self-attention module and configured to split the permutation result data into a first part and a second part by channel averaging; then invoke the bidirectional interactive graph self-attention module to run to convert the first part into a scaling factor and a translation amount, multiply the second part by the scaling factor, sum the product and the translation amount to obtain second part transformation result; finally, perform channel splicing on the first part and the second part transformation result to obtain output data of the standardization flow model to which the affine coupling model belongs.

2. The gesture-based optimization policy based anomaly detection system of claim 1, wherein, the tracking device comprises, in sequence: a pose estimator configured to extract human joint information from a video of pedestrians and obtain joint coordinates of each joint to obtain the pedestrian pose data; a tracker configured to maintain identity tracking of the same pedestrian in consecutive video frames and assign a unique number to each pedestrian to obtain cross-frame identity track information corresponding to the specified pedestrian.

3. The gesture-based optimization policy based anomaly detection system of claim 2, wherein, the optimization device comprises: a region division module in communication with the tracker and configured to allow a user to perform clustering of all joints involved in the pedestrian pose data into high motion areas and low motion areas; The posture optimization module, in communication with the region division module and the detection device, is configured to obtain joint motion amplitude probability densities of high motion regions and low motion regions respectively by using kernel density estimation algorithm, and to perform posture optimization based on probability density adaptive scaling on each joint coordinate respectively to obtain the optimized posture data.

4. The gesture-based optimization policy based anomaly detection system of claim 3, wherein, The posture optimization module is configured to perform the following steps to obtain the optimized posture data: A1: extracting a plurality of continuous video frames from the pedestrian posture data, and extracting all joint coordinates of each joint of the specified pedestrian in each of the plurality of continuous video frames; A2: calculating the Euclidean distance of joint coordinates of each joint in adjacent video frames, taking the Euclidean distance as the motion amplitude to obtain the motion amplitude of each joint in each video frame; A3: integrating the motion amplitudes of all joints in the high motion region in the plurality of continuous video frames into a high motion region motion amplitude set, and integrating the motion amplitudes of all joints in the low motion region in the plurality of continuous video frames into a low motion region motion amplitude set; A4: fitting a probability density function to the high motion region motion amplitude set by using a kernel density estimation algorithm to obtain the joint motion amplitude probability density of the high motion region, and fitting a probability density function to the low motion region motion amplitude set by using a kernel density estimation algorithm to obtain the joint motion amplitude probability density of the low motion region; A5: linearly combining the two joint motion amplitude probability densities obtained in step A4 to obtain a scaling function, and calculating the scaling function value of the motion amplitude of each joint in each video frame; A6: multiplying the scaling function value of the motion amplitude of each joint in each video frame by the joint coordinate of the joint in the video frame, and taking the obtained operation result as the optimized coordinate of each joint in each video frame; A7: adjusting the picture of each video frame according to the optimized coordinate of each joint in the video frame to obtain the picture adjustment result of the plurality of continuous video frames, and taking the picture adjustment result of the plurality of continuous video frames as the optimized posture data.

5. The gesture-based optimization policy based anomaly detection system of claim 4, wherein, The bidirectional interactive graph self-attention module is configured to perform the following steps: B1: generating a topological structure adjacency matrix representing the physical structure of the internal connection of the human body based on the natural connection relationship of the human body skeleton, constructing a motion saliency adjacency matrix representing whether there is a connection between two joints and the connection strength based on joint motion saliency prior, and setting an initial adaptive adjacency matrix; B2: performing mapping processing of the first part by the self-attention mechanism to obtain a self-attention output; B3: sequentially performing global average pooling processing, one-dimensional convolution processing, Gaussian error linear activation processing, two-dimensional convolution processing, and S-shaped activation function processing on the self-attention output to obtain channel importance features; B4: multiplying the initial adaptive adjacency matrix by the channel importance features to obtain an adaptive adjacency matrix; B5: performing a graph convolution operation based on the topological structure adjacency matrix, a graph convolution operation based on the motion saliency adjacency matrix and a graph convolution operation based on the adaptive adjacency matrix on the first part respectively, and summing the results of the graph convolution operations to obtain spatial features; B6: sequentially performing global average pooling processing, convolution processing and S-shaped activation function processing on the spatial features to obtain spatial attention weights; B7: multiplying the self-attention output by the spatial attention weights from the left to obtain self-attention features; B8: summing the spatial features and the self-attention features to obtain a scaling factor and a translation amount.

6. A method for anomaly behavior detection based on posture optimization strategy, characterized in that, The abnormal behavior detection system based on the posture optimization strategy according to any one of claims 1-5, comprising the following steps: S1: extracting pedestrian posture data of a specified pedestrian from a video of the pedestrian through a tracking device; the pedestrian posture data comprises video frames and joint coordinates of multiple joint nodes of the body in the video frames; S2: clustering all joint nodes involved in the pedestrian posture data into high motion areas and low motion areas by a user through an optimization device; S3: obtaining joint motion amplitude probability densities of the high motion areas and the low motion areas respectively by using a kernel density estimation algorithm through the optimization device, and performing posture optimization based on probability density adaptive scaling on each joint coordinate to obtain optimized posture data; S4: converting the optimized posture data into probability distribution data through a detection device, and then obtaining an abnormal behavior detection result according to a comparison result of the probability distribution data and normal behavior detection data.

Images