A human posture anomaly detection method based on multi-branch spatio-temporal modeling

Abstract

A human posture anomaly detection method based on multi-branch spatio-temporal modeling, the method first designs a multi-branch local-global feature fusion module, by constructing a local feature extraction branch for each sub-region of the body, the limbs, torso and other parts of the human body are modeled in detail posture feature, and the dynamic fusion of local and global features is realized by combining the self-adaptive weighting mechanism, forming a multi-level feature representation system, thereby simultaneously improving the detection accuracy of sudden local anomalies and global posture anomalies; At the same time, a time sequence attention enhancement module is proposed, which adopts a time sequence attention mechanism that integrates bidirectional recurrent units, and by parallelly integrating forward and backward time sequence context information, the modeling capability of the model for long time span posture dynamic changes is strengthened, and the recognition robustness for persistent abnormal postures is significantly improved. The method shows superior performance in local sudden anomaly and persistent anomaly detection tasks.

Description

Technical Field

[0001] This invention relates to a method for detecting abnormal human posture, specifically a method for detecting abnormal human posture based on multi-branch spatiotemporal modeling, belonging to the field of video anomaly detection technology. Background Technology

[0002] With the rapid development of intelligent technologies, abnormal behavior detection has become a key technical means to improve the effectiveness of public safety management. In various complex scenarios, such as densely populated public places (including tourist attractions, transportation hubs, and commercial centers) and complex industrial environments (such as production workshops, warehousing and logistics facilities, and energy facilities), traditional monitoring methods face severe challenges. The dynamic changes in pedestrian flow, occlusion interference, and behavioral diversity in public scenes, as well as mechanical noise, complex backgrounds, and equipment interference in industrial environments, all significantly complicate anomaly detection. Traditional methods based on manual monitoring and simple motion detection algorithms not only require substantial human resources but also exhibit significant limitations when facing complex tasks such as multi-target tracking and identification of sudden abnormal events. Especially in open and highly dynamic environments, traditional methods often have detection blind spots, making it difficult to achieve 24 / 7, all-round security monitoring, leading to missed detection of critical abnormal behaviors and creating safety hazards.

[0003] Existing anomaly detection methods are mainly divided into two categories: visual feature detection based on video frames and behavior feature detection. Visual feature detection methods based on video frames typically identify abnormal behavior by detecting visual information such as inter-frame differences and optical flow. Although these methods perform well in certain application scenarios, their performance is unstable in scenarios with a large number of abnormal events. Behavior feature detection methods detect abnormal behavior by analyzing human behavior trajectories and posture changes. These methods are more robust to environmental changes, but using this behavior feature extraction method in public scenarios often leads to complex image processing algorithms, high computational cost, and poor real-time performance.

[0004] With the development of deep learning technology, skeletal point-based behavioral feature extraction methods have gradually attracted attention. Skeletal points are key points on the human body that can accurately reflect a person's posture and pose, and behavioral features can be effectively extracted through skeletal points. In public scenes, skeletal point-based video anomaly detection, compared to traditional anomaly detection methods that require a large amount of abnormal data, can effectively detect abnormal behavior by modeling normal behavior. However, existing methods are not effective for some subtle limb postures and continuous abnormal postures, and are prone to misjudging them as normal behavior. Summary of the Invention

[0005] The purpose of this invention is to provide a human posture anomaly detection method based on multi-branch spatiotemporal modeling. By enhancing local features and optimizing temporal modeling, it can improve the detection accuracy of sudden limb anomalies while effectively improving the ability to identify continuous abnormal postures.

[0006] To achieve the above objectives, this invention provides a method for detecting human pose anomalies based on multi-branch spatiotemporal modeling, comprising the following steps:

[0007] S1: Video data is acquired using a camera, and a skeletal point extraction network is used to extract skeletal point map data X = {B, T, V, C} from the video sequence, where B represents the number of batch samples, T represents the number of video frames, V represents the number of keypoints, C represents the keypoint coordinates (x, y), and an adjacency matrix A representing the connection relationships of skeletal points. The information representing the skeletal point coordinates in the extracted skeletal point map data is expanded by one dimension to obtain the skeletal point coordinate information. For the skeletal point map data, there are 17 keypoints V. 17 ={P0, P1, ... P 16}, New neck key point P 17 It is the midpoint between the left shoulder P5 and the right shoulder P6. Then, the skeletal point map data is activated, normalized, and channel-permuted to obtain the skeletal point data X′.

[0008] S2: Spatial segmentation is performed on the normalized and channel-permutated skeletal point data X′ to obtain key point data representing the upper and lower body; spatial graph convolutional networks are constructed for the two sets of key points for the upper and lower body respectively, and local spatial features of the upper and lower body are captured by independent graph structure modeling. Then, the local spatial features of the upper and lower body are weighted and fused with the global spatial features extracted from the skeletal point data X′ to form a joint feature representation with multi-level spatial expression capabilities.

[0009] S3: A bidirectional temporal modeling strategy is adopted. For the bidirectional recurrent unit, the temporal dependencies of the skeletal point data X′ are calculated by forward and backward recurrent neural networks respectively, so as to obtain the bidirectional temporal score containing long-distance temporal context information. For the attention module, local one-dimensional convolution and global temporal pooling operations are used in parallel in the temporal dimension to establish short-term receptive fields and global receptive fields respectively, and local abnormal patterns and global salient features are extracted to capture instantaneous abnormal pose patterns and overall motion trends. Finally, the bidirectional temporal score, local abnormal patterns and global salient features are fused to generate a comprehensive attention score.

[0010] S4: The attention score calculated in step S3 is weighted and fused with the joint feature representation extracted in step S2 to obtain the final feature used to calculate the anomaly score, thereby enhancing the expressive power of key spatiotemporal features. Subsequently, the features after weighted fusion of steps S3 and S2 are mapped to the latent space through a normalized flow architecture to obtain the updated pose sequence. The anomaly score is calculated based on probability density estimation. If the score exceeds the preset detection threshold, the current input skeleton point map data X is determined to be abnormal behavior.

[0011] The specific steps of the fusion feature generation method in step S2 of this invention are as follows:

[0012] S21: For the skeletal point data X′ processed in step S1, in order to distinguish local pose features, the skeletal point data X′ is divided into two sub-parts, the upper body and the lower body, and features are extracted from each sub-part.

[0013] X′1,X′2=split(X′,dim=-2)

[0014] Where: X′1 represents the joint features of the upper body;

[0015] X′2 represents the joint features of the lower body;

[0016] S22: In the process of extracting local pose features of the upper and lower body, graph convolution operation (STGCN) is performed on the poses of the upper and lower body respectively to extract the spatiotemporal features of the local pose. For the upper body, its corresponding skeleton connection adjacency matrix A1 is defined, and the upper body features are extracted by graph convolution operation (STGCN):

[0017] H′1=STGCN1(X′1,A1·I1)

[0018] Similarly, for the lower body, we define its corresponding skeletal adjacency matrix A2, and extract the lower body features using graph convolution (STGCN) operation:

[0019] H′2=STGCN1(X′2,A2·I2)

[0020] Where: I1 and I2 represent the corresponding importance matrices, which control the important weights of local skeleton connections;

[0021] S23: After extracting local features through graph convolution (STGCN), the features H′1 representing the upper body and H′2 representing the lower body are concatenated to form a comprehensive local pose feature representation.

[0022] H concat =concat(H′1,H′2,dim=-2)

[0023] Subsequently, the feature dimensions are further adjusted using a multilayer perceptron to generate the fused local feature representation H. fused :

[0024] H fused =MLP(H concat );

[0025] S24: Global pose features are extracted using GCN. To combine local and global pose features, weighted fusion is used. By learning weight parameters, the Graph Convolutional Network (GCN) can be adjusted according to the importance of local and global pose features in different scenarios.

[0026] H′=α·GCN(X)+β·H fused

[0027] Where: α and β are learnable weight parameters, α+β=1, to ensure that the fused features remain intact;

[0028] H′ represents the overall feature resulting from the fusion of local and global pose features.

[0029] This approach allows for dynamic adjustment of the contributions of local and global features, making the model more flexible for different scenarios.

[0030] The specific steps of the temporal attention generation method in step S3 of this invention are as follows:

[0031] S31: For the input skeletal point data X′∈RB ×C×T×V The skeletal point data X′ is transformed in dimension through a fully connected layer, adjusting it into an intermediate representation X. L ∈R B×C×T×M Where M represents the number of neurons in the bidirectional recurrent unit, and M is the set number of hidden units in the bidirectional recurrent neural network. Subsequently, for each independent time step t, the corresponding feature slice x(t)∈R is extracted. B×C×1×M The data are then fed into forward-propagating GRU units and backward-propagating GRU units respectively for temporal feature modeling. These two recurrent neural networks, operating in opposite directions, capture the temporal contextual dependencies of the sequence data. The forward GRU processes historical information from the beginning to the current time step, while the backward GRU processes future information from the end to the current time step, resulting in hidden information in both directions. Finally, by fusing the hidden information from these two directions, a new memory information F is formed. L ,Right now:

[0032] F L =[l(1),l(2),…,l(T)] T ∈R B×C×T×M

[0033] in: This represents the memory information of the t-th frame;

[0034] and The outputs are the forward and backward loop units, respectively;

[0035] S32: For the attention calculation module, the skeletal point map data X is first up-dimensioned. Then, a one-dimensional convolution is used to establish a local receptive field in the time dimension to capture short-term anomalous patterns. At the same time, max pooling is used to extract global salient features in the temporal sequence. The short-term anomalous features and global salient features are fused after dimensionality reduction and fully connected layer transformation. Finally, the attention weights F are generated through the sigmoid function. A The specific formula is expressed as follows:

[0036] F A =σ(W2Conv1d(W1X)+W3MaxPool(W1X))∈R B×C×T×V

[0037] Where: W1 is the upgraded fully connected layer;

[0038] W2 and W3 represent dimension-reduced fully connected layers;

[0039] σ refers to the sigmoid function;

[0040] The memory information and attention score are fused by element-wise multiplication, and the dimensions are adjusted to obtain a comprehensive attention score through residual connection. The comprehensive attention score is then weighted onto the skeletal point data X′ to obtain the skeletal attention feature X''.

[0041] The specific steps of the abnormal score calculation method in step S4 of this invention are as follows:

[0042] S41: Fuse the skeletal attention feature X'' from step S3 with the global feature H' from S2 to obtain the feature tensor G'. Divide the feature tensor G' into two subsets, shift and scale:

[0043] G′→(shift, scale);

[0044] S42: First, calculate the global average pooling value of the feature tensor G′, and after processing it with the tanh activation function, add 1 to obtain the dynamic temperature coefficient. Based on this, divide the original scaling parameter scale by the temperature coefficient τ and then feed it into the Sigmoid function for nonlinear transformation, multiply by 2, and add a factor derived from the minimum value 1e. -6 The bias term, composed of the learnable noise suppression parameter β, ultimately yields the optimized scaling factor:

[0045] τ=1+tanh(GlobalAvgBool(G′))

[0046]

[0047] S43: Residual feature calibration is employed.

[0048] G”=(G′+α·shift)⊙scale_adj

[0049] α is the adaptive decay factor: α = exp(-||shift||2)

[0050] S44: The log-determinant' calculation updates the determinant logdet by taking the logarithm of the scale and summing over the specified dimension.

[0051]

[0052] S45: Using G ” The probability density score representing the anomaly is calculated using logdet′, and the threshold is calculated using a dynamic thresholding mechanism.

[0053] threshold=Percentile(anomaly_score_history,95%)+δ·MAD

[0054] score=1 / 2(G”^TΣ^(-1)G”+log|Σ|)-logdet′

[0055] MAD represents the mean absolute deviation of the current window score;

[0056] δ represents the sensitivity coefficient;

[0057] The final calculated anomaly score is compared with a threshold. If the anomaly score is greater than the threshold, it indicates that there is an anomaly in the current skeletal point map data X.

[0058] Compared to existing technologies, the skeletal point-based detection method of this invention focuses primarily on local human posture and continuous abnormal behavior. It analyzes key point data of the human body, independent of the scene background, thus exhibiting stronger robustness to environmental changes. In open and dynamically changing environments such as public places, skeletal point detection can more stably identify human behavior. Furthermore, the skeletal point method can capture the specific posture of the human body more meticulously, especially in complex crowd scenes, effectively distinguishing different types of behavior. For example, abnormal behaviors such as fighting, running, and falling can be identified through skeletal point data, revealing the specific posture form without being easily disturbed by the background. Moreover, for dynamic scenes like public places, skeletal point-based anomaly detection only requires analyzing the position and relative changes of key points on the human body, significantly reducing computational load compared to traditional pixel-level image processing. Therefore, in densely populated and complex environments, the skeletal point method can provide higher real-time performance. Attached Figure Description

[0059] Figure 1 Flowchart for the invention;

[0060] Figure 2 Flowchart for enhancing global attitude to local attitude;

[0061] Figure 3 This is a flowchart of the temporal attention process. Detailed Implementation

[0062] In high-risk work environments such as industrial manufacturing, construction, and logistics, workers may experience safety accidents due to sudden violations of operating procedures (such as accidental contact with machinery or dangerous postures) or sustained fatigue postures (such as prolonged bending or repetitive movements). Traditional monitoring relies on manual inspections or simple rules, making it difficult to accurately identify risks associated with complex postures. The human posture anomaly detection method based on multi-branch spatiotemporal modeling proposed in this invention can effectively solve these two problems.

[0063] The invention will now be further described with reference to the accompanying drawings.

[0064] like Figures 1-3 As shown, a method for detecting human pose anomalies based on multi-branch spatiotemporal modeling includes the following steps:

[0065] S1: Video data is collected by cameras installed in the industrial production workshop. Object detection algorithms are used to extract personnel information from the videos. A skeletal point extraction network is then used to extract skeletal point map data X = {B, T, V, C} from the video sequence, where B represents the number of batch samples, T represents the number of video frames, V represents the number of keypoints, C represents the keypoint coordinates (x, y), and an adjacency matrix A representing the connection relationships between skeletal points. The information representing skeletal point coordinates in the extracted skeletal point map data is expanded by one dimension to obtain the skeletal point coordinate information. For the skeletal point map data, there are 17 keypoints V. 17 ={P0, P1, ... P 16}, New neck key point P 17 It is the midpoint between left shoulder P5 and right shoulder P6:

[0066] Expand as

[0067] For the processed skeletal point data obtained after expanding the dimensions, activation normalization is performed. A linear transformation is then applied along the dimension representing the keypoint coordinates C to make its mean zero and standard deviation one. This transformation includes two parameters: translation and scaling. These two parameters can be adjusted through learning.

[0068] X′=s·X+b

[0069] Where: s represents the learnable scaling parameter, b represents the learnable translation parameter, and X′ represents the normalized skeletal point data; to calculate the inverse operation, we have:

[0070]

[0071] The log determinant, another output of activation normalization, is an important metric for measuring the change in transform volume and is used to calculate the log-likelihood of the change in data distribution.

[0072]

[0073] The normalized skeletal point data X′ is subjected to spatial channel swapping and temporal sequence permutation. After permuting T:{t1, t2, ..., tn} with C:{x,y} in the skeletal point map data, the following is obtained:

[0074]

[0075] S2: Spatial segmentation is performed on the normalized and channel-permutated skeletal point data X′ to obtain key point data representing the upper and lower body, which are divided into two sets of key points: upper body and lower body. Spatial graph convolutional networks are constructed for the two sets of key points, respectively, and local spatial features of the upper and lower body are captured by independent graph structure modeling. Then, the local spatial features of the upper and lower body are weighted and fused with the global spatial features extracted from the skeletal point data X′ to form a joint feature representation with multi-level spatial expression capabilities.

[0076] The specific steps of the fusion feature generation method in step S2 are as follows:

[0077] S21: For the skeletal point data X′ processed in step S1, in order to distinguish local pose features, the pose sequence is divided into two sub-parts, the upper body and the lower body, and features are extracted from each sub-part.

[0078] X′1,X′2=split(X′,dim=-2)

[0079] Where: X′1 represents the joint features of the upper body;

[0080] X′2 represents the joint features of the lower body;

[0081] S22: In the process of extracting local pose features of the upper and lower body, graph convolution operation (STGCN) is performed on the poses of the upper and lower body respectively to extract the spatiotemporal features of the local pose. For the upper body, its corresponding skeleton connection adjacency matrix A1 is defined, and the upper body features are extracted by graph convolution operation (STGCN):

[0082] H′1=STGCN1(X′1,A1·I1)

[0083] Similarly, for the lower body, we define its corresponding skeletal adjacency matrix A2, and extract the lower body features using graph convolution (STGCN) operation:

[0084] H′2=STGCN1(X′2,A2·I2)

[0085] Where: I1 and I2 represent the corresponding importance matrices, which control the important weights of local skeleton connections;

[0086] S23: After extracting local features through graph convolution (STGCN), the features H′1 representing the upper body and H′2 representing the lower body are concatenated to form a comprehensive local pose feature representation.

[0087] H concat =concat(H′1,H′2,dim=-2)

[0088] Subsequently, the feature dimensions are further adjusted using a multilayer perceptron to generate the fused local feature representation H. fused :

[0089] H fused =MLP(H concat );

[0090] S24: Global pose features are extracted using GCN. To combine local and global pose features, weighted fusion is used. By learning weight parameters, the Graph Convolutional Network (GCN) can be adjusted according to the importance of local and global pose features in different scenarios.

[0091] H′=α·GCN(X)+β·H fused

[0092] Where: α and β are learnable weight parameters, α+β=1, to ensure that the fused features remain intact;

[0093] This approach allows for dynamic adjustment of the contributions of local and global features, making the model more flexible for different scenarios.

[0094] S3: To address the persistent fatigue postures that workers may experience (such as prolonged bending and repetitive movements), a bidirectional temporal modeling strategy is adopted. For bidirectional recurrent units, the temporal dependencies of skeletal point data X′ are calculated through forward and backward recurrent neural networks, thereby obtaining bidirectional temporal scores containing long-distance temporal context information. For the attention module, local one-dimensional convolution and global temporal pooling operations are used in parallel in the temporal dimension to establish short-term receptive fields and global receptive fields, respectively, to extract local abnormal patterns and global salient features to capture instantaneous abnormal posture patterns and overall movement trends. Finally, the bidirectional temporal scores, local abnormal patterns, and global salient features are fused to generate a comprehensive attention score.

[0095] The specific steps of the temporal attention generation method in step S3 are as follows:

[0096] S31: For the input skeletal point data X′∈R B×C×T×V The skeletal point data X′ is transformed in dimension through a fully connected layer, adjusting it into an intermediate representation X. L ∈R B×C×T×M Where M represents the number of neurons in the bidirectional recurrent unit, and M is the set number of hidden units in the bidirectional recurrent neural network. Subsequently, for each independent time step t, the corresponding feature slice x(t)∈R is extracted. B×C×1×MThe data are then fed into forward-propagating GRU units and backward-propagating GRU units respectively for temporal feature modeling. These two recurrent neural networks, operating in opposite directions, capture the temporal contextual dependencies of the sequence data. The forward GRU processes historical information from the beginning to the current time step, while the backward GRU processes future information from the end to the current time step, resulting in hidden information in both directions. Finally, by fusing the hidden information from these two directions, a new memory information F is formed. L ,Right now:

[0097] F L =[l(1),l(2),…,l(T)] T ∈R B×C×T×M

[0098] in: This represents the memory information of the t-th frame;

[0099] and The outputs are the forward and backward loop units, respectively;

[0100] S32: For the attention calculation module, the skeletal point map data X is first up-dimensioned. Then, a one-dimensional convolution is used to establish a local receptive field in the time dimension to capture short-term anomalous patterns. At the same time, max pooling is used to extract global salient features in the temporal sequence. The short-term anomalous features and global salient features are fused after dimensionality reduction and fully connected layer transformation. Finally, the attention weights F are generated through the sigmoid function. A The specific formula is expressed as follows:

[0101] F A =σ(W2Conv1d(W1X)+W3MaxPool(W1X))∈R B×C×T×V

[0102] Where: W1 is the upgraded fully connected layer;

[0103] W2 and W3 represent dimension-reduced fully connected layers;

[0104] σ refers to the sigmoid function;

[0105] The memory information and attention score are fused by element-wise multiplication, and the dimensions are adjusted to obtain a comprehensive attention score through residual connection. Then, the comprehensive attention score is weighted onto the skeletal point data X′ to obtain X''.

[0106] S4: The attention score calculated in step S3 is weighted and fused with the joint feature representation extracted in step S2 to obtain the final feature used to calculate the anomaly score, thereby enhancing the expressive power of key spatiotemporal features. Subsequently, the features after weighted fusion of steps S3 and S2 are mapped to the latent space through a normalized flow architecture to obtain the updated pose sequence. The anomaly score is calculated based on probability density estimation. If the score exceeds the preset detection threshold, the current input skeleton point map data X is determined to be abnormal behavior.

[0107] The specific steps for calculating the outlier score in step S4 are as follows:

[0108] S41: Fuse the skeletal attention feature X'' from step S3 with the global feature H' from S2 to obtain the feature tensor G'. Divide the feature tensor G' into two subsets, shift and scale:

[0109] G′→(shift, scale);

[0110] S42: First, calculate the global average pooling value of the feature tensor G′, and after processing it with the tanh activation function, add 1 to obtain the dynamic temperature coefficient. Based on this, divide the original scaling parameter scale by the temperature coefficient τ and then feed it into the Sigmoid function for nonlinear transformation, multiply by 2, and add a factor derived from the minimum value 1e. -6 The bias term, composed of the learnable noise suppression parameter β, ultimately yields the optimized scaling factor:

[0111] τ=1+tanh(GlobalAvgBool(G′))

[0112]

[0113] S43: Residual feature calibration is employed.

[0114] G″=(G′+α·shift)⊙scale_adj

[0115] α is the adaptive decay factor: α = exp(-||shift||2)

[0116] S44: The log-determinant' calculation updates the determinant logdet by taking the logarithm of the scale and summing over the specified dimension.

[0117]

[0118] S45: Calculate the probability density score representing the anomaly using G″ and logdet′, and calculate the threshold using a dynamic thresholding mechanism;

[0119] threshold=Percentile(anomaly_score_history,95%)+δ·MAD

[0120] score=1 / 2(G″^TΣ^(-1)G″+log|Σ|)-logdet′

[0121] MAD represents the mean absolute deviation of the current window score;

[0122] δ represents the sensitivity coefficient;

[0123] The final calculated anomaly score is compared with a threshold. If the anomaly score is greater than the threshold, it indicates that there is an anomaly in the current skeletal point map data X.

Claims

1. A method for detecting human pose anomalies based on multi-branch spatiotemporal modeling, characterized in that, Includes the following steps: S1: Use a camera to capture video data and employ a skeletal point extraction network to extract skeletal point map data from the video sequence. ,in Represents the number of samples in a batch. Represents the number of video frames. Represents the number of key points. Represents the coordinates of key points and the adjacency matrix representing the connection relationships between skeletal points. The information representing the coordinates of bone points in the extracted skeletal point map data is expanded by one dimension to obtain the bone point coordinate information. There are 17 key points in the skeletal point map data. New key points added to the neck It's the left shoulder. and right shoulder The midpoint is then used to perform activation normalization and channel permutation on the skeletal point map data to obtain the skeletal point data. ; S2: Normalized and channel-permutated skeletal point data Spatial segmentation is performed to obtain keypoint data representing the upper and lower body. Spatial graph convolutional networks are constructed for each of the two keypoint sets, capturing local spatial features of the upper and lower body through independent graph structure modeling. Subsequently, these local spatial features are combined with the skeletal point data. The extracted global spatial features are weighted and fused to form a joint feature representation with multi-level spatial representation capabilities; S3: A bidirectional temporal modeling strategy is adopted. For bidirectional recurrent units, skeletal point data are calculated separately through forward and backward recurrent neural networks. The temporal dependencies are identified to obtain bidirectional temporal scores containing long-distance temporal context information. For the attention module, local one-dimensional convolution and global temporal pooling operations are used in parallel in the temporal dimension to establish short-term receptive fields and global receptive fields respectively, extracting local abnormal patterns and global salient features to capture instantaneous abnormal pose patterns and overall motion trends. Finally, the bidirectional temporal scores, local abnormal patterns and global salient features are fused to generate a comprehensive attention score. S4: The attention score calculated in step S3 is weighted and fused with the joint feature representation extracted in step S2 to obtain the final feature used to calculate the anomaly score, enhancing the expressive power of key spatiotemporal features. Subsequently, the weighted fused features from steps S3 and S2 are mapped to the latent space through a normalized flow architecture to obtain the updated pose sequence, and the anomaly score is calculated based on probability density estimation. If the score exceeds a preset detection threshold, the current input skeletal point map data is determined to be... Abnormal behavior exists within; The specific steps of the temporal attention generation method in step S3 are as follows: S31: For the input skeletal point data , through a fully connected layer on the skeletal point data Perform a dimensional transformation to adjust it into an intermediate representation. ,in This represents the number of neuron units in the bidirectional recurrent unit. This refers to the number of hidden units set in a bidirectional recurrent neural network. Then, for each independent time step... Extract the corresponding feature slices The data are then fed into forward propagation GRU units and backward propagation GRU units respectively for temporal feature modeling. These two recurrent neural networks in opposite directions capture the temporal contextual dependencies of the sequence data. The forward GRU processes historical information from the beginning to the current time, while the backward GRU processes future information from the end to the current time, thus obtaining hidden information in two directions. Finally, by fusing the hidden information from these two directions, a new memory information is formed. ,Right now: ,in: Indicates the first The memory information of a frame; and The outputs are the forward and backward loop units, respectively; S32: For the attention calculation module, first, the skeletal point map data... The system then employs dimensionality enhancement, followed by the use of one-dimensional convolution to establish a local receptive field in the temporal dimension, capturing short-term anomalous patterns. Simultaneously, max pooling is used to extract temporally significant global salient features. These short-term anomalous features and global salient features are then fused after dimensionality reduction via a fully connected layer. Finally, attention weights are generated using the sigmoid function. The specific formula is expressed as follows: ,in: It is a fully connected layer of higher dimension; and This represents a dimension-reduced fully connected layer; refer to function; The memory information and attention score are fused through element-wise multiplication, and the dimensionality is adjusted to obtain a comprehensive attention score through residual connections. The comprehensive attention score is then weighted onto the skeletal point data. Obtain skeletal attention features .

2. The method for detecting human posture anomalies based on multi-branch spatiotemporal modeling according to claim 1, characterized in that, The specific steps of the fusion feature generation method in step S2 are as follows: S21: For the bone point data processed in step S1 To distinguish local pose features, the skeletal point data The body is divided into two sub-parts: the upper body and the lower body, and features are extracted from each sub-part. , ,in: Indicates the joint features of the upper body; Indicates the joint features of the lower body; S22: In the process of extracting local pose features of the upper and lower body, STGCN graph convolution operation is performed on the poses of the upper and lower body respectively to extract the spatiotemporal features of the local pose. For the upper body, the corresponding skeleton connection adjacency matrix is ​​defined. And STGCN extracts upper body features through graph convolution operation: Similarly, for the lower body, define its corresponding skeletal adjacency matrix. And STGCN extracts lower body features through graph convolution operation: ,in: and These represent the corresponding importance matrices, controlling the important weights of local skeleton connections; S23: After STGCN extracts local features through graph convolution operations, it will represent the upper body features. and lower body features The data is then stitched together to form a comprehensive representation of the local pose features. Subsequently, the feature dimensions are further adjusted using a multilayer perceptron to generate a fused local feature representation. : ； S24: Global pose features are extracted using GCN. To combine local and global pose features, weighted fusion is used. By learning weight parameters, GCN adjusts its settings according to the importance of local and global pose features in different scenarios. ,in: and These are learnable weight parameters; This represents the overall feature resulting from the fusion of local and global pose features.

3. The method for detecting human posture anomalies based on multi-branch spatiotemporal modeling according to claim 1, characterized in that, The specific steps for calculating the outlier score in step S4 are as follows: S41: Apply the skeletal attention features from step S3 Overall characteristics of S2 The feature tensor is obtained by fusion ,Will Divided into two subsets and : ； S42: First calculate The global average pooling value is processed by the tanh activation function and then incremented by 1 to obtain the dynamic temperature coefficient. Based on this, the original scaling parameters are... Divide by the temperature coefficient τ, feed into the Sigmoid function for nonlinear transformation, multiply by 2, and add a factor derived from the minimum value 1e. -6 The bias term, composed of the learnable noise suppression parameter β, ultimately yields the optimized scaling factor: ， ； S43: Residual feature calibration is employed. G'' = (G' + α·shift) ⊙ scale_adj α is the adaptive decay factor: S44: The calculation updates the determinant logdet by taking the logarithm of the scale and summing over the specified dimension: S45: Use and Calculate the probability density score representing the anomaly, and then calculate the threshold using a dynamic thresholding mechanism; threshold = Percentile(anomaly_score_history, 95%) + ·MAD score = 1 / 2(G''^TΣ^(-1)G'' + log|Σ|) - log ' MAD represents the mean absolute deviation of the current window score; Indicates the sensitivity coefficient; The final calculated anomaly score is compared with a threshold. If the anomaly score is greater than the threshold, it indicates that there is an anomaly in the current sequence.

Images