A skeletal point-based violent action anomaly detection method

Abstract

A kind of skeleton point-based violent action anomaly detection method, the space-time relationship between video frames is modeled using graph convolution network. In order to let the network better distinguish normal and abnormal actions, the input of graph convolution is embedded with velocity features. Since the traditional graph convolution network ignores the length and direction of the skeleton points and the spatial relationship between non-adjacent skeleton points, an adaptive graph convolution network is used to replace the adjacency matrix representing the relationship between the skeleton point nodes with a learnable parameter matrix, and the Euclidean distance between non-adjacent skeleton points is modeled. The output of GCN is modeled as a high-dimensional distribution using a normalized flow model, which maps complex distribution data to a simple and easy-to-handle distribution, and the model is converged by minimizing the negative log-likelihood. It can effectively capture the complex distribution of normal behavior in the video, thus showing higher sensitivity when detecting abnormal behavior.

Description

Technical Field

[0001] The present invention relates to a motion anomaly detection method, in particular to a violent motion anomaly detection method based on skeleton points, and belongs to the technical field of video anomaly detection. Background Art

[0002] With the development of intelligent monitoring systems, abnormal behavior detection has become a core technology for improving public safety management. Intelligent monitoring systems play a crucial role, particularly in densely populated and complex environments like scenic spots. Traditional monitoring methods, relying on manual monitoring and simple motion detection algorithms, are time-consuming and labor-intensive. Furthermore, they often struggle to cope with changing scenarios and unforeseen abnormal events in complex environments. In open, densely populated public spaces like scenic spots, traditional monitoring methods not only fail to effectively improve safety management efficiency but can also miss many critical abnormal behaviors, preventing safety hazards from being discovered in a timely manner.

[0003] Existing anomaly detection methods are mainly divided into two categories: video frame-based visual feature detection and behavioral feature-based detection. Video frame-based visual feature detection methods typically identify abnormal behavior by detecting visual information such as inter-frame differences and optical flow. Although these methods perform well in certain application scenarios, they are unstable in scenarios such as scenic spots where there are a large number of abnormal events. Behavioral feature-based detection methods detect abnormal behavior by analyzing human behavior trajectories and posture changes. These methods are highly robust to environmental changes, but using these behavioral feature extraction methods in scenic spots often leads to complex image processing algorithms, high computational load, and poor real-time performance.

[0004] With the development of deep learning technology, behavioral feature extraction methods based on skeleton points have gradually attracted attention. Skeleton points are key points on the human body that can accurately reflect a person's posture and movements. They can effectively extract behavioral features. Skeleton-based video anomaly detection. In scenic spots, compared to traditional anomaly detection methods that require large amounts of abnormal data, skeleton-based detection methods can also effectively detect abnormal behavior by modeling normal behavior. However, existing methods have poor recognition performance for some violent movements with large body amplitude changes, such as running, jumping, and fighting, and are easily misclassified as normal behavior. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for detecting violent motion anomalies based on skeleton points, which can improve the accuracy and robustness of the anomaly detection model while ensuring real-time reasoning speed.

[0006] In order to achieve the above object, the present invention provides a method for detecting violent motion anomalies based on skeleton points, comprising the following steps:

[0007] S1: Use a camera to collect video data, and use a skeleton point extraction network to extract skeleton point graph data G = {b, f, k, d} from the video sequence of the video data, where b represents the number of batch samples, f represents the number of video frames, k represents the number of key points, d represents the key point coordinates (x, y) and the adjacency matrix A representing the connection relationship of the skeleton points;

[0008] S2: Expand the information representing the coordinates of the skeleton points in the skeleton point image data extracted in step S1 by one dimension to obtain the coordinate information of the skeleton points. For the skeleton point image data, there are 17 key points K 17 ={P0, P1, ...P 16}, newly added neck key point P 17 It is the midpoint between the left shoulder P5 and the right shoulder P6:

[0009] Expand to

[0010] And normalize the coordinate information of the skeleton points and exchange channels:

[0011]

[0012] S3: Using the skeleton point data obtained in step S2, calculate the difference between each point in adjacent frames and calculate the speed. As speed information, and use the fully connected network to learn the changing law of speed;

[0013] S4: replacing the custom adjacency matrix representing the relationship between bone points in the bone point graph data extracted in step S1 with a learnable parameter matrix;

[0014] S5: For the skeleton point data processed in step S2, calculate the Euclidean distance between non-adjacent skeleton points and model the spatial structure;

[0015] S6: Perform spatiotemporal graph convolution on the skeleton point data processed in step S5 and the learnable adjacency matrix replaced by S4;

[0016] S7: The features extracted by spatiotemporal graph convolution in step S6 are fed into the normalized flow network to calculate the probability density score representing the anomaly.

[0017] The step S3 specifically comprises: for each joint point in the input skeleton point data, obtaining the position coordinates of the joint point in space and calculating the velocity information according to the position coordinates, adding and summing the obtained position and velocity embedded information, and fusing them into a new joint feature;

[0018] The specific steps of obtaining new joint features in step S3 are as follows:

[0019] S31: Input skeleton point graph data, where P represents the coordinates of the skeleton point position = {P it |t∈[1,f],i∈[1,n]}, corresponding to a skeleton sequence with n joints and f frames, velocity information v i,t =P i,t -P i-1,t , the position coordinates and velocity information are encoded into high-dimensional space, namely Ph i,t and vh i,t , where Ph i,t Represents the position information of high-dimensional space, vh i,t Represents the velocity information of high-dimensional space, by summing z i,t =Ph i,t +vh i,t blend them together;

[0020] S32: Embedding information Ph in high-dimensional spatial position i,t For example, use two fully connected layers to i,t Encode to get:

[0021] Ph i,t =σ(W2(σ(W1P i,t +b1))+b2),

[0022] Where: and is the weight matrix;

[0023] b1 and b2 are bias matrices;

[0024] σ is the relu activation function;

[0025] Get high-dimensional spatial velocity embedding features:

[0026] vh i,t =σ(W2(σ(W1v i,t +b1))+b2).

[0027] The specific steps of step S4 are as follows:

[0028] S41: Obtain an adjacency matrix A, where A represents a user-defined adjacency matrix representing the physical structure of the human body;

[0029] S42: Obtain the adjacency matrix B representing the connection strength K , B K is a k×k adjacency matrix. Compared with A, B K The elements of B are parameterized and optimized along with other parameters during training. K There are no constraints on the value of

[0030] S43: Obtaining the adaptive adjacency matrix CK , C K Is a data-dependent adjacency matrix that learns a unique adjacency matrix for each sample. In order to determine whether there is a connection between two bone points and the strength of the connection, a normalized embedding Gaussian function is applied to calculate the similarity of the two bone points:

[0031]

[0032] Where: k is the total number of vertices;

[0033] The dot product is used to measure the similarity between two skeleton points in the embedding space; specifically, given an input feature map fin of size Cin×T×k, it is first transformed into Ce×T×k using two embedding functions, namely θ and Then a 1×1 convolutional layer is selected as the embedding function; the two embedded feature maps are rearranged and reshaped into N×CeT matrix and CeT×T matrix; then they are multiplied to obtain the similarity matrix C of size k×k k , where C k Each element in Represents the bone point v i and bone point v j The similarity of the matrix is ​​normalized to 0-1 and used as the edge of the two skeleton points. Since the normalized Gaussian uses the softmax operation, C k Expressed as:

[0034]

[0035] Where: W θ and are the embedding functions θ and Parameters;

[0036] There is no direct replacement of the original A with B K or C K , but add them to it;

[0037] B K The values ​​of θ and The parameters are initialized to 0, so that the flexibility of the model is enhanced without reducing the original performance, and the adaptive network formula is obtained:

[0038]

[0039] The specific steps of step S5 are as follows:

[0040] S51: For input C×T×k, T represents the number of frames in a sequence, k represents the number of bone points (vertices), and C is 2, which represents the 2D coordinates x and y of the corresponding bone point in the image;

[0041] In order to calculate the Euclidean distance, we need to obtain the coordinates of the two skeleton points x1, y1 and x2, y2. The formula is:

[0042] distance = (x2 - x1) 2 +(y2-y1) 2

[0043] The distance finally obtained is a matrix with a dimension of (T-1, N), in which each element represents the distance between a skeleton point and other skeleton points in the current frame;

[0044] S52: In order to obtain the weight matrix Wd representing the spatial structure of the bone points, it is necessary to calculate the average of the sum of each bone point and all other bone points, and subtract the Euclidean distance between each bone point and all other bone points from this average, and then calculate the maximum value. For a certain bone point, the formula is:

[0045] αi,j=max(ee(i,j))

[0046] Where: αi,j represents the spatial relationship between node i and node j;

[0047] e represents the matrix of average Euclidean distances between pairwise nodes in the sequence.

[0048] The specific steps of step S7 are as follows:

[0049] S71: The model of the normalization flow processes the pose sequence data directly. It learns an invertible mapping between the data distribution PX and the potential distribution PZ (Gaussian Z ~ N(μnormal, I)). The architecture includes K steps, each of which consists of three layers: Actnorm, permutation and affine coupling layers. Actnorm is an activation normalization layer that uses the input data to obtain scaling factors and offsets, similar to batch normalization.

[0050] S72: For the input C×T×N, where C is the number of input channels and represents the coordinates of the bone point (x, y), the permutation layer permutes the order of the input channels, i.e. (x, y) → (y, x);

[0051] S73: For the data that has passed the permutation layer, half of the dimension C remains unchanged, while the other half is affine transformed based on the other half. This results in a simple inverse transformation and Jacobian calculation. Compared with performing an affine transformation on the entire dimension C during backpropagation, the amount of calculation is reduced. The final calculated anomaly score is compared with the threshold. When the anomaly score is greater than the threshold, it indicates that there is an anomaly in the current sequence.

[0052] Compared with the existing technology, the detection method based on skeleton points of the present invention mainly focuses on the posture and movement of people, and performs analysis through the key point data of the human body. It does not rely on the scene background and is therefore more robust to changes in the environment. In an open and dynamically changing environment such as a scenic spot, skeleton point detection can more stably identify human behavior. At the same time, the use of the skeleton point method can capture the specific movements of the human body in more detail, especially in complex crowd scenes, and can effectively distinguish different types of behavior. For example, abnormal behaviors such as fighting, running, and falling can be identified through skeleton point data in the specific form of the action, without being easily interfered by the background. Moreover, for dynamic scenes such as scenic spots, anomaly detection based on skeleton points only needs to analyze the position and relative changes of the key points of the human body. Compared with traditional image pixel-level processing, the amount of calculation is significantly reduced. Therefore, in scenes such as scenic spots with dense crowds and complex environments, the skeleton point method can provide higher real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of the invention;

[0054] Figure 2 Flowchart for fusing velocity features and spatial structure matrix;

[0055] Figure 3 Flowchart for obtaining a learnable adjacency matrix. DETAILED DESCRIPTION

[0056] The present invention will be further described below with reference to the accompanying drawings.

[0057] like Figure 1 As shown in FIG, a method for detecting violent motion anomalies based on skeleton points is provided, wherein the acquisition and preprocessing normalization of skeleton point image data include two steps S1 and S2:

[0058] S1: Use a camera to collect video data, and use a skeleton point extraction network to extract skeleton point graph data G = {b, f, k, d} from the video sequence of the video data, where b represents the number of batch samples, f represents the number of video frames, k represents the number of key points, d represents the key point coordinates (x, y) and the adjacency matrix A representing the connection relationship of the skeleton points;

[0059] S2: Expand the information representing the coordinates of the skeleton points in the skeleton point graph data extracted in step S1 by one dimension to obtain the skeleton point information. For the skeleton point graph data, there are 17 key points K 17 ={P0, P1, ...P 16}, newly added neck key point P 17 It is the midpoint between the left shoulder P5 and the right shoulder P6:

[0060] Expand to

[0061] And normalize the coordinate information of the skeleton points and exchange channels:

[0062]

[0063] like Figure 2 As shown, step 3 specifically includes:

[0064] S3: Using the skeleton point data obtained in step S2, calculate the difference between each point in adjacent frames and calculate the speed. As speed information, and use the fully connected network to learn the changing law of speed;

[0065] like Figure 3 As shown, step 4 specifically includes:

[0066] S4: replacing the custom adjacency matrix representing the relationship between bone points in the bone point graph data extracted in step S1 with a learnable parameter matrix;

[0067] like Figure 2 As shown, step 5 specifically includes:

[0068] S5: For the skeleton point data processed in step S2, calculate the Euclidean distance between non-adjacent skeleton points and model the spatial structure;

[0069] S6: Perform spatiotemporal graph convolution on the skeleton point data processed in step S5 and the learnable adjacency matrix replaced by S4;

[0070] S7: The features extracted by spatiotemporal graph convolution in step S6 are fed into the normalized flow network to calculate the probability density score representing the anomaly.

[0071] The step S3 specifically comprises: for each joint point in the input skeleton point data, obtaining the position coordinates of the joint point in space and calculating the velocity information according to the position coordinates, adding and summing the obtained position and velocity embedded information, and fusing them into a new joint feature;

[0072] The specific steps of obtaining new joint features in step S3 are as follows:

[0073] S31: Input skeleton point graph data, where P represents the coordinates of the skeleton point position = {P it |t∈[1,f],i∈[1,n]}, corresponding to a skeleton sequence with n joints and f frames, velocity information v i,t =P i,t -P i-1,t , the position coordinates and velocity information are encoded into high-dimensional space, namely Ph i,t and vh i,t , where Phi,t Represents the position information of high-dimensional space, vh i,t Represents the velocity information of high-dimensional space, by summing z i,t =Ph i,tt +vh i,t blend them together;

[0074] S32: Embedding information Ph in high-dimensional spatial position i,t For example, use two fully connected layers to i,t Encode to get:

[0075] Ph i,t =σ(W2(σ(W1P i,t +b1))+b2),

[0076] Where: and is the weight matrix;

[0077] b1 and b2 are bias matrices;

[0078] σ is the relu activation function;

[0079] Get high-dimensional spatial velocity embedding features:

[0080] vh i,t =σ(W2(σ(W1v i,t +b1))+b2).

[0081] The specific steps of step S4 are as follows:

[0082] S41: Obtain an adjacency matrix A, where A represents a user-defined adjacency matrix representing the physical structure of the human body;

[0083] S42: Obtain the adjacency matrix B representing the connection strength K , B K is a k×k adjacency matrix. Compared with A, B K The elements of B are parameterized and optimized along with other parameters during training. K There are no constraints on the value of

[0084] S43: Obtaining the adaptive adjacency matrix C K , C K Is a data-dependent adjacency matrix that learns a unique adjacency matrix for each sample. In order to determine whether there is a connection between two bone points and the strength of the connection, a normalized embedding Gaussian function is applied to calculate the similarity of the two bone points:

[0085]

[0086] Where: k is the total number of vertices;

[0087] The dot product is used to measure the similarity between two skeleton points in the embedding space; specifically, given an input feature map fin of size Cin×T×k, it is first transformed into Ce×T×k using two embedding functions, namely θ and Then a 1×1 convolutional layer is selected as the embedding function; the two embedded feature maps are rearranged and reshaped into N×CeT matrix and CeT×T matrix; then they are multiplied to obtain the similarity matrix C of size k×k k , where C k Each element in Represents the bone point v i and bone point v j The similarity of the matrix is ​​normalized to 0-1 and used as the edge of the two skeleton points. Since the normalized Gaussian uses the softmax operation, C k Expressed as:

[0088]

[0089] Where: W θ and are the embedding functions θ and Parameters;

[0090] There is no direct replacement of the original A with B K or C K , but add them to it;

[0091] B K The values ​​of θ and The parameters are initialized to 0, so that the flexibility of the model is enhanced without reducing the original performance, and the adaptive network formula is obtained:

[0092]

[0093] The specific steps of step S5 are as follows:

[0094] S51: For input C×T×k, T represents the number of frames in a sequence, k represents the number of bone points (vertices), and C represents the 2D coordinates x, y of the corresponding bone point in the image;

[0095] In order to calculate the Euclidean distance, we need to obtain the coordinates of the two skeleton points x1, y1 and x2, y2. The formula is:

[0096] distance = (x2 - x1) 2 +(y2-y1) 2

[0097] The distance finally obtained is a matrix with a dimension of (T-1, N), in which each element represents the distance between a skeleton point and other skeleton points in the current frame;

[0098] S52: In order to obtain the weight matrix Wd representing the spatial structure of the bone points, it is necessary to calculate the average of the sum of each bone point and all other bone points, and subtract the Euclidean distance between each bone point and all other bone points from this average, and then calculate the maximum value. For a certain bone point, the formula is:

[0099] αi,j=max(ee(i,j))

[0100] Where: αi,j represents the spatial relationship between node i and node j;

[0101] e represents the matrix of average Euclidean distances between pairwise nodes in the sequence.

[0102] The specific steps of step S7 are as follows:

[0103] S71: The model of the normalization flow processes the pose sequence data directly. It learns an invertible mapping between the data distribution PX and the potential distribution PZ (Gaussian Z ~ N(μnormal, I)). The architecture includes K steps, each of which consists of three layers: Actnorm, permutation and affine coupling layers. Actnorm is an activation normalization layer that uses the input data to obtain scaling factors and offsets, similar to batch normalization.

[0104] S72: For the input C×T×N, C is the number of input channels, which is 2 and represents the coordinates of the bone point (x, y). The permutation layer permutes the order of the input channels, i.e. (x, y) → (y, x).

[0105] S73: For the data that has passed the permutation layer, half of the dimension C remains unchanged, while the other half is affine transformed based on the other half. This results in a simple inverse transformation and Jacobian calculation. Compared with performing an affine transformation on the entire dimension C during backpropagation, the amount of calculation is reduced. The final calculated anomaly score is compared with the threshold. When the anomaly score is greater than the threshold, it indicates that there is an anomaly in the current sequence.

[0106] An embodiment of the present invention is given for application in abnormal detection of dynamic environment in scenic spots

[0107] S1: Video data is collected by cameras installed in the scenic area. Skeleton point extraction network is used to extract skeleton point graph data G = {b, f, k, d} from the video sequence of the video data, where b represents the number of batch samples, which is 32, f represents the number of video frames, 64, k represents the number of key points, 17, d represents the coordinates (x, y) of the key points, and the adjacency matrix A representing the connection relationship of the skeleton points has a dimension of {18, 18};

[0108] S2: Expand the information representing the coordinates of the skeleton points in the skeleton point graph data extracted in step S1 by one dimension to obtain the skeleton point information. For the skeleton point graph data, there are 17 key points K 17 ={P0, P1, ...P 16}, newly added neck key point P 17 It is the midpoint between the left shoulder P5 and the right shoulder P6:

[0109] Expand to

[0110] And normalize the coordinate information of the skeleton points and exchange channels:

[0111]

[0112] like Figure 2 As shown, step 3 specifically includes:

[0113] S3: In the scenic area, the limb movement speed of abnormal and normal movements is about 2 to 3 times different. Therefore, the skeleton point data obtained in step S2 is used to calculate the difference between each point in adjacent frames and calculate the speed. As speed information, and use the fully connected network to learn the changing law of speed;

[0114] like Figure 3 As shown, step 4 specifically includes:

[0115] S4: Considering the large flow of people and the frequent body movements in scenic spots, in order to better distinguish normal from abnormal movements, the custom adjacency matrix representing the relationship between skeleton points in the skeleton point graph data extracted in step S1 is replaced with a learnable parameter matrix;

[0116] like Figure 2 As shown, step 5 specifically includes:

[0117] S5: For the skeleton point data processed in step S2, calculate the Euclidean distance between non-adjacent skeleton points and model the spatial structure;

[0118] S6: Perform spatiotemporal graph convolution on the skeleton point data processed in step S5 and the learnable adjacency matrix replaced by S4;

[0119] S7: The features extracted by spatiotemporal graph convolution in step S6 are fed into the normalized flow network to calculate the probability density score representing the anomaly.

[0120] The step S3 specifically comprises: for each joint point in the input skeleton point data, obtaining the position coordinates of the joint point in space and calculating the velocity information according to the position coordinates, adding and summing the obtained position and velocity embedded information, and fusing them into a new joint feature;

[0121] The specific steps of obtaining new joint features in step S3 are as follows:

[0122] S31: Input skeleton point graph data, where P represents the coordinates of the skeleton point position = {P it |t∈[1,f],i∈[1,n]}, corresponding to a skeleton sequence with n joints and f frames, velocity information v i,t =P i,t -P i-1,t , the position coordinates and velocity information are encoded into high-dimensional space, namely Ph i,t and vh i,t , where Ph i,t Represents the position information of high-dimensional space, vh i,t Represents the velocity information of high-dimensional space, by summing z i,t =Ph i,t +vh i,t blend them together;

[0123] S32: Embedding information Ph in high-dimensional spatial position i,t For example, use two fully connected layers to i,t Encode to get:

[0124] Ph i,t =σ(W2(σ(W1P i,t +b1))+b2),

[0125] Where: and is the weight matrix;

[0126] b1 and b2 are bias matrices;

[0127] σ is the relu activation function;

[0128] Get high-dimensional spatial velocity embedding features:

[0129] vh i,t =σ(W2(σ(W1v i,t +b1))+b2).

[0130] The specific steps of step S4 are as follows:

[0131] S41: Obtain an adjacency matrix A, where A represents a user-defined adjacency matrix representing the physical structure of the human body;

[0132] S42: Obtain the adjacency matrix B representing the connection strength K , B K is an 18×18 adjacency matrix. Compared with A, B K The elements of B are parameterized and optimized along with other parameters during training. K There are no constraints on the value of

[0133] S43: Obtaining the adaptive adjacency matrix C K , C K Is a data-dependent adjacency matrix that learns a unique adjacency matrix for each sample. In order to determine whether there is a connection between two bone points and the strength of the connection, a normalized embedding Gaussian function is applied to calculate the similarity of the two bone points:

[0134]

[0135] Where: k is the total number of vertices, which is 18;

[0136] The dot product is used to measure the similarity between two skeleton points in the embedding space; specifically, given an input feature map fin of size Cin×T×k, it is first transformed into ce×T×k using two embedding functions, namely θ and Then a 1×1 convolutional layer is selected as the embedding function; the two embedded feature maps are rearranged and reshaped into N×CeT matrix and CeT×T matrix; then they are multiplied to obtain the similarity matrix C of size k×k k , where C k Each element in Represents the bone point v i and bone point v j The similarity of the matrix is ​​normalized to 0-1 and used as the edge of the two skeleton points. Since the normalized Gaussian uses the softmax operation, C k Expressed as:

[0137]

[0138] Where: W θ and are the embedding functions θ and Parameters;

[0139] There is no direct replacement of the original A with B K or C K , but add them to it;

[0140] B K The values ​​of θ and The parameters are initialized to 0, so that the flexibility of the model is enhanced without reducing the original performance, and the adaptive network formula is obtained:

[0141]

[0142] The specific steps of step S5 are as follows:

[0143] S51: For input C×T×k, T represents the number of frames in a sequence, k represents the number of skeleton points, and the number of C is 2, representing the 2D coordinates x, y of the corresponding skeleton points in the image;

[0144] In order to calculate the Euclidean distance, we need to obtain the coordinates of the two skeleton points x1, y1 and x2, y2. The formula is:

[0145] distance = (x2 - x1) 2 +(y2-y1) 2

[0146] The distance finally obtained is a matrix with a dimension of (T-1, N), in which each element represents the distance between a skeleton point and other skeleton points in the current frame;

[0147] S52: In order to obtain the weight matrix Wd representing the spatial structure of the bone points, it is necessary to calculate the average of the sum of each bone point and all other bone points, and subtract the Euclidean distance between each bone point and all other bone points from this average, and then calculate the maximum value. For a certain bone point, the formula is:

[0148] αi,j=max(ee(i,j))

[0149] Where: αi,j represents the spatial relationship between node i and node j;

[0150] e represents the matrix of average Euclidean distances between pairwise nodes in the sequence.

[0151] The specific steps of step S7 are as follows:

[0152] S71: The model of the normalization flow processes the pose sequence data directly. It learns an invertible mapping between the data distribution PX and the potential distribution PZ (Gaussian Z ~ N(μnormal, I)). The architecture includes 16 steps, each of which consists of three layers: Actnorm, permutation and affine coupling layers. Actnorm is an activation normalization layer that uses the input data to obtain scaling factors and offsets, similar to batch normalization.

[0153] S72: For the input C×T×N, C is the number of input channels, which is 2 and represents the coordinates of the bone point (x, y). The permutation layer permutes the order of the input channels, i.e. (x, y) → (y, x).

[0154] S73: After the permutation layer, half of the data in dimension C remains unchanged, while the other half undergoes an affine transformation based on the other half. This results in a simple inverse transformation and Jacobian calculation. Compared to performing an affine transformation on the entire dimension C during backpropagation, this reduces the computational effort. The final calculated anomaly score is compared with a threshold. If the anomaly score exceeds the threshold, an anomaly is present in the current sequence. When an anomaly is detected, the person's location in the image is marked and an alarm is issued.

Claims

1. A method for detecting abnormalities of violent movements based on skeleton points, characterized in that: The following steps are involved: S1: Use the camera to collect video data and use the skeleton point extraction network to extract skeleton point data from the video sequence of the video data ,in represents the number of batch samples, Represents the number of video frames, represents the number of key points, Represents the key point coordinates And the adjacency matrix representing the connection relationship of the bone points ; S2: Expand the information representing the coordinates of the skeleton points in the skeleton point graph data extracted in step S1 by one dimension to obtain the coordinate information of the skeleton points. There are 17 key points in the skeleton point graph data. , newly added neck key points It's the left shoulder and right shoulder Midpoint of: , which expands to ; And normalize the coordinate information of the skeleton points and exchange channels: ； S3: Using the skeleton point data obtained in step S2, calculate the difference between each point in adjacent frames and calculate the speed. As speed information, and use the fully connected network to learn the changing law of speed; S4: replacing the custom adjacency matrix representing the relationship between bone points in the bone point graph data extracted in step S1 with a learnable parameter matrix; S5: For the skeleton point data processed in step S2, calculate the Euclidean distance between non-adjacent skeleton points and model the spatial structure; S6: Perform spatiotemporal graph convolution on the skeleton point data processed in step S5 and the learnable adjacency matrix replaced by S4; S7: The features extracted by spatiotemporal graph convolution in step S6 are fed into the normalized flow network to calculate the probability density score representing the anomaly; The specific steps of step S5 are as follows: S51: For input , Indicates the number of frames a sequence contains, represents the number of key points, is the number of input channels; In order to calculate the Euclidean distance, you need to get the coordinates of the two bone points , the formula is: , the final distance is a dimension of A matrix of shape, where each element represents the distance between a bone point and other bone points in the current frame; S52: To obtain the weight matrix representing the spatial structure of the bone points , we need to calculate the average of the sum of each bone point and all other bone points, and subtract the Euclidean distance between each bone point and all other bone points from this average, and then calculate the maximum value. For a certain bone point, the formula is: , where: Representative Node and nodes spatial relationship; A matrix representing the average Euclidean distances between pairs of nodes in the sequence.

2. The method for detecting abnormalities of violent movements based on skeleton points according to claim 1, characterized in that: The step S3 specifically comprises: for each joint point in the input skeleton point data, obtaining the position coordinates of the joint point in space and calculating the velocity information according to the position coordinates, embedding the obtained position and velocity information, adding and summing them, and fusing them into a new joint feature; The specific steps of obtaining new joint features in step S3 are as follows: S31: Input the skeleton point image data, which represents the coordinates of the skeleton point position , corresponding to joints and Skeleton sequence of frames, velocity information , the position coordinates and velocity information are encoded into high-dimensional space respectively, that is, ,in Represents the location information of high-dimensional space, Represents the velocity information of high-dimensional space, by summing blend them together; S32: Embedding information with high-dimensional spatial positions For example, use two fully connected layers to Encode to get: ， Where: is the weight matrix; is the bias matrix; is the relu activation function; Get high-dimensional velocity embedding features: 。 3. The method for detecting abnormalities of violent movements based on skeleton points according to claim 2, characterized in that: The specific steps of step S4 are as follows: S41: Get adjacency matrix ,in Represents a custom adjacency matrix that represents the physical structure of the human body; S42: Obtaining the adjacency matrix representing the connection strength , is a adjacency matrix, and compared to, The elements of are parameterized and optimized along with other parameters during training, There are no constraints on the value of S43: Get adaptive adjacency matrix , Is a data-dependent adjacency matrix that learns a unique adjacency matrix for each sample. In order to determine whether there is a connection between two bone points and the strength of the connection, a normalized embedding Gaussian function is applied to calculate the similarity of the two bone points: , where: is the number of key points; The dot product is used to measure the similarity between two bone points in the embedding space; specifically, given a size of Input feature map , first use two embedding functions to transform it into ,Right now , then select a 1×1 convolution layer as the embedding function; rearrange and reshape the feature maps of the two previous embeddings into Matrix and matrices; then multiply them together to get a size of Similarity matrix ,in, Each element in Represents bone points and bone points The similarity of the matrix is ​​normalized to 0-1 and used as the edge of the two skeleton points. Since the normalized Gaussian uses the softmax operation, Expressed as: , where: They are embedded functions Parameters; No direct original Replace with or , but add them to it; The value and The parameters are initialized to 0, so that the flexibility of the model is enhanced without reducing the original performance, and the adaptive network formula is obtained: 。 4. The method for detecting abnormalities of violent movements based on skeleton points according to claim 1, characterized in that: The specific steps of step S7 are as follows: S71: The normalized flow model processes the pose sequence data directly, which learns the reversible mapping between the data distribution PX and the potential distribution PZ, Gaussian ,The architecture consists of K steps, each step consists of three layers: , permutation and affine coupling layers, is an activation normalization layer that uses the input data to obtain scaling factors and offsets, similar to batch normalization; S72: For input , represents the coordinates of the bone point , the permutation layer permutes the order of the input channels, i.e. ; S73: Data after the replacement layer, for This dimension, half remains unchanged, and the other is an affine transformation based on the other half, which results in a simple inverse transformation and Jacobian calculation. When performing backpropagation, compared to This dimension is all affine transformed to reduce the amount of calculation; the final calculated anomaly score is compared with the threshold. When the anomaly score is greater than the threshold, it indicates that there is an anomaly in the current sequence.

Images