A Reversible Normalized Streaming Video Anomaly Detection Method Based on Human Skeleton Nodes

Abstract

This invention discloses a reversible normalized streaming video anomaly detection method based on human skeletal nodes. The process is as follows: Human skeletal node sequences are extracted from the acquired video data, and the skeletal nodes are normalized and subjected to confidence filtering to obtain the skeletal node sequence. Feature enhancement is performed based on the topological relationship of the human skeleton, and the skeletal node sequence is input into a multilayer reversible transform network. The skeletal sequence features are mapped to a latent space through multilayer reversible transform. A probabilistic model of the normal behavior distribution is performed in the latent space, and a spatiotemporal masking operation is applied to suppress the influence of pose noise on feature modeling. The reversible transform network is used for inverse mapping to obtain the reconstructed skeletal sequence features. A frame-by-frame anomaly score is generated based on the negative log-likelihood value of the skeletal node sequence in the latent space, thus realizing the detection of abnormal behavior in the video. This invention only requires acquiring normal behavior video data to complete the anomaly detection, and has the advantages of high detection stability and high computational efficiency.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and video intelligent analysis technology, specifically relating to a reversible normalized streaming video anomaly detection method based on human skeleton nodes. Background Technology

[0002] With the rapid development of fields such as intelligent monitoring, smart transportation, industrial automation, and public safety management, video behavior understanding and anomaly detection technology has gradually become an important research direction in the field of computer vision.

[0003] Currently, video behavior understanding systems typically consist of two key stages: video feature extraction and behavior recognition. Video feature extraction methods have evolved from traditional manual feature extraction to automated feature learning through deep learning. In video behavior recognition, existing methods include single-stream architectures, two-stream architectures, time-segmented models, two-stage learning models, and methods based on the visual Transformer. These methods have made some progress in behavior recognition and action localization tasks, but they generally suffer from high computational complexity, strong dependence on large amounts of labeled data, complex model structures, and limited ability to model long-term dependencies. Especially in anomaly detection scenarios, due to the scarcity and diversity of anomaly samples, supervised learning-based methods struggle to comprehensively cover unknown anomaly categories, resulting in insufficient generalization ability.

[0004] In recent years, generative models have gained increasing attention in the field of anomaly detection. Among them, normalized flow models, as a reversible probabilistic generative model, map complex data distributions to simpler distributions through a series of reversible mappings, enabling accurate modeling of data probability density. Compared to variational autoencoders and generative adversarial networks, normalized flow models offer advantages such as accurate likelihood estimation, stable inference processes, controllable training processes, and structural invertibility. Existing research shows that normalized flow models have achieved significant results in image generation, semi-supervised learning, and human motion generation, and their application in video anomaly detection is being explored. However, existing methods often rely on autoencoders for feature extraction before flow modeling, resulting in a relatively complex overall structure and insufficient robustness to pose noise and spatiotemporal interference. Furthermore, existing video anomaly detection methods often directly model RGB video, incurring significant computational overhead and hindering deployment on resource-constrained devices or real-time monitoring systems.

[0005] In addition, video data itself has characteristics such as strong time dependence, complex environment, obvious lighting changes and unavoidable pose estimation errors. Traditional deep networks are easily affected by noise interference during the modeling process, resulting in unstable anomaly scores and affecting detection accuracy and system reliability. Summary of the Invention

[0006] To address the shortcomings of existing video anomaly detection methods, such as complex model structures, strong dependence on anomaly samples, high computational cost, insufficient robustness to pose estimation noise, and limited probabilistic modeling capabilities, this invention proposes a reversible normalized streaming video anomaly detection method based on human skeleton nodes. This method uses video data containing the human object to be detected as input. By constructing a reversible normalized streaming video anomaly detection method based on human skeleton nodes, it achieves accurate modeling of the distribution of normal behavior and realizes unsupervised anomaly detection through negative log-likelihood, thereby improving anomaly detection accuracy and model stability while ensuring computational efficiency.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A reversible normalized streaming video anomaly detection method based on human skeletal nodes, the method comprising the following steps:

[0009] S1. Collect video data containing the object to be detected. In view of the problem that existing detection methods directly process RGB images, resulting in large computational load and susceptibility to lighting and background interference, this invention first extracts human skeleton nodes from video data and performs normalization and confidence filtering to obtain skeleton node sequence X.

[0010] S2. Utilizing the topological connections between human joints, fully extract spatial structural features and enhance the features of the skeletal node sequence X. Define the connections between joints using an adjacency matrix, calculate the feature differences between adjacent joints as relative edge features, extract node-level structural representations using convolutional mapping and nonlinear transformation, and further generate a set of structural condition features by employing channel-level residual concatenation and channel alignment mapping. , The structural condition features corresponding to the nth (n=1,2,3…,N) reversible transformation network are represented, and the set of structural condition features is injected into the reversible transformation network in layers.

[0011] S3. To address the problem that existing generative models cannot accurately calculate data probability density, resulting in a lack of statistical interpretability in abnormal scores, this invention employs a multi-layer reversible transformation network to map skeletal sequence features to a latent space for probabilistic modeling, combining the skeletal node sequence X with the set of structural condition features. The N layers of the invertible transformation network are input respectively, and the feature space is transformed through the forward process of the invertible transformation network to obtain the latent code Z in the latent space;

[0012] S4. To address the problem of random noise inevitably interfering with the distribution of the latent space during anomaly detection, leading to false detections, this invention employs a spatiotemporal masking operation on the latent code Z to suppress latent dimensions sensitive to attitude noise, thereby obtaining a constrained latent code Z. c ;

[0013] S5. The constrained latent code Z c The features of the reconstructed skeletal sequence are obtained by performing inverse mapping operations on N layers of an invertible transform network. ;

[0014] S6. Calculate the negative log-likelihood value of the skeletal node sequence X under the standard Gaussian distribution based on the latent code Z. Use the obtained negative log-likelihood value as the basis for anomaly scoring. The larger the negative log-likelihood value, the more the corresponding skeletal sequence deviates from the normal behavior distribution.

[0015] S7. Using video data containing only normal behavior, repeat steps S2 to S6 to perform end-to-end iterative calculation. During the iterative calculation, use the reconstructed skeleton sequence features and the original skeleton node sequence to calculate the loss value of the loss function, use the latent encoding under the standard Gaussian distribution to calculate the loss value of the negative log-likelihood loss function, and minimize the above two loss values. At the same time, update the parameters of the reversible transformation network through backpropagation until the preset maximum number of iterations is reached.

[0016] S8. Perform steps S1 to S6 on the video data to be detected to obtain the negative log-likelihood value of the video data. This provides an anomaly measurement standard with clear physical meaning. An anomaly score is generated frame by frame based on the negative log-likelihood value of the video data. The anomaly score is used to determine whether there is abnormal behavior in the video.

[0017] Furthermore, step S1 is performed as follows:

[0018] S11. Collect video data containing the object to be detected as a video frame sequence. From video frame sequence Extracting key point information of the human skeleton, among which... Represents the image of frame t. This represents the total number of video frames, for the t-th frame. Human key point detection is performed to obtain a representation of the human skeletal structure: ,in Indicates the number of joints. This represents the two-dimensional coordinates of the v-th joint in the t-th frame of the image. The detection confidence of this joint is used to construct a three-channel skeletal feature representation in the t-th frame image: This step reduces the dimensionality of high-dimensional video data to a sparse skeletal feature representation, significantly reducing subsequent computational overhead and making it suitable for edge devices.

[0019] S12. Representation of three-channel skeleton features using a sliding window method. The process involves iterating through the data, starting with frame t and sliding the window for one frame at a time. The three-channel skeletal feature representations from T consecutive frames within each sliding window are concatenated into a single skeletal temporal sample. The sliding window length is T. Furthermore, the two-dimensional coordinate channels of the joints in the constructed skeletal temporal sample are normalized to obtain the normalized skeletal feature representation for frame t. ;

[0020] S13, Set threshold For the skeletal feature representation of frame t ,like There is at least one confidence value in the set of confidence values ​​in the third channel. Then determine the skeletal feature representation of the t-th frame. If the pre-set confidence requirements are not met, the data is discarded, resulting in the skeletal node sequence X. The confidence filtering operation effectively removes noisy data caused by occlusion or detection errors, improving the quality of the input data from the source and effectively enhancing the network's robustness to complex environments.

[0021] Furthermore, step S2 is as follows:

[0022] S21. Perform a weighted aggregation operation on the skeletal node sequence X and the predefined human skeletal topology adjacency matrix A to obtain the neighborhood structure features. X[c,t,v] represents the feature value of the v-th joint on the c-th channel in the t-th frame of the sliding window, if and only if there is a connecting edge A[v,u]=1 between joint v and joint u in the predefined structure, otherwise A[v,u]=0. This design explicitly introduces prior knowledge of human kinematics, enabling the network to capture the cooperative motion patterns between joints more accurately.

[0023] S22. For each joint, based on the adjacency matrix A, calculate the feature difference between each joint and its neighboring joints, and concatenate the feature difference with the node's own features in the channel dimension to obtain the relative edge feature E.

[0024] S23. Apply a nonlinear transformation to the opposite edge feature E using a convolutional mapping function, and perform max pooling on the adjacent node dimension to obtain the node-level structure representation. The convolution mapping function consists of two... It consists of convolution, batch normalization, and LeakyReLU activation functions;

[0025] S24. Using the output of the (n-1)th layer reversible transform network as input, execute steps S21 to S23 to obtain the node-level structure representation of the nth layer. The expression is defined as n=2,3,…,N, and channel-level residual concatenation is used to merge the node-level structural representation obtained in the current layer with the node-level structural representation obtained in the previous layer. Then, the structure injection feature is obtained through the channel alignment mapping function. Among them, the channel alignment mapping function uses two layers. Implemented in the form of convolution, it combines the set of structural conditional features. As conditional structure priors, they are injected into the corresponding reversible transformation networks in a hierarchical one-to-one correspondence. The fusion of multi-scale features enhances the network's ability to perceive local subtle movements and overall posture changes, thereby improving the discriminative power of feature representation.

[0026] Furthermore, the input to the nth layer multilayer reversible transform network includes the output of the (n-1)th layer reversible transform network, the initial input skeletal node sequence X, and structural condition features. The skeletal node sequence X is processed through N invertible transformation networks to obtain the latent code Z in the latent space. The latent code Z follows a standard normal distribution. The log probability density of the skeletal node sequence X is equal to the log probability density of the latent code Z minus the logarithm of the determinant of the Jacobian matrix of the transformation function. This achieves an accurate probability density estimation of the normal behavior distribution. Furthermore, the negative of the obtained log probability density of the skeletal node sequence X is used as the anomaly score. The higher the anomaly score, the more the skeletal node sequence deviates from the normal motion distribution. The lower the anomaly score, the more the skeletal node sequence belongs to the normal motion manifold. Since the mapping is invertible, the network can perform both forward probability evaluation and inverse reconstruction, which can improve the stability and reliability of the anomaly score.

[0027] Further, step S4 is as follows: Let the learnable latent gating vector be... Potential gate vector Each element in the algorithm takes a value of [0,1]. The latent code Z is then compared with the gating function. Element-wise multiplication yields the constrained latent code. If a certain latent dimension is sensitive to input perturbations, the corresponding gating weight is optimized to be close to 0, thereby achieving noise suppression. Conversely, if a certain dimension can stably reflect the normal motion pattern, the weight of that dimension approaches 1. Through calculation iteration, the latent gating vector can effectively filter noise components in the latent space, further improving the network's generalization ability and detection accuracy in real monitoring scenarios.

[0028] Furthermore, the loss function is defined as follows:

[0029] loss function This includes a negative log-likelihood term and a reconstruction error term. The negative log-likelihood term is the inverse of the log probability density of the skeletal node sequence X, maximizing the log-likelihood value of normal samples in the latent space to ensure the accuracy of the distribution mapping. The reconstruction error term is generated by the constrained latent encoding. The reconstructed skeletal sequence features are obtained by inverting the computation process of inputting into the reversible transform network. Then the obtained reconstructed skeletal sequence features The results are calculated from the skeletal node sequence X, and finally obtained by weighted summation of the negative log-likelihood term and the reconstruction error term:

[0030] ,

[0031] in, Represents the negative log-likelihood term. Represents the reconstruction error term. This represents adjustable weight parameters. The joint loss function can avoid the pattern collapse or reconstruction ambiguity problems that may be caused by a single loss function.

[0032] Furthermore, the video data originates from video frame sequence data of real or simulated monitoring scenarios.

[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0034] 1. Possesses precise probability density modeling capabilities, and anomaly scoring exhibits clear statistical interpretability and high stability. This invention, based on the principle of invertible neural networks, establishes a rigorous mathematical mapping from the data space to the probability space. Due to the invertibility of the mapping and the computability of the Jacobian determinant, the network can explicitly calculate the exact probability density of a sequence under a normal behavioral distribution. Therefore, the generated anomaly scores have strict statistical significance: the higher the score, the greater the probability that the sample deviates from the normal distribution. This probability density-based scoring mechanism avoids score drift caused by training fluctuations in traditional discriminative networks, significantly improving the reliability and stability of the detection results.

[0035] 2. Unsupervised training requires only normal samples, significantly reducing data annotation costs and adapting to open scenarios. For example, in step S7 of this invention, the network iterative calculation process only requires video data of normal behavior, optimizing network parameters by minimizing the negative log-likelihood loss and reconstruction error loss. Traditional supervised learning methods rely on a large amount of labeled data covering various abnormal situations. However, in actual monitoring scenarios, abnormal events are scarce and their forms are unpredictable, making it difficult for the network to generalize. As described in step S8 of this invention, any skeleton node sequence that does not conform to the learned normal distribution pattern will have its mapping in the latent space deviate from the standard Gaussian distribution, resulting in an extremely high negative log-likelihood value. This approach fundamentally eliminates the dependence on abnormal annotation, significantly reducing data collection and manual annotation costs, and enabling the network to effectively identify novel abnormal behaviors, making it highly valuable for open environment applications.

[0036] 3. Low data dimensionality and lightweight network structure result in high computational efficiency, making it suitable for real-time monitoring and edge deployment. For example, in step S1 of this invention, the input data is a sequence of skeletal nodes that has been filtered and normalized with confidence, rather than high-dimensional RGB image pixels. Furthermore, the reversible transformation network in this invention employs a feature enhancement and hierarchical injection structure based on topological relationships, resulting in fewer parameters compared to traditional deep convolutional networks. Existing methods based on RGB video require processing massive amounts of pixel information, leading to significant computational redundancy and susceptibility to interference from lighting and cluttered backgrounds. This invention reduces the dimensionality of high-dimensional video data to a sparse sequence of skeletal keypoints from the source, combined with a lightweight reversible neural network, significantly reducing matrix operations and memory usage. This design significantly improves network inference speed, meeting the demands of real-time video stream processing, and can be easily deployed on embedded devices or edge computing nodes with limited computing power, solving the pain point of traditional methods being difficult to implement. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of the reversible standardized streaming video anomaly detection method disclosed in this invention;

[0039] Figure 2 This is a schematic diagram of the composition of the multilayer reversible neural network in the reversible normalized streaming video anomaly detection method disclosed in this invention;

[0040] Figure 3 This is a schematic diagram illustrating the anomaly detection performance of the present invention on the UBnormal dataset;

[0041] Figure 4 This is a schematic diagram illustrating the anomaly detection effect of the present invention on the ShanghaiTech dataset. Detailed Implementation

[0042] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0043] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0044] Example 1

[0045] This embodiment discloses a reversible normalized streaming video anomaly detection method based on human skeletal nodes, specifically including the following steps:

[0046] S11. Collect video data containing the object to be detected as a video frame sequence. From video frame sequence Extracting key point information of the human skeleton, among which... Represents the image of frame t. This represents the total number of video frames, for the t-th frame. Human key point detection is performed to obtain a representation of the human skeletal structure: ,in Indicates the number of joints. This represents the two-dimensional coordinates of the v-th joint in the t-th frame of the image. The detection confidence of this joint is used to construct a three-channel skeletal feature representation in the t-th frame image: ;

[0047] Alphapose was used for human keypoint detection, with fast_ResNet-50 pre-trained weights and a set number of keypoints. =18.

[0048] S12. Representation of three-channel skeleton features using a sliding window method. The process involves iterating through the data, starting with frame t and sliding the window for one frame at a time. The three-channel skeletal feature representations from T consecutive frames within each sliding window are concatenated into a single skeletal temporal sample. The sliding window length is T. Furthermore, the two-dimensional coordinate channels of the joints in the constructed skeletal temporal sample are normalized to obtain the normalized skeletal feature representation for frame t. ;

[0049] The sliding window length is set to T=16.

[0050] S13, Set threshold For the skeletal feature representation of frame t ,like There is at least one confidence value in the set of confidence values ​​in the third channel. Then determine the skeletal feature representation of the t-th frame. If the pre-set reliability requirement is not met, the node sequence X is discarded, resulting in the skeletal node sequence X.

[0051] Among them, setting a threshold =0.1.

[0052] S21. Perform a weighted aggregation operation on the skeletal node sequence X and the predefined human skeletal topology adjacency matrix A to obtain the neighborhood structure features. , where X[c,t,v] represents the feature value of the v-th joint on the c-th channel in the t-th frame of the sliding window, if and only if there is a connected edge A[v,u]=1 between joint v and joint u in the predefined structure, otherwise A[v,u]=0;

[0053] S22. For each joint, based on the adjacency matrix A, calculate the feature difference between each joint and its neighboring joints, and concatenate the feature difference with the node's own features in the channel dimension to obtain the relative edge feature E.

[0054] S23. Apply a nonlinear transformation to the opposite edge feature E using a convolutional mapping function, and perform max pooling on the adjacent node dimension to obtain the node-level structure representation. The convolution mapping function consists of two... It consists of convolution, batch normalization, and LeakyReLU activation functions;

[0055] S24. Using the output of the (n-1)th layer reversible transform network as input, execute steps S21 to S23 to obtain the node-level structure representation of the nth layer. The expression is defined as n=2,3,…,N, and channel-level residual concatenation is used to merge the node-level structural representation obtained in the current layer with the node-level structural representation obtained in the previous layer. Then, the structure injection feature is obtained through the channel alignment mapping function. Among them, the channel alignment mapping function uses two layers. Implemented in the form of convolution, it combines the set of structural conditional features. As conditional structural priors, they are injected into the corresponding invertible transformation networks in a one-to-one correspondence according to the hierarchy;

[0056] The number of structural condition feature sets is set to N=12.

[0057] S3. Pass the skeletal node sequence X through N invertible transformation networks to obtain the latent code Z in the latent space. The latent code Z follows a standard normal distribution. The log probability density of the skeletal node sequence X is equal to the log probability density of the latent code Z minus the logarithm of the determinant of the Jacobian matrix of the transformation function. The negative of the log probability density of the obtained skeletal node sequence X is used as the anomaly score. The higher the anomaly score, the more the skeletal node sequence deviates from the normal motion distribution. The lower the anomaly score, the more the skeletal node sequence belongs to the normal motion manifold.

[0058] To correspond with the set of structural condition features, the number of layers in the reversible transformation network is set to N=12; the dimension of the latent encoding Z is the same as that of the skeletal node sequence X, which is 3* *T=864.

[0059] S4. Define a learnable latent gating vector. The latent code Z is combined with the gating function Element-wise multiplication yields the constrained latent code. Then the constrained latent encoding In the reverse process of inputting into the reversible neural network, the inputs are given in the reverse order of the forward process, and the inverse transformation operation is performed step by step from layer N to layer 1, but without using the structural condition feature set. Finally, the reconstructed skeletal sequence features are obtained. ;

[0060] Among them, setting potential gating vectors and constrained latent encoding The dimensions are the same as the skeletal node sequence X, which is 3*. *T=864.

[0061] S5. Using video data containing only normal behavior, repeat steps S2 to S6 for end-to-end iterative computation; during the iterative computation, use the reconstructed skeleton sequence features and the original skeleton node sequence to calculate the loss value of the loss function. The loss value of the negative log-likelihood loss function is calculated using the latent encoding under the standard Gaussian distribution. And minimize the joint loss function of weighted summation. : Meanwhile, the parameters of the reversible transformation network are updated through backpropagation until the preset maximum number of iterations is reached;

[0062] Among them, adjustable weight parameters are set. =0.3.

[0063] S6. Based on the latent code Z obtained in S3, calculate the negative log-likelihood value of the skeletal node sequence X under the standard Gaussian distribution. The obtained negative log-likelihood value is directly used as the anomaly score.

[0064] In this embodiment, the dataset used is from the UBnormal dataset, which contains approximately 450,000 frames of data. End-to-end iterative computation is employed: first, the network parameters are initialized, and the initial learning rate is set to... The Adam optimizer was used, and the program was trained for a total of 30 epochs. In each iteration, the minimum... The invertible neural network and potential gating vectors are updated using the backpropagation algorithm. The parameters are adjusted until the preset maximum number of iterations is reached. After the network training is complete, video data containing the objects to be detected is input, and steps S1 to S6 of the claims are executed sequentially, outputting frame-by-frame negative log-likelihood values ​​as anomaly scores. Figure 3 The figure shows an instance from the UBnormal dataset, with the horizontal axis representing the frame ID and the vertical axis representing the processed anomaly score. The white area represents normal periods where the score remains low; the gray area represents abnormal periods where the score changes significantly. Quantitative comparison results are shown in Table 1. The method of this invention achieves an AUROC score of 0.735 on the UBnormal dataset, which is superior to existing mainstream methods, indicating that the method of this invention has high detection accuracy.

[0065] Table 1. Comparison of results between the video anomaly detection method disclosed in this invention and existing methods.

[0066]

[0067] Example 2

[0068] This embodiment discloses a reversible normalized streaming video anomaly detection method based on human skeletal nodes, specifically including the following steps:

[0069] S1. Referring to the operation steps S11-S13 in Example 1, some parameters are changed as follows: Set the sliding window length T=18, and the threshold... =0.1.

[0070] S2. Referring to the operation steps S21-S24 in Example 1, change some parameters as follows: Set the number of structural condition feature sets N=16.

[0071] S3. Referring to the S3 operation steps in Example 1, some parameters are changed as follows: To correspond to the structural condition feature set, the number of reversible transformation network layers N=16, and the dimension of the latent encoding Z is the same as that of the skeletal node sequence X, which is 3* *T=972.

[0072] S4. Referring to the operation steps in S4 of Example 1, some parameters are changed as follows: Set the potential gating vector. and constrained latent encoding The dimensions are the same as the skeletal node sequence X, which is 3*. *T=972.

[0073] S5. Referring to the operation steps in S5 of Example 1, change some parameters as follows: Set adjustable weight parameters. =0.2.

[0074] S6. Based on the latent code Z obtained in S3, calculate the negative log-likelihood value of the skeletal node sequence X under the standard Gaussian distribution. The obtained negative log-likelihood value is directly used as the anomaly score.

[0075] In this embodiment, the dataset used is from the ShanghaiTech dataset, and end-to-end iterative computation is employed: first, the network parameters are initialized, and the initial learning rate is set to... The Adam optimizer was used, and the program was trained for a total of 30 epochs. In each iteration, the minimum... The invertible neural network and potential gating vectors are updated using the backpropagation algorithm. The parameters are adjusted until the preset maximum number of iterations is reached. After the network training is complete, video data containing the objects to be detected is input, and steps S1 to S6 of the claims are executed sequentially, outputting frame-by-frame negative log-likelihood values ​​as anomaly scores. Figure 4 The figure shows an example from the ShanghaiTech dataset, with the horizontal axis representing the frame ID and the vertical axis representing the processed anomaly score. The white area represents normal periods where the score remains low; the gray area represents abnormal periods where the score changes significantly. Quantitative comparison results are shown in Table 2. The method of this invention achieves an AUROC index of 0.837 on the ShanghaiTech dataset, which is superior to existing mainstream methods, indicating that the method of this invention has high detection accuracy.

[0076] Table 2. Comparison of Results between the Video Anomaly Detection Method Disclosed in This Invention and Existing Methods

[0077]

[0078] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously.

[0079] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for anomaly detection in reversible normalized streaming video based on human skeletal nodes, characterized in that, The reversible normalized stream video anomaly detection method includes the following steps: S1. Collect video data containing the object to be detected, extract human skeleton nodes from the video data, and perform normalization and confidence filtering operations on the human skeleton nodes to obtain skeleton node sequence X. S2. Perform feature enhancement on the skeletal node sequence X, construct a joint adjacency matrix based on predefined human skeletal topological relationships, and then extract multi-scale spatial structural features from the joint adjacency matrix to generate a set of structural condition features. , This represents the structural condition features corresponding to the nth layer (n=1,2,3,…,N) of the invertible transformation network. S3. Combine the skeletal node sequence X and the set of structural condition features. The N layers of the invertible transformation network are input respectively, and the feature space is transformed through the forward process of the invertible transformation network to obtain the latent code Z in the latent space; S4. Apply a spatiotemporal masking operation to the latent code Z to suppress the latent dimension sensitive to pose noise, thus obtaining the constrained latent code Z. c ; S5. The constrained latent code Z c The features of the reconstructed skeletal sequence are obtained by performing inverse mapping operations on N layers of an invertible transform network. ; S6. Calculate the negative log-likelihood value of the skeletal node sequence X under the standard Gaussian distribution based on the latent code Z. Use the obtained negative log-likelihood value as the basis for anomaly scoring. The larger the negative log-likelihood value, the more the corresponding skeletal sequence deviates from the normal behavior distribution. S7. Using video data containing only normal behavior, repeat steps S2 to S6 to perform end-to-end iterative calculation. During the iterative calculation, use the reconstructed skeleton sequence features and the original skeleton node sequence to calculate the loss value of the loss function, use the latent encoding under the standard Gaussian distribution to calculate the loss value of the negative log-likelihood loss function, and minimize the above two loss values. At the same time, update the parameters of the reversible transformation network through backpropagation until the preset maximum number of iterations is reached. S8. Perform steps S1 to S6 on the video data to be detected to obtain the negative log-likelihood value of the video data, generate an anomaly score frame by frame based on the negative log-likelihood value of the video data, and determine whether there is abnormal behavior in the video based on the anomaly score.

2. The method for reversible normalized streaming video anomaly detection based on human skeleton nodes according to claim 1, characterized in that, The process of step S1 is as follows: S11. Collect video data containing the object to be detected as a video frame sequence. From video frame sequence Extracting key point information of the human skeleton, among which... Represents the image of frame t. This represents the total number of video frames, for the t-th frame. Human key point detection is performed to obtain a representation of the human skeletal structure: ,in Indicates the number of joints. This represents the two-dimensional coordinates of the v-th joint in the t-th frame of the image. The detection confidence of this joint is used to construct a three-channel skeletal feature representation in the t-th frame image: ; S12. Representation of three-channel skeleton features using a sliding window method. The process involves iterating through the data, starting with frame t and sliding the window for one frame at a time. The three-channel skeletal feature representations from T consecutive frames within each sliding window are concatenated into a single skeletal temporal sample. The sliding window length is T. Furthermore, the two-dimensional coordinate channels of the joints in the constructed skeletal temporal sample are normalized to obtain the normalized skeletal feature representation for frame t. ; S13, Set threshold For the skeletal feature representation of frame t ,like There is at least one confidence value in the set of confidence values ​​in the third channel. Then determine the skeletal feature representation of the t-th frame. If the pre-set reliability requirement is not met, the node sequence X is discarded, resulting in the skeletal node sequence X.

3. The method for reversible normalized streaming video anomaly detection based on human skeleton nodes according to claim 1, and the process of step S1 according to claim 2, characterized in that, The process of step S2 is as follows: S21. As described in claim 2, perform a weighted aggregation operation on the bone node sequence X and the predefined human bone topology adjacency matrix A to obtain the neighborhood structure features. , where X[c,t,v] represents the feature value of the v-th joint on the c-th channel in the t-th frame of the sliding window, and A[v,u]=1 if and only if there is a connected edge between joint v and joint u in the predefined structure, otherwise A[v,u]=0; S22. For each joint, based on the adjacency matrix A, calculate the feature difference between each joint and its neighboring joints, and concatenate the feature difference with the node's own features in the channel dimension to obtain the relative edge feature E. S23. Apply a nonlinear transformation to the opposite edge feature E using a convolutional mapping function, and perform max pooling on the adjacent node dimension to obtain the node-level structure representation. The convolution mapping function consists of two... It consists of convolution, batch normalization, and LeakyReLU activation functions; S24. Using the output of the (n-1)th layer reversible transform network as input, execute steps S21 to S23 to obtain the node-level structure representation of the nth layer. The expression is given by n=2,3,…,N. Channel-level residual concatenation is used to merge the node-level structural representation obtained in the current layer with the node-level structural representation obtained in the previous layer. Then, the structure-injected features are obtained through a channel alignment mapping function. Among them, the channel alignment mapping function uses two layers. Implemented in the form of convolution, it combines the set of structural conditional features. As conditional structure priors, they are injected into the corresponding invertible transformation networks in a one-to-one correspondence according to the hierarchy.

4. The method for reversible normalized streaming video anomaly detection based on human skeleton nodes according to claim 1, characterized in that, The input to the nth layer multilayer reversible transform network includes the output of the (n-1)th layer reversible transform network, the initial input skeletal node sequence X, and structural condition features. The skeletal node sequence X is processed through N invertible transformation networks to obtain the latent code Z in the latent space. The latent code Z follows a standard normal distribution. The log probability density of the skeletal node sequence X is equal to the log probability density of the latent code Z minus the logarithm of the determinant of the Jacobian matrix of the transformation function. The negative of the log probability density of the obtained skeletal node sequence X is used as the anomaly score. The higher the anomaly score, the more the skeletal node sequence deviates from the normal motion distribution. The lower the anomaly score, the more the skeletal node sequence belongs to the normal motion manifold.

5. The method for reversible normalized streaming video anomaly detection based on human skeleton nodes according to claim 1, characterized in that, The process of step S4 is as follows: Let the learnable potential gating vector be... Potential gate vector Each element in the algorithm takes a value of [0,1]. The latent code Z is then compared with the gating function. Element-wise multiplication yields the constrained latent code. If a certain potential dimension is sensitive to input perturbations, the gating weight corresponding to that dimension is optimized to close to 0, thereby achieving noise suppression. Conversely, if a certain dimension can stably reflect the normal motion pattern, the weight of that dimension approaches 1.

6. The method for reversible normalized streaming video anomaly detection based on human skeleton nodes according to claim 1, characterized in that, The loss function is defined as follows: loss function This includes a negative log-likelihood term and a reconstruction error term. The negative log-likelihood term is the negative of the log probability density of the skeletal node sequence X, and the reconstruction error term is derived from the constrained latent encoding. The reconstructed skeletal sequence features are obtained by inverting the computation process of inputting into the reversible transform network. Then, the obtained reconstructed skeletal sequence features The results are calculated from the skeletal node sequence X, and finally obtained by weighted summation of the negative log-likelihood term and the reconstruction error term: , in, Represents the negative log-likelihood term. This represents the reconstruction error term. This represents an adjustable weight parameter.

7. The method for reversible normalized streaming video anomaly detection based on human skeleton nodes according to claim 1, characterized in that, The video data originates from video frame sequences of real or simulated surveillance scenarios.

Images