Video monitoring personnel behavior identification method and system

Abstract

The invention relates to the technical field of image processing, and discloses a video monitoring personnel behavior identification method and system. The method comprises the following steps: performing action node and joint angle extraction processing on a positioning area to obtain human body action skeleton sequence data; constructing an action connection graph and a time sequence association edge, and carrying out behavior pattern learning processing to obtain space-time behavior characteristics; performing action semantic mapping processing on the spatio-temporal behavior features and the human body behavior feature vectors, and performing behavior time sequence chain construction processing to obtain behavior semantic description features; and performing behavior pattern matching and anomaly measurement processing on the behavior semantic description features, performing feature discrimination processing to obtain a behavior type identifier and an anomaly degree index, performing behavior persistence analysis processing, and performing behavior risk level evaluation processing to obtain a behavior recognition result and early warning information. According to the invention, based on multi-modal feature fusion and spatio-temporal behavior modeling, accurate identification of personnel behaviors and dynamic assessment of risk levels are realized.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method and system for identifying human behavior in video surveillance. Background Art

[0002] In recent years, with the widespread application of video surveillance systems in the field of public security, intelligent video analysis technology has received increasing attention. Existing methods for identifying human behavior mainly use technologies such as feature extraction based on image processing, behavior classification based on deep learning, and anomaly detection based on rule bases. These methods extract human appearance features, motion features, and posture features from video sequences, and combine behavior pattern analysis and temporal relationship modeling to achieve automatic identification of human behavior and abnormal behavior warning. Current technical solutions usually decompose the behavior recognition problem into multiple subtasks such as feature extraction, behavior classification, and anomaly detection, and use methods such as multimodal feature fusion and deep learning to improve recognition accuracy.

[0003] However, existing technical solutions still have some shortcomings when dealing with human behavior recognition in complex scenarios: first, traditional feature extraction methods often only focus on features of a single dimension, and lack holistic modeling of human behavior in time and space dimensions; second, existing behavior recognition algorithms have a relatively shallow understanding of action semantics, and it is difficult to accurately capture the logical connections and evolution laws between behaviors; third, abnormal behavior detection is mostly based on simple threshold judgments, lacking in-depth analysis of behavior patterns and refined assessment of risk levels, resulting in insufficient accuracy and timeliness of warning information. Summary of the invention

[0004] The present application provides a method and system for identifying human behavior in video surveillance, which is used to achieve accurate identification of human behavior and dynamic assessment of risk level based on multimodal feature fusion and spatiotemporal behavior modeling.

[0005] In the first aspect, the present application provides a method for identifying human behavior through video surveillance, which comprises: extracting human motion texture features and motion trajectory features from an input monitoring video sequence, performing feature fusion processing through behavior saliency analysis, and obtaining a human behavior feature vector; performing skeletal joint positioning processing on the human behavior feature vector, and performing action node and joint angle extraction processing on the positioning area, and obtaining human motion skeleton sequence data; constructing an action connection graph and time-series association edges for the human motion skeleton sequence data, performing behavior pattern learning processing through motion paradigm analysis, and obtaining spatiotemporal behavior features; performing action semantic mapping processing on the spatiotemporal behavior features and human behavior feature vectors, and obtaining behavior semantic description features through behavior timing chain construction processing; performing behavior pattern matching and anomaly measurement processing on the behavior semantic description features, and performing feature discrimination processing through behavior similarity comparison, and obtaining a behavior type identifier and anomaly degree index; performing behavior continuity analysis processing on the behavior type identifier and anomaly degree index, and obtaining behavior recognition results and warning information through behavior risk level assessment processing.

[0006] In a second aspect, the present application provides a system for identifying human behavior through video surveillance, the system comprising:

[0007] The acquisition module is used to extract the texture features and motion trajectory features of human actions from the input surveillance video sequence, perform feature fusion processing through behavior significance analysis, and obtain the human behavior feature vector;

[0008] A positioning module is used to perform skeletal joint positioning processing on the human behavior feature vector, and to extract action nodes and joint angles from the positioning area to obtain human action skeleton sequence data;

[0009] A construction module is used to construct an action connection graph and a time-series associated edge for the human action skeleton sequence data, and to perform behavior pattern learning processing through motion paradigm analysis to obtain spatiotemporal behavior characteristics;

[0010] A mapping module is used to perform action semantic mapping processing on the spatiotemporal behavior characteristics and human behavior feature vectors, and obtain behavior semantic description characteristics through behavior time sequence chain construction processing;

[0011] A measurement module is used to perform behavior pattern matching and abnormal measurement processing on the behavior semantic description features, perform feature discrimination processing by behavior similarity comparison, and obtain a behavior type identifier and an abnormality degree index;

[0012] The analysis module is used to perform behavior continuity analysis on the behavior type identification and abnormality index, and obtain behavior recognition results and warning information through behavior risk level assessment.

[0013] In the technical solution provided by the present application, by extracting the texture features and motion trajectory features of human actions from the monitoring video sequence, and combining the behavior saliency analysis to realize feature fusion, the appearance and motion information of human behavior are effectively captured, and the integrity of feature expression is improved. By locating the skeleton joints and extracting the action nodes of the human behavior feature vector, the key parts of the human body are accurately located, and the accurate description of the human posture is realized. Based on the human action skeleton sequence data, the action connection graph and the time-series correlation edge are constructed, and the behavior pattern learning is combined with the motion paradigm analysis to establish the spatiotemporal structure representation of the behavior, and the modeling ability of complex behavior patterns is enhanced. The spatiotemporal behavior features and human behavior feature vectors are mapped to action semantics, and the behavior time chain is constructed to realize the effective mapping from the bottom feature to the high-level semantics, and the accuracy of behavior understanding is improved. Based on the behavior semantic description features, behavior pattern matching and abnormal measurement are carried out, and feature discrimination is carried out by comparing the behavior similarity, and the abnormal behavior pattern is effectively identified, which improves the sensitivity of abnormal behavior detection. Finally, the behavior type identification and abnormal degree index are continuously analyzed and the risk level is evaluated, which realizes the timely warning of abnormal behavior and ensures the safety of the monitoring scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0015] Figure 1 A schematic diagram of an embodiment of a method for identifying human behavior in video surveillance in an embodiment of the present application;

[0016] Figure 2 Schematic diagram of an embodiment of a personnel behavior recognition system for video surveillance in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The embodiment of the present application provides a method and system for identifying the behavior of people in video surveillance. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the method for identifying human behavior in video surveillance in the embodiment of the present application includes:

[0019] Step S101, extracting human action texture features and motion trajectory features from the input surveillance video sequence, performing feature fusion processing through behavior significance analysis, and obtaining a human action feature vector;

[0020] Step S102, performing skeleton joint positioning processing on the human behavior feature vector, and performing action node and joint angle extraction processing on the positioning area to obtain human action skeleton sequence data;

[0021] Step S103, constructing an action connection graph and temporal association edges for the human action skeleton sequence data, performing behavior pattern learning processing through motion paradigm analysis, and obtaining spatiotemporal behavior characteristics;

[0022] Step S104, performing action semantic mapping processing on the spatiotemporal behavior features and the human behavior feature vectors, and constructing and processing the behavior time sequence chain to obtain behavior semantic description features;

[0023] Step S105: Performing behavior pattern matching and abnormality measurement processing on the behavior semantic description features, performing feature discrimination processing by behavior similarity comparison, and obtaining a behavior type identifier and an abnormality degree index;

[0024] Step S106: Perform behavior continuity analysis on the behavior type identifier and the abnormality index, and perform behavior risk level assessment to obtain behavior recognition results and warning information.

[0025] It is understandable that the execution subject of the present application may be a video surveillance personnel behavior recognition system, or a terminal or a server, which is not limited here. The present application embodiment is described by taking a server as the execution subject as an example.

[0026] Specifically, feature extraction is performed on the input surveillance video sequence. Texture feature extraction is performed on the video frame through Gabor filter. In this process, two-dimensional Gabor filter groups of different directions and scales are set to perform convolution operations on the image to extract local texture information containing human movements, including clothing texture, limb contours and other features. At the same time, the motion vector field between adjacent frames is calculated by optical flow method to obtain the characteristics of human motion trajectory. The optical flow method obtains the speed and direction information of the moving target by calculating the displacement vector of image pixels between consecutive frames. The extracted texture features and motion features are analyzed for significance, the weight coefficients of the feature regions are calculated, and the feature components that contribute more to behavior recognition are highlighted. Finally, a comprehensive human behavior feature vector is constructed through weighted fusion. Based on the obtained human behavior feature vector, the human skeleton joints are accurately located. First, the human body region is determined by the region segmentation algorithm, and the human body contour mask is generated. Then, the key bone nodes are located in the region by using geometric constraints, including 17 main joints such as head, neck, shoulder, elbow, wrist, hip, knee and ankle. The rationality of the spatial distribution of the nodes is verified by introducing the human anatomical constraint relationship, and the connection relationship and three-dimensional angle information between the joints are calculated. For each joint point, its spatial coordinates and confidence score are recorded, and jitter is eliminated through temporal smoothing, ultimately obtaining skeleton sequence data that describes changes in human posture.

[0027] Based on the skeleton sequence data, an action connection graph is constructed. The nodes in the graph represent the skeletal joints, and the edges represent the physical connection between the joints. By calculating the Euclidean distance and angle change between the nodes, weight values ​​are assigned to the edges to characterize the strength of the joint movement. A temporal sliding window is used to analyze the correlation between consecutive frames. The window size is set to 25 frames and the step length is 5 frames. Temporal correlation edges are established in the time dimension to characterize the temporal evolution characteristics of the action. Pattern learning is performed on the constructed spatiotemporal graph structure to extract node features and edge features. The spatial and temporal information are fused through graph convolution operations to obtain a unified spatiotemporal behavior feature representation. Subsequently, the spatiotemporal behavior features are mapped to the human behavior feature vector at the semantic level to establish the corresponding relationship between the action and the semantic concept. By semantically stratifying the action sequence, the complex behavior is decomposed into basic action units, and the temporal dependency and causal relationship between the action units are analyzed. A behavior rule library is constructed based on the scene semantic constraints, and semantic reasoning is performed on the action sequence to identify the action intention. Through action rule matching and semantic association analysis, a behavior temporal chain is constructed to describe the semantic evolution process of the behavior and generate behavior semantic description features containing temporal structure.

[0028] Based on the behavior semantic description features, behavior pattern matching and anomaly detection are performed. First, the behavior features are encoded into standardized feature vectors and compared with the predefined behavior pattern library for similarity. The cosine similarity is used to measure the distance between feature vectors, and an adaptive threshold is set for behavior classification. At the same time, the abnormal measurement value of the behavior is calculated, and the degree of abnormality of the behavior is evaluated by analyzing the degree of deviation between the behavior features and the normal behavior pattern. Based on the similarity calculation and anomaly measurement results, the behavior type identification and abnormality degree index are obtained. Finally, the behavior type and abnormality degree are continuously analyzed to evaluate the duration and evolution trend of the behavior. The changing rules of the behavior features are statistically analyzed through the time series window to predict the development trend of the behavior. Combined with the scenario risk level, a risk propagation model is established to comprehensively evaluate the dangerousness of the behavior and generate graded warning information.

[0029] For example, in a shopping mall surveillance scene, the camera captured a person wandering near a jewelry counter. The system first extracted the person's motion texture features and motion trajectory, extracted a 64-dimensional texture feature vector through a Gabor filter group, and calculated the motion velocity field using the optical flow method, showing that the person repeatedly moved back and forth within a 3-meter range at a speed of 0.3-0.5 m / s. By analyzing its posture features through skeleton extraction and locating 17 key nodes, it was found that the head and torso postures changed frequently, the bending angle fluctuated between 30-45 degrees, and the head turned more than 120 degrees when looking around. Combining spatiotemporal behavior features and semantic analysis, the system identified this behavior pattern of repeated wandering and frequent looking as a suspicious behavior type. By comparing with the feature library of normal shopping behavior, the behavior similarity was calculated to be 0.35 and the abnormality metric value was 0.85, which significantly exceeded the warning threshold of 0.7. Further analysis by the system revealed that the behavior lasted for 16 minutes and occurred in the high-risk area of ​​the jewelry district. The risk level assessment value reached 8.5 (out of 10 points). After comprehensive assessment, the system generated a second-level warning information, prompting security personnel to pay close attention and conduct on-site verification.

[0030] In the embodiment of the present application, by extracting the texture features and motion trajectory features of human actions from the monitoring video sequence, and combining the behavior significance analysis to realize feature fusion, the appearance and motion information of human behavior are effectively captured, and the integrity of feature expression is improved. By locating the skeleton joints and extracting the action nodes of the human behavior feature vector, the key parts of the human body are accurately located, and the accurate description of the human posture is realized. Based on the human action skeleton sequence data, the action connection graph and the time-series correlation edge are constructed, and the behavior pattern learning is combined with the motion paradigm analysis to establish the spatiotemporal structure representation of the behavior, and enhance the modeling ability of complex behavior patterns. The spatiotemporal behavior features and human behavior feature vectors are mapped to action semantics, and the behavior time chain is constructed to realize the effective mapping from the bottom feature to the high-level semantics, and the accuracy of behavior understanding is improved. Based on the behavior semantic description features, behavior pattern matching and abnormal measurement are carried out, and feature discrimination is carried out by comparing the behavior similarity, and the abnormal behavior pattern is effectively identified, which improves the sensitivity of abnormal behavior detection. Finally, the behavior type identification and abnormal degree index are continuously analyzed and the risk level is evaluated, so as to realize the timely early warning of abnormal behavior and ensure the safety of the monitoring scene.

[0031] In a specific embodiment, the process of executing step S101 may specifically include the following steps:

[0032] (1) performing frame sampling processing on the input surveillance video sequence to obtain video frame sequence data, and performing grayscale and normalization processing on the video frame sequence data to obtain preprocessed image data;

[0033] (2) extracting texture features from the preprocessed image data using a Gabor filter to obtain a human motion texture feature matrix, and calculating motion vectors from the preprocessed image data using an optical flow method to obtain a human motion trajectory feature matrix;

[0034] (3) performing feature dimension alignment processing on the human motion texture feature matrix and the human motion trajectory feature matrix to obtain an aligned feature matrix, and performing principal component analysis processing on the aligned feature matrix to obtain dimension-reduced feature data;

[0035] (4) Performing regional weight calculation on the reduced-dimensional feature data through saliency calculation to obtain a feature weight matrix, and performing weighted combination processing on the reduced-dimensional feature data and the feature weight matrix to obtain an initial fusion feature;

[0036] (5) Performing feature correlation analysis on the initial fusion features through the covariance matrix to obtain a feature correlation matrix, and performing feature selection on the feature correlation matrix to obtain a core feature set;

[0037] (6) Performing frequency domain feature extraction processing on the core feature set through Fourier transform to obtain a frequency domain feature vector, and performing frequency domain feature screening processing on the frequency domain feature vector to obtain a frequency domain feature set;

[0038] (7) performing feature concatenation processing on the frequency domain feature set and the core feature set to obtain a combined feature vector, and performing feature standardization processing on the combined feature vector to obtain a standardized feature vector;

[0039] (8) Performing time-frequency feature analysis on the standardized feature vector through wavelet transform to obtain time-frequency feature data, and performing multi-scale feature extraction on the time-frequency feature data to obtain a human behavior feature vector.

[0040] Specifically, a frame sampling operation is performed, the sampling frequency is set to 25 frames / second, and a video frame sequence is obtained by uniform sampling. The sampled video frames are grayed, and the RGB three-channel image is converted into a single-channel grayscale image. At the same time, the pixel values ​​are normalized, and the pixel value range is uniformly mapped to the [0,1] interval to obtain standardized preprocessed image data. Based on the preprocessed image, the Gabor filter is used to extract texture features. The Gabor filter is a bandpass filter that can effectively capture local texture information. A Gabor filter group with 8 directions and 4 scales is designed to perform convolution operations on the image. The response result of each filter forms a feature map, and finally a human motion texture feature matrix consisting of 32 feature maps is obtained. At the same time, the Lucas-Kanade optical flow method is used to calculate the pixel displacement field between adjacent frames, and the optical flow equation is solved:

[0041]

[0042] in, Represents the spatial gradient of the image in the x and y directions,

[0043] represents the time gradient, v x 、v y Represent the desired optical flow velocity component, obtain the motion vector field, and construct the human motion trajectory feature matrix.

[0044] The acquired texture feature matrix and motion feature matrix are dimensionally aligned, and the two types of features are unified to the same spatial resolution through spatial interpolation to obtain the aligned feature matrix. The aligned feature matrix is ​​then subjected to principal component analysis (PCA) dimensionality reduction, the eigenvalues ​​and eigenvectors of the feature covariance matrix are calculated, and the principal components with a cumulative contribution rate of 95% are selected as the reduced dimension feature data.

[0045] Based on the feature data after dimensionality reduction, the saliency weight of the feature area is calculated. The saliency calculation is based on the local contrast and spatial distribution of the feature, using the following formula:

[0046]

[0047] Among them, W(x, y) is the significance weight, (x, y) represents the coordinates of the feature point, (μ x , μ y ) represents the center position, (σ x ,σ y_ ) represents the spatial distribution parameter, and C(x, y) represents the local contrast. After obtaining the feature weight matrix, it is weightedly combined with the reduced-dimensional feature data to generate the initial fusion feature. Feature correlation analysis is performed by calculating the covariance matrix of the initial fusion feature, the correlation between different feature components is evaluated, and features with low correlation are selected to form a core feature set. The core feature set is subjected to a one-dimensional Fourier transform to obtain the frequency domain feature vector, and the main frequency components are retained through energy threshold screening to form a frequency domain feature set. The frequency domain feature set and the core feature set are feature spliced ​​to construct a combined feature vector, and feature normalization is performed to obtain a standardized feature vector. Finally, the standardized feature vector is subjected to a wavelet transform, and the Daubechies wavelet basis function is used for multi-scale decomposition to extract the time-frequency features at different scales, and finally the human behavior feature vector is obtained.

[0048] Taking the identification of customer shopping behavior in supermarket monitoring as an example, the monitoring video is sampled at 25 frames per second with a resolution of 1920×1080 pixels. The customer's clothing texture and body shape features are extracted through Gabor filters to form 32 feature maps, each of which is 240×135 pixels in size. The motion velocity field is calculated by the optical flow method, showing that the customer moves between shelves at a speed of 0.5 meters per second. The dimension of the feature matrix after alignment is 240×135×64. The first 20 principal components are retained after PCA dimensionality reduction, and the dimension of the reduced feature is reduced to 240×135×20. The significance calculation shows that the weight values ​​of the customer's hands and shopping cart areas are relatively high, reaching more than 0.8. Feature correlation analysis selects 12 core features, and the frequency components of 80% of the energy are retained after Fourier transform. Finally, a 256-dimensional human behavior feature vector is obtained for subsequent behavior recognition analysis.

[0049] In a specific embodiment, the process of executing step S102 may specifically include the following steps:

[0050] (1) performing human body region segmentation processing on the human behavior feature vector to obtain human body region mask data, and performing bounding box positioning processing on the human body region mask data to obtain human body region bounding box data;

[0051] (2) generating candidate points of skeleton nodes for the human body region bounding box data to obtain a skeleton node candidate set, and performing node screening on the skeleton node candidate set through geometric constraints to obtain the coordinates of key skeleton nodes;

[0052] (3) Performing human anatomical constraint processing on the coordinates of key bone nodes to obtain an anatomical node relationship diagram, and constructing joint connection relationships on the anatomical node relationship diagram to obtain bone joint connection data;

[0053] (4) performing joint angle calculation processing on the skeletal joint connection data to obtain a joint angle sequence, and performing angle range verification processing on the joint angle sequence to obtain valid joint angle data;

[0054] (5) performing motion trajectory construction processing on the effective joint angle data to obtain a joint motion trajectory set, and performing action node extraction processing on the joint motion trajectory set to obtain action key node data;

[0055] (6) Performing time alignment processing on the key node data of the action to obtain an aligned skeleton sequence, and performing skeleton feature extraction processing on the aligned skeleton sequence to obtain human body action skeleton sequence data.

[0056] Specifically, human body region segmentation is performed, and the image is pixel-level classified by the semantic segmentation method, and the image is divided into human body region and background region, and binary human body region mask data is generated. In the mask data, the pixel value of the human body region is 1, and the pixel value of the background region is 0. Then the mask data is subjected to connected region analysis and boundary extraction, and the minimum circumscribed rectangle is calculated to obtain the bounding box data containing the human body, and the upper left corner coordinates and lower right corner coordinates of the bounding box are recorded. In the obtained human body region bounding box, the hierarchical feature point detection method is used to generate skeleton node candidate points. First, grid points are uniformly sampled and generated in the bounding box area, and the grid spacing is 8 pixels. Then, the local feature response value of each grid point is calculated, and the point with higher response value is selected as the candidate point. The candidate point set is screened by geometric constraints, which include distance constraints and angle constraints between point pairs. The key node coordinates that conform to the human body skeleton structure are obtained by threshold screening.

[0057] Human anatomical constraints are introduced into the coordinates of the selected bone nodes. According to the physiological structural characteristics of the human skeleton, the spatial position relationship and motion range restrictions between 17 key nodes are defined. A node relationship graph is constructed through anatomical constraint rules, where nodes represent key bone points and edges represent anatomical connection relationships between nodes. Based on the relationship graph, bone joint connection data is further constructed to record the connection properties and degrees of freedom of motion of each pair of connected joints.

[0058] Calculate the joint angle for the skeletal joint connection data. The angle calculation formula formed by three adjacent key points is:

[0059]

[0060] Among them, P1, P2, and P3 represent the coordinates of three adjacent key points. and represents a vector, and θ is the joint angle. Calculate the three-dimensional Euler angle of each joint:

[0061]

[0062] Among them, a, β, and γ represent the rotation angles around the x, y, and z axes, respectively. x , n y , n z ) represents the normal vector of the joint coordinate system, (a x , a y ) represents the projection component of the axis vector of the joint coordinate system on the xy plane. The angle validity is verified according to the physiological range of motion of the human joint, and the valid joint angle data is screened.

[0063] The joint motion trajectory is constructed based on the effective joint angle data. The cubic spline interpolation method is used to fit the motion path of the joint points in the time series to generate a continuous trajectory curve. The key points of the trajectory curve are extracted, and the points with significant curvature changes are selected as the key nodes of the action. The spatiotemporal coordinates and motion state parameters of the key nodes are recorded. Finally, the key node data of the action are aligned in time series, and the dynamic time warping algorithm is used to standardize the action sequences of different lengths, uniformly sample them to the same time length, and obtain the aligned skeleton sequence. By extracting the spatial structure features and temporal change features of the aligned skeleton sequence, the skeleton sequence data describing the human body action is finally generated.

[0064] Taking the analysis of long jump as an example, the athlete image is first segmented to obtain the human region mask with the bounding box coordinates of [(150,100),(450,500)]. 85 candidate points are detected in the bounding box, and 17 key bone points are retained after geometric constraint screening. The joint connection relationship is constructed according to the anatomical constraints, and 16 main joint connections are recorded. The calculated hip joint angle sequence changes from 120 degrees to 75 degrees during the take-off phase, and the knee joint angle changes from 165 degrees to 90 degrees, both of which are within the physiological angle range. Through spline interpolation, a continuous trajectory of 25 frames in 0.5 seconds is obtained, and 5 key action nodes are extracted, including the end of the run-up, take-off, air posture, landing preparation and landing completion. After time alignment, a standardized 80-frame skeleton sequence is generated, which fully records the changes in human posture during the long jump.

[0065] In a specific embodiment, the process of executing step S103 may specifically include the following steps:

[0066] (1) Performing skeleton node connection processing on human motion skeleton sequence data to obtain an initial motion connection graph, and performing edge weight calculation processing on the initial motion connection graph to obtain an motion connection graph;

[0067] (2) performing action feature extraction processing on the nodes in the action connection graph to obtain a node feature matrix, and performing node similarity calculation processing on the node feature matrix to obtain a node similarity matrix;

[0068] (3) Performing time series sliding window processing on the node similarity matrix to obtain time series window data, and performing time series edge construction processing on the time series window data to obtain a time series associated edge set;

[0069] (4) Perform edge weight distribution processing on the time-series associated edge set to obtain the time-series associated edges, and perform graph fusion processing on the time-series associated edges and the action connection graph to obtain the spatiotemporal action graph;

[0070] (5) performing action pattern clustering processing on the spatiotemporal action graph to obtain action pattern clusters, and performing paradigm feature extraction processing on the action pattern clusters to obtain action paradigm features;

[0071] (6) Performing behavioral pattern encoding processing on the action paradigm features to obtain a behavioral coding sequence, and performing feature extraction processing on the behavioral coding sequence through paradigm analysis to obtain spatiotemporal behavioral features.

[0072] Specifically, the connection relationship between skeleton nodes is established. According to the anatomical characteristics of the human body, 17 key bone points are connected through 16 edges to form an initial action connection diagram. The edge connection relationship includes the main bone connections such as the head to the neck, the neck to the trunk, the trunk to the left and right shoulders, the shoulder to the elbow, and the elbow to the wrist. The weight value of each edge is calculated using the following formula:

[0073]

[0074] Among them, w ij represents the weight of the edge between node i and node j, d ij represents the Euclidean distance between two nodes, v i and v j Represents the movement speed of the two nodes, and obtains a weighted action connection graph. Then, the action features are extracted for each node in the action connection graph. The node features contain information such as spatial position, movement speed, acceleration, etc. For node k, its feature vector is calculated as follows:

[0075]

[0076] Among them, (x k ,y k ) is the node space coordinate, Δxk and Δy k The node feature matrix is ​​obtained by using the displacement increment and Δt as the time interval. The feature similarity between nodes is calculated and the node similarity matrix is ​​constructed by using the cosine similarity metric. The node similarity matrix is ​​subjected to time series analysis. The sliding window size is set to 25 frames and the step length is 5 frames. Local time series features are extracted by sliding in the time dimension. In each time series window, the correlation of the node motion mode is analyzed and time series correlation edges are established. The time series correlation edges represent the dynamic correlation relationship between nodes at different times, including attributes such as time span and motion correlation.

[0077] The formed temporal correlation edge set is weighted, and the weight value reflects the strength of the temporal correlation. The temporal correlation edge is fused with the original action connection graph to construct a spatiotemporal action graph containing spatial structure and temporal evolution. Each node in the spatiotemporal action graph contains both spatial position information and temporal change characteristics. Action pattern clustering is performed based on the spatiotemporal action graph, and the spectral clustering method is used to classify similar action patterns into the same category. Clustering considers the spatial distribution and temporal change characteristics of the nodes to obtain action pattern clusters with similar motion laws. Typical features are extracted from each pattern cluster, including the spatial configuration and motion trajectory of key nodes, to form action paradigm features. Action paradigm features describe the standard pattern of a certain type of action, which has strong distinguishability and representativeness.

[0078] The action paradigm features are encoded into a fixed-dimensional feature sequence, and the encoding process takes into account the two dimensions of spatial structure and temporal change. The key patterns in the sequence are extracted through paradigm analysis, and finally the spatiotemporal behavior features that can characterize the complete behavior features are obtained.

[0079] Taking basketball shooting action analysis as an example, we first construct a skeleton connection diagram containing 17 nodes, and calculate that the edge weight from the shoulder joint to the elbow joint is 0.85, and the edge weight from the elbow joint to the wrist is 0.78. When extracting node features, the displacement of the right wrist node in 0.5 seconds is (0.8m, 1.2m), the speed is (1.6m / s, 2.4m / s), and the acceleration is (3.2m / s 2 ,4.8m / s 2 ). Using a 25-frame sliding window to analyze the timing features, it was found that the movements of the three joints of the arm (shoulder, elbow, and wrist) were highly correlated, with a similarity of 0.92. A complete action graph was obtained through spatiotemporal fusion, which contains 80 spatial edges and 120 temporal edges. Cluster analysis revealed three main action pattern clusters: preparation phase, force phase, and follow-up phase. The paradigm features of each phase were extracted, such as the arm joint angle change range of 45°-170° and the movement duration of 0.3 seconds in the force phase. Finally, a 384-dimensional spatiotemporal behavior feature vector was generated for subsequent behavior recognition analysis.

[0080] In a specific embodiment, the process of executing step S104 may specifically include the following steps:

[0081] (1) Performing action semantic layering processing on the spatiotemporal behavior characteristics and human behavior feature vectors to obtain layered action semantic elements, and performing basic action deconstruction processing on the layered action semantic elements to obtain a basic action library;

[0082] (2) Perform scene-action matching processing on the basic action library to obtain scene-related action sequences, and perform action correlation derivation processing on the scene-related action sequences to obtain an action correlation feature matrix;

[0083] (3) Performing action sequence modeling on the action association feature matrix to obtain an action dependency network, and performing semantic reasoning on the action dependency network to obtain action intention features;

[0084] (4) Mining the temporal regularity of action intention features to obtain an action rule set, and performing behavior semantic mapping on the action rule set to obtain a behavior semantic mapping graph;

[0085] (5) Performing hierarchical semantic abstraction processing on the behavior semantic mapping graph to obtain a hierarchical behavior description, and performing action constraint construction processing on the hierarchical behavior description to obtain a behavior constraint network;

[0086] (6) Performing spatiotemporal correlation verification processing on the behavior constraint network to obtain a verification behavior sequence, and performing semantic feature aggregation processing on the verification behavior sequence to obtain behavior semantic description features.

[0087] Specifically, a three-layer semantic structure is constructed: the bottom layer contains basic action units, such as raising hands, turning around, bending over, etc.; the middle layer contains combined action sequences, such as picking up items, checking items, etc.; the high layer contains complete behavioral semantics, such as shopping, patrolling, etc. The input features are decomposed according to the semantic level to obtain hierarchical action semantic elements, and then the basic action deconstruction of each layer of semantic elements is performed to extract the core components and temporal relationships of the actions, and a standardized basic action library is constructed. Based on the constructed basic action library, scene action matching is performed, and the standard action units in the action library are compared with the action sequences in the actual scene. By calculating the similarity and temporal consistency of the action features, the action sequence related to the current scene is identified. The matching scene-related action sequences are subjected to correlation analysis, and the temporal dependency and semantic correlation between the actions are calculated to generate an action correlation feature matrix. Each element in the matrix represents the correlation strength between two actions.

[0088] The action-related feature matrix is ​​sequence modeled to construct an action dependency network in the form of a directed acyclic graph. The nodes in the network represent basic action units, the edges represent the dependencies between actions, and the weights of the edges reflect the strength of the dependencies. The combination rules and semantic dependencies of actions are analyzed through the network structure. Semantic reasoning is performed on the dependency network, and the potential intentions of the action sequence are inferred by combining the scene context information to obtain the feature vector that describes the purpose and intention of the action. The temporal regularity mining of the action intention features is performed to analyze the change pattern of the action sequence in the time dimension. By identifying information such as recurring action combinations and action transition probabilities, a series of rule sets that describe the temporal regularity of actions are summarized. The rule set is mapped to the semantic space, the correspondence between the action rules and the behavior semantics is established, and a behavior semantic mapping map is constructed.

[0089] The obtained behavior semantic mapping graph is hierarchically abstracted to elevate specific action rules to a higher level of behavior semantic description. Through semantic aggregation and generalization, a multi-level behavior description system is formed, including semantic expressions at the action level, intention level, and behavior level. Based on the hierarchical behavior description, an action constraint network is constructed to define the temporal constraints, spatial constraints, and semantic constraints between actions. Finally, the behavior constraint network is verified for spatiotemporal correlation to check whether the action sequence meets various constraints. Through verification, valid behavior sequences that meet the constraints are screened out, and semantic feature aggregation is performed on these sequences to extract common features and distinguishing features, and finally generate complete behavior semantic description features.

[0090] Taking abnormal behavior identification in monitoring as an example, the input action sequence includes basic actions such as "fast walking-frequent looking-stopping and observing-quickly leaving". Through semantic stratification, it is deconstructed into underlying action units: walking speed 2.5m / s, head turning frequency 2 times / second, stop time 15 seconds, etc. In the scene matching stage, compared with the normal shopping behavior library, it is found that the abnormality of the action combination reaches 0.85. Action association analysis shows that the correlation between frequent looking and stop observation is 0.92, indicating a strong correlation. The dependency network reflects that the sequence has obvious surveillance characteristics, and the action intention tends to scene exploration. Time sequence regularity analysis found that similar action patterns repeated 4 times within 3 minutes, and the behavior pattern showed periodicity. After semantic mapping and constraint verification, the behavior sequence was finally labeled as "suspicious stomping behavior", with an abnormality of 0.88, triggering a security warning.

[0091] In a specific embodiment, the process of executing step S105 may specifically include the following steps:

[0092] (1) Performing behavior type encoding processing on the behavior semantic description features to obtain a behavior encoding vector, and performing classification standard matching processing on the behavior encoding vector to obtain a behavior category feature;

[0093] (2) extracting standard behavior patterns from the behavior category features to obtain a behavior pattern library, and calculating similarity between the behavior category features and the behavior pattern library to obtain a behavior similarity matrix;

[0094] (3) Performing threshold segmentation processing on the behavior similarity matrix to obtain similar behavior groups, and calculating the intra-group differences of the similar behavior groups to obtain the behavior deviation index;

[0095] (4) performing abnormality evaluation processing on the behavior deviation index to obtain a behavior abnormality measurement value, and performing grade classification processing on the behavior abnormality measurement value to obtain an abnormality degree grade;

[0096] (5) performing feature combination processing on the abnormality level and the behavior category features to obtain a behavior feature set, and performing category determination processing on the behavior feature set to obtain a behavior type identifier;

[0097] (6) Performing statistical analysis on the behavior type identification and abnormality level to obtain the behavior statistical characteristics, and performing indicator quantification processing on the behavior statistical characteristics to obtain the behavior type identification and abnormality level index.

[0098] Specifically, the input behavior semantic description features are first encoded for behavior types. The encoding process uses a one-hot encoding method to map different behavior types into binary vectors of fixed dimensions, and each position represents a basic behavior pattern. For example, in the supermarket monitoring scenario, the encoding dimension is 32, where the 1st to 8th bits represent mobile behaviors (such as walking, running, and staying), the 9th to 16th bits represent interactive behaviors (such as taking items and putting items back), the 17th to 24th bits represent posture behaviors (such as bending, reaching out, and looking around), and the 25th to 32nd bits represent social behaviors (such as talking, following, and gathering). The generated behavior encoding vector is matched with the predefined classification criteria to obtain the behavior category features. Subsequently, the behavior category features are subjected to standard behavior pattern extraction to construct a behavior library containing typical behavior patterns. Each pattern in the behavior pattern library contains descriptions of three dimensions: temporal features, spatial features, and semantic features. The similarity between the behavior category features to be analyzed and the standard patterns in the pattern library is calculated, and the similarity between feature vectors is measured using cosine distance to generate an NxM-dimensional behavior similarity matrix, where N is the number of behaviors to be analyzed and M is the number of standard patterns.

[0099] The behavioral similarity matrix is ​​segmented by adaptive threshold, and the threshold is set as the interval of the similarity mean plus or minus the standard deviation. Behaviors with similarity higher than the threshold are clustered into the same group to form similar behavior groups. For each similar behavior group, the differences between behaviors within the group are calculated, including time difference, spatial difference and semantic difference, and the behavioral deviation index is obtained comprehensively. The deviation index reflects the degree of abnormality of the behavior relative to the standard pattern. The abnormality is evaluated based on the behavioral deviation index. The deviation index is mapped to the interval of 0-1 to obtain a standardized behavioral abnormality measurement value. According to the distribution characteristics of the abnormality measurement value, the degree of abnormality is divided into five levels: normal (0-0.2), slight abnormality (0.2-0.4), moderate abnormality (0.4-0.6), severe abnormality (0.6-0.8) and extreme abnormality (0.8-1.0).

[0100] Combine the abnormality level with the behavior category features to construct a multi-dimensional behavior feature set. The feature set contains behavior type information, abnormality level information, time series change information, and scene context information. Analyze the feature set through the preset judgment rules to determine the final behavior type identification. The behavior type identification contains specific behavior categories and abnormal attribute tags. Finally, perform statistical analysis on the behavior type identification and abnormality level, and calculate the statistical characteristics of different types of behaviors, such as frequency of occurrence, duration, and conversion probability. Quantify the statistical features, generate a standardized indicator system, and finally output the behavior type identification and abnormality level indicators.

[0101] Taking the abnormal behavior identification in the bank business hall as an example, the monitoring system captures a person's behavior sequence. First, the behavior is encoded to generate a 32-dimensional binary vector, in which the stay position (3rd position), frequent looking (23rd position), and solitude (28th position) are all 1, indicating the existence of these basic behaviors. Compared with the standard behavior pattern library, the similarity matrix is ​​calculated. The similarity between this behavior and the normal queue waiting mode is 0.35, and the similarity with the suspicious stay mode is 0.82. A similarity threshold of 0.65 is used for segmentation and it is classified into the suspicious behavior group. Calculating the intra-group difference, it is found that the behavior deviates significantly from the normal mode in terms of stay time (more than 30 minutes) and looking frequency (an average of 6 times per minute), and the behavior deviation index reaches 0.78. After the abnormality evaluation, it is determined to be a serious abnormality level. Combined with the behavior feature analysis, the "suspicious stomping" behavior type is identified. The final abnormality index generated is 0.78, triggering the second-level warning of the security system.

[0102] In a specific embodiment, the process of executing step S106 may specifically include the following steps:

[0103] (1) Perform time series window segmentation processing on the behavior type identification and abnormality degree index to obtain a time series analysis sequence, and perform behavior duration statistics processing on the time series analysis sequence to obtain the behavior duration feature;

[0104] (2) Analyzing and processing the behavior evolution trend of the behavior persistence characteristics to obtain behavior trend data, and predicting the development of the behavior trend data to obtain the behavior development sequence;

[0105] (3) Evaluate the degree of danger of the behavior development sequence to obtain risk assessment indicators, and classify the risk assessment indicators into risk levels to obtain risk level data;

[0106] (4) Perform scenario-related analysis on the risk level data to obtain scenario risk characteristics, and perform risk propagation path analysis on the scenario risk characteristics to obtain a risk propagation map;

[0107] (5) Performing early warning level determination processing on the risk propagation map to obtain early warning level data, and performing early warning information generation processing on the early warning level data to obtain an early warning message set;

[0108] (6) Summarize and process the warning message set and behavior development sequence to obtain behavior recognition results and warning information.

[0109] Specifically, in the process of processing the behavior type identification and abnormality index, the time series window segmentation method is first used to set the window size to 30 seconds and the sliding step size to 10 seconds to segment the continuous behavior sequence. The behavior data in each time window is statistically analyzed to record the distribution of behavior types and the changing trend of abnormality. By cumulatively counting the duration, frequency of occurrence and conversion law of each behavior type, the characteristic vector describing the continuous characteristics of the behavior is obtained. Trend analysis is performed based on the continuous characteristics of the behavior, and the changing law of the characteristic value in the time series is statistically analyzed. The analysis includes the evolution trend of the behavior type, the progressive relationship of the abnormality degree and the conversion characteristics of the behavior pattern. The time series prediction method is used for the trend data to predict the development trend of the behavior in the future. The prediction content includes the probability of change of the behavior type, the growth rate of the abnormality degree and the potential behavior conversion point.

[0110] The predicted behavior development sequence is evaluated for its degree of danger. The evaluation indicators include three dimensions: harmfulness, scope of impact, and urgency. Harmfulness is quantified based on the destructive power and threat level of the behavior. The scope of impact considers the spatial scale and number of people affected by the behavior. The urgency is based on the time urgency of the development of the situation. The scores of these three dimensions are combined to generate standardized risk assessment indicators. According to the numerical distribution of the evaluation indicators, the risk level is divided into five levels: slight risk (0-0.2), low risk (0.2-0.4), moderate risk (0.4-0.6), high risk (0.6-0.8) and extremely serious risk (0.8-1.0). The risk level data is correlated with specific scenarios. Considering the characteristics of the scenario, such as importance, vulnerability, and protection ability, the actual impact of the risk in different scenarios is evaluated. For example, the same risk level in a densely populated area is more threatening than in a remote area. Based on the results of the scenario correlation analysis, a risk propagation path map is constructed to describe the diffusion process of the risk in the spatial and temporal dimensions. The nodes in the map represent the risk impact points, the edges represent the risk propagation channels, and the weights reflect the propagation probability.

[0111] The warning level is determined based on the risk propagation map, and four warning levels are set: blue warning (minor threat), yellow warning (medium threat), orange warning (serious threat) and red warning (particularly serious threat). Corresponding disposal measures and response processes are specified for each warning level, and a warning message set containing threat description, impact range, and disposal suggestions is generated. Finally, a comprehensive analysis and information summary of the warning message set and behavior development sequence are conducted to form a complete behavior identification result and warning information. The summary content includes behavior type determination, abnormality degree assessment, risk level determination, and warning measure recommendations.

[0112] Taking the security monitoring of a large shopping mall as an example, the monitoring system detected a behavioral sequence of a group of suspicious persons. Within the 30-second time window, three main behaviors were recorded: repeated patrols (accounting for 45%), scattered wandering (accounting for 35%), and whispered conversations (accounting for 20%). The behavior persistence characteristics show that the patrol behavior was repeated 6 times within 15 minutes, each lasting 2-3 minutes. Trend analysis predicts that the activity range of this group of people will expand to the valuable commodity area, and the degree of abnormality is on the rise, rising by 0.3 levels within 15 minutes. The risk level assessment shows that the current risk index is 0.75, which is a high risk level. Scenario analysis found that this area is the jewelry area of ​​the mall, with a large passenger flow density and a high level of protection. The risk propagation path analysis shows that the risk impact range will cover the entire jewelry area within 5 minutes. Based on the above analysis, an orange warning is triggered and a warning message is generated: "A suspicious group of multiple people is found in the jewelry area, showing obvious characteristics of stomping. It is recommended that security personnel increase the patrol frequency and focus on key areas."

[0113] The above describes the method for identifying the behavior of a person in video surveillance in the embodiment of the present application. The following describes the system for identifying the behavior of a person in video surveillance in the embodiment of the present application. Figure 2 In the embodiment of the present application, an embodiment of the video surveillance personnel behavior recognition system includes:

[0114] The acquisition module 201 is used to extract the human action texture features and motion trajectory features from the input monitoring video sequence, perform feature fusion processing through behavior significance analysis, and obtain the human action feature vector;

[0115] A positioning module 202 is used to perform skeleton joint positioning processing on the human behavior feature vector, and extract action nodes and joint angles from the positioning area to obtain human action skeleton sequence data;

[0116] A construction module 203 is used to construct an action connection graph and a time-series associated edge for the human action skeleton sequence data, and to perform behavior pattern learning processing through motion paradigm analysis to obtain spatiotemporal behavior characteristics;

[0117] A mapping module 204 is used to perform action semantic mapping processing on the spatiotemporal behavior features and the human behavior feature vectors, and obtain behavior semantic description features through behavior time sequence chain construction processing;

[0118] The measurement module 205 is used to perform behavior pattern matching and abnormal measurement processing on the behavior semantic description features, perform feature discrimination processing by behavior similarity comparison, and obtain a behavior type identifier and an abnormality degree index;

[0119] The analysis module 206 is used to perform behavior continuity analysis on the behavior type identifier and abnormality index, and obtain behavior recognition results and warning information through behavior risk level assessment.

[0120] Through the collaboration of the above components, the human action texture features and motion trajectory features are extracted from the surveillance video sequence, and feature fusion is achieved by combining behavioral saliency analysis, which effectively captures the appearance and motion information of human behavior and improves the integrity of feature expression. By locating the skeleton joints and extracting the action nodes of the human behavior feature vector, the key parts of the human body are accurately located, and the accurate description of the human posture is achieved. Based on the human action skeleton sequence data, the action connection graph and time-series correlation edges are constructed, and the behavior pattern learning is combined with the motion paradigm analysis to establish the spatiotemporal structure representation of the behavior, which enhances the modeling ability of complex behavior patterns. The spatiotemporal behavior features and human behavior feature vectors are mapped to action semantics, and the behavior time chain is constructed to achieve effective mapping from low-level features to high-level semantics, thereby improving the accuracy of behavior understanding. Behavior pattern matching and anomaly measurement are performed on the basis of the behavior semantic description features, and feature discrimination is performed by comparing the behavior similarity, effectively identifying abnormal behavior patterns, and improving the sensitivity of abnormal behavior detection. Finally, the behavior type identification and abnormal degree indicators are continuously analyzed and the risk level is evaluated, which realizes the timely warning of abnormal behavior and ensures the safety of the monitoring scene.

[0121] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying human behavior in video surveillance, characterized in that: The method for identifying human behavior through video surveillance comprises: Extract human action texture features and motion trajectory features from the input surveillance video sequence, perform feature fusion processing through behavior saliency analysis, and obtain human behavior feature vectors; Performing skeleton joint positioning processing on the human behavior feature vector, and extracting action nodes and joint angles from the positioning area to obtain human action skeleton sequence data; Constructing an action connection graph and a temporal correlation edge for the human action skeleton sequence data, performing behavior pattern learning processing through motion paradigm analysis, and obtaining spatiotemporal behavior characteristics; Performing action semantic mapping processing on the spatiotemporal behavior features and human behavior feature vectors, and constructing and processing the behavior time sequence chain to obtain behavior semantic description features; Performing behavior pattern matching and abnormality measurement processing on the behavior semantic description features, performing feature discrimination processing by behavior similarity comparison, and obtaining a behavior type identifier and an abnormality degree index; The behavior type identification and abnormality degree index are subjected to behavior continuity analysis and processing, and the behavior risk level is evaluated to obtain behavior recognition results and early warning information.

2. The method for identifying human behavior in video surveillance according to claim 1, characterized in that: The human action texture features and motion trajectory features are extracted from the input surveillance video sequence, and feature fusion processing is performed through behavior saliency analysis to obtain a human action feature vector, including: Performing frame sampling processing on the input monitoring video sequence to obtain video frame sequence data, and performing grayscale and normalization processing on the video frame sequence data to obtain preprocessed image data; Performing texture feature extraction processing on the preprocessed image data through a Gabor filter to obtain a human motion texture feature matrix, and performing motion vector calculation processing on the preprocessed image data through an optical flow method to obtain a human motion trajectory feature matrix; Performing feature dimension alignment processing on the human action texture feature matrix and the human motion trajectory feature matrix to obtain an aligned feature matrix, and performing principal component analysis processing on the aligned feature matrix to obtain dimension-reduced feature data; Performing regional weight calculation processing on the reduced-dimensional feature data through saliency calculation to obtain a feature weight matrix, and performing weighted combination processing on the reduced-dimensional feature data and the feature weight matrix to obtain an initial fusion feature; Performing feature correlation analysis on the initial fusion features through a covariance matrix to obtain a feature correlation matrix, and performing feature selection on the feature correlation matrix to obtain a core feature set; Performing frequency domain feature extraction processing on the core feature set through Fourier transform to obtain a frequency domain feature vector, and performing frequency domain feature screening processing on the frequency domain feature vector to obtain a frequency domain feature set; Performing feature splicing processing on the frequency domain feature set and the core feature set to obtain a combined feature vector, and performing feature standardization processing on the combined feature vector to obtain a standardized feature vector; The standardized feature vector is subjected to time-frequency feature analysis processing through wavelet transform to obtain time-frequency feature data, and the time-frequency feature data is subjected to multi-scale feature extraction processing to obtain a human behavior feature vector.

3. The method for identifying human behavior in video surveillance according to claim 1, characterized in that: The method of performing skeleton joint positioning processing on the human behavior feature vector and extracting action nodes and joint angles from the positioning area to obtain human action skeleton sequence data includes: Performing human body region segmentation processing on the human body behavior feature vector to obtain human body region mask data, and performing bounding box positioning processing on the human body region mask data to obtain human body region bounding box data; Performing a skeleton node candidate point generation process on the human body region bounding box data to obtain a skeleton node candidate set, and performing a node screening process on the skeleton node candidate set through geometric constraints to obtain key skeleton node coordinates; Performing human anatomical constraint processing on the key bone node coordinates to obtain an anatomical node relationship diagram, and performing joint connection relationship construction processing on the anatomical node relationship diagram to obtain bone joint connection data; Performing joint angle calculation processing on the skeletal joint connection data to obtain a joint angle sequence, and performing angle range verification processing on the joint angle sequence to obtain valid joint angle data; Performing motion trajectory construction processing on the effective joint angle data to obtain a joint motion trajectory set, and performing action node extraction processing on the joint motion trajectory set to obtain action key node data; The action key node data is subjected to time sequence alignment processing to obtain an aligned skeleton sequence, and the aligned skeleton sequence is subjected to skeleton feature extraction processing to obtain human body action skeleton sequence data.

4. The method for identifying human behavior in video surveillance according to claim 1, characterized in that: The step of constructing an action connection graph and a temporal correlation edge for the human action skeleton sequence data, and performing behavior pattern learning processing through motion paradigm analysis to obtain spatiotemporal behavior features includes: Performing skeleton node connection processing on the human body motion skeleton sequence data to obtain an initial motion connection graph, and performing edge weight calculation processing on the initial motion connection graph to obtain an motion connection graph; Performing action feature extraction processing on the nodes in the action connection graph to obtain a node feature matrix, and performing node similarity calculation processing on the node feature matrix to obtain a node similarity matrix; Performing time series sliding window processing on the node similarity matrix to obtain time series window data, and performing time series edge construction processing on the time series window data to obtain a time series associated edge set; Performing edge weight distribution processing on the time series associated edge set to obtain time series associated edges, and performing graph fusion processing on the time series associated edges and the action connection graph to obtain a spatiotemporal action graph; Performing action pattern clustering processing on the spatiotemporal action graph to obtain action pattern clusters, and performing paradigm feature extraction processing on the action pattern clusters to obtain action paradigm features; The action paradigm features are subjected to behavior pattern coding processing to obtain a behavior coding sequence, and the behavior coding sequence is subjected to feature extraction processing through paradigm analysis to obtain spatiotemporal behavior features.

5. The method for identifying human behavior in video surveillance according to claim 1, characterized in that: The step of performing action semantic mapping on the spatiotemporal behavior features and the human behavior feature vectors and constructing and processing the behavior time sequence chain to obtain behavior semantic description features includes: Performing action semantic layering processing on the spatiotemporal behavior characteristics and human behavior feature vectors to obtain layered action semantic elements, and performing basic action deconstruction processing on the layered action semantic elements to obtain a basic action library; Performing scene action matching processing on the basic action library to obtain a scene-related action sequence, and performing action correlation derivation processing on the scene-related action sequence to obtain an action correlation feature matrix; Performing action sequence modeling processing on the action association feature matrix to obtain an action dependency network, and performing semantic reasoning processing on the action dependency network to obtain action intention features; Performing temporal regularity mining processing on the action intention features to obtain an action rule set, and performing behavior semantic mapping processing on the action rule set to obtain a behavior semantic mapping graph; Performing hierarchical semantic abstraction processing on the behavior semantic mapping graph to obtain a hierarchical behavior description, and performing action constraint construction processing on the hierarchical behavior description to obtain a behavior constraint network; The behavior constraint network is subjected to spatiotemporal correlation verification processing to obtain a verification behavior sequence, and the verification behavior sequence is subjected to semantic feature aggregation processing to obtain a behavior semantic description feature.

6. The method for identifying human behavior in video surveillance according to claim 1, characterized in that: The behavior pattern matching and abnormality measurement processing are performed on the behavior semantic description features, and feature discrimination processing is performed by behavior similarity comparison to obtain the behavior type identification and abnormality degree index, including: Performing behavior type encoding processing on the behavior semantic description feature to obtain a behavior encoding vector, and performing classification standard matching processing on the behavior encoding vector to obtain a behavior category feature; Performing standard behavior pattern extraction processing on the behavior category features to obtain a behavior pattern library, and performing similarity calculation processing on the behavior category features and the behavior pattern library to obtain a behavior similarity matrix; Performing threshold segmentation processing on the behavior similarity matrix to obtain similar behavior groups, and performing intra-group difference calculation processing on the similar behavior groups to obtain a behavior deviation index; Performing abnormality evaluation processing on the behavior deviation indicator to obtain a behavior abnormality measurement value, and performing grade classification processing on the behavior abnormality measurement value to obtain an abnormality degree grade; Performing feature combination processing on the abnormality level and behavior category features to obtain a behavior feature set, and performing category determination processing on the behavior feature set to obtain a behavior type identifier; The behavior type identification and the abnormality level are statistically analyzed to obtain the behavior statistical characteristics, and the behavior statistical characteristics are quantitatively processed to obtain the behavior type identification and the abnormality level index.

7. The method for identifying human behavior in video surveillance according to claim 1, characterized in that: The behavior type identification and abnormality index are subjected to behavior continuity analysis and processing, and the behavior risk level is evaluated to obtain behavior identification results and warning information, including: Performing time series window segmentation processing on the behavior type identifier and the abnormality degree index to obtain a time series analysis sequence, and performing behavior duration statistics processing on the time series analysis sequence to obtain a behavior duration feature; Performing behavior evolution trend analysis on the behavior persistence characteristics to obtain behavior trend data, and performing event development prediction processing on the behavior trend data to obtain a behavior development sequence; Performing risk assessment processing on the behavior development sequence to obtain a risk assessment index, and performing risk level classification processing on the risk assessment index to obtain risk level data; Performing scenario association analysis on the risk level data to obtain scenario risk characteristics, and performing risk propagation path analysis on the scenario risk characteristics to obtain a risk propagation map; Performing warning level determination processing on the risk propagation map to obtain warning level data, and performing warning information generation processing on the warning level data to obtain a warning message set; The warning message set and the behavior development sequence are aggregated and processed to obtain behavior recognition results and warning information.

8. A video surveillance personnel behavior recognition system, used to implement the video surveillance personnel behavior recognition method according to any one of claims 1 to 7, characterized in that: The personnel behavior recognition system of the video surveillance comprises: The acquisition module is used to extract the texture features and motion trajectory features of human actions from the input surveillance video sequence, perform feature fusion processing through behavior significance analysis, and obtain the human behavior feature vector; A positioning module is used to perform skeletal joint positioning processing on the human behavior feature vector, and to extract action nodes and joint angles from the positioning area to obtain human action skeleton sequence data; A construction module is used to construct an action connection graph and a time-series associated edge for the human action skeleton sequence data, and to perform behavior pattern learning processing through motion paradigm analysis to obtain spatiotemporal behavior characteristics; A mapping module is used to perform action semantic mapping processing on the spatiotemporal behavior characteristics and human behavior feature vectors, and obtain behavior semantic description characteristics through behavior time sequence chain construction processing; A measurement module is used to perform behavior pattern matching and abnormal measurement processing on the behavior semantic description features, perform feature discrimination processing by behavior similarity comparison, and obtain a behavior type identifier and an abnormality degree index; The analysis module is used to perform behavior continuity analysis on the behavior type identification and abnormality index, and obtain behavior recognition results and warning information through behavior risk level assessment.

Images