Dynamic scene recognition method and device for intelligent video monitoring equipment

Abstract

The invention provides a dynamic scene identification method and device for intelligent video monitoring equipment. According to the dynamic scene recognition method and device of the intelligent video monitoring equipment, continuous modeling of the target track in the dynamic scene and accurate recognition of the multi-target interaction relation can be achieved, the group behavior recognition accuracy is remarkably improved, and the problem of track breakage caused by spatio-temporal feature splitting in a traditional method is effectively solved.

Description

Technical Field

[0001] This invention relates to the field of surveillance technology, and in particular to a method and apparatus for dynamic scene recognition in intelligent video surveillance equipment. Background Technology

[0002] Intelligent video surveillance, as a crucial technological support for public safety, is widely used in complex scenarios such as transportation hubs and commercial complexes. With the development of AI vision technology, existing systems often employ a cascaded architecture of Convolutional Neural Networks (CNNs) and Long Short-Term Memory Networks (LSTMs), achieving dynamic scene recognition through the collaborative operation of spatial feature extraction and temporal series modeling. Specifically, this technology encompasses the entire process from video frame sequence generation and spatiotemporal feature extraction to multi-task processing. CNNs handle single-frame spatial feature extraction, LSTMs process temporal information, attention mechanisms focus on key areas, and multi-task mechanisms simultaneously perform target detection and behavior prediction. However, this traditional architecture suffers from systemic flaws in complex dynamic scenarios, resulting in a technical bottleneck where the group behavior recognition rate struggles to exceed 85%.

[0003] In existing technologies, the problem of fragmented spatiotemporal feature processing is particularly prominent. Specifically, the CNN-LSTM concatenated architecture requires extracting single-frame spatial features before splicing temporal information, resulting in a trajectory fragmentation rate of 15%-20% for moving targets, and it cannot model the physical interaction relationships between targets (such as follow / aggregate vectors). Based on this, static multi-task mechanisms suffer from a target tracking ID switching rate exceeding 10% in rainy or foggy weather due to the lack of sensor parameters, and a false alarm rate exceeding 5% in low-light nighttime scenes. Simultaneously, fixed-interval frame extraction (e.g., 30fps) wastes computational resources during smooth motion, with a critical event start frame loss rate exceeding 30%. More seriously, existing systems output discrete detection boxes and classification labels, failing to present event association chains such as abandoned objects and gathered crowds, requiring security personnel to manually correlate information across multiple screens, resulting in an abnormal event response delay exceeding 5 seconds. These deficiencies directly impact the accurate identification of group behavior in dynamic scenes and the timely handling of security incidents. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose a dynamic scene recognition method for intelligent video surveillance equipment.

[0006] The second objective of this invention is to provide a dynamic scene recognition device for intelligent video surveillance equipment.

[0007] To achieve the above objectives, a first aspect of the present invention provides a dynamic scene recognition method for intelligent video surveillance equipment, comprising: S1: Acquire dynamic scene video streams through multi-angle cameras, calculate the average motion vector of the scene based on dense optical flow field, and dynamically switch between high-frequency sampling mode and mutation frame detection mode according to the comparison result of motion vector and preset threshold to generate a spatiotemporal key frame sequence containing action mutation frames. S2, input the spatiotemporal keyframe sequence into the 3D spatiotemporal graph convolutional network, use a 3D separable convolutional kernel with dynamically adapted sampling rate to extract spatiotemporal features, and construct a spatiotemporal relationship graph with the detected target as the node and the node attributes including position velocity vector and appearance features; S3 updates the node features of the spatiotemporal relationship graph through a graph attention mechanism. The attention coefficient is jointly determined by the feature similarity and spatial relative position of the current node and its neighboring nodes. The edge weight is calculated by a linear combination of the cosine similarity of the relative motion trend between targets and the exponential decay function of spatial distance. The linear coefficient is a learnable parameter. S4 constructs a scene event knowledge graph based on the node states output by the graph neural network. When a preset abnormal event node is detected to form a closed-loop associated path, a linkage alarm is triggered and a dynamic scene semantic graph with spatiotemporal stamps is output.

[0008] In one embodiment of the present invention, S1 includes: S11, when the average motion vector of the scene exceeds the threshold, the high-frequency sampling mode is activated, and the action change frame is identified by the joint criterion that the inter-frame structural similarity is less than 0.7 or the optical flow acceleration L2 norm exceeds the preset threshold Γ. S12 performs spatiotemporal alignment processing on the retained mutation frames and adjacent temporal frames to generate a spatiotemporal keyframe sequence containing timestamps and motion vectors, compressing the data volume to 14.5% of the original video.

[0009] In one embodiment of the present invention, S2 includes: S21 uses a 5×7×7 separable convolutional kernel to dynamically adapt to a 60fps sampling rate and outputs a 128-dimensional spatiotemporal feature map, where the length of the convolutional kernel in the time dimension is automatically adjusted according to the current sampling frequency. S22, When constructing the spatiotemporal relationship graph, the node attributes include the target's three-dimensional position coordinates, velocity vector direction angle, and appearance feature vector. The edge weight calculation formula is α·cos(v_i,v_j)+β·exp(-||p_i-p_j||), where α and β are trainable parameters learned through backpropagation.

[0010] In one embodiment of the present invention, S3 includes: S31 uses a weighted summation of neighborhood node features to update node features. When calculating the attention coefficient, a spatial constraint term is introduced, and an exponential decay penalty factor is applied to nodes that are more than 5 meters apart. S32 uses a multilayer perceptron to perform cluster analysis on the 32-dimensional relationship feature vectors, classifying the interaction relationship types into three categories: follow, cluster, and avoid. Each type of relationship corresponds to an independent attention weight matrix.

[0011] In one embodiment of the present invention, S4 includes: S41 defines an abnormal clustering event as more than 5 targets within a radius of 3 meters for 10 seconds, a fast movement event as a speed exceeding 5 meters / second for 3 seconds, and a left-behind object event as a static target whose duration exceeds 30 seconds. S42, when a closed-loop path A→B→C is detected, the audible and visual alarm is activated and a safety event report containing a heat map of the target trajectory is generated, where A is an abnormal cluster node, B is a left-behind object node, and C is a fast-moving node.

[0012] To achieve the above objectives, a second aspect of the present invention provides a dynamic scene recognition method apparatus for intelligent video surveillance equipment, comprising: The dynamic sampling control module is used to acquire dynamic scene video streams through multi-angle cameras, calculate the average motion vector of the scene based on the dense optical flow field, and dynamically switch between high-frequency sampling mode and sudden frame detection mode according to the comparison result of motion vector and preset threshold, so as to generate a spatiotemporal key frame sequence containing action sudden frames. The 3D spatiotemporal feature processing module is used to input the spatiotemporal keyframe sequence into the 3D spatiotemporal graph convolutional network, extract spatiotemporal features using a 3D separable convolutional kernel with dynamically adapted sampling rate, and construct a spatiotemporal relationship graph with the detected target as the node and the node attributes including position and velocity vectors and appearance features. The graph attention interaction analysis module is used to update the node features of the spatiotemporal relationship graph through the graph attention mechanism. The attention coefficient is jointly determined by the feature similarity and spatial relative position of the current node and its neighboring nodes. The edge weight is calculated by a linear combination of the cosine similarity of the relative motion trend between targets and the exponential decay function of spatial distance. The linear coefficient is a learnable parameter. The event knowledge graph construction and alarm module is used to construct a scene event knowledge graph based on the node status output by the graph neural network. When a preset abnormal event node is detected to form a closed-loop associated path, a linkage alarm is triggered and a dynamic scene semantic graph with spatiotemporal stamps is output.

[0013] The method and apparatus of this invention can realize continuous modeling of target trajectories and accurate identification of multi-target interaction relationships in dynamic scenes, significantly improving the accuracy of group behavior recognition and system environmental adaptability.

[0014] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0015] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a dynamic scene recognition method for an intelligent video surveillance device according to an embodiment of the present invention; Figure 2 This is a structural diagram of a dynamic scene recognition device for an intelligent video surveillance equipment according to an embodiment of the present invention. Detailed Implementation

[0016] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] The following description, with reference to the accompanying drawings, illustrates a dynamic scene recognition method and apparatus for intelligent video surveillance equipment according to an embodiment of the present invention.

[0019] Example 1 Figure 1 This is a flowchart of a dynamic scene recognition method for an intelligent video surveillance device according to an embodiment of the present invention, such as... Figure 1 As shown, it includes: S1 acquires dynamic scene video streams through multi-angle cameras, calculates the average motion vector of the scene based on the dense optical flow field, and dynamically switches between high-frequency sampling mode and mutation frame detection mode according to the comparison result of the motion vector and the preset threshold to generate a spatiotemporal keyframe sequence containing action mutation frames.

[0020] Specifically, this step involves acquiring dynamic scene video streams using multi-angle cameras and calculating the scene's average motion vector based on a dense optical flow field. This enables intelligent switching of dynamic sampling modes, ultimately generating a spatiotemporal keyframe sequence containing frames with abrupt action changes. This step is the core component of the entire dynamic scene recognition system, enabling efficient video processing and key event capture.

[0021] Furthermore, the system employs a multi-angle steerable PTZ camera array (including a 120° wide-angle lens and a 40x optical zoom telephoto lens) to capture 1080P resolution video streams at a base frequency of 25fps. The front-end FPGA chip incorporates an optical flow computing accelerator to calculate the dense optical flow field between consecutive video frames in real time, and calculates the average motion vector based on the optical flow vector distribution, with units of pixels per frame. When this vector exceeds a preset threshold θ (e.g., 3.5 pixels per frame), the system automatically switches to a high-frequency sampling mode (above 60fps) to capture fast motion events; otherwise, it enters a sudden frame detection mode, identifying sudden motion frames through a joint criterion of inter-frame structural similarity (SSIM) and optical flow acceleration.

[0022] Furthermore, this step is applicable to complex dynamic monitoring scenarios such as stations and squares. Especially when emergencies occur (such as abandoned objects, crowd gatherings, or rapid movement), it can accurately capture key frames, providing high-quality input for subsequent 3D spatiotemporal graph convolutional networks (3D-STGCN), thereby improving the accuracy of behavior recognition and event association.

[0023] Furthermore, the dynamic sampling strategy effectively balances computational resources and event capture accuracy, resolving the contradiction between resource waste and keyframe loss in the traditional fixed sampling mode. This lays a solid foundation for subsequent multi-task decision-making and event loop determination, demonstrating significant practical value and innovation.

[0024] Furthermore, S1 includes: S11. When the average motion vector of the scene exceeds the threshold, the high-frequency sampling mode is activated, and the action mutation frame is identified by the joint criterion that the inter-frame structural similarity is less than 0.7 or the optical flow acceleration L2 norm exceeds the preset threshold Γ.

[0025] Specifically, when the average motion vector of the scene exceeds a preset threshold θ, the system will activate a high-frequency sampling mode of 60fps or higher. This step is one of the keyframe processing mechanisms in the dynamic scene recognition method of the intelligent video surveillance equipment of this invention, aiming to improve the ability to capture fast-moving events while optimizing the efficiency of computing resource utilization. In some implementations, this motion vector is obtained by statistically analyzing the dense optical flow field between consecutive video frames through the optical flow calculation module in the front-end FPGA chip. Specifically, it calculates the average magnitude of the optical flow vector of all pixels, in pixels per frame. When this average value exceeds a set threshold θ (e.g., 3.5 pixels per frame), it indicates that there is a significant change in motion in the scene, and the system will automatically switch to the high-frequency sampling mode to capture video at a rate of 60 frames per second, thereby ensuring high-precision capture of key actions.

[0026] Furthermore, in high-frequency sampling mode, the system still needs to identify frames with abrupt action changes to avoid redundant data caused by the increased sampling frequency. To this end, this invention employs a joint criterion mechanism, combining inter-frame structural similarity (SSIM) and the L2 norm of optical flow acceleration for judgment. Specifically, SSIM measures the similarity of adjacent frames in structural information; when its value is below 0.7, it indicates a significant structural change between frames, potentially corresponding to abrupt action changes. Simultaneously, the L2 norm of optical flow acceleration quantifies the acceleration change of the target motion; when it exceeds a preset threshold Γ, it also triggers abrupt frame marking. Optionally, the system can retain frames that meet either condition and extract several frames before and after them to construct a complete event context.

[0027] The technical advantage of this invention lies in the fact that, through a combined mechanism of dynamic sampling and mutation frame recognition, the system can significantly reduce the amount of invalid frames processed while ensuring that no critical events are missed, thereby improving overall recognition efficiency and response speed. In a specific embodiment, this mechanism compresses a 10-second video segment of the original 600 frames to 87 frames, reducing the data volume to 14.5%, while achieving an event start frame capture rate of 99.7% and controlling the computation latency to within 76ms, providing high-quality, low-redundancy input data for subsequent 3D spatiotemporal graph convolutional network processing.

[0028] S12 performs spatiotemporal alignment processing on the retained mutation frames and adjacent temporal frames to generate a spatiotemporal keyframe sequence containing timestamps and motion vectors, compressing the data volume to 14.5% of the original video.

[0029] Specifically, this step involves spatiotemporally aligning the retained mutation frames with adjacent temporal frames to generate a spatiotemporal keyframe sequence containing timestamps and motion vectors, and compressing the data volume to 14.5% of the original video. This step is the core of video frame processing and feature extraction in this invention, aiming to improve the efficiency and accuracy of dynamic scene recognition.

[0030] Furthermore, this step first calculates the average motion vector of the scene based on the dense optical flow field to determine the intensity of motion in the current video segment. When the motion vector exceeds a preset threshold θ (e.g., 3.5 pixels / frame), the system enters a high-frequency sampling mode (above 60fps) to capture rapidly changing scene content; otherwise, the system enters a keyframe mutation detection mode, using inter-frame structural similarity (SSIM) and the L2 norm of optical flow acceleration as joint criteria. Specifically, the system retains frames with an SSIM below 0.7 or an optical flow acceleration L2 norm exceeding a preset threshold Γ to ensure the integrity of key action mutation frames.

[0031] Furthermore, the system performs spatiotemporal alignment processing on the retained abrupt change frames and adjacent frames, including timestamp synchronization and motion vector interpolation. Timestamps are precisely annotated using the system clock and camera frame rate synchronization mechanism, with errors controlled within ±10ms. Motion vectors are calculated using optical flow estimation algorithms (such as Farneback or deep learning optical flow models) and interpolated using inter-frame displacement compensation techniques to construct continuous motion trajectories. The aligned frame sequence is input into a 3D spatiotemporal graph convolutional network (3D-STGCN) for feature extraction. Its graph structure uses the detected target as nodes, and the edge weights are calculated as a linear combination of the cosine similarity of the relative motion trend and the exponential decay function of the spatial distance, where α and β are learnable parameters.

[0032] S2 inputs the spatiotemporal keyframe sequence into a 3D spatiotemporal graph convolutional network, uses a dynamically adaptable 3D separable convolutional kernel to extract spatiotemporal features, and constructs a spatiotemporal relationship graph with the detected target as the node and the node attributes including position velocity vectors and appearance features.

[0033] Specifically, the spatiotemporal keyframe sequence is input into a 3D spatiotemporal graph convolutional network (3D-STGCN). A dynamically adaptable 3D separable convolutional kernel is used to extract spatiotemporal features, and a spatiotemporal relationship graph is constructed with the detected target as a node, whose attributes include position and velocity vectors and appearance features. This is one of the core steps in achieving dynamic scene recognition and behavior modeling in this invention. This step, by fusing multimodal information from both temporal and spatial dimensions, significantly improves the accuracy of target behavior recognition and semantic understanding in complex scenes.

[0034] Further, this step first inputs the spatiotemporal keyframe sequence processed by the adaptive frame extraction algorithm into the 3D-STGCN network. This network uses a 5×7×7 three-dimensional separable convolutional kernel, where the temporal dimension (5) is dynamically adapted according to the input frame rate. For example, in the 60fps sampling mode, the temporal kernel length can be extended to 5 frames to ensure accurate modeling of fast motion. The separable convolutional structure decouples spatial and temporal convolutions, processing intra-frame spatial features and inter-frame temporal changes separately, thereby reducing computational complexity and improving feature extraction efficiency. The output of the convolution operation is a 128-dimensional spatiotemporal feature map, which is used for subsequent graph structure construction.

[0035] Furthermore, the system treats each detected target (such as a pedestrian or an object) as a node in the graph. Node attributes include the target's position coordinates, velocity vector (e.g., 0.8 m / s, 30° direction), and appearance features (e.g., color histogram, texture features, etc.). Edge weights are calculated as a linear combination of the cosine similarity of the relative motion trends between targets and the exponential decay function of spatial distance, using the following formula: , where α and β are learnable parameters that are optimized through backpropagation to adapt to different interaction modes in different scenarios.

[0036] Furthermore, it is particularly applicable to complex monitoring scenarios with high density and multi-target interactions such as stations and shopping malls. By constructing a spatio-temporal relationship graph, the system can identify behavior patterns such as following, clustering, and avoidance among targets, providing a structured semantic representation for subsequent abnormal behavior detection and event correlation. In addition, this graph structure supports the graph attention mechanism to aggregate multi-hop neighbor information, further enhancing the context awareness ability of node features.

[0037] Furthermore, this step realizes the joint spatio-temporal modeling of target behaviors, effectively solving the problem of spatio-temporal feature fragmentation in the traditional CNN+LSTM architecture. By introducing a graph structure, the system can directly encode the interaction relationships between targets, improving the accuracy of group behavior recognition to 97.9% and significantly reducing the trajectory breakage phenomenon. At the same time, this graph structure provides a basic support for subsequent event closed-loop detection and linkage alarm, enhancing the intelligent decision-making ability and interpretability of the system.

[0038] Furthermore, S2 includes: S21, using a 5×7×7 separable convolution kernel to dynamically adapt to a 60fps sampling rate and output a 128-dimensional spatio-temporal feature map, where the length of the convolution kernel in the time dimension is automatically adjusted according to the current sampling frequency.

[0039] Specifically, this step involves using a 5×7×7 separable convolution kernel to perform spatio-temporal feature extraction for dynamically adapting to a 60fps sampling rate on a video frame sequence, and outputting a 128-dimensional spatio-temporal feature map, where the length of the convolution kernel in the time dimension is automatically adjusted according to the current sampling frequency. This step is the core feature extraction module in the entire dynamic scene recognition method, and its technical implementation is based on the primary processing stage of a three-dimensional spatio-temporal graph convolutional network (3D-STGCN), aiming to efficiently and accurately model the motion trajectories and interaction relationships of targets in the video.

[0040] Furthermore, this step adopts a separable convolution (DepthwiseSeparableConvolution) structure, decomposing the standard three-dimensional convolution into two independent operations of spatial convolution and temporal convolution, thereby significantly reducing the computational complexity. Specifically, the spatial convolution uses a 7×7 convolution kernel to extract the spatial features of the image, while the temporal convolution dynamically adjusts its length according to the current sampling frequency (such as 60fps). For example, at 60fps, the length of the temporal convolution kernel can be set to 5 frames to cover approximately 83 milliseconds of continuous motion information. This dynamic adjustment mechanism is realized through a time step adaptive module, which calculates the sampling window length in the time dimension according to the input frame rate, ensuring time continuity and motion representation ability at different sampling frequencies.

[0041] Furthermore, the feature map output by this convolutional kernel has 128 channels, with each channel encoding the target's motion trend, appearance changes, and local interaction information in the spatiotemporal dimension. The temporal convolutional kernel length is adjusted as follows: when the sampling frequency is 60fps, the temporal kernel length is set to 5 frames; when the sampling frequency drops to 30fps, the length is expanded to 10 frames to compensate for the decrease in temporal resolution. In addition, the convolutional operation supports INT8 quantization inference to adapt to the low-power requirements of edge computing units, and the inference latency is controlled to <20ms, meeting the requirements of real-time video processing.

[0042] Furthermore, this step is primarily used in the video frame sequence processing stage of multi-camera monitoring systems, and is particularly suitable for high-density dynamic scenes such as stations and squares. By dynamically adapting the convolutional kernel length, the system can maintain high-precision modeling of target motion at different sampling frequencies, providing high-quality spatiotemporal feature inputs for subsequent graph neural network construction and multi-task decision-making.

[0043] Furthermore, by employing spatiotemporal joint modeling, the problem of fragmented spatiotemporal features in the traditional CNN+LSTM architecture is effectively solved, significantly improving the modeling accuracy of the continuity of target trajectories and interaction relationships. Experiments show that this method can improve the recognition accuracy to 97.9% in group behavior recognition tasks, eliminating the trajectory breakage problem and laying a solid foundation for subsequent anomaly detection and linkage alarms.

[0044] S22, When constructing the spatiotemporal relationship graph, the node attributes include the target's three-dimensional position coordinates, velocity vector direction angle, and appearance feature vector. The edge weight calculation formula is α·cos(v_i,v_j)+β·exp(-||p_i-p_j||), where α and β are trainable parameters learned through backpropagation.

[0045] Specifically, in the process of constructing the spatiotemporal relationship graph, setting node attributes and calculating edge weights are the core steps for realizing target behavior modeling and group interaction recognition in dynamic scenes. Specifically, each node represents a detected moving target, and its attributes include three-dimensional position coordinates (x, y, z), velocity vector direction angle (θ), and appearance feature vectors (such as color histogram, texture features, target category embedding, etc.). The three-dimensional position coordinates are obtained through joint estimation of millimeter-wave radar and camera calibration parameters, with an accuracy of ±0.1 meters; the velocity vector direction angle is calculated based on the target displacement vector between consecutive frames, using Euler angles, with the angle error controlled within ±3°; the appearance feature vector is extracted by the target detection model (such as YOLOv7), and after normalization, serves as the static attribute input for the graph nodes.

[0046] The formula for calculating edge weight is: α·cos(v_i,v_j)+β·exp(-||p_i-p_j||), Here, v_i and v_j represent the velocity vectors of node i and node j, respectively, and cos(v_i,v_j) represents the cosine similarity of their velocity directions, used to measure the consistency of motion trends between targets; p_i and p_j are the three-dimensional position coordinates of nodes i and j, ||p_i-p_j|| represents their Euclidean distance, and exp(-||p_i-p_j||) is an exponential decay function used to simulate the influence of spatial distance between targets on the interaction strength. α and β are learnable parameters optimized during training using the backpropagation algorithm, initially set to 0.5, and finally converged to the optimal solution using gradient descent to balance the contributions of motion trends and spatial distance in relationship modeling.

[0047] This step plays a crucial role in the 3D Spatiotemporal Graph Convolutional Network (3D-STGCN). By constructing a graph structure with physical semantics, the model can simultaneously perceive the motion trajectory and interactive behavior of targets. For example, in a station plaza scene, if nodes #15 and #22 have similar velocity directions and are spatially close, their edge weights are higher, indicating that they exhibit following or clustering behavior. Furthermore, this graph structure serves as input to the graph attention mechanism, achieving dynamic modeling and prediction of target behavior through weighted aggregation of neighborhood features, thereby significantly improving the accuracy and robustness of group behavior recognition.

[0048] S3 updates the node features of the spatiotemporal relationship graph through a graph attention mechanism. The attention coefficient is jointly determined by the feature similarity between the current node and its neighboring nodes and their spatial relative positions. The edge weights are calculated by a linear combination of the cosine similarity of the relative motion trend between targets and the exponential decay function of spatial distance. The linear coefficients are learnable parameters.

[0049] Specifically, this step updates the node features of the spatiotemporal relationship graph through a graph attention mechanism, which is the core link in this invention to realize target interaction modeling and behavior reasoning.

[0050] Furthermore, this mechanism is based on a Graph Neural Network (GNN) framework, modeling the detected targets in video surveillance scenes as nodes in a graph structure. Each node's feature vector contains multimodal information such as the target's position coordinates, velocity vector, and appearance features (e.g., color histogram, HOG features, or embedding vectors extracted by CNN). Edges in the graph represent the interaction relationships between targets, and their weights are calculated as a linear combination of the cosine similarity of the relative motion trends between targets and an exponentially decaying function of the spatial distance. The linear coefficients α and β are learnable parameters that are automatically optimized during training using a backpropagation algorithm.

[0051] Furthermore, the dimension of the weight matrix of the attention mechanism is... , where d is the dimension of the node feature vector (e.g., 128-dimensional), and the initial values ​​of α and β are set to 0.5, dynamically adjusted during training using the Adam optimizer. The aggregated node features are updated using a non-linear activation function (e.g., ReLU or GELU), outputting a new state vector for subsequent multi-task decision-making and event modeling.

[0052] Furthermore, by introducing a joint spatial and motion attention mechanism, the graph neural network's ability to model target interaction relationships is significantly enhanced, solving problems such as inaccurate target relationship modeling and delayed behavior recognition in traditional methods, and providing high-quality semantic graph input for subsequent multi-task decision-making and linkage alarm.

[0053] Furthermore, S3 includes: S31 updates node features by weighted summation of neighborhood node features. When calculating the attention coefficient, a spatial constraint term is introduced, and an exponential decay penalty factor is applied to nodes that are more than 5 meters away.

[0054] Specifically, this step updates node features using a weighted summation of neighborhood node features and introduces a spatial constraint term in the attention coefficient calculation, applying an exponential decay penalty factor to nodes more than 5 meters apart. This is one of the core graph attention mechanisms in the 3D-STGCN of this invention. Its technical implementation principle is based on the graph attention mechanism in graph neural networks (GNNs). By introducing an exponential decay function for spatial distance, it enhances the model's ability to model the physical relationships between targets, thereby improving the accuracy and robustness of group behavior recognition in dynamic scenes.

[0055] Furthermore, each detected target is modeled as a node in a graph. Node attributes include the target's position coordinates, velocity vector (magnitude and direction), and appearance features (such as color histogram, HOG features, etc.). The edge weights between nodes consist of two parts: first, the cosine similarity of the relative motion trends between targets, used to measure the consistency of motion directions; and second, the exponentially decaying function of the Euclidean distance between targets. This spatial constraint term is implemented through learnable parameters. and The attention coefficient is formed by linearly combining the attention coefficient with the feature similarity.

[0056] Furthermore, this step significantly enhances the graph neural network's ability to perceive the physical relationships between targets, enabling the system to more accurately identify target behavior patterns, such as following, gathering, and avoiding, in complex dynamic scenarios, thereby providing high-quality semantic graph input for subsequent event loop determination and linkage alarm.

[0057] S32. Perform clustering analysis on the 32-dimensional relationship feature vector through a multi-layer perceptron, and classify the interaction relationship types into three categories: following, aggregation, and avoidance. Each type of relationship corresponds to an independent attention weight matrix.

[0058] Specifically, in this step, clustering analysis is performed on the 32-dimensional relationship feature vector through a multi-layer perceptron (MLP), the interaction relationship types are classified into three categories: following, aggregation, and avoidance, and an independent attention weight matrix is assigned to each type of relationship. This is the core link in realizing the behavior modeling between targets and event correlation recognition in this invention.

[0059] Furthermore, this step first receives the 32-dimensional relationship feature vector output by the three-dimensional spatio-temporal graph convolutional network (3D-STGCN), which encodes key interaction attributes such as the relative motion trend, spatial distance, and velocity direction between targets. Subsequently, these feature vectors are input into an MLP model composed of two layers of fully connected neural networks, which adopts the ReLU activation function and the L2 regularization strategy to avoid overfitting. The output layer of the MLP is the clustering probability distribution of the three types of interaction relationships, which is normalized through the Softmax function, and finally each relationship feature vector is mapped to a following, aggregation, or avoidance category label.

[0060] Furthermore, an independent attention weight matrix is configured for each type of relationship, and this matrix is optimized through the backpropagation algorithm during the training process. Specifically, when the graph attention mechanism (GraphAttentionNetwork, GAT) aggregates the features of neighborhood nodes, it selects the corresponding weight matrix according to the relationship type for weighted summation, so as to realize the differential modeling of different interaction modes. For example, for the following relationship, the attention weight matrix pays more attention to the relative speed and direction consistency between targets; for the aggregation relationship, it emphasizes spatial proximity and residence time; while for the avoidance relationship, it focuses on the reverse of the motion vector and the rate of distance change.

[0061] Furthermore, the input layer dimension of the MLP is 32, the hidden layer is 64, the output layer is 3, the learning rate is set to 0.001, and the Adam optimizer is used for training. The clustering process is initialized using the K-means algorithm and is iteratively optimized in combination with the soft classification results output by the MLP, finally achieving an accurate division of the relationship types. This step plays a key role in behavior semantic parsing and event chain construction in the system, provides structured input for subsequent spatio-temporal consistency constraints and linkage alarm mechanisms, and significantly improves the system's semantic understanding ability and decision-making efficiency for complex dynamic scenarios.

[0062] S4. Construct a scene event knowledge graph based on the node states output by the graph neural network. When it is detected that a closed-loop association path is formed by preset abnormal event nodes, trigger a linkage alarm and output a dynamic scene semantic graph with time stamps.

[0063] Specifically, this step constructs a scene event knowledge graph based on the node states output by a graph neural network (GNN). When a preset abnormal event node forms a closed-loop association path, a linkage alarm is triggered, and a dynamic scene semantic graph with spatiotemporal stamps is output. This step is the core link in the entire intelligent video surveillance system for realizing event semantic reasoning and security decision-making.

[0064] Furthermore, the node state vectors output by the Graph Neural Network (3D-STGCN) include the target's ID number, position coordinates, velocity vector, appearance features, and behavioral state encoding. The system constructs these node states into a dynamic spatiotemporal relationship graph, where nodes represent entity targets in the monitored scene (such as pedestrians, abandoned objects, etc.), and edges represent the interaction relationships between targets (such as following, gathering, avoiding, etc.). Edge weights are calculated as a linear combination of the cosine similarity of the relative motion trends between targets and the exponentially decaying function of spatial distance.

[0065] Furthermore, pre-defined criteria for identifying abnormal events are established. For example, an abnormal clustering event is defined as the presence of more than 5 targets within a 3-meter radius for more than 10 seconds; a rapid movement event is defined as a speed exceeding 5 meters per second for 3 seconds; and a leftover object event is defined as a static target lasting for more than 30 seconds. When these event nodes form a closed-loop path in the knowledge graph, the system identifies it as a potential security threat and triggers an alarm mechanism.

[0066] The technical effect of this step is to significantly improve the system's accuracy in identifying abnormal events in complex scenarios and its response efficiency. Through closed-loop path detection, the system can automatically reason about event chains, shorten the response time of linked alarms to 1.8 seconds, and improve decision-making efficiency by more than 100%. It effectively solves the problems of delayed event identification and poor interpretability, and has high practical value and innovation.

[0067] Furthermore, S4 includes: S41 defines an abnormal clustering event as more than 5 targets within a radius of 3 meters for 10 seconds, a fast movement event as a speed exceeding 5 meters / second for 3 seconds, and a leftover object event as a static target lasting more than 30 seconds.

[0068] Specifically, this step defines three key dynamic scene events: abnormal aggregation events, rapid movement events, and abandoned object events. Its technical implementation is based on the spatiotemporal feature modeling of target behavior in video surveillance and event judgment logic, which is the core link of event recognition and linkage alarm in the entire intelligent video surveillance system.

[0069] Furthermore, this step constructs a scene event knowledge graph, mapping the spatiotemporal attributes of targets, such as their motion trajectory, speed, and dwell time, to nodes and edges in a graph structure, thus achieving structured event recognition. Specifically, the determination of abnormal clustering events is based on the spatial distribution density of targets. When more than 5 targets are continuously detected in a certain area (within a 3-meter radius centered on the target) (using ID and coordinate information output by the target detection and tracking module), and this state lasts for more than 10 seconds, the system determines it as abnormal clustering. Fast movement events are calculated using the target velocity vector. When the instantaneous speed of a target exceeds 5 meters per second, and this high-speed state is maintained continuously for more than 3 seconds, a fast movement event is triggered. The determination of abandoned object events depends on the target's motion state. When a target does not undergo significant displacement (speed below 0.1 meters per second) for 30 consecutive seconds, and is not marked as removed or occluded by other targets, the system determines it as an abandoned object.

[0070] Furthermore, the 3-meter radius setting is based on the minimum danger threshold of crowd gathering in typical monitoring scenarios, the density threshold of 5 targets refers to the definition of abnormal gathering in public place safety standards, the speed threshold of 5 meters per second corresponds to the typical speed range of human running or fast-moving objects, the 3-second duration is used to eliminate instantaneous motion interference, and the 30-second static duration is set according to the time window in which objects left behind in a security incident may cause risks, ensuring that the system responds to potential threats within a reasonable time.

[0071] Furthermore, through structured event definition and closed-loop path detection mechanisms, the system can automatically identify and correlate multiple abnormal behaviors, forming an interpretable event chain, significantly improving the accuracy and response speed of alarms. For example, in this embodiment, the system triggers a linked alarm by detecting the correlation between a gathering of 8 people and a suitcase left behind, enabling security personnel to obtain complete event information within 1.8 seconds and achieve efficient handling.

[0072] S42, when a closed-loop path A→B→C is detected, the audible and visual alarm is activated and a safety event report containing a heat map of the target trajectory is generated, where A is an abnormal cluster node, B is a left-behind object node, and C is a fast-moving node.

[0073] Specifically, when the system detects a closed-loop path A→B→C, it identifies three types of events—abnormal aggregation (A), abandoned objects (B), and rapid movement (C)—forming a logical closed loop in the spatiotemporal semantic graph. The system will then automatically activate the audible and visual alarms and generate a security event report containing a heatmap of the target trajectory. This step is the core component of the entire intelligent video surveillance system for achieving security early warning and event response, and its technical implementation is based on the dynamic construction of an event knowledge graph and a closed-loop path identification mechanism.

[0074] Furthermore, the system constructs a scene event knowledge graph through a spatiotemporal consistency constraint module, where event nodes A, B, and C correspond to preset abnormal behavior patterns. Specifically, an abnormal clustering event is defined as the presence of more than 5 targets within a 3-meter radius for more than 10 seconds; a leftover object event is defined as a static target persisting for more than 30 seconds; and a rapid movement event is defined as a speed exceeding 5 meters per second lasting for more than 3 seconds. The system calculates the edge weights between event nodes using the 32-dimensional relational feature vectors output by the graph neural network, and quantifies the motion trends and spatial relationships between targets using a linear combination method, where α and β are learnable parameters optimized through backpropagation.

[0075] Furthermore, when a closed-loop path A→B→C is detected, the system will trigger a linked alarm mechanism. The activation of the audible and visual alarm is based on a preset response threshold; for example, if the sum of edge weights exceeds 0.75, the system determines it as a high-risk closed-loop event. Simultaneously, the system generates a security event report, which includes a target trajectory heatmap. The heatmap maps timestamps (accurate to 100 milliseconds) to spatial coordinates and uses color gradients (e.g., red indicates a high-density trajectory) to display the target's movement path before and after the event, facilitating rapid location and retracing of the event process by security personnel.

[0076] Furthermore, through closed-loop path identification, the system can effectively distinguish between normal behavior and potential threat behavior, improving the accuracy and timeliness of incident response. Its technological value lies in achieving end-to-end automated processing from raw video data to actionable security decisions, significantly reducing the need for manual intervention and improving the overall intelligence level and response efficiency of the security system.

[0077] Furthermore, the system first constructs a semantic graph of the dynamic scene using a 3D spatiotemporal graph convolutional network (3D-STGCN). Each node represents a detected physical target (such as pedestrians, abandoned objects, etc.), and the node attributes include target ID, position coordinates, velocity vector, appearance features, and behavioral state encoding. When the system identifies a target as an abandoned object, the node diameter is linearly scaled from the default 30px to 45px according to a preset threat level mapping rule. The scaling process uses a temporal interpolation algorithm to ensure a smooth transition within 100 milliseconds, thereby enhancing the continuity and stability of visual perception. The edge width is dynamically adjusted according to the interaction intensity between targets. The interaction intensity is calculated from the relational feature vector output by the graph convolution, and specifically mapped to a line width range of 1-5px after normalization. The line width is positively correlated with the interaction intensity. For example, when there is a following behavior between two targets and the relative speed is less than 1m / s, the edge width can be set to 3px, while when there is a rapid approach or gathering behavior, the edge width can be increased to 5px.

[0078] Furthermore, the initial value for node diameter scaling is 30px, with a maximum value of 45px. The scaling factor is determined by a linear mapping of threat level (0-5). For example, at threat level 3, the diameter is 39px. Edge width adjustment is mapped to 1-5px based on the interaction intensity normalized value (0-1). The specific mapping function is: line width = 1 + 4 × normalized intensity value. In addition, the system supports timeline slider operation with a time resolution of 100ms, facilitating security personnel to review the development of events.

[0079] In practical applications, this step is primarily used in the visualization decision support module of intelligent video surveillance systems, especially in high-density pedestrian areas such as train stations, airports, and shopping malls. It uses dynamic graphs to intuitively display abnormal behavior and potential threats, improving security response efficiency. Its technical value lies in enhancing the interpretability and operability of the system output, enabling security personnel to quickly identify and handle complex security incidents such as abandoned objects and crowds without switching between multiple screens. This reduces the alarm response time to less than 1.8 seconds, significantly improving the real-time performance and intelligence level of the security system.

[0080] The present invention discloses a dynamic scene recognition method for intelligent video surveillance equipment, which can realize high-precision spatiotemporal joint modeling and multi-task adaptive processing of targets in dynamic scenes, significantly improve the accuracy and real-time performance of event recognition in complex environments, reduce computing resource consumption, and enhance the interpretability of system decisions.

[0081] Example 2 This invention proposes a dynamic scene recognition method and apparatus for intelligent video surveillance equipment. The dynamic scene recognition method and apparatus for intelligent video surveillance equipment according to embodiments of this invention can achieve continuous modeling of target trajectories and accurate identification of multi-target interaction relationships in dynamic scenes, significantly improving the accuracy of group behavior recognition and effectively solving the trajectory breakage problem caused by the fragmentation of spatiotemporal features in traditional methods.

[0082] In one embodiment of the present invention, a dynamic scene recognition method for an intelligent video surveillance device includes the following steps: Multi-angle dynamic scene video streams are acquired using a steerable pan-tilt camera. An optical flow-driven adaptive frame extraction algorithm generates a spatiotemporal keyframe sequence. This algorithm calculates the average motion vector of the scene based on the dense optical flow field between consecutive video frames. When the motion vector exceeds a threshold θ, a high-frequency sampling mode of 60fps or higher is activated; otherwise, a keyframe abrupt change detection mode is enabled. Abrupt change frames are identified using a joint criterion of inter-frame structural similarity and optical flow acceleration. Frames with a structural similarity below 0.7 or an optical flow acceleration L2 norm exceeding a preset threshold Γ are retained. The generated frame sequence is input into a 3D spatiotemporal graph convolutional network. This network contains 5×7×7 separable convolutional kernels with dynamically adapted sampling frequencies in the time dimension for initial spatiotemporal feature extraction and constructs a network with the detected target as a node and node attribute packages... A spatiotemporal relationship graph containing position and velocity vectors and appearance features is generated. The edge weights are calculated by a linear combination of the cosine similarity of the relative motion trend between targets and the exponential decay function of spatial distance. The linear coefficients are learnable parameters α and β. Then, the node features are updated by aggregating multi-hop neighbor information through a graph attention mechanism. Based on the real-time illuminance value, weather code, and target density parameters obtained by environmental sensors, the weight allocation of three tasks—target detection, abnormal behavior prediction, and cross-camera target tracking—is dynamically adjusted. A multilayer perceptron mapping function from environmental parameters to task weights is established. Dynamic weights are injected into the multi-task loss function. When the illuminance is below 50 lux, the low-illuminance enhancement subnetwork of the detection task branch is activated. When the target density is greater than 15 people / frame, the group behavior modeling module of the tracking task is enabled. Finally, the spatiotemporal consistency constraint module integrates the outputs of multiple tasks to establish a cross-camera field-of-view topology map and uses polar geometric constraints to verify the target re-identification results. When the constructed scene event knowledge graph detects abnormal clustering, rapid movement, and left-behind object events forming a closed-loop path, a linkage alarm is triggered, and a dynamic scene semantic graph with spatiotemporal stamps is output.

[0083] Furthermore, the graph attention mechanism of the three-dimensional spatiotemporal graph convolutional network is specifically implemented as follows: the node feature update process adopts the weighted summation method of the features of neighboring nodes, the attention coefficient is jointly determined by the feature similarity and spatial relative position of the current node and its neighboring nodes, the weight matrix is ​​optimized through backpropagation learning, and the aggregated features are output by a nonlinear activation function to output the updated node state vector, which encodes the following, clustering and avoidance interaction relationships between targets.

[0084] Furthermore, the implementation of the dynamic weight allocation includes: the environmental parameter input vector is processed by a two-layer fully connected neural network to generate initial weights for the three tasks, which are then normalized to weight coefficients using a softmax function. The target detection task uses an improved CIoU bounding box regression loss function, the abnormal behavior prediction task uses a temporal focus loss function to handle the class imbalance problem, and the cross-camera target tracking task uses an association embedding loss function. Finally, the total loss of the multi-task task is the weighted sum of the losses of the three tasks, and the weight coefficients are updated every 5 seconds based on environmental sensor data.

[0085] Furthermore, the operation of the spatiotemporal consistency constraint module includes: constructing a topological map of overlapping fields of view using multi-camera calibration parameters; when a target crosses the field of view of a camera, obtaining the epipolar constraint equation through essential matrix decomposition to verify the spatial consistency of the target re-identification result; constructing a scene event knowledge graph based on the node states output by the graph neural network, defining an abnormal clustering event as more than 5 targets within a radius of 3 meters for 10 seconds, a fast movement event as a speed exceeding 5 meters / second for 3 seconds, and a left-behind object event as a static target lasting for more than 30 seconds; when the above event nodes form a closed-loop associated path in the knowledge graph, activating the audible and visual alarm and generating a security event report.

[0086] Furthermore, the generation rules for the dynamic scene semantic graph are as follows: graph nodes include target ID numbers, behavior state codes, and associated event labels, and the node size dynamically scales with the target threat level; edge connection lines represent the interaction relationship between targets, the line width is proportional to the interaction intensity, and the relationship type is determined by clustering the 32-dimensional relationship feature vector output by the graph convolutional network; the graph overlay time axis slider realizes dynamic scene backtracking, with a time resolution of 100 milliseconds.

[0087] In one embodiment of the present invention, an intelligent video surveillance device implementing the above method is characterized by comprising: a pan-tilt-zoom camera group configured with a 120-degree wide-angle lens and a 40x optical zoom telephoto lens; an embedded processing unit integrating an optical flow computing chip and a 3D graph convolution accelerator; a scene perception sensor array including a digital illuminometer with a range of 0-100,000 lux, a multispectral weather sensor supporting rain, snow, and fog recognition, and a millimeter-wave radar with a detection range of 200 meters; and a dynamic visualization terminal equipped with a GPU array for real-time rendering of spatiotemporal relationship graphs and semantic graphs.

[0088] Furthermore, the embedded processing unit adopts a three-level hierarchical computing architecture: the front-end FPGA chip performs optical flow field calculation and adaptive frame extraction, with processing latency controlled within 20 milliseconds; the edge computing unit is equipped with a dedicated 3D-STGCN inference engine, supporting INT8 quantization inference; the cloud training platform generates virtual scene data through a digital twin system, updates the scene adaptation parameter library weekly, and distributes it to edge devices through a secure channel.

[0089] Furthermore, the working mode of the multispectral weather sensor is as follows: it detects the refraction characteristics of raindrops in the visible light band, identifies the scattering pattern of fog particles in the near-infrared band, senses the thermal radiation characteristics of snowflake crystals in the mid-infrared band, and outputs weather type codes by fusing the three band data through a pre-trained ResNet-18 network.

[0090] In one embodiment of the present invention, system initialization and data acquisition are performed as follows: A pan-tilt-zoom camera module (including a 120° wide-angle lens and a 40x telephoto lens) initiates preset position scanning to capture multi-angle video (25fps, 1080P) of the station square. Simultaneously, a digital illuminometer (real-time value 85 lux), a multispectral weather sensor, and a millimeter-wave radar (detecting 143 targets), all integrated within the equipment compartment, continuously upload environmental parameters. The front-end FPGA chip activates an optical flow accelerator to calculate the dense optical flow field between consecutive video frames, measuring an average scene motion vector of 4.2 pixels / frame (exceeding the threshold θ=3.5).

[0091] Furthermore, adaptive video frame processing: The system automatically switches to high-frequency sampling mode (60fps) and performs abrupt frame detection on the current 10-second video segment (600 frames): it calculates the inter-frame structural similarity (SSIM), and when the SSIM value in frames 203-204 drops sharply to 0.63 (<0.7 threshold), it simultaneously detects that the optical flow acceleration L2 norm in frame 315 reaches 12.7. (Exceeding Γ=10), the system extracts the above key frames and adjacent temporal frames to generate a spatiotemporal key frame sequence containing 87 frames, and the data volume is compressed to 14.5% of the original video.

[0092] Furthermore, 3D spatiotemporal graph convolutional feature extraction: Spatiotemporal keyframe sequences are input into the 3D-STGCN network: Primary feature extraction: Spatiotemporal features are extracted using 5×7×7 separable convolutional kernels (temporal kernel dynamically adapted to 60fps sampling rate), outputting a 128-dimensional feature map; Relationship graph construction: 42 pedestrian targets are detected, and a graph structure is constructed using these targets as nodes. For example, the attributes of node #15 are: position (125, 240), velocity (0.8m / s, 30°), and appearance features [0.76, 0.12, ...]; Edge weight calculation: The cosine similarity of the relative motion trend between nodes #15 and #22 is 0.91, and the Euclidean distance is 2.3m (α, β are optimized through backpropagation); Graph attention aggregation: Features are aggregated from the 8 neighbors of node #15, and the one with the highest attention coefficient (#22, a=0.83) represents the following behavior. Furthermore, environmental adaptive multi-task decision-making: The environmental parameter vector [85, 02, 143] is input into a multilayer perceptron (structure: 128×64×3): the original weights are output as follows: [target detection 0.41, abnormal behavior 0.33, cross-camera tracking 0.26]; after Softmax normalization, the dynamic weight coefficients are obtained as (0.38, 0.31, 0.31). Furthermore, the multi-task engine performs the following: Target detection: Activating the CIoU loss function, it identifies abandoned suitcases in the northwest corner (confidence 0.94); Abnormal behavior: Based on temporal focus loss, it detects the cluster of #15-#22 (lasting 12 seconds beyond the threshold); Cross-camera tracking: Millimeter-wave radar assists in associating the trajectory of target #107 within the field of view of cameras #3 and #7. Furthermore, a spatiotemporal consistency fusion alarm is triggered, and the spatiotemporal constraint module executes: Cross-camera verification: The epipolar constraints of cameras 3 and 7 were calculated by essential matrix factorization to confirm that the re-identification error of target #107 during field of view switching is less than 3 pixels. Event graph construction: Node A (abnormal clustering): 8 people, including #15-#22, continuously gathered in an area with a radius of 2.8 meters; Node B (leftover object): The suitcase remained stationary for 32 seconds; the weight of edge AB is 0.79 (calculated through a 32-dimensional relational feature vector); the A→B associated path was detected to form a closed loop, triggering the linkage alarm rule.

[0093] Furthermore, the dynamic semantic graph output generates a timestamped semantic graph: Node visualization: Crowds are displayed as red cluster nodes (45px in diameter), and suitcases are displayed as flashing yellow nodes (30px in diameter); Relationship connection: The edge line from the gathering area to the suitcase is 4px wide (intensity level 3); Backtrack function: Operate the timeline slider to rewind the scene 10 seconds ago, showing the trajectory of target #107 from the gathering area to the suitcase; Furthermore, the equipment is linked for control, and the system automatically executes pre-planned actions: Control the telephoto lens of camera No. 7 to zoom in and focus on the abandoned suitcase, activate the nearest broadcasting device via PoE network to play an alarm voice, and push semantic graphs and event reports to the security terminal.

[0094] Example 3 To achieve the above embodiments, such as Figure 2 As shown, this embodiment also provides a dynamic scene recognition method device 10 for intelligent video surveillance equipment, including: The dynamic sampling control module 100 is used to acquire dynamic scene video streams through multi-angle cameras, calculate the average motion vector of the scene based on the dense optical flow field, and dynamically switch between high-frequency sampling mode and sudden frame detection mode according to the comparison result of motion vector and preset threshold, and generate a spatiotemporal key frame sequence containing action sudden frames. The three-dimensional spatiotemporal feature processing module 200 is used to input the spatiotemporal keyframe sequence into the three-dimensional spatiotemporal graph convolutional network, extract spatiotemporal features using a three-dimensional separable convolutional kernel with dynamically adapted sampling rate, and construct a spatiotemporal relationship graph with the detected target as the node and the node attributes including position velocity vectors and appearance features. The graph attention interaction analysis module 300 is used to update the node features of the spatiotemporal relationship graph through the graph attention mechanism. The attention coefficient is jointly determined by the feature similarity and spatial relative position of the current node and its neighboring nodes. The edge weight is calculated by a linear combination of the cosine similarity of the relative motion trend between targets and the exponential decay function of spatial distance. The linear coefficient is a learnable parameter. The event knowledge graph construction and alarm module 400 is used to construct a scene event knowledge graph based on the node status output by the graph neural network. When a preset abnormal event node is detected to form a closed-loop associated path, a linkage alarm is triggered and a dynamic scene semantic graph with spatiotemporal stamps is output.

[0095] Furthermore, the dynamic sampling control module 100 is also used for: When the average motion vector of the scene exceeds the threshold, the high-frequency sampling mode is activated, and the action change frame is identified by the joint criterion that the inter-frame structural similarity is less than 0.7 or the optical flow acceleration L2 norm exceeds the preset threshold Γ. The retained mutation frames are spatiotemporally aligned with adjacent time-series frames to generate a spatiotemporal keyframe sequence containing timestamps and motion vectors, compressing the data volume to 14.5% of the original video.

[0096] Furthermore, the three-dimensional spatiotemporal feature processing module 200 is also used for: Using a 5×7×7 separable convolutional kernel to dynamically adapt to a 60fps sampling rate, a 128-dimensional spatiotemporal feature map is output, where the length of the temporal dimension convolutional kernel is automatically adjusted according to the current sampling frequency. When constructing the spatiotemporal relationship graph, the node attributes include the target's three-dimensional position coordinates, velocity vector direction angle, and appearance feature vector. The edge weight calculation formula is α·cos(v_i,v_j)+β·exp(-||p_i-p_j||), where α and β are trainable parameters learned through backpropagation.

[0097] Furthermore, the graph attention interaction analysis module 300 is also used for: The node features are updated by weighted summation of neighborhood node features. A spatial constraint term is introduced when calculating the attention coefficient, and an exponential decay penalty factor is applied to nodes that are more than 5 meters away. Cluster analysis of the 32-dimensional relationship feature vectors was performed using a multilayer perceptron, classifying the interaction relationship types into three categories: follow, cluster, and avoid. Each category of relationship corresponds to an independent attention weight matrix.

[0098] Furthermore, the event knowledge graph construction and alarm module 400 is also used for: An abnormal clustering event is defined as more than 5 targets within a radius of 3 meters for 10 seconds; a fast movement event is defined as a speed exceeding 5 meters / second for 3 seconds; and a left-behind object event is defined as a static target whose duration exceeds 30 seconds. When a closed-loop path A→B→C is detected, the audible and visual alarm is activated and a safety incident report containing a heatmap of the target trajectory is generated, where A is the abnormal cluster node, B is the leftover object node, and C is the fast-moving node.

[0099] The present invention discloses a dynamic scene recognition device for intelligent video surveillance equipment, which can realize high-precision spatiotemporal joint modeling and multi-task adaptive processing of targets in dynamic scenes, significantly improve the accuracy and real-time performance of event recognition in complex environments, reduce computing resource consumption, and enhance the interpretability of system decisions.

[0100] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0101] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for dynamic scene recognition of a smart video surveillance device, characterized in that, The method comprises the following steps: S1, acquiring a dynamic scene video stream by a multi-angle camera, calculating a scene average motion vector based on a dense optical flow field, dynamically switching a high-frequency sampling mode or a sudden change frame detection mode according to a comparison result of the motion vector and a preset threshold, and generating a time-space key frame sequence containing action sudden change frames; S2, inputting the time-space key frame sequence into a three-dimensional time-space graph convolution network, extracting time-space features by using a three-dimensional separable convolution kernel with a dynamically adapted sampling rate, and constructing a time-space relation graph with detected targets as nodes and node attributes containing position and velocity vectors and appearance features; S3, updating the node features of the time-space relation graph by a graph attention mechanism, wherein the attention coefficients are determined by the feature similarity and the spatial relative position of the current node and the neighbor nodes, and the edge weights are calculated by a linear combination of the relative motion trend cosine similarity and the spatial distance exponential decay function, and the linear coefficient is a learnable parameter; S4, constructing a scene event knowledge graph based on the node states output by the graph neural network, triggering a linkage alarm and outputting a dynamic scene semantic graph with time-space stamps when a preset abnormal event node forms a closed loop association path.

2. The method of claim 1, wherein, The S1 comprises: S11, when the scene average motion vector exceeds the threshold, starting the high-frequency sampling mode, and identifying the action sudden change frames by a joint criterion that the inter-frame structural similarity is less than 0.7 or the optical flow acceleration two-norm exceeds a preset threshold Γ; S12, performing time-space alignment processing on the reserved sudden change frames and the adjacent time sequence frames to generate a time-space key frame sequence containing time stamps and motion vectors, and compressing the data amount to 14.5% of the original video.

3. The method of claim 1, wherein, The S2 comprises: S21, using a 5*7*7 separable convolution kernel to dynamically adapt the 60fps sampling rate, and outputting a 128-dimensional time-space feature map, wherein the time dimension convolution kernel length is automatically adjusted according to the current sampling frequency; S22, when constructing the time-space relation graph, the node attributes contain the three-dimensional position coordinates, the velocity vector direction angle and the appearance feature vector of the target, and the edge weight calculation formula is α·cos(v_i,v_j)+β·exp(-||p_i-p_j||), wherein α and β are trainable parameters learned by back propagation.

4. The method of claim 1, wherein, The S3 comprises: S31, updating the node features by using a neighborhood node feature weighted summation method, and introducing a spatial constraint term when calculating the attention coefficients to apply an exponential decay penalty factor to the nodes with a distance exceeding 5 meters; S32, performing clustering analysis on the 32-dimensional relation feature vector by a multilayer perceptron, and dividing the interaction relation types into three categories: following, gathering and avoiding, and each category of relation corresponds to an independent attention weight matrix.

5. The method of claim 1, wherein, The S4 comprises: S41, defining an abnormal gathering event as more than 5 targets within a radius of 3 meters for 10 seconds, a fast moving event as a speed exceeding 5 meters / second for 3 seconds, and a left object event as a static target lasting more than 30 seconds; S42, when detecting an A→B→C closed loop path, activating an audible and visual alarm and generating a safety event report containing a target trajectory heat map, wherein A is an abnormal gathering node, B is a left object node, and C is a fast moving node.

6. A method of dynamic scene identification of a smart video surveillance device, characterized in that, The method comprises the following steps: The dynamic sampling control module is configured to collect a dynamic scene video stream through a multi-angle camera, calculate a scene average motion vector based on a dense optical flow field, dynamically switch a high-frequency sampling mode or a mutation frame detection mode according to a comparison result of the motion vector and a preset threshold, and generate a time-space key frame sequence containing action mutation frames. The three-dimensional time-space feature processing module is configured to input the time-space key frame sequence into a three-dimensional time-space graph convolution network, extract time-space features by using a three-dimensional separable convolution kernel with a dynamically adapted sampling rate, and construct a time-space relation graph with nodes representing detected targets and node attributes including position and velocity vectors and appearance features. The graph attention interaction analysis module is configured to update node features of the time-space relation graph by using a graph attention mechanism, wherein attention coefficients are determined based on feature similarities and spatial relative positions of current nodes and neighbor nodes, and edge weights are calculated based on a linear combination of a relative motion trend cosine similarity between targets and an exponential decay function of spatial distances, with a linear coefficient being a learnable parameter. The event knowledge graph construction and alarm module is configured to construct a scene event knowledge graph based on node states output by the graph neural network, and trigger a linkage alarm and output a dynamic scene semantic graph with time-space stamps when a preset abnormal event node forms a closed-loop correlation path.

7. The apparatus of claim 6, wherein, The dynamic sampling control module is further configured to: start the high-frequency sampling mode when the scene average motion vector exceeds the threshold, and identify action mutation frames based on a joint criterion that an inter-frame structural similarity is less than 0.7 or an optical flow acceleration two-norm exceeds a preset threshold Γ; perform time-space alignment processing on the retained mutation frames and adjacent time sequence frames to generate a time-space key frame sequence containing time stamps and motion vectors, and compress data to 14.5% of the original video.

8. The apparatus of claim 6, wherein, The three-dimensional time-space feature processing module is further configured to: use a 5×7×7 separable convolution kernel to dynamically adapt to a 60fps sampling rate, and output a 128-dimensional time-space feature map, wherein a time dimension convolution kernel length is automatically adjusted according to a current sampling frequency; when constructing the time-space relation graph, node attributes include three-dimensional position coordinates, velocity vector direction angles and appearance feature vectors of targets, and an edge weight calculation formula is α·cos(v_i,v_j)+β·exp(-||p_i-p_j||), wherein α and β are trainable parameters learned through back propagation.

9. The apparatus of claim 6, wherein, The graph attention interaction analysis module is further configured to: update node features by using a neighborhood node feature weighted summation method, and introduce a spatial constraint term when calculating attention coefficients to apply an exponential decay penalty factor to nodes with a distance exceeding 5 meters; perform clustering analysis on 32-dimensional relation feature vectors by using a multi-layer perceptron, and divide interaction relation types into three categories, i.e., following, gathering and avoiding, each category corresponding to an independent attention weight matrix.

10. The apparatus of claim 6, wherein, The event knowledge graph construction and alarm module is further configured to: define an abnormal gathering event as more than 5 targets within a radius of 3 meters lasting for 10 seconds, a fast moving event as a speed exceeding 5 meters / second and lasting for 3 seconds, and a left object event as a static target lasting for more than 30 seconds. When the A→B→C closed loop path is detected, activate the sound-light alarm and generate a safety event report containing the target trajectory heat map, where A is the abnormal aggregation node, B is the left object node, and C is the fast-moving node.

Images