A group abnormal behavior mining system and method based on a fusion graph neural network

Abstract

This invention relates to the field of group behavior mining technology, and discloses a group anomaly behavior mining system and method fused with graph neural networks. The method involves: extracting density-enhanced node features from monitoring video frames of multiple camera nodes; calculating global and local features based on the density-enhanced node features and performing bidirectional temporal encoding to obtain bidirectional encoded features; calculating a group momentum similarity matrix based on the bidirectional encoded features, and superimposing the group momentum similarity matrix onto a self-attention score for encoding to obtain group co-evolutionary features; performing spatiotemporal encoding and decoding on the density-enhanced node features and calculating the topology-preserving reconstruction error; and generating group anomaly behavior detection results based on the group co-evolutionary features, the topology-preserving reconstruction error, and the prototype distance anomaly probability. This invention improves the robustness of anomaly detection and its generalization ability to novel anomaly patterns, solving the problem of insufficient detection accuracy of single models in complex scenes and under conditions of few samples.

Description

Technical Field

[0001] This invention relates to the field of group behavior mining technology, and in particular to a group abnormal behavior mining system and method that integrates graph neural networks. Background Technology

[0002] Currently, techniques for detecting abnormal group behavior employ spatiotemporal graph neural networks to model the dependencies between multiple camera nodes. However, existing technologies suffer from the following problems: First, the impact of changes in group density is not considered during node feature extraction, resulting in insufficient feature representation capabilities in high-density crowded scenes and low-density sparse scenes. Second, graph neural networks use static aggregation strategies, which cannot dynamically adjust the attention weight allocation between nodes according to the temporal context, making it difficult to accurately capture the spatiotemporal dependency evolution patterns across nodes. Third, the temporal encoding process ignores the impact of topological structural mutations on historical memory, resulting in a sluggish response to structural anomalies. Summary of the Invention

[0003] The main objective of this invention is to provide a system and method for mining abnormal behavior in groups by fusing graph neural networks. This invention improves the robustness of anomaly detection and the ability to generalize to novel anomaly patterns, and solves the problem of insufficient detection accuracy of a single model in complex scenarios and under conditions with few samples.

[0004] To achieve the above objectives, this invention provides a method for mining abnormal group behavior by fusing graph neural networks, comprising the following steps: Extract density-enhanced node features from surveillance video frames of multiple camera nodes; Global and local features are calculated based on the density-enhanced node features, and bidirectional temporal coding is performed on the global and local features to obtain bidirectional coded features; The population momentum similarity matrix is ​​calculated based on the bidirectional coding features, and the population momentum similarity matrix is ​​superimposed on the self-attention score for encoding to obtain the population co-evolution features; Spatiotemporal encoding and decoding are performed on the density-enhanced node features to calculate the topology-preserving reconstruction error; Based on the group's cooperative evolution characteristics, the topology preservation and reconstruction error, and the prototype distance anomaly probability, the group's abnormal behavior detection results are generated.

[0005] Optionally, in a first implementation of the first aspect of the present invention, extracting density-enhanced node features from the monitoring video frames of multiple camera nodes includes: Spatial feature vectors are extracted from surveillance video frames from multiple camera nodes, and optical flow vector fields are calculated. The optical flow vector magnitude of each pixel in the monitoring area is extracted based on the optical flow vector field, and the optical flow vector magnitude is integrated to obtain the magnitude integral. The magnitude integral is then divided by the area of ​​the monitoring area to obtain the population density scalar. The spatial feature vector is concatenated with the population density scalar to obtain the concatenated feature vector; The concatenated feature vector is subjected to weight matrix transformation and bias operation to obtain a gated weight vector. The spatial feature vector is then multiplied element-wise with the gated weight vector to obtain density-enhanced node features.

[0006] Optionally, in a second implementation of the first aspect of the present invention, global features and local features are calculated based on the density-enhanced node features, and bidirectional temporal coding is performed on the global features and the local features to obtain bidirectional coded features, including: Based on the graph's adjacent edges and node degrees, the density-enhanced node features are summed using a neighborhood weighted method to obtain local features; Calculate the temporal difference features of the density-enhanced node features at adjacent time points, and perform projection transformation and weighted aggregation on the current node features, target node features, and the temporal difference features to obtain global features; A fusion coefficient is calculated based on the local features, the global features, and the population density scalar, and the local features and the global features are fused based on the fusion coefficient to obtain fused features; The fused features are subjected to bidirectional temporal coding to obtain bidirectional coded features.

[0007] Optionally, in a third implementation of the first aspect of the present invention, a fusion coefficient is calculated based on the local features, the global features, and the population density scalar, and the local features and the global features are fused based on the fusion coefficient to obtain fused features, including: The global features are subjected to dimensionality reduction transformation to obtain dimensionality-reduced global features. The local features, the dimensionality-reduced global features, and the population density scalar are then subjected to weight matrix transformation and bias operation to obtain fusion coefficients. The local features are multiplied element-wise by the fusion coefficients to obtain the weighted local features, and the dimensionality-reduced global features are multiplied element-wise by the complements of the fusion coefficients to obtain the weighted global features. The fused features are obtained by summing the weighted local features and the weighted global features element by element.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, the fused features are subjected to bidirectional temporal coding to obtain bidirectional coded features, including: Calculate the normalized cosine distance of the adjacency matrix at adjacent time points, and use the normalized cosine distance as the topological change rate; Calculate the forget gate parameters based on the topological change rate and the fusion features; The memory cells of the forward long short-term memory network are updated based on the forget gate parameters to obtain the forward hidden state, and the memory cells of the backward long short-term memory network are updated based on the forget gate parameters to obtain the backward hidden state. The forward hidden state and the backward hidden state are concatenated to obtain the bidirectional coding feature.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, a population momentum similarity matrix is ​​calculated based on the bidirectional coding features, and the population momentum similarity matrix is ​​superimposed on the self-attention score for encoding to obtain population co-evolutionary features, including: The temporal gradients of the bidirectional encoded features at adjacent time points are calculated and the temporal gradients of all nodes in the graph are summed to obtain the global population momentum vector. Construct a group momentum similarity matrix based on the global group momentum vector; The bidirectional encoded features are projected using query key values ​​and self-attention scores are calculated. The population momentum similarity matrix and the self-attention scores are then weighted and summed, and the population co-evolution features are obtained through multi-layer Transformer encoding.

[0010] Optionally, in a sixth implementation of the first aspect of the present invention, spatiotemporal encoding and decoding of the density-enhancing node features and calculation of the topology-preserving reconstruction error include: Based on the density-enhanced node features, spatial coding features are extracted, and the spatial coding features are temporally encoded to obtain a spatiotemporal coding vector; The spatiotemporal coding vector is reconstructed to obtain the reconstructed node features; Calculate the pixel-level reconstruction error between the density-enhanced node features and the reconstructed node features; The topological similarity of nodes connected by adjacent edges in the computational graph is preserved under the spatiotemporal encoding vector and reconstructed node features, while maintaining the error. The topology-preserving reconstruction error is obtained by weighted summation of the pixel-level reconstruction error and the topology similarity preservation error.

[0011] Optionally, in a seventh implementation of the first aspect of the present invention, generating a group abnormal behavior detection result based on the group cooperative evolution characteristics, the topology preservation reconstruction error, and the prototype distance anomaly probability includes: The group co-evolutionary features are mapped to supervised anomaly scores. A normalized threshold is generated based on the topology-preserving reconstruction error of normal samples in the training set. The topology-preserving reconstruction error is then normalized and mapped based on the normalized threshold to obtain unsupervised anomaly scores. The anomaly probability of the prototype distance is calculated based on normal and anomalous samples in the support set. A comprehensive anomaly score is calculated based on the supervised anomaly score, the unsupervised anomaly score, and the prototype distance anomaly probability, and a group abnormal behavior detection result is generated based on the comprehensive anomaly score.

[0012] Optionally, in the eighth implementation of the first aspect of the present invention, calculating the prototype distance anomaly probability based on normal samples and anomalous samples in the support set includes: The normal samples and abnormal samples in the support set are subjected to weight matrix transformation and activation function operation respectively to obtain the normal sample embedding vector and the abnormal sample embedding vector. The normal class prototype is obtained by averaging the embedding vectors of the normal samples, and the abnormal class prototype is obtained by averaging the embedding vectors of the abnormal samples. Clustering the embedding vectors of the abnormal samples yields multiple abnormal subclass prototype centers; The query sample is mapped to a query sample embedding vector, and the first Euclidean distance from the query sample embedding vector to the normal class prototype and the second Euclidean distance to the center of each abnormal subclass prototype are calculated. The anomaly probability of the prototype distance is calculated based on the first Euclidean distance and the second Euclidean distance.

[0013] This invention also provides a system for mining abnormal group behavior by fusing graph neural networks, comprising: The extraction module is used to extract density-enhanced node features from surveillance video frames of multiple camera nodes; The calculation module is used to calculate global features and local features based on the density-enhanced node features, and to perform bidirectional temporal coding on the global features and the local features to obtain bidirectional coded features; The encoding module is used to calculate the population momentum similarity matrix based on the bidirectional encoding features, and to superimpose the population momentum similarity matrix onto the self-attention score for encoding to obtain the population co-evolution features; The error calculation module is used to perform spatiotemporal encoding and decoding on the density-enhanced node features and calculate the topology-preserving reconstruction error. An abnormal behavior detection module is used to generate abnormal behavior detection results of the group based on the group cooperative evolution characteristics, the topology preservation reconstruction error, and the prototype distance anomaly probability.

[0014] In summary, this invention addresses the issue of inconsistent feature representation across different density scenarios by using a density-enhanced node feature mechanism that integrates optical flow density scalar with spatial feature vector gating. This allows for dynamic adjustment of node expressive power based on population density. Furthermore, by introducing temporal difference features as attention weights in the context information computation graph and combining them with density-adaptive gating to fuse local and global features, the invention achieves dynamic adjustment of attention allocation between nodes and adaptive switching between local and global aggregation strategies, effectively capturing the spatiotemporal dependency evolution patterns across nodes. Finally, by adjusting the forgetting gate parameters of the Long Short-Term Memory network through topology change rate, the invention makes the model sensitive to topological changes, enhancing its responsiveness to structural anomalies. By superimposing the group momentum similarity matrix onto the Transformer self-attention score, the co-evolutionary pattern of the global group is captured, improving the ability to identify abnormal behaviors at the group level. By simultaneously optimizing pixel-level reconstruction and node-to-node similarity preservation through topology-preserving reconstruction error constraints, the topological relationship consistency of the graph structure is maintained. Through multi-model weighted fusion decision-making based on supervised anomaly scores, unsupervised anomaly scores, and prototype distance anomaly probabilities, the advantages of labeled learning, unsupervised reconstruction, and few-shot metric learning are combined, improving the robustness of anomaly detection and the generalization ability to novel anomaly patterns, and solving the problem of insufficient detection accuracy of a single model in complex scenes and under few-shot conditions. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the steps of a method for mining abnormal group behavior by fusing graph neural networks in one embodiment of the present invention; Figure 2 This is a block diagram of the group abnormal behavior mining system fused with graph neural networks in an embodiment of the present invention.

[0016] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0018] Reference Figure 1 This embodiment provides a method for mining abnormal group behavior by fusing graph neural networks, including the following steps: S1, extract density-enhanced node features from the surveillance video frames of multiple camera nodes; S2, calculate global and local features based on density-enhanced node features, and perform bidirectional temporal coding on global and local features to obtain bidirectional coded features; S3. Calculate the population momentum similarity matrix based on bidirectional coding features, and superimpose the population momentum similarity matrix onto the self-attention score for encoding to obtain the population co-evolutionary features; S4 performs spatiotemporal encoding and decoding on the features of density-enhanced nodes and calculates the topology-preserving reconstruction error. S5 generates abnormal behavior detection results for the group based on the characteristics of group co-evolution, topology preservation and reconstruction error, and the probability of abnormal prototype distance.

[0019] In one example, density-enhanced node features are extracted from surveillance video frames from multiple camera nodes, including: Spatial feature vectors are extracted from surveillance video frames from multiple camera nodes, and optical flow vector fields are calculated. The optical flow vector magnitude of each pixel in the monitoring area is extracted based on the optical flow vector field, and the optical flow vector magnitude is integrated to obtain the magnitude integral. The magnitude integral is then divided by the area of ​​the monitoring area to obtain the population density scalar. The spatial feature vector is concatenated with the population density scalar to obtain the concatenated feature vector; The concatenated feature vectors are subjected to weight matrix transformation and bias operation to obtain the gated weight vector. The spatial feature vectors are then multiplied element-wise with the gated weight vector to obtain the density-enhanced node features.

[0020] In this example, video stream data from multiple fixed-location camera nodes is received, and a continuous video frame sequence is extracted at a fixed frame rate. For the input frame of each camera node at the current timestamp, spatial features are extracted using a pre-trained convolutional neural network. The fourth-stage residual block in the ResNet-50 network is used as the feature output channel. An intermediate feature map of size 14×14×2048 is obtained through convolutional layers, and global average pooling is used to compress the spatial dimension. At the same time, a 1×1 convolutional kernel is used to reduce the number of channels to 512 dimensions, thus obtaining the spatial feature vector of each camera node at each time step. A pre-trained ResNet-50 convolutional neural network was used as the spatial feature extractor. The input was a surveillance video frame image with a size of 224×224×3. Multi-scale features were extracted through the conv1 convolutional layer (7×7 convolutional kernel, 64 channels, stride 2), the max pooling layer (3×3, stride 2), and the residual block groups from conv2_x to conv4_x (containing 3, 4, 6, and 3 residual units respectively, with channel numbers of 64, 128, 256, and 512 respectively). The output feature map of the conv4_x layer was truncated to a size of 14×14×512 and compressed into a 512-dimensional spatial feature vector through the Global Average Pooling2D layer. The momentum parameter of the BatchNormalization layer was set to 0.9, and the negative slope of the ReLU activation function was set to 0. To address the temporal dimension between consecutive frames, the Farneback dense optical flow algorithm is introduced to perform pixel-by-pixel registration of adjacent frames, calculating the optical flow components of each pixel in the horizontal and vertical directions to construct a two-dimensional optical flow vector field. For all pixels in the monitoring area corresponding to a camera node, the optical flow vector magnitude is extracted, i.e., the Euclidean norm is calculated for each pixel coordinate point to obtain a pixel-level velocity amplitude map. The optical flow magnitude over the entire monitoring area is then integrated over an area to reflect the overall activity level of the group's movement in the monitoring image. Dividing the integral value of the magnitude by the total pixel area constant of the monitoring area yields a normalized density scalar value, which serves as a density index characterizing the intensity of crowd movement in a local area. The density scalar value is copied into a vector form with the same dimension as the spatial feature vector through a dimension expansion operation, and then concatenated with the original spatial feature vector along the channel direction to form a new concatenated feature vector. Subsequently, a linear transformation is performed through a fully connected layer. The concatenated feature vector is then input into a gated mapping network, where an affine transformation is performed using the initialized weight matrix and bias vector. The output value is restricted to between 0 and 1 by the Sigmoid activation function, resulting in a gated weight vector with the same dimension as the original spatial feature vector. Logically, the gated weight vector is equivalent to a mask that dynamically adjusts the saliency of the channels.The original spatial feature vector and the gate weight vector are subjected to the Hadamard product operation along the element dimension. That is, each spatial feature is multiplied by the corresponding gate coefficient to achieve feature enhancement. In dense crowd scenes, the activation response of key channels is improved, while inefficient noise dimensions are suppressed in sparse scenes, resulting in density-enhanced node features with sensitivity to group movement.

[0021] The density-enhanced gating network takes a 512-dimensional spatial feature vector and a 1-dimensional population density scalar as input. First, the population density scalar is copied and expanded 512 times to form a 512-dimensional density vector, which is then concatenated with the spatial feature vector to obtain a 513-dimensional concatenated feature vector. This concatenated feature vector is then input to the fully connected layer for weight matrix transformation (weight matrix W). ρ The dimension is 512×513, and the bias vector is b. ρ The dimension is 512. After being mapped to the (0,1) interval by the Sigmoid activation function, a 512-dimensional gated weight vector is obtained. The gated weight vector is multiplied element-wise with the spatial feature vector to obtain the 512-dimensional density-enhanced node features. The fully connected layer is initialized uniformly by Xavier and the learning rate is set to 0.0001.

[0022] In one example, global and local features are calculated based on density-enhanced node features, and bidirectional temporal encoding is performed on the global and local features to obtain bidirectional encoded features, including: Based on the graph's adjacent edges and node degree, the density-enhanced node features are summed by neighborhood weighting to obtain local features. Calculate the temporal difference features of density-enhanced node features at adjacent time points, and perform projection transformation and weighted aggregation on the current node features, target node features, and temporal difference features to obtain global features; The fusion coefficient is calculated based on local features, global features, and population density scalar, and the local features and global features are fused based on the fusion coefficient to obtain the fused features; The fused features are bidirectionally time-series encoded to obtain bidirectional encoded features.

[0023] In this example, based on the pre-constructed set of adjacent edges and corresponding edge weights in the graph structure, a neighborhood-weighted summation is performed on the density-enhanced node features. For each camera node, its set of one-hop adjacent nodes defined in the graph is traversed, and a normalization coefficient is calculated based on the edge weights and the degree information of the nodes in the graph. This coefficient is used as a weighting factor to perform a weighted summation on the density-enhanced features of neighboring nodes. Simultaneously, the weighted sum is input into a shared linear mapping network and processed through a nonlinear activation function to obtain the local feature representation of each node at the current time step, characterizing the local collaborative relationship between the node and its neighboring cameras in space. To capture the evolutionary dynamics of group behavior over time, a temporal difference mechanism is introduced. For each target node, the difference between its density-enhanced feature vectors at adjacent time steps is calculated to obtain the temporal difference feature, which reflects the changing trend of the node's state between two consecutive time steps. Based on the graph attention mechanism, the density enhancement features of the current node, the density enhancement features of the target node, and the temporal difference features of the target node are each introduced into a projection matrix for subspace mapping. The mapped vectors are then concatenated to form a context-enhanced feature representation. The attention parameter vector is then input to calculate the attention logit score, which is normalized using a softmax function to generate context-aware global attention weights. These global attention weights are then used to weight and sum the projected features of all nodes, forming a global feature encoding long-range dependencies across the entire graph. The global feature is compressed and mapped to the same dimension as the local features. The local features, the compressed global features, and the collective density scalar of the nodes are then concatenated into a combined feature, which is input into a gating network. Affine transformation and a sigmoid activation function are used to calculate the fusion coefficients. These fusion coefficients act as modulation weights, controlling the proportion of local and global feature fusion. In high-density scenarios, the fusion coefficients tend to favor global paths, while in low-density scenarios, local path features are emphasized. The fusion coefficients are used to weight the outputs of local and global paths, forming a unified fused feature. A bidirectional temporal coding structure is introduced, and a bidirectional LSTM is selected as the core framework of the encoder. The fused feature sequence is encoded from both the forward and backward time dimensions. At each time step, the forward hidden state and the backward hidden state are concatenated to obtain bidirectional encoded features that simultaneously contain historical and future temporal context information.

[0024] The graph convolutional network takes n nodes as input (512-dimensional density-enhanced node features) as input. The edge connection rule for the graph adjacency matrix A is that edges are established when the camera field-of-view overlap is greater than a threshold θ = 0.15. Symmetric normalization of the adjacency matrix yields the normalized adjacency matrix à = D. (-1 / 2) AD (-1 / 2) Where D is the degree matrix, and the features of the neighboring nodes are weighted and summed using the normalized adjacency matrix, and then processed by the weight matrix W. local(Dimensionality 256×512) Dimensionality reduction transformation and ReLU activation function yield 256-dimensional local features. The weight matrix is ​​initialized with He, and the Dropout ratio is set to 0.3 to prevent overfitting.

[0025] Multi-head Graph Attention Network: Eight attention heads are set up, each with a projection dimension of 64. The input is 512-dimensional density-enhanced node features. First, the feature differences between adjacent time steps are calculated to obtain a 512-dimensional temporal difference feature vector. The current node features, target node features, and temporal difference features are then projected through the projection matrix W. h (64×512 dimensions) transformed into a 64-dimensional query, key, and value vector; temporal difference features are then projected through a separate matrix W. hdelta A transformation of (64×512 dimensions) concatenates the three elements into a 192-dimensional vector and the attention parameter vector a. h (Dimension 192×1) Dot product and LeakyReLU (negative slope 0.2) are used to calculate the attention log value. The entire graph nodes are normalized to obtain the context-aware attention weight. The value vector is weighted and aggregated based on the attention weight. The outputs of the 8 heads are concatenated to form a 512-dimensional global feature. The projection matrix is ​​initialized using Glorot.

[0026] In one example, a fusion coefficient is calculated based on local features, global features, and a population density scalar. The local and global features are then fused based on this fusion coefficient to obtain fused features, including: The global features are dimensionality reduced to obtain dimensionality-reduced global features. The local features, dimensionality-reduced global features, and population density scalar are then subjected to weight matrix transformation and bias operation to obtain the fusion coefficients. The local features are multiplied element-wise by the fusion coefficients to obtain the weighted local features, and the reduced global features are multiplied element-wise by the complements of the fusion coefficients to obtain the weighted global features. The fused features are obtained by summing the weighted local features and weighted global features element by element.

[0027] In this example, a dimensionality reduction transformation is performed on the global features. Since the dimensionality of the global features is higher than that of the local features, a linear dimensionality reduction transformation operation is introduced to match the representation dimension of the local path. This linear dimensionality reduction transformation operation is implemented through a trainable, learnable fully connected layer. The original high-dimensional global feature vector is input into the dimensionality reduction mapping matrix, and a bias term is added to output a vector with the same dimension as the local features, resulting in a dimensionality-reduced global feature with consistent structure. Combining the local features, the dimensionality-reduced global features, and the population density scalar, the three elements are concatenated along the feature dimension direction to form a joint vector. The concatenation result is input into a fusion gating unit, which contains a learnable weight matrix and a bias vector. After an affine transformation, it is connected to a sigmoid activation function to output a fusion coefficient vector ranging from 0 to 1. Logically, the fusion coefficient can be viewed as a channel-level weighted mask to balance the contribution ratio of local and global paths. The fusion coefficients are multiplied element-wise with their corresponding local feature vectors to obtain a weighted local feature representation. Simultaneously, the element-wise complement of the fusion coefficients is calculated by subtracting the fusion coefficient vector from the unit vector. This result is then multiplied element-wise with the reduced global feature vector to form a weighted global feature representation. The weighted local features and weighted global features are then added element-wise at their corresponding positions, summed channel-wise, to achieve dynamic fusion of local information and long-range dependencies, resulting in the fused feature.

[0028] In one example, bidirectional temporal encoding is performed on the fused features to obtain bidirectional encoded features, including: Calculate the normalized cosine distance of the adjacency matrix at adjacent time points, and use the normalized cosine distance as the topological change rate; Calculate forget gate parameters based on topological change rate and fusion features; The memory units of the forward long short-term memory network are updated based on the forget gate parameters to obtain the forward hidden state, and the memory units of the backward long short-term memory network are updated based on the forget gate parameters to obtain the backward hidden state. The forward hidden state and the backward hidden state are concatenated to obtain the bidirectional encoded features.

[0029] In this example, the adjacency matrix of the graph structure at two consecutive time steps is obtained from the time series. For each camera node, the row vectors corresponding to its adjacency matrix at the current and previous time steps are extracted, representing the connection relationships between this node and all other nodes in the graph. L2 norm normalization is performed on these two vectors, and the cosine similarity between them is calculated. Subtracting the cosine similarity value from 1 yields the topological change rate of the node at that time step, reflecting whether the spatial structure of the group has undergone drastic adjustments, such as crowd aggregation, dispersion, or reorganization. The topological change rate is used as an external dynamic input and concatenated with the fusion features of the node at the current time step to form a new input vector. This vector is then input into the forgetting gate structure in the Long Short-Term Memory (LSTM) network. An affine transformation is performed through a set of trainable weight matrices and bias parameters. The topological change rate is then broadcast to a dimension consistent with the fusion features, allowing the forgetting gate to consider the joint effects of node feature changes and graph structure changes when generating the gating factor. The forgetting gate parameter, between 0 and 1, is then calculated using the Sigmoid activation function. This gating coefficient adjusts the strength of historical state retention during memory updates. In the forward LSTM path, the forget gate coefficient is used to modulate the memory state of the previous time step, and the current input is modulated by the input gate to update the new memory unit, generating the forward hidden state of the current time step. The same operation is repeated for the backward LSTM path, with the processing order reversed from the last time step. The forget gate of the backward path is still calculated using the corresponding fused feature sequence and the corresponding topological change rate, and the memory unit is updated and the backward hidden state is calculated accordingly. At each time step, the two hidden state vectors obtained from the forward and backward paths are concatenated by channel to form a bidirectional encoded feature containing past information and future context.

[0030] The bidirectional Long Short-Term Memory (LSTM) network consists of two LSTMs, each containing 128 hidden units. The input is a 256-dimensional fusion feature of time series length m. First, the cosine distance between adjacent row vectors in the adjacency matrix is ​​calculated to obtain a topological change rate scalar. This topological change rate is then transformed with the weights of the fusion feature and the previous hidden state (weight matrix W). f Dimensions 128×384, U f The sum of 128×1 units is mapped to the forgetting gate parameter f using the Sigmoid function. t Input gate i t Candidate memory c̃ t and output gate o t The weight matrix W is calculated using the standard LSTM formula. i W c W o All dimensions are 128×384, and the memory unit c t Through c t =f t ⊙c(t-1) +i t ⊙c̃ t Update, hidden state h t =o t ⊙Tanh(c t The forward and backward hidden states are concatenated to form a 256-dimensional bidirectional encoded feature. The LSTM weights are orthogonally initialized, and the forget gate bias is initialized to 1.0.

[0031] The Transformer encoder consists of four Transformer coding layers, each with four attention heads. The input is a 256-dimensional bidirectional encoded feature sequence. First, sinusoidal positional encodings PE(t,2k)=sin(t / 10000^(2k / 256)) and PE(t,2k+1)=cos(t / 10000^(2k / 256)) are added. The gradients of the bidirectional encoded features at adjacent time steps are calculated and summed over all nodes in the graph to obtain a 256-dimensional global population momentum vector. The cosine similarity of the momentum vectors at different time steps constitutes the population momentum similarity matrix M. crowd The query and key are queried through the projection matrix W. Q W K (256×256) transformation, the dimension d of each head k =64, the self-attention score is calculated as QK^T / √64, then the weighting coefficient λ is added. crowd The group momentum similarity matrix with a value of 0.3, after softmax normalization, is used to compare the value vectors (W). V Projection (256×256 dimension) weighted summation, multi-head output through output projection matrix W O The (256×256) transformation, residual connections, and LayerNormalization (ε=1e-6) are then input into the feedforward network FFN. FFN contains two fully connected layers (the first layer has 512-dimensional ReLU activation, and the second layer has 256-dimensional linear activation). The population co-evolution features are obtained by residual connections and LayerNormalization again. The Transformer weights are initialized with Xavier and the Dropout ratio is set to 0.1.

[0032] In one example, a population momentum similarity matrix is ​​calculated based on bidirectional encoding features, and this matrix is ​​then superimposed on the self-attention score for encoding to obtain population co-evolutionary features, including: The temporal gradients of the bidirectional encoded features at adjacent time points are calculated and the temporal gradients of all nodes in the graph are summed to obtain the global population momentum vector. Construct a population momentum similarity matrix based on the global population momentum vector; The bidirectional encoded features are projected using query key values ​​and self-attention scores are calculated. The group momentum similarity matrix and self-attention scores are weighted and summed, and the group co-evolution features are obtained through multi-layer Transformer encoding.

[0033] In this example, the bidirectional encoded features of each camera node are subjected to time-series differencing. Specifically, at each time step, the bidirectional feature vector at the current time step is subtracted from the bidirectional feature vector at the previous time step to obtain the temporal gradient vector of the node within the time period, reflecting the evolution direction and magnitude of the node's behavior over time. At each time point, the temporal gradient vectors of all nodes are summed and divided by the total number of nodes to perform normalization, resulting in a group momentum vector representing the current global population evolution trend. This group momentum vector reflects the average direction of individual changes and encodes the collaborative behavioral evolution characteristics of the overall behavior within the graph structure. To measure the consistency of the group behavior evolution trend across different time points, the cosine similarity between each pair of group momentum vectors corresponding to all time points is calculated using the time dimension as an index, forming a group momentum similarity matrix. Each element of the matrix represents the degree of similarity in the group evolution direction between two time steps; a value closer to 1 indicates a more consistent population movement trend at the two time points, while a value closer to -1 indicates opposite behavioral change directions. The group momentum similarity matrix is ​​introduced as a global collaborative constraint into the Transformer's self-attention mechanism to enhance the global perception capability of time series modeling. After adding absolute position encoding to the bidirectional encoded features at each time step, they are input into trainable query, key, and value projection matrices respectively. Affine mapping is used to generate query vectors, key vectors, and value vectors for each time step. The standard self-attention score matrix is ​​calculated by dividing the dot product of the query vector and all key vectors by a scaling factor. The group momentum similarity matrix and the self-attention score matrix are then summed element-wise along the time dimension. Prior knowledge of the overall evolutionary trend of the group is introduced during the attention weight generation process, enabling the attention mechanism to not only focus on the local similarity of features between current nodes but also perceive the changing trend of similarity in the overall group's behavioral rhythm. The fused attention weights are used to weight the value vectors at all times to generate the context representation corresponding to each attention head. The final multi-head attention output is constructed by concatenating the output vectors of multiple attention heads and performing linear transformations. After residual connection and layer normalization, the result is input into the feedforward network layer, and multiple Transformer encoder modules are repeatedly stacked to continuously enhance the model's ability to model long-distance temporal dependencies and group collaborative dynamics, and output the group collaborative evolution characteristics.

[0034] The convolutional-LSTM spatiotemporal autoencoder consists of four convolutional layers. The input is density-enhanced node features reshaped to 14×14×512. The first convolutional layer (64 channels, 3×3 kernel, stride 2, same padding) outputs 7×7×64; the second convolutional layer (128 channels) outputs 4×4×128; the third convolutional layer (256 channels) outputs 2×2×256; and the fourth convolutional layer (512 channels) outputs 1×1×512. Each convolutional layer is followed by BatchNormalization and ReLU activation, flattening the output into 512-dimensional spatial encoded features. The temporal encoder is a two-layer LSTM (128 hidden units per layer), with the input being a sequence of 512-dimensional spatial encoded features over m time steps, outputting a 128-dimensional spatiotemporal encoded vector. The temporal decoder is a two-layer LSTM inverse... The system outputs 512×m dimensional features after unfolding m time steps. The spatial decoder contains four deconvolutional layers (DeConv). The first deconvolutional layer (256 channels, 3×3 kernel, stride 2) outputs 2×2×256, the second layer (128 channels) outputs 4×4×128, the third layer (64 channels) outputs 8×8×64, the fourth layer (32 channels) outputs 16×16×32, and the last 1×1 convolutional layer outputs 14×14×512 reconstructed node features. The convolutional kernels are initialized with He, and the LSTM is initialized with orthogonality. The reconstruction loss is the sum of the pixel-level squared error and the topological similarity preservation error with a weighted coefficient λ_topo=0.5. The Adam optimizer learning rate is set to 0.0001, the batch size is 32, and the training lasts for 100 epochs.

[0035] In one example, spatiotemporal encoding and decoding are performed on density-enhanced node features to calculate the topology-preserving reconstruction error, including: Spatial coding features are extracted based on density-enhanced node features, and these spatial coding features are then temporally encoded to obtain a spatiotemporal coding vector. The spatiotemporal coding vector is reconstructed to obtain the reconstructed node features; Calculate the pixel-level reconstruction error between density-enhanced node features and reconstructed node features; The topological similarity of node pairs connected by adjacent edges in the computational graph is preserved under the spatiotemporal encoded vector and reconstructed node features, thus maintaining the error. The topology-preserving reconstruction error is obtained by weighted summation of the pixel-level reconstruction error and the topology similarity preservation error.

[0036] In this example, the density enhancement node features of each node at each time stamp are reshaped into a three-dimensional spatial tensor to restore its spatial structure in the ResNet feature map, for example, a 14×14×512 feature map. The three-dimensional spatial tensor is input into a four-layer two-dimensional convolutional neural network as a spatial encoder module. Through layer-by-layer downsampling and channel upsampling operations, the spatial dimension of the input feature map is gradually compressed to 1×1, while the number of channels is maintained or increased layer by layer from 512 until a fixed-length spatial encoding vector is obtained through a flattening operation. After obtaining the spatial encoding at each time step, the spatial encoding features at multiple consecutive time points are arranged in chronological order, on a node-by-node basis, to form a temporal encoding input sequence. This sequence is then input into a two-layer stacked Long Short-Term Memory (LSTM) network for temporal encoding processing. The first LSTM layer extracts the feature evolution patterns within a local time window, while the second LSTM layer compresses and integrates the dynamic change information of the entire time period, outputting a fixed-dimensional spatiotemporal encoding vector. The spatiotemporal encoding vector is a compact representation of the spatiotemporal behavior of each node throughout the entire observation period. A two-layer stacked LSTM decoder progressively unfolds the spatiotemporal encoded vector to reconstruct a temporal feature sequence with the same length as the original. During training, a teacher forcing strategy is used to enable the decoder to use real input at each step to enhance convergence, while during inference, the predicted values ​​from the previous step are recursively unfolded. The reconstructed temporal features at each time step are reshaped into a 1×1×512 tensor and input into the transposed convolutional module, which is a mirror image of the encoder. That is, four deconvolutional layers are used to progressively upsample the compressed representation to reconstruct a 14×14×512 reconstructed feature map, which is then flattened into a vector to obtain reconstructed node features with the same dimension as the original density-enhanced features. The sum of the element-wise squared Euclidean distances between the density-enhanced node features and the reconstructed node features at each time step is calculated as the pixel-level reconstruction error, and the summation over all nodes and time steps is used as the first reconstruction metric. A structure-aware topology preservation mechanism is introduced. For node pairs connected by edges in the original graph, the cosine similarity between their original density-enhanced space and their reconstructed space is calculated at each time point. Then, the absolute error between the two similarities is calculated to obtain the relative relationship offset of the node pair in the topological representation due to reconstruction. The summation is then performed over all edges and all time steps to form the second topology preservation error index. The pixel-level reconstruction error and the topology preservation error are then weighted and summed according to a set weight coefficient to form the total topology preservation reconstruction error.

[0037] In one example, anomalous group behavior detection results are generated based on group co-evolutionary characteristics, topology-preserving reconstruction error, and prototype distance anomaly probability, including: The group co-evolutionary features are mapped to supervised anomaly scores. A normalized threshold is generated based on the topology-preserving reconstruction error of normal samples in the training set. The topology-preserving reconstruction error is then normalized and mapped based on the normalized threshold to obtain unsupervised anomaly scores. The anomaly probability of the prototype distance is calculated based on normal and anomalous samples in the support set. A comprehensive anomaly score is calculated based on the supervised anomaly score, the unsupervised anomaly score, and the prototype distance anomaly probability. The results of group abnormal behavior detection are then generated based on the comprehensive anomaly score.

[0038] In this example, the group co-evolutionary features are input into a fully connected mapping unit with a Sigmoid activation function. The unit parameters are learned through supervised training and labeled anomaly category information. Its output is a supervised anomaly score ranging from 0 to 1, which represents the similarity between the current node's behavioral trajectory and the anomaly samples in the training set within the encoded time window. Since this process is directly driven by anomaly labels, it has high end-to-end discriminative ability. Simultaneously, the topology-preserving reconstruction error of each node in the time series dimension is obtained from the autoencoder reconstruction path. This error represents the deviation between the original features and the reconstructed features in terms of feature value accuracy and structural similarity. First, based on the set of reconstruction error values ​​in the training set containing only normal samples, the 95th percentile of the reconstruction error value set is statistically analyzed and defined as a normalization threshold. Then, the original reconstruction error value of each node is divided by the normalization threshold and input into the Sigmoid function for normalization mapping, resulting in an unsupervised anomaly score in the [0,1] interval, reflecting the anomaly estimation of the node under the unsupervised reconstruction mechanism. To enhance the model's ability to classify complex group abnormal behaviors in scenarios with few samples, a metric learning-driven prototype network structure is introduced. The limited number of labeled support set samples are divided into normal and abnormal classes, with the abnormal class further subdivided into multiple subclasses. Multiple abnormal prototype centers are generated using K-means clustering. The group co-evolutionary features of the current query node are then converted into fixed-dimensional embedding vectors via an embedding network, and the Euclidean distances to normal and abnormal prototypes are calculated. The distances are then converted into anomaly probabilities using a Softmax function, resulting in the prototype distance anomaly probability, which serves as the third scoring source. The supervised anomaly score, unsupervised anomaly score, and prototype distance anomaly probability are weighted and fused proportionally with weights of 0.5, 0.3, and 0.2, respectively. These weights are based on the Transformer end-to-end model's strong learning ability (giving it the highest weight), the unsupervised path's sensitivity to noise (not dependent on labels), and the prototype network's limited support set coverage (giving it a low weight). A unified comprehensive anomaly score is obtained by weighted summation of these three factors. Set an optimal threshold based on the ROC curve selected on the validation set. When the comprehensive anomaly score exceeds the threshold, it is determined that there is group abnormal behavior in the area observed by the current camera node during this time period, and the corresponding detection results are generated. The output includes binary anomaly labels and continuous anomaly confidence scores.

[0039] The prototype network embedding layer consists of three fully connected layers. The input is a 256-dimensional population co-evolutionary feature. The first fully connected layer is 256-dimensional with ReLU activation, the second is 128-dimensional with ReLU activation, and the third is 64-dimensional with no activation function. The output is a 64-dimensional embedding vector. Each layer is followed by a Dropout layer (scale 0.3) to prevent overfitting. The weight matrix is ​​initialized using Xavier. It supports averaging the embedding vectors of normal samples to obtain 64-dimensional normal class prototypes and averaging the embedding vectors of abnormal samples to obtain 64-dimensional abnormal class prototypes. K-means clustering (K=5, maximum iterations 100, initialization method k-means++) is performed on the abnormal sample embedding vectors to obtain five 64-dimensional abnormal subclass prototype centers. The Euclidean distance from the query sample embedding vector to each prototype is calculated as ||z||. query -c k || 2 Distance negative value divided by temperature coefficient τ proto =0.1 and then normalized by softmax to obtain the prototype distance anomaly probability. The prototype network is trained using the episode training method. Each episode randomly samples 5 normal samples and 5 abnormal samples to form the support set.

[0040] In one example, the prototype distance anomaly probability is calculated based on normal and anomalous samples in the support set, including: The normal samples and abnormal samples in the support set are subjected to weight matrix transformation and activation function operation respectively to obtain the normal sample embedding vector and the abnormal sample embedding vector. The normal class prototype is obtained by averaging the embedding vectors of normal samples, and the abnormal class prototype is obtained by averaging the embedding vectors of abnormal samples. Clustering of the embedding vectors of abnormal samples yields multiple prototype centers of abnormal subclasses; Map the query sample to the query sample embedding vector, and calculate the first Euclidean distance from the query sample embedding vector to the normal class prototype and the second Euclidean distance to the center of each abnormal subclass prototype. The anomaly probability of the prototype distance is calculated based on the first Euclidean distance and the second Euclidean distance.

[0041] In this example, the co-evolutionary feature vectors of all normal and abnormal samples are extracted from the support set and uniformly input into an embedding mapping network consisting of multiple fully connected layers. The weight matrix and bias parameters of each layer are trainable, and ReLU activation functions are used between layers to introduce non-linearity. The final output is a fixed-dimensional embedding vector, resulting in sets of embedding vectors for normal and abnormal samples. The embedding vectors of all normal samples are averaged along the sample dimension to obtain a normal class prototype vector, which serves as the central representation of normal group behavior in the metric space. Similarly, the embedding vectors of all abnormal samples are averaged to construct a single abnormal class prototype. Clustering is performed on the abnormal sample embedding vectors using the K-means algorithm, clustering them into several classes. The number of cluster centers is set to K, with a value of 5, to cover typical group abnormal subtypes such as aggregation, dispersion, congestion, conflict, and retrograde movement. Through iterative solution, K abnormal subclass prototype centers are converged, forming multiple category anchor points representing specific abnormal patterns in the feature space. For each query sample to be detected, it is input into an embedding mapping network to obtain its corresponding query sample embedding vector. Then, the Euclidean distance from the query sample embedding vector to the normal class prototype is calculated, denoted as the first Euclidean distance. Simultaneously, the Euclidean distances from the query sample embedding vector to the centers of the K abnormal subclass prototypes are calculated, forming a set of second Euclidean distance vectors. A temperature adjustment parameter is introduced, and all distances are subjected to Softmax normalization. The calculated result is the prototype distance anomaly probability. Using negative distance as a similarity metric, the negative distance from the query sample to the normal prototype is exponentially calculated as the denominator, and the negative distances to each abnormal subclass prototype are exponentially calculated and summed as the numerator. Dividing the two yields the anomaly probability. The anomaly probability reflects whether the query sample is closer to the abnormal class prototype or the normal class prototype in the metric space, thus enabling effective discrimination and inference of unseen abnormal behavior under small sample data conditions.

[0042] The anomaly score fusion network consists of two parts: Supervised anomaly scores map the 256-dimensional population co-evolution features to a [0,1] scalar through a fully connected layer (weight matrix 256×1, bias scalar) and a Sigmoid activation function, and are trained using binary cross-entropy loss; Unsupervised anomaly scores use the 95th percentile of the topology-preserving reconstruction error of the normal samples in the training set as the normalization threshold τ. r Divide the topology-preserving reconstruction error by τ r The scores were then normalized to the [0,1] interval using the Sigmoid function. The three anomaly scores were then weighted and summed according to their weights w1=0.5, w2=0.3, and w3=0.2 to obtain a comprehensive anomaly score. The decision threshold τ was then used. decision =0.6 is determined by the optimal F1-score of the ROC curve of the validation set. The comprehensive anomaly score is greater than the decision threshold and is judged as abnormal behavior.

[0043] Reference Figure 2 This embodiment provides a system for mining abnormal group behavior by fusing graph neural networks, including: Extraction module 1 is used to extract density-enhanced node features from surveillance video frames of multiple camera nodes; Calculation module 2 is used to calculate global and local features based on density-enhanced node features, and to perform bidirectional temporal coding on the global and local features to obtain bidirectional coded features; Encoding module 3 is used to calculate the group momentum similarity matrix based on bidirectional encoding features, and to superimpose the group momentum similarity matrix onto the self-attention score for encoding to obtain the group co-evolution features; Error calculation module 4 is used to perform spatiotemporal encoding and decoding on the features of density-enhanced nodes and calculate the topology-preserving reconstruction error. The abnormal behavior detection module 5 is used to generate abnormal behavior detection results of the group based on the group's cooperative evolution characteristics, topology preservation reconstruction error, and prototype distance anomaly probability.

[0044] In this embodiment, the specific implementation of each unit in the above system embodiment is described in the above method embodiment, and will not be repeated here.

[0045] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, system, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, system, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, system, article, or method that includes that element.

[0046] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for mining abnormal group behavior by fusing graph neural networks, characterized in that, include: Extract density-enhanced node features from surveillance video frames of multiple camera nodes; Global and local features are calculated based on the density-enhanced node features, and bidirectional temporal coding is performed on the global and local features to obtain bidirectional coded features; The population momentum similarity matrix is ​​calculated based on the bidirectional coding features, and the population momentum similarity matrix is ​​superimposed on the self-attention score for encoding to obtain the population co-evolution features; Spatiotemporal encoding and decoding are performed on the density-enhanced node features to calculate the topology-preserving reconstruction error; Based on the group's cooperative evolution characteristics, the topology preservation and reconstruction error, and the prototype distance anomaly probability, the group's abnormal behavior detection results are generated.

2. The method for mining abnormal group behavior using fused graph neural networks according to claim 1, characterized in that, Extract density-enhanced node features from surveillance video frames of multiple camera nodes, including: Spatial feature vectors are extracted from surveillance video frames from multiple camera nodes, and optical flow vector fields are calculated. The optical flow vector magnitude of each pixel in the monitoring area is extracted based on the optical flow vector field, and the optical flow vector magnitude is integrated to obtain the magnitude integral. The magnitude integral is then divided by the area of ​​the monitoring area to obtain the population density scalar. The spatial feature vector is concatenated with the population density scalar to obtain the concatenated feature vector; The concatenated feature vector is subjected to weight matrix transformation and bias operation to obtain a gated weight vector. The spatial feature vector is then multiplied element-wise with the gated weight vector to obtain density-enhanced node features.

3. The method for mining abnormal group behavior using fused graph neural networks according to claim 2, characterized in that, Global and local features are calculated based on the density-enhanced node features, and bidirectional temporal coding is performed on the global and local features to obtain bidirectional coded features, including: Based on the graph's adjacent edges and node degrees, the density-enhanced node features are summed using a neighborhood weighted method to obtain local features; Calculate the temporal difference features of the density-enhanced node features at adjacent time points, and perform projection transformation and weighted aggregation on the current node features, target node features, and the temporal difference features to obtain global features; A fusion coefficient is calculated based on the local features, the global features, and the population density scalar, and the local features and the global features are fused based on the fusion coefficient to obtain fused features; The fused features are subjected to bidirectional temporal coding to obtain bidirectional coded features.

4. The method for mining abnormal group behavior using fused graph neural networks according to claim 3, characterized in that, A fusion coefficient is calculated based on the local features, the global features, and the population density scalar. The local features and global features are then fused based on the fusion coefficient to obtain fused features, including: The global features are subjected to dimensionality reduction transformation to obtain dimensionality-reduced global features. The local features, the dimensionality-reduced global features, and the population density scalar are then subjected to weight matrix transformation and bias operation to obtain fusion coefficients. The local features are multiplied element-wise by the fusion coefficients to obtain the weighted local features, and the dimensionality-reduced global features are multiplied element-wise by the complements of the fusion coefficients to obtain the weighted global features. The fused features are obtained by summing the weighted local features and the weighted global features element by element.

5. The method for mining abnormal group behavior using fused graph neural networks according to claim 3, characterized in that, The fused features are subjected to bidirectional temporal coding to obtain bidirectional coded features, including: Calculate the normalized cosine distance of the adjacency matrix at adjacent time points, and use the normalized cosine distance as the topological change rate; Calculate the forget gate parameters based on the topological change rate and the fusion features; The memory cells of the forward long short-term memory network are updated based on the forget gate parameters to obtain the forward hidden state, and the memory cells of the backward long short-term memory network are updated based on the forget gate parameters to obtain the backward hidden state. The forward hidden state and the backward hidden state are concatenated to obtain the bidirectional coding feature.

6. The method for mining abnormal group behavior using fused graph neural networks according to claim 5, characterized in that, Based on the bidirectional encoding features, a population momentum similarity matrix is ​​calculated, and this matrix is ​​superimposed on the self-attention score for encoding to obtain population co-evolutionary features, including: The temporal gradients of the bidirectional encoded features at adjacent time points are calculated and the temporal gradients of all nodes in the graph are summed to obtain the global population momentum vector. Construct a group momentum similarity matrix based on the global group momentum vector; The bidirectional encoded features are projected using query key values ​​and self-attention scores are calculated. The population momentum similarity matrix and the self-attention scores are then weighted and summed, and the population co-evolution features are obtained through multi-layer Transformer encoding.

7. The method for mining abnormal group behavior using fused graph neural networks according to claim 1, characterized in that, Spatiotemporal encoding and decoding are performed on the density-enhanced node features to calculate the topology-preserving reconstruction error, including: Based on the density-enhanced node features, spatial coding features are extracted, and the spatial coding features are temporally encoded to obtain a spatiotemporal coding vector; The spatiotemporal coding vector is reconstructed to obtain the reconstructed node features; Calculate the pixel-level reconstruction error between the density-enhanced node features and the reconstructed node features; The topological similarity of nodes connected by adjacent edges in the computational graph is preserved under the spatiotemporal encoding vector and reconstructed node features, while maintaining the error. The topology-preserving reconstruction error is obtained by weighted summation of the pixel-level reconstruction error and the topology similarity preservation error.

8. The method for mining abnormal group behavior using fused graph neural networks according to claim 1, characterized in that, Based on the aforementioned group cooperative evolution characteristics, the aforementioned topology preservation reconstruction error, and the prototype distance anomaly probability, a group abnormal behavior detection result is generated, including: The group co-evolutionary features are mapped to supervised anomaly scores. A normalized threshold is generated based on the topology-preserving reconstruction error of normal samples in the training set. The topology-preserving reconstruction error is then normalized and mapped based on the normalized threshold to obtain unsupervised anomaly scores. The anomaly probability of the prototype distance is calculated based on normal and anomalous samples in the support set. A comprehensive anomaly score is calculated based on the supervised anomaly score, the unsupervised anomaly score, and the prototype distance anomaly probability, and a group abnormal behavior detection result is generated based on the comprehensive anomaly score.

9. The method for mining abnormal group behavior using fused graph neural networks according to claim 8, characterized in that, The prototype distance anomaly probability is calculated based on normal and anomalous samples in the support set, including: The normal samples and abnormal samples in the support set are subjected to weight matrix transformation and activation function operation respectively to obtain the normal sample embedding vector and the abnormal sample embedding vector. The normal class prototype is obtained by averaging the embedding vectors of the normal samples, and the abnormal class prototype is obtained by averaging the embedding vectors of the abnormal samples. Clustering the embedding vectors of the abnormal samples yields multiple abnormal subclass prototype centers; The query sample is mapped to a query sample embedding vector, and the first Euclidean distance from the query sample embedding vector to the normal class prototype and the second Euclidean distance to the center of each abnormal subclass prototype are calculated. The anomaly probability of the prototype distance is calculated based on the first Euclidean distance and the second Euclidean distance.

10. A system for mining abnormal group behavior by fusing graph neural networks, characterized in that, The steps for implementing the group anomaly behavior mining method of the fused graph neural network according to any one of claims 1 to 9 include: The extraction module is used to extract density-enhanced node features from surveillance video frames of multiple camera nodes; The calculation module is used to calculate global features and local features based on the density-enhanced node features, and to perform bidirectional temporal coding on the global features and the local features to obtain bidirectional coded features; The encoding module is used to calculate the population momentum similarity matrix based on the bidirectional encoding features, and to superimpose the population momentum similarity matrix onto the self-attention score for encoding to obtain the population co-evolution features; The error calculation module is used to perform spatiotemporal encoding and decoding on the density-enhanced node features and calculate the topology-preserving reconstruction error. An abnormal behavior detection module is used to generate abnormal behavior detection results of the group based on the group cooperative evolution characteristics, the topology preservation reconstruction error, and the prototype distance anomaly probability.

Images