A group behavior recognition method based on multi-scale feature extraction

Abstract

The application discloses a kind of group behavior recognition methods based on multiscale feature extraction, the sensor data of collection is preprocessed, and the individual refined feature is obtained by multiscale feature extraction module after preprocessed sensor data, then the individual refined feature is input into interactive relationship extraction module to capture the interactive relationship between individual, and the refined feature vector is obtained;Finally, the number of subgroups in refined feature vector is extracted by subgroup base prediction module, and the feature adjacency matrix of refined feature vector is constructed, input into graph clustering module to carry out subgroup division, and generate the global feature of subgroup, then the recognition result is obtained by group behavior classifier.The individual feature and relationship feature of sensor data are mined and extracted in the application, the generalization ability of model is enhanced, the robustness and accuracy of group behavior recognition are improved, and the effective identification of model to group behavior is realized.

Description

Technical Field

[0001] This application belongs to the field of behavior recognition technology, specifically relating to a group behavior recognition method based on multi-scale feature extraction. Background Technology

[0002] Group behavior recognition based on sensor data is a promising and highly regarded research area. This field aims to understand the overall behavioral patterns formed by the interactions between multiple individuals, and its findings have wide applications in urban planning, traffic management, sociological research, and many other fields. Recognizing group behavior in complex environments is a highly challenging task. This is mainly because group behavior is not a simple summation of individual behaviors, but requires comprehensive analysis of individual behaviors and the complex interactions between individuals to achieve bottom-up inference of behavioral patterns.

[0003] In recent years, with the rapid development of IoT technology and the widespread adoption of wireless sensor networks and wearable devices, the extensive application of various sensors has provided a rich source of data for group behavior recognition. Simultaneously, the widespread application of machine learning and deep learning technologies has significantly promoted research on group behavior recognition based on sensor data. However, most current research on group behavior still focuses on the fields of vision and image processing. In contrast, group behavior recognition based on sensor data has advantages such as low cost, lack of geographical limitations, and strong privacy protection. Today, smart terminal devices commonly integrate various sensor modules, such as accelerometers, magnetometers, gyroscopes, and GPS, providing reliable technical support and practical feasibility for using smart terminals for group behavior recognition.

[0004] Despite the widespread application of various sensor technologies and the introduction of deep learning methods providing abundant data resources and theoretical foundations for group behavior recognition research, many challenges remain to be addressed in this field. First, feature extraction of group behavior remains a complex problem. Existing feature extraction methods typically extract individual features and relationships between individuals at only one scale. How to mine and extract individual and relationship features at different scales, and then design effective model structures to improve the performance of group behavior recognition, is a problem that urgently needs to be explored. Second, large-scale groups usually contain several subgroups and interfering individuals. How to reasonably divide subgroups and exclude interfering individuals to help the model understand group behavior in large-scale scenarios remains a challenging task. Summary of the Invention

[0005] The purpose of this application is to provide a group behavior recognition method based on multi-scale feature extraction, so as to improve the robustness and accuracy of group behavior recognition.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows:

[0007] A group behavior recognition method based on multi-scale feature extraction includes:

[0008] A recognition network model is constructed and trained, which includes a multi-scale feature extraction module, an interaction relationship extraction module, a subgroup cardinality prediction module, a graph clustering module, and a group behavior classifier;

[0009] The collected sensor data is preprocessed, and the preprocessed sensor data is then processed by a multi-scale feature extraction module to obtain individual refined features.

[0010] The refined individual features are input into the interaction relationship extraction module to capture the interaction relationships between individuals, resulting in a refined feature vector.

[0011] The subgroup cardinality prediction module extracts the number of subgroups in the refined feature vector and constructs the feature adjacency matrix of the refined feature vector. This matrix is ​​then input into the graph clustering module for subgroup division and generates global features of the subgroups. Finally, the recognition result is obtained through the group behavior classifier.

[0012] The multi-scale feature extraction module performs the following operations:

[0013] Upsample the preprocessed sensor data into the channel;

[0014] The upsampled sensor data is uniformly divided along the channel dimension according to a preset scale to obtain channel features;

[0015] Scale features are extracted for each channel feature to obtain the scale features corresponding to each channel feature, and then the features are concatenated to obtain the target scale features.

[0016] The target scale features are input into an efficient channel attention network to obtain individual refined features.

[0017] Furthermore, the number of channel features is four, and the extraction of features at different scales for each channel feature, followed by concatenation, yields the target scale feature, including:

[0018] The first channel feature is directly used as part of the target scale feature via a far-hop link;

[0019] The second channel feature is processed through a general convolution operation and a linear layer to extract the hidden feature of the second channel feature. The first half of the hidden feature is merged with the third channel feature after convolution, and the information of the other half is used as part of the target scale feature.

[0020] The third channel feature undergoes a convolution operation and is merged with the first half of the hidden feature of the second channel feature. It is then further extracted through convolutional and linear layers to obtain the hidden feature of the third channel feature. The first half of the hidden feature of the third channel feature is merged with the next channel feature after convolution, and the second half of the dimension is part of the target scale feature.

[0021] The fourth channel feature undergoes a convolution operation and is merged with the first half of the hidden feature of the third channel feature. It is then further extracted through convolutional and linear layers to obtain the fourth channel feature hidden feature, which is used as part of the target scale feature.

[0022] Finally, the target scale features are obtained by splicing them together.

[0023] Furthermore, the interaction relationship extraction module performs the following operations:

[0024] The refined features of each individual are evenly divided along the channel dimension according to a preset scale number to obtain scale features;

[0025] Using individuals as nodes and similarity as edge weights, a graph structure is constructed for each scale feature;

[0026] The graph structure is input into the Graph Transformer module to update node features, and then input into the feedforward neural network for nonlinear transformation to obtain scale-enhanced features;

[0027] The scale-enhanced features are recombined to obtain individual-enhanced features;

[0028] The Transformer module further extracts the deep relationships between features at different scales from the individual enhanced feature input map, and then performs normalization processing to obtain a refined feature vector.

[0029] Furthermore, the subgroup cardinality prediction module performs the following operations:

[0030] The input features first undergo a linear transformation, then the ReLU activation function is used to increase the model's expressive power, and finally another linear transformation is performed to output the predicted number of subgroups.

[0031] Furthermore, the graph clustering module performs the following operations:

[0032] Based on the identity matrix and the feature adjacency matrix, a Laplacian matrix is ​​constructed, and then eigenvalue decomposition is performed to obtain the node embedding matrix;

[0033] Clustering algorithms are applied to the individuals in the node embedding matrix to complete the subgroup division;

[0034] The refined feature vectors of each subgroup are aggregated using average pooling to obtain the global feature vector of each subgroup.

[0035] Furthermore, the loss functions used in training the recognition network model include: subgroup cardinality loss function, subgroup member loss function, and group behavior recognition loss function.

[0036] This application proposes a group behavior recognition method based on multi-scale feature extraction. In the process of group behavior recognition, a multi-scale feature extraction module is used to extract and expand individual features in sensor data to obtain individual behavior information. The interaction relationship extraction module is used to fully explore the relationship features between individuals. Combined with the subgroup partitioning task, potential behavior patterns in sensor data are mined, which enhances the generalization ability of the model and improves the robustness and accuracy of group behavior recognition, thus realizing the effective recognition of group behavior by the model. Attached Figure Description

[0037] Figure 1 This is a flowchart of the group behavior recognition method based on multi-scale feature extraction proposed in this application.

[0038] Figure 2 This is a schematic diagram of the network model structure for identification in this application.

[0039] Figure 3 This is a schematic diagram of the structure of the multi-scale feature extraction module in an embodiment of this application.

[0040] Figure 4 This is a schematic diagram of the interaction relationship extraction module in an embodiment of this application. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0043] One embodiment of this application, such as Figure 1 As shown, a group behavior recognition method based on multi-scale feature extraction is provided, including:

[0044] Step S1: Construct and train the recognition network model, which includes a multi-scale feature extraction module, an interaction relationship extraction module, a subgroup cardinality prediction module, a graph clustering module, and a group behavior classifier.

[0045] The recognition network model constructed in this embodiment is as follows: Figure 2 As shown, it includes: a multi-scale feature extraction module, an interaction relationship extraction module, a subgroup cardinality prediction module, a graph clustering module, and a group behavior classifier.

[0046] Step S2: Preprocess the collected sensor data. The preprocessed sensor data is then processed by a multi-scale feature extraction module to obtain individual refined features.

[0047] The embodiment uses a sliding window segmentation method to preprocess the collected sensor data, dividing the longer time series samples into multiple shorter time series segments.

[0048] In this embodiment, the sensors are worn on the wrist of the experimental subject. The sensors include an accelerometer and a gyroscope, which can record acceleration and angular velocity data along the x, y, and z axes, respectively. This embodiment selects 50 Hz as the data acquisition frequency, acquiring a complete set of data every 4 seconds. The acquired data is processed using a sliding window segmentation method. The sliding window length is set to 200, with an overlap rate of 50%. Starting from the beginning of the time series, the sliding window gradually slides backward, generating a subsequence of length 200 each time, until the long-time time series sample is divided into multiple short-time series segments, each short-time series segment being counted as a data point X.

[0049] The preprocessed sensor data is input into the multi-scale feature extraction module to obtain individual refined features. The multi-scale feature extraction module mainly includes channel upsampling, channel partitioning, multi-scale convolution, and the efficient channel attention network ECANet.

[0050] In this embodiment, the multi-scale feature extraction module is as follows: Figure 3 As shown, perform the following operations:

[0051] Step 2.1: Upsample the preprocessed sensor data by channel.

[0052] To apply 2D convolution to sensor data, the sensor input data is first upsampled in the channel dimension, expanding the (1,T,C0)-dimensional input data X into (H,W,C)-dimensional data X′, where T is the time span, C0 is the initial number of channels, C is the number of channels after expansion, and H and W are the height and width of the sensor data after expansion into two dimensions. Channel upsampling enhances the model's feature representation ability and enables the embedding of low-dimensional features. The formula can be expressed as:

[0053]

[0054] UpSampling represents the upsampling operation.

[0055] Step 2.2: Divide the upsampled sensor data into channels at a preset scale to obtain channel features.

[0056] In this embodiment, to facilitate subsequent multi-scale feature extraction, the number of scales to be divided, n, is first preset. This means that the upsampled sensor data is uniformly divided into n parts according to the number of scales n along the channel dimension. The formula can be expressed as:

[0057]

[0058] ChannelSplit represents the channel splitting operation. In this embodiment, the number of scales n is 4, and the value of i can be 0, 1, 2 or 3. The initial number of channels for each scale is 8.

[0059] Step 2.3: Extract scale features for each channel feature separately to obtain the scale features corresponding to each channel feature, and then concatenate them to obtain the target scale features.

[0060] Features of each channel Scale features are extracted sequentially, s i Represents scale information.

[0061] Taking four channel features as an example, such as Figure 3 As shown. First channel feature Without any processing, it is directly used as part of the target scale feature Y via a far-skip link. The far jump link operation solves the gradient vanishing problem in model training, allowing models to have deeper network structures, retaining important information in the initial features, and accelerating the convergence speed of the model.

[0062] The second channel feature will undergo a general convolution operation and pass through a linear layer. Thus, the hidden features of the second channel feature are extracted. The first half of the dimension is merged with the third channel feature after convolution, and the information of the other half of the dimension is used as part of the target scale feature Y. In this process, convolution operations are used to extract hidden information from the features of the current channel, and linear layers... Essentially, it is a weight matrix that is updated with gradients according to the target loss function during backpropagation, automatically adjusting its parameters to select the feature dimensions that contribute strongly to the next channel feature and update the weights of each dimension in the current channel feature.

[0063] Third channel feature After one convolution operation, and with The first half of the dimensions are merged, and then passed through a convolutional neural network and a linear layer. Further extraction yields hidden features. Hidden features The first half of the dimension is merged with the feature of the next channel after convolution, and the second half of the dimension is a part of the target scale feature Y. In this process, the linear layer It will automatically adjust its parameters to select feature dimensions that contribute significantly to the information of the next channel.

[0064] Fourth channel features After one convolution operation, and with the hidden features of the third channel, the feature is... The first half of the dimension is merged, and then further extracted through convolutional and linear layers to obtain the hidden features of the fourth channel. As part of the target scale features

[0065] The above and That is, the scale features corresponding to the features of each channel.

[0066] It should be noted that if the number of scales is larger, the steps for subsequent channels are similar to those for the third channel feature. In the last channel feature, the hidden features... Instead of segmenting, the entire feature is treated as part of the target scale feature Y. The process of scale feature extraction can be represented by the following formula:

[0067]

[0068]

[0069] Here, Conv is a 2D convolution, and the kernel size is set to 1. and To extract the first and second halves of a tensor's dimension, Concat represents a tensor concatenation operation. This is a linear layer, where Y represents the target scale feature. The target scale feature Y contains features of different scales, enhancing the expressive power of the features.

[0070] It should be noted that in the multi-scale feature extraction module, the branches corresponding to each channel feature will select convolutional kernels of different sizes. The first part of the feature is directly used as part of the target feature. Each subsequent feature will pass through a convolutional layer, with the kernel size increasing sequentially, resulting in features with receptive fields of different sizes, that is, features of different scales.

[0071] Step 2.4: Input the target scale features into an efficient channel attention network to obtain individual refined features.

[0072] The target scale feature Y is then input into the efficient channel attention network ECANet to refine the channel features. The main process of ECANet is as follows: First, global average pooling is performed on each channel to generate a channel description vector h. t This process compresses the spatial dimension, retaining only channel information. (h) t A one-dimensional convolutional layer is input to capture local interactions between channels. Adaptive selection of the kernel size k ensures a moderate range of local interactions; in this embodiment, a kernel size of 3 is chosen. The output of the one-dimensional convolution is then mapped to the [0,1] interval using a sigmoid activation function to generate channel attention weights. These attention weights are applied to the target-scale feature Y to highlight important channels and suppress irrelevant channels. The ECANet formula can be expressed as:

[0073] h t =AvgPooling(Y) (5)

[0074]

[0075]

[0076] Where AvgPooling is the average pooling operation. Conv1D is a 1D convolutional neural network with kernel size j. σ is the sigmoid function. The individual refined feature h is the final output of the multi-scale feature extraction module.

[0077] Step S3: Input the refined individual features into the interaction relationship extraction module to capture the interaction relationships between individuals and obtain the refined feature vector.

[0078] Information at different scales may have complex interdependencies, and analyzing them as a whole can easily dilute important information. Therefore, in this step, such as... Figure 4 As shown, the refined features of multiple input individuals are processed by the interaction relationship extraction module to extract the interaction relationships, thereby enabling the model to more accurately capture features at a specific scale, enhance the model's flexibility and controllability, and improve its interpretability.

[0079] The interaction relationship extraction module in this embodiment performs the following operations:

[0080] Step 3.1: Divide the refined features of each individual into segments along the channel dimension according to a preset scale number to obtain scale features.

[0081] This step refines the individual features of the input data by dividing them into n parts along the channel dimension, thus obtaining scale features. Each feature In this context, m represents the individual ID, the total number of individuals is N, and s i Represents scale information, scale features Channel dimension and The channel dimension is consistent with, and is consistent with The scale information represented by s i This is because ECANet's localization operation when calculating channel attention weights ensures that each scale of information can be represented independently on the channel, without being fused with features from other scales, and is merely used as a way to update weights.

[0082] Step 3.2: Using individuals as nodes and similarity as the edge weights, construct a graph structure for each scale feature.

[0083] To model the relationships between individuals and extract higher-level group features, this step constructs a graph-based network structure, where the nodes of the graph represent individuals in the group, and the node features are... Edges represent the relationships between individuals. For example... Figure 4 As shown, for each scale of features, a corresponding graph structure is constructed by calculating similarity based on individual relationship features. in It is a set of nodes, where each node corresponds to an individual in the group; It is a set of edges used to represent the dependencies between individuals. The weights of the edges are obtained by calculating the similarity between features of individuals at the same scale. The calculation formula is expressed as follows:

[0084]

[0085] Where ∈ is a positive number greater than 0, used to avoid division by zero errors in similarity calculation. These represent the features of different individuals within the scale feature. Using the above method, a weighted undirected graph is constructed. in It is the edge weight matrix.

[0086] Step 3.3: Input the graph structure into the Graph Transformer module to update the node features, and then input it into the feedforward neural network for nonlinear transformation to obtain scale-enhanced features.

[0087] For each completed graph at different scales The node features are updated using two-layer graph Transformer modules at different scales. Each graph Transformer includes a multi-head attention mechanism to enhance the model's feature capture capability. Multi-head attention processes the input node features through multiple parallel attention heads, concatenates them, and maps them to the output space, thereby capturing different channel dependency patterns. Subsequently, the node features updated by the graph attention module are input into a feedforward neural network (FFN) for nonlinear transformation to obtain scale-enhanced features.

[0088] Step 3.4: Recombine the scale-enhanced features to obtain individual-enhanced features.

[0089] For each scale, after processing with Graph Transformer and FFN, the updated scale-enhanced features are output, where the features of each individual are represented as follows: p′ mi Recombining individuals into individual enhanced features, using p′ m express.

[0090] Step 3.5: The Transformer module further extracts the deep relationships between features at different scales from the individual enhanced feature input map, and then performs normalization processing to obtain a refined feature vector.

[0091] Individual augmentation feature p′ m The global graph Transformer module is used to further extract deep relationships between features at different scales, followed by LayerNorm normalization to obtain the final refined feature vector Y of the model. norm (Also known as normalized features). The LayerNorm normalization operation reduces the risk of gradient explosion or vanishing during training and accelerates model convergence by standardizing the feature distribution.

[0092] The interaction relationship extraction module enables efficient fusion of features across multiple scales and fully captures the interaction relationships between individuals, thereby improving the model's feature representation capabilities and adaptability to complex scenarios.

[0093] It should be noted that the graph Transformer model, feedforward neural network, and LayerNorm normalization operation are all relatively mature technologies in this field, and will not be elaborated on here.

[0094] Step S4: Extract the number of subgroups in the refined feature vector through the subgroup cardinality prediction module, construct the feature adjacency matrix of the refined feature vector, input it into the graph clustering module for subgroup division, generate the global features of the subgroups, and then obtain the recognition result through the group behavior classifier.

[0095] Refined feature vector Y norm Individual and relational features have been fully extracted from the input features. This step utilizes the refined feature vector Y. norm A feature adjacency matrix is ​​constructed, the number of subgroups is determined by the subgroup cardinality prediction module, the subgroups are partitioned using the graph clustering module, and finally the group behavior recognition result is obtained through the group behavior classifier.

[0096] First, the eigenvector Y is refined using a group. norm Construct a feature adjacency matrix A. The purpose of constructing the feature adjacency matrix is ​​to express the relationships between individuals in the population. If the feature of each individual in the refined feature vector of the population is p′ m In the characteristic adjacency matrix, a ij This represents the strength of the relationship between the i-th and j-th individuals. The formula for calculating the relationship strength is as follows:

[0097]

[0098] To improve numerical stability and accommodate subsequent graph operations, the adjacency matrix is ​​normalized to obtain a symmetric normalized adjacency matrix. The calculation formula is as follows:

[0099]

[0100] Where D is the degree matrix, D ii =∑ j a ij A normalized adjacency matrix is ​​used to represent the global relationships between individuals in a group.

[0101] This embodiment also relies on the group refinement feature vector Y norm The subpopulation cardinality prediction module estimates the number of subpopulations K in the input population. In this module, the input features first undergo a linear transformation, then the ReLU activation function is used to enhance the model's expressive power, and finally, another linear transformation is performed to output the predicted subpopulation number K. The calculation formula is as follows:

[0102] K = σ(W2·ReLU(W1·Y) norm +b1)+b2) (11)

[0103] Where W1 and W2 are weight matrices, b1 and b2 are bias terms, and σ is the sigmoid activation function.

[0104] Subsequently, based on the predicted number of subgroups K and the adjacency matrix Perform graph clustering to divide the samples into subgroups and obtain the global features of the subgroups.

[0105] In one specific embodiment, the graph clustering module performs the following operations:

[0106] Step 4.1: Construct the Laplacian matrix based on the identity matrix and the feature adjacency matrix, and then perform eigenvalue decomposition to obtain the node embedding matrix.

[0107] In this embodiment, graph clustering models the graph structure of the population using the Laplacian matrix L. By performing eigenvalue decomposition on the Laplacian matrix, the top K eigenvectors are obtained, which constitute a low-dimensional node embedding matrix. This can be expressed as:

[0108]

[0109] U = eig(L,K) (13)

[0110] Where U is the node embedding matrix composed of eigenvectors, I is the identity matrix, and eig represents the eigenvalue decomposition of the matrix, which calculates the eigenvalues ​​and eigenvectors of the matrix.

[0111] Step 4.2: Apply a clustering algorithm to the individuals in the node embedding matrix to complete the subgroup division.

[0112] Then, the K-Means clustering algorithm is applied to the points (i.e. individuals) in the node embedding matrix to assign each individual to one of the K subgroups, thus completing the subgroup division.

[0113] Step 4.3: Aggregate the individual refined feature vectors of each subgroup through average pooling to obtain the global feature vector of each subgroup.

[0114] After subgrouping, the network further integrates the results to generate global feature representations for each subgroup. Individual features within each subgroup are aggregated using average pooling to obtain a global feature vector for each subgroup. This subgroup feature vector summarizes the behavioral patterns or characteristics of the entire subgroup. The formula is expressed as:

[0115]

[0116] Where c i is the subgroup label to which the i-th individual belongs, and k represents the index of the subgroup. Let be the refined feature vector of the i-th individual.

[0117] Finally, the global feature vectors of the subgroups are fed into a group behavior classifier, which uses a fully connected layer plus a softmax structure to identify and classify the behavior of each subgroup. The classification result is the probability distribution of each subgroup on the behavior category.

[0118] In another embodiment of this application, the loss function in training the recognition network model consists of two parts. Since subgrouping helps the network learn the interaction characteristics between individuals at the spatial and relational levels, individuals belonging to the same subgroup often have similar behavioral patterns or interaction methods. This attribution information can supplement the implicit semantics in behavior recognition. Therefore, the model will simultaneously learn the intrinsic patterns of subgrouping and group behavior, using the features of the subgrouping task as a shared representation for the group behavior recognition task, allowing the model to focus on more representative local relationships, thereby providing more focused contextual information for group behavior recognition. The subgrouping task predicts the subgroup cardinality of the group and, based on the subgroup cardinality, performs graph clustering on the hidden features of individuals to obtain the final classification result. The loss function for training the subgrouping task is defined as follows:

[0119]

[0120]

[0121] in, The subgroup cardinality loss function is used, and mean squared error is selected. K is the predicted subgroup cardinality for the i-th sample. i It is the true subgroup cardinality of the i-th sample. The subgroup membership loss function is binary cross-entropy loss. It is the predicted membership relation, with values ​​ranging from [0,1], M ij It represents the actual membership relationship, with a value of 0 or 1, and N is the total number of samples.

[0122] In this embodiment, a classifier is used to perform the group behavior recognition task. The classifier consists of a single fully connected neural network and a softmax activation function. The loss function for group behavior recognition in this embodiment uses cross-entropy loss to measure the difference between the predicted group behavior category distribution and the true category distribution. The loss function can be defined as follows:

[0123]

[0124] in, P is the predicted probability of the i-th sample in class c. ic It represents the true distribution of the i-th sample in the c-th class, where C is the number of behavior categories and N is the total number of samples.

[0125] The overall loss function in this embodiment This can be expressed as the following formula:

[0126]

[0127] Here, λ1 and λ2 represent weight coefficients, which can control the contribution of the three types of loss to model training, thereby optimizing the overall performance of the model.

[0128] In this application, the technical solution was also experimentally verified. The experiments were based on the UT-Group dataset modified from UT-Data and the self-created WBSensor dataset, and compared with current mainstream subgroup partitioning algorithms and group behavior recognition algorithms. The results of the subgroup partitioning task were measured using F1 score, precision, recall, mean average precision (mAP), and IOU-AOC. The results of the group behavior recognition task were measured using F1 score, precision, recall, and accuracy. The experimental results are shown in Tables 1 and 2. Table 1 shows the experimental results for subgroup partitioning, and Table 2 shows the experimental results for group behavior recognition.

[0129] Table 1

[0130]

[0131] Table 2

[0132]

[0133] The experimental results of the proposed method are divided into two parts, evaluating the model's performance on two tasks: subgroup partitioning and group behavior recognition. In the subgroup partitioning task, the proposed method achieved an F1 score of 58.40%, precision of 55.30%, recall of 62.00%, mean precision of 77.60%, and IOU-AOC of 38.30% on the UT-Group dataset. On the WBSensor dataset, the F1 score was 64.16%, precision of 62.12%, recall of 66.35%, mean precision of 81.67%, and IOU-AOC of 40.77%. The proposed method outperforms other methods on all metrics. In the task of group behavior recognition, the proposed method achieves an F1 score of 84.44%, precision of 86.12%, recall of 82.83%, and accuracy of 84.52% on the UT-Group dataset; and an F1 score of 88.37%, precision of 89.21%, recall of 87.56%, and accuracy of 93.86% on the WBSensor dataset. The proposed method also outperforms other methods across all metrics for this task.

[0134] In summary, the method of this invention has certain advantages over other algorithms. This method can more comprehensively understand the group behavior reflected in sensor data and efficiently extract group features from sensor data, thereby significantly improving the accuracy and robustness of group behavior recognition.

[0135] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for identifying group behavior based on multi-scale feature extraction, characterized in that, The group behavior recognition method based on multi-scale feature extraction includes: A recognition network model is constructed and trained, which includes a multi-scale feature extraction module, an interaction relationship extraction module, a subgroup cardinality prediction module, a graph clustering module, and a group behavior classifier; The collected sensor data is preprocessed, and the preprocessed sensor data is then processed by a multi-scale feature extraction module to obtain individual refined features. The refined individual features are input into the interaction relationship extraction module to capture the interaction relationships between individuals, resulting in a refined feature vector. The subgroup cardinality prediction module extracts the number of subgroups in the refined feature vector and constructs the feature adjacency matrix of the refined feature vector. This matrix is ​​then input into the graph clustering module for subgroup division and generates global features of the subgroups. Finally, the recognition result is obtained through the group behavior classifier. The multi-scale feature extraction module performs the following operations: Upsample the preprocessed sensor data by channel, reducing the dimension of the preprocessed sensor data from (1) ,T,C 0 ) expands to ( H,W,C ),in T For the time span, C 0 This is the initial number of channels. C To expand the number of channels, H,W The height and width after expansion; The upsampled sensor data is uniformly divided along the channel dimension according to a preset scale to obtain channel features; Scale features are extracted for each channel feature to obtain the scale features corresponding to each channel feature, and then the features are concatenated to obtain the target scale features. The target scale features are input into an efficient channel attention network to obtain individual refined features; The number of channel features is four. The process of extracting features at different scales for each channel feature and then concatenating them to obtain the target scale feature includes: The first channel feature is directly used as part of the target scale feature via a far-hop link; The second channel feature is processed through a general convolution operation and a linear layer to extract the hidden feature of the second channel feature. The first half of the hidden feature is merged with the third channel feature after convolution, and the information of the other half is used as part of the target scale feature. The third channel feature undergoes a convolution operation and is merged with the first half of the hidden feature of the second channel feature. It is then further extracted through convolutional and linear layers to obtain the hidden feature of the third channel feature. The first half of the hidden feature of the third channel feature is merged with the next channel feature after convolution, and the second half of the dimension is part of the target scale feature. The fourth channel feature undergoes a convolution operation and is merged with the first half of the hidden feature of the third channel feature. It is then further extracted through convolutional and linear layers to obtain the fourth channel feature hidden feature, which is used as part of the target scale feature. Finally, the target scale features are obtained by splicing them together. The interaction relationship extraction module performs the following operations: The refined features of each individual are segmented along the channel dimension according to a preset number of scales to obtain scale features; Using individuals as nodes and similarity as edge weights, a graph structure is constructed for each scale feature; The graph structure is input into the Graph Transformer module to update node features, and then input into the feedforward neural network for nonlinear transformation to obtain scale-enhanced features; The scale-enhanced features are recombined to obtain individual-enhanced features; The Transformer module further extracts the deep relationships between features at different scales from the individual enhanced feature input map, and then performs normalization processing to obtain a refined feature vector.

2. The group behavior recognition method based on multi-scale feature extraction according to claim 1, characterized in that, The subgroup cardinality prediction module performs the following operations: The input features first undergo a linear transformation, then the ReLU activation function is used to increase the model's expressive power, and finally another linear transformation is performed to output the predicted number of subgroups.

3. The group behavior recognition method based on multi-scale feature extraction according to claim 1, characterized in that, The graph clustering module performs the following operations: Based on the identity matrix and the feature adjacency matrix, a Laplacian matrix is ​​constructed, and then eigenvalue decomposition is performed to obtain the node embedding matrix; Clustering algorithms are applied to the individuals in the node embedding matrix to complete the subgroup division; The refined feature vectors of each subgroup are aggregated using average pooling to obtain the global feature vector of each subgroup.

4. The group behavior recognition method based on multi-scale feature extraction according to claim 1, characterized in that, The loss functions used in training the recognition network model include: subgroup cardinality loss function, subgroup member loss function, and group behavior recognition loss function.

Images