A multi-level behavior recognition method for dense crowd scenes

Abstract

The present invention discloses a method for multi-level behavior recognition in dense crowd scenes, comprising the following steps: step s1, extracting corresponding feature vectors in the scene through a convolutional neural network; step s2, constructing an initial basic graph neural network according to the corresponding feature vectors in the scene; step s3, calculating the feature vectors of the basic graph neural network to obtain an image similarity matrix E; step s4, the basic graph neural network calculating the spatial distance matrix D between two pedestrians; step s5, the basic graph neural network obtaining a relationship matrix R according to the similarity matrix E and the spatial distance matrix D; step s6, the basic neural network using the relationship matrix R to update the feature vectors in the scene; step s7, the layered graph network performing two aggregations on the updated feature vectors to obtain pedestrian feature n I , grouping feature n P , global feature n G ; Step s8, the multi-layer perceptron network obtains the action category prediction result; This method realizes the one-time recognition of multi-level action information in the scene, and improves the efficiency of behavior recognition.

Description

Technical field:

[0001] The present invention belongs to the fields of artificial intelligence, computer vision, and design behavior recognition technology, and in particular relates to a method for multi-level behavior recognition in dense crowd scenes. Background technology:

[0002] The goal of this paper is to identify the behavior of densely populated crowds in surveillance scenarios at several levels: identifying the action categories of individual pedestrians, identifying the clustering and grouping of pedestrians, identifying the action categories of clustered groups, and identifying the overall action category of the scene. Previous methods typically solve these four tasks separately. This invention aims to solve all four tasks simultaneously using a unified architecture.

[0003] The background technologies involved in the present invention include:

[0004] (1) Feature extraction network [1]: The appearance features of the target are a very effective feature and have been widely used in behavior recognition tasks. Currently, commonly used appearance features usually include artificial features and features extracted by deep learning networks. However, artificial features often have limitations. This paper uses an end-to-end deep learning network to extract the features of the target box. By inputting a large amount of training data sets, the feature extraction capability of the model is improved.

[0005] (2) Graph Neural Networks[2]: Graph neural networks build a graph based on the feature information of multiple instances, using instance features as nodes and relationships between instances as edges. The graph node features are updated and optimized in a specific way. This paper uses a basic graph neural network that integrates spatial distance information to update and optimize the visual features of pedestrians, and uses a hierarchical graph neural network structure to hierarchically aggregate visual feature information, thereby optimizing the network's information extraction capabilities.

[0006] (3) Motion analysis[3]: Existing motion analysis problems include individual behavior recognition, group behavior recognition, and social group detection. These methods are usually independent of each other, with one method often focusing on solving a specific motion analysis problem. The present invention uses a unified architecture to solve multiple motion analysis problems simultaneously, greatly improving computational efficiency.

[0007] (4) Spatial distance measurement method: Using information from a two-dimensional image to measure the spatial distance between people in an actual three-dimensional scene is an important basis for determining the grouping and aggregation of pedestrians. Existing methods usually use the Euclidean distance between the center points of the detection frame to represent this distance, but this calculation method is not applicable to scenes with large depth of field and wide field of view. The present invention uses an area-normalized spatial distance measurement method to significantly improve the accuracy of spatial distance measurement, thereby greatly improving the accuracy of social group detection. Summary of the invention:

[0008] To address the challenges of the prior art, the present invention provides a method for multi-level behavior recognition in dense crowd scenes. This method can simultaneously identify individual pedestrian action categories, pedestrian clustering and grouping, the action categories of pedestrian groups, and the action categories of the entire scene. This method utilizes a unified network architecture to achieve one-time recognition of multi-level action information in the scene, significantly reducing repeated computations, improving behavior recognition efficiency, and achieving excellent recognition accuracy. The present invention utilizes computer vision methods to analyze the action categories and clustering of pedestrians in dense crowd scenes, playing an important role in many applications. This invention addresses the problem that traditional behavior recognition tasks often focus on identifying the actions of a single pedestrian, the interactive behaviors of a small number of pedestrians, or the collective behaviors of a group of pedestrians. However, existing behavior recognition tasks often focus on only one of these tasks and fail to simultaneously address all of these. The present invention proposes a new multi-level behavior recognition task for multi-person scenes. Using a unified end-to-end network, this method performs full-scene, multi-level behavior recognition in dense crowd scenes. This method simultaneously identifies individual pedestrian action categories, their social groupings, the social behavior categories of pedestrian groups, and the activity categories of the entire scene.

[0009] In order to solve the existing technology, the present invention adopts the following technical solutions:

[0010] 1. A method for multi-level behavior recognition in dense crowd scenes, based on convolutional neural networks, basic graph neural networks, layered graph neural networks, and multi-layer perceptron networks; comprising the following steps:

[0011] Step s1: extract the feature vectors b1, b2, ..., b corresponding to the pedestrian target detection box in the scene through the convolutional neural network. n ;

[0012] Step s2, constructing an initial basic graph neural network based on the corresponding feature vectors in the scene;

[0013] Step s3, calculating the similarity between the two feature vectors of the basic graph neural network to obtain the target similarity matrix E;

[0014] Step s4, the basic graph neural network uses an area-normalized spatial distance measurement method to calculate the spatial distance matrix D between each pair of pedestrians;

[0015] Step s5, the basic graph neural network obtains the relationship matrix R according to the similarity matrix E and the spatial distance matrix D;

[0016] Step s6, the basic neural network uses the relationship matrix R to update the feature vector in the scene;

[0017] Step s7, the layered graph network aggregates the updated feature vector twice to obtain the pedestrian feature n I , grouping feature n P , global feature n G ;

[0018] Step s8: The multi-layer perceptron network obtains the action category prediction result according to the following formula:

[0019]

[0020]

[0021] a G =F g (n G )

[0022] Among them: a I is the result of single behavior recognition, a P is the behavior recognition result of the group, a G is the overall behavior recognition result of the scene. Further, the similarity between the feature vectors of the basic graph neural network is calculated to obtain the image similarity matrix E. The process is as follows:

[0023] Build a graph G = (N, E) for the pedestrians in the scene; where N represents the node information along the way and E represents the edge information in the graph; that is, the node similarity matrix.

[0024] The node N of the graph is initialized to the deep features extracted in step (1). For nodes u and v, the calculation formula of the corresponding graph edge information is:

[0025] e u,v =F1(f u )F2(f v ) T ,u,v∈N

[0026] Among them, F1 and F2 are multi-layer perceptron network structures, f u 、f v Represent the features corresponding to node u and node v respectively. Side information e u,v Then it represents the similarity between nodes u and v. By calculating the similarity between any two nodes, we can get the similarity matrix E of the entire graph.

[0027] Furthermore, the basic neural network uses the relationship matrix R to update the feature vector process in the scene:

[0028] According to the initialized graph network and the calculated similarity matrix, the node information of the graph network is updated. The specific steps are as follows:

[0029] Use the following formula to update graph node information:

[0030]

[0031] Among them F n Update the network for a node consisting of fully connected layers, f u , f' u They are the node features before and after the update, node v represents the node connected to node u, and e u,v Calculated in the above steps;

[0032] Aggregate the node features before and after the update. The specific formula is:

[0033]

[0034] Can get is the final pedestrian feature, the subscript u indicates that the pedestrian number corresponding to the feature is u, and the operator Represents the bitwise addition of elements in a eigenvector.

[0035] Furthermore, the basic graph neural network uses an area-normalized spatial distance measurement method to calculate the spatial distance matrix D between two pedestrians;

[0036] The initial spatial distance between pedestrians is calculated using the area-normalized spatial distance measurement method. The formula is as follows:

[0037]

[0038] Where: x u , x v is the horizontal coordinate of the midpoint of the lower edge of the detection frame, y u ,y v is the vertical coordinate of the midpoint of the lower edge of the detection frame; S u 、S v is the area of ​​the detection box;

[0039] The spatial distance mask matrix is ​​calculated using the following formula:

[0040]

[0041] Where: ρ is a pre-set hyperparameter.

[0042] Furthermore, the basic graph neural network obtains the relationship matrix R according to the similarity matrix E and the spatial distance matrix D according to the following formula:

[0043]

[0044] Among them: Calculated from the distance matrix D as described in claim 3, λ is a pre-set hyperparameter used to balance the weight between the similarity matrix and the spatial distance matrix, and ⊙ represents the multiplication of corresponding elements of the matrix;

[0045] Furthermore, the layered graph network aggregates the updated feature vector twice to obtain the pedestrian feature n I , grouping feature n P , global feature n G process;

[0046] According to the relationship matrix R, a clustering algorithm is used to obtain pedestrian grouping prediction results;

[0047] Use the aggregation network to aggregate all pedestrian features n in each group I Aggregate according to the following formula to obtain the grouping feature n P :

[0048]

[0049] Where: The subscript k indicates that the group number corresponding to the feature is k, that is, the kth group G k , represents the corresponding features of all pedestrians belonging to this group, and AiO represents the feature aggregation operation. Indicates the grouping feature corresponding to the k-th group;

[0050] Use the aggregation network to group the features n P And the pedestrian features n that do not belong to any group I Aggregate according to the following formula to obtain the global feature n G :

[0051]

[0052] in: Indicates that u does not belong to any group, that is, the single person feature that does not participate in the group Indicates the grouping characteristics formed by all groups n G Represents the global features obtained by aggregation.

[0053] Beneficial effects:

[0054] The present invention first uses a convolutional neural network to extract features of pedestrians in a scene. A basic graph neural network that incorporates distance information is then used to update these features. The updated pedestrian features are then used to determine their action categories. A hierarchical graph network is then used to aggregate the pedestrian features twice. The first aggregated feature information is used to determine the sub-clustering of pedestrians and their grouped action categories. The second aggregated feature information is then used to determine the overall action category of the scene.

[0055] 1. This invention uses a unified network structure to simultaneously identify the individual action categories of pedestrians in a scene, the clustering and grouping of pedestrians, the action categories of pedestrian groups, and the action categories of the entire scene. This allows for a single computation to generate the output results for multiple tasks, significantly improving computational efficiency.

[0056] 2. The present invention solves multiple subtasks jointly, and each subtask benefits from each other, which to a certain extent increases the recognition ability of the model and improves the accuracy of each subtask.

[0057] 3. This invention combines appearance feature information with spatial distance information to characterize the distance between pedestrians in a scene, and uses this distance information to update the pedestrians' depth features. These two types of information complement each other, significantly improving the model's representational capabilities and, in turn, the accuracy of behavior recognition. Description of the drawings:

[0058] Figure 1 This is a flow chart of a method for multi-level behavior recognition in dense crowd scenes according to the present invention. Specific implementation method:

[0059] like Figure 1 As shown, the invention proposes a method for multi-level behavior recognition in dense crowd scenes. The method uses a convolutional neural network to extract the features of pedestrians in the scene and uses a basic graph neural network that integrates distance information to update the features. The pedestrian's action category can be obtained through the updated pedestrian features. A layered graph network is then used to aggregate the pedestrian features twice. The pedestrian's clustering and grouped action categories are obtained through the feature information after the first aggregation. The action category of the entire scene is obtained through the feature information after the second aggregation. Specifically, it includes the following contents:

[0060] A method for multi-level behavior recognition in dense crowd scenes, based on convolutional neural networks, basic graph neural networks, layered graph neural networks, and multi-layer perceptron networks, is characterized by comprising the following steps:

[0061] Step s1: extract the feature vectors b1, b2, ..., b corresponding to the pedestrian target detection box in the scene through the convolutional neural network. n ;

[0062] Step s2, constructing an initial basic graph neural network based on the corresponding feature vectors in the scene;

[0063] Step s3, calculating the similarity between the two feature vectors of the basic graph neural network to obtain the target similarity matrix E;

[0064] Step s4, the basic graph neural network uses an area-normalized spatial distance measurement method to calculate the spatial distance matrix D between each pair of pedestrians;

[0065] Step s5, the basic graph neural network obtains the relationship matrix R according to the similarity matrix E and the spatial distance matrix D;

[0066] Step s6, the basic neural network uses the relationship matrix R to update the feature vector in the scene;

[0067] Step s7, the layered graph network aggregates the updated feature vector twice to obtain the pedestrian feature n I , grouping feature n P , global feature n G ;

[0068] Step s8: The multi-layer perceptron network obtains the action category prediction result according to the following formula:

[0069]

[0070]

[0071] a G =F g (n G )

[0072] Among them: a I is the result of single behavior recognition, a P is the behavior recognition result of the group, a G is the overall behavior recognition result of the scene. Further, the similarity between the feature vectors of the basic graph neural network is calculated to obtain the image similarity matrix E. The process is as follows:

[0073] Build a graph G = (N, E) for the pedestrians in the scene; where N represents the node information along the way and E represents the edge information in the graph; that is, the node similarity matrix.

[0074] The node N of the graph is initialized to the deep features extracted in step (1). For nodes u and v, the calculation formula of the corresponding graph edge information is:

[0075] e u,v =F1(f u )F2(f v ) T ,u,v∈N

[0076] Among them, F1 and F2 are multi-layer perceptron network structures, f u 、f v Represent the features corresponding to node u and node v respectively. Side information e u,vThen it represents the similarity between nodes u and v. By calculating the similarity between any two nodes, we can get the similarity matrix E of the entire graph.

[0077] Furthermore, the basic neural network uses the relationship matrix R to update the feature vector process in the scene:

[0078] According to the initialized graph network and the calculated similarity matrix, the node information of the graph network is updated. The specific steps are as follows:

[0079] Use the following formula to update graph node information:

[0080]

[0081] Among them: F n Update the network for a node consisting of fully connected layers, f u , f' u They are the node features before and after the update, node v represents the node connected to node u, and e u,v Calculated in the above steps;

[0082] Aggregate the node features before and after the update. The specific formula is:

[0083]

[0084] Can get is the final pedestrian feature, the subscript u indicates that the pedestrian number corresponding to the feature is u, and the operator Represents the bitwise addition of elements in a eigenvector.

[0085] Furthermore, the basic graph neural network uses an area-normalized spatial distance measurement method to calculate the spatial distance matrix D between two pedestrians;

[0086] The initial spatial distance between pedestrians is calculated using the area-normalized spatial distance measurement method. The formula is as follows:

[0087]

[0088] Where: x u , x v is the horizontal coordinate of the midpoint of the lower edge of the detection frame, y u ,y v is the vertical coordinate of the midpoint of the lower edge of the detection frame; S u 、S v is the area of ​​the detection box;

[0089] The spatial distance mask matrix is ​​calculated using the following formula:

[0090]

[0091] Where: ρ is a pre-set hyperparameter.

[0092] Furthermore, the basic graph neural network obtains the relationship matrix R according to the similarity matrix E and the spatial distance matrix D according to the following formula:

[0093]

[0094] in: Calculated from the distance matrix D as described in claim 3, λ is a pre-set hyperparameter used to balance the weight between the similarity matrix and the spatial distance matrix, and ⊙ represents the multiplication of corresponding elements of the matrix;

[0095] Furthermore, the layered graph network aggregates the updated feature vector twice to obtain the pedestrian feature n I , grouping feature n P , global feature n G process;

[0096] According to the relationship matrix R, a clustering algorithm is used to obtain pedestrian grouping prediction results;

[0097] Use the aggregation network to aggregate all pedestrian features n in each group I Aggregate according to the following formula to obtain the grouping feature n P :

[0098]

[0099] Where: The subscript k indicates that the group number corresponding to the feature is k, that is, the kth group G k , represents the corresponding features of all pedestrians belonging to this group, and AiO represents the feature aggregation operation. Indicates the grouping feature corresponding to the k-th group;

[0100] Use the aggregation network to group the features n P And the pedestrian features n that do not belong to any group I Aggregate according to the following formula to obtain the global feature n G :

[0101]

[0102] in: Indicates that u does not belong to any group, that is, the single person feature that does not participate in the group Indicates the grouping characteristics formed by all groups n G Represents the global features obtained by aggregation.

[0103] 2. Implementation of multi-level behavior recognition method for multi-person scenarios.

[0104] (1) Construction of basic graph neural network and updating of node information

[0105] First, we build a basic graph neural network for pedestrian feature extraction and graph network initialization. The specific steps are as follows:

[0106] The deep convolutional neural network Inception-v3 is used to extract the global features of the image, and the feature extraction method RoIAlign[4] is used to extract the feature information of each pedestrian as the initial node representation of the basic graph neural network.

[0107] Construct a graph G = (N, E) for the pedestrians in the scene. N represents the node information along the way, and E represents the edge information in the graph. The initial graph structure is a fully connected graph. The node N of the graph is initialized to the deep features extracted in step (1). For nodes u and v, the calculation formula for the corresponding graph edge information is:

[0108] e u,v =F1(f u )F2(f v ) T ,u,v∈N

[0109] Among them: F1, F2 are multi-layer perceptron network structures, f u 、f v Represent the features corresponding to node u and node v respectively. Side information e u,v Then it represents the similarity between nodes u and v. By calculating the similarity between any two nodes, we can get the similarity matrix E of the entire graph.

[0110] According to the initialized graph network and the calculated similarity matrix, the node information of the graph network is updated. The specific steps are as follows:

[0111] Use the following formula to update graph node information:

[0112]

[0113] Among them: F n Update the network for a node consisting of fully connected layers, f u , f' u They are the node features before and after the update, node v represents the node connected to node u, and e u,v Calculated in the above steps;

[0114] Aggregate the node features before and after the update. The specific formula is:

[0115]

[0116] Can get is the final pedestrian feature, the subscript u indicates that the pedestrian number corresponding to the feature is u, and the operator Represents the bitwise addition of elements in a eigenvector.

[0117] (2) Construction of spatial distance matrix

[0118] Using the position information of the pedestrian detection frame, calculate the spatial distance between pedestrians, and construct the spatial distance matrix and spatial distance mask. The specific steps are as follows:

[0119] (1) The initial spatial distance between pedestrians is calculated using the area-normalized spatial distance measurement method. The formula is as follows:

[0120]

[0121] where x u , x v is the horizontal coordinate of the midpoint of the lower edge of the detection frame, y u ,y v is the vertical coordinate of the midpoint of the lower edge of the detection frame; S u 、S v is the area of ​​the detection box.

[0122] The spatial distance mask matrix is ​​calculated using the following formula

[0123]

[0124] Where: ρ is a pre-set hyperparameter.

[0125] (3) Calculate the relationship matrix R for node update

[0126] According to the similarity matrix E, spatial distance matrix D and spatial distance mask The final relationship matrix R between the graph nodes can be obtained, and the calculation formula is as follows:

[0127]

[0128] in: Calculated from the distance matrix D as described in claim 3, λ is a pre-set hyperparameter used to balance the weight between the similarity matrix and the spatial distance matrix, and ⊙ represents the multiplication of corresponding elements of the matrix;

[0129] After calculating the final relationship matrix R, the node information is aggregated twice using the hierarchical graph network to obtain the output results of each subtask. The specific steps are as follows:

[0130] (1) Based on the relationship matrix R obtained in step (3), a clustering algorithm is used to obtain pedestrian grouping prediction results;

[0131] (2) Use AllinOne aggregation network (AiO) to aggregate all pedestrian features n in each group I Perform aggregation to obtain grouping feature n P , the formula is as follows:

[0132]

[0133] Where: The subscript k indicates that the group number corresponding to the feature is k, that is, the kth group G k , represents the corresponding features of all pedestrians belonging to this group, and AiO represents the feature aggregation operation. Indicates the grouping feature corresponding to the k-th group;

[0134] (3) Use AiO network to group feature n P And the pedestrian features n that do not belong to any group I Perform aggregation to obtain global features n G , the formula is as follows:

[0135]

[0136] in: Indicates that u does not belong to any group, that is, the single person feature that does not participate in the group Indicates the grouping characteristics formed by all groups n G Represents the global features obtained by aggregation.

[0137] (3) According to the above steps, n is obtained I 、n P 、n G , use three multi-layer perceptron networks to obtain the output results of each subtask, where a I is the result of single behavior recognition, a P is the behavior recognition result of the group, a G is the overall behavior recognition result of the scene. The formula is as follows:

[0138]

[0139]

[0140] a G =F g (n G )

[0141] (5) Network loss function. The network uses the cross entropy loss function to constrain the training results and thus update the network. The overall loss function formula of the network is as follows:

[0142]

[0143] in They represent the loss functions of the four subtasks: individual behavior recognition, group social behavior recognition, overall activity type recognition, and group group detection.

[0144] The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solutions of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the scope of the present invention and the scope of protection of the claims, those skilled in the art may make various specific modifications based on the teachings of the present invention, all of which fall within the scope of protection of the present invention.

[0145] References

[0146] [1]Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking theinceptionarchitecture for computer vision. CVPR 2016.

[0147] [2] Zhao, J., Han, R., Gan, Y., Wan, L., Feng, W., Wang, S.: Human identification and interaction detection in cross-view multi-person videos with wearable cameras. ACM MM2020.

[0148] [3]Wu, J., Wang, L., Wang, L., Guo, J., Wu, G.: Learning actor relation graphs for group activity recognition. In: CVPR 2019.

[0149] [4]He, K., Gkioxari, G., Doll′ar, P., Girshick, R.: Mask R-CNN. In: ICCV 2017.

Claims

1. A method for multi-level behavior recognition in dense crowd scenes, based on convolutional neural networks, basic graph neural networks, layered graph neural networks, and multi-layer perceptron networks; characterized in that ； The steps include: Step s1: extract the feature vector corresponding to the pedestrian target detection frame in the scene through the convolutional neural network ; Step s2, constructing an initial basic graph neural network based on the corresponding feature vectors in the scene; Step s3: Calculate the similarity between the two feature vectors of the basic graph neural network to obtain the target similarity matrix ; Step s4: The basic graph neural network uses the area normalized spatial distance measurement method to calculate the spatial distance matrix between pedestrians. ; Step s5, the basic graph neural network is based on the similarity matrix and spatial distance matrix Obtain the relationship matrix R; Step s6, the basic graph neural network uses the relationship matrix R to update the feature vector in the scene; Step s7: The layered graph neural network aggregates the updated feature vector twice to obtain pedestrian features. , grouping features , global features ; Step s8: The multi-layer perceptron network obtains the action category prediction result according to the following formula: ； ； ； in: is the result of single behavior recognition, is the behavior recognition result of the group, The overall behavior recognition result of the scene; The layered graph neural network aggregates the updated feature vector twice to obtain pedestrian features. , grouping features , global features ; According to the relationship matrix R, a clustering algorithm is used to obtain pedestrian grouping prediction results; Use the aggregation network to aggregate all pedestrian features in each group Aggregate and obtain grouping features according to the following formula : ； Where: The subscript k indicates that the group number corresponding to the feature is k, that is, the kth group , Represents the corresponding features of all pedestrians belonging to this group, represents the feature aggregation operation, Indicates the grouping feature corresponding to the k-th group; Group features using aggregation networks and pedestrian characteristics that do not belong to any group Aggregate and obtain global features according to the following formula : ； in: Indicates that u does not belong to any group, that is, the single person feature that does not participate in the group , Indicates the grouping characteristics formed by all groups Represents the global features obtained by aggregation.

2. The method for multi-level behavior recognition in dense crowd scenes according to claim 1, characterized in that: The similarity between the two feature vectors of the basic graph neural network is calculated to obtain the target similarity matrix , the process is as follows: Mapping pedestrians in the scene ;in Represents the node information in the graph, Represents the edge information in the graph; that is, the node similarity matrix; Calculate similarity based on the edge information of the graph: ； in is a multi-layer perceptron network structure. Represents the features corresponding to node u and node v, side information It represents the similarity between nodes u and v. By calculating the similarity between any two nodes, the similarity matrix E of the entire graph can be obtained.

3. The method for multi-level behavior recognition in dense crowd scenes according to claim 2, characterized in that: The basic graph neural network uses the relationship matrix R to update the feature vector process in the scene: According to the initialized graph network and the calculated similarity matrix, the node information of the graph network is updated. The specific steps are as follows: Use the following formula to update graph node information: ； in: Update the network for a node consisting of fully connected layers, are the node features before and after the update, respectively. Node v represents the node connected to node u. Calculated in the above steps; Aggregate the node features before and after the update. The specific formula is: ； Can get is the final pedestrian feature, the subscript u indicates that the pedestrian number corresponding to the feature is u, and the operator Represents the bitwise addition of elements in a eigenvector.

4. The method for multi-level behavior recognition in dense crowd scenes according to claim 1, characterized in that: The basic graph neural network uses the area normalized spatial distance measurement method to calculate the spatial distance matrix between pedestrians. ;include: The initial spatial distance between pedestrians is calculated using the area-normalized spatial distance measurement method. The formula is as follows: ； in: , is the horizontal coordinate of the midpoint of the lower edge of the detection frame, , The vertical coordinate of the midpoint of the lower edge of the detection frame; 、 is the area of ​​the detection box; The spatial distance mask matrix is ​​calculated using the following formula: : ； in are pre-set hyperparameters.

5. The method for multi-level behavior recognition in dense crowd scenes according to claim 3 is characterized by: The basic graph neural network is based on the similarity matrix and spatial distance matrix The relationship matrix R is obtained according to the following formula; ； in: is a pre-set hyperparameter used to balance the weights between the similarity matrix and the spatial distance matrix. Represents the multiplication of corresponding matrix elements.

Images