Human skeleton behavior recognition method based on global adaptive graph convolutional network

Abstract

The invention discloses a human skeleton behavior recognition method based on a global adaptive graph convolutional network. The method comprises the steps of processing a data set, constructing a human skeleton behavior recognition network, training the human skeleton behavior recognition network, testing the human skeleton behavior recognition network and recognizing behaviors. According to the method, the dynamic learning topological structure is represented by learning actions on different time scales, and the time weight is adaptively adjusted, so that local and global relationships between frame joints are simulated. By capturing the complex relationship between remote nodes, the ability of the model to understand the global dependency relationship is enhanced. Compared with the prior art, a contrast experiment is carried out, the experiment shows that the accuracy of the method is superior to that of the contrast experiment, the accuracy reaches 93.7% and 97.5% on the NTU RGB + D evaluation standard, the accuracy reaches 90.9% and 92.2% on the NTU RGB + D 120 evaluation standard, the accuracy of behavior recognition is improved, and the method can be popularized and used in the technical field of image processing.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a method for recognizing human skeleton behavior. Background Art

[0002] Action recognition is a task that classifies action categories by receiving video data as input. It is an important branch of computer vision research and is widely used in various fields such as video surveillance, medical care, education research, and human-computer interaction. In recent years, RGB-based and skeleton-based action recognition methods have been proposed. Compared with RGB-based methods, skeleton information directly reflects the structural information of the human body, can capture the essential characteristics of human movement, has lower cost, and is more robust to interference factors such as complex backgrounds and different lighting conditions.

[0003] Skeleton behavior recognition method based on unraveling adaptive multi-dimensional dynamic graph convolutional network A behavior recognition method based on graph convolutional network is disclosed, which effectively captures the interaction relationship of each pair of joint points and learns the important detail features of the joint points, and enhances the ability to extract spatial structural features. However, this method does not fully model the local features and global dependencies of the joints, and uses the same topological structure for each frame, resulting in insufficient modeling of skeleton dynamic information, affecting recognition accuracy. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a human skeleton behavior recognition method based on a global adaptive graph convolutional network with high recognition accuracy.

[0005] The technical solution adopted to solve the above technical problems consists of the following steps:

[0006] (1) Processing Dataset

[0007] The total number of behavior categories K for the NTU RGB+D sequence is 60, and the total number of behavior categories K for the NTU RGB+D 120 sequence is 120.

[0008] Process the NTU RGB+D sequence and the NTU RGB+D 120 sequence;

[0009] The human skeleton diagram is represented as:

[0010] G={V,E}

[0011] V={v1,v2,…,v N}

[0012] Among them, v N Represents a set of N joints, where N is 25 and E is the physical connection between the joints.

[0013] (2) Constructing a human skeleton behavior recognition network

[0014] The human skeleton behavior recognition network is composed of a spatiotemporal graph convolutional network basic block 1 and a spatiotemporal graph convolutional network basic block 2, a spatiotemporal graph convolutional network basic block 3, a spatiotemporal graph convolutional network basic block 4, a spatiotemporal graph convolutional network basic block 5, a spatiotemporal graph convolutional network basic block 6, a spatiotemporal graph convolutional network basic block 7, a spatiotemporal graph convolutional network basic block 8, a spatiotemporal graph convolutional network basic block 9, a global average pooling layer, and a softmax layer connected in series in sequence.

[0015] The spatiotemporal graph convolution network basic block 1 is composed of a global diffuse attention graph convolution module, an adaptive attention spatiotemporal aggregation module, and a multi-scale adaptive time convolution module. The output ends of the global diffuse attention graph convolution module and the adaptive attention spatiotemporal aggregation module are connected to the multi-scale adaptive time convolution module; the structures of the spatiotemporal graph convolution network basic block 2, spatiotemporal graph convolution network basic block 3, spatiotemporal graph convolution network basic block 4, spatiotemporal graph convolution network basic block 5, spatiotemporal graph convolution network basic block 6, spatiotemporal graph convolution network basic block 7, spatiotemporal graph convolution network basic block 8, and spatiotemporal graph convolution network basic block 9 are the same as the structure of the spatiotemporal graph convolution network basic block 1.

[0016] (3) Training the human skeleton behavior recognition network

[0017] 1) Construct loss function

[0018] The loss function L is constructed as follows:

[0019]

[0020] Among them, the total number of behavior categories K of the NTU RGB+D sequence is 60, M is the number of samples, and M is 64; the total number of behavior categories K of the NTU RGB+D120 sequence is 120, and M is 64; i is a sample in M, j is a category in K, and H ij Whether the behavior category of sample i is j, p ij is the probability that sample i belongs to the true behavior category.

[0021] 2) Training human skeleton behavior recognition network

[0022] The training set is input into the human skeleton action recognition network for training. The SGD optimizer with Nesterov momentum of 0.9 and weight decay of 0.0004 is used. The training cycle is 80. A warm-up strategy is used in the first five cycles to stabilize the training process. Cosine annealing decay is used with a learning rate of 0.0001. For the NTU RGB+D and NTU RGB+D 120 datasets, the batch size is set to 64 and the training is carried out until the loss function converges.

[0023] (4) Testing the human skeleton behavior recognition network

[0024] The test set is input into the trained human skeleton behavior recognition network, and the output of the human skeleton behavior recognition network is fused to obtain the final human skeleton behavior recognition result. Top-1 is used as the evaluation index, and the best result is selected from the accuracy vector as the classification result. If the result is consistent with the true label, the classification is correct, otherwise the classification is wrong. The Top-1 accuracy is determined by the following formula:

[0025]

[0026] in, represents the predicted category of the human skeleton action recognition network for the i-th sample, that is, the category with the highest confidence, g i represents the true category of the i-th sample, i.e., the label, is an intermediate variable.

[0027] (5) Identification behavior

[0028] The skeleton data X of frame t t As input, get the joint global feature Z s and spatiotemporal aggregation features F agg :

[0029] Z s =Q s (X t )

[0030] F agg =P(X t )

[0031] Among them, t∈(1,2,..,T), T is the total number of frames in the skeleton sequence, Q s (·) is the global diffuse attention graph convolution module, P(·) is the adaptive attention spatiotemporal aggregation module; the human skeleton behavior recognition result Y is obtained as follows:

[0032] Y=σ(GAP(F out ))

[0033]

[0034] F t =R(Z spatial )

[0035] Z spatial =αF agg +(1-α)Z s

[0036] Among them, α is the learning parameter, α∈(0,1), GAP is the global average pooling, σ is the softmax operation, F out is the behavior characteristic of human skeleton, F t is the long-term and short-term time feature, R(·) is the multi-scale adaptive time convolution module, Z spatial It is the skeleton space feature.

[0037] In the step (2) of constructing a human skeleton behavior recognition network of the present invention, the global diffuse attention graph convolution module is composed of a pooling layer 1, a convolution layer 1, a self-attention module 1, a pooling layer 2, a convolution layer 2, and a self-attention module 2. The pooling layer 1 is connected in series with the convolution layer 1 and the self-attention module 1 in sequence. The pooling layer 2 is connected in series with the convolution layer 2 and the self-attention module 2 in sequence. The output ends of the self-attention module 1 and the self-attention module 2 are connected.

[0038] The convolution kernels of the convolution layer 1 and the convolution layer 2 of the present invention are 1×1 and the step size is 1.

[0039] In step (2) of the present invention, the adaptive attention spatiotemporal aggregation module is composed of a convolutional layer 3, a pooling layer 3, a convolutional layer 4, a pooling layer 4, a softmax layer, and a spatiotemporal attention module. The convolutional layer 3 and the pooling layer 3 are connected in series and then connected to an input end of the softmax layer. The convolutional layer 4 and the pooling layer 4 are connected in series and then connected to another input end of the softmax layer. The output end of the softmax layer is connected to the spatiotemporal attention module.

[0040] The convolution kernels of the convolution layers 3 and 4 of the present invention are 1×1 and the step size is 1.

[0041] The multi-scale adaptive time convolution module of the present invention is composed of a convolution layer 5, a convolution layer 6, a convolution layer 7, a convolution layer 8, a convolution expansion layer 1, a convolution expansion layer 2, and a pooling layer 5. The convolution layer 5 is connected in series with the convolution expansion layer 1, the convolution layer 6 is connected in series with the convolution expansion layer 2, the convolution layer 7 is connected in series with the pooling layer 5, and the output ends of the convolution expansion layer 1, the convolution expansion layer 2, the pooling layer 5, and the convolution layer 8 are connected.

[0042] The convolution kernel of the convolution layer 5 of the present invention is 1×1 and the step size is 1, and the sizes of the convolution layers 6, 7 and 8 are the same as that of the convolution layer 5; the convolution kernel of the convolution expansion layer 1 is 5×1 and the expansion coefficient is 3, and the convolution kernel of the convolution expansion layer 2 is 5×1 and the expansion coefficient is 1.

[0043] The present invention dynamically learns the topological structure and adaptively adjusts the time weights by learning the action representations on different time scales to simulate the local and global relationships between frame joints. At the same time, by capturing the complex relationships between distant nodes, the model's ability to understand global dependencies is further enhanced. The present invention and the prior art were compared in experiments, and the experiments showed that the accuracy of the present invention was better than that of the comparative experiments. The present invention can learn the global joint spatial relationship of each frame in the skeleton sequence, while capturing the local dynamics and overall temporal patterns of the action, improving the accuracy of behavior recognition, and can be promoted and used in the field of image processing technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flow chart of embodiment 1 of the present invention.

[0045] Figure 2 This is a schematic diagram of the structure of the human skeleton behavior recognition network.

[0046] Figure 3 yes Figure 2 Schematic diagram of the structure of the basic block 1 of the spatiotemporal graph convolutional network.

[0047] Figure 4 yes Figure 3 Schematic diagram of the structure of the global diffuse attention graph convolution module in.

[0048] Figure 5 yes Figure 3 Schematic diagram of the structure of the adaptive attention spatiotemporal aggregation module.

[0049] Figure 6 yes Figure 3 Schematic diagram of the structure of the multi-scale adaptive temporal convolution module. DETAILED DESCRIPTION

[0050] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0051] Example 1

[0052] The human skeleton behavior recognition method based on the global adaptive graph convolutional network of this embodiment consists of the following steps (see Figure 1 ):

[0053] (1) Processing Dataset

[0054] The total number of behavior categories K for the NTU RGB+D sequence is 60, and the total number of behavior categories K for the NTU RGB+D 120 sequence is 120.

[0055] The NTU RGB+D sequence and the NTU RGB+D 120 sequence are processed.

[0056] The human skeleton diagram is represented as:

[0057] G={V,E}

[0058] V={v1,v2,…,v N}

[0059] Among them, v N Represents a set of N joints, where N is 25 and E is the physical connection between the joints.

[0060] (2) Constructing a human skeleton behavior recognition network

[0061] Figure 2 A schematic diagram of the structure of a human skeleton behavior recognition network in Example 1 of the present invention is given. Figure 2 In the figure, the human skeleton behavior recognition network of this embodiment is composed of a spatiotemporal graph convolutional network basic block 1 and a spatiotemporal graph convolutional network basic block 2, a spatiotemporal graph convolutional network basic block 3, a spatiotemporal graph convolutional network basic block 4, a spatiotemporal graph convolutional network basic block 5, a spatiotemporal graph convolutional network basic block 6, a spatiotemporal graph convolutional network basic block 7, a spatiotemporal graph convolutional network basic block 8, a spatiotemporal graph convolutional network basic block 9, a global average pooling layer, and a softmax layer connected in series in sequence.

[0062] Figure 3 Given Figure 2 Schematic diagram of the structure of the basic block 1 of the spatiotemporal graph convolutional network. Figure 3 In the embodiment, the spatiotemporal graph convolution network basic block 1 is composed of a global diffuse attention graph convolution module, an adaptive attention spatiotemporal aggregation module, and a multi-scale adaptive time convolution module. The output ends of the global diffuse attention graph convolution module and the adaptive attention spatiotemporal aggregation module are connected to the multi-scale adaptive time convolution module; the structures of the spatiotemporal graph convolution network basic block 2, spatiotemporal graph convolution network basic block 3, spatiotemporal graph convolution network basic block 4, spatiotemporal graph convolution network basic block 5, spatiotemporal graph convolution network basic block 6, spatiotemporal graph convolution network basic block 7, spatiotemporal graph convolution network basic block 8, and spatiotemporal graph convolution network basic block 9 are the same as the structure of the spatiotemporal graph convolution network basic block 1.

[0063] Figure 4 Given Figure 3 Schematic diagram of the structure of the global diffuse attention graph convolution module in. Figure 4In the embodiment, the global diffuse attention graph convolution module is composed of pooling layer 1, convolution layer 1, self-attention module 1, pooling layer 2, convolution layer 2, and self-attention module 2. Pooling layer 1 is connected in series with convolution layer 1 and self-attention module 1, and pooling layer 2 is connected in series with convolution layer 2 and self-attention module 2. The output end of self-attention module 1 is connected to the output end of self-attention module 2. The convolution kernels of convolution layer 1 and convolution layer 2 in the embodiment are 1×1 and the step size is 1.

[0064] Figure 5 Given Figure 3 Schematic diagram of the structure of the adaptive attention spatiotemporal aggregation module in . Figure 5 In the embodiment, the adaptive attention spatiotemporal aggregation module is composed of convolution layer 3, pooling layer 3, convolution layer 4, pooling layer 4, softmax layer, and spatiotemporal attention module. Convolution layer 3 and pooling layer 3 are connected in series and then connected to one input end of the softmax layer. Convolution layer 4 and pooling layer 4 are connected in series and then connected to another input end of the softmax layer. The output end of the softmax layer is connected to the spatiotemporal attention module. The convolution kernels of convolution layer 3 and convolution layer 4 in this embodiment are 1×1 and the stride is 1.

[0065] Figure 6 Given Figure 3 Schematic diagram of the structure of the multi-scale adaptive temporal convolution module. Figure 6 In the embodiment, the multi-scale adaptive time convolution module is composed of convolution layer 5, convolution layer 6, convolution layer 7, convolution layer 8, convolution expansion layer 1, convolution expansion layer 2, and pooling layer 5. Convolution layer 5 is connected in series with convolution expansion layer 1, convolution layer 6 is connected in series with convolution expansion layer 2, convolution layer 7 is connected in series with pooling layer 5, and the output ends of convolution expansion layer 1, convolution expansion layer 2, pooling layer 5, and convolution layer 8 are connected. The convolution kernel of convolution layer 5 of the embodiment is 1×1 and the step size is 1. The sizes of convolution layer 6, convolution layer 7, and convolution layer 8 are the same as those of convolution layer 5. The convolution kernel of convolution expansion layer 1 of the embodiment is 5×1 and the expansion coefficient is 3, and the convolution kernel of convolution expansion layer 2 is 5×1 and the expansion coefficient is 1.

[0066] (3) Training the human skeleton behavior recognition network

[0067] 1) Construct loss function

[0068] The loss function L is constructed as follows:

[0069]

[0070] Among them, the total number of behavior categories K of the NTU RGB+D sequence is 60, M is the number of samples, and M is 64; the total number of behavior categories K of the NTU RGB+D120 sequence is 120, and M is 64; i is a sample in M, j is a category in K, and Hij Whether the behavior category of sample i is j, p ij is the probability that sample i belongs to the real behavior category;

[0071] 2) Training human skeleton behavior recognition network

[0072] The training set is input into the human skeleton action recognition network for training. The SGD optimizer with Nesterov momentum of 0.9 and weight decay of 0.0004 is used. The training cycle is 80. A warm-up strategy is used in the first five cycles to stabilize the training process. Cosine annealing decay is used with a learning rate of 0.0001. For the NTU RGB+D and NTU RGB+D 120 datasets, the batch size is set to 64 and the training is carried out until the loss function converges.

[0073] (4) Testing the human skeleton behavior recognition network

[0074] The test set is input into the trained human skeleton behavior recognition network, and the output of the human skeleton behavior recognition network is fused to obtain the final human skeleton behavior recognition result. Top-1 is used as the evaluation index, and the best result is selected from the accuracy vector as the classification result. If the result is consistent with the true label, the classification is correct, otherwise the classification is wrong. The Top-1 accuracy is determined by the following formula:

[0075]

[0076] in, represents the predicted category of the human skeleton action recognition network for the i-th sample, that is, the category with the highest confidence, g i represents the true category of the i-th sample, i.e., the label, is an intermediate variable.

[0077] (5) Identification behavior

[0078] The skeleton data X of frame t t As input, get the joint global feature Z s and spatiotemporal aggregation features F agg :

[0079] Z s =Q s (X t )

[0080] F agg =P(X t )

[0081] Among them, t∈(1,2,..,T), T is the total number of frames in the skeleton sequence, Q s(·) is the global diffuse attention graph convolution module, P(·) is the adaptive attention spatiotemporal aggregation module; the human skeleton behavior recognition result Y is obtained as follows:

[0082] Y=σ(GAP(F out ))

[0083]

[0084] F t =R(Z spatial )

[0085] Z spatial =αF agg +(1-α)Z s

[0086] Where α is a learning parameter, α∈(0,1), in this embodiment, α is set to 0.5, GAP is the global average pooling, σ is the softmax operation, and F out is the behavior characteristic of human skeleton, F t is the long-term and short-term time feature, R(·) is the multi-scale adaptive time convolution module, Z spatial It is the skeleton space feature.

[0087] Completed the human skeleton behavior recognition method based on global adaptive graph convolutional network.

[0088] Example 2

[0089] The human skeleton behavior recognition method based on the global adaptive graph convolutional network of this embodiment consists of the following steps:

[0090] (1) Processing Dataset

[0091] This step is the same as in Example 1.

[0092] (2) Constructing a human skeleton behavior recognition network

[0093] This step is the same as in Example 1.

[0094] (3) Training the human skeleton behavior recognition network

[0095] This step is the same as in Example 1.

[0096] (4) Testing the human skeleton behavior recognition network

[0097] This step is the same as in Example 1.

[0098] (5) Identification behavior

[0099] The skeleton data X of frame t t As input, get the joint global feature Z sand spatiotemporal aggregation features F agg :

[0100] Z s =Q s (X t )

[0101] F agg =P(X t )

[0102] Among them, t∈(1,2,..,T), T is the total number of frames in the skeleton sequence, Q s (·) is the global diffuse attention graph convolution module, P(·) is the adaptive attention spatiotemporal aggregation module; the human skeleton behavior recognition result Y is obtained as follows:

[0103] Y=σ(GAP(F out ))

[0104]

[0105] F t =R(Z spatial )

[0106] Z spatial =αF agg +(1-α)Z s

[0107] Wherein, α is a learning parameter, α∈(0,1), and the value of α in this embodiment is 0.1. The meanings and value ranges of other parameters and variables are the same as those in Embodiment 1.

[0108] Completed the human skeleton behavior recognition method based on global adaptive graph convolutional network.

[0109] Completed the human skeleton behavior recognition method based on global adaptive graph convolutional network.

[0110] Example 3

[0111] The human skeleton behavior recognition method based on the global adaptive graph convolutional network of this embodiment consists of the following steps:

[0112] (1) Processing Dataset

[0113] This step is the same as in Example 1.

[0114] (2) Constructing a human skeleton behavior recognition network

[0115] This step is the same as in Example 1.

[0116] (3) Training the human skeleton behavior recognition network

[0117] This step is the same as in Example 1.

[0118] (4) Testing the human skeleton behavior recognition network

[0119] This step is the same as in Example 1.

[0120] (5) Identification behavior

[0121] The skeleton data X of frame t t As input, get the joint global feature Z s and spatiotemporal aggregation features F agg :

[0122] Z s =Q s (X t )

[0123] F agg =P(X t )

[0124] Among them, t∈(1,2,..,T), T is the total number of frames in the skeleton sequence, Q s (·) is the global diffuse attention graph convolution module, P(·) is the adaptive attention spatiotemporal aggregation module; the human skeleton behavior recognition result Y is obtained as follows:

[0125] Y=σ(GAP(F out ))

[0126]

[0127] F t =R(Z spatial )

[0128] Z spatial =αF agg +(1-α)Z s

[0129] Wherein, α is a learning parameter, α∈(0,1), and the value of α in this embodiment is 0.9. The meanings and value ranges of other parameters and variables are the same as those in Embodiment 1.

[0130] Completed the human skeleton behavior recognition method based on global adaptive graph convolutional network.

[0131] In order to verify the beneficial effects of the method of the present invention, a comparative experiment was conducted using Example 1 of the present invention on the commonly used human skeleton behavior datasets NTU RGB+D and NTU RGB+D 120 with "Channel-wise topology refinement graphconvolution for skeleton-based action recognition" (referred to as comparative experiment 1), "Constructing stronger and faster baselines for skeleton-based actionrecognition" (referred to as comparative experiment 2), and "Infogcn: Representation learning for humanskeleton-based action recognition" (referred to as comparative experiment 3). The behavior recognition results were evaluated using the Top-1 accuracy.

[0132] The NTU RGB+D dataset has two evaluation metrics: (1) Cross-Subject (X-Sub): 40,320 action samples of 20 subjects are divided into a training set according to their personal IDs, and the test set is 16,560 action samples of another 20 subjects. (2) Cross-View (X-View): 37,920 samples performed by subjects captured by cameras 2 and 3 are used as the training set, and 18,960 samples performed by subjects captured by camera 1 are used as the test set.

[0133] There are two evaluation metrics for the NTU-RGB+D 120 dataset: (1) Cross-Subject (X-Sub): 630,226 samples generated by 106 subjects are used for training and 50,922 samples are used for testing; (2) Cross-Set (X-Set): Even-numbered ID categories are used as training sets and odd-numbered ID categories are used as test sets, containing a total of 54,471 training samples and 59,477 test samples.

[0134] Determine the Top-1 accuracy as follows:

[0135]

[0136] in, represents the predicted category of the human skeleton action recognition network for the i-th sample, that is, the category with the highest confidence, g i represents the true category of the i-th sample, i.e., the label, is an intermediate variable.

[0137] The experimental and calculation results are shown in Table 1.

[0138] Table 1 Experimental results of Example 1 method and comparative experiment

[0139]

[0140] As shown in Table 1, compared with the comparative test, the human skeleton behavior recognition method based on the global adaptive graph convolutional network of Example 1 achieved 93.7% and 97.5% accuracy rates in the NTU RGB+D evaluation standard, and 90.9% and 92.2% accuracy rates in the NTU RGB+D 120 evaluation standard, respectively. This proves the effectiveness and feasibility of the method of the present invention.

Claims

1. A human skeleton behavior recognition method based on a global adaptive graph convolutional network, characterized in that It consists of the following steps: (1) Processing Dataset The total number of behavior categories K for the NTU RGB+D sequence is 60, and the total number of behavior categories K for the NTU RGB+D 120 sequence is 120; Process the NTU RGB+D sequence and the NTU RGB+D 120 sequence; The human skeleton diagram is represented as: G={V,E} V={v1,v2,…,v N } Among them, v N represents a set of N joints, N is 25, and E is the physical connection between the joints; (2) Constructing a human skeleton behavior recognition network The human skeleton behavior recognition network is composed of a spatiotemporal graph convolution network basic block 1 and a spatiotemporal graph convolution network basic block 2, a spatiotemporal graph convolution network basic block 3, a spatiotemporal graph convolution network basic block 4, a spatiotemporal graph convolution network basic block 5, a spatiotemporal graph convolution network basic block 6, a spatiotemporal graph convolution network basic block 7, a spatiotemporal graph convolution network basic block 8, a spatiotemporal graph convolution network basic block 9, a global average pooling layer, and a softmax layer connected in series in sequence; The spatiotemporal graph convolution network basic block 1 is composed of a global diffuse attention graph convolution module, an adaptive attention spatiotemporal aggregation module, and a multi-scale adaptive time convolution module. The output ends of the global diffuse attention graph convolution module and the adaptive attention spatiotemporal aggregation module are connected to the multi-scale adaptive time convolution module; the structures of the spatiotemporal graph convolution network basic block 2, the spatiotemporal graph convolution network basic block 3, the spatiotemporal graph convolution network basic block 4, the spatiotemporal graph convolution network basic block 5, the spatiotemporal graph convolution network basic block 6, the spatiotemporal graph convolution network basic block 7, the spatiotemporal graph convolution network basic block 8, and the spatiotemporal graph convolution network basic block 9 are the same as the structure of the spatiotemporal graph convolution network basic block 1; (3) Training the human skeleton behavior recognition network 1) Construct loss function The loss function L is constructed as follows: The total number of behavior categories K for the NTU RGB+D sequence is 60, M is the number of samples, and M is 64; the total number of behavior categories K for the NTU RGB+D 120 sequence is 120, and M is 64; i is a sample in M, j is a category in K, and H ij Is the behavior category of sample i j, p ij is the probability that sample i belongs to the real behavior category; 2) Training human skeleton behavior recognition network The training set is input into the human skeleton action recognition network for training. The SGD optimizer with Nesterov momentum of 0.9 and weight decay of 0.0004 is used. The training cycle is 80. A warm-up strategy is used in the first five cycles to stabilize the training process. Cosine annealing decay is used with a learning rate of 0.0001. For the NTU RGB+D and NTU RGB+D 120 datasets, the batch size is set to 64 and training is performed until the loss function converges. (4) Testing the human skeleton behavior recognition network The test set is input into the trained human skeleton behavior recognition network, and the output of the human skeleton behavior recognition network is fused to obtain the final human skeleton behavior recognition result. Top-1 is used as the evaluation index, and the best result is selected from the accuracy vector as the classification result. If the result is consistent with the true label, the classification is correct, otherwise the classification is wrong. The Top-1 accuracy is determined by the following formula: in, represents the predicted category of the human skeleton action recognition network for the i-th sample, that is, the category with the highest confidence, g i represents the true category of the i-th sample, i.e., the label, is an intermediate variable; (5) Identification behavior The skeleton data X of frame t t As input, get the joint global feature Z s and spatiotemporal aggregation features F agg : Z s =Q s (X t ) F agg =P(X t ) Among them, t∈(1,2,..,T), T is the total number of frames in the skeleton sequence, Q s (·) is the global diffuse attention graph convolution module, P(·) is the adaptive attention spatiotemporal aggregation module; the human skeleton behavior recognition result Y is obtained as follows: Y=σ(GAP(F out )) F t =R(Z spatial ) Z spatial =αF agg +(1-α)Z s Among them, α is the learning parameter, α∈(0,1), GAP is the global average pooling, σ is the softmax operation, F out is the behavior characteristic of human skeleton, F t is the long-term and short-term temporal features, R(·) is the multi-scale adaptive temporal convolution module, and Z spatial It is the skeleton space feature.

2. The method for human skeleton behavior recognition based on a global adaptive graph convolutional network according to claim 1, characterized in that: In step (2) of constructing a human skeleton behavior recognition network, the global diffuse attention graph convolution module is composed of a pooling layer 1, a convolution layer 1, a self-attention module 1, a pooling layer 2, a convolution layer 2, and a self-attention module 2. The pooling layer 1 is connected in series with the convolution layer 1 and the self-attention module 1 in sequence. The pooling layer 2 is connected in series with the convolution layer 2 and the self-attention module 2 in sequence. The output ends of the self-attention module 1 and the self-attention module 2 are connected.

3. The method for human skeleton behavior recognition based on a global adaptive graph convolutional network according to claim 2, characterized in that: The convolution kernels of the convolution layer 1 and the convolution layer 2 are 1×1 and the step size is 1.

4. The method for human skeleton behavior recognition based on a global adaptive graph convolutional network according to claim 1, characterized in that: In step (2), the adaptive attention spatiotemporal aggregation module is composed of a convolutional layer 3, a pooling layer 3, a convolutional layer 4, a pooling layer 4, a softmax layer, and a spatiotemporal attention module. The convolutional layer 3 and the pooling layer 3 are connected in series and then connected to an input end of the softmax layer. The convolutional layer 4 and the pooling layer 4 are connected in series and then connected to another input end of the softmax layer. The output end of the softmax layer is connected to the spatiotemporal attention module.

5. The method for human skeleton behavior recognition based on global adaptive graph convolutional network according to claim 4, characterized in that: The convolution kernels of the convolution layer 3 and the convolution layer 4 are 1×1 and the step size is 1.

6. The method for human skeleton behavior recognition based on a global adaptive graph convolutional network according to claim 2, characterized in that: The multi-scale adaptive time convolution module is composed of convolution layer 5, convolution layer 6, convolution layer 7, convolution layer 8, convolution expansion layer 1, convolution expansion layer 2, and pooling layer 5. Convolution layer 5 is connected in series with convolution expansion layer 1, convolution layer 6 is connected in series with convolution expansion layer 2, convolution layer 7 is connected in series with pooling layer 5, and the output ends of convolution expansion layer 1, convolution expansion layer 2, pooling layer 5, and convolution layer 8 are connected.

7. The method for human skeleton behavior recognition based on global adaptive graph convolutional network according to claim 6, characterized in that: The convolution kernel of the convolution layer 5 is 1×1 and the step size is 1. The sizes of convolution layers 6, 7 and 8 are the same as those of convolution layer 5. The convolution kernel of the convolution expansion layer 1 is 5×1 and the expansion coefficient is 3. The convolution kernel of the convolution expansion layer 2 is 5×1 and the expansion coefficient is 1.

Images