Human pose recognition method based on multi-dimensional cooperative attention network

Abstract

A human posture recognition method based on a multi-dimensional cooperative attention network, which is composed of the steps of selecting and preprocessing a data set, dividing the data set, constructing a multi-dimensional cooperative attention neural network, training the multi-dimensional cooperative attention neural network, testing the multi-dimensional cooperative attention network, and evaluating the performance of the multi-dimensional cooperative attention neural network. The neural network is adopted in the present application, and the channel refinement convolution and multi-dimensional cooperative attention effectively aggregate the joint features among different channels while reducing the computational complexity. The constructed neural network environment perception module improves the receptive field to capture more comprehensive environmental information. The present application has the advantages of simple network structure, good network performance, wide application range, etc., and can be used in human posture recognition systems, dance teaching systems, etc.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to human posture recognition. BACKGROUND

[0002] Human posture recognition can also be referred to as human key point detection, which is to recognize the positions of human key points such as elbows, shoulders, eyes and knees in a picture, and connect the recognized key points according to the limb relationship of the human body to form a human skeleton model. Human posture recognition is one of the hot topics in the field of computer vision in recent years. Solving the problem of human posture recognition can provide information for human fall detection, intelligent monitoring, human behavior recognition and human-computer interaction.

[0003] In recent years, with the rise of deep learning and the application of convolutional neural networks in a series of image processing, the field of human posture recognition has developed rapidly. However, there are still technical problems that are difficult to solve in the fields of background occlusion, complex posture, poor lighting conditions and multi-person posture recognition. The stacked hourglass network (SHN) adopts a very classic U-shaped architecture and performs posture recognition in an end-to-end manner. The cascaded pyramid network (CPN) is divided into two steps of GlobalNet and RefineNet, which first recognizes simple key points and then integrates the recognized feature representations to recognize complex key points. However, the above methods are difficult to meet the precision requirements of practical applications. The high-resolution network (HRNet) always maintains high-resolution feature maps as the main line compared to traditional down-sampling network structures. When down-sampling, a new low-resolution sub-network is generated. The high-resolution feature of the image is maintained from head to tail, and information fusion is performed on multiple parallel sub-networks to further balance the accuracy and computational complexity. However, when facing complex background conditions, there is still the technical problem of low precision. ViTPose proposed in 2022 effectively improves the detection accuracy, but at the same time brings a very high computational burden, which requires high computational load capacity of the device. SUMMARY

[0004] The technical problem to be solved by the present application is to overcome the shortcomings of the above-mentioned prior art and provide a human posture recognition method based on a multi-dimensional collaborative attention network with good network performance and wide application range.

[0005] The technical solution adopted to solve the above technical problem comprises the following steps:

[0006] (1) Select and preprocess the data set

[0007] Select the Microsoft disclosed COCO2017 dataset, which annotates 2500000 human instances, and each pedestrian sample is annotated with 17 human key points: nose a x 1, eyes b x 2, ears c x 2, shoulders d x 2, elbows e x 2, wrists f x 2, hips g x 2, knee joints h x 2, and ankles i x 2.

[0008] The COCO2017 dataset is preprocessed as follows:

[0009] 1) Crop the image into a 256x192 pixel picture.

[0010] 2) Randomly flip the dataset and scale it by 0.25 times.

[0011] (2) Divide the dataset

[0012] The dataset is divided into training set, validation set and test set according to the ratio of 6:2:2.

[0013] (3) Construct a multi-dimensional collaborative attention neural network

[0014] The multi-dimensional collaborative attention neural network is composed of convolution layer 1, bottleneck module, feature fusion module 1, feature fusion module 2, feature fusion module 3, and environment perception module connected in sequence.

[0015] The bottleneck module is composed of convolution layer 2, batch normalization layer 1, ReLU activation function layer 1, channel refinement convolution block, batch normalization layer 2, ReLU activation function layer 2, convolution layer 3, batch normalization layer 3, and multi-dimensional collaborative attention layer connection. Convolution layer 2 is connected in sequence with batch normalization layer 1, ReLU activation function layer 1, channel refinement convolution block, batch normalization layer 2, ReLU activation function layer 2, convolution layer 3, and batch normalization layer 3. One output end of the convolution layer 2 is connected to another input end of the batch normalization layer 3, and another output end is connected to another input end of the channel refinement convolution block through the multi-dimensional collaborative attention layer.

[0016] (4) Train the multi-dimensional collaborative attention network

[0017] 1) Construct the loss function

[0018] The loss function Loss is constructed as follows:

[0019]

[0020] Where y i is the true value, h(x i ) is the output vector of the network, and N is the number of samples, which is a finite positive integer.

[0021] 2) Train the multi-dimensional collaborative attention network

[0022] The training set is input into the multi-dimensional collaborative attention network for training, the training period is 210, the batch size is 32, the initial learning rate is set to 0.001, and the learning rate is changed to 0.0001 after 170 iterations, and the training is performed until the loss function Loss converges.

[0023] (5) Test the multi-dimensional collaborative attention network

[0024] The test set is input into the trained multi-dimensional collaborative attention network for testing, and the human key point a, b, c, d, e, f, d, h, i human posture detection result is output.

[0025] (6) Evaluate the performance of the multi-dimensional collaborative attention network

[0026] The key point similarity evaluation index is used to evaluate the effect of the multi-dimensional collaborative attention network on the test set.

[0027] The key point similarity OKS is determined according to the following formula:

[0028]

[0029] Wherein, represents the square of the Euclidean distance between the i-th predicted key point position and the corresponding GT information, s 2 represents the target area, k i represents the key point normalization factor of type i, δ(v i ) represents a function of selecting visible points for calculation, v i represents the flag bit provided by the GT information for the i-th real key point, P is the number of human key points, and the value of P is 17.

[0030] In step (3) of constructing the multi-dimensional collaborative attention neural network of the application, the channel refinement convolution block is connected by a linear transformation layer, an average pooling layer, a channel refinement layer, a Tanh polarization function layer 3, a convolution layer 4 and a convolution layer 5. The linear transformation layer is connected in series with the average pooling layer, the channel refinement layer, the Tanh polarization function layer 3 and the convolution layer 4 in turn, the input end of the convolution layer 5 is connected with the other output end of the linear transformation layer, and the output end is connected with the other input end of the convolution layer 4.

[0031] In step (3) of constructing the multi-dimensional collaborative attention neural network of the application, the environment perception module is connected in series by a large core convolution module and a multi-scale convolution module.

[0032] The multiscale convolution module of the application is connected by a convolution layer 6, a convolution layer 7, a convolution layer 8, a convolution layer 9, a convolution layer 10, a convolution layer 11, a convolution layer 12, a convolution layer 13 and a convolution layer 14, the convolution layer 6 is connected in series with the convolution layer 7, the convolution layer 10, the convolution layer 11 and the convolution layer 14 in turn, one output end of the convolution layer 7 is connected with one input end of the convolution layer 14 through the convolution layer 8 and the convolution layer 9, and the other output end of the convolution layer 7 is connected with the other input end of the convolution layer 14 through the convolution layer 12 and the convolution layer 13.

[0033] The convolution kernel size of the convolution layer 6 is 1*1 and the step is 1, the convolution kernel size of the convolution layer 7 is 5*5 and the step is 2, the convolution kernel size of the convolution layer 8 is 1*7 and the step is 2, the convolution kernel size of the convolution layer 9 is 7*1 and the step is 2, the convolution kernel size of the convolution layer 10 is 1*11 and the step is 2, the convolution kernel size of the convolution layer 11 is 11*1 and the step is 2, the convolution kernel size of the convolution layer 12 is 1*21 and the step is 2, the convolution kernel size of the convolution layer 13 is 21*1 and the step is 2, and the convolution kernel size of the convolution layer 14 is 1*1 and the step is 2.

[0034] The construction method of the channel refinement convolution block of the application is as follows:

[0035] The channel refinement layer M1 is constructed according to formula (1):

[0036] M1 = σ (ψ (x) - φ (x) ) (1)

[0037] Wherein, σ represents a Tanh activation function, ψ (x) and φ (x) represent linear transformation operations, r represents a channel dimension reduction coefficient, The number of channels after the feature map dimension reduction, W*1 and 1*W represent the height*width of the feature map.

[0038] In the step (3) of constructing the multi-dimensional cooperative attention neural network in the application, the construction method of the large kernel convolution module comprises the following steps:

[0039] 1) Feature vector channel fusion

[0040] The feature vector channel fusion is performed according to formula (2):

[0041]

[0042] Wherein, U j Indicates a parallel branch, j is a limited positive integer, Indicates a convolution kernel with a size of 1*1.

[0043] 2) Feature splicing

[0044] The feature splicing is determined according to formula (3):

[0045]

[0046] 3) processing the spatial attention feature map

[0047] The spatial attention feature map is processed according to formula (4):

[0048] SA=F{SA avg ,SA max} (4)

[0049]

[0050] wherein, SA avg ,SA max is an intermediate variable, respectively represent average pooling and maximum pooling, F represents a convolution kernel with a size of 1x1 and a channel number equal to the number of branches.

[0051] 4) outputting the large kernel convolution module feature

[0052] The large kernel convolution module feature Y is output according to formula (5):

[0053] Y=X×S (5)

[0054]

[0055] SA j =σ(SA j )

[0056] wherein, X represents an input feature, M represents the number of parallel branches, M is a finite positive integer, and σ(SA i ) represents a sigmoid activation function.

[0057] In formula (2) of step 1) feature vector channel fusion, U j represents parallel branches, and j ranges from 1 to 6.

[0058] In formula (5) of step 4) outputting the large kernel convolution module feature, M represents the number of parallel branches, and M∈[1,6].

[0059] In step (4) of the present application, in the formula of constructing the loss function of the multi-dimensional collaborative attention network, N is the number of samples, and N∈[5000,100000].

[0060] Compared with the prior art, the present application has the following advantages:

[0061] The improved bottleneck module is adopted in the application, the network model better aggregates the joint features among different channels, the operation complexity is reduced, the data most conforming to the expectation is generated, the training and optimization of the subsequent network are facilitated, and better results of the model are obtained.

[0062] The environmental perception module is used in the application, the model can flexibly integrate features, capture more comprehensive environmental information, and improve the recognition rate and detection accuracy of the image in a complex background. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 is a flowchart of embodiment 1 of the application.

[0064] Figure 2 is a structural schematic diagram of the multi-dimensional collaborative attention neural network of the application.

[0065] Figure 3 is Figure 2 a structural schematic diagram of the bottleneck module.

[0066] Figure 4 is Figure 3 a structural schematic diagram of the channel refinement convolution block.

[0067] Figure 5 is Figure 2 a structural schematic diagram of the environmental perception module.

[0068] Figure 6 is Figure 5 a structural schematic diagram of the multi-scale convolution module. DETAILED DESCRIPTION

[0069] The application will be further described in detail below in combination with the drawings and embodiments, but the application is not limited to the following embodiments.

[0070] Embodiment 1

[0071] In Figure 1 the embodiment, the human pose recognition method based on the multi-dimensional collaborative attention network comprises the following steps:

[0072] (1) Select and preprocess the data set

[0073] The COCO2017 data set disclosed by Microsoft is selected, wherein 2500000 human instances are labeled, and each pedestrian sample is labeled with 17 human key points: nose a x 1, eyes b x 2, ears c x 2, shoulders d x 2, elbows e x 2, wrists f x 2, hips g x 2, knee joints h x 2, and ankles i x 2.

[0074] The COCO2017 dataset was preprocessed as follows:

[0075] 1) Crop the image to 256×192 pixels.

[0076] 2) Randomly flip the dataset and scale it by 0.25 times.

[0077] (2) Split the dataset

[0078] The dataset is divided into training, validation, and test sets in a 6:2:2 ratio.

[0079] (3) Constructing a multidimensional collaborative attention neural network

[0080] Figure 2 A schematic diagram of the multidimensional collaborative attention neural network structure in this embodiment is provided. Figure 2 In this embodiment, the multidimensional collaborative attention neural network is composed of a convolutional layer 1 connected in series with a bottleneck module, a feature fusion module 1, a feature fusion module 2, a feature fusion module 3, and an environment perception module.

[0081] Figure 3 Given Figure 2 A structural diagram of the bottleneck module. Figure 3 In this embodiment, the bottleneck module is composed of convolutional layer 2, batch normalization layer 1, ReLU activation function layer 1, channel thinning convolutional block, batch normalization layer 2, ReLU activation function layer 2, convolutional layer 3, batch normalization layer 3, and multidimensional collaborative attention layer. Convolutional layer 2 is connected in series with batch normalization layer 1, ReLU activation function layer 1, channel thinning convolutional block, batch normalization layer 2, ReLU activation function layer 2, convolutional layer 3, and batch normalization layer 3. One output of convolutional layer 2 is connected to the other input of batch normalization layer 3, and the other output is connected to the other input of channel thinning convolutional block through multidimensional collaborative attention layer.

[0082] Because this invention employs an improved bottleneck module, the network model can better aggregate joint features between different channels, reducing computational complexity and generating data that best meets expectations. This is beneficial for subsequent network training and optimization, resulting in better model performance.

[0083] Figure 4 Given Figure 3 A schematic diagram of the structure of the mid-channel refined convolution block. Figure 4In the embodiment, the channel refinement convolution block is composed of a linear transformation layer, an average pooling layer, a channel refinement layer, a Tanh polarization function layer 3, a convolution layer 4, and a convolution layer 5 connected in sequence, the linear transformation layer is connected in sequence with the average pooling layer, the channel refinement layer, the Tanh polarization function layer 3, and the convolution layer 4, and the input end of the convolution layer 5 is connected with another output end of the linear transformation layer and the output end is connected with another input end of the convolution layer 4.

[0084] The construction method of the channel refinement convolution block is as follows:

[0085] The channel refinement convolution block M1 is constructed according to formula (1):

[0086] M1 = σ (ψ (x) - φ (x) ) (1)

[0087] wherein σ represents a Tanh activation function, ψ (x) and φ (x) represent linear transformation operations, r represents a channel dimension reduction coefficient, represents the number of channels after the dimension reduction of the feature map, and W x 1 and 1 x W represent the height x width of the feature map.

[0088] Figure 5 The structure diagram of the environment perception module is given. Figure 2 In the embodiment, the environment perception module is composed of a large-core convolution module and a multi-scale convolution module connected in sequence. Figure 5 In the embodiment, the environment perception module is composed of a large-core convolution module and a multi-scale convolution module connected in sequence.

[0089] The construction method of the large-core convolution module of the embodiment comprises the following steps:

[0090] 1) Feature vector channel fusion

[0091] The feature vector channel fusion is performed according to formula (2):

[0092]

[0093] wherein U j represents a parallel branch, j takes a value in the range of 1-6, and in the embodiment, j takes a value in the range of 4, represents a convolution kernel with a size of 1 x 1.

[0094] 2) Feature splicing

[0095] The feature splicing is determined according to formula (3):

[0096]

[0097] 3) Processing the spatial attention feature map

[0098] The spatial attention feature map is processed according to formula (4):

[0099] SA = F{SA avg SA max} (4)

[0100]

[0101] Among them, SA avg SA max As an intermediate variable, These represent average pooling and max pooling, respectively, and F represents a convolution kernel of size 1×1 with the number of channels equal to the number of branches;

[0102] 4) Output large kernel convolution module features

[0103] Output the large kernel convolution module features Y according to equation (5):

[0104] Y = X × S (5)

[0105]

[0106] SA j =σ(SA) j )

[0107] Where X represents the input feature, M represents the number of parallel branches, M∈[1,6], and in this embodiment M is 4, σ(SA i ) represents the sigmoid activation function.

[0108] Figure 6 Given Figure 5 A schematic diagram of the structure of a multi-scale convolutional module is shown in the diagram. Figure 6 In this embodiment, the multi-scale convolution module is composed of convolutional layers 6, 7, 8, 9, 10, 11, 12, 13, and 14 connected together. Convolutional layer 6 is connected in series with convolutional layers 7, 10, 11, and 14. One output of convolutional layer 7 is connected to one input of convolutional layer 14 through convolutional layers 8 and 9, and the other output of convolutional layer 7 is connected to the other input of convolutional layer 14 through convolutional layers 12 and 13.

[0109] In this embodiment, the kernel size of convolutional layer 6 is 1×1 with a stride of 1, the kernel size of convolutional layer 7 is 5×5 with a stride of 2, the kernel size of convolutional layer 8 is 1×7 with a stride of 2, the kernel size of convolutional layer 9 is 7×1 with a stride of 2, the kernel size of convolutional layer 10 is 1×11 with a stride of 2, the kernel size of convolutional layer 11 is 11×1 with a stride of 2, the kernel size of convolutional layer 12 is 1×21 with a stride of 2, the kernel size of convolutional layer 13 is 21×1 with a stride of 2, and the kernel size of convolutional layer 14 is 1×1 with a stride of 2.

[0110] Due to the use of the environment perception module, the model can flexibly integrate features, capture more comprehensive environment information, and improve the recognition rate and detection accuracy of images in complex backgrounds.

[0111] The construction method of the channel refinement convolution block of the embodiment is as follows:

[0112] The channel refinement convolution block M1 is constructed according to formula (1):

[0113] M1 = sigma (psi (x) - phi (x)) (1)

[0114] Wherein, sigma represents the Tanh activation function, psi (x) and phi (x) represent linear transformation operations, r represents the channel dimension reduction coefficient, represents the number of channels after dimension reduction of the feature map, Wx1 and 1xW represent the heightxwidth of the feature map.

[0115] (4) Training the multi-dimensional collaborative attention network

[0116] 1) Constructing the loss function

[0117] The loss function Loss is constructed according to the following formula:

[0118]

[0119] Wherein, y i is the true value, h (x i ) is the output vector of the network, N is the sample number, N [5000, 100000], and the value of N in the embodiment is 50000.

[0120] 2) Training the multi-dimensional collaborative attention network

[0121] The training set is input into the multi-dimensional collaborative attention network for training, the training period is 210, the batch size is 32, the initial learning rate is set to 0.001, and the learning rate becomes 0.0001 after 170 iterations, and the training is stopped when the loss function Loss converges.

[0122] (5) Testing the multi-dimensional collaborative attention network

[0123] The test set is input into the trained multi-dimensional collaborative attention network for testing, and the human key point a, b, c, d, e, f, j, h, i human pose detection result is output.

[0124] (6) Evaluating the performance of the multi-dimensional collaborative attention network

[0125] The key point similarity evaluation index is used to evaluate the effect of the multi-dimensional collaborative attention network on the test set:

[0126] The key point similarity OKS is determined according to the following formula:

[0127]

[0128] Wherein, d i 2 represents the square of the Euclidean distance between the i-th predicted key point position and the corresponding GT information, s 2 represents the target area, k i represents the key point normalization factor of type i, δ(v i ) represents a function of selecting visible points for calculation, v i represents the flag bit of the i-th real key point provided by the GT information.

[0129] The human pose recognition method based on the multi-dimensional collaborative attention network is completed.

[0130] Embodiment 2

[0131] The human pose recognition method based on the multi-dimensional collaborative attention network of the embodiment consists of the following steps:

[0132] (1) Select and preprocess the data set

[0133] This step is the same as that of embodiment 1.

[0134] (2) Divide the data set

[0135] This step is the same as that of embodiment 1.

[0136] (3) Construct a multi-dimensional collaborative attention neural network

[0137] The structure of the multi-dimensional collaborative attention neural network is the same as that of embodiment 1.

[0138] The construction method of the channel refinement convolution block of the embodiment is the same as that of embodiment 1.

[0139] The environment perception module of the embodiment is composed of a large core convolution module and a multi-scale convolution module in series. The construction method of the large core convolution module consists of the following steps:

[0140] 1) Feature vector channel fusion

[0141] The feature vector channel fusion is performed according to formula (2):

[0142] The expression of formula (2) is the same as that of embodiment 1.

[0143] In formula (2), U j represents a parallel branch, and j takes a value in the range of 1-6, and in the embodiment, j takes a value of 1.

[0144] 2) Feature splicing

[0145] This step is the same as that in Embodiment 1.

[0146] 3) Processing the spatial attention feature map

[0147] This step is the same as that in Embodiment 1.

[0148] 4) Outputting the large-core convolution module feature

[0149] The large-core convolution module feature Y is outputted according to formula (5):

[0150] The expression of formula (5) is the same as that in Embodiment 1.

[0151] In formula (5), M represents the number of parallel branches, M ∈ [1, 6], and the value of M in this embodiment is 1. The meanings and values of other parameters are the same as those in Embodiment 1.

[0152] (4) Training the multi-dimensional collaborative attention network

[0153] 1) Constructing a loss function

[0154] The loss function Loss is constructed according to the following formula:

[0155] The expression of the loss function Loss is the same as that in Embodiment 1.

[0156] In the expression of the loss function Loss, N is the number of samples, N ∈ [5000, 100000], and the value of N in this embodiment is 5000. The meanings and values of other parameters are the same as those in Embodiment 1.

[0157] The other steps are the same as those in Embodiment 1. The human pose recognition method based on the multi-dimensional collaborative attention network is completed.

[0158] Embodiment 3

[0159] The human pose recognition method based on the multi-dimensional collaborative attention network in this embodiment comprises the following steps:

[0160] (1) Selecting and preprocessing a data set

[0161] This step is the same as that in Embodiment 1.

[0162] (2) Dividing the data set

[0163] This step is the same as that in Embodiment 1.

[0164] (3) Constructing a multi-dimensional collaborative attention neural network

[0165] The structure of the multi-dimensional collaborative attention neural network is the same as that in Embodiment 1.

[0166] The construction method of the channel refinement convolution block of the embodiment is the same as that of embodiment 1.

[0167] The environment perception module of the embodiment is composed of a large-core convolution module and a multi-scale convolution module in series. The construction method of the large-core convolution module comprises the following steps:

[0168] 1) Feature vector channel fusion

[0169] The feature vector channel fusion is performed according to formula (2):

[0170] The expression of formula (2) is the same as that of embodiment 1.

[0171] In formula (2), U j represents a parallel branch, j takes a value in the range of 1-6, and j takes a value of 6 in the embodiment

[0172] 2) Feature splicing

[0173] This step is the same as that of embodiment 1.

[0174] 3) Processing the spatial attention feature map

[0175] This step is the same as that of embodiment 1.

[0176] 4) Outputting the large-core convolution module feature

[0177] The large-core convolution module feature Y is outputted according to formula (5):

[0178] The expression of formula (5) is the same as that of embodiment 1.

[0179] In formula (5), M represents the number of parallel branches, M ∈ [1, 6], M takes a value of 6 in the embodiment, and the meanings and values of other parameters are the same as those of embodiment 1.

[0180] (4) Training the multi-dimensional collaborative attention network

[0181] 1) Constructing a loss function

[0182] The loss function Loss is constructed according to the following formula:

[0183] The expression of the loss function Loss is the same as that of embodiment 1.

[0184] In the expression of the loss function Loss, N is the number of samples, N ∈ [5000, 100000], N takes a value of 100000 in the embodiment, and the meanings and values of other parameters are the same as those of embodiment 1.

[0185] The other steps are the same as those of embodiment 1. The human pose recognition method based on the multi-dimensional collaborative attention network is completed.

[0186] In order to verify the beneficial effects of the present application, the inventors conducted a laboratory simulation comparative experiment using the human pose recognition method based on the multi-dimensional collaborative attention network of Example 1 of the present application. Various experimental conditions are as follows:

[0187] 1. Experimental environment

[0188] Experimental environment: processor uses AMD EPYC 9754, graphics card is NVIDIA GeForce RTX 3080Ti, operating system type 64-bit, uses python 3.8, deep learning framework is PyTorch1.8.1.

[0189] 2. Experimental content and results

[0190] Example 1 of the present application (referred to as the present application) and an unbiased data processing method (hereinafter referred to as comparative method 1), a human pose estimation method using a high-resolution network (referred to as comparative method 2), a human pose estimation method using a lightweight high-resolution network-18 (referred to as comparative method 3), and a human pose estimation method using a lightweight high-resolution network-30 (referred to as comparative method 4) were tested and compared. Evaluation indicators include: average accuracy (referred to as AP), accuracy when key point similarity OKS is 0.5 (referred to as AP 50 ), accuracy when key point similarity OKS is 0.75 (referred to as AP 75 ), medium scale target accuracy (referred to as AP M ), large scale target accuracy (referred to as AP L ), and average recall rate (referred to as AR). The test results are shown in Table 1, and the comparative experiment results are shown in Table 2.

[0191] Table 1 Test set results of the present application method and the comparative method

[0192] Experimental method Input image AP (%) AP 50 (%)]] AP 75 (%)]] AP M (%)]] AP L (%)]] AR (%) Comparative method 1 384ⅹ288 76.1 92.5 83.5 72.8 82.0 81.3 Comparative method 2 384ⅹ288 74.9 92.5 82.8 71.3 80.9 80.1 The present invention 384ⅹ288 76.5 93.7 83.6 73.9 80.7 80.5

[0193] Table 2 Comparative simulation experiment results of the present application method and the comparative method

[0194] Experimental method Input image AP (%) AP 50 (%)]] AP 75 (%)]] AP M (%)]] AP L (%)]] AR (%) Comparative method 1 256ⅹ192 75.2 92.4 82.9 72.0 80.8 80.4 Comparative method 2 256ⅹ192 73.4 89.5 80.7 70.2 80.1 78.9 Comparative method 3 256ⅹ192 64.8 86.7 73.0 62.1 70.5 71.2 Comparative method 4 256ⅹ192 67.2 88.0 75.0 64.3 73.1 73.3 The present invention 256ⅹ192 75.6 93.2 82.6 73.1 79.8 79.6

[0195] As can be seen from Tables 1 and 2, the present application has higher average recall rate, large scale target accuracy, accuracy when key point similarity OKS is 0.75, average accuracy, accuracy when key point similarity OKS is 0.5, and medium scale target accuracy than the comparative experiment method.

Claims

1.A human pose recognition method based on a multi-dimensional cooperative attention network, characterized by Consists of the following steps: (1) Select and preprocess the data set Select the Microsoft published COCO2017 data set, which has 2,500,000 annotated human instances, and each pedestrian sample is annotated with 17 human key points: nose a x 1, eyes b x 2, ears c x 2, shoulders d x 2, elbows e x 2, wrists f x 2, hips g x 2, knees h x 2, ankles i x 2; The COCO2017 data set is preprocessed as follows: 1) Crop the image to a 256x192 pixel picture; 2) Randomly flip the data set and scale it by 0.25 times; (2) Divide the data set Divide the data set into training set, validation set and test set according to the ratio of 6:2:2; (3) Construct a multi-dimensional collaborative attention neural network The multi-dimensional collaborative attention neural network is composed of convolution layer 1 and bottleneck module, feature fusion module 1, feature fusion module 2, feature fusion module 3, and environment perception module connected in sequence. The bottleneck module is connected by convolution layer 2, batch normalization layer 1, ReLU activation function layer 1, channel refinement convolution block, batch normalization layer 2, ReLU activation function layer 2, convolution layer 3, batch normalization layer 3, and multi-dimensional collaborative attention layer, with convolution layer 2 connected to batch normalization layer 1, ReLU activation function layer 1, channel refinement convolution block, batch normalization layer 2, ReLU activation function layer 2, convolution layer 3, and batch normalization layer 3 connected in sequence, with one output end of convolution layer 2 connected to another input end of batch normalization layer 3, and another output end connected to another input end of channel refinement convolution block through multi-dimensional collaborative attention layer. (4) Train the multi-dimensional collaborative attention network 1) Construct the loss function The loss function Loss is constructed as follows: where y i is the true value, h(x i ) is the output vector of the network, and N is the number of samples, which is a finite positive integer. 2) Train the multi-dimensional collaborative attention network Input the training set into the multi-dimensional collaborative attention network for training, with a training period of 210, a batch size of 32, an initial learning rate of 0.001, a learning rate of 0.0001 after 170 iterations, and training until the loss function Loss converges; (5) Test the multi-dimensional collaborative attention network Input the test set into the trained multi-dimensional collaborative attention network for testing, and output the human key point a, b, c, d, e, f, d, h, i human pose detection result; (6) Evaluate the performance of the multi-dimensional collaborative attention network Use the key point similarity evaluation index to evaluate the effect of the multi-dimensional collaborative attention network on the test set: Determine the key point similarity OKS as follows: wherein, represents the square of the Euclidean distance between the ith predicted key point position and the corresponding GT information, s 2 represents the target area, k i represents the key point normalization factor of type i, δ(v i ) represents a function of selecting visible points for calculation, v i represents the flag bit provided by the GT information for the ith real key point, P is the number of human key points, and the value of P is 17. 2.The method of human pose recognition based on multi-dimensional cooperative attention network according to claim 1, characterized in that: In step (3) of constructing the multi-dimensional collaborative attention neural network, the channel refinement convolution block is connected by a linear transformation layer, an average pooling layer, a channel refinement layer, a Tanh polarization function layer 3, a convolution layer 4, and a convolution layer 5, with the linear transformation layer connected to the average pooling layer, the channel refinement layer, the Tanh polarization function layer 3, and the convolution layer 4 in sequence, with the input end of the convolution layer 5 connected to another output end of the linear transformation layer, and the output end connected to another input end of the convolution layer 4. 3.The method of human pose recognition based on multi-dimensional cooperative attention network according to claim 1, characterized in that: In step (3) of constructing the multi-dimensional collaborative attention neural network, the environment perception module is connected by a large kernel convolution module and a multi-scale convolution module. 4.The method of human pose recognition based on multi-dimensional cooperative attention network according to claim 3, characterized in that: The multi-scale convolution module is connected by the convolution layer 6, the convolution layer 7, the convolution layer 8, the convolution layer 9, the convolution layer 10, the convolution layer 11, the convolution layer 12, the convolution layer 13 and the convolution layer 14, the convolution layer 6 is connected in series with the convolution layer 7, the convolution layer 10, the convolution layer 11 and the convolution layer 14, one output end of the convolution layer 7 is connected with one input end of the convolution layer 14 through the convolution layer 8 and the convolution layer 9, and the other output end of the convolution layer 7 is connected with the other input end of the convolution layer 14 through the convolution layer 12 and the convolution layer 13. 5.The method of human pose recognition based on multi-dimensional cooperative attention network according to claim 4, characterized in that: The convolution kernel size of the convolution layer 6 is 1*1 with a step of 1, the convolution kernel size of the convolution layer 7 is 5*5 with a step of 2, the convolution kernel size of the convolution layer 8 is 1*7 with a step of 2, the convolution kernel size of the convolution layer 9 is 7*1 with a step of 2, the convolution kernel size of the convolution layer 10 is 1*11 with a step of 2, the convolution kernel size of the convolution layer 11 is 11*1 with a step of 2, the convolution kernel size of the convolution layer 12 is 1*21 with a step of 2, the convolution kernel size of the convolution layer 13 is 21*1 with a step of 2, and the convolution kernel size of the convolution layer 14 is 1*1 with a step of 2. 6.The method of human pose recognition based on multi-dimensional cooperative attention network according to claim 2, characterized in that The construction method of the channel refinement convolution block is as follows: The channel refinement layer M1 is constructed according to formula (1): M1 = σ (ψ (x) - φ (x) ) (1) wherein, σ represents a Tanh activation function, and ψ(x), φ(x) represent linear transformation operations, r represents a channel dimension reduction coefficient, represents the number of channels after feature map dimension reduction, and Wx1 and 1xW represent the height x width of the feature map. 7.The method of human pose recognition based on multi-dimensional cooperative attention network according to claim 3, characterized in that: In step (3) of constructing the multi-dimensional collaborative attention neural network, the construction method of the large kernel convolution module comprises the following steps: 1) Feature vector channel fusion The feature vector channel fusion is performed according to formula (2): wherein U j represents a parallel branch, j takes a finite positive integer, represents a convolution kernel of size 1x1; 2) Feature splicing The feature splicing is determined according to formula (3): 3) Processing the spatial attention feature map The spatial attention feature map is processed according to formula (4): SA = F{SA avg , SA max} (4) wherein, SA avg , SA max is an intermediate variable, respectively represent average pooling and max pooling, F represents a convolution kernel with a size of 1x1 and a number of channels equal to the number of branches. 4) Outputting the large kernel convolution module feature The large kernel convolution module feature Y is outputted according to formula (5): Y = X * S (5) SA j = σ(SA j ) where X denotes the input feature, M denotes the number of branches in parallel, M takes a finite positive integer, σ(SA i ) denotes the sigmoid activation function. 8.The method of human pose recognition based on multi-dimensional cooperative attention network according to claim 7, characterized in that: In the formula (2) of step 1) feature vector channel fusion, the U j represents parallel branches, and j has a value range of 1-6. 9.The method of human pose recognition based on multi-dimensional cooperative attention network according to claim 7, characterized in that: In formula (5) of step 4) of outputting the large kernel convolution module feature, M represents the number of parallel branches, and M ∈ [1, 6]. 10.The method of human pose recognition based on multi-dimensional cooperative attention network according to claim 1, characterized in that: In formula (1) of step (4) of constructing the loss function of the multi-dimensional collaborative attention network, N represents the number of samples, and N ∈ [5000, 100000].

Images