Human body posture recognition method based on multi-dimensional cooperative attention network

Abstract

The invention discloses a human body posture recognition method based on a multi-dimensional cooperative attention network. The method comprises the steps of selecting and preprocessing a data set, dividing the data set to construct a multi-dimensional cooperative attention neural network, training the multi-dimensional cooperative attention neural network, testing the multi-dimensional cooperative attention neural network and evaluating the performance of the multi-dimensional cooperative attention neural network. According to the method, the neural network is adopted, and joint features among different channels are effectively aggregated while the calculation complexity is reduced through channel refinement convolution and multi-dimensional cooperative attention. The environment sensing module of the constructed neural network improves the receptive field to capture more comprehensive environment information. The method has the advantages of relatively simple network structure, relatively good network performance, relatively wide application range and the like, and can be applied to a human body posture recognition system, a dance teaching system and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to human body posture recognition. Background Art

[0002] Human posture recognition, also known as human key point detection, is the recognition of the positions of key human points such as elbows, shoulders, eyes, knees, etc. in the image, and the connection of the identified key parts according to the relationship between human limbs to form a human skeleton model. It is one of the hot topics in the field of computer vision in recent years. Solving the problem of human posture recognition can provide information for human fall detection, intelligent monitoring, human behavior recognition, human-computer interaction, etc.

[0003] In recent years, with the rise of deep learning and the application of convolutional neural networks in a series of image processing, the field of human posture recognition has developed rapidly, but there are difficult to solve technical problems in background occlusion, complex posture, poor lighting conditions, and multi-person posture recognition. The stacked hourglass network (SHN) adopts a very classic U-shaped architecture and performs posture recognition end-to-end. The cascaded pyramid network (CPN) is divided into two steps: GlobalNet and RefineNet. It first identifies simple key points, and then integrates the identified feature representations to identify complex key points, layer by layer. However, the above methods are difficult to meet the accuracy requirements of practical applications. Compared with the traditional downsampling network structure, the high-resolution network (HRNet) always maintains high-resolution feature maps as the main line of operation. When downsampling, a new low-resolution sub-network will be generated. Under the premise of maintaining the high-resolution features of the image from beginning to end, multiple parallel sub-networks are integrated to further balance the accuracy and computational complexity, but when facing complex background conditions, there is still a technical problem of low accuracy. ViTPose proposed in 2022 effectively improves the detection accuracy, but at the same time brings an extremely high computational burden, and has high requirements on the computing load capacity of the device. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a human posture recognition method based on a multi-dimensional collaborative attention network with good network performance and a wide range of applications.

[0005] The technical solution adopted to solve the above technical problems consists of the following steps:

[0006] (1) Selecting and preprocessing datasets

[0007] The COCO2017 dataset released by Microsoft was selected, which contains 2,500,000 labeled human instances. Each pedestrian sample is labeled with 17 key points of the human body: nose a×1, eyes b×2, ears c×2, shoulders d×2, elbows e×2, wrists f×2, hips g×2, knees h×2, and ankles i×2.

[0008] The COCO2017 dataset is preprocessed as follows:

[0009] 1) Crop the image into 256×192 pixels.

[0010] 2) Randomly flip the dataset and scale it by 0.25 times.

[0011] (2) Dividing the Dataset

[0012] The dataset is divided into training set, validation set, and test set according to the ratio of 6:2:2.

[0013] (3) Constructing a multi-dimensional collaborative attention neural network

[0014] The multi-dimensional collaborative attention neural network is composed of a convolutional layer 1 and a bottleneck module, a feature fusion module 1, a feature fusion module 2, a feature fusion module 3, and an environment perception module connected in series.

[0015] The bottleneck module is composed of convolution layer 2, batch normalization layer 1, ReLU activation function layer 1, channel refinement convolution block, batch normalization layer 2, ReLU activation function layer 2, convolution layer 3, batch normalization layer 3, and multi-dimensional collaborative attention layer. Convolution layer 2 is connected in series with batch normalization layer 1, ReLU activation function layer 1, channel refinement convolution block, batch normalization layer 2, ReLU activation function layer 2, convolution layer 3, and batch normalization layer 3 in sequence. One output end of convolution layer 2 is connected to the other input end of batch normalization layer 3, and the other output end is connected to the other input end of the channel refinement convolution block through the multi-dimensional collaborative attention layer.

[0016] (4) Training a multi-dimensional collaborative attention network

[0017] 1) Construct loss function

[0018] Construct the loss function Loss as follows:

[0019]

[0020] Among them, y i is the true value, h(x i ) is the output vector of the network, N is the number of samples, and N is a finite positive integer.

[0021] 2) Training a multi-dimensional collaborative attention network

[0022] The training set is input into the multi-dimensional collaborative attention network for training. The training cycle is 210, the batch size is 32, the initial learning rate is set to 0.001, and it becomes 0.0001 after 170 iterations. The training is carried out until the loss function Loss converges.

[0023] (5) Testing multi-dimensional collaborative attention networks

[0024] The test set is input into the trained multi-dimensional collaborative attention network for testing, and the human posture detection results of the key points a, b, c, d, e, f, d, h, i are output.

[0025] (6) Evaluating the performance of multi-dimensional collaborative attention networks

[0026] The key point similarity evaluation index is used to evaluate the effect of the multi-dimensional collaborative attention network on the test set.

[0027] Determine the key point similarity OKS as follows:

[0028]

[0029] in, represents the square of the Euclidean distance between the i-th predicted key point position and the corresponding GT information, s 2 represents the target area, k i represents the normalization factor of the key point with id type i, δ(v i ) represents the function that selects visible points for calculation, v i It indicates the flag of the i-th real key point provided by GT information, P is the number of key points of the human body, and the value of P is 17.

[0030] In step (3) of the present invention, a multi-dimensional collaborative attention neural network is constructed, wherein the channel refinement convolution block is composed of a linear transformation layer, an average pooling layer, a channel refinement layer, a Tanh polarization function layer 3, a convolution layer 4, and a convolution layer 5 connected in series. The linear transformation layer is connected in series with the average pooling layer, the channel refinement layer, the Tanh polarization function layer 3, and the convolution layer 4 in sequence. The input end of the convolution layer 5 is connected to another output end of the linear transformation layer, and the output end is connected to another input end of the convolution layer 4.

[0031] In step (3) of constructing a multi-dimensional collaborative attention neural network of the present invention, the environment perception module is composed of a large-kernel convolution module and a multi-scale convolution module connected in series.

[0032] The multi-scale convolution module of the present invention is composed of a convolution layer 6, a convolution layer 7, a convolution layer 8, a convolution layer 9, a convolution layer 10, a convolution layer 11, a convolution layer 12, a convolution layer 13, and a convolution layer 14. The convolution layer 6 is connected in series with the convolution layer 7, the convolution layer 10, the convolution layer 11, and the convolution layer 14 in sequence. An output end of the convolution layer 7 is connected to an input end of the convolution layer 14 through the convolution layer 8 and the convolution layer 9, and the other output end of the convolution layer 7 is connected to the other input end of the convolution layer 14 through the convolution layer 12 and the convolution layer 13.

[0033] The convolution kernel size of the convolution layer 6 of the present invention is 1×1, and the step size is 1, the convolution kernel size of the convolution layer 7 is 5×5, and the step size is 2, the convolution kernel size of the convolution layer 8 is 1×7, and the step size is 2, the convolution kernel size of the convolution layer 9 is 7×1, and the step size is 2, the convolution kernel size of the convolution layer 10 is 1×11, and the step size is 2, the convolution kernel size of the convolution layer 11 is 11×1, and the step size is 2, the convolution kernel size of the convolution layer 12 is 1×21, and the step size is 2, the convolution kernel size of the convolution layer 13 is 21×1, and the step size is 2, and the convolution kernel size of the convolution layer 14 is 1×1, and the step size is 2.

[0034] The construction method of the channel refinement convolution block of the present invention is as follows:

[0035] Construct the channel refinement layer M1 according to formula (1):

[0036] M1=σ(ψ(x)-φ(x)) (1)

[0037] Among them, σ represents the Tanh activation function, ψ(x) and φ(x) represent linear transformation operations, r represents the channel dimension reduction coefficient, It represents the number of channels of the feature map after dimensionality reduction, and W×1 and 1×W represent the height×width of the feature map.

[0038] In step (3) of the present invention, in constructing a multi-dimensional collaborative attention neural network, the method for constructing the large kernel convolution module consists of the following steps:

[0039] 1) Feature vector channel fusion

[0040] According to formula (2), feature vector channel fusion is performed:

[0041]

[0042] Among them, U j represents a branch in parallel, j is a finite positive integer. Represents a convolution kernel of size 1×1.

[0043] 2) Feature stitching

[0044] Determine feature splicing according to formula (3):

[0045]

[0046] 3) Processing the spatial attention feature map

[0047] The spatial attention feature map is processed according to formula (4):

[0048] SA=F{SA avg , S.A. max} (4)

[0049]

[0050] Among them, SA avg , S.A. max is the intermediate variable, They represent average pooling and maximum pooling respectively, and F represents a convolution kernel with a size of 1×1 and the number of channels equal to the number of branches.

[0051] 4) Output large kernel convolution module features

[0052] According to formula (5), the large kernel convolution module feature Y is output:

[0053] Y=X×S (5)

[0054]

[0055] SA j =σ(SA j )

[0056] Among them, X represents the input feature, M represents the number of parallel branches, M is a finite positive integer, σ(SA i ) represents the sigmoid activation function.

[0057] In step 1) of the feature vector channel fusion formula (2), the U j Indicates parallel branches, and the value of j ranges from 1 to 6.

[0058] In step 4), in formula (5) for outputting the large kernel convolution module feature, the M represents the number of parallel branches, M∈[1,6].

[0059] In the expression of 1) constructing the loss function of step (4) of the present invention for training the multi-dimensional collaborative attention network, N is the number of samples, N∈[5000,100000].

[0060] Compared with the prior art, the present invention has the following advantages:

[0061] Since the present invention adopts an improved bottleneck module, the network model can better aggregate the joint features between different channels, reduce the computational complexity, generate the most expected data, and facilitate the subsequent network training and optimization, so that the model can obtain better results.

[0062] The present invention uses an environmental perception module, which enables the model to flexibly integrate features, capture more comprehensive environmental information, and improve the recognition rate and detection accuracy of images under complex backgrounds. The present invention has the advantages of good network performance, wide application range, high recognition rate and detection accuracy of images, and can be used for the recognition of human action images. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of embodiment 1 of the present invention.

[0064] Figure 2 It is a schematic diagram of the structure of the multi-dimensional collaborative attention neural network of the present invention.

[0065] Figure 3 yes Figure 2 Schematic diagram of the bottleneck module.

[0066] Figure 4 yes Figure 3 Schematic diagram of the structure of the mid-channel refinement convolutional block.

[0067] Figure 5 yes Figure 2 Schematic diagram of the structure of the environment perception module.

[0068] Figure 6 yes Figure 5 Schematic diagram of the structure of the multi-scale convolution module. DETAILED DESCRIPTION

[0069] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the following embodiments.

[0070] Example 1

[0071] exist Figure 1 In the embodiment, the human body posture recognition method based on multi-dimensional collaborative attention network consists of the following steps:

[0072] (1) Selecting and preprocessing datasets

[0073] The COCO2017 dataset released by Microsoft was selected, which contains 2,500,000 labeled human instances. Each pedestrian sample is labeled with 17 key points of the human body: nose a×1, eyes b×2, ears c×2, shoulders d×2, elbows e×2, wrists f×2, hips g×2, knees h×2, and ankles i×2.

[0074] The COCO2017 dataset is preprocessed as follows:

[0075] 1) Crop the image into 256×192 pixels.

[0076] 2) Randomly flip the dataset and scale it by 0.25 times.

[0077] (2) Dividing the Dataset

[0078] The dataset is divided into training set, validation set, and test set according to the ratio of 6:2:2.

[0079] (3) Constructing a multi-dimensional collaborative attention neural network

[0080] Figure 2 A schematic diagram of the structure of the multi-dimensional collaborative attention neural network of this embodiment is given. Figure 2 In the embodiment, the multi-dimensional collaborative attention neural network is composed of a convolutional layer 1 and a bottleneck module, a feature fusion module 1, a feature fusion module 2, a feature fusion module 3, and an environmental perception module connected in series in sequence.

[0081] Figure 3 Given Figure 2 Schematic diagram of the bottleneck module. Figure 3 In the embodiment, the bottleneck module is composed of convolution layer 2, batch normalization layer 1, ReLU activation function layer 1, channel refinement convolution block, batch normalization layer 2, ReLU activation function layer 2, convolution layer 3, batch normalization layer 3, and multi-dimensional collaborative attention layer. Convolution layer 2 is connected in series with batch normalization layer 1, ReLU activation function layer 1, channel refinement convolution block, batch normalization layer 2, ReLU activation function layer 2, convolution layer 3, and batch normalization layer 3 in sequence. One output end of convolution layer 2 is connected to another input end of batch normalization layer 3, and the other output end is connected to another input end of the channel refinement convolution block through the multi-dimensional collaborative attention layer.

[0082] Since the present invention adopts an improved bottleneck module, the network model can better aggregate the joint features between different channels, reduce the computational complexity, generate the most expected data, and facilitate the subsequent network training and optimization, so that the model can obtain better results.

[0083] Figure 4 Given Figure 3 Schematic diagram of the structure of the channel refinement convolution block. Figure 4In the embodiment, the channel refinement convolution block is composed of a linear transformation layer, an average pooling layer, a channel refinement layer, a Tanh polarization function layer 3, a convolution layer 4, and a convolution layer 5. The linear transformation layer is connected in series with the average pooling layer, the channel refinement layer, the Tanh polarization function layer 3, and the convolution layer 4 in sequence. The input end of the convolution layer 5 is connected to the other output end of the linear transformation layer, and the output end is connected to the other input end of the convolution layer 4.

[0084] The construction method of the channel refinement convolution block is as follows:

[0085] Construct the channel refinement convolution block M1 according to formula (1):

[0086] M1=σ(ψ(x)-φ(x)) (1)

[0087] Among them, σ represents the Tanh activation function, ψ(x) and φ(x) represent linear transformation operations, r represents the channel dimension reduction coefficient, It represents the number of channels of the feature map after dimensionality reduction, and W×1 and 1×W represent the height×width of the feature map.

[0088] Figure 5 Given Figure 2 Schematic diagram of the structure of the environment perception module. Figure 5 In the embodiment, the environment perception module is composed of a large-kernel convolution module and a multi-scale convolution module connected in series.

[0089] The construction method of the large kernel convolution module of this embodiment consists of the following steps:

[0090] 1) Feature vector channel fusion

[0091] According to formula (2), feature vector channel fusion is performed:

[0092]

[0093] Among them, U j Indicates a branch connected in parallel, the value range of j is 1 to 6, and the value range of j in this embodiment is 4. Represents a convolution kernel of size 1×1.

[0094] 2) Feature stitching

[0095] Determine feature splicing according to formula (3):

[0096]

[0097] 3) Processing the spatial attention feature map

[0098] The spatial attention feature map is processed according to formula (4):

[0099] SA=F{SA avg SA max} (4)

[0100]

[0101] Among them, SA avg , S.A. max is the intermediate variable, They represent average pooling and maximum pooling respectively, and F represents a convolution kernel with a size of 1×1 and the number of channels equal to the number of branches;

[0102] 4) Output large kernel convolution module features

[0103] According to formula (5), the large kernel convolution module feature Y is output:

[0104] Y=X×S (5)

[0105]

[0106] SA j =σ(SA j )

[0107] Where X represents the input feature, M represents the number of parallel branches, M∈[1,6], and the value of M in this embodiment is 4. i ) represents the sigmoid activation function.

[0108] Figure 6 Given Figure 5 Schematic diagram of the structure of the multi-scale convolution module in Figure 6 In the figure, the multi-scale convolution module of the present embodiment is composed of convolution layer 6, convolution layer 7, convolution layer 8, convolution layer 9, convolution layer 10, convolution layer 11, convolution layer 12, convolution layer 13, and convolution layer 14. Convolution layer 6 is connected in series with convolution layer 7, convolution layer 10, convolution layer 11, and convolution layer 14 in sequence. An output end of convolution layer 7 is connected to an input end of convolution layer 14 through convolution layer 8 and convolution layer 9, and the other output end of convolution layer 7 is connected to the other input end of convolution layer 14 through convolution layer 12 and convolution layer 13.

[0109] In this embodiment, the convolution kernel size of the convolution layer 6 is 1×1 and the step size is 1, the convolution kernel size of the convolution layer 7 is 5×5 and the step size is 2, the convolution kernel size of the convolution layer 8 is 1×7 and the step size is 2, the convolution kernel size of the convolution layer 9 is 7×1 and the step size is 2, the convolution kernel size of the convolution layer 10 is 1×11 and the step size is 2, the convolution kernel size of the convolution layer 11 is 11×1 and the step size is 2, the convolution kernel size of the convolution layer 12 is 1×21 and the step size is 2, the convolution kernel size of the convolution layer 13 is 21×1 and the step size is 2, and the convolution kernel size of the convolution layer 14 is 1×1 and the step size is 2.

[0110] Since the present invention uses an environmental perception module, the model can flexibly integrate features, capture more comprehensive environmental information, and improve the recognition rate and detection accuracy of images under complex backgrounds.

[0111] The construction method of the channel refinement convolution block in this embodiment is as follows:

[0112] Construct the channel refinement convolution block M1 according to formula (1):

[0113] M1=σ(ψ(x)-φ(x)) (1)

[0114] Among them, σ represents the Tanh activation function, ψ(x) and φ(x) represent linear transformation operations, r represents the channel dimension reduction coefficient, It represents the number of channels of the feature map after dimensionality reduction, and W×1 and 1×W represent the height×width of the feature map.

[0115] (4) Training a multi-dimensional collaborative attention network

[0116] 1) Construct loss function

[0117] Construct the loss function Loss as follows:

[0118]

[0119] Among them, y i is the true value, h(x i ) is the output vector of the network, N is the number of samples, N∈[5000,100000], and the value of N in this embodiment is 50000.

[0120] 2) Training a multi-dimensional collaborative attention network

[0121] The training set is input into the multi-dimensional collaborative attention network for training. The training cycle is 210, the batch size is 32, the initial learning rate is set to 0.001, and it is changed to 0.0001 after 170 iterations. The training is carried out until the loss function Loss converges;

[0122] (5) Testing multi-dimensional collaborative attention networks

[0123] The test set is input into the trained multi-dimensional collaborative attention network for testing, and the human posture detection results of the key points a, b, c, d, e, f, j, h, i are output.

[0124] (6) Evaluating the performance of multi-dimensional collaborative attention networks

[0125] Use the key point similarity evaluation index to evaluate the effect of the multi-dimensional collaborative attention network on the test set:

[0126] Determine the key point similarity OKS as follows:

[0127]

[0128] Among them, d i 2 represents the square of the Euclidean distance between the i-th predicted key point position and the corresponding GT information, s 2 represents the target area, k i represents the normalization factor of the key point with id type i, δ(v i ) represents the function that selects visible points for calculation, v i Indicates the flag bit of the i-th true key point provided by GT information.

[0129] Completed the human posture recognition method based on multi-dimensional collaborative attention network.

[0130] Example 2

[0131] The human body posture recognition method based on multi-dimensional collaborative attention network of this embodiment consists of the following steps:

[0132] (1) Selecting and preprocessing datasets

[0133] This step is the same as in Example 1.

[0134] (2) Dividing the Dataset

[0135] This step is the same as in Example 1.

[0136] (3) Constructing a multi-dimensional collaborative attention neural network

[0137] The multi-dimensional collaborative attention neural network structure is the same as that in Example 1.

[0138] The construction method of the channel refinement convolution block in this embodiment is the same as that in Embodiment 1.

[0139] The environment perception module of this embodiment is composed of a large-core convolution module and a multi-scale convolution module connected in series. The construction method of the large-core convolution module consists of the following steps:

[0140] 1) Feature vector channel fusion

[0141] According to formula (2), feature vector channel fusion is performed:

[0142] The expression of formula (2) is the same as that of Example 1.

[0143] In formula (2), U j Indicates a branch connected in parallel, the value of j ranges from 1 to 6, and the value of j in this embodiment is 1.

[0144] 2) Feature stitching

[0145] This step is the same as in Example 1.

[0146] 3) Processing the spatial attention feature map

[0147] This step is the same as in Example 1.

[0148] 4) Output large kernel convolution module features

[0149] According to formula (5), the large kernel convolution module feature Y is output:

[0150] The expression of formula (5) is the same as that of Example 1.

[0151] In formula (5), M represents the number of branches connected in parallel, M∈[1,6]. In this embodiment, the value of M is 1. The meanings and values ​​of other parameters are the same as those in embodiment 1.

[0152] (4) Training a multi-dimensional collaborative attention network

[0153] 1) Construct loss function

[0154] Construct the loss function Loss as follows:

[0155] The expression of the loss function Loss is the same as that in Example 1.

[0156] In the expression of the loss function Loss, N is the number of samples, N∈[5000,100000], and the value of N in this embodiment is 5000. The meanings and values ​​of other parameters are the same as those in Embodiment 1.

[0157] The other steps are the same as those in Example 1. A human posture recognition method based on a multi-dimensional collaborative attention network is completed.

[0158] Example 3

[0159] The human body posture recognition method based on multi-dimensional collaborative attention network of this embodiment consists of the following steps:

[0160] (1) Selecting and preprocessing datasets

[0161] This step is the same as in Example 1.

[0162] (2) Dividing the Dataset

[0163] This step is the same as in Example 1.

[0164] (3) Constructing a multi-dimensional collaborative attention neural network

[0165] The multi-dimensional collaborative attention neural network structure is the same as that in Example 1.

[0166] The construction method of the channel refinement convolution block in this embodiment is the same as that in Embodiment 1.

[0167] The environment perception module of this embodiment is composed of a large-core convolution module and a multi-scale convolution module connected in series. The construction method of the large-core convolution module consists of the following steps:

[0168] 1) Feature vector channel fusion

[0169] According to formula (2), feature vector channel fusion is performed:

[0170] The expression of formula (2) is the same as that of Example 1.

[0171] In formula (2), U j Indicates a branch connected in parallel. The value of j ranges from 1 to 6. In this embodiment, the value of j is 6.

[0172] 2) Feature stitching

[0173] This step is the same as in Example 1.

[0174] 3) Processing the spatial attention feature map

[0175] This step is the same as in Example 1.

[0176] 4) Output large kernel convolution module features

[0177] According to formula (5), the large kernel convolution module feature Y is output:

[0178] The expression of formula (5) is the same as that of Example 1.

[0179] In formula (5), M represents the number of branches connected in parallel, M∈[1,6]. The value of M in this embodiment is 6. The meanings and values ​​of other parameters are the same as those in embodiment 1.

[0180] (4) Training a multi-dimensional collaborative attention network

[0181] 1) Construct loss function

[0182] Construct the loss function Loss as follows:

[0183] The expression of the loss function Loss is the same as that in Example 1.

[0184] In the expression of the loss function Loss, N is the number of samples, N∈[5000,100000], and the value of N in this embodiment is 100000. The meanings and values ​​of other parameters are the same as those in Embodiment 1.

[0185] The other steps are the same as those in Example 1. A human posture recognition method based on a multi-dimensional collaborative attention network is completed.

[0186] In order to verify the beneficial effects of the present invention, the inventors conducted a laboratory simulation comparison experiment using the human posture recognition method based on a multi-dimensional collaborative attention network of Example 1 of the present invention. The various experimental conditions are as follows:

[0187] 1. Experimental environment

[0188] Experimental environment: The processor is AMD EPYC 9754, the graphics card is NVIDIA GeForce RTX 3080Ti, the operating system type is 64-bit, Python 3.8 is used, and the deep learning framework is PyTorch1.8.1.

[0189] 2. Experimental content and results

[0190] A test box comparison test was conducted using Example 1 of the present invention (hereinafter referred to as the present invention) and an unbiased data processing method (hereinafter referred to as comparison method 1), a human posture estimation method using a high-resolution network (hereinafter referred to as comparison method 2), a human posture estimation method using a lightweight high-resolution network-18 (hereinafter referred to as comparison method 3), and a human posture estimation method using a lightweight high-resolution network-30 (hereinafter referred to as comparison method 4). The evaluation indicators include: average accuracy (hereinafter referred to as AP), accuracy when the key point similarity OKS is 0.5 (hereinafter referred to as AP 50 ). The accuracy (AP for short) when the key point similarity OKS is 0.75 75 ), medium-scale target accuracy (AP for short) M ), the accuracy of large-scale targets (AP for short) L ), Average Recall Rate (AR for short). The test results are shown in Table 1, and the comparative experimental results are shown in Table 2.

[0191] Table 1 Test set results of the method of the present invention and the comparative method

[0192] Experimental methods Input Image AP(%) <![CDATA[AP 50 (%)]]> <![CDATA[AP 75 (%)]]> <![CDATA[AP M (%)]]> <![CDATA[AP L (%)]]> AR(%) Comparison method 1 384ⅹ288 76.1 92.5 83.5 72.8 82.0 81.3 Comparison Method 2 384ⅹ288 74.9 92.5 82.8 71.3 80.9 80.1 The present invention 384ⅹ288 76.5 93.7 83.6 73.9 80.7 80.5

[0193] Table 2 Comparative simulation results of the method of the present invention and the comparative method

[0194] Experimental methods Input Image AP(%) <![CDATA[AP 50 (%)]]> <![CDATA[AP 75 (%)]]> <![CDATA[AP M (%)]]> <![CDATA[AP L (%)]]> AR(%) Comparison method 1 256ⅹ192 75.2 92.4 82.9 72.0 80.8 80.4 Comparison Method 2 256ⅹ192 73.4 89.5 80.7 70.2 80.1 78.9 Comparison method 3 256ⅹ192 64.8 86.7 73.0 62.1 70.5 71.2 Comparison method 4 256ⅹ192 67.2 88.0 75.0 64.3 73.1 73.3 The present invention 256ⅹ192 75.6 93.2 82.6 73.1 79.8 79.6

[0195] It can be seen from Tables 1 and 2 that the present invention is higher than the comparative experimental method in ensuring the average recall rate, large-scale target accuracy, accuracy when the key point similarity OKS is 0.75, average accuracy, accuracy when the key point similarity OKS is 0.5, and medium-scale target accuracy.

Claims

1. A human posture recognition method based on a multi-dimensional collaborative attention network, characterized by It consists of the following steps: (1) Selecting and preprocessing datasets The COCO2017 dataset released by Microsoft is selected, which contains 2,500,000 labeled human instances. Each pedestrian sample is labeled with 17 key points of the human body: nose a×1, eyes b×2, ears c×2, shoulders d×2, elbows e×2, wrists f×2, hips g×2, knees h×2, and ankles i×2. The COCO2017 dataset is preprocessed as follows: 1) Crop the image into 256×192 pixels; 2) Randomly flip the data set and scale it by 0.25 times; (2) Dividing the Dataset The data set is divided into training set, validation set, and test set according to 6:2:2; (3) Constructing a multi-dimensional collaborative attention neural network The multi-dimensional collaborative attention neural network is composed of convolution layer 1 and bottleneck module, feature fusion module 1, feature fusion module 2, feature fusion module 3, and environment perception module connected in series in sequence; The bottleneck module is composed of a convolution layer 2, a batch normalization layer 1, a ReLU activation function layer 1, a channel refinement convolution block, a batch normalization layer 2, a ReLU activation function layer 2, a convolution layer 3, a batch normalization layer 3, and a multi-dimensional collaborative attention layer. The convolution layer 2 is connected in series with the batch normalization layer 1, the ReLU activation function layer 1, the channel refinement convolution block, the batch normalization layer 2, the ReLU activation function layer 2, the convolution layer 3, and the batch normalization layer 3 in sequence. One output end of the convolution layer 2 is connected to another input end of the batch normalization layer 3, and the other output end is connected to another input end of the channel refinement convolution block through the multi-dimensional collaborative attention layer. (4) Training a multi-dimensional collaborative attention network 1) Construct loss function Construct the loss function Loss as follows: Among them, y i is the true value, h(x i ) is the output vector of the network, N is the number of samples, and N is a finite positive integer; 2) Training a multi-dimensional collaborative attention network The training set is input into the multi-dimensional collaborative attention network for training. The training cycle is 210, the batch size is 32, the initial learning rate is set to 0.001, and it is changed to 0.0001 after 170 iterations. The training is carried out until the loss function Loss converges; (5) Testing multi-dimensional collaborative attention networks The test set is input into the trained multi-dimensional collaborative attention network for testing, and the human posture detection results of the key points a, b, c, d, e, f, d, h, i are output; (6) Evaluating the performance of multi-dimensional collaborative attention networks Use the key point similarity evaluation index to evaluate the effect of the multi-dimensional collaborative attention network on the test set: Determine the key point similarity OKS as follows: in, represents the square of the Euclidean distance between the i-th predicted key point position and the corresponding GT information, s 2 represents the target area, k i represents the normalization factor of the key point with id type i, δ(v i ) represents the function that selects visible points for calculation, v i It indicates the flag of the i-th real key point provided by GT information, P is the number of key points of the human body, and the value of P is 17.

2. The method for human posture recognition based on multi-dimensional collaborative attention network according to claim 1, characterized in that: In step (3) of constructing a multi-dimensional collaborative attention neural network, the channel refinement convolution block is composed of a linear transformation layer, an average pooling layer, a channel refinement layer, a Tanh polarization function layer 3, a convolution layer 4, and a convolution layer 5. The linear transformation layer is connected in series with the average pooling layer, the channel refinement layer, the Tanh polarization function layer 3, and the convolution layer 4 in sequence. The input end of the convolution layer 5 is connected to the other output end of the linear transformation layer, and the output end is connected to the other input end of the convolution layer 4.

3. The method for human posture recognition based on multi-dimensional collaborative attention network according to claim 1, characterized in that: In step (3) of constructing a multi-dimensional collaborative attention neural network, the environment perception module is composed of a large-kernel convolution module and a multi-scale convolution module connected in series.

4. The method for human posture recognition based on multi-dimensional collaborative attention network according to claim 3, characterized in that: The multi-scale convolution module is composed of convolution layer 6, convolution layer 7, convolution layer 8, convolution layer 9, convolution layer 10, convolution layer 11, convolution layer 12, convolution layer 13, and convolution layer 14. Convolution layer 6 is connected in series with convolution layer 7, convolution layer 10, convolution layer 11, and convolution layer 14 in sequence. An output end of convolution layer 7 is connected to an input end of convolution layer 14 through convolution layer 8 and convolution layer 9, and the other output end of convolution layer 7 is connected to the other input end of convolution layer 14 through convolution layer 12 and convolution layer 13.

5. The method for human posture recognition based on multi-dimensional collaborative attention network according to claim 4, characterized in that: The convolution kernel size of the convolution layer 6 is 1×1, and the step size is 1, the convolution kernel size of the convolution layer 7 is 5×5, and the step size is 2, the convolution kernel size of the convolution layer 8 is 1×7, and the step size is 2, the convolution kernel size of the convolution layer 9 is 7×1, and the step size is 2, the convolution kernel size of the convolution layer 10 is 1×11, and the step size is 2, the convolution kernel size of the convolution layer 11 is 11×1, and the step size is 2, the convolution kernel size of the convolution layer 12 is 1×21, and the step size is 2, the convolution kernel size of the convolution layer 13 is 21×1, and the step size is 2, and the convolution kernel size of the convolution layer 14 is 1×1, and the step size is 2.

6. The human posture recognition method based on multi-dimensional collaborative attention network according to claim 2 is characterized in that The construction method of the channel refinement convolution block is as follows: Construct the channel refinement layer M1 according to formula (1): M1=σ(ψ(x)-φ(x)) (1) Among them, σ represents the Tanh activation function, ψ(x) and φ(x) represent linear transformation operations, r represents the channel dimension reduction coefficient, It represents the number of channels of the feature map after dimensionality reduction, and W×1 and 1×W represent the height×width of the feature map.

7. The method for human posture recognition based on multi-dimensional collaborative attention network according to claim 1, characterized in that: In step (3) of constructing a multi-dimensional collaborative attention neural network, the method for constructing the large kernel convolution module consists of the following steps: 1) Feature vector channel fusion According to formula (2), feature vector channel fusion is performed: Among them, U j represents a branch in parallel, j is a finite positive integer. Represents a convolution kernel of size 1×1; 2) Feature stitching Determine feature splicing according to formula (3): 3) Processing the spatial attention feature map The spatial attention feature map is processed according to formula (4): IN=F{IN avg ,IN max } (4) Among them, SA avg , S.A. max is the intermediate variable, They represent average pooling and maximum pooling respectively, and F represents a convolution kernel with a size of 1×1 and the number of channels equal to the number of branches; 4) Output large kernel convolution module features According to formula (5), the large kernel convolution module feature Y is output: Y=X×S (5) IN j =σ(SA j ) Among them, X represents the input feature, M represents the number of parallel branches, M is a finite positive integer, σ(SA i ) represents the sigmoid activation function.

8. The method for human posture recognition based on multi-dimensional collaborative attention network according to claim 7, characterized in that: In step 1) of the feature vector channel fusion formula (2), the U j Indicates parallel branches, and the value of j ranges from 1 to 6.

9. The method for human posture recognition based on multi-dimensional collaborative attention network according to claim 7, characterized in that: In step 4), in formula (5) for outputting the large kernel convolution module feature, the M represents the number of parallel branches, M∈[1,6].

10. The method for human posture recognition based on multi-dimensional collaborative attention network according to claim 1, characterized in that: In the expression of 1) constructing the loss function of step (4) training the multi-dimensional collaborative attention network, N is the number of samples, N∈[5000,100000].

Images