Three-dimensional human body posture estimation method based on attention map convolution

Abstract

The invention relates to a three-dimensional human body posture estimation method based on attention map convolution, and belongs to the technical field of human body three-dimensional posture estimation, and the method comprises the following steps: S1, carrying out the two-dimensional posture estimation of a human body image through employing a cascaded pyramid network (CPN); s2, extracting the global spatial features by using a Transform attention mechanism based on the attention of the Transform; s3, extracting local spatial features by using a graph convolutional network; and S4, fusing the global spatial features and the local spatial features to obtain a final three-dimensional attitude coordinate. According to the invention, the accuracy of human body three-dimensional attitude estimation is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of human three-dimensional posture estimation, and relates to a three-dimensional human posture estimation method based on attention graph convolution. Background Art

[0002] As one of the most important research topics in computer vision, human pose estimation is in a period of rapid development with the development of computer vision technology and has been increasingly used in real life. Human pose estimation aims to depict the human pose by analyzing the human form in objects such as pictures and videos, which involves multiple tasks such as target recognition, image segmentation, and regression detection. Compared with traditional two-dimensional pose estimation, three-dimensional human pose estimation can describe the human pose more accurately and has higher research and application value. At present, three-dimensional human pose estimation has become a research hotspot in the field of computer vision and an important foundation for many research works. The three-dimensional pose of the human body extracted from images or videos can not only be used for action recognition, but also for tasks such as three-dimensional mesh reconstruction, providing strong support for applications in fields such as human behavior analysis, virtual reality, and augmented reality. With the continuous advancement of deep learning and computer vision technology, three-dimensional human pose estimation will show its importance and potential in a wider range of fields.

[0003] Current 3D human pose estimation methods can be roughly divided into two types: (1) Direct regression method, which directly predicts 3D pose coordinates from 2D images without first converting them into 2D pose representations. This method is capable of end-to-end network training, but has high requirements for network structure and data preprocessing. (2) 2D skeleton-based methods, which are usually divided into two stages: first, a pre-trained 2D pose estimation network is used to extract the skeleton sequence, which is then input into the 3D pose estimation network for dimensionality enhancement. Due to the maturity of existing 2D pose estimation algorithms, the 2D skeleton-based method greatly simplifies the task complexity and outperforms the direct regression method, so it has become the mainstream. This method can reduce the complexity of the network structure and is easier to apply and deploy in actual environments.

[0004] The semantic graph convolution proposed by Zhao et al. pointed out that by learning the semantic relationship between adjacent nodes implied in the edge of the graph, the limitations of the shared kernel matrix can be effectively solved. However, the problem of loss of node information caused by only calculating the features of adjacent nodes is improved, and the integrity of node features in the information transmission process is ensured. Xiao et al. improved the self-loop-free adjacency matrix into a symmetric normalized adjacency matrix with self-loops, and used different parameter transformation matrices to transform the parameters of the node itself and its adjacent nodes respectively. Xu et al. combined the idea of ​​the hourglass stacking network with the graph convolution network and proposed an hourglass-based graph convolution network, which consists of repeated encoder-decoders, in which the features of the graph structure are processed at three different scales of the human skeleton representation. This multi-scale architecture enables the model to learn local and global feature representations simultaneously, thereby better capturing the details and overall structure of human posture. These methods usually take into account the global and local information of the skeleton, but ignore the differences in the dependencies between different joints and the information of the joint points themselves. In addition, insufficient consideration of global information will also affect the accuracy of 3D posture estimation. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a 3D human posture estimation method based on attention graph convolution

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A 3D human posture estimation method based on attention graph convolution comprises the following steps:

[0008] S1: 2D pose estimation of human body images using cascaded pyramid network CPN;

[0009] S2: Extract global spatial features using the Transformer-based attention mechanism;

[0010] S3: Extract local spatial features using graph convolutional networks;

[0011] S4: Fuse the global space features and local space features to obtain the final 3D posture coordinates.

[0012] Further, step S1 specifically includes the following steps:

[0013] S11: Input the human body image into the cascade pyramid network CPN for two-dimensional posture estimation to obtain the two-dimensional joint point coordinates C i ∈R N×2 , where N is the number of joint points;

[0014] S12: The obtained two-dimensional key point coordinates are encoded using a skeleton embedding module composed of a multi-layer fully connected network to obtain a high-dimensional vector.

[0015] Furthermore, in step S2, each joint is regarded as a patch, and feature extraction is performed between all patches according to the general visual transformation pipeline. First, the coordinates of each joint are mapped to a high-dimensional space using a trainable linear projection. The obtained feature R J×C The joint sequence is input into the spatial transformer encoder, which integrates the information of all joints using a self-attention mechanism;

[0016] The Transformer attention block follows the scaled dot product attention, and the attention calculation formula for the query, key, and value matrices Q, K, V in each head is:

[0017]

[0018] where {Q,K,V}∈R J×C , N represents the number of tokens, i.e. the number of joint points, C represents the dimension of each token, and h multi-head attentions are defined as follows:

[0019] MSA=Concat(head1,head2,...,head h )W O

[0020] head i =Attention(Q i ,K i ,V i ),i∈h

[0021] The linear projection weight is W O ∈R C×C , in the attention encoder, each joint token p∈P N is from the two-dimensional coordinates C∈R N×2 Joint C i Projected, the joint token p is passed through the matrix E pos ∈R N×C Embed location information:

[0022] X=Norm(L e (C i )+E pos ),X∈R N×C

[0023] Where Norm is layer normalization, L e It is a linear embedding layer, and the spatial dependency between joint points is calculated through the attention module as follows:

[0024] Ui =XW m ,m∈{Q,K,V}

[0025] Y=MSA(U Q ,U K ,U V )+X

[0026] R=MLP(Norm(Y))+Y

[0027] Where Y is the attention output of joint token X, U i is the matrix of X mapped by linear transformation, W m is the linear transformation weight matrix corresponding to the query, key, and value in the joint, Norm is the layer normalization, and R is the final output of the global attention.

[0028] Furthermore, the joint coordinates are represented as a graph G = {V, E}, where V is a set of N joint points and E represents an edge. GCN is represented as:

[0029]

[0030] in, is the input feature of layer l, is the output feature after GCN operation, D l and D l+1 are the input feature dimension and output feature dimension of each joint, A is the adjacency matrix of the undirected graph, I is the identity matrix, (A+I) represents the adjacency matrix with self-connection, D is the degree matrix of A;

[0031] is the symmetric normalization of the self-loop adjacency matrix (A+I), is a trainable transformation matrix for transforming and extracting joint features of layer l; from the perspective of spatial domain, H (l) W (l) The input joint feature H (l) Feature extraction;

[0032] Using two different transformation matrices and W1 (l) To extract two different feature information, as shown in the following formula:

[0033]

[0034] These are two features extracted for two different types of information;

[0035] Add a learnable EA matrix M to the graph convolution structure (l), EA includes the learning of its own joints and neighboring joints, and the adjacency relationship of the EA matrix is ​​expressed as:

[0036]

[0037] where the adjacency matrix (A+I) is used as a mask, ⊙ represents the Hadamard product of two matrices, and ρ i is the softmax operator, which is used to achieve row normalization so that the sum of the edge attention weights of all j adjacent joints to joint i is equal to 1; Through the learnable matrix M (l) OK, replace it After that, the dependency between adjacent joints of the same skeleton is no longer the same; the contribution information to its own joints and the contribution information to adjacent joints are extracted separately. is decomposed into two parts, namely, the self-relation and The relationship with neighbor joints is decomposed into the following formula:

[0038]

[0039] Where ⊙ represents the Hadamard product of two matrices. and As a relationship matrix, multiply by and Aggregate the features of the node and its neighboring nodes:

[0040]

[0041] in Represents the characteristics of self-node aggregation, Features that represent the aggregation of adjacent nodes. Both features are local features.

[0042] Further, step S4 specifically includes:

[0043] The three features of each joint are fused using the Concatenate fusion method, expressed as:

[0044]

[0045] Where Cat represents the connection operation;

[0046] The output feature dimensions are then transformed to the same dimension and finally a simple MLP block with layer normalization and a linear layer is used to return the final 3D pose coordinates R 1×(N×3) .

[0047] Furthermore, the mean square error (MSE) between the estimated value and the true value is used as the loss function of the estimated joint position and the true joint position, which is defined as:

[0048]

[0049] in is the estimated 3D pose coordinate of joint i, J i is the estimated coordinate of the true 3D pose of joint i.

[0050] The beneficial effect of the present invention is that the present invention uses graph convolution and Transformer technology to extract spatial features in both local and global aspects. In acquiring local information, the different interdependencies between two joints in the human skeleton are taken into account, and a learnable EA matrix is ​​added to well learn the dependencies between its own joints and adjacent joints. In addition, the Transformer technology can better acquire the global information between skeletons. Through the skeleton information of the two modules, the accuracy of human three-dimensional posture estimation is improved to a certain extent.

[0051] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0053] Figure 1 The overall block diagram of 3D human pose estimation. DETAILED DESCRIPTION

[0054] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0055] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0056] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0057] like Figure 1 As shown, the present invention provides a 3D human body posture estimation method based on attention graph convolution, which uses graph convolution and Transformer technology to extract spatial features locally and globally, and improves the accuracy of human body 3D posture estimation to a certain extent. In order to implement the above method, the present invention provides the following 5 modules:

[0058] 1. Two-dimensional key point extraction module:

[0059] The present invention uses the existing two-dimensional pose estimation detector cascade pyramid network (CPN) to perform two-dimensional pose estimation on the image. First, the image is input into the cascade pyramid network (CPN) for two-dimensional pose estimation to obtain the two-dimensional joint point coordinates C i ∈R N×2 , where N is the number of joint points, which is set to 17. Then, the obtained two-dimensional key point coordinates are encoded using a skeleton embedding module (composed of a multi-layer fully connected network) to obtain a high-dimensional vector.

[0060] 2. Transformer-based global information extraction module:

[0061] Each joint (i.e., two coordinates) is considered as a patch, and feature extraction is performed between all patches following the general visual transformation pipeline. First, the coordinates of each joint are mapped to a high-dimensional space using a trainable linear projection, which is called spatial patch embedding. The obtained feature R J×C The joint sequence is input into the spatial transformer encoder, which integrates the information of all joints using a self-attention mechanism.

[0062] The Transformer attention block follows the scaled dot product attention. The attention calculation formula for the query, key, and value matrices Q, K, and V in each head is:

[0063]

[0064] where {Q,K,V}∈R J×C , N represents the number of tokens, i.e. the number of joint points, and C represents the dimension of each token. The h multi-head attention is defined as follows:

[0065] MSA=Concat(head1,head2,...,head h )W O

[0066] head i =Attention(Q i ,K i ,V i ),i∈h

[0067] The linear projection weight is W O ∈R C×C In the attention encoder of this method, each joint token p∈P N is from the two-dimensional coordinates C∈R N×2 Joint C i The joint token p is projected through the matrix E pos ∈R N×C Embed location information:

[0068] X=Norm(L e (C i )+E pos ),X∈R N×C

[0069] Where Norm is layer normalization, L e is a linear embedding layer. The spatial dependency between joint points is calculated through the attention module as follows:

[0070] U i =XW m ,m∈{Q,K,V}

[0071] Y=MSA(U Q ,U K ,U V )+X

[0072] R=MLP(Norm(Y))+Y

[0073] Where Y is the attention output of joint token X, U i is the matrix of X mapped by linear transformation, W m is the linear transformation weight matrix corresponding to the query, key, and value in the joint. Norm is the layer normalization, and R is the final output of the global attention.

[0074] 3. Local information extraction module based on graph convolution:

[0075] Since the skeleton structure is a topological graph with fixed connection relationships, this part of the work is carried out based on the spatial method of graph convolution. The 17 joint coordinates obtained above can be represented as a graph G = {V, E}, where V is a set of N joint points and E represents the edge. The traditional GCN can be expressed as:

[0076]

[0077] in, is the input feature of layer l, is the output feature after GCN operation. l and D l+1 are the input feature dimension and output feature dimension of each joint respectively. A is the adjacency matrix of the undirected graph, I is the identity matrix, so (A+I) represents the adjacency matrix with self-connection. D is the degree matrix of A.

[0078] is the symmetric normalization of the self-loop adjacency matrix (A+I), is a trainable transformation matrix for transforming and extracting joint features of layer l. From the perspective of spatial domain, H (l) W (l) It can be understood as the input joint feature H (l) feature extraction.

[0079] The coordination relationship of the human body, the information contributed by this node to other global joints should also be different. Only one transformation matrix W is used (l) It is unreasonable to extract all these features without distinguishing these three different information. Different transformation matrices W (l) Different information can be extracted. To this end, a structure that uses multiple transformation matrices to extract features from multiple branches is proposed to solve the above problem. There are two different transformation matrices used to learn how to extract two different feature information, as shown in the following formula:

[0080]

[0081] These are two features extracted for two different types of information.

[0082] Considering that the physiological structure of the human body leads to different dependencies between adjacent nodes of the same bone, a learnable EA matrix M is added to the traditional graph convolution structure. (l) , EA includes the learning of its own joints and neighboring joints. The adjacency relationship of the EA matrix can be expressed as:

[0083]

[0084] where the adjacency matrix (A+I) is used as a mask, ⊙ represents the Hadamard product of two matrices, and ρ i is the softmax operator, which is used to achieve row normalization, so that the sum of the edge attention weights of all j adjacent joints to joint i is equal to 1. is the learnable matrix M (l) Determined, so no longer symmetrical, replace it After that, the dependency between adjacent joints of the same bone is no longer the same. As shown in the above formula, the contribution information of the own joint and the contribution information of the adjacent joints are extracted separately. Therefore, in order to aggregate these two different features separately, It is also decomposed into two parts, namely, the self-relation and The relationship with neighbor joints. The decomposition formula is as follows:

[0085]

[0086] Where ⊙ represents the Hadamard product of two matrices. According to the idea of ​​spatial domain feature aggregation in GCN, and As a relationship matrix, multiply by and Aggregate the features of the node and its neighboring nodes:

[0087]

[0088] in Represents the characteristics of self-node aggregation, Features that represent the aggregation of adjacent nodes. Both features are local features.

[0089] 4. Feature Fusion:

[0090] After the above steps, each joint has three different features, which represent the information from the joint itself, adjacent joints and other global joints. Here we choose the Concatenate fusion method, and the final feature fusion is expressed as:

[0091]

[0092] Where Cat represents the connection operation. Since the connection operation increases the dimension of the feature, in these three types of feature extraction, its final output feature dimension needs to be converted to the same dimension in order to maintain the same feature dimension when using the connection fusion method. Finally, a simple MLP block with layer normalization and a linear layer is used to return the final 3D pose coordinates R 1×(N×3) .

[0093] 5. Loss function:

[0094] In regression-based methods, the mean square error (MSE) between the estimated value and the true value is usually used as the loss function. In this paper, the estimated joint position and the true joint position also use the MSE loss function, which is defined as:

[0095]

[0096] in is the estimated 3D pose coordinate of joint i, J i is the estimated coordinate of the true 3D pose of joint i.

[0097] The present invention uses the public data sets Human3.6M and MPI-INF-3DHP for experiments. Human3.6M is currently the most popular indoor 3D human posture estimation benchmark. It contains 11 subjects performing 15 typical actions, including a variety of asymmetric walking postures (such as walking with hands in pockets, walking with a shoulder bag), sitting postures, lying postures, various waiting postures and other types of postures, generating a total of 3.6 million video frames; As mentioned above, although Human3.6M has a large amount of data, the scene is single. In order to solve this problem, MPI-INF-3DHP adds data enhancement processing for the foreground and background to the data set. Specifically, the collection of its training set is completed in an indoor green screen scene with the help of a multi-eye camera. For the original collected data, the human body, background, etc. are first segmented, and then the background and foreground are replaced with different textures to achieve the purpose of data enhancement. The test set covers three different scenes, including indoor green screen scenes, ordinary indoor scenes, and outdoor scenes. Therefore, MPI-INF-3DHP is more helpful in evaluating the generalization ability of the algorithm.

[0098] The present invention uses MPJPE (mean per joint position error) as an indicator to evaluate the performance of 3D HPE. MPJPE is calculated using the Euclidean distance between the estimated 3D joint position and the ground truth position. The MPJPE formula is as follows:

[0099]

[0100] Experimental results show that the three-dimensional human posture estimated by this method has high accuracy.

[0101] In the above embodiments, the description's reference to "this embodiment" indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0102] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0103] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0104] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0105] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0106] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0107] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0108] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0109] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0110] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0111] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A 3D human pose estimation method based on attention graph convolution, characterized by: The following steps are involved: S1: 2D pose estimation of human body images using cascaded pyramid network CPN; S2: Extract global spatial features using the Transformer-based attention mechanism; S3: Extract local spatial features using graph convolutional networks; S4: Fuse the global space features and local space features to obtain the final 3D posture coordinates.

2. The 3D human posture estimation method based on attention graph convolution according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11: Input the human body image into the cascade pyramid network CPN for two-dimensional posture estimation to obtain the two-dimensional joint point coordinates C i ∈R N×2 , where N is the number of joint points; S12: The obtained two-dimensional key point coordinates are encoded using a skeleton embedding module composed of a multi-layer fully connected network to obtain a high-dimensional vector.

3. The 3D human posture estimation method based on attention graph convolution according to claim 1, characterized in that: In step S2, each joint is considered as a patch, and features are extracted between all patches according to the general visual transformation pipeline. First, the coordinates of each joint are mapped to a high-dimensional space using a trainable linear projection. The obtained features R J×C The joint sequence is input into the spatial transformer encoder, which integrates the information of all joints using a self-attention mechanism; The Transformer attention block follows the scaled dot product attention, and the attention calculation formula for the query, key, and value matrices Q, K, V in each head is: where {Q,K,V}∈R J×C , N represents the number of tokens, i.e. the number of joint points, C represents the dimension of each token, and h multi-head attentions are defined as follows: MSA=Concat(head1,head2,...,head h )W O head i =Attention(Q i ,K i ,V i ),i∈h The linear projection weight is W O ∈R C×C , in the attention encoder, each joint token p∈P N is the two-dimensional coordinate C∈R N×2 Joint C i Projected, the joint token p is passed through the matrix E pos ∈R N×C Embed location information: X=Norm(L e (C i )+E pos ),X∈R N×C Where Norm is layer normalization, L e It is a linear embedding layer, and the spatial dependency between joint points is calculated through the attention module as follows: U i =XW m ,m∈{Q,K,V} Y=MSA(U Q ,U K ,U V )+X R=MLP(Norm(Y))+Y Where Y is the attention output of joint token X, U i is the matrix of X mapped by linear transformation, W m is the linear transformation weight matrix corresponding to the query, key, and value in the joint, Norm is the layer normalization, and R is the final output of the global attention.

4. The 3D human posture estimation method based on attention graph convolution according to claim 1, characterized in that: The joint coordinates are represented as a graph G = {V, E}, where V is a set of N joint points and E represents an edge. GCN is represented as: in, is the input feature of layer l, is the output feature after GCN operation, D l and D l+1 are the input feature dimension and output feature dimension of each joint, A is the adjacency matrix of the undirected graph, I is the identity matrix, (A+I) represents the adjacency matrix with self-connection, D is the degree matrix of A; is the symmetric normalization of the self-loop adjacency matrix (A+I), is a trainable transformation matrix for transforming and extracting joint features of layer l; from the perspective of spatial domain, H (l) W (l) The input joint feature H (l) Feature extraction; Using two different transformation matrices W0 (l) and W1 (l) To extract two different feature information, as shown in the following formula: These are two features extracted for two different types of information; Add a learnable EA matrix M to the graph convolution structure (l) , EA includes the learning of its own joints and neighboring joints, and the adjacency relationship of the EA matrix is ​​expressed as: where the adjacency matrix (A+I) is used as a mask, ⊙ represents the Hadamard product of two matrices, and ρ i is the softmax operator, which is used to achieve row normalization so that the sum of the edge attention weights of all j adjacent joints to joint i is equal to 1; Through the learnable matrix M (l) OK, replace it After that, the dependency between adjacent joints of the same skeleton is no longer the same; the contribution information to its own joints and the contribution information to adjacent joints are extracted separately. is decomposed into two parts, namely, the self-relation and The relationship with neighbor joints is decomposed into the following formula: Where ⊙ represents the Hadamard product of two matrices. and As a relationship matrix, multiply by and Aggregate the features of the node and its neighboring nodes: in Represents the characteristics of self-node aggregation, Features that represent the aggregation of adjacent nodes. Both features are local features.

5. The 3D human posture estimation method based on attention graph convolution according to claim 1, characterized in that: Step S4 specifically includes: The three features of each joint are fused using the Concatenate fusion method, expressed as: Where Cat represents the connection operation; The output feature dimensions are then transformed to the same dimension and finally a simple MLP block with layer normalization and a linear layer is used to return the final 3D pose coordinates R 1×(N×3) .

6. The 3D human posture estimation method based on attention graph convolution according to claim 1, characterized in that: The mean square error MSE between the estimated value and the true value is used as the loss function of the estimated joint position and the true joint position, which is defined as: in is the estimated 3D pose coordinate of joint i, J i is the estimated coordinate of the true 3D pose of joint i.

Images