Three-dimensional human body posture estimation method based on space-time Transform

Abstract

The invention relates to a three-dimensional human body posture estimation method based on space-time Transform, which belongs to the technical field of human body three-dimensional posture estimation, and comprises the following steps: S1, carrying out two-dimensional posture estimation on an input human body posture image by using a cascaded pyramid network (CPN); s2, utilizing a space transformer block based on a Transform attention mechanism to obtain space information between joint points; s3, inputting the spatial information into a global time sequence feature learning module based on a Transform attention mechanism, and outputting global time sequence feature information; s4, different joints are separated in the time dimension, parallel modeling is conducted on the different joints, and local time sequence feature information is output through a local time sequence feature learning module based on a Transform attention mechanism; and S5, carrying out fusion and regression on the global time sequence feature information and the local time sequence feature information to obtain a three-dimensional attitude coordinate.

Description

Technical Field

[0001] The present invention belongs to the technical field of human three-dimensional posture estimation, and relates to a three-dimensional human posture estimation method based on spatiotemporal Transformer. Background Art

[0002] As a key research area in computer vision, human pose estimation has made significant progress in technology development and application in recent years. It aims to accurately describe the human pose information by analyzing the human form in images or videos, involving multiple tasks such as target recognition, image segmentation, and regression detection. Compared with traditional two-dimensional pose estimation, three-dimensional human pose estimation can capture the human pose information more finely, so it has a wider research and application prospect.

[0003] At present, 3D human posture estimation has become a research hotspot in the field of computer vision and is also the basis of many important application fields. The 3D human posture information extracted from images or videos can not only be used for action recognition, but also for tasks such as 3D model reconstruction, providing important support for human behavior analysis, virtual reality, augmented reality and other fields.

[0004] Current 3D human pose estimation methods can be mainly divided into two categories: one is the direct regression method, which directly predicts 3D pose coordinates from 2D images. This method can perform end-to-end network training, but has high requirements for network structure and data preprocessing; the other is the method based on 2D coordinates. First, a pre-trained 2D pose estimation network is used to extract the skeleton sequence, which is then input into the 3D pose estimation network for dimensionality enhancement. The 2D skeleton-based method has become the mainstream because it uses the existing 2D pose estimation algorithm to simplify the task complexity and has better performance than the direct regression method. This method reduces the complexity of the network structure and is easier to deploy in actual environments.

[0005] In addition, with the excellent effects of attention mechanism on various tasks in recent years, the framework based on attention mechanism has also begun to be applied to the monocular single-person 3D human pose estimation task, and has achieved excellent results in improving the depth ambiguity problem. In 2017, the Transformer model was proposed and showed good performance in many computer vision tasks. Zheng et al. first proposed a pure Transformer-based 3D human pose estimation method in videos, in which no convolutional architecture is required. The network structure is designed as a spatiotemporal Transformer structure. First, the spatial information of the relative position between the joints of each frame is extracted, and the time series information is extracted through the temporal Transformer structure. However, this structure has a large amount of calculation and there is a problem of adjacent frame redundancy. Hassanin et al. proposed a Transformer-based pose estimation model, which added two interactive modules, cross-joint and cross-frame, to the Transformer structure, and explicitly encoded the local and global dependencies between body joints. Zhang et al. proposed a mixed spatio-temporal encoder MixSTE (Mixed Spatio-Temporal Encoder), which has a temporal transformer to model the temporal motion of each joint and a spatial transformer block to learn the spatial correlation between joints. The two blocks are used alternately to obtain better spatio-temporal feature encoding. In addition, the network output is extended from the central frame of the input video to the entire frame, thereby improving the consistency between the input sequence and the output sequence. Tang et al. proposed a spatio-temporal interleaved transformer STCFormer (Spatio-Temporal Criss-cross Transformer), which explores the spatial correlation and temporal correlation of 3D human pose estimation in videos in a dual-path manner. Among them, STCFormer is constructed by stacking multiple STC blocks, each of which divides the joint features into two groups along the channel dimension and models the spatial and temporal interactions on each group separately. By doing so, the receptive field of the STC block is like a criss-cross spatial and temporal axis. In addition, STCFormer uses the dynamic chain structure of the human body to model the local context and form a new position embedding function.

[0006] Although these methods have considered the acquisition of skeleton information, there are still deficiencies in the acquisition of local information, which may affect the accurate estimation of human 3D posture. In videos, there is an obvious temporal relationship between consecutive frames, and the temporal features at different time scales may be crucial for posture estimation. Therefore, effectively processing multi-scale temporal features can help improve the accuracy of human 3D posture estimation. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide a 3D human posture estimation method based on spatiotemporal Transformer. Aiming at the problem that complex spatiotemporal information is difficult to handle in 3D human posture estimation, firstly, in spatial modeling, a Transformer attention mechanism is proposed to obtain spatial information between joints. Secondly, in temporal modeling, a method of fusing two Transformer attention mechanisms for multi-scale temporal feature extraction is proposed to obtain temporal feature information between multiple frames. By global and local spatiotemporal modeling between joints, the complex spatiotemporal information between joints in human posture is obtained, thereby achieving accurate estimation of human 3D posture.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A 3D human posture estimation method based on spatiotemporal Transformer comprises the following steps:

[0010] S1: Use the cascaded pyramid network CPN to estimate the 2D pose of the input human image;

[0011] S2: Taking the joint points estimated by 2D posture as input, the spatial Transformer module STB is used to obtain the spatial information between the joint points of a single frame image;

[0012] S3: Input the spatial information of multiple frames into the global temporal Transformer module, and output the global temporal feature information of the joint points of multiple frames;

[0013] S4: Separate different joints in the time dimension, model different joints in parallel, and use the local temporal Transformer module to output local temporal feature information;

[0014] S5: Fuse and regress the global temporal feature information and the local temporal feature information to obtain the three-dimensional posture coordinates.

[0015] Further, step S1 specifically includes the following steps:

[0016] S11: Input the human body posture image into the cascade pyramid network CPN for two-dimensional posture estimation to obtain the two-dimensional joint point coordinates C N,T ∈R N×T×2 , where N is the number of joints and T is the number of frames;

[0017] S12: The two-dimensional key point coordinates C N,T Use a skeleton embedding module composed of multiple layers of fully connected networks to encode and obtain a high-dimensional vector The characteristic dimension of each joint is d m .

[0018] Further, in step S2, the STB learns the spatial correlation between joints in each frame, given a two-dimensional key point with N joints, and regards each joint as a marker of spatial attention;

[0019] First, taking the two-dimensional joint points as input, a linear embedding layer is used to project each key point onto the spatial feature;

[0020] Then use the position matrix Embed spatial location information;

[0021] Then the spatial features of the i-th frame are Input into the spatial attention mechanism of STB, model the dependencies between all joints, and output the high-dimensional features in the spatial transformer block of the i-th frame

[0022] Further, in step S3, the output of the spatial module Convert to As the input of the global temporal Transformer module, learn a global cross-frame temporal information. Output global temporal feature information Convert its dimensions into

[0023] Furthermore, in step S4, all body joints are treated as a token in the local temporal Transformer module, and different joints are separated in the time dimension so that the trajectory of each joint is a separate token. Different joints of the human body are modeled in parallel; from the perspective of the time dimension, different motion trajectories of the human joints are modeled separately, and the joint separation operation is as follows:

[0024]

[0025] where p i,j ∈P N,T represents the i-th joint in the j-th frame, F represents the temporal encoder function, and the output of the l-th TTB encoder

[0026] Furthermore, the global temporal Transformer module and the local temporal Transformer module are both based on the Transformer attention mechanism, which follows the scaled dot product attention. The attention calculation formula of the query, key, and value matrices Q, K, and V in each head is:

[0027]

[0028] in N represents the number of tokens, i.e. the number of joint points, and C represents the dimension of each token; h multi-head attentions are defined as follows:

[0029] MSA=Concat(head1,head2,...,head h )W O

[0030] head i =Attention(Q i ,K i ,V i ),i∈h

[0031] The linear projection weight is In the attention encoder, each joint token p∈P N is from the two-dimensional coordinate C N ∈R N×2 Joint C i Projected; the combined token p is passed through the matrix Embed location information:

[0032]

[0033] Where Norm is layer normalization, L e is a linear embedding layer; the spatial dependency between joint points is calculated through the attention mechanism as follows:

[0034] U i =XW m ,m∈{Q,K,V}

[0035] Y=MSA(U Q ,U K ,U V )+X

[0036] R=MLP(Norm(Y))+Y

[0037] Where Y is the attention output of joint token X, U i is the matrix of X mapped by linear transformation, W m is the linear transformation weight matrix corresponding to the query, key, and value in the joint; Norm is the layer normalization, and R is the final output of the global attention.

[0038] Further, in step S5, the two obtained time series feature information are fused using the Concatenate fusion method, and the final feature fusion is expressed as:

[0039] R=Cat(R1,R2)

[0040] Where Cat represents the connection operation; in the two feature extractions, the final output feature dimensions of each are converted to the same dimension; finally, an MLP block with layer normalization and a linear layer is used to return the final 3D pose coordinates R T×(N×3) .

[0041] Furthermore, the estimated joint positions and the true joint positions use the MSE loss function, which is defined as:

[0042]

[0043] in is the estimated 3D pose coordinate of joint i, J i is the estimated coordinate of the true 3D pose of joint i.

[0044] The beneficial effects of the present invention are as follows: the present invention extracts multi-scale temporal features by fusing global and local Transformer attention mechanisms, obtains temporal feature information between multiple frames, obtains complex temporal and spatial information between joints in human posture by global and local spatiotemporal modeling between joints, and improves the estimation accuracy of human three-dimensional posture.

[0045] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0047] Figure 1 The overall block diagram of 3D human pose estimation. DETAILED DESCRIPTION

[0048] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0049] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0050] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0051] like Figure 1 As shown, the present invention provides a 3D human posture estimation method based on spatiotemporal Transformer. Firstly, in spatial modeling, a Transformer attention mechanism is proposed to obtain spatial information between joints. Secondly, in temporal modeling, a method of fusing two Transformer attention mechanisms to extract multi-scale temporal features is proposed to obtain temporal feature information between multiple frames. By global and local spatiotemporal modeling between joints, the complex spatiotemporal information between joints in human posture is obtained, thereby realizing accurate estimation of human 3D posture.

[0052] For the above method, the present invention provides the following 5 modules to implement:

[0053] 1. Two-dimensional key point extraction module:

[0054] The present invention uses the existing two-dimensional pose estimation detector cascade pyramid network (CPN) to perform two-dimensional pose estimation on the image. First, the image is input into the cascade pyramid network (CPN) for two-dimensional pose estimation to obtain the two-dimensional joint point coordinates C N,T ∈R N×T×2 , where N is the number of joint points set to 17, and T is the number of frames. Take it as input. Then get the two-dimensional key point coordinates C N,T Use the skeleton embedding module (composed of multiple layers of fully connected networks) to encode the high-dimensional vector The characteristic dimension of each joint is d m .

[0055] 2. Spatiotemporal feature learning module

[0056] 1) Use the spatial transformer module STB to learn the spatial correlation between joints in each frame. Given a 2D keypoint with N joints, we treat each joint as a marker for spatial attention. First, taking the 2D joint points as input, a linear embedding layer is used to project each keypoint onto a high-dimensional feature, which is called spatial feature in STB. Then the position matrix Embed the spatial position information. Then the spatial features of the i-th frame Input into the spatial attention mechanism of STB, model the dependencies between all joints, and output the high-dimensional features in the i-th frame STB

[0057]

[0058] 2) Set the output of STB Convert to As the input of the global temporal Transformer module, learn a global cross-frame temporal information. Output global temporal feature information Convert its dimensions into

[0059]

[0060] 3) In order to inject effective motion trajectories into the learned representation, the temporal correspondence of each joint is considered so as to explicitly model the correlation of the same joints in the dynamic sequence. All body joints are treated as a token in the local temporal Transformer module. Different joints are separated in the temporal dimension so that the trajectory of each joint is a separate token. Different joints of the human body are modeled in parallel. From the perspective of the time dimension, different motion trajectories of human joints are modeled separately to better represent the time correlation. The joint separation operation is as follows:

[0061]

[0062] where p i,j ∈P N,T represents the i-th joint in the j-th frame, F represents the temporal encoder function, and the output of the l-th TTB encoder

[0063] 3. Transformer-based attention module:

[0064] In both the spatial Transformer module STB and the temporal Transformer module, the same Transformer attention block is used. The Transformer attention block follows the scaled dot product attention. The attention calculation formula for the query, key, and value matrices Q, K, and V in each head is:

[0065]

[0066] in N represents the number of tokens, i.e. the number of joint points, and C represents the dimension of each token. The h multi-head attention is defined as follows:

[0067] MSA=Concat(head1,head2,...,head h )W O

[0068] head i =Attention(Q i ,K i ,V i ),i∈h

[0069] The linear projection weight is In the attention encoder of this method, each joint token p∈P N is from the two-dimensional coordinate C N ∈R N×2 Joint C i The projected. Union token p through the matrix Embed location information:

[0070]

[0071] Where Norm is layer normalization, L e is a linear embedding layer. The spatial dependency between joint points is calculated through the Transformer attention module as follows:

[0072] U i =XW m ,m∈{Q,K,V}

[0073] Y=MSA(U Q ,U K ,U V )+X

[0074] R=MLP(Norm(Y))+Y

[0075] Where Y is the attention output of joint token X, U i is the matrix of X mapped by linear transformation, W m is the linear transformation weight matrix corresponding to the query, key, and value in the joint. Norm is the layer normalization, and R is the final output of the global attention.

[0076] 4. Feature fusion and regression module:

[0077] After the above steps, the two time series information are fused. Here, the Concatenate fusion method is selected, and the final feature fusion is expressed as:

[0078] R=Cat(R1,R2)

[0079] Where Cat represents the concatenation operation. Since the concatenation operation increases the dimension of the feature, the final output feature dimension needs to be converted to the same dimension in both feature extractions so as to maintain the same feature dimension when using the concatenation fusion method. Finally, a simple MLP block with layer normalization and a linear layer is used to return the final 3D pose coordinates R T×(N×3) .

[0080] 5. Loss function module:

[0081] In regression-based methods, the mean square error (MSE) between the estimated value and the true value is usually used as the loss function. In this paper, the estimated joint position and the true joint position also use the MSE loss function, which is defined as:

[0082]

[0083] in is the estimated 3D pose coordinate of joint i, J i is the estimated coordinate of the true 3D pose of joint i.

[0084] For this method, the mean per joint position error MPJPE is used as the metric to evaluate the performance of 3D HPE. MPJPE is calculated using the Euclidean distance between the estimated 3D joint positions and the ground truth positions. The MPJPE formula is as follows:

[0085]

[0086] The experimental data used in this embodiment are the public data sets Human3.6M and MPI-INF-3DHP. Human3.6M is currently the most popular indoor 3D human posture estimation benchmark. It contains 11 subjects performing 15 typical actions, including a variety of asymmetric walking postures (such as walking with hands in pockets, walking with bags on shoulders), sitting postures, lying postures, various waiting postures and other types of postures, generating a total of 3.6 million video frames; As mentioned above, although Human3.6M has a large amount of data, the scene is single. In order to solve this problem, MPI-INF-3DHP adds data enhancement processing for the foreground and background to the data set. Specifically, the collection of its training set is completed in an indoor green screen scene with the help of a multi-eye camera. For the original collection data, the human body, background, etc. are first segmented, and then the background and foreground are replaced with different textures to achieve the purpose of data enhancement. The test set covers three different scenes, including indoor green screen scenes, ordinary indoor scenes, and outdoor scenes. Therefore, MPI-INF-3DHP is more helpful in evaluating the generalization ability of the algorithm. The experimental results show that this method can significantly improve the accuracy of human three-dimensional posture.

[0087] In the above embodiments, the description's reference to "this embodiment" indicates that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple occurrences of "this embodiment" do not necessarily all refer to the same embodiment.

[0088] In the above-described embodiments, although the invention has been described in conjunction with specific embodiments of the invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other storage structures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed. Embodiments of the invention are intended to encompass all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0089] This embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, any one of the methods in this embodiment is implemented.

[0090] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0091] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0092] The computer-readable storage medium in this embodiment can be understood by ordinary technicians in this field: all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0093] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used to communicate, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes each step of the above method.

[0094] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0095] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0096] The present invention can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.

[0097] The present invention may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A 3D human posture estimation method based on spatiotemporal Transformer, characterized by: The following steps are involved: S1: Use the cascaded pyramid network CPN to estimate the 2D pose of the input human image; S2: Taking the joint points estimated by 2D posture as input, the spatial Transformer module STB is used to obtain the spatial information between the joint points of a single frame image; S3: Input the spatial information of multiple frames into the global temporal Transformer module, and output the global temporal feature information of the joint points of multiple frames; S4: Separate different joints in the time dimension, model different joints in parallel, and use the local temporal Transformer module to output local temporal feature information; S5: Fuse and regress the global temporal feature information and the local temporal feature information to obtain the three-dimensional posture coordinates.

2. The 3D human posture estimation method based on spatiotemporal Transformer according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11: Input the human body posture image into the cascade pyramid network CPN for two-dimensional posture estimation to obtain the two-dimensional joint point coordinates C N,T ∈R N×T×2 , where N is the number of joints and T is the number of frames; S12: The two-dimensional key point coordinates C N,T Use a skeleton embedding module composed of multiple layers of fully connected networks to encode and obtain a high-dimensional vector The characteristic dimension of each joint is d m .

3. The 3D human posture estimation method based on spatiotemporal Transformer according to claim 1, characterized in that: In step S2, the STB learns the spatial correlation between joints in each frame, given a two-dimensional key point with N joints, and regards each joint as a marker of spatial attention; First, taking the two-dimensional joint points as input, a linear embedding layer is used to project each key point onto the spatial feature; Then use the position matrix Embed spatial location information; Then the spatial features of the i-th frame are Input into the spatial attention mechanism of STB, model the dependencies between all joints, and output the high-dimensional features in the spatial transformer block of the i-th frame 4. The 3D human posture estimation method based on spatiotemporal Transformer according to claim 1, characterized in that: In step S3, the output of the spatial module Convert to As the input of the global temporal Transformer module, learn a global cross-frame temporal information; output global temporal feature information Convert its dimensions into 5. The 3D human posture estimation method based on spatiotemporal Transformer according to claim 1, characterized in that: In step S4, all body joints are treated as a token in the local temporal Transformer module, and different joints are separated in the time dimension so that the trajectory of each joint is a separate token. Different joints of the human body are modeled in parallel; from the perspective of the time dimension, different motion trajectories of the human joints are modeled separately, and the joint separation operation is as follows: where p i,j ∈P N,T represents the i-th joint in the j-th frame, F represents the temporal encoder function, and the output of the l-th TTB encoder 6. The 3D human posture estimation method based on spatiotemporal Transformer according to claim 1, characterized in that: The global temporal Transformer module and the local temporal Transformer module are both based on the Transformer attention mechanism, which follows the scaled dot product attention. The attention calculation formula of the query, key, and value matrices Q, K, and V in each head is: in N represents the number of tokens, i.e. the number of joint points, and C represents the dimension of each token; h multi-head attentions are defined as follows: MSA=Concat(head1,head2,...,head h )W O head i =Attention(Q i ,K i ,V i ),i∈h The linear projection weight is In the attention encoder, each joint token p∈P N is from the two-dimensional coordinate C N ∈R N×2 Joint C i Projected; the combined token p is passed through the matrix Embed location information: Where Norm is layer normalization, L e is a linear embedding layer; the spatial dependency between joint points is calculated through the attention mechanism as follows: U i =XW m ,m∈{Q,K,V} Y=MSA(U Q ,U K ,U V )+X R=MLP(Norm(Y))+Y Where Y is the attention output of joint token X, U i is the matrix of X mapped by linear transformation, W m is the linear transformation weight matrix corresponding to the query, key, and value in the joint; Norm is the layer normalization, and R is the final output of the global attention.

7. The 3D human posture estimation method based on spatiotemporal Transformer according to claim 1, characterized in that: In step S5, the two obtained time series feature information are fused using the Concatenate fusion method, and the final feature fusion is expressed as: R=Cat(R1,R2) Where Cat represents the connection operation; in the two feature extractions, the final output feature dimensions of each are converted to the same dimension; finally, an MLP block with layer normalization and a linear layer is used to return the final 3D pose coordinates R T ×(N×3) .

8. The method for 3D human posture estimation based on spatiotemporal Transformer according to claim 1, characterized in that: The estimated joint positions and the true joint positions use the MSE loss function, defined as: in is the estimated 3D pose coordinate of joint i, J i is the estimated coordinate of the true 3D pose of joint i.

Images