Single-hand reconstruction method based on attitude decoupling
By employing a pose-decoupled single-hand reconstruction method, and utilizing comparative learning of pose-related and irrelevant features, the problem of insufficient 3D annotation data is solved, achieving stability and robustness in 3D hand reconstruction, which is suitable for applications such as virtual reality and human-computer interaction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI UNIV
- Filing Date
- 2026-01-31
- Publication Date
- 2026-05-12
Smart Images
Figure CN122023720A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image information processing and computer vision technology, and in particular relates to a single-hand reconstruction method based on pose decoupling. Background Technology
[0002] With the rapid development of technologies such as virtual reality, augmented reality, human-computer interaction, smart homes, and wearable devices, 3D reconstruction of the hand has become an important research direction in the field of computer vision. As one of the most flexible and information-rich interactive parts of the human body, the hand's posture, movements, and shape can be directly used to express commands, control interfaces, assist in viewing medical images, and enhance immersive experiences. Real-time and accurate reconstruction of the hand's 3D structure is of great significance for improving the naturalness of interaction and the operability of the system.
[0003] Traditional 3D hand reconstruction typically relies on specialized hardware such as multi-view imaging systems, depth cameras, or wearable data gloves. While these methods can provide relatively accurate 3D information, they are costly, involve complex equipment, and have limited applicability, making them unsuitable for the needs of ordinary consumer devices. In recent years, with the development of deep learning, methods for estimating the 3D structure of the hand from a single RGB image have gradually become a mainstream research direction. However, this approach still faces many technical bottlenecks.
[0004] First, the hand, as a highly non-rigid structure, contains more than twenty degrees of freedom. The bending, self-occlusion, and complex pose changes of different fingers lead to a large amount of deformation and blurring information in 2D images, posing significant challenges to 3D reconstruction. Second, existing deep learning methods typically rely on large amounts of data with 3D annotations for training. However, high-quality 3D annotations often require expensive multi-camera systems and meticulous annotation processes, resulting in data scarcity and high acquisition costs. In real-world scenarios, only a small amount of 3D-annotated data is often available, making it difficult for traditional supervised learning-based methods to train generalized 3D reconstruction models.
[0005] To alleviate the problem of scarce 3D annotations, self-supervised and contrastive learning methods have been gradually introduced into hand 3D reconstruction tasks. Contrastive learning improves the model's discriminative ability by constructing positive and negative sample pairs. However, existing methods often directly use overall visual features for contrast optimization, making it difficult to distinguish between "pose-related information" and "appearance-related information." Since hand images have a large number of appearance differences in terms of lighting, background, skin color, and texture, if the model cannot decouple pose features from appearance factors, it will lead to unstable pose representation and insufficient generalization ability.
[0006] Therefore, there is an urgent need for a method that can automatically learn pose-related discriminative representations from unannotated images and explicitly remove pose-irrelevant factors such as lighting, background, and texture, so as to achieve accurate hand 3D pose and mesh reconstruction even when 3D annotation data is insufficient. Summary of the Invention
[0007] To address the aforementioned technical issues, this invention proposes a single-hand reconstruction method based on pose decoupling. This method explicitly introduces a decoupling mechanism between pose-related and pose-independent features within a contrastive learning framework, enabling the model to learn robust pose discrimination features on a large number of unlabeled images and still obtain high-precision 3D reconstruction results when fine-tuned using a small amount of labeled data.
[0008] To achieve the above objectives, this invention provides a single-hand reconstruction method based on attitude decoupling, comprising: Obtain an RGB image containing the hand, perform data augmentation processing on the RGB image, and obtain multiple augmented images; The enhanced image is input into a single-hand reconstruction model to obtain the hand reconstruction result. The single-hand reconstruction model includes a pose decoupling module, a feature encoding module, and a three-dimensional reconstruction module. The pose decoupling module is used to extract the basic hand features of the enhanced image and map the basic hand features into pose-related features and pose-independent features; The feature encoding module is used to calculate the pose-related contrast loss and the pose-independent contrast loss based on the pose-related features and the pose-independent features, respectively. The 3D reconstruction module is used to estimate the coordinates of the 3D joints and the 3D mesh vertices of the hand based on the posture-related contrast loss and the posture-independent contrast loss, and output the hand reconstruction result.
[0009] Optionally, extracting the basic hand features from the enhanced image and mapping the basic hand features to pose-related features includes: The enhanced image is input into the backbone network for processing to obtain a fine-grained feature map; The fine-grained feature map is deconvolutionally upsampled to obtain a high-resolution feature map; Average pooling and convolution processing are performed on the high-resolution feature map in different directions to obtain one-dimensional heatmaps for keypoint detection in different directions; The coordinates of the two-dimensional joints of the hand are extracted from the one-dimensional heatmap using soft-argmax, and the coordinates of the two-dimensional joints of the hand are converted into a Gaussian heatmap. The attitude-related features are obtained by performing feature projection on the Gaussian heatmap.
[0010] Optionally, before converting the two-dimensional hand joint coordinates into a Gaussian heatmap, the method further includes: The two-dimensional joint coordinates of the hand are optimized using a two-dimensional pose estimation loss, wherein the two-dimensional pose estimation loss... for: ; in, Let j represent the number of joints, and j represent the j-th joint. This represents the predicted coordinates of the j-th joint. Represents the corresponding real coordinates, It is an L1 norm.
[0011] Optionally, extracting the basic hand features of the enhanced image and mapping the basic hand features to pose-independent features includes: The enhanced image is input into the backbone network for processing to obtain a fine-grained feature map; The fine-grained feature map is then subjected to global average pooling to obtain the feature vector; The feature vector is input into a projection head containing two fully connected layers and one batch normalization layer, and mapped to the pose-independent feature.
[0012] Optionally, calculate the attitude-related contrast loss. include: ; ; in, It's a temperature over-parameter. Cosine similarity is used to measure the similarity between two feature vectors. Embedded representations for pose-related features Indicates For anchor points, For positive samples, the pose-related contrastive loss term is used. Indicates For anchor points, denoted as the pose-related contrastive loss term for positive samples, m is the index of the negative sample, and M is the total number of negative samples participating in the contrastive learning. () represents an exponential function.
[0013] Optionally, calculate pose-independent contrastive loss. include: ; in, For pose-independent feature embedding representation. Indicates For anchor points, For positive samples, the pose-independent contrastive loss term is used. Indicates For anchor points, For positive samples, the pose-independent contrastive loss term is used. Indicates an indicator function, when The value is 1 if the condition is met, and 0 otherwise. These are negative sample pairs, and N represents the number of sample pairs participating in the contrastive learning.
[0014] Optionally, based on the pose-dependent contrast loss and pose-independent contrast loss, the coordinates of the three-dimensional joints of the hand and the coordinates of the three-dimensional mesh vertices of the hand are estimated, and the hand reconstruction results are output, including: The feature encoding module is optimized using the pose-related contrast loss and pose-independent contrast loss to obtain the optimized feature encoding module; The optimized feature encoding module is then transferred to the 3D reconstruction module. The 3D reconstruction module is used to estimate the coordinates of the 3D joints and the vertices of the 3D mesh of the hand, and the hand reconstruction result is output.
[0015] Optionally, the 3D reconstruction module includes: a pose regression unit (PoseNet unit), a mesh regression unit (MeshNet unit), a projection unit, and a mapping unit; The PoseNet unit is used to predict the 2.5D coordinates of hand joints based on the output features of the feature encoding module. The MeshNet unit is used to predict the 2.5D coordinates of the hand mesh based on the 2.5D coordinates; The projection unit is used to back-project the 2.5D coordinates of the hand mesh into three-dimensional space to obtain the vertex coordinates of the three-dimensional mesh; The mapping unit is used to map the coordinates of the three-dimensional mesh vertex to the coordinates of the three-dimensional hand joints according to the preset joint regression matrix, and use the coordinates of the three-dimensional hand joints as the hand reconstruction result.
[0016] Optionally, the reconstruction method further includes: A total loss function is constructed based on the two-dimensional pose estimation loss, pose-related contrast loss, and pose-independent contrast loss. The total loss function is then used to optimize the single-handed reconstruction model by weighted summation.
[0017] Compared with the prior art, the present invention has the following advantages and technical effects: 1. By employing a feature learning mechanism with pose decoupling, the impact of pose changes on the accuracy of single-hand reconstruction is effectively reduced. This invention decouples hand features into pose-related features and pose-independent features, and introduces corresponding contrastive learning losses for each to constrain them. This enables the model to distinguish between pose changes and inherent hand morphology information during the feature learning stage. Compared with existing methods that directly perform regression in the coupled feature space, this invention effectively reduces the interference of complex pose changes on the reconstruction results, improving the stability and robustness of single-hand 3D reconstruction under conditions of large pose changes.
[0018] 2. This invention enhances feature discrimination capabilities and improves the consistency and reliability of reconstruction results by utilizing contrastive learning. Through pose-dependent contrastive loss and pose-independent contrastive loss, this invention guides the feature representation of the same hand instance to remain consistent across different viewpoints or data augmentation conditions, while simultaneously widening the feature distance between different instances. Compared to existing methods that rely on a single supervisory signal, this invention can significantly enhance the discriminativeness of feature embedding without additional annotation costs, thereby improving the consistency and reliability of 3D reconstruction results.
[0019] 3. This invention employs a phased 2D-to-3D reconstruction structure, reducing the difficulty of 3D regression and improving overall reconstruction accuracy. The invention first predicts the 2.5D coordinates of hand joints using a pose regression unit, then predicts the 2.5D coordinates of the hand mesh using a mesh regression unit, and finally completes the step-by-step reconstruction process from 2D to 3D by combining projection and mapping units. Compared to existing methods that directly regress 3D meshes from image features, this invention decomposes the complex 3D reconstruction problem into multiple sub-tasks, reducing the difficulty of model learning and improving the accuracy of 3D mesh and joint prediction.
[0020] 4. The structure is clear and the modules are well-defined, exhibiting good versatility and scalability. The 3D reconstruction module of this invention consists of pose regression units, mesh regression units, projection units, and mapping units. Each unit has a clearly defined function and is independent of the others, facilitating replacement or expansion in different application scenarios. Compared to existing end-to-end coupled structures, this invention is more conducive to system integration and engineering implementation.
[0021] 5. Applicable to single-hand 3D reconstruction under monocular image conditions, with a wide range of applications. This invention does not require multi-view input or complex external devices, and can achieve single-hand 3D reconstruction using only a single image. It is suitable for various application scenarios such as virtual reality, human-computer interaction, and augmented reality, and has high practical value. Attached Figure Description
[0022] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a single-hand reconstruction method based on attitude decoupling according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the attitude decoupling module according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the pose-dependent and pose-independent contrastive learning modules according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the three-dimensional reconstruction module according to an embodiment of the present invention. Detailed Implementation
[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0024] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0025] This embodiment proposes a single-hand reconstruction method based on attitude decoupling, such as... Figure 1 As shown, the specific steps include: Obtain an RGB image containing the hand, perform data augmentation processing on the RGB image, and obtain multiple augmented images; The enhanced image is input into a single-hand reconstruction model to obtain the hand reconstruction result. The single-hand reconstruction model includes a pose decoupling module, a feature encoding module, and a three-dimensional reconstruction module. The pose decoupling module is used to extract the basic hand features of the enhanced image and map the basic hand features into pose-related features and pose-independent features; The feature encoding module is used to calculate the pose-related contrast loss and the pose-independent contrast loss based on the pose-related features and the pose-independent features, respectively. The 3D reconstruction module is used to estimate the coordinates of the 3D joints and the 3D mesh vertices of the hand based on the posture-related contrast loss and the posture-independent contrast loss, and output the hand reconstruction result.
[0026] Specifically, step S1: Input a color (RGB) image containing a hand into the feature extractor, scale the image to a preset size, and apply various data augmentation operations to generate multi-view, multi-appearance image enhancement pairs; Step S2: Input the augmented images obtained in Step S1 into the pose decoupling module to extract basic hand features and map the basic features into pose-related / irrelevant feature subspaces; Step S3: Construct pose-dependent and pose-independent contrastive learning losses, and constrain the pose-dependent / pose-independent contrastive losses respectively; Step S4: Transfer the feature encoder obtained through pose decoupling and contrastive learning to the 3D reconstruction module, predict the coordinates of the 3D joints and the 3D mesh vertices of the hand through supervised fine-tuning, and finally output the hand reconstruction result.
[0027] Further, extracting the basic hand features from the enhanced image and mapping these basic hand features to pose-related features includes: The enhanced image is input into the backbone network for processing to obtain a fine-grained feature map; The fine-grained feature map is deconvolutionally upsampled to obtain a high-resolution feature map; Average pooling and convolution processing are performed on the high-resolution feature map in different directions to obtain one-dimensional heatmaps for keypoint detection in different directions; The coordinates of the two-dimensional joints of the hand are extracted from the one-dimensional heatmap using soft-argmax, and the coordinates of the two-dimensional joints of the hand are converted into a Gaussian heatmap. The attitude-related features are obtained by performing feature projection on the Gaussian heatmap.
[0028] Further, extracting the basic hand features from the enhanced image and mapping these basic hand features to pose-independent features includes: The enhanced image is input into the backbone network for processing to obtain a fine-grained feature map; The fine-grained feature map is then subjected to global average pooling to obtain the feature vector; The feature vector is input into a projection head containing two fully connected layers and one batch normalization layer, and mapped to the pose-independent feature.
[0029] Specifically, Figure 2 The detailed structure of the pose decoupling module is shown. This module is used to explicitly learn pose-related and pose-independent features from the input image. The specific steps are as follows: Step 101: The original image is processed using ResNet-50 to obtain fine-grained feature maps. ; Step 102: Through deconvolution pairs Upsampling is performed to obtain a high-resolution feature map. ; Step 103: Adjust the x-axis and y-axis directions respectively. Perform average pooling to obtain and ; Step 104: For each and Applying 1×1 convolution, heatmaps of the joints along the x and y axes are obtained. and ; Step 105: Extract two-dimensional coordinates from the heatmap using soft-argmax. ; Step 106: Design the two-dimensional attitude estimation loss function as follows: ; in, Let j represent the number of joints, and j represent the j-th joint. This represents the predicted coordinates of the j-th joint. Represents the corresponding real coordinates, It is an L1 norm.
[0030] Step 107: Estimate the two-dimensional attitude coordinates Convert to Gaussian heatmap We use three convolutional layers and one fully connected layer to project features onto the Gaussian heatmap to obtain pose-related feature embeddings. ; Step 108: Extract feature maps from the backbone network Perform global average pooling to obtain a 3D eigenvectors The feature vector is input into a projection head containing two fully connected layers and one batch normalization layer, which maps it to... 3D pose-independent feature embedding representation .
[0031] Furthermore, Figure 3 This demonstrates a contrastive learning module for pose-related and pose-independent features. This module is used to learn two types of feature representations for pose, and the specific steps are as follows: Step 201: Given an input image I and its pose p, obtain positive sample pairs through pose-invariant data augmentation methods (such as color jittering, blurring, cropping, etc.). Simultaneously, pose-related data augmentation techniques (such as rotation operations) are employed to generate negative sample pairs. and These two pairs of negative samples have different pose information; Step 202: Extract features from positive and negative sample pairs, and input the features into the pose decoupling module to extract the corresponding pose-related feature embedding representations. ; Step 203: Design the attitude-related contrastive loss function as follows: ; ; in, It's a temperature over-parameter. Cosine similarity is used to measure the similarity between two feature vectors. Embedding representations for pose-related features Indicates For anchor points, For positive samples, the pose-related contrastive loss term is used. Indicates For anchor points, denoted as the pose-related contrastive loss term for positive samples, m is the index of the negative sample, and M is the total number of negative samples participating in the contrastive learning. () represents an exponential function.
[0032] Step 204: Randomly select N images and use pose-invariant data augmentation to generate two related views of the same gesture as positive sample pairs. For each positive sample pair, pose-independent feature embeddings are extracted using the pose decoupling module. ; Step 205: Design the pose-independent contrastive loss function as follows: ; in, For pose-independent feature embedding representation. Indicates For anchor points, For positive samples, the pose-independent contrastive loss term is used. Indicates For anchor points, For positive samples, the pose-independent contrastive loss term is used. Indicates an indicator function, when The value is 1 if the condition is met, and 0 otherwise. These are negative sample pairs, and N represents the number of sample pairs participating in the contrastive learning.
[0033] Step 206: Three losses work together to optimize the model: the 2D pose estimation loss, the pose-dependent contrastive loss, and the pose-independent contrastive loss. Therefore, the total loss is as follows: ; Furthermore, based on the posture-dependent contrast loss and posture-independent contrast loss, the coordinates of the three-dimensional joints of the hand and the coordinates of the three-dimensional mesh of the hand are estimated, and the hand reconstruction results are output, including: The feature encoding module is optimized using the pose-related contrast loss and pose-independent contrast loss to obtain the optimized feature encoding module; The optimized feature encoding module is then transferred to the 3D reconstruction module. The 3D reconstruction module is used to estimate the coordinates of the 3D joints and the vertices of the 3D mesh of the hand, and the hand reconstruction result is output.
[0034] Furthermore, the 3D reconstruction module includes: a PoseNet unit, a MeshNet unit, a projection unit, and a mapping unit; The PoseNet unit is used to predict the 2.5D coordinates of hand joints based on the output features of the feature encoding module. The MeshNet unit is used to predict the 2.5D coordinates of the hand mesh based on the 2.5D coordinates; The projection unit is used to back-project the 2.5D coordinates of the hand mesh into three-dimensional space to obtain the vertex coordinates of the three-dimensional mesh; The mapping unit is used to map the coordinates of the three-dimensional mesh vertex to the coordinates of the three-dimensional hand joints according to the preset joint regression matrix, and use the coordinates of the three-dimensional hand joints as the hand reconstruction result.
[0035] Specifically, Figure 4 The detailed structure of the 3D reconstruction module is shown. This module adopts the same structure as the I2L-MeshNet architecture and consists of two sub-modules: PoseNet and MeshNet. The specific steps are as follows: Step 301: Input the encoder features of the input image into PoseNet, and directly predict the 2.5D coordinates of 21 hand joints through regression. ; Step 302: Use the obtained 2.5D joint coordinates as structural guides and input them into MeshNet. MeshNet first predicts the 2.5D coordinates of the hand mesh. ; Step 303: Using the camera projection matrix corresponding to the image and the camera extrinsic parameters, backproject the 2.5D mesh coordinates into the real 3D space to obtain the final 3D mesh points of the hand. The process is as follows: ; in This represents the inverse transformation matrix that maps 2.5D coordinates from image space to the camera coordinate system. This represents the camera's extrinsic translation vector. Indicates the inverse transformation of camera projection; Step 304: The joint coordinates are obtained from mesh regression: ; in The joint regression matrix used by I2L-MeshNet maps mesh vertices to 21 3D joint positions.
[0036] Furthermore, the reconstruction method also includes: A total loss function is constructed based on the two-dimensional pose estimation loss, pose-related contrast loss, and pose-independent contrast loss. The total loss function is then used to optimize the single-handed reconstruction model by weighted summation.
[0037] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A single-hand reconstruction method based on posture decoupling, characterized in that, include: Obtain an RGB image containing the hand, perform data augmentation processing on the RGB image, and obtain multiple augmented images; The enhanced image is input into a single-hand reconstruction model to obtain the hand reconstruction result. The single-hand reconstruction model includes a pose decoupling module, a feature encoding module, and a three-dimensional reconstruction module. The pose decoupling module is used to extract the basic hand features of the enhanced image and map the basic hand features into pose-related features and pose-independent features; The feature encoding module is used to calculate the pose-related contrast loss and the pose-independent contrast loss based on the pose-related features and the pose-independent features, respectively. The 3D reconstruction module is used to estimate the coordinates of the 3D joints and the 3D mesh vertices of the hand based on the posture-related contrast loss and the posture-independent contrast loss, and output the hand reconstruction result.
2. The single-hand reconstruction method based on attitude decoupling according to claim 1, characterized in that, Extracting the basic hand features from the enhanced image and mapping these basic hand features to pose-related features includes: The enhanced image is input into the backbone network for processing to obtain a fine-grained feature map; The fine-grained feature map is deconvolutionally upsampled to obtain a high-resolution feature map; Average pooling and convolution processing are performed on the high-resolution feature map in different directions to obtain one-dimensional heatmaps for keypoint detection in different directions; The coordinates of the two-dimensional joints of the hand are extracted from the one-dimensional heatmap using soft-argmax, and the coordinates of the two-dimensional joints of the hand are converted into a Gaussian heatmap. The attitude-related features are obtained by performing feature projection on the Gaussian heatmap.
3. The single-hand reconstruction method based on attitude decoupling according to claim 2, characterized in that, Before converting the hand's two-dimensional joint coordinates into a Gaussian heatmap, the following steps are also included: The two-dimensional joint coordinates of the hand are optimized using a two-dimensional pose estimation loss, wherein the two-dimensional pose estimation loss... for: ； in, Let j represent the number of joints, and j represent the j-th joint. This represents the predicted coordinates of the j-th joint. Represents the corresponding real coordinates, It is an L1 norm.
4. The single-hand reconstruction method based on attitude decoupling according to claim 1, characterized in that, Extracting the basic hand features from the enhanced image and mapping these basic hand features to pose-independent features includes: The enhanced image is input into the backbone network for processing to obtain a fine-grained feature map; The fine-grained feature map is then subjected to global average pooling to obtain the feature vector; The feature vector is input into a projection head containing two fully connected layers and one batch normalization layer, and mapped to the pose-independent feature.
5. The single-hand reconstruction method based on attitude decoupling according to claim 3, characterized in that, Calculate attitude-related contrast loss include: ； ； in, It's a temperature over-parameter. Cosine similarity is used to measure the similarity between two feature vectors. Embedding representations for pose-related features Indicates For anchor points, For positive samples, the pose-related contrastive loss term is used. Indicates For anchor points, denoted as the pose-related contrastive loss term for positive samples, m is the index of the negative sample, and M is the total number of negative samples participating in the contrastive learning. () represents an exponential function.
6. The single-hand reconstruction method based on attitude decoupling according to claim 5, characterized in that, Calculate pose-independent contrast loss include: ； in, For pose-independent feature embedding representation. Indicates For anchor points, For positive samples, the pose-independent contrastive loss term is used. Indicates For anchor points, For positive samples, the pose-independent contrastive loss term is used. Indicates an indicator function, when The value is 1 if the condition is met, and 0 otherwise. These are negative sample pairs, and N represents the number of sample pairs participating in the contrastive learning.
7. The single-hand reconstruction method based on attitude decoupling according to claim 6, characterized in that, Based on the posture-dependent contrast loss and posture-independent contrast loss, the coordinates of the 3D joints and the vertex coordinates of the 3D mesh of the hand are estimated, and the hand reconstruction results are output, including: The feature encoding module is optimized using the pose-related contrast loss and pose-independent contrast loss to obtain the optimized feature encoding module; The optimized feature encoding module is then transferred to the 3D reconstruction module. The 3D reconstruction module is used to estimate the coordinates of the 3D joints and the vertices of the 3D mesh of the hand, and the hand reconstruction result is output.
8. The single-hand reconstruction method based on attitude decoupling according to claim 7, characterized in that, The 3D reconstruction module includes: a pose regression unit, a mesh regression unit, a projection unit, and a mapping unit; The posture regression unit is used to predict the 2.5D coordinates of hand joints based on the output features of the feature encoding module. The mesh regression unit is used to predict the 2.5D coordinates of the hand mesh based on the 2.5D coordinates; The projection unit is used to back-project the 2.5D coordinates of the hand mesh into three-dimensional space to obtain the vertex coordinates of the three-dimensional mesh; The mapping unit is used to map the coordinates of the three-dimensional mesh vertex to the coordinates of the three-dimensional hand joints according to the preset joint regression matrix, and use the coordinates of the three-dimensional hand joints as the hand reconstruction result.
9. A single-hand reconstruction method based on attitude decoupling according to claim 8, characterized in that, The reconstruction method further includes: A total loss function is constructed based on the two-dimensional pose estimation loss, pose-related contrast loss, and pose-independent contrast loss. The total loss function is then used to optimize the single-handed reconstruction model by weighted summation.