Unlock instant, AI-driven research and patent intelligence for your innovation.

Method and apparatus for synthesizing realistic hand poses based on blending generative adversarial networks

A hand, image technique used in the field of generating realistic hand poses

Pending Publication Date: 2022-01-07
TENCENT AMERICA LLC
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, such technology is still lacking: an adjustment scheme for the foreground (hand) and background (natural scene where the hand appears)

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and apparatus for synthesizing realistic hand poses based on blending generative adversarial networks
  • Method and apparatus for synthesizing realistic hand poses based on blending generative adversarial networks
  • Method and apparatus for synthesizing realistic hand poses based on blending generative adversarial networks

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] Embodiments of the present application relate to generating realistic hand gestures. Existing hand pose estimation algorithms can be greatly improved by augmenting the training data with generated hand poses that are naturally annotated with ground truth. Specifically, an augmented reality simulator can synthesize hand poses with accurate 3D hand keypoint annotations. These synthetic hand poses may look unnatural and are not suitable for training. Although the synthesized hand poses come with precise joint labels, however, they look unnatural and unsuitable for training.

[0022] In order to generate more realistic hand poses, in the embodiment of the present application, each synthetic hand pose is blended with the real background, and a hybrid generative adversarial network (BlendGAN) is developed, which can align the synthetic hand poses with the real background tonal and color distributions, and can generate high-quality hand poses.

[0023] figure 1 An overview...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A method of synthesizing an image of a hand using a blending generative adversarial network (BlendGAN) includes obtaining a synthetic 3 -dimensional (3D) hand pose including a 3D model of a hand; obtaining a real background image; combining the synthetic 3D hand pose with the real background image to create a synthetic hand image; and blending the synthetic hand image using the BlendGAN to create a blended synthetic hand image.

Description

[0001] Cross References to Related Applications [0002] This application claims priority to U.S. Published Application No. 16 / 578,555, filed September 23, 2019, with the U.S. Patent Office, the entire contents of which are incorporated herein by reference. Background technique [0003] Three-dimensional (3D) hand pose estimation from a single red, green, and blue (RGB) image is important but challenging due to the lack of large enough hand pose datasets and accurate 3D hand keypoint annotations for training sex. Estimating hand pose from monocular RGB images has achieved significant progress due to the rapid development of deep neural networks (DNNs). These DNN-based methods combine learning hand representation and estimating pose. Although effective, DNN-based methods are highly dependent on a large amount of training data. However, using artificial hand keypoint annotations to collect all hand poses of interest for training is very expensive. [0004] Recent work on han...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06T15/50G06T19/00G06T7/70G06T7/13G06T7/90G06N3/04G06N3/08
CPCG06T7/70G06T2207/20084G06T15/503G06T19/00H04L67/10G06N3/08G06N3/047G06N3/045G06T7/13G06T7/90G06T2207/30196G06T2207/20081G06T2219/004
Inventor 林斯姚谢于晟谭辉韩连漪范伟
Owner TENCENT AMERICA LLC