Virtual human generation method and device based on monocular vision

A monocular vision and virtual human technology, applied in neural learning methods, biological neural network models, image data processing, etc., can solve problems such as lack of a large amount of input information, high difficulty, etc., and achieve good interactive effects

Pending Publication Date: 2021-11-16
TSINGHUA UNIV
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, in related technologies, it is very difficult to generate a 3D virtual perso

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Virtual human generation method and device based on monocular vision
  • Virtual human generation method and device based on monocular vision

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0026] Example one

[0027] figure 1 A flow diagram of a single-visual visual virtual human generating method provided in accordance with an embodiment of the present application, such as figure 1 As shown, it may include:

[0028] Step 101 acquire a single color image.

[0029] Step 102: Extract the image characteristics of a single color image, and estimate the posture of the human body torso and limbs based on image features of the single color image to complete the reconstruction of the torso and limbs of the virtual character model.

[0030] The present application obtains the image characteristics of the single-graphic color image in high dimensional space by using the depth residual network processing, and then uses the three-dimensional coordinates of the human key point based on the image feature based on the image feature. And use full-connected neural networks to return to the body joint rotation parameters and the body of the human body according to the three-dimensio...

Example Embodiment

[0042] Example 2

[0043] Further, based on the above-described embodiment, a single-visual visual virtual human generating method is further provided, the present application also provides a single-visual visual virtual person generating apparatus 200, figure 2 A structural diagram of a single-visual visual virtual person generating device provided in accordance with an embodiment of the present application, such as figure 2 As shown, it may include:

[0044] Get module 201 for acquiring a single color image;

[0045] The first estimation module 202 is an image feature for extracting a single color image, and estimates the posture of the human body torso and limbs based on the image characteristics of the single color image to complete the reconstruction of the torso and limbs of the virtual character model;

[0046] The second estimation module 203 is configured to locate the hand position information in a single color image in a single-color image, and intercept the hand image ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a virtual human generation method based on monocular vision. The method comprises the following steps: acquiring a monocular color image; extracting image features of the monocular color image, estimating postures of the trunk and the limbs of the human body, and completing reconstruction of the trunk and the limbs of the virtual character model; positioning hand position information in the monocular color image, intercepting a hand image area in the monocular color image, and extracting image features of the hand image area to estimate hand rotation parameters; positioning face position information in the monocular color image, intercepting a face image area in the monocular color image, extracting image features of the face image area, estimating shape parameters and expression coefficients of the face, and generating a face model; and completing hand reconstruction by using the hand rotation parameter, and replacing a face region of the three-dimensional human body model by using the face model to complete face reconstruction so as to obtain a virtual character model. Therefore, the invention provides a method for generating the virtual human body model in real time through the monocular color image.

Description

technical field [0001] The present application relates to the technical fields of computer vision and computer graphics, and in particular to a monocular vision-based virtual human generation method, device and storage medium. Background technique [0002] The generation of virtual human has a very wide range of applications in the field of virtual reality and mixed reality. The generation of a virtual human requires that the reconstruction result be consistent with that of a real human being, and it needs to contain a lot of details. Therefore, it is necessary to perform a complete motion capture of the target object, that is, to simultaneously reconstruct the large-scale activities of the body trunk and limbs, the gesture movement of the hands, and the face. expression changes. [0003] However, in the related art, it is very difficult to generate a 3D virtual person only through a monocular color image due to the lack of a large amount of input information. Contents of...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T13/40G06T17/00G06K9/00G06N3/04G06N3/08
CPCG06T13/40G06T17/00G06N3/08G06N3/045
Inventor 徐枫周玉枭
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap