Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Virtual human generation method and device based on monocular vision

A monocular vision and virtual human technology, applied in neural learning methods, biological neural network models, image data processing, etc., can solve problems such as lack of a large amount of input information, high difficulty, etc., and achieve good interactive effects

Pending Publication Date: 2021-11-16
TSINGHUA UNIV
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, in related technologies, it is very difficult to generate a 3D virtual person only through monocular color images due to the lack of a large amount of input information.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Virtual human generation method and device based on monocular vision
  • Virtual human generation method and device based on monocular vision

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0027] figure 1 It is a schematic flowchart of a monocular vision-based virtual human generation method provided according to an embodiment of the present application, such as figure 1 As shown, can include:

[0028] Step 101. Obtain a monocular color image.

[0029] Step 102, extracting the image features of the monocular color image, and estimating the posture of the human torso and limbs based on the image features of the monocular color image, so as to complete the reconstruction of the torso and limbs of the virtual character model.

[0030] Among them, the application uses the deep residual network to process the obtained monocular color image to obtain the image features of the monocular color image in the high-dimensional space, and then uses the convolutional neural network to estimate the three-dimensional coordinates of the key points of the human body based on the image features. And use the fully connected neural network to return the rotation parameters of the ...

Embodiment 2

[0043] Further, based on the monocular vision-based virtual human generation method provided in the above embodiments, the embodiment of the present application also provides a monocular vision-based virtual human generation device 200, figure 2 It is a schematic structural diagram of a virtual human generation device based on monocular vision provided according to an embodiment of the present application, such as figure 2 As shown, can include:

[0044] An acquisition module 201, configured to acquire a monocular color image;

[0045] The first estimation module 202 is used to extract the image features of the monocular color image, and estimate the posture of the human trunk and limbs based on the image features of the monocular color image, so as to complete the reconstruction of the torso and limbs of the virtual character model;

[0046] The second estimating module 203 is used to respectively locate the hand position information in the monocular color image by using t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a virtual human generation method based on monocular vision. The method comprises the following steps: acquiring a monocular color image; extracting image features of the monocular color image, estimating postures of the trunk and the limbs of the human body, and completing reconstruction of the trunk and the limbs of the virtual character model; positioning hand position information in the monocular color image, intercepting a hand image area in the monocular color image, and extracting image features of the hand image area to estimate hand rotation parameters; positioning face position information in the monocular color image, intercepting a face image area in the monocular color image, extracting image features of the face image area, estimating shape parameters and expression coefficients of the face, and generating a face model; and completing hand reconstruction by using the hand rotation parameter, and replacing a face region of the three-dimensional human body model by using the face model to complete face reconstruction so as to obtain a virtual character model. Therefore, the invention provides a method for generating the virtual human body model in real time through the monocular color image.

Description

technical field [0001] The present application relates to the technical fields of computer vision and computer graphics, and in particular to a monocular vision-based virtual human generation method, device and storage medium. Background technique [0002] The generation of virtual human has a very wide range of applications in the field of virtual reality and mixed reality. The generation of a virtual human requires that the reconstruction result be consistent with that of a real human being, and it needs to contain a lot of details. Therefore, it is necessary to perform a complete motion capture of the target object, that is, to simultaneously reconstruct the large-scale activities of the body trunk and limbs, the gesture movement of the hands, and the face. expression changes. [0003] However, in the related art, it is very difficult to generate a 3D virtual person only through a monocular color image due to the lack of a large amount of input information. Contents of...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T13/40G06T17/00G06K9/00G06N3/04G06N3/08
CPCG06T13/40G06T17/00G06N3/08G06N3/045
Inventor 徐枫周玉枭
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com