Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Nonlinear 3DMM face reconstruction and posture normalization method and device, medium and equipment

A non-linear, normalized technology, applied in the field of computer vision, can solve problems such as faces that cannot handle large pose changes well

Pending Publication Date: 2021-01-12
BEIJING TECHSHINO TECH +1
View PDF0 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, they only use 2D faces and sparse ground truth, so they cannot handle faces with large pose variations well

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Nonlinear 3DMM face reconstruction and posture normalization method and device, medium and equipment
  • Nonlinear 3DMM face reconstruction and posture normalization method and device, medium and equipment
  • Nonlinear 3DMM face reconstruction and posture normalization method and device, medium and equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0096] Implementation of the present invention provides a nonlinear 3DMM face reconstruction method, such as figure 1 As shown, the method includes:

[0097] Step S100: use the training set to train the nonlinear 3DMM model.

[0098] Among them, the training set includes multiple 2D face image samples, and the nonlinear 3DMM model includes a CNN encoder, a multi-layer perception shape decoder, a CNN texture decoder, and a rendering layer.

[0099] The nonlinear 3DMM model of the present invention includes an encoder and two decoders, the encoder is a convolutional neural network, and the convolutional neural network (convolution neural network, CNN) is a deep learning method through which data can be extracted The deep-level feature descriptor, and the descriptor can be applied to tasks such as vision, speech and text. One of the two decoders is a shape decoder, the other is a texture decoder, the shape decoder is a deep convolutional neural network (CNN), the texture decode...

Embodiment 2

[0139] Embodiments of the present invention provide a nonlinear 3DMM face reconstruction device corresponding to the nonlinear 3DMM face reconstruction method described in Embodiment 1, such as Figure 5 As shown, the device includes:

[0140] The training module 10 is used to use the training set to train the nonlinear 3DMM model.

[0141] Among them, the training set includes multiple 2D face image samples, and the nonlinear 3DMM model includes a CNN encoder, a multi-layer perception shape decoder, a CNN texture decoder, and a rendering layer.

[0142] During training, the 2D face image sample input into the nonlinear 3DMM model is estimated by the CNN encoder to obtain the camera projection parameters, shape parameters and texture parameters. The multi-layer perception shape decoder decodes the shape parameters into 3D shapes, and the CNN texture decoder will The texture parameter is decoded into a 3D texture, and the rendering layer obtains the rendered image according to...

Embodiment 3

[0166] The methods described in the above embodiments provided in this specification can implement business logic through computer programs and record them on a storage medium, and the storage medium can be read and executed by a computer to achieve the effect of the solution described in Embodiment 1 of this specification. Therefore, the present invention also provides a computer-readable storage medium for nonlinear 3DMM face reconstruction corresponding to the nonlinear 3DMM face reconstruction method in Embodiment 1, including a memory for storing processor-executable instructions, the instruction When executed by the processor, the steps of the nonlinear 3DMM face reconstruction method in Embodiment 1 are realized.

[0167]In view of the obstacles of the existing linear 3DMM in its data, supervision and linear basis, the present invention learns a nonlinear 3DMM model of facial shape and texture from a set of unconstrained 2D face images by innovating the learning paradigm...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a nonlinear 3DMM face reconstruction and attitude normalization method and device, a medium and equipment, and belongs to the field of computer vision. The method comprises thefollowing steps: training a model, and inputting a 2D face image into the model to obtain a 3D face, wherein the model comprises a CNN encoder, a shape decoder, a texture decoder and a rendering layer; a CNN encoder estimates a 2D face image sample to obtain a camera projection parameter, a shape parameter and a texture parameter, and a shape decoder and a texture decoder decode the shape parameter and the texture parameter into a 3D shape and a 3D texture. During training, a rendering layer obtains a rendering image, and a model is trained through a loss function. And during prediction, therendering layer performs 3D rendering to obtain a 3D face. The method has higher representation capability than a linear 3DMM, training and prediction are carried out end to end, network training canbe carried out by utilizing a 2D image without 3D face scanning, and the reconstructed 3D face is high in recognition accuracy after normalization.

Description

technical field [0001] The present invention relates to the field of computer vision, in particular to a nonlinear 3DMM face reconstruction method, device, computer-readable storage medium and equipment, and a face posture normalization method, device, and computer based on the nonlinear 3DMM reconstruction method Readable storage media and devices. Background technique [0002] In the face image recognition technology, the posture of the face is an important factor affecting the face recognition rate. The face image recognition of the prior art is mainly the recognition of the front face image or the small gesture (angle) face image. The recognition results of large-pose face images are not ideal. In order to improve the recognition accuracy, it is necessary to normalize the face images (especially large-pose face images). [0003] The aforementioned face images, small pose face images and large pose face images are all 2D face images. The facial posture normalization met...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/00G06T17/00G06T15/00G06N3/04
CPCG06T17/00G06T15/005G06V20/64G06V40/161G06N3/045
Inventor 周军刘利朋江武明丁松
Owner BEIJING TECHSHINO TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com