Parameter estimation model training method and device, equipment and storage medium

A technology for estimating models and training methods, which is applied in the field of image processing, can solve the problems that the estimation accuracy of reconstruction parameters cannot be guaranteed, the operation of 3D face reconstruction is cumbersome and complex, and the acquisition of reconstruction parameters requires a lot, so as to improve the estimation accuracy and deformation process. Accurate, the effect of ensuring accuracy

Pending Publication Date: 2021-03-19
BIGO TECH PTE LTD
View PDF0 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] At present, the estimation of the reconstruction parameters corresponding to each principal component base is usually to directly use the pixel value of each feature point in the 2D face image as the supervision information of the 3D face deformation to estimate the reconstruction parameters corresponding to each principal component base. However, since it is a pathological problem from 2D images to 3D reconstruction, only the pixel values ​​of feature points are used as supervision information, which cannot guarantee the

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Parameter estimation model training method and device, equipment and storage medium
  • Parameter estimation model training method and device, equipment and storage medium
  • Parameter estimation model training method and device, equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0033]Example one

[0034]Figure 1A A flow chart of a training method for a parameter estimation model provided by the embodiment of the present invention, and this embodiment can be applied to any three-dimensional face reconstruction that needs to be estimated to estimate the corresponding reconstruction parameters. The training method of the parameter estimation model provided in this embodiment can be performed by a training device of the parameter estimation model provided by the embodiment of the present invention, which can be implemented by software and / or hardware, and integrated in the computer device in which the method is performed. in.

[0035]Specific, referenceFigure 1A The method can include the steps of:

[0036]S110, each training sample in the face image training is input to a pre-built neural network model, estimated the three-dimensional face reconstruction designated reconstruction parameters, and inputs the reconstructed parameter into a pre-built three-dimensional v...

Example Embodiment

[0054]Example 2

[0055]Figure 2A A flow chart of a training method for a parameter estimation model provided by the second embodiment of the present invention,Figure 2b A structural diagram of a three-dimensional variable model for three-dimensional face reconstruction is provided in the method of the second embodiment of the present invention. This embodiment is optimized based on the above embodiment.

[0056]It should be noted that the three-dimensional variable model in the present embodiment consists of a double primary component analysis (PCA) model and single PCA model, such asFigure 2b As shown, the double PCA model is mainly used to model the changes in face shape and expression in the process of three-dimensional face reconstruction, and the single PCA model is mainly used to change the change in face reaction rate during the three-dimensional face reconstruction. Modeling.

[0057]Among them, a three-dimensional average face in the dual PCA model in this embodiment may define a f...

Example Embodiment

[0076]Example three

[0077]Figure 3A A flow chart of a training method for a parameter estimation model provided in the third embodiment of the present invention,Figure 3B The principle of the training process of the parameter estimation model provided by the third embodiment of the present invention is shown. This embodiment is optimized based on the above embodiment. Specifically, such asFigure 3A As shown, the loss function under the plurality of two-dimensional supervision information can include: image pixel loss function, key point loss function, identity feature loss function, retrieval rate penalty function, and 3D face reconstruction designated reconstruction parameters The regular item corresponding to the reconstruction parameter is mainly explained in detail for the specific setting of the loss function under the training process in the training process.

[0078]Optional, such asFigure 3A As shown, in this embodiment, the steps can be included:

[0079]S301, inputs each training...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a parameter estimation model training method and device, equipment and a storage medium. The method comprises the following steps: inputting a face image training set into a pre-constructed neural network model, estimating reconstruction parameters specified by three-dimensional face reconstruction, inputting the reconstruction parameters into a pre-constructed three-dimensional deformation model, and reconstructing a three-dimensional face corresponding to a training sample; calculating loss functions between the three-dimensional face and the training sample under multiple items of two-dimensional supervision information, and adjusting the weight corresponding to each loss function; and generating a corresponding fitting loss function based on each loss function and the corresponding weight, and performing reverse correction on the neural network model by using the fitting loss function to obtain a trained parameter estimation model. According to the technicalscheme provided by the invention, the parameter estimation model is trained through the loss function under multiple items of two-dimensional supervision information, so that the reference information in the training process is more comprehensive, and the estimation accuracy of the reconstruction parameters adopted during three-dimensional face reconstruction is improved.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of image processing, and in particular, to a training method, device, device and storage medium for a parameter estimation model. Background technique [0002] With the development of video technology, there is an increasing demand for creating realistic face models in entertainment applications such as face animation, face recognition, and augmented reality (Augmented Reality, AR), which require face display. At present, it is very difficult to create a realistic 3D face model. It is necessary to reconstruct the corresponding 3D face from one or more 2D face images or depth images, so that it includes face shape, color, illumination and head. Various types of 3D information such as the internal rotation angle. [0003] In the existing 3D face reconstruction methods, a large amount of 3D scan data of faces is usually collected first, and then the 3D scan data is used to construct a cor...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T17/00G06K9/00G06K9/62G06N3/02G06N3/08
CPCG06T17/00G06N3/08G06N3/02G06V40/165G06F18/2135G06T7/00G06N3/04G06N3/045
Inventor 张小伟胡钟元刘更代
Owner BIGO TECH PTE LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap