A training method, device, equipment and storage medium for parameter estimation model

Abstract

The present invention discloses a training method, device, equipment and storage medium for a parameter estimation model. The method comprises: inputting a training set of face images into a pre-constructed neural network model, estimating the reconstruction parameters specified for three-dimensional face reconstruction, and inputting the reconstruction parameters into a pre-constructed three-dimensional deformation model to reconstruct the three-dimensional face corresponding to the training sample; calculating the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervisory information, and adjusting the weight corresponding to each loss function; generating a corresponding fitting loss function based on each loss function and the corresponding weight, and using the fitting loss function to reversely correct the neural network model to obtain a trained parameter estimation model. The technical solution provided by the present invention trains the parameter estimation model through the loss function under multiple two-dimensional supervisory information, so that the reference information in the training process is more comprehensive, and the estimation accuracy of the reconstruction parameters used in three-dimensional face reconstruction is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing technology, and in particular to a training method, apparatus, device and storage medium for a parameter estimation model. Background Art

[0002] With the development of video technology, there is an increasing demand for the creation of realistic face models in entertainment applications such as face animation, face recognition and augmented reality (AR) that require face display. Currently, creating a realistic 3D face model is a very difficult task, which requires reconstructing the corresponding 3D face from one or more 2D face images or depth images, including various 3D information such as face shape, color, lighting and head rotation angle.

[0003] In the existing three-dimensional face reconstruction method, a large amount of three-dimensional scanning data of human faces is usually collected first, and then the three-dimensional scanning data is used to construct a corresponding three-dimensional deformable model (3D morphable model, 3DMM). At this time, the three-dimensional deformable model will include an average facial shape of a standard face, a principal component basis representing changes in facial identity, and a principal component basis representing changes in facial expression. Then, based on the current two-dimensional face image to be reconstructed, reconstruction parameters corresponding to the two sets of principal component bases are estimated, and the two sets of principal component bases are adjusted accordingly through the reconstruction parameters to deform the average facial shape accordingly, thereby reconstructing the corresponding three-dimensional face.

[0004] At present, the estimation of the reconstruction parameters corresponding to each principal component basis is usually to directly use the pixel values ​​of each feature point in the two-dimensional face image as the supervision information of the three-dimensional face deformation to estimate the reconstruction parameters corresponding to each principal component basis. However, since the reconstruction from two-dimensional images to three-dimensional images is a pathological problem, only the pixel values ​​of the feature points are used as supervision information, and the estimation accuracy of the reconstruction parameters cannot be guaranteed; alternatively, the multi-perspective two-dimensional face images or depth information to be reconstructed are used as input to estimate the reconstruction parameters corresponding to each principal component basis, but it is necessary to collect multiple face images and even special sensors to collect depth images, which limits the reconstruction scene and requires too many reconstruction parameter collection requirements, making the three-dimensional face reconstruction operation more cumbersome and complicated. Summary of the invention

[0005] The embodiments of the present invention provide a training method, apparatus, device and storage medium for a parameter estimation model, which optimize the training method of the parameter estimation model used to estimate corresponding reconstruction parameters during three-dimensional face reconstruction, improve the estimation accuracy of the reconstruction parameters used during three-dimensional face reconstruction, and reduce the operational complexity of three-dimensional face reconstruction while ensuring the accuracy of three-dimensional face reconstruction.

[0006] In a first aspect, an embodiment of the present invention provides a method for training a parameter estimation model, the method comprising:

[0007] Input each training sample in the face image training set into a pre-built neural network model, estimate the reconstruction parameters specified for 3D face reconstruction, and input the reconstruction parameters into a pre-built 3D deformable model to reconstruct the 3D face corresponding to the training sample;

[0008] Calculating a loss function between the three-dimensional face and the training sample under multiple two-dimensional supervisory information, and adjusting a weight corresponding to each loss function;

[0009] A corresponding fitting loss function is generated based on each loss function and the corresponding weight, and the neural network model is reversely corrected using the fitting loss function to obtain a trained parameter estimation model.

[0010] In a second aspect, an embodiment of the present invention provides a training device for a parameter estimation model, the device comprising:

[0011] A 3D reconstruction module, for inputting each training sample in the face image training set into a pre-built neural network model, estimating reconstruction parameters specified for 3D face reconstruction, and inputting the reconstruction parameters into a pre-built 3D deformable model to reconstruct a 3D face corresponding to the training sample;

[0012] A loss function calculation module, used to calculate the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervision information, and adjust the weight corresponding to each loss function;

[0013] The model training module is used to generate a corresponding fitting loss function based on each loss function and the corresponding weight, and use the fitting loss function to reversely correct the neural network model to obtain a trained parameter estimation model.

[0014] In a third aspect, an embodiment of the present invention provides a computer device, the computer device comprising:

[0015] one or more processors;

[0016] A storage device for storing one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the training method of the parameter estimation model described in any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the training method of the parameter estimation model described in any embodiment of the present invention.

[0019] The training method, apparatus, device and storage medium of a parameter estimation model provided by an embodiment of the present invention pre-construct a corresponding neural network model for each reconstruction parameter required for three-dimensional face reconstruction. Each training sample in a face image training set is input into the neural network model to estimate the reconstruction parameters required for the training sample in three-dimensional face reconstruction. The reconstruction parameters are input into a pre-constructed three-dimensional deformation model to reconstruct the three-dimensional face corresponding to the training sample. At this time, a fitting loss function for the neural network model training can be generated by calculating the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervisory information and adjusting the weight corresponding to each loss function. The fitting loss function only needs to use the two-dimensional face information under multiple supervisory information. , without referring to additional three-dimensional face information before reconstructing the three-dimensional face, the neural network model can be continuously reversely corrected to obtain a trained parameter estimation model, thereby optimizing the training method of the parameter estimation model used to estimate the corresponding reconstruction parameters in the three-dimensional face reconstruction, and training the parameter estimation model through a loss function under multiple two-dimensional supervision information, so that the reference information in the training process is more comprehensive, and the estimation accuracy of the reconstruction parameters used in the three-dimensional face reconstruction is improved; at the same time, the trained parameter estimation model is subsequently used to estimate the reconstruction parameters in the three-dimensional face reconstruction, so that the deformation process of the face image to be reconstructed in the three-dimensional deformation model is more accurate, thereby ensuring the accuracy of the three-dimensional face reconstruction, without deploying additional information equipment in the three-dimensional face reconstruction process, and reducing the operational complexity of the three-dimensional face reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0021] Figure 1A A flowchart of a method for training a parameter estimation model provided in Embodiment 1 of the present invention;

[0022] Figure 1B A schematic diagram of the principle of the training process of the parameter estimation model provided in the first embodiment of the present invention;

[0023] Figure 1C A schematic diagram of the principle of a three-dimensional face reconstruction process in the method provided in Embodiment 1 of the present invention;

[0024] Figure 2A A flowchart of a method for training a parameter estimation model provided in Embodiment 2 of the present invention;

[0025] Figure 2B A schematic diagram of the structure of a three-dimensional deformable model used for three-dimensional face reconstruction in the method provided in the second embodiment of the present invention;

[0026] Figure 3A A flowchart of a method for training a parameter estimation model provided in Embodiment 3 of the present invention;

[0027] Figure 3B A schematic diagram of the principle of the training process of the parameter estimation model provided in the third embodiment of the present invention;

[0028] Figure 3C A schematic diagram of dynamically selecting key feature points in the method provided in Embodiment 3 of the present invention;

[0029] Figure 3D A schematic diagram of the distribution of key feature points in the method provided in Embodiment 3 of the present invention;

[0030] Figure 4 A schematic diagram of the structure of a training device for a parameter estimation model provided in a fourth embodiment of the present invention;

[0031] Figure 5 A schematic diagram of the structure of a computer device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only the parts related to the present invention, rather than all structures, are shown in the accompanying drawings. In addition, the embodiments of the present invention and the features in the embodiments may be combined with each other without conflict.

[0033] Embodiment 1

[0034] Figure 1A This is a flow chart of a parameter estimation model training method provided in Embodiment 1 of the present invention. This embodiment can be applied to any situation of 3D face reconstruction that requires estimation of corresponding reconstruction parameters. The parameter estimation model training method provided in this embodiment can be executed by a parameter estimation model training device provided in an embodiment of the present invention. The device can be implemented in software and / or hardware and integrated in a computer device that executes this method.

[0035] Specifically, refer to Figure 1A , the method may include the following steps:

[0036] S110, input each training sample in the face image training set into a pre-built neural network model, estimate the reconstruction parameters specified for three-dimensional face reconstruction, and input the reconstruction parameters into a pre-built three-dimensional deformation model to reconstruct the three-dimensional face corresponding to the training sample.

[0037] Optionally, currently, face animation, face recognition, augmented reality, and face beautification all require the application of 3D face reconstruction technology. 3D face reconstruction refers to the reconstruction of 3D geometric shape, albedo, illumination information, head rotation angle, and other 3D information of a 2D face image, wherein the 3D geometric shape can be composed of a set of vertices in a 3D space, and each vertex can be uniquely determined by a corresponding 3D coordinate (x, y, z). At this time, for the existing 3D face reconstruction, the main thing is to pre-construct a corresponding 3D deformation model, and use the 3D deformation model to model the 3D face shape and appearance of any 2D face image, and propose that any face shape can be represented as a standard face and a set of linear combinations of principal component vectors representing face shape changes and expression changes, so as to deform the average face shape and adjust the face expression through different linear combinations of principal component vectors, thereby reconstructing the corresponding 3D face.

[0038] Among them, the reconstruction parameters specified for the three-dimensional face reconstruction in this embodiment refer to the various parameters that can represent the linear combination form of the referenced principal component vectors when the three-dimensional deformable model deforms the average face shape and adjusts the facial expression, as well as the parameters that affect the realistic effect of the three-dimensional face, such as lighting parameters and face position, posture, etc.; at this time, the reconstruction parameters can control the corresponding deformation of the average face shape and the change of facial expression, thereby generating a corresponding three-dimensional face, and improving the detailed display of the lighting conditions and posture angles in the three-dimensional face. It can be seen that the accuracy of estimating the reconstruction parameters for the two-dimensional face image to be reconstructed can directly affect the realism of the three-dimensional face corresponding to the reconstruction of the two-dimensional face image. Therefore, in order to make the reconstructed three-dimensional face more realistic, it is required to accurately estimate the various reconstruction parameters specified for the three-dimensional face reconstruction from the two-dimensional face image to be reconstructed. Specifically, the reconstruction parameters in this embodiment may include deformation parameters representing changes in face shape, expression parameters representing changes in face expression, albedo parameters representing changes in face albedo, illumination parameters representing changes in face illumination, position parameters representing face translation, and rotation parameters representing face head posture, etc.; wherein the albedo parameters include RGB color information of the two-dimensional face image to be reconstructed.

[0039] Optionally, in order to accurately estimate the various reconstruction parameters specified for three-dimensional face reconstruction for any two-dimensional face image to be reconstructed, this embodiment will initially construct a neural network model, and then use a large number of face image samples to perform parameter estimation training on the neural network model, so as to train a parameter estimation model that can accurately estimate the reconstruction parameters specified for three-dimensional face reconstruction, wherein the neural network model adopts a convolutional neural network. When performing parameter estimation training on the neural network model, a corresponding face image training set will first be constructed, and the face image training set contains a large number of face images from different sources and different types as training samples for the neural network model; Figure 1B As shown, each training sample in the face image training set is continuously input into the pre-built neural network model, and then the neural network model compares and analyzes the facial features in the input training sample with the facial features of the standard face, and determines the various operation parameters required to transform the standard face into the face in the training sample, thereby estimating the values ​​of the reconstruction parameters required to be used when the training sample is used for three-dimensional face reconstruction. At this time, in this embodiment, the accuracy of the estimated reconstruction parameters can be determined by analyzing the degree of realism of the reconstructed three-dimensional face after the three-dimensional deformation model uses the reconstruction parameters to reconstruct the training sample for three-dimensional face.

[0040] Therefore, after estimating the reconstruction parameters used for three-dimensional face reconstruction of the training sample through the neural network model, the reconstruction parameters will also be input into the pre-constructed three-dimensional deformation model. The three-dimensional deformation model will use the reconstruction parameters to perform corresponding deformation, expression changes and albedo changes on the standard face defined therein, and adjust the corresponding three-dimensional detail display information accordingly, so as to reconstruct the three-dimensional face corresponding to the training sample. Subsequently, by comparing the similarity between the training sample and the reconstructed three-dimensional face, the realism of the reconstructed three-dimensional face can be analyzed.

[0041] S120, calculating the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervision information, and adjusting the weight corresponding to each loss function.

[0042] Optionally, in order to analyze the degree of realism of the reconstructed three-dimensional face relative to the faces in the training samples, the present embodiment will pre-set a corresponding loss function, and use the loss function to compare the similarity between the reconstructed three-dimensional face and the training samples; the multiple two-dimensional supervision information in the present embodiment indicates that the similarity between the reconstructed three-dimensional face and the training samples in each supervision dimension can be more comprehensively judged through multiple two-dimensional supervision information of different numbers. At this time, setting the loss function under multiple two-dimensional supervision information can avoid the reconstruction error of the three-dimensional face as comprehensively as possible; exemplarily, the loss function under multiple two-dimensional supervision information in the present embodiment may include: image pixel loss function, key point loss function, identity feature loss function, albedo penalty function and regularization terms corresponding to the target reconstruction parameters in the reconstruction parameters specified for the three-dimensional face reconstruction.

[0043] Specifically, after reconstructing the three-dimensional face corresponding to the training sample, the reconstruction error between the reconstructed three-dimensional face and the training sample in each dimension of the supervision information can be calculated through the currently set loss function under multiple two-dimensional supervision information, that is, the specific value of the loss function, and then the realism of the reconstructed three-dimensional face under each supervision information is analyzed according to the specific value of each loss function, so as to judge the estimation accuracy of the trained neural network model under each supervision information, and then adjust the weight corresponding to each loss function accordingly to improve the estimation ability during the next training.

[0044] Exemplarily, this embodiment can clearly determine the ability of the current training process to accurately reconstruct image pixels, key points, identity features, albedo and various reconstruction parameters during three-dimensional face reconstruction by respectively calculating the image pixel loss function, key point loss function, identity feature loss function, albedo penalty function and regularization terms corresponding to the target reconstruction parameters in the reconstruction parameters specified for three-dimensional face reconstruction between the three-dimensional face and the training sample, and adjust the weights of each loss function according to the specific reconstruction capability, and continue training, thereby continuously improving the ability to estimate reconstruction parameters during three-dimensional face reconstruction.

[0045] In addition, since the reconstructed 3D face may only contain the 3D face image, and the training sample displays the background image of non-face images in addition to the face image, in order to ensure the accuracy of the loss calculation between the 3D face and the training sample, Figure 1B As shown, before calculating the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervisory information, this embodiment may also include: rendering the three-dimensional face using a differentiable renderer to use the rendered three-dimensional face training parameter estimation model.

[0046] Specifically, a differentiable renderer is used to render a three-dimensional face, so that the texture and image of the rendered three-dimensional face can be closer to the training sample, so that the rendered three-dimensional face can be used to train the parameter estimation model more accurately in the future.

[0047] S130, generating a corresponding fitting loss function based on each loss function and the corresponding weight, and using the fitting loss function to reversely correct the neural network model to obtain a trained parameter estimation model.

[0048] Specifically, after calculating the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervision information, and adjusting the weight corresponding to each loss function, the weight corresponding to each loss function is used to integrate the loss functions under multiple two-dimensional supervision information to generate a corresponding fitting loss function, and the fitting loss function is used as the loss function of the entire training process of the neural network model, and then the fitting loss function is used to perform back propagation according to the entire training process, so as to correct the network parameters in the neural network model, and then the corrected neural network model is used according to the above steps to continue to estimate the reconstruction parameters of the next training sample when reconstructing the three-dimensional face, so as to continue to execute the training process, and continuously The neural network model is reversely corrected, and the neural network model that has finally completed the training is used as the trained parameter estimation model. At this time, the fitting loss function in this embodiment only needs to use the two-dimensional face information under multiple supervisory information, and there is no need to refer to additional three-dimensional face information before reconstructing the three-dimensional face. The neural network model can be continuously reversely corrected to obtain the trained parameter estimation model, thereby optimizing the training method of the parameter estimation model used to estimate the corresponding reconstruction parameters in the three-dimensional face reconstruction. The parameter estimation model is trained by the loss function under multiple two-dimensional supervisory information, so that the reference information in the training process is more comprehensive, thereby improving the estimation accuracy of the reconstruction parameters used in the three-dimensional face reconstruction.

[0049] For example, if the loss functions under multiple two-dimensional supervision information are: image pixel loss function L photometric , key point loss function L landmark , identity feature loss function L perception , albedo penalty function L box And the regularization term corresponding to the target reconstruction parameters (adjustment parameter α of the principal component vector representing the change of face shape, adjustment parameter δ of the principal component vector representing the change of face expression, adjustment parameter β of the principal component vector representing the change of face albedo and illumination parameter γ) in the reconstruction parameters specified for 3D face reconstruction, then the fitting loss function in this embodiment can be:

[0050] L=λ p L photometric +λ l L landmark+λ id L perception +λ b L box +λ α ||α||2+λ β ||β||2+λ δ ||δ||2+λ γ ||γ||2; where λ p is the weight corresponding to the image pixel loss function, λ l is the weight corresponding to the key point loss function, λ id is the weight corresponding to the identity feature loss function, λ b is the weight corresponding to the albedo penalty function, λ α , β , δ and λ γ The weights corresponding to the regularization terms corresponding to the target reconstruction parameters in the reconstruction parameters specified for 3D face reconstruction.

[0051] Furthermore, when the trained parameter estimation model is obtained, the parameter estimation model can accurately estimate the reconstruction parameters of any two-dimensional face image to be reconstructed when performing three-dimensional face reconstruction. Therefore, after the present embodiment uses the fitting loss function to reversely correct the neural network model to obtain the trained parameter estimation model, it can also include: inputting the two-dimensional face image to be reconstructed into the parameter estimation model, estimating the reconstruction parameters specified for the three-dimensional face reconstruction, and inputting the reconstruction parameters into a pre-constructed three-dimensional deformation model to reconstruct the three-dimensional face corresponding to the two-dimensional face image.

[0052] Specifically, since the shooting sizes of the two-dimensional face images to be reconstructed are different, in order to ensure the accuracy of three-dimensional face reconstruction, Figure 1CAs shown, the present embodiment first performs face detection on the two-dimensional face image to be reconstructed, thereby cropping out the two-dimensional face image including the face area, and performing corresponding scaling operation on the cropped two-dimensional face image to a corresponding size (such as 224*224 pixels) to meet the image requirements of the parameter estimation model for estimating reconstruction parameters during three-dimensional face reconstruction, and then the preprocessed two-dimensional face image is input into the parameter estimation model, and the parameter estimation model accurately estimates the various reconstruction parameters required for the two-dimensional face image to be used for three-dimensional face reconstruction, and then the reconstruction parameters are input into the pre-constructed three-dimensional deformation model, and the three-dimensional deformation model uses the reconstruction parameters to perform corresponding deformation and expression changes on the standard face defined therein, and adjusts the corresponding three-dimensional detail display information, so as to reconstruct the three-dimensional face corresponding to the two-dimensional face image, so that the deformation process of the face image to be reconstructed in the three-dimensional deformation model is more accurate, thereby ensuring the accuracy of three-dimensional face reconstruction, without deploying additional information equipment during the three-dimensional face reconstruction process, and reducing the operational complexity of three-dimensional face reconstruction.

[0053] The technical solution provided in this embodiment pre-constructs a corresponding neural network model for each reconstruction parameter required for three-dimensional face reconstruction. Each training sample in the face image training set is input into the neural network model to estimate the reconstruction parameters required for the training sample in three-dimensional face reconstruction. The reconstruction parameters are input into the pre-constructed three-dimensional deformation model to reconstruct the three-dimensional face corresponding to the training sample. At this time, a fitting loss function for the neural network model training can be generated by calculating the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervisory information and adjusting the weight corresponding to each loss function. The fitting loss function only needs to use the two-dimensional face information under multiple supervisory information, and does not need to reconstruct the three-dimensional face. By referring to additional three-dimensional face information before, the neural network model can be continuously reversely corrected to obtain a trained parameter estimation model, thereby optimizing the training method of the parameter estimation model used to estimate corresponding reconstruction parameters in three-dimensional face reconstruction, and training the parameter estimation model through a loss function under multiple two-dimensional supervision information, so that the reference information in the training process is more comprehensive, and the estimation accuracy of the reconstruction parameters used in three-dimensional face reconstruction is improved; at the same time, the trained parameter estimation model is subsequently used to estimate the reconstruction parameters in three-dimensional face reconstruction, so that the deformation process of the face image to be reconstructed in the three-dimensional deformation model is more accurate, thereby ensuring the accuracy of three-dimensional face reconstruction, without deploying additional information equipment in the three-dimensional face reconstruction process, and reducing the operational complexity of three-dimensional face reconstruction.

[0054] Embodiment 2

[0055] Figure 2A A flowchart of a method for training a parameter estimation model provided in Embodiment 2 of the present invention, Figure 2B This is a schematic diagram of the structure of a three-dimensional deformable model used for three-dimensional face reconstruction in the method provided in the second embodiment of the present invention. This embodiment is optimized on the basis of the above embodiment.

[0056] It should be noted that the three-dimensional deformation model in this embodiment is composed of a dual principal component analysis (PCA) model and a single PCA model. Figure 2B As shown, the double PCA model is mainly used to model the changes in facial appearance and expression during the three-dimensional face reconstruction process, while the single PCA model is mainly used to model the changes in facial albedo during the three-dimensional face reconstruction process.

[0057] Among them, the dual PCA model in this embodiment can define a three-dimensional average face, a first principal component basis representing changes in facial appearance, and a second principal component basis representing changes in facial expression, which can be expressed as: is the three-dimensional average face defined in the double PCA model, I base is the first principal component basis representing the change of face shape, E base is the second principal component basis representing the change of facial expression, α is the facial identity parameter that needs to be estimated when the facial image to be reconstructed is subjected to three-dimensional face reconstruction and is used to indicate the change of the first principal component basis, δ is the facial expression parameter that needs to be estimated when the facial image to be reconstructed is subjected to three-dimensional face reconstruction and is used to indicate the change of the second principal component basis.

[0058] At the same time, the single PCA model in this embodiment may define the average albedo of the face and the third principal component basis representing the change of the albedo of the face, which may be expressed as: is the average albedo of the face defined in the single PCA model, T base is the third principal component basis representing the change of face albedo in the single PCA model, β is the albedo parameter indicating the change of the third principal component basis that needs to be estimated when performing 3D face reconstruction on the face image to be reconstructed. At this time, the albedo parameter contains the RGB color information of each vertex in the face image to achieve color reconstruction during 3D face reconstruction.

[0059] In addition, in order to ensure the three-dimensional detail features when reconstructing the three-dimensional face, the three-dimensional deformation model of this embodiment may also define an illumination parameter γ representing the illumination change of the face, a position parameter t representing the translation of the face, and a rotation parameter p representing the head posture of the face. Among them, this embodiment may use spherical harmonic illumination to approximate the illumination in the three-dimensional scene to estimate the corresponding illumination parameter γ.

[0060] Therefore, in order to accurately reconstruct a three-dimensional face, the reconstruction parameters specified for three-dimensional face reconstruction in this embodiment may be (α, δ, β, γ, t, p).

[0061] Optionally, in order to ensure the accuracy of three-dimensional face reconstruction, the number of first principal component bases representing changes in facial appearance in the dual PCA model of this embodiment can be set to 80, the number of second principal component bases representing changes in facial expressions can be set to 30, the number of third principal component bases representing changes in facial albedo in the single PCA model can be set to 79, the number of illumination parameters can be set to 27, including 9 color parameters for each of the three color channels of R, G, and B, and the number of position parameters and rotation parameters can be set to 3 respectively. The number of principal component bases and illumination parameters provided in this embodiment is only an example, and can be set specifically according to the corresponding reconstruction requirements. This embodiment is not limited to this.

[0062] Specifically, Figure 2A As shown, this embodiment mainly explains in detail the specific reconstruction process of performing 3D face reconstruction using a 3D deformable model composed of a dual PCA model and a single PCA model.

[0063] Optional, such as Figure 2A As shown, this embodiment may include the following steps:

[0064] S210, collecting three-dimensional face scanning data with uniform illumination under a multi-dimensional data source, and performing deformation analysis, expression change analysis and albedo analysis on the three-dimensional face scanning data, to obtain the corresponding three-dimensional average face, average face albedo, first principal component basis, second principal component basis and third principal component basis.

[0065] Optionally, in order to accurately determine the principal components that may affect the realism of three-dimensional face reconstruction, this embodiment first uses 3D scanning technology to scan a large amount of facial information with uniform lighting under multi-dimensional data sources such as different races, ages, genders, skin colors and expressions, thereby collecting three-dimensional face scanning data with uniform lighting under multi-dimensional data sources, and then performing corresponding deformation analysis, expression change analysis and albedo analysis on a large amount of three-dimensional face scanning data to obtain the corresponding three-dimensional average face and average face albedo, as well as the first principal component representing the change in facial appearance, the second principal component representing the change in facial expression and the third principal component representing the change in facial albedo, so as to subsequently construct the corresponding three-dimensional deformation model to accurately reconstruct the face image in three dimensions.

[0066] S220, input each training sample in the face image training set into a pre-built neural network model to estimate the reconstruction parameters specified for three-dimensional face reconstruction.

[0067] S230, inputting the reconstruction parameters respectively matched with the first principal component basis and the second principal component basis into the dual PCA model, deforming the three-dimensional average face, and obtaining a corresponding three-dimensional deformed face.

[0068] Optionally, after estimating the various reconstruction parameters for three-dimensional face reconstruction of the training samples, the reconstruction parameters are input into the three-dimensional deformation model to deform the three-dimensional average face. At this time, the three-dimensional deformation model is composed of a double PCA model and a single PCA model. Different PCA models have different reconstruction functions. The double PCA model is mainly used to model the changes in facial appearance and expression during the three-dimensional face reconstruction process, and the single PCA model is mainly used to model the changes in facial albedo during the three-dimensional face reconstruction process. Therefore, the samples are trained in turn through the double PCA model and the single PCA model for three-dimensional face reconstruction.

[0069] Specifically, first, reconstruction parameters that match the first principal component basis and the second principal component basis defined in the double PCA model are screened out from the estimated reconstruction parameters, and then the screened reconstruction parameters are input into the double PCA model. The model representation function of the double PCA model is used to perform corresponding appearance changes and expression changes on the three-dimensional average face defined therein, so as to obtain the corresponding three-dimensional deformed face. Subsequently, the single PCA model is used to continue to change the albedo of the three-dimensional deformed face to reconstruct the corresponding three-dimensional face.

[0070] S240, inputting the three-dimensional deformable face and the reconstruction parameters matched with the third principal component basis into the single PCA model, performing albedo correction on the three-dimensional deformable face based on the average albedo of the face, and obtaining a reconstructed three-dimensional face.

[0071] Optionally, after obtaining the corresponding three-dimensional deformable face through the double PCA model, the reconstruction parameters that match the third principal component basis defined in the single PCA model are screened out again from the estimated reconstruction parameters, and then the three-dimensional deformable face and the reconstruction parameters that match the third principal component basis are input into the single PCA model. The model representation function of the above single PCA model is used to perform albedo correction on the three-dimensional deformable face based on the standard albedo of the face, so as to obtain the reconstructed three-dimensional face.

[0072] In addition, in order to ensure the three-dimensional detail features of the three-dimensional face, the three-dimensional face may be further optimized using the illumination parameters, position parameters, and rotation parameters defined in the three-dimensional deformation model.

[0073] S250, calculating the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervision information, and adjusting the weight corresponding to each loss function.

[0074] S260, generating a corresponding fitting loss function based on each loss function and the corresponding weight, and using the fitting loss function to reversely correct the neural network model to obtain a trained parameter estimation model.

[0075] The technical solution provided in this embodiment uses a double PCA model to construct a three-dimensional deformation model during the training process of the parameter estimation model to ensure the accuracy of three-dimensional face reconstruction, so that the specific losses between the reconstructed three-dimensional face and the training samples in multiple two-dimensional supervision information are reflected as much as possible in the estimation errors of the reconstruction parameters. At this time, the parameter estimation model is trained through the loss function under multiple two-dimensional supervision information, so that the reference information in the training process is more comprehensive and accurate, thereby improving the estimation accuracy of the reconstruction parameters used in three-dimensional face reconstruction.

[0076] Embodiment 3

[0077] Figure 3A A flowchart of a method for training a parameter estimation model provided in Embodiment 3 of the present invention, Figure 3B The schematic diagram of the principle of the training process of the parameter estimation model provided in the third embodiment of the present invention is shown in FIG. This embodiment is optimized on the basis of the above embodiment. Specifically, Figure 3A As shown, the loss functions under multiple two-dimensional supervision information in this embodiment may include: image pixel loss function, key point loss function, identity feature loss function, albedo penalty function and regularization terms corresponding to target reconstruction parameters in the reconstruction parameters specified for three-dimensional face reconstruction. At this time, the specific setting method of the loss function under multiple two-dimensional supervision information that the parameter estimation model refers to during the training process is mainly explained in detail.

[0078] Optional, such as Figure 3A As shown, this embodiment may include the following steps:

[0079] S301, input each training sample in the face image training set into a pre-built neural network model, estimate the reconstruction parameters specified for three-dimensional face reconstruction, and input the reconstruction parameters into a pre-built three-dimensional deformation model to reconstruct the three-dimensional face corresponding to the training sample.

[0080] S302, segmenting the corresponding skin mask from the training sample.

[0081] Optionally, the mask is a binary image composed of pixel values ​​0 and 1. When setting the image pixel loss function in this embodiment, in order to ensure the accuracy of image pixel loss during the training process, by applying a skin mask in the training sample, the pixel values ​​of the facial skin area in the training sample can be set to 1, and the pixel values ​​of the non-facial skin area can be set to 0, so that the corresponding facial skin area can be accurately segmented from the training sample using a skin segmentation algorithm, so as to avoid interference of pixel features in the non-facial skin area with three-dimensional face reconstruction.

[0082] S303, based on the skin mask, calculate the pixel error between the three-dimensional face and the same pixel point in the face skin area in the training sample to obtain the corresponding image pixel loss function.

[0083] Optionally, after segmenting the corresponding skin mask from the training sample, the pixel points at the same pixel position can be found from the reconstructed three-dimensional face and the training sample, and then based on the segmented skin mask, it can be accurately determined whether each identical pixel point is in the facial skin area. By calculating the pixel error of each pixel point in the facial skin area between the three-dimensional face and the training sample, the overall pixel error of the three-dimensional face and the training sample in the facial skin area is analyzed, thereby obtaining the corresponding image pixel loss function. This image pixel loss function only compares the pixel errors in the facial skin area before and after reconstruction, while shielding the pixel influence of the non-facial skin area, so that the facial identity features and albedo information in the estimated reconstruction parameters are more accurate.

[0084] Exemplarily, the image pixel loss function in this embodiment may be:

[0085] in, is the pixel value of the pixel point (j, k) in the i-th training sample, is the pixel value of the pixel point (j, k) in the 3D face reconstructed from the i-th training sample, M jk is the pixel value of the pixel point (j, k) in the skin mask. At this time, M is the pixel value of the pixel point in the skin mask in the face skin area. jk is 1 if the value is true, otherwise it is 0.

[0086] S304, extracting key feature points at preset positions from the training samples, and determining the visibility of each key feature point.

[0087] Optionally, in order to ensure the one-to-one matching of key feature points in the reconstructed three-dimensional face and the training samples, when setting the key point loss function in the training process, this embodiment can use the Landmark algorithm to extract key feature points at preset positions in each facial area from the training samples, such as 17 key feature points on the face contour, 5 key feature points on the left and right eyebrows, 6 key feature points on the left and right eyes, 9 key feature points on the nose, and 20 key feature points on the mouth; illustratively, a total of 68 key feature points can be used in this embodiment, such as Figure 3D At this time, due to the different head postures of the human faces in different training samples, the key feature points at some preset positions may not be visible. Therefore, after extracting the key feature points at each preset position from the training samples, it is necessary to further determine whether each key feature point is visible. Subsequently, only the reconstruction of the visible key feature points needs to be judged for loss, so as to achieve the reconstruction of training samples under various head postures.

[0088] S305, calculating the position reconstruction error of each visible key feature point between the three-dimensional face and the training sample, and obtaining a corresponding key point loss function.

[0089] Optionally, after determining each visible key feature point, by analyzing whether the pixel position of each visible key feature point on the reconstructed 3D face and the training sample is consistent, the position reconstruction error of each visible key feature point before and after reconstruction is calculated, and then the corresponding key point loss function is obtained. At this time, for faces with large head rotation angles in the training sample, only the visible half-side key feature points need to be selected to calculate the corresponding key point reconstruction loss, and the invisible key feature points do not participate in the loss calculation of the key point reconstruction.

[0090] In addition, since the head posture of the reconstructed three-dimensional face and the training sample may be different during the training process, in order to ensure the matching of the same pixel point in the three-dimensional face and the training sample, the present embodiment will adopt a method of dynamically selecting key feature points in the reconstructed three-dimensional face to determine the vertices that match each visible key feature point in the training sample; illustratively, according to the head posture in the three-dimensional face, the three-dimensional mesh vertex that matches each visible key feature point is dynamically selected from the three-dimensional face, and the position information of the three-dimensional mesh vertex in the three-dimensional face is used as the reconstructed position of the visible key feature point to calculate the position reconstruction error of the visible key feature point between the three-dimensional face and the training sample.

[0091] Specifically, the head posture such as the translation position and rotation angle of the head in the reconstructed 3D face is first analyzed, and then according to the head posture in the 3D face and the face part represented by each visible key feature point, the 3D mesh vertex matching each visible key feature point in the 3D face is dynamically selected, such as Figure 3C As shown, the position information of the three-dimensional mesh vertex in the three-dimensional face is used as the reconstructed position of the visible key feature point, and the reconstructed position of each visible key feature point in the three-dimensional face is calculated in the above manner, so as to subsequently calculate the position reconstruction error of each visible key feature point between the three-dimensional face and the training sample.

[0092] Exemplarily, the key point loss function in this embodiment may be:

[0093] in, is the position coordinate of the jth key feature point in the i-th training sample, is the position coordinate of the jth key feature point in the 3D face reconstructed from the i-th training sample, v ij Indicates the visibility of the jth key feature point. The visible key feature point is set to 1, and the invisible key feature point is set to 0. j is the weight of the jth key feature point in the loss function. Different weights are used for different facial parts (such as eyes, mouth, contour points, etc.). These weights are adjusted by w j to control the size.

[0094] S306, input the training sample and the reconstructed three-dimensional face into a pre-built face recognition model respectively, to obtain a first identity feature corresponding to the training sample and a second identity feature corresponding to the three-dimensional face.

[0095] Optionally, for the identity feature loss function, the essence is to analyze whether the identity features before and after reconstruction have changed. Therefore, in this embodiment, for identity feature recognition, a corresponding face recognition model can be pre-constructed to extract the identity features before and after reconstruction through the face recognition model; specifically, Figure 3B As shown, the training sample and the reconstructed three-dimensional face are first input into the face recognition model respectively, and the face recognition model is used to analyze the facial features in the training sample and the three-dimensional face respectively, so as to determine the first identity feature corresponding to the training sample and the second identity feature corresponding to the three-dimensional face respectively, so as to subsequently judge whether there is an error in the identity features before and after reconstruction.

[0096] In addition, since the facial head posture in the training sample may have a certain rotation angle, making some facial areas in the training sample invisible, the identity features extracted from a single angle may have certain errors. Therefore, in order to ensure the accuracy of the first identity feature, the first identity feature corresponding to the training sample in this embodiment can also be calculated using the following steps: collect facial images of the same face as the training sample taken from multiple angles, and input each facial image into a pre-constructed three-dimensional deformation model to extract the identity sub-features corresponding to each facial image; integrate the identity sub-features to obtain the first identity feature corresponding to the training sample.

[0097] Specifically, by analyzing the faces in the training samples, multiple facial images with the same face as the training sample are additionally taken from multiple angles, and then the facial images taken at each multiple angle are respectively input into the three-dimensional deformation model. The identity features of each facial image can be extracted through the first principal component basis representing the changes in the facial appearance in the three-dimensional deformation model, so as to obtain the identity sub-features corresponding to each facial image. At this time, the features of each identity sub-feature are integrated to obtain the first identity feature corresponding to the training sample, thereby ensuring the comprehensiveness and accuracy of the first identity feature.

[0098] S307, calculating a corresponding identity feature loss function according to the similarity between the first identity feature and the second identity feature.

[0099] Optionally, after obtaining the first identity feature corresponding to the training sample and the second identity feature corresponding to the three-dimensional face, in order to analyze whether there are errors in the identity features before and after reconstruction, it is necessary to first determine the similarity between the first identity feature and the second identity feature, and then calculate the corresponding identity feature loss function based on the similarity.

[0100] Exemplarily, the identity feature loss function in this embodiment may be:

[0101] in, is the first identity feature corresponding to the i-th training sample, is the second identity feature corresponding to the three-dimensional face reconstructed for the i-th training sample.

[0102] S308, calculating the albedo of each vertex in the three-dimensional face.

[0103] Optionally, this embodiment can calculate the albedo of each pixel by detecting information such as the color and reflected light intensity of each pixel in the training sample, and then set the albedo of each vertex on the reconstructed three-dimensional face according to the position matching of each vertex in the reconstructed three-dimensional face with each pixel in the training sample to ensure the consistency of the face albedo before and after reconstruction.

[0104] S309, calculating a corresponding albedo penalty function based on the albedo of each vertex in the three-dimensional face and a preset albedo range.

[0105] Optionally, in order to ensure that the albedo of each vertex in the reconstructed three-dimensional face is not too dark or too bright, the present embodiment will make corresponding adjustments to the albedo on each vertex in the three-dimensional face. At this time, the present embodiment will pre-set a reasonable preset albedo interval. The preset albedo interval in the present embodiment may be [0.05, 0.95], so that the albedo of each vertex in the reconstructed three-dimensional face can all be within the preset albedo interval. Therefore, by analyzing whether the albedo of each vertex in the three-dimensional face is within the preset albedo interval, the corresponding albedo penalty function can be calculated to continuously optimize the albedo on the reconstructed three-dimensional face during the training process.

[0106] Exemplarily, the albedo penalty function in this embodiment may be:

[0107] Among them, T i,j is the albedo of the jth pixel in the i-th training sample, is the albedo of the j-th pixel in the 3D face reconstructed from the i-th training sample.

[0108] It should be noted that S302 and S303 in this embodiment are calculation steps of the image pixel loss function, S304 and S305 are calculation steps of the key point loss function, S306 and S307 are calculation steps of the identity feature loss function, and S308 and S309 are calculation steps of the albedo penalty function. At this time, the calculation steps corresponding to the image pixel loss function, the key point loss function, the identity feature loss function and the albedo penalty function in this embodiment can be executed simultaneously or sequentially, without limitation.

[0109] S310, adjusting the weight corresponding to each loss function.

[0110] S311, generating a corresponding fitting loss function based on each loss function and the corresponding weight, and using the fitting loss function to reversely correct the neural network model to obtain a trained parameter estimation model.

[0111] The technical solution provided in this embodiment optimizes individual loss functions in the loss functions under multiple two-dimensional supervision information by using skin masks and dynamically selecting key feature points in training samples, thereby ensuring the training accuracy of the parameter estimation model. The parameter estimation model is trained by using loss functions under multiple two-dimensional supervision information, so that the reference information in the training process is more comprehensive, thereby improving the estimation accuracy of the reconstruction parameters used in three-dimensional face reconstruction.

[0112] Embodiment 4

[0113] Figure 4 A schematic diagram of a training device for a parameter estimation model provided in Embodiment 4 of the present invention, specifically, Figure 4 As shown, the device may include:

[0114] The 3D reconstruction module 410 is used to input each training sample in the face image training set into a pre-built neural network model, estimate the reconstruction parameters specified for 3D face reconstruction, and input the reconstruction parameters into a pre-built 3D deformable model to reconstruct the 3D face corresponding to the training sample;

[0115] A loss function calculation module 420, used to calculate the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervision information, and adjust the weight corresponding to each loss function;

[0116] The model training module 430 is used to generate a corresponding fitting loss function based on each loss function and the corresponding weight, and use the fitting loss function to reversely correct the neural network model to obtain a trained parameter estimation model.

[0117] The technical solution provided in this embodiment pre-constructs a corresponding neural network model for each reconstruction parameter required for three-dimensional face reconstruction. Each training sample in the face image training set is input into the neural network model to estimate the reconstruction parameters required for the training sample in three-dimensional face reconstruction. The reconstruction parameters are input into the pre-constructed three-dimensional deformation model to reconstruct the three-dimensional face corresponding to the training sample. At this time, a fitting loss function for the neural network model training can be generated by calculating the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervisory information and adjusting the weight corresponding to each loss function. The fitting loss function only needs to use the two-dimensional face information under multiple supervisory information, and does not need to reconstruct the three-dimensional face. By referring to additional three-dimensional face information before, the neural network model can be continuously reversely corrected to obtain a trained parameter estimation model, thereby optimizing the training method of the parameter estimation model used to estimate corresponding reconstruction parameters in three-dimensional face reconstruction, and training the parameter estimation model through a loss function under multiple two-dimensional supervision information, so that the reference information in the training process is more comprehensive, and the estimation accuracy of the reconstruction parameters used in three-dimensional face reconstruction is improved; at the same time, the trained parameter estimation model is subsequently used to estimate the reconstruction parameters in three-dimensional face reconstruction, so that the deformation process of the face image to be reconstructed in the three-dimensional deformation model is more accurate, thereby ensuring the accuracy of three-dimensional face reconstruction, without deploying additional information equipment in the three-dimensional face reconstruction process, and reducing the operational complexity of three-dimensional face reconstruction.

[0118] The training device for the parameter estimation model provided in this embodiment can be applied to the training method for the parameter estimation model provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0119] Embodiment 5

[0120] Figure 5 A schematic diagram of the structure of a computer device provided in Embodiment 5 of the present invention is shown in FIG. Figure 5 As shown, the device includes a processor 50, a storage device 51 and a communication device 52; the number of processors 50 in the device can be one or more. Figure 5 A processor 50 is taken as an example; the processor 50, the storage device 51 and the communication device 52 in the device can be connected by a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0121] A computer device provided in this embodiment can be used to execute the training method of the parameter estimation model provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0122] Embodiment 6

[0123] Embodiment 6 of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the training method of the parameter estimation model in any of the above embodiments can be implemented. The method may specifically include:

[0124] Input each training sample in the face image training set into a pre-built neural network model, estimate the reconstruction parameters specified for 3D face reconstruction, and input the reconstruction parameters into a pre-built 3D deformable model to reconstruct the 3D face corresponding to the training sample;

[0125] Calculating a loss function between the three-dimensional face and the training sample under multiple two-dimensional supervisory information, and adjusting a weight corresponding to each loss function;

[0126] A corresponding fitting loss function is generated based on each loss function and the corresponding weight, and the neural network model is reversely corrected using the fitting loss function to obtain a trained parameter estimation model.

[0127] Of course, the storage medium containing computer executable instructions provided in an embodiment of the present invention is not limited to the method operations described above, and can also execute related operations in the training method of the parameter estimation model provided in any embodiment of the present invention.

[0128] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0129] It is worth noting that in the embodiment of the training device for the above-mentioned parameter estimation model, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0130] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A training method for a parameter estimation model, characterized in that: include: Input each training sample in the face image training set into a pre-built neural network model, estimate the reconstruction parameters specified for 3D face reconstruction, and input the reconstruction parameters into a pre-built 3D deformable model to reconstruct the 3D face corresponding to the training sample; Calculating a loss function between the three-dimensional face and the training sample under multiple two-dimensional supervisory information, and adjusting a weight corresponding to each loss function; Generate a corresponding fitting loss function based on each loss function and the corresponding weight, and use the fitting loss function to reversely correct the neural network model to obtain a trained parameter estimation model; The three-dimensional deformation model is composed of a dual principal component analysis (PCA) model and a single PCA model, wherein the dual principal component analysis (PCA) model defines a three-dimensional average face, a first principal component basis representing changes in face shape, and a second principal component basis representing changes in facial expression; The single PCA model defines an average face albedo and a third principal component basis representing the variation of the face albedo.

2. The method according to claim 1, characterized in that The loss functions under the multiple two-dimensional supervision information include: image pixel loss function, key point loss function, identity feature loss function, albedo penalty function and regularization items corresponding to target reconstruction parameters in the reconstruction parameters specified for the three-dimensional face reconstruction.

3. The method according to claim 2, characterized in that If the image pixel loss function between the three-dimensional human face and the training sample is calculated, the method further comprises: Segmenting a corresponding skin mask from the training sample; Accordingly, the calculating of the image pixel loss function between the three-dimensional face and the training sample includes: Based on the skin mask, a pixel error between the three-dimensional human face and a same pixel point in the training sample within the human face skin region is calculated to obtain a corresponding image pixel loss function.

4. The method according to claim 2, characterized in that: If a key point loss function between the three-dimensional human face and the training sample is calculated, the method further comprises: Extracting key feature points at preset positions from the training samples, and determining the visibility of each of the key feature points; Accordingly, the calculating of the key point loss function between the three-dimensional face and the training sample includes: The position reconstruction error of each visible key feature point between the three-dimensional human face and the training sample is calculated to obtain a corresponding key point loss function.

5. The method according to claim 4, characterized in that Before calculating the position reconstruction error between the three-dimensional human face and the training sample for each visible key feature point, the method further includes: According to the head posture in the three-dimensional human face, a three-dimensional mesh vertex matching each of the visible key feature points is dynamically selected from the three-dimensional human face, and the position information of the three-dimensional mesh vertex in the three-dimensional human face is used as the reconstructed position of the visible key feature point to calculate the position reconstruction error of the visible key feature point between the three-dimensional human face and the training sample.

6. The method according to claim 2, characterized in that If the identity feature loss function between the three-dimensional human face and the training sample is calculated, the method further includes: Inputting the training sample and the reconstructed three-dimensional face into a pre-built face recognition model respectively to obtain a first identity feature corresponding to the training sample and a second identity feature corresponding to the three-dimensional face; Accordingly, the calculating of the identity feature loss function between the three-dimensional human face and the training sample includes: According to the similarity between the first identity feature and the second identity feature, a corresponding identity feature loss function is calculated.

7. The method according to claim 6, characterized in that The first identity feature corresponding to the training sample is also calculated using the following steps: Collecting facial images of the same face as the training sample taken from multiple angles, and inputting each facial image into a pre-built three-dimensional deformation model, respectively, to extract identity sub-features corresponding to each facial image; The identity sub-features are integrated to obtain a first identity feature corresponding to the training sample.

8. The method according to claim 2, characterized in that: If the albedo penalty function of the three-dimensional face is calculated, the method further comprises: Calculating the albedo of each vertex in the three-dimensional face; Accordingly, the albedo penalty function of the three-dimensional face is calculated, including: Based on the albedo of each vertex in the three-dimensional face and a preset albedo interval, a corresponding albedo penalty function is calculated.

9. The method according to claim 1, characterized in that: Inputting the reconstruction parameters into a pre-built three-dimensional deformable model to reconstruct a three-dimensional face corresponding to the training sample includes: Inputting the reconstruction parameters respectively matched with the first principal component basis and the second principal component basis into the dual principal component analysis (PCA) model, deforming the three-dimensional average human face, and obtaining a corresponding three-dimensional deformed human face; The three-dimensional deformable face and the reconstruction parameters matched with the third principal component basis are input into the single PCA model, and the albedo of the three-dimensional deformable face is corrected based on the average albedo of the face to obtain a reconstructed three-dimensional face.

10. The method according to claim 9, characterized in that Before inputting the reconstruction parameters into the pre-built three-dimensional deformable model to reconstruct the three-dimensional face corresponding to the training sample, the method further includes: Three-dimensional face scanning data with uniform illumination under a multi-dimensional data source is collected, and deformation analysis, expression change analysis and albedo analysis are performed on the three-dimensional face scanning data to obtain the corresponding three-dimensional average face, average face albedo, first principal component basis, second principal component basis and third principal component basis.

11. The method according to claim 9, characterized in that The three-dimensional deformation model also defines illumination parameters representing changes in illumination of the face, position parameters representing translation of the face, and rotation parameters representing the head posture of the face.

12. The method according to any one of claims 1 to 11, characterized in that: Before calculating the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervision information, the method further includes: The three-dimensional human face is rendered using a differentiable renderer, so as to train the parameter estimation model using the rendered three-dimensional human face.

13. The method according to any one of claims 1 to 11, characterized in that: After the neural network model is reversely corrected by using the fitting loss function to obtain a trained parameter estimation model, the method further includes: The two-dimensional face image to be reconstructed is input into the parameter estimation model to estimate the reconstruction parameters specified for three-dimensional face reconstruction, and the reconstruction parameters are input into a pre-built three-dimensional deformation model to reconstruct the three-dimensional face corresponding to the two-dimensional face image.

14. A training device for a parameter estimation model, characterized in that: include: A 3D reconstruction module, for inputting each training sample in the face image training set into a pre-built neural network model, estimating reconstruction parameters specified for 3D face reconstruction, and inputting the reconstruction parameters into a pre-built 3D deformable model to reconstruct a 3D face corresponding to the training sample; A loss function calculation module, used to calculate the loss function between the three-dimensional face and the training sample under multiple two-dimensional supervision information, and adjust the weight corresponding to each loss function; A model training module, used to generate a corresponding fitting loss function based on each loss function and the corresponding weight, and use the fitting loss function to reversely correct the neural network model to obtain a trained parameter estimation model; The three-dimensional deformation model is composed of a dual principal component analysis (PCA) model and a single PCA model, wherein the dual principal component analysis (PCA) model defines a three-dimensional average face, a first principal component basis representing changes in face shape, and a second principal component basis representing changes in facial expression; The single PCA model defines an average face albedo and a third principal component basis representing the variation of the face albedo.

15. A computer device, characterized in that: The computer device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the training method of the parameter estimation model as described in any one of claims 1-13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the training method of the parameter estimation model as described in any one of claims 1 to 13 is implemented.

Images