Face-focusing digital human body reconstruction method

Abstract

The invention discloses a face-focusing digital human body reconstruction method, which relates to the technical field of digital human body reconstruction, and comprises the following steps: sequentially carrying out super-division processing and semantic segmentation on a face region obtained by face recognition to obtain a current face RGB image frame; obtaining a new parameter of a Gaussian primitive of a head grid corresponding to the current RGB image frame based on a multi-layer perceptron according to the initialization parameter and the expression parameter of the current RGB image frame; optimizing parameters of Gaussian primitives of a head grid corresponding to the current RGB image frame through a training strategy of minimizing loss, iteratively processing other RGB image frames, and combining all optimized Gaussian primitives to obtain a reconstructed digital head; and rendering according to the reconstructed digital head and the reconstructed digital human body to obtain a human body image focusing the face. The method not only pays attention to the reconstruction of the whole human body, but also strengthens the high-precision reconstruction of the face, and can present a smooth and vivid human body animation effect in subsequent rendering.

Description

Technical Field

[0001] The present invention relates to the technical field of digital human reconstruction, and in particular to a digital human reconstruction method focusing on face. Background Art

[0002] Traditional digital human reconstruction methods usually focus on body reconstruction and ignore facial details, which may result in a blurry or unnatural face. On the contrary, they only focus on face reconstruction and ignore body reconstruction. In order to achieve more natural human animation and more realistic facial rendering effects, a high-precision digital human reconstruction model needs to be built to make the rendering effect of digital humans in virtual environments more realistic and delicate.

[0003] Traditional digital human reconstruction methods mainly include the following: Existing methods for full-body reconstruction and fine facial reconstruction mainly include the following aspects: The method based on Nerf for digital human reconstruction has the problem of low quality of both the whole body and the face; the method of using learning dynamic neural network to reconstruct digital humans has the problem of being unable to achieve accurate facial animation in fast-moving frames and subtle frames, and there are large artifacts for unseen postures; the method based on Gaussian for digital human reconstruction still has obvious shortcomings in the reconstruction effect of the face area; some methods use monocular video to reconstruct the head alone. For virtual reality, it is obviously unreasonable for the only interactive object to be the head; the SMPL-X model requires multi-view training and has high requirements for hardware equipment; ExAvatar and GaussianAvatar have incorrect expressions and artifacts.

[0004] Therefore, a digital human body reconstruction method focusing on the face is developed to solve the above problems. Summary of the invention

[0005] The present invention provides a digital human reconstruction method focusing on the face, so as to solve the problem that the digital human face reconstructed by the existing digital human reconstruction method is relatively blurry or unnatural.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0007] The present invention provides a digital human body reconstruction method focusing on the face, comprising:

[0008] Acquire first information, wherein the first information includes a sequence of RGB image frames of a dynamic human body;

[0009] Acquire second information and third information according to the first information, wherein the second information includes shape parameters, expression parameters, and head camera pose of the FLAME model corresponding to each predicted RGB image frame, and the third information includes shape parameters, posture parameters, and human camera pose of the digital human body reconstruction model corresponding to each predicted RGB image frame;

[0010] Constructing a head grid corresponding to each RGB image frame based on a preset FLAME model according to the second information;

[0011] Uniformly sampling the UV texture map of the head mesh corresponding to each RGB image frame to obtain a corresponding sampling result;

[0012] Performing Gaussian field initialization on the corresponding head grid according to the corresponding sampling result to obtain initialization parameters of the Gaussian basis element of the corresponding head grid;

[0013] Enter the current RGB image frame processing steps:

[0014] Perform face recognition on the current RGB image frame, perform super-resolution processing and semantic segmentation on the face area recognized by the face, and obtain the current face RGB image frame;

[0015] According to the initialization parameters and the expression parameters of the current RGB image frame, new parameters of the Gaussian basis element of the head grid corresponding to the current RGB image frame are obtained based on a multi-layer perceptron;

[0016] A head rendering image is obtained according to the new parameters of the Gaussian primitives of the head mesh corresponding to the current RGB image frame and the head camera pose rendering, the loss between the head rendering image and the current face RGB image frame is calculated, and the parameters of the Gaussian primitives of the head mesh corresponding to the current RGB image frame are optimized by a training strategy that minimizes the loss, and then the current RGB image frame processing step is ended;

[0017] Iteratively process other RGB image frames according to the current RGB image frame processing step until the parameters of the Gaussian primitives corresponding to the head grids of all the RGB image frames are optimized, and combine all the optimized Gaussian primitives to obtain a reconstructed digital head;

[0018] Inputting the third information into a preset digital human body reconstruction model, and outputting a reconstructed digital human body;

[0019] A human body image with a focused face is obtained by rendering according to the reconstructed digital head and the reconstructed digital human body.

[0020] Further, acquiring the second information and the third information according to the first information includes:

[0021] Get the RGB image frame sequence of the dynamic human body captured by the monocular camera;

[0022] Inputting the RGB image frame sequence of the dynamic human body captured by the monocular camera into a pre-trained human head parameter prediction model, and outputting second information;

[0023] The RGB image frame sequence of the dynamic human body captured by the monocular camera is input into a pre-trained human body parameter prediction model to output third information.

[0024] Furthermore, the initialization parameter G is expressed as:

[0025] Wherein, μ represents the texture coordinate, and the texture coordinate μ can be used to locate a fixed point on the head mesh. , the point is the position of the Gaussian basis element, represents the rotation coefficient, represents the scaling factor, α represents the opacity, h represents the spherical harmonic function, Initialized to quaternion (1,0,0,0), Initialized to the distance between the nearest Gaussian primitive, the calculation formula is as follows:

[0026] in, Indicates the departure point The center position of the nearest Gaussian primitive, the opacity α of each Gaussian primitive is initialized to 0.1, and the spherical harmonic function h is initialized to A tensor of dimension N, where N is the number of Gaussian basis elements, the order of spherical harmonics is d=3, the value of the tensor at (N,3,0) is 0.5, and the rest are 0. The dist function is used to calculate the Euclidean distance between two coordinate points.

[0027] Furthermore, face recognition is performed on the current RGB image frame, and super-resolution processing and semantic segmentation are performed on the face area recognized by the face, so as to obtain the current face RGB image frame, including:

[0028] Perform face recognition on the current RGB image frame based on the face recognition algorithm to obtain the face area;

[0029] Performing super-resolution processing on the face area to obtain a clear face image;

[0030] Perform semantic segmentation on the clear face image to obtain a current face RGB image frame.

[0031] Further, according to the initialization parameters and the expression parameters of the current RGB image frame, new parameters of the Gaussian basis element of the head grid corresponding to the current RGB image frame are obtained based on a multi-layer perceptron, including:

[0032] encoding the texture coordinates;

[0033] Input the encoding result and the expression parameters of the current RGB image frame into a multi-layer perceptron, and output the offset, rotation and scaling of the Gaussian primitive position of the head grid;

[0034] Adding the offset to the position of the Gaussian primitive to obtain a new position of the Gaussian primitive corresponding to the head grid of the current RGB image frame;

[0035] Multiplying the rotation amount by the rotation coefficient to obtain a new rotation coefficient of the Gaussian basis element of the head grid corresponding to the current RGB image frame;

[0036] The scaling amount is multiplied by the scaling factor to obtain a new scaling factor of the Gaussian basis element of the head grid corresponding to the current RGB image frame.

[0037] Furthermore, the new parameters of the Gaussian primitives of the head grid corresponding to the current RGB image frame are expressed as:

[0038]

[0039]

[0040] in, , , represents the offset, rotation and scaling respectively, D represents the multilayer perceptron, θ i represents the expression parameters of the current RGB image frame, represents the result of the encoding, , , The new position, new rotation coefficient and new scaling coefficient of the Gaussian basis element of the head grid corresponding to the current RGB image frame are sequentially represented.

[0041] Furthermore, the calculation steps of the loss between the head rendering picture and the current face RGB image frame are as follows:

[0042] The head rendering image is obtained by spherical harmonic rendering through the new position, new rotation coefficient, new scaling coefficient, and opacity of the Gaussian primitive corresponding to the head mesh of the current RGB image frame. ;

[0043] Render the head image The loss reduction optimization is performed with the current face RGB image frame, and the formula is as follows:

[0044]

[0045]

[0046] in Dynamic human body images captured by the camera, is the position of the face recognition frame, crop is the image cropping, super is the image super-resolution, is the result of the super-resolution processing, is the result of the semantic segmentation, is the total loss function, and loss is the loss.

[0047] Furthermore, the total loss function for:

[0048]

[0049] Where λ1 and λ2 are constants, and Huber loss for:

[0050]

[0051] in, Indicates that the camera takes an image. Represents a rendered image, ;

[0052] Its perceptual similarity loss for:

[0053]

[0054] in Represents a rendered image The feature map of layer i in the VGG deep network is Indicates that the camera captures an image In the VGG deep network, the i-th layer feature map, N is the number of feature layers for calculating the loss, Representation calculation The L2 norm of , and then square it.

[0055] Further, rendering a human body image with a focused face according to the reconstructed digital head and the reconstructed digital human body includes:

[0056] A dynamic human body rendering image is obtained according to the reconstructed digital human body and the predicted human body camera pose rendering ;

[0057] A dynamic human head rendering image is obtained by rendering according to the reconstructed digital head, the predicted head camera pose and the predicted expression parameters. ;

[0058] The mask result of the semantic segmentation corresponding to the reconstructed digital head Rendering of the dynamic human head Segmentation to get the face image ;

[0059] The dynamic human body rendering and the facial image Fusion is performed to obtain the human body image of the focused face , the formula is as follows:

[0060] .

[0061] Furthermore, the digital human body reconstruction model is a SMPL model.

[0062] The beneficial effects of the present invention are:

[0063] The present invention proposes a face-focused digital human body reconstruction method which not only focuses on the reconstruction of the human body as a whole, but also enhances the high-precision reconstruction of the face, and can present a smooth and realistic human body animation effect in subsequent rendering. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of a method for reconstructing a digital human focusing on the face of the present application. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0066] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] The terms “first”, “second”, etc. are only used for distinguishing descriptions and should not be understood as indicating or implying relative importance.

[0068] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0069] like Figure 1 As shown, this embodiment provides a digital human body reconstruction method focusing on the face of the present invention, comprising:

[0070] Acquire first information, wherein the first information includes a sequence of RGB image frames of a dynamic human body;

[0071] Acquire second information and third information according to the first information, wherein the second information includes shape parameters, expression parameters, and head camera pose of the FLAME model corresponding to each predicted RGB image frame, and the third information includes shape parameters, posture parameters, and human camera pose of the digital human body reconstruction model corresponding to each predicted RGB image frame;

[0072] Constructing a head grid corresponding to each RGB image frame based on a preset FLAME model according to the second information;

[0073] Uniformly sampling the UV texture map of the head mesh corresponding to each RGB image frame to obtain a corresponding sampling result;

[0074] Performing Gaussian field initialization on the corresponding head grid according to the corresponding sampling result to obtain initialization parameters of the Gaussian basis element of the corresponding head grid;

[0075] Enter the current RGB image frame processing steps:

[0076] Perform face recognition on the current RGB image frame, perform super-resolution processing and semantic segmentation on the face area recognized by the face, and obtain the current face RGB image frame;

[0077] According to the initialization parameters and the expression parameters of the current RGB image frame, new parameters of the Gaussian basis element of the head grid corresponding to the current RGB image frame are obtained based on a multi-layer perceptron;

[0078] A head rendering image is obtained according to the new parameters of the Gaussian primitives of the head mesh corresponding to the current RGB image frame and the head camera pose rendering, the loss between the head rendering image and the current face RGB image frame is calculated, and the parameters of the Gaussian primitives of the head mesh corresponding to the current RGB image frame are optimized by a training strategy that minimizes the loss, and then the current RGB image frame processing step is ended;

[0079] Iteratively process other RGB image frames according to the current RGB image frame processing step until the parameters of the Gaussian primitives corresponding to the head grids of all the RGB image frames are optimized, and combine all the optimized Gaussian primitives to obtain a reconstructed digital head;

[0080] Inputting the third information into a preset digital human body reconstruction model, and outputting a reconstructed digital human body;

[0081] A human body image with a focused face is obtained by rendering according to the reconstructed digital head and the reconstructed digital human body.

[0082] In some embodiments, obtaining the second information and the third information according to the first information includes:

[0083] Get the RGB image frame sequence of the dynamic human body captured by the monocular camera;

[0084] Inputting the RGB image frame sequence of the dynamic human body captured by the monocular camera into a pre-trained human head parameter prediction model, and outputting second information;

[0085] The RGB image frame sequence of the dynamic human body captured by the monocular camera is input into a pre-trained human body parameter prediction model to output third information.

[0086] In some embodiments, the initialization parameter G is expressed as:

[0087]

[0088] Wherein, μ represents the texture coordinate, and the texture coordinate μ can be used to locate a fixed point on the head mesh. , the point is the position of the Gaussian basis element, represents the rotation coefficient, represents the scaling factor, α represents the opacity, h represents the spherical harmonic function, Initialized to quaternion (1,0,0,0), Initialized to the distance between the nearest Gaussian primitive, the calculation formula is as follows:

[0089]

[0090] in, Indicates the departure point The center position of the nearest Gaussian primitive, the opacity α of each Gaussian primitive is initialized to 0.1, and the spherical harmonic function h is initialized to A tensor of dimension , where is the number of Gaussian basis elements, the order of spherical harmonics is d=3, the value of the tensor at (N,3,0) is 0.5, and the rest are 0. The dist function is used to calculate the Euclidean distance between two coordinate points.

[0091] In some embodiments, face recognition is performed on the current RGB image frame, and super-resolution processing and semantic segmentation are performed on the face region obtained by face recognition in sequence to obtain the current face RGB image frame, including:

[0092] Perform face recognition on the current RGB image frame based on the face recognition algorithm to obtain the face area;

[0093] Performing super-resolution processing on the face area to obtain a clear face image;

[0094] Perform semantic segmentation on the clear face image to obtain a current face RGB image frame.

[0095] Further, according to the initialization parameters and the expression parameters of the current RGB image frame, new parameters of the Gaussian basis element of the head grid corresponding to the current RGB image frame are obtained based on a multi-layer perceptron, including:

[0096] encoding the texture coordinates;

[0097] Input the encoding result and the expression parameters of the current RGB image frame into a multi-layer perceptron, and output the offset, rotation and scaling of the Gaussian primitive position of the head grid;

[0098] Adding the offset to the position of the Gaussian primitive to obtain a new position of the Gaussian primitive corresponding to the head grid of the current RGB image frame;

[0099] Multiplying the rotation amount by the rotation coefficient to obtain a new rotation coefficient of the Gaussian basis element of the head grid corresponding to the current RGB image frame;

[0100] The scaling amount is multiplied by the scaling factor to obtain a new scaling factor of the Gaussian basis element of the head grid corresponding to the current RGB image frame.

[0101] In some embodiments, the new parameters of the Gaussian primitives corresponding to the head grid of the current RGB image frame are expressed as:

[0102]

[0103]

[0104] in, , , represents the offset, rotation and scaling respectively, D represents the multilayer perceptron, θ i represents the expression parameters of the current RGB image frame, represents the result of the encoding, , , The new position, new rotation coefficient and new scaling coefficient of the Gaussian basis element of the head grid corresponding to the current RGB image frame are sequentially represented.

[0105] In some embodiments, the calculation steps of the loss between the head rendering picture and the current face RGB image frame are as follows:

[0106] The head rendering image is obtained by spherical harmonic rendering through the new position, new rotation coefficient, new scaling coefficient, and opacity of the Gaussian primitive corresponding to the head mesh of the current RGB image frame. ;

[0107] Render the head image The loss reduction optimization is performed with the current face RGB image frame, and the formula is as follows:

[0108]

[0109]

[0110] in Dynamic human body images captured by the camera, is the position of the face recognition frame, crop is the image cropping, super is the image super-resolution, is the result of the super-resolution processing, is the result of the semantic segmentation, is the total loss function, and loss is the loss.

[0111] In some embodiments, the total loss function for:

[0112]

[0113] Where λ1 and λ2 are constants, and Huber loss for:

[0114]

[0115] in, Indicates that the camera takes an image. Represents a rendered image, ;

[0116] Its perceptual similarity loss for:

[0117]

[0118] in Represents a rendered image The feature map of layer i in the VGG deep network is Indicates that the camera captures an image In the VGG deep network, the i-th layer feature map, N is the number of feature layers for calculating the loss, Representation calculation The L2 norm of , and then square it.

[0119] like Figure 1 As shown, in some embodiments, obtaining a human body image with a focused face according to the reconstructed digital head and the reconstructed digital human body rendering includes:

[0120] A dynamic human body rendering image is obtained according to the reconstructed digital human body and the predicted human body camera pose rendering ;

[0121] A dynamic human head rendering image is obtained by rendering according to the reconstructed digital head, the predicted head camera pose and the predicted expression parameters. ;

[0122] The mask result of the semantic segmentation corresponding to the reconstructed digital head Rendering of the dynamic human head Segmentation to get the face image ;

[0123] The dynamic human body rendering and the facial image Fusion is performed to obtain the human body image of the focused face , the formula is as follows:

[0124] .

[0125] In some embodiments, the digital human body reconstruction model is a SMPL model.

[0126] The advantages of the present invention compared to the prior art are:

[0127] 1. Based on the traditional digital human reconstruction model, the FLAME model is combined to construct a human reconstruction method with high-precision facial detail expression. It not only focuses on the reconstruction of the human body as a whole, but also strengthens the high-precision reconstruction of the face. Based on the reconstructed human model, real-time rendering of human animation is achieved, the dynamic effect of facial expression changes is naturally generated, and the coordination between movements and expressions is ensured, presenting a smooth and realistic human animation effect;

[0128] 2. During the training process, the loss function is designed to optimize the human body reconstruction parameters by considering the color, normal map, depth map, joint point data and facial features of the pixels in the corresponding perspective image, and the 3D points reconstructed in the facial area are fused in real time;

[0129] 3. In the human animation rendering stage, the human body posture is captured in real time through a monocular camera, and the 3D posture data and camera parameters of the human body in the picture are predicted, thereby driving the digital human to change to a human animation in a new posture. Specifically, the rotation difference and scaling difference are calculated based on the triangular facets corresponding to the human body mesh model in the current posture and the human body mesh model in the standard posture, and the final primitive parameter value in the Gaussian animation field is calculated based on this. The human body animation in the current posture can be generated by rendering the Gaussian animation field in real time according to the predicted camera parameters. The face area in the image is recognized and tracked in real time, the face posture and camera posture are extracted, and a facial animation Gaussian field is generated, which is used to update the face area in the human Gaussian animation field, thereby rendering human animation with rich facial expressions;

[0130] 4. This method does not require the traditional method of multi-view training, and only uses single-purpose video. It reconstructs high-fidelity digital humans and achieves high-definition face and full-body rendering;

[0131] 5. Fast rendering speed. This method based on Gaussian splash technology is faster than the traditional Nerf-based method;

[0132] 6. The face rendering quality is high. Compared with the monocular video reconstruction method based on the SMPL-X model, this method can achieve higher accuracy and animation details by segmenting the face and reconstructing it separately.

[0133] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A digital human body reconstruction method focusing on the face, characterized in that: include: Acquire first information, wherein the first information includes a sequence of RGB image frames of a dynamic human body; Acquire second information and third information according to the first information, wherein the second information includes shape parameters, expression parameters, and head camera pose of the FLAME model corresponding to each predicted RGB image frame, and the third information includes shape parameters, posture parameters, and human camera pose of the digital human body reconstruction model corresponding to each predicted RGB image frame; Constructing a head grid corresponding to each RGB image frame based on a preset FLAME model according to the second information; Uniformly sampling the UV texture map of the head mesh corresponding to each RGB image frame to obtain a corresponding sampling result; Performing Gaussian field initialization on the corresponding head grid according to the corresponding sampling result to obtain initialization parameters of the Gaussian basis element of the corresponding head grid; Enter the current RGB image frame processing steps: Perform face recognition on the current RGB image frame, perform super-resolution processing and semantic segmentation on the face area recognized by the face, and obtain the current face RGB image frame; According to the initialization parameters and the expression parameters of the current RGB image frame, new parameters of the Gaussian basis element of the head grid corresponding to the current RGB image frame are obtained based on a multi-layer perceptron; Obtain a head rendering image according to the new parameters of the Gaussian primitives of the head mesh corresponding to the current RGB image frame and the head camera pose rendering, calculate the loss between the head rendering image and the current face RGB image frame, optimize the parameters of the Gaussian primitives of the head mesh corresponding to the current RGB image frame by a training strategy that minimizes the loss, and then end the current RGB image frame processing step; Iteratively process other RGB image frames according to the current RGB image frame processing step until the parameters of the Gaussian primitives corresponding to the head grids of all the RGB image frames are optimized, and combine all the optimized Gaussian primitives to obtain a reconstructed digital head; Inputting the third information into a preset digital human body reconstruction model, and outputting a reconstructed digital human body; A human body image with a focused face is obtained by rendering according to the reconstructed digital head and the reconstructed digital human body.

2. The method for reconstructing a digital human body focusing on the face according to claim 1, characterized in that: Acquiring second information and third information according to the first information includes: Get the RGB image frame sequence of the dynamic human body captured by the monocular camera; Inputting the RGB image frame sequence of the dynamic human body captured by the monocular camera into a pre-trained human head parameter prediction model, and outputting second information; The RGB image frame sequence of the dynamic human body captured by the monocular camera is input into a pre-trained human body parameter prediction model to output third information.

3. The method for reconstructing a digital human body focusing on the face according to claim 1, characterized in that: The initialization parameter G is expressed as: ， Wherein, μ represents the texture coordinate, and the texture coordinate μ can be used to locate a fixed point on the head mesh. , the point is the position of the Gaussian basis element, represents the rotation coefficient, represents the scaling factor, α represents the opacity, h represents the spherical harmonic function, Initialized to quaternion (1,0,0,0), Initialized to the distance between the nearest Gaussian primitive, the calculation formula is as follows: ， in, Indicates the departure point The center position of the nearest Gaussian primitive, the opacity α of each Gaussian primitive is initialized to 0.1, and the spherical harmonic function h is initialized to A tensor of dimension N, where N is the number of Gaussian basis elements, the order of spherical harmonics is d=3, the value of the tensor at (N,3,0) is 0.5, and the rest are 0. The dist function is used to calculate the Euclidean distance between two coordinate points.

4. A face-focused digital human reconstruction method according to claim 1 or 3, characterized in that: Perform face recognition on the current RGB image frame, perform super-resolution processing and semantic segmentation on the face area identified by face recognition, and obtain the current face RGB image frame, including: Perform face recognition on the current RGB image frame based on the face recognition algorithm to obtain the face area; Performing super-resolution processing on the face area to obtain a clear face image; Perform semantic segmentation on the clear face image to obtain a current face RGB image frame.

5. The method for reconstructing a digital human body focused on the face according to claim 3, characterized in that: According to the initialization parameters and the expression parameters of the current RGB image frame, new parameters of the Gaussian basis element of the head grid corresponding to the current RGB image frame are obtained based on a multi-layer perceptron, including: encoding the texture coordinates; Input the encoding result and the expression parameters of the current RGB image frame into a multi-layer perceptron, and output the offset, rotation and scaling of the Gaussian primitive position of the head grid; Adding the offset to the position of the Gaussian primitive to obtain a new position of the Gaussian primitive corresponding to the head grid of the current RGB image frame; Multiplying the rotation amount by the rotation coefficient to obtain a new rotation coefficient of the Gaussian basis element of the head grid corresponding to the current RGB image frame; The scaling amount is multiplied by the scaling factor to obtain a new scaling factor of the Gaussian basis element of the head grid corresponding to the current RGB image frame.

6. The method for reconstructing a digital human body focusing on the face according to claim 5, characterized in that: The new parameters of the Gaussian primitives of the head grid corresponding to the current RGB image frame are expressed as: ， ， in, , , represents the offset, rotation and scaling respectively, D represents the multilayer perceptron, θ i represents the expression parameters of the current RGB image frame, represents the result of the encoding, , , The new position, new rotation coefficient and new scaling coefficient of the Gaussian basis element of the head grid corresponding to the current RGB image frame are sequentially represented.

7. The method for reconstructing a digital human body focused on the face according to claim 6, characterized in that: The calculation steps of the loss between the head rendering picture and the current face RGB image frame are as follows: The head rendering image is obtained by spherical harmonic rendering through the new position, new rotation coefficient, new scaling coefficient, and opacity of the Gaussian primitive corresponding to the head mesh of the current RGB image frame. ; Render the head image The loss reduction optimization is performed with the current face RGB image frame, and the formula is as follows: ， ， in Dynamic human body images captured by the camera, is the position of the face recognition frame, crop is the image cropping, super is the image super-resolution, is the result of the super-resolution processing, is the result of the semantic segmentation, is the total loss function, and loss is the loss.

8. The method for reconstructing a digital human body focused on the face according to claim 7, characterized in that: The total loss function for: ， Where λ1 and λ2 are constants, and Huber loss for: ， in, Indicates that the camera takes an image. Represents a rendered image, ; Its perceptual similarity loss for: ， in Represents a rendered image The feature map of layer i in the VGG deep network is Indicates that the camera captures an image In the VGG deep network, the i-th layer feature map, N is the number of feature layers for calculating the loss, Representation calculation The L2 norm of is then squared.

9. The method for reconstructing a digital human body focused on the face according to claim 8, characterized in that: Rendering a human body image with a focused face according to the reconstructed digital head and the reconstructed digital human body includes: A dynamic human body rendering image is obtained according to the reconstructed digital human body and the predicted human body camera pose rendering ; A dynamic human head rendering image is obtained by rendering according to the reconstructed digital head, the predicted head camera pose and the predicted expression parameters. ; The mask result of the semantic segmentation corresponding to the reconstructed digital head Rendering of the dynamic human head Segmentation to get the face image ; The dynamic human body rendering and the facial image Fusion is performed to obtain the human body image of the focused face , the formula is as follows: 。 10. The method for reconstructing a digital human body focused on the face according to claim 1, characterized in that: The digital human body reconstruction model is an SMPL model.

Images