A dynamic human body reconstruction method based on neural radiance field

Abstract

The application discloses a dynamic human body reconstruction method based on a neural radiation field, and comprises the following steps: S1, establishing a dynamic human body reconstruction network based on a neural radiation field, wherein the dynamic human body reconstruction network comprises an action branch for acquiring an action field and a reconstruction branch for acquiring a neural radiation field; S2, initializing the action branch and the reconstruction branch in a specified resting posture by using a human body parameterized model; S3, using real images in a given posture and given camera internal and external parameters as a supervision signal, and simultaneously training the action branch and the reconstruction branch by taking the minimum mean square error between the real images and synthetic images as a loss function; and S4, inputting a new human body posture and a camera pose to the trained dynamic human body reconstruction network, and rendering a human body image in any view point in the new posture.

Description

Technical Field

[0001] This invention relates to the field of computer vision and 3D reconstruction, and in particular to a dynamic human body reconstruction method based on neural radiation fields. Background Technology

[0002] How to directly obtain a drivable human body model from image input has long been a major challenge in the fields of computer vision and computer graphics. Simultaneously acquiring the static human body surface and the surface deformation caused by movement from video (images) is an ill-posed problem without a stable and unique solution. Therefore, research in this field often adopts a two-stage approach: first static modeling, then automatic or manual bone binding.

[0003] Traditional 3D human body reconstruction methods, such as multi-view scanning methods, rely on acquisition devices and systems such as multi-view synchronous cameras and depth cameras to obtain relatively detailed static human body models. With the development of deep learning, existing 3D human body reconstruction methods, by leveraging network structures such as convolutional neural networks and graph neural networks, and based on reconstruction ideas such as pixel alignment and volume rendering, have made it possible to obtain 3D human body models using fewer viewpoints.

[0004] On the other hand, most existing 3D human body reconstruction methods can only acquire static human body sculptures and cannot change the human body model's movements according to new motion inputs. How to transform the reconstructed static human body model into a driveable model has become a research focus. Human geometry is highly structured and regular, so early conversion methods treated the deformation of the human body surface as rigid motion following the movement of the skeleton. These methods use a large number of high-quality static human body models as input to acquire the human skeleton and assign each vertex of the human body surface model to each skeleton set, so that the vertex follows the movement of the skeleton. However, this series of methods relies on a pre-given static scan of the human body and lacks the ability to process non-rigid deformations caused by movement, such as clothing wrinkles and detailed textures. In recent years, the proposal of the neural radiation field method has made it possible to acquire dynamic human bodies, and to obtain a sequence of frame human body 3D models by directly or indirectly inserting timestamps and constructing motion fields guided by human posture.

[0005] However, current methods for driving human reconstruction based on neural radiation fields have two prominent problems. First, the methods have poor generalization ability under new poses outside the dataset: when the input pose differs greatly from the poses in the training dataset, the methods often give completely wrong driving results. Second, the current methods take a long time to optimize for a single person, making them difficult to apply in practice: the optimization time for a single person's neural radiation field representation can be as long as tens of hours. Summary of the Invention

[0006] In view of this, the present invention proposes a dynamic human body reconstruction method based on neural radiation field, which solves the technical problems of long optimization time and poor driving effect in the above-mentioned prior art.

[0007] To solve the above-mentioned technical problems, the present invention proposes the following technical solution:

[0008] A dynamic human body reconstruction method based on neural radiation field includes the following steps: S1, establishing a dynamic human body reconstruction network based on neural radiation field, the dynamic human body reconstruction network including an action branch for acquiring action field and a reconstruction branch for acquiring neural radiation field;

[0009] S2. Initialize the action branch and reconstruction branch under the specified resting pose using the human body parameterization model; S3. Train the action branch and reconstruction branch simultaneously using the real image under the given pose and given camera intrinsic and extrinsic parameters as the supervision signal and the minimum mean square error between the real image and the synthesized image as the loss function; S4. Input the new human pose and camera pose into the trained dynamic human reconstruction network and render the synthesized human image at any viewpoint under the new pose.

[0010] Furthermore, the action branch takes the pre-regressed human parameterized model as input and maps each sampling point on any ray in the action space to a specified resting posture space; the reconstruction branch renders the pixel color on the imaging plane corresponding to the ray based on the volume density and color of each sampling point in the resting posture space.

[0011] Furthermore, the action branch includes a skeletal action branch and an action field fine-tuning branch; the skeletal action branch takes a discretized human body parameterized model corresponding to the current individual and action as input and outputs a skeletal action field represented by a three-dimensional mesh; the action field fine-tuning branch takes the motion vector of any three-dimensional point under the control of the skeleton and the joint rotation angle corresponding to the current action of the human body as input to obtain the action residual; the offset of any three-dimensional point with the skeletal action is added to the corresponding action residual as the output of the action branch.

[0012] Furthermore, the reconstruction branch uses a sparse activation grid to store learnable parameters representing the volume density and spherical harmonic illumination coefficient in space, represents the observed color of the sampling point under different viewpoints as a spherical harmonic function, represents the light penetration probability at any point in space as a volume density, and uses the volume rendering formula to obtain the pixel color on the imaging plane corresponding to the ray.

[0013] Further, step S2 specifically includes: S21, obtaining a human parameterized model using an existing multi-view human parameterized model regression method, then specifying the mesh size, setting meshes whose eight vertices are all outside the human body to 0, and the rest to 1, to obtain a discretized human parameterized model; S22, initializing action branches: using the skeletal motion field obtained from the discretized human parameterized model under the current action and the human parameterized model under the specified resting posture as supervision signals, pre-training the skeletal motion branches of the action branches; S23, initializing reconstruction branches: in the initial stage, the sparse activation mesh of the reconstruction branch is densely activated at a lower resolution, and according to the discretized human parameterized model, points outside the human parameterized model are set to a smaller volume density value, and meshes that are wholly or partially inside the human body are set to a larger volume density value.

[0014] Further, step S23 specifically includes: first, adjusting the pre-estimated parameterized model to a specified resting pose and translating it to the center point of the world coordinate system, and then scaling it to a cube space with a fixed side length; initializing the sparse active mesh as a mesh with a lower resolution; if the distance between any mesh center point and its corresponding nearest parameterized model surface is less than a fixed value, then the initial value of the mesh volume density is initialized to a larger value greater than a specified amount, otherwise the initial value of the volume density is initialized to a smaller value less than a specified amount.

[0015] Furthermore, step S3 also includes: in order to encourage the reconstruction branch to eliminate non-surface regions as soon as possible, a regularization term is added to the training process to make the sparsely activated mesh sparsely activated only near the surface, wherein the regularization term is the average value of all mesh volume density values.

[0016] Furthermore, step S3 also includes: using additional ground truth image motion data to discriminately optimize the generalization of the dynamic human body reconstruction network to new poses.

[0017] Furthermore, the generalization of the dynamic human reconstruction network to new poses is optimized by using additional ground truth-less image motion data in a discriminative manner. Specifically, this includes: using additional ground truth-less image motion data and a parameterized model obtained from the current individual body shape parameters as network input; using a discriminator to determine whether the current synthesis result output by the network is a real image augmented view synthesis performance; using a human pose estimation network with frozen weights to estimate the two-dimensional joint heatmap of the corresponding synthesized image; and using the two-dimensional joint image obtained by projecting the corresponding input parameterized model as pseudo-ground truth for weakly supervised training.

[0018] Further, step S4 specifically includes: given a human body parameterized model under a new pose and replacing its body shape parameters with the driving target body shape parameters; inputting the adjusted human body parameterized model into the skeletal motion branch of the motion branch and caching the position offset caused by the skeletal motion; densely sampling to generate rays on the imaging plane corresponding to the given camera pose, querying the skeletal motion and fine-tuning the motion field fine-tuning branch to obtain the motion residuals of each point on the ray; adding the position offsets caused by the skeletal motion to the motion residuals to obtain the coordinates of each point in the specified resting pose space; querying the corresponding volume density and color using the coordinates of each point in the resting space as an index, and obtaining the synthesized image under the given new pose and new viewpoint using the volume rendering formula.

[0019] The beneficial effects of this invention are as follows: This invention can not only acquire a static human body model in a specified pose, but also, given data from other poses (actions), acquire the image synthesis (rendering) result from any viewpoint in a new pose. This invention significantly improves the optimization speed by utilizing sparse activation meshes and efficient initialization methods. The geometry and color of the human body model in a specified resting pose are represented by sparse activation meshes to reduce rendering and optimization time. Furthermore, the dynamic performance optimization method proposed in this invention improves the accuracy of view synthesis in new poses by the obtained dynamic human body representation, while ensuring that the human body pose in the synthesized image is consistent with the input pose. Attached Figure Description

[0020] Figure 1 This is a flowchart of the dynamic human body reconstruction method based on neural radiation field according to an embodiment of the present invention.

[0021] Figure 2 This is a diagram of the dynamic human body reconstruction network architecture based on the neural radiation field in an embodiment of the present invention. Detailed Implementation

[0022] The present patent application will now be described in further detail with reference to specific embodiments and the accompanying drawings. It should be emphasized that the following description is merely exemplary and is not intended to limit the scope or application of the present patent application.

[0023] Non-limiting and non-exclusive embodiments will be described with reference to the following figures, wherein the same reference numerals denote the same parts unless otherwise specifically stated.

[0024] Those skilled in the art will recognize that numerous variations are possible with respect to the above description, and therefore the embodiments are merely illustrative of one or more specific implementations.

[0025] This invention proposes a dynamic human body reconstruction method based on neural radiation fields. Please refer to [link / reference]. Figure 1 The method generally includes the following steps S1 to S4:

[0026] S1. Establish a dynamic human body reconstruction network based on neural radiation field. The dynamic human body reconstruction network includes an action branch for acquiring the action field and a reconstruction branch for acquiring the neural radiation field.

[0027] S2. Initialize the motion branches and reconstruction branches for the specified resting posture using the human parametric model;

[0028] S3. Using real images under given pose and given camera intrinsic and extrinsic parameters as supervision signals, and using the minimum mean square error between real images and synthetic images as the loss function, train both action branches and reconstruction branches simultaneously.

[0029] S4. Input the new human pose and camera pose into the trained dynamic human reconstruction network, and render and synthesize human images from any viewpoint under the new pose.

[0030] The network architecture constructed in step S1 is described in detail below. Figure 2 This is an architecture diagram of the dynamic human body reconstruction network based on neural radiation fields constructed according to an embodiment of the present invention. Please refer to... Figure 2 This dynamic human reconstruction network includes an action branch and a reconstruction branch. The action branch takes a pre-regressed parametric human model as input. Each sampling point on any ray in the action space is mapped to a specified resting pose space through the action branch. The reconstruction branch provides the volume density and color of each sampling point in the resting pose space, and the color of the pixel on the imaging plane corresponding to the ray is given by the volume rendering formula. Specifically, the action branch decomposes the action into a coarse skeletal action field and obtains a dense action field through point-by-point non-rigid motion, so that any point x in the current action space... o Mapped to a point x in the resting posture space c For the resting posture space point x obtained from the action branch c The reconstruction branch queries the corresponding position of the point in the grid and uses trilinear interpolation to obtain the volume density and color (spherical harmonic illumination coefficient) of the point. Overall, when using several sampling points on the ray in the action space as input, the action branch twists the ray into a curve in the resting pose space, the reconstruction branch obtains the color and volume density of the sampling points, and integrates along the curve to obtain the color of the corresponding pixel.

[0031] Continue to refer to Figure 2The motion branch includes a skeletal motion branch and a motion field fine-tuning branch for modeling non-rigid deformations. In some embodiments, the skeletal motion branch uses 3D-Unet as its basic structure, taking a discretized parametric model of the human body corresponding to the current human body and its actions as input, and obtaining a rigid motion field (i.e., skeletal motion field) under skeletal motion through 3D-Unet. Simultaneously, to distinguish between individuals of similar body types and the same person wearing different clothing, individuals wearing different clothing are treated as different individuals, and learnable encodings for the corresponding individuals are added to the bottleneck layer of the skeletal motion branch. The skeletal motion branch outputs a skeletal motion field represented by a 3D mesh, where each voxel or vector in the mesh measures the degree to which any point in the current mesh is dominated by the specified skeletal motion. Considering non-rigid surfaces such as clothing, the motion field fine-tuning branch estimates the non-rigid deformation of any point in space under the current pose, based on the rigid motion field. Specifically, for any 3D point, the motion field fine-tuning branch takes the motion vector of the 3D point under skeletal dominance and the joint rotation angle corresponding to the current human body as input to obtain the motion residual vector. The output of the motion branch is the sum of the offset of any 3D point with the bone movement and its corresponding motion residual.

[0032] Specifically, a coarse skeletal motion field is obtained by inputting the skeletal motion branches constructed using 3D-Unet into a discretized parametric model. o Adding the corresponding positional motion vector v(xo) to obtain the skeletal motion vector x s To differentiate individuals with similar body sizes, an additional learnable encoding is introduced in the bottleneck layer of 3D-Unet; the network skeleton of the motion field fine-tuning branch is a multilayer perceptron network (MLP) with residual structure, taking the joint rotation matrix R, the sampling point position xo, and the rough motion field v(xo) at the sampling point as input, and outputting non-rigid deformation:

[0033] x c =x s +MLp(x o ,v(x o ), R).

[0034] The reconstruction branch represents the neural radiation field in space using learnable parameters stored in a sparsely activated mesh, and represents the volume density and color information in space as learnable parameters in the corresponding location mesh, and optimizes this using view synthesis loss. Specifically, the reconstruction branch stores learnable parameters representing volume density and spherical harmonic illumination coefficient (color) in space using a sparsely activated mesh, represents the observed color of any point under different viewpoints as a spherical harmonic function, and represents the light penetration probability at any point in space as volume density. During the optimization process, spaces excluding the human body are gradually deactivated, and the mesh is further subdivided only in areas with high volume density values.

[0035] In some embodiments, the reconstruction branch is a sparse activation grid, represented and indexed in the form of an octree, where each grid stores several learnable parameters. In this example, each grid contains 28 learnable parameters, one of which corresponds to the volume density value, and the other 27 represent the coefficients of the second-order spherical harmonic function of the RGB three channels.

[0036] During the optimization process, for meshes whose volume density value drops below a specified threshold after a certain number of iterations, they are deactivated by setting them to 0. To ensure reconstruction accuracy, an upsampling process is performed every 20 iterations if there is no significant loss: the meshes that have not yet been deactivated are expanded into 8 sub-meshes, the volume density value is halved, and the ray sampling interval is halved.

[0037] For the ray emitted from the camera optical center in the motion space, several 3D points are sampled on it at fixed intervals. The motion branch maps each sampled point to the resting pose space. The volume density and spherical harmonic illumination coefficient are obtained by trilinear interpolation, and then volume rendering is performed.

[0038] The first step in volume rendering is to determine the observed color using the spherical harmonic illumination coefficients and the estimated observation direction. After mapping each point on the sampling ray to the resting pose space, the direction of each point is considered as the direction of the curve at that point, and is approximated by the direction of the line segment between two sampling points. The formula for determining color c can then be expressed as:

[0039]

[0040] in, The spherical harmonic coefficients, Using spherical harmonics, in this example, since the data illumination is relatively uniform, the spherical harmonic order is chosen as l = 2. There are a total of 27 spherical harmonic coefficients for the RGB three color channels. The list of spherical harmonic basis functions used is as follows:

[0041]

[0042] Step 2: Integrate to obtain the color of the corresponding pixel. In practice, this can be approximated using Monte Carlo methods.

[0043]

[0044] Where c pixel σ represents the color of the imaging plane position corresponding to the ray. i ,d j ,c i The i-th sampling point x i Volume density, sampling interval, and observed color.

[0045] The initialization of step S2 is described in detail below.

[0046] 1) Acquisition and discretization (gridization) of human body parametric model.

[0047] This invention employs an existing multi-view human parametric model regression method to obtain a human parametric model. During the discretization stage, a specified mesh size is used, and meshes whose eight vertices are all outside the human body are set to 0, while the rest are set to 1. Furthermore, the mesh points set to 1 can be assigned semantic labels such as face, legs, etc., based on the human body part they are closest to. The number of label types is determined by the parametric model used, and can generally be considered as 23 different parts.

[0048] 2) Initialize action branches.

[0049] To prevent the skeletal motion branches from failing to converge, pre-training of the skeletal motion branches is performed during the initialization phase. The skeletal motion fields obtained from the discretized human body parameterized model under the current action and the human body parameterized model under the specified resting posture are used as supervision signals to pre-train the skeletal motion branches of the action branches.

[0050] 3) Rebuild branch initialization.

[0051] This invention utilizes a pre-estimated parametric model (e.g., an existing image processing model, the Skinned Multi-Person Linear Model, SMPL model) for initialization to minimize queries and calculations that do not include human body surface areas. Specifically, the pre-estimated parametric model is first adjusted to a specified resting pose and translated to the center point of the world coordinate system, then scaled into a cube space with a fixed side length. The aforementioned sparse activation mesh is initialized as a low-resolution mesh. If the distance between any mesh center point and its corresponding nearest parametric model surface is less than a fixed value, the initial density of that mesh is initialized to a value greater than a specified amount; otherwise, the initial density is initialized to a smaller value less than the specified amount.

[0052] The reconstruction branch utilizes only surface and color information from the grid storage space. Initially, the grid is densely activated at a lower resolution. Based on the discretized human body parameterization model described in 1) above, points outside the model are assigned smaller volume density values, while grids wholly or partially within the human body are assigned larger initial volume density values; in this example, these are 0.1 and 1, respectively. During training, the grid will be further deactivated or subdivided to improve physical resolution.

[0053] The network training and optimization in step S3 are described in detail below.

[0054] First, there is the supervised training process of the network. In the early stages of optimization, the network adopts a fully supervised training method, using real images with given poses and camera intrinsic and extrinsic parameters as supervision signals, and using the minimum mean square error between real and synthetic images as the loss function to train both action and reconstruction branches simultaneously. At the same time, in order to encourage the reconstruction network to eliminate non-surface regions as quickly as possible, a regularization term is added to make the mesh sparsely activated only near the surface. The regularization term is the average value of the density values ​​of all mesh volumes.

[0055] During the supervised training phase, the action branch and the reconstruction branch are trained jointly. Given a specified pose and viewpoint, the network forward propagates to acquire the corresponding synthesized image, and the loss function is defined as the minimum mean square error between the synthesized image and the ground truth image. To ensure that the sparse activation mesh is sparsely activated only near the human body surface, this embodiment of the invention applies a regularization constraint to the volume density and considers only the portion of the volume density greater than a certain value during each mesh subdivision. This fixed value is determined based on experience and the specific volume rendering method, and its range depends on the representation of the volume density; the specific value is determined empirically. When using a volume density setting of 0 to +∞ and its corresponding volume rendering method, a value of around 1 is recommended. When using a truncated volume density of 0 to 1, a smaller value is recommended.

[0056] Secondly, dynamic performance optimization is performed. In existing datasets, the video length of each individual is relatively short and the poses and movements are relatively simple. This means that the network training process only considers a portion of the poses present in the dataset, which may lead to the network converging to a local optimum. On the other hand, existing large-scale action databases can basically encompass any common pose in life. In supervised training, the network needs to include training data containing both human parameterized models and real images from a specified viewpoint. In this case, each individual is optimized using only limited action data. To enhance the network's generalization ability for new poses, this embodiment of the invention utilizes ground-value-free image action data collected from the action database to discriminately optimize the network's dynamic performance under various poses, thereby enhancing the network's generalization ability for new poses.

[0057] In this embodiment of the invention, additional ground truth image motion data and a parameterized model obtained from the current individual body shape parameters are used as network inputs. A discriminator determines whether the current synthesis result output by the network is a real image augmented view synthesis performance. A human pose estimation network with frozen weights is used to estimate the two-dimensional joint heatmap of the corresponding synthesized image. The two-dimensional joint image obtained by projecting the corresponding input parameterized model is used as a pseudo-ground truth for weakly supervised training.

[0058] The reasoning process for step S4 is described in detail below.

[0059] After training, the network can be used to synthesize images of individuals wearing specific clothing from any new viewpoint and in any new pose. During the driving (inference) phase, the network takes the new viewpoint camera parameters, joint rotation angles, and 3D coordinates of the joints as input. It uses the action branch to map any query point (any point in the current action space) to the specified resting pose space. The reconstruction branch uses the volume density and color of the query point in the resting pose to obtain the synthesized result of the new view under the new pose through the volume rendering process.

[0060] Specifically, during the inference phase, given a human body parameterized model in any pose, its body shape parameters are replaced with the driving target body shape parameters. The adjusted (replaced) human body parameterized model is input into the skeletal motion branch of the action branch, and the inference results (positional offsets caused by skeletal motion) are cached. Rays are generated by dense sampling on the imaging plane corresponding to the given camera pose. The skeletal motion is queried, and the motion field fine-tuning branch is used to fine-tune and obtain the motion residuals at each point on the ray. The motion residuals are added to the positional offsets caused by the skeletal motion to obtain the coordinates of each point in the specified resting pose space. The corresponding volume density and color are queried using the coordinates of each point in the resting space as an index. The volume rendering formula is used to obtain the synthesized image under the new pose and new viewpoint specified in the driving (inference) phase. The processing of the volume rendering formula has been detailed above and will not be repeated here.

[0061] In summary, the dynamic human body reconstruction method based on neural radiation field proposed in this invention has the following functional characteristics: First, the geometry and color of the human body model under a specified resting posture are represented by a sparse activation mesh to reduce rendering and optimization time; Second, a dense action field guided by posture is constructed by using a three-dimensional convolutional and non-rigid deformation network; Third, for action data without corresponding ground truth images, a discriminative method is used for optimization to improve the generalization of the method under new actions.

[0062] Although exemplary embodiments considered to be present in this patent application have been described and illustrated, those skilled in the art will understand that various alterations and modifications can be made thereto without departing from the spirit of this patent application. Furthermore, many modifications can be made to adapt specific situations to the doctrine of this patent application without departing from the central concepts described herein. Therefore, this patent application is not limited to the specific embodiments disclosed herein, but may include others that fall within the scope of this patent application.

Claims

1. A method of dynamic human body reconstruction based on neural radiance fields, characterized in that, The method comprises the following steps: S1, establishing a dynamic human body reconstruction network based on a neural radiance field, the dynamic human body reconstruction network comprising an action branch for obtaining an action field and a reconstruction branch for obtaining a neural radiance field; S2, initializing the action branch and the reconstruction branch in a specified resting posture using a human body parameterized model; The action branch comprises a skeletal action branch and an action field fine-tuning branch; the skeletal action branch takes a discretized human body parameterized model corresponding to a current individual and an action as input and outputs a skeletal action field in a three-dimensional grid representation; the action field fine-tuning branch takes a motion vector of a three-dimensional point under the control of a skeleton and a joint rotation angle corresponding to a current action human body as input to obtain an action residual; and the output of the action branch is obtained by adding the offset of the three-dimensional point with the skeletal action to the corresponding action residual; The reconstruction branch stores learnable parameters of volume density and spherical harmonic illumination coefficients in a sparse activation grid representation space, expresses the observed color of a sampling point under different viewing angles in a spherical harmonic function, expresses the light penetration probability of any point in space in a volume density, and obtains the pixel color on the imaging plane corresponding to the ray by using a volume rendering formula; S3, training the action branch and the reconstruction branch simultaneously using real images under a given posture and given camera internal and external parameters as a supervision signal and using the minimum mean square error between the real images and the synthesized images as a loss function; S4, inputting a new human body posture and camera pose to the trained dynamic human body reconstruction network to render a synthesized human body image at any viewing point under the new posture.

2. The dynamic human body reconstruction method of claim 1, wherein: The action branch takes a human body parameterized model obtained by pre-regression as input and maps each sampling point on an arbitrary ray in an action space to a specified resting posture space; The reconstruction branch renders the pixel color on the imaging plane corresponding to the ray based on the volume density and color of each sampling point in the resting posture space.

3. The dynamic human body reconstruction method of claim 1, wherein, Step S2 specifically comprises: S21, obtaining a human body parameterized model by using an existing multi-view human body parameterized model regression method, then specifying a grid size, setting the grid vertices all outside the human body to 0 and the rest to 1 to obtain a discretized human body parameterized model; S22, initializing the action branch: using the skeletal action field obtained from the discretized human body parameterized model under a current action and the human body parameterized model under a specified resting posture as a supervision signal, pre-training the skeletal action branch of the action branch; S23, initializing the reconstruction branch: in the initial stage, the sparse activation grid of the reconstruction branch is densely activated at a low resolution, and according to the discretized human body parameterized model, the points outside the human body parameterized model are set to a small volume density value, and the grids that are wholly or partially inside the human body are set to a large volume density value.

4. The dynamic human body reconstruction method of claim 3, wherein, The step S23 specifically comprises: firstly adjusting the pre-estimated parameterized model to the specified rest pose, translating to the center point of the world coordinate system, and scaling to a cube space with a fixed side length; initializing the sparse activation grid as a low-resolution grid, and initializing the initial value of the grid density as a large value greater than a specified amount if the distance between any grid center point and the nearest surface of the corresponding parameterized model is less than a specified value, or initializing the grid density as a small value less than the specified amount.

5. The dynamic human body reconstruction method of claim 3, wherein, The step S3 further comprises: to facilitate the reconstruction branch to exclude non-surface regions as soon as possible, a regularization term is added to the training process to facilitate the sparse activation grid to be sparsely activated only near the surface, wherein the regularization term is the average value of all grid density values.

6. The dynamic human body reconstruction method of claim 1, wherein, The step S3 further comprises: using additional image motion data without ground truth to optimize the generalization of the dynamic human body reconstruction network to new poses in a discriminative manner.

7. The dynamic human body reconstruction method of claim 5, wherein, Using additional image motion data without ground truth to optimize the generalization of the dynamic human body reconstruction network to new poses in a discriminative manner specifically comprises: Using additional image motion data without ground truth and the parameterized model obtained by the current individual body type parameter as network input, judging whether the current synthesized result output by the network is a real image intensive view synthesis performance through the discriminator, and estimating the two-dimensional joint heat map of the corresponding synthesized image through the human pose estimation network with frozen weights, to project the two-dimensional joint image obtained by the corresponding input parameterized model as a pseudo ground truth for weakly supervised training.

8. The dynamic human body reconstruction method of claim 1, wherein, The step S4 specifically comprises: Given the human parameterized model under the new pose and replacing the body type parameter with the driven target body type parameter; inputting the adjusted human parameterized model into the bone action branch of the motion branch, and caching the position offset caused by the bone action; Densely sampling rays on the imaging plane corresponding to the given camera pose, querying the bone action and fine-tuning the action residual on each point on the ray through the action field fine-tuning branch; Adding the position offset caused by the bone action to the action residual to obtain the coordinates of each point in the specified rest pose space; Using the body density and color corresponding to the coordinates of each point in the rest space as an index to obtain the synthesized image under the given new pose and new view point through the volume rendering formula.

Images