Real-time stable 3D human body avatar driving method and system

Abstract

The invention discloses a real-time stable 3D human body avatar driving method and system, and relates to the technical field of human body avatar driving. EHPS output parameters are mapped to an input space of a target digital human model through standardization processing and a lightweight neural network by using a cross-model mapping module, so that different EHPS method outputs can be compatible with digital human modeling input, and the incompatibility between the EHPS output parameters and digital human driving input is solved; an incremental smoothing strategy is adopted, time stability is enhanced, frame-level jitter is eliminated, high-frequency fluctuation of parameters on a time sequence is reduced, differential processing is carried out on the parameters in a local time window, and the consistency and stability of digital human driving are remarkably improved; a linear hybrid deformation technology and a 3D Gaussian point cloud rendering technology are adopted, efficient and high-quality digital human animation generation is realized, a faster end-to-end reasoning speed can be realized, and a 3D human body avatar driving solution with good balance between generation precision and processing speed is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of human avatar driving, and particularly relates to a real-time stable 3D human avatar driving method and system. BACKGROUND

[0002] With the rapid development of virtual reality, augmented reality and other technologies, the demand for generating three-dimensional animatable digital humans and real-time driving from monocular videos is growing. The core goal of such technologies is to generate three-dimensional animatable digital humans from monocular videos captured by ordinary cameras and to drive the three-dimensional virtual image in real time, automatically recovering the three-dimensional pose and expression of the character. This process not only requires the system to have high reconstruction accuracy and animation naturalness, but also must meet the real-time and interactive application requirements of driving.

[0003] The current mainstream research route usually divides the task into two independent technical modules: one is the human pose and shape estimation (Expressive Human Pose and Shape Estimation, EHPS), and the other is the modeling and driving of animatable three-dimensional digital humans. The EHPS method estimates the structured parameters of the human body, such as joint angles, skeleton poses, body proportions, etc. from images or video sequences. There are a large number of research results in this field, which can be mainly divided into multi-stage optimization methods and end-to-end learning methods. The former, such as the Hand4Whole method, realizes high-precision estimation by gradually optimizing a parameterized human body model (such as SMPL or SMPL-X), but the computational overhead is large and it is difficult to apply in real time. The latter, such as the OSX, HyBrik, SMPLerX methods, uses deep neural networks to directly predict human body parameters from images, significantly improving the computational efficiency, but often ignores the temporal consistency between frames, resulting in obvious pose jumps and animation jitter problems in video processing.

[0004] In the field of digital human modeling and driving, traditional methods mostly use grid-based parametric human models to generate pose animation through skeleton driving and morphing functions. Such methods have clear structure and strong controllability, but have limitations in expressing dynamic details such as clothing wrinkles and facial micro-expressions. In recent years, to improve visual realism, modeling methods based on implicit representation have emerged, such as Neural Radiance Fields (NeRF) and three-dimensional Gaussian body representation. In particular, methods such as GaussianAvatar and GauHuman use Gaussian body modeling to achieve smooth animation through pose control, improving modeling accuracy and rendering efficiency, and becoming one of the mainstream trends in current digital human modeling. However, modeling methods based on implicit representation are not compatible with mainstream parametric models, making it difficult to directly reuse existing animation pipelines. Moreover, due to the lack of explicit motion constraints in implicit representation, the stability of time-series motion is poor. These methods usually consume a large amount of computational resources, making it difficult to deploy in real-time conversation scenarios and video synthesis tasks. SUMMARY

[0005] To address the shortcomings of the prior art, the present application proposes a real-time stable 3D human avatar driving method and system, which realizes a 3D human avatar driving solution with good balance between generation accuracy and processing speed.

[0006] The technical solution of the present application is as follows:

[0007] The present application provides a real-time stable 3D human avatar driving method, comprising the following steps:

[0008] Extracting human low-dimensional parameters from video, including shape parameters, facial expression parameters and pose parameters;

[0009] Obtaining optimal spatial displacement parameters from video;

[0010] Modeling digital human according to video to obtain target digital human model;

[0011] Standardizing the extracted human parameters and spatial displacement parameters, inputting the standardized human parameters and spatial displacement parameters into a multi-layer perception machine, and completing the spatial conversion of human parameters through a mapping function. The mapped human parameters are subjected to secondary normalization processing, and finally the standardized human parameters suitable for the target digital human model are output;

[0012] Differentially time-smoothing the standardized human parameters suitable for the target digital human model to obtain time-smoothed human parameters;

[0013] Driving and rendering the target digital human model based on the time-smoothed human parameters.

[0014] Further, the human low-dimensional parameter extraction from the video specifically comprises:

[0015] S1: Extracting continuous image frames F from the input monocular video stream t , t = 1, 2, 3…t', t is the time sequence order of the image frame in the monocular video stream, and t' is the number of image frames;

[0016] S2: Preprocessing the image frame with a resolution higher than the set resolution to obtain a low-resolution image I t , t = 1, 2, 3…t'; the preprocessing includes downsampling and normalization;

[0017] S3: Extracting human low-dimensional parameters from the low-resolution image I t using the EHPS method, including pose parameters θ, shape parameters β, and expression parameters ψ;

[0018] S4: Dynamic verification of the extracted human low-dimensional parameters to delete abnormal parameters and obtain the final human low-dimensional parameters.

[0019] Further, the optimal spatial displacement parameter obtained from the video specifically comprises:

[0020] Obtaining an SMPL-X template, the vertices in the SMPL-X template being V i , simultaneously extracting 2D key points K t from the low-resolution image I i using MMPose, then constructing an objective function based on the projection error between the vertices V i in the SMPL-X template and the 2D key points K i extracted by MMPose:

[0021]

[0022] Wherein, is the optimal spatial displacement parameter, T is the spatial displacement parameter, V i ∈R N×3 is the i-th vertex in the SMPL-X template, i is the number of the vertex, N is the number of the vertex, K i ∈R N×2 is the 2D key point corresponding to V i , π(.) is a projection function; (x, y, z) is the position of the vertex in three-dimensional space, f x and f y represent the focal length of the camera in the x-axis and y-axis respectively, p x and p y represent the coordinates of the principal point on the image plane;

[0023] Then the objective function is iteratively solved by the LM algorithm to obtain the optimal spatial displacement parameter

[0024] Furthermore, the multi-layer perceptron includes a 3-layer fully connected structure, through the mapping function Complete the spatial transformation of human body parameters, where is the human body parameter after mapping, f(.) represents the human body parameter space mapping function, and Φ represents the back propagation optimization weight matrix;

[0025] The multilayer perceptron is trained in an end-to-end manner, and its loss function is defined as:

[0026]

[0027] Among them, L map is the loss function, β target ,θ target , ψ target are the standard parameters of the target digital human model;

[0028] Auxiliary loss:

[0029]

[0030] Among them, L geo for auxiliary losses;

[0031] By jointly optimizing L map With L geo Train the multilayer perceptron.

[0032] Furthermore, the differentiated time smoothing process is specifically as follows: for the standardized posture parameters First convert the axis-angle representation to quaternion q t , detect the direction mutation by the dot product sign of the quaternion of adjacent image frames: if the dot product of the quaternion of adjacent image frames is less than zero, q t Then, the Savitzky-Golay filter with a set window length is applied to the quaternion sequence composed of the quaternions of all image frames, and the filtering result is converted back to the axis-angle representation to obtain the attitude parameters after time smoothing; for the shape parameters With expression parameters Directly use the Savitzky-Golay filter with the same window size.

[0033] A second aspect of the present invention provides a real-time and stable 3D human avatar driving system for implementing a real-time and stable 3D human avatar driving method, comprising a parameter extraction module, an optimal spatial displacement parameter solving module, a digital human modeling module, a cross-model mapping module, a temporal motion smoothing module, and a digital human driving and rendering module;

[0034] The parameter extraction module extracts the human low-dimensional parameters in SMPL-X format from the input video, including shape parameters beta, facial expression parameters psi and pose parameters theta;

[0035] The optimal spatial displacement parameter solving module is configured to obtain the optimal spatial displacement parameters from the video

[0036] The digital human modeling module is configured to model the digital human according to the video by using an ExAvatar method to obtain a target digital human model.

[0037] The cross-model mapping module is configured to perform standardization processing on the extracted human parameters theta, beta and psi and the spatial displacement parameters and map the standardized human parameters to the input space of the target digital human model by using a multi-layer perception machine to obtain standardized human parameters suitable for the target digital human model and

[0038] The time sequence motion smoothing module is configured to perform differential time smoothing processing on the standardized human parameters suitable for the target digital human model and to obtain human parameters after time smoothing processing and to ensure the consistency and stability of the digital human driving.

[0039] The digital human driving and rendering module drives and renders the target digital human model by using a linear blending deformation technology and a 3D Gaussian point cloud rendering technology based on the human parameters after time smoothing processing.

[0040] The third aspect of the present application provides an electronic device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to execute the steps of the real-time stable 3D human avatar driving method.

[0041] The fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by the processor to execute the steps of the real-time stable 3D human avatar driving method.

[0042] Compared with the prior art, the present application has the following advantages:

[0043] The technical scheme provided by the present application designs a unified framework, which combines the human pose and shape estimation (EHPS) technology and the 3D digital human modeling technology; uses a cross-model mapping module to map the EHPS output parameters to the input space of the target digital human model through standardization processing and a lightweight neural network, so that the outputs of different EHPS methods can be compatible with the input of digital human modeling, solving the incompatibility of EHPS output parameters and digital human modeling input; and adopts an incremental smoothing strategy in the time sequence motion smoothing module to enhance the time stability and eliminate frame-level jitter, reducing the high-frequency fluctuations of parameters in the time sequence, and differentiating the parameters in the local time window, which significantly improves the consistency and stability of the digital human driving; the 3D Gaussian point cloud rendering technology is adopted to realize efficient and high-quality digital human animation generation, and faster end-to-end inference speed can also be realized, realizing a 3D human avatar driving solution with good balance between generation accuracy and processing speed.

[0044] Compared with the prior art, the present application has better image quality and meets the real-time application requirements, and is a 3D human avatar driving solution with good balance between generation accuracy and processing speed. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 A flowchart of a real-time stable 3D human avatar driving method and system in an embodiment of the present application is shown in the figure.

[0046] Figure 2 A flowchart of spatial displacement estimation in an embodiment of the present application is shown in the figure.

[0047] Figure 3 An MLP diagram in an embodiment of the present application is shown in the figure.

[0048] Figure 4 A flowchart of time smoothing processing in an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0049] The present application will be described in detail below in combination with the drawings and embodiments.

[0050] The application provides a real-time stable 3D human avatar driving method and system. By seamlessly connecting the mesh parameters efficiently extracted by the EHPS method (such as SMPLer-X) and the digital human modeling method (such as ExAvatar), the posture, shape and expression parameters of the human body are quickly extracted from the monocular video, and the digital human model is driven to generate dynamic effects. Using the cross-model mapping technology, the incompatibility of the EHPS output parameters and the digital human modeling input is solved, and the dynamic consistency and stability are improved through time smoothing processing. At the same time, the 3D Gaussian point cloud rendering technology is adopted to realize efficient and high-quality digital human animation generation, which is suitable for real-time video application scenarios. A 3D human avatar driving solution with good balance between generation accuracy and processing speed is realized.

[0051] A real-time stable 3D human avatar driving method, as shown in Figure 1 , comprising the following steps:

[0052] Step 1: Extracting human low-dimensional parameters from video, including shape parameters β, facial expression parameters ψ and posture parameters θ;

[0053] Step 1.1: Extracting continuous image frames F t (t = 1, 2, 3…t') from the input monocular video stream, t is the time sequence order of the image frame in the monocular video stream, and t' is the number of image frames;

[0054] Step 1.2: Preprocessing the image frames with resolution higher than the set resolution to obtain low-resolution images I t (t = 1, 2, 3…t'); the preprocessing includes downsampling and normalization;

[0055] In this embodiment, for image frames with high resolution higher than 512x512, such as high-resolution images of 1920x1080, the bilinear interpolation method is used to downsample to 512x512 resolution, and then the RGB three channels are normalized to adapt to the input specification of the network;

[0056] Step 1.3: Extracting human low-dimensional parameters from low-resolution images I t using the EHPS method, including posture parameters θ (covering the rotation angles of body, hand and facial joints), shape parameters β (describing body characteristics) and expression parameters ψ (characterizing facial muscle movement); the EHPS method includes SMPLer-X or OSX, etc.;

[0057] In this embodiment, the SMPLer-X parameter extraction module is used to extract human low-dimensional parameters, and based on ViT (Vision Transformer) coding, the input image is first divided into 32x24 16x16 patches, and after linear embedding and adding position coding, it is spliced with learnable TaskToken, and is input into a multi-layer Transformer encoder to extract global features; then a lightweight Neck module is used to predict the bounding boxes of the hands and face, and ROIAlign is used to crop the region features, which are input into three regression heads after transposed convolution upsampling, and finally the 72-dimensional pose parameters θ, 10-dimensional shape parameters β and 10-dimensional expression parameters ψ of SMPL-X are directly predicted;

[0058] Step 1.4: Dynamically checking the extracted human low-dimensional parameters, deleting the abnormal parameters, and obtaining the final human low-dimensional parameters;

[0059] In this embodiment, after the extraction is completed, the parameters are dynamically checked, for example, whether the joint angle exceeds the physiological limit (such as the elbow joint bending more than 180°), and the abnormal data is removed to avoid the error accumulation of the subsequent module, and the effective parameters are temporarily stored in the buffer area for calling by the cross-model mapping module;

[0060] Step 2: As shown in Figure 2 , the optimal spatial displacement parameters

[0061] Specifically, an SMPL-X template is obtained, and the vertices in the SMPL-X template are V i , and at the same time, MMPose is used to extract 2D key points K t from a low-resolution image I i , then based on the projection error between the vertices V i in the SMPL-X template and the 2D key points K i extracted by MMPose, a target function is constructed, and the formula is as follows:

[0062]

[0063] Among them, is the optimal spatial displacement parameter, T is the spatial displacement parameter, V i ∈R N×3 is the i-th vertex in the SMPL-X template, i is the number of the vertex, N is the number of the vertex, K i ∈R N×2 is the 2D key point corresponding to V i , π(.) is a projection function defined by the camera intrinsic parameters (focal length f and principal point p); (x, y, z) is the position of the vertex in the three-dimensional space, f x and f yrespectively denote the focal length of the camera on the x-axis and y-axis, p x and p y denote the coordinates of the principal point on the image plane, which is usually approximated as the image center;

[0064] The objective function is solved iteratively by the LM (Levenberg-Marquardt) algorithm to obtain the optimal spatial displacement parameters Ensure that the spatial position of the digital human in the rendered scene is consistent with the viewing angle of the input video. This process is incrementally calculated within a sliding window and only requires 5 iterations to converge, with a single frame taking less than 2ms;

[0065] Step 3: Use the ExAvatar method to model the digital human according to the video, and obtain the target digital human model;

[0066] Step 4: Standardize the extracted body parameters θ, β, ψ and spatial displacement parameters to eliminate scale deviations caused by algorithm differences. Then, the standardized body parameters and spatial displacement parameters are input into a multi-layer perceptron to complete the spatial conversion of the body parameters through a mapping function. The mapped body parameters are subjected to secondary normalization to finally output the standardized body parameters that adapt to the target digital human model and

[0067] The standardization operation is as follows: the axis angle representation of the pose parameter θ is normalized to the range [-π, π], and the shape parameter β and the expression parameter ψ are adjusted to zero mean and unit variance through Z-score standardization;

[0068] The multi-layer perceptron is as shown in Figure 3 : it includes a 3-layer fully connected structure with hidden layer dimensions of 256, 128, and 64 respectively, and the activation function is LeakyReLU with a slope of 0.2. The spatial conversion of the body parameters is completed through a mapping function , where f(.) represents the body parameter space mapping function and Φ represents the backpropagation optimization weight matrix;

[0069] The multi-layer perceptron is trained in an end-to-end manner, and its loss function is defined as:

[0070]

[0071] where L map is the loss function, β target , θ target , ψ target are the standard parameters of the target digital human model, and are the true values from the data set;

[0072] To enhance the physical rationality of the mapping, a geometric consistency constraint is introduced as an auxiliary loss to calculate the reprojection error between the SMPL-X template vertex mapping and the 2D key points. The formula is as follows:

[0073]

[0074] Among them, L geo is the auxiliary loss; by jointly optimizing L map With L geo (minimized after summation), the present invention ensures the natural presentation of details such as hand postures and facial expressions in the target model while preserving the semantics of the parameters.

[0075] Step 5: Standardize the human body parameters of the target digital human model and Perform differentiated time smoothing to improve the visual coherence of dynamic digital humans and obtain human body parameters after time smoothing. and

[0076] As attached Figure 4 Shown: For standardized posture parameters First convert the axis-angle representation to quaternion q t , detect the direction mutation by the dot product sign of the quaternion of adjacent image frames: if the dot product of the quaternion of adjacent image frames is less than zero (i.e. q t-1 .q t <0), q t The sign flip is negated to eliminate the discontinuity caused by the sign flip. Subsequently, a Savitzky-Golay filter (polynomial order 2) with a window length of 9 is applied to the quaternion sequence composed of the quaternions of all image frames to suppress high-frequency jitter while retaining the motion trend. The filtered result is converted back to the axis-angle representation to obtain the attitude parameters after time smoothing. For the shape parameters With expression parameters Directly use the Savitzky-Golay filter with the same window size to avoid sudden changes in body shape and facial expressions.

[0077] Step 6: Based on the human body parameters after temporal smoothing, the target digital human model is driven and rendered using linear blending deformation technology and 3D Gaussian point cloud rendering technology;

[0078] Specifically: Based on the linear blending deformation technology, the joint rotation is combined with the mesh vertex deformation to generate a three-dimensional mesh representation consistent with human body movement. The formula is as follows:

[0079]

[0080] Where V∈R N×3denotes vertices in the SMPL-X template, W is a linear blend shape model, T p is a pose transformation function, is a joint position computation function, denotes the initial joint position computed from the shape parameters , ω ∈ R K×3 is a weight matrix of vertices and joints, K is the number of vertices in the SMPL-X template, N = K = 10475, i.e., the number of vertices of the SMPL-X template;

[0081] Subsequently, based on a 3D Gaussian point cloud rendering technology, the mesh is converted into a sparse Gaussian point distribution and rendered in screen space, thereby completing the 3D digital human driving task.

[0082] The embodiment also provides a real-time stable 3D digital human driving system for implementing a real-time stable 3D digital human driving method, comprising a parameter extraction module, an optimal spatial displacement parameter solving module, a digital human modeling module, a cross-model mapping module, a time sequence motion smoothing module, and a digital human driving and rendering module.

[0083] The parameter extraction module extracts low-dimensional human body parameters in SMPL-X format from an input video by using an EHPS method such as SMPLer-X or OSX, and the low-dimensional human body parameters comprise shape parameters β, facial expression parameters ψ, and pose parameters θ.

[0084] The optimal spatial displacement parameter solving module is configured to obtain optimal spatial displacement parameters

[0085] The digital human modeling module is configured to model a digital human by using an ExAvatar method according to a video, and obtain a target digital human model.

[0086] The cross-model mapping module is configured to perform standardization processing on the extracted human body parameters θ, β, and ψ and the spatial displacement parameters , and map the standardized human body parameters to an input space of the target digital human model by using a multi-layer perception machine, to obtain standardized human body parameters and

[0087] The time sequence motion smoothing module is configured to perform differential time smoothing processing on the standardized human body parameters and in a local time window, to obtain human body parameters and after time smoothing processing, thereby ensuring consistency and stability of digital human driving.

[0088] The digital human driving and rendering module drives and renders the target digital human model based on the human body parameters after time smoothing processing by using linear blend shape (LBS) technology and 3D Gaussian point cloud rendering (3DGS) technology.

[0089] The embodiment further provides an electronic device, comprising a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of the real-time stable 3D human avatar driving method.

[0090] The embodiment further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of the real-time stable 3D human avatar driving method.

[0091] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present application.

Claims

1. A real-time and stable 3D human avatar driving method, characterized in that: The following steps are involved: Extract low-dimensional parameters of the human body from the video, including shape parameters, facial expression parameters and posture parameters; Obtain optimal spatial displacement parameters from the video; Perform digital human modeling based on the video to obtain the target digital human model; The extracted human body parameters and spatial displacement parameters are standardized and input into the multi-layer perceptron. The spatial transformation of the human body parameters is completed through the mapping function. The mapped human body parameters are subjected to secondary normalization and finally the standardized human body parameters that are adapted to the target digital human model are output. Performing differentiated time smoothing processing on the standardized human body parameters of the adapted target digital human model to obtain the human body parameters after time smoothing processing; Based on the human body parameters after time smoothing, the target digital human model is driven and rendered.

2. The real-time and stable 3D human avatar driving method according to claim 1, characterized in that: The step of extracting low-dimensional parameters of the human body from the video specifically includes: S1: Extract continuous image frames F from the input monocular video stream t , t=1,2,3…t', t is the temporal sequence of the image frames in the monocular video stream, and t' is the number of image frames; S2: Preprocess the image frame with a resolution higher than the set resolution to obtain a low-resolution image I t , t=1,2,3…t'; the preprocessing includes downsampling and normalization; S3: Using EHPS method to obtain low-resolution image I t Extract low-dimensional parameters of the human body, including posture parameters θ, shape parameters β and expression parameters ψ; S4: Dynamically verify the extracted low-dimensional parameters of the human body, delete the abnormal parameters, and obtain the final low-dimensional parameters of the human body.

3. The real-time and stable 3D human avatar driving method according to claim 1, characterized in that: The optimal spatial displacement parameters are obtained from the video, specifically: Get the SMPL-X template, the vertex in the SMPL-X template is V i , and use MMPose to extract the low-resolution image I t Extract 2D key points K from i , then based on the vertex V in the SMPL-X template i With the 2D key points K extracted by MMPose i The projection error constructs the objective function: in, is the optimal spatial displacement parameter, T is the spatial displacement parameter, V i ∈R N×3 is the i-th vertex in the SMPL-X template, i is the vertex number, N is the number of vertices, K i ∈R N×2 For V i The corresponding 2D key point, π(.) is the projection function; (x, y, z) is the position of the vertex in three-dimensional space, f x and f y Represents the focal length of the camera on the x-axis and y-axis, p x and p y represents the coordinates of the principal point on the image plane; Then the objective function is iteratively solved by the LM algorithm to obtain the optimal spatial displacement parameter 4. The real-time stable 3D human avatar driving method according to claim 1, characterized in that: The multi-layer perceptron includes a 3-layer fully connected structure, through the mapping function Complete the spatial transformation of human body parameters, where is the human body parameter after mapping, f(.) represents the human body parameter space mapping function, and Φ represents the back propagation optimization weight matrix; The multilayer perceptron is trained in an end-to-end manner, and its loss function is defined as: Among them, L map is the loss function, β target ,θ target , ψ target are the standard parameters of the target digital human model; Auxiliary loss: Among them, L geo for auxiliary losses; By jointly optimizing L map With L geo Train the multilayer perceptron.

5. The real-time and stable 3D human avatar driving method according to claim 1, characterized in that: The differentiated time smoothing process is specifically as follows: for the standardized posture parameters First convert the axis-angle representation to quaternion q t , detect the direction mutation by the dot product sign of the quaternion of adjacent image frames: if the dot product of the quaternion of adjacent image frames is less than zero, q t Then, the Savitzky-Golay filter with a set window length is applied to the quaternion sequence composed of the quaternions of all image frames, and the filtering result is converted back to the axis-angle representation to obtain the attitude parameters after time smoothing; for the shape parameters With expression parameters Directly use the Savitzky-Golay filter with the same window size.

6. A real-time stable 3D human avatar driving system, used to implement the real-time stable 3D human avatar driving method according to any one of claims 1 to 5, characterized in that: It includes parameter extraction module, optimal spatial displacement parameter solution module, digital human modeling module, cross-model mapping module, temporal motion smoothing module and digital human driving and rendering module; The parameter extraction module extracts low-dimensional parameters of the human body in SMPL-X format from the input video, including shape parameters β, facial expression parameters ψ and posture parameters θ; The optimal spatial displacement parameter solving module is used to obtain the optimal spatial displacement parameters from the video The digital human modeling module is used to perform digital human modeling based on the video using the ExAvatar method to obtain a target digital human model; The cross-model mapping module extracts the human body parameters θ, β, ψ and spatial displacement parameters Perform standardization processing and use a multi-layer perceptron to map the standardized human body parameters to the input space of the target digital human model to obtain the standardized human body parameters that are adapted to the target digital human model. and The temporal motion smoothing module is used to adapt the standardized human body parameters of the target digital human model within a local time window. and Perform differentiated time smoothing to obtain the human body parameters after time smoothing and Ensure the consistency and stability of digital human drivers; The digital human driving and rendering module drives and renders the target digital human model based on the human body parameters after time smoothing using linear blending deformation technology and 3D Gaussian point cloud rendering technology.

7. An electronic device, characterized in that: include: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the real-time and stable 3D human avatar driving method according to any one of claims 1 to 5 are performed.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the real-time and stable 3D human avatar driving method according to any one of claims 1 to 5.

Images