Digital human real-time generation method and system fusing hierarchical optical transmission model
By combining a layered light transmission model and a neural network accelerator, high-precision, low-cost real-time skin rendering for digital humans was achieved, solving the problems of material distortion and dynamic response lag in existing technologies, and making it suitable for consumer-grade devices.
Patent Information
- Application Number
- CN202511625827.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-07
- Publication Date
- 2026-03-20
AI Technical Summary
Existing digital human modeling methods suffer from issues such as material distortion, sluggish dynamic response, and high hardware costs in skin rendering, making it difficult to meet the needs of consumer-grade devices.
A layered light transmission model is adopted. By collecting the optical parameters of the epidermis, dermis and subcutaneous tissue layers of the skin, a radiation flux equation is constructed. The skin material with dynamic reflectivity field is generated by the adaptive photon mapping algorithm and neural network accelerator, and then rendered in real time by the skeletal animation system.
It improves skin rendering accuracy, optimizes real-time performance, enhances dynamic adaptability, solves the problem of asynchronous skin texture and muscle movement, and reduces hardware costs.
Smart Images
Figure CN121708170A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of digital human modeling and real-time rendering technology, and in particular to a method and system for real-time generation of digital humans that integrates a layered light transmission model. Background Technology
[0002] Digital human technology, a cross-disciplinary integration of computer graphics, biomechanics, and artificial intelligence, has brought new opportunities and breakthroughs to numerous fields. In the film and gaming industry, high-precision digital humans can reduce character production costs by 67%. In the healthcare field, the patient-specific digital human models integrated into the da Vinci Surgical System enable pre-operative simulations with an accuracy exceeding 92%. In the field of metaverse interaction, researchers predict a significant increase in the global digital human market size by 2025. Furthermore, digital human technology has also driven breakthroughs in fundamental theories such as skin optical transmission, muscle dynamics simulation, and neural actuation.
[0003] There are currently several methods and principles for digital human modeling, but they all have some technical problems. For example, the 3D scanning reconstruction method uses a multi-camera array combined with laser scanning technology, but this method has high equipment costs, time-consuming data processing at the pore level, and is difficult to promote and apply. The parametric model generation method uses preset topology and slider adjustment to generate digital humans, but this method leads to a lack of biological realism and is not friendly to special physical characteristics, making it impossible to adapt to special physical characteristics. The neural radiation field method generates digital humans through multi-view neural network rendering, but this method not only has long single-model training time, but also low support for dynamic expressions, and the generated digital humans are too stiff to be directly applied. Overall, existing methods for generating digital humans have several shortcomings in skin rendering: For example, material distortion: the traditional BSSRDF (Bidirectional Scattering Surface Reflectance Distribution Function, a model used to describe multiple scatterings of light within an object) model simplifies the skin as a single layer, leading to confusion between the light transmission characteristics of the dermis (collagen fiber structure) and subcutaneous tissue (fat layer), resulting in a measured reflection spectrum error >25%; dynamic response lag: when facial expressions change, existing methods require more than 300ms to recalculate scattering parameters, causing a disconnect between skin texture and muscle movement (delay error >1.3mm); and excessively high hardware costs: film-grade skin rendering requires multiple RTX 6000 graphics cards (high cost and high power consumption) to achieve 30fps, which is insufficient for consumer-grade devices. Summary of the Invention
[0004] In order to solve the problems existing in the prior art, the present invention provides the following technical solution.
[0005] The first aspect of this invention provides a method for real-time generation of digital humans, comprising: S101, collect optical parameters of the epidermis, dermis and subcutaneous tissue of the skin, the optical parameters including extinction coefficient, scattering anisotropy factor and scattering coefficient; S102, construct a layered skin light transmission model, and establish radiation flux equations for the epidermis, dermis and subcutaneous tissue layers based on radiation transmission theory and interlayer coupling relationship; S103, based on the radiation flux equations of the epidermis, dermis and subcutaneous tissue layers, uses an adaptive photon mapping algorithm to perform parallel Monte Carlo sampling on the epidermis, dermis and subcutaneous tissue layers respectively to obtain the radiation transmission results of each layer; the Henyey-Greenstein phase function is used for the dermis layer and the Rayleigh scattering model is used for the subcutaneous tissue layer. S104 uses a neural network accelerator to fuse the radiative transmission results of each layer to generate a skin material containing a dynamic reflectivity field. S105 binds the skin material obtained in S104 to the skeletal animation system in real time and outputs the rendered digital human.
[0006] Step S101 includes: Optical parameters of the epidermis, dermis, and subcutaneous tissue layers were acquired using a three-band polarization multispectral imaging device. Specifically, the optical parameters of the epidermis were acquired using the visible light band; the optical parameters of the dermis were acquired using the near-infrared I band; and the optical parameters of the subcutaneous tissue layer were acquired using the near-infrared II band.
[0007] Preferably, in step S102, The radiation flux equation for the epidermis is: ; The radiation flux equation for the dermis is: ; The radiation flux equation for the subcutaneous tissue layer is: ; in , and , respectively, represent the radiation flux of the epidermis, dermis, and subcutaneous tissue; t represents time; , and These are the diffusion coefficients of the epidermis, dermis, and subcutaneous tissue layers, respectively. , and The values were calculated using the extinction coefficient and scattering anisotropy factor of the corresponding layers, respectively. , and These are the extinction coefficients of the epidermis, dermis, and subcutaneous tissue layers, respectively. This is the coupling coefficient between the epidermis and the dermis. , which is the interfacial transmittance between the epidermis and the dermis. pass and Calculated; , and These are the refractive indices of the epidermis, dermis, and subcutaneous tissue layers, respectively. This is the coupling coefficient between the dermis and the subcutaneous tissue layer. , The transmittance of the interface between the dermis and the subcutaneous tissue layer, through and Calculated; Let x be the photon flux source term of the epidermis, and x be the spatial coordinate.
[0008] Preferably, in S103, the adaptive photon mapping algorithm includes: The sampling density in the facial muscle attachment area was 64 samples / mm². Based on real-time facial motion capture of muscle deformation angles, the subcutaneous tissue sampling weight w is adjusted according to the following formula: , Where Δθ is the muscle deformation angle. , .
[0009] Preferably, in step S104, the neural network accelerator employs a multi-scale feature fusion architecture, including: The input layer is used to receive the radiation transmission results from each layer. The intermediate layer adopts a 3-level U-Net encoding and decoding structure, with each level containing 8 residual modules; The output layer is used to generate skin materials that include diffuse, specular, and subsurface scattering components.
[0010] A second aspect of the present invention provides a real-time digital human generation system, comprising: An optical parameter acquisition module is used to acquire optical parameters of the epidermis, dermis and subcutaneous tissue of the skin. The optical parameters include extinction coefficient, scattering anisotropy factor and scattering coefficient. The layered model construction module is used to construct a layered skin light transmission model and establish radiation flux equations for the epidermis, dermis and subcutaneous tissue layers based on radiation transmission theory and interlayer coupling relationship. The sampling module is used to perform parallel Monte Carlo sampling of the epidermis, dermis and subcutaneous tissue layers using an adaptive photon mapping algorithm to obtain the radiative transmission results of each layer. The rendering module is used to fuse the radiative transmission results of each layer through a neural network accelerator to generate a skin material containing a dynamic reflectivity field. The dynamic binding module is used to bind the skin material generated by the rendering module to the skeletal animation system in real time, and output the rendered digital human.
[0011] Preferably, the sampling module includes an adaptive photon mapping algorithm unit, used for: The sampling density in the facial muscle attachment area was 64 samples / mm². Based on real-time facial motion capture of muscle deformation angles, the subcutaneous tissue sampling weight w is adjusted according to the following formula: , Where Δθ is the muscle deformation angle. , .
[0012] Preferably, the optical parameter acquisition module integrates a three-band polarization multispectral imaging device to acquire optical parameters of the epidermis, dermis, and subcutaneous tissue layer, respectively.
[0013] The present invention also provides a memory storing a plurality of instructions for implementing the real-time digital human generation method as described in the first aspect.
[0014] The present invention also provides an electronic device, including a processor and a memory connected to the processor, the memory storing a plurality of instructions which can be loaded and executed by the processor to enable the processor to perform the real-time digital human generation method as described in Embodiment 1.
[0015] The beneficial effects of this invention are as follows: The technical solution provided by this invention deeply embeds the results of skin optical optimization into the digital human modeling process from optical parameter acquisition to rendering completion. Through the superposition of several technologies such as layered optical modeling, dynamic sampling optimization, and hardware accelerated rendering, not only is the skin rendering accuracy improved and the real-time performance optimized, but the dynamic adaptability is also greatly enhanced. This solves the technical problems of unrealistic skin, sluggish dynamic response, and asynchronous skin texture and muscle movement in existing technologies at the source of digital human generation. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating the real-time digital human generation method described in this invention.
[0017] Figure 2This is a functional structure diagram of the real-time digital human generation system described in this invention. Detailed Implementation
[0018] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.
[0019] The method provided by this invention can be implemented in a terminal environment that may include one or more of the following components: a processor, a memory, and a display screen. The memory stores at least one instruction, which is loaded and executed by the processor to implement the method described in the following embodiments.
[0020] A processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the terminal, and performs various functions and processes data by running or executing instructions, programs, code sets or instruction sets stored in memory, and by calling data stored in memory.
[0021] Memory can include random access memory (RAM) or read-only memory (ROM). Memory can be used to store instructions, programs, code, code sets, or instructions.
[0022] The display screen is used to show the user interface of each application.
[0023] In addition, those skilled in the art will understand that the structure of the terminal described above does not constitute a limitation on the terminal. The terminal may include more or fewer components, or combine certain components, or have different component arrangements. For example, the terminal may also include radio frequency circuits, input units, sensors, audio circuits, power supplies, and other components, which will not be described in detail here. Example 1
[0024] like Figure 1 As shown, this embodiment of the invention provides a method for real-time generation of digital humans, including: S101, collect optical parameters of the epidermis, dermis and subcutaneous tissue of the skin, the optical parameters including extinction coefficient, scattering anisotropy factor and scattering coefficient; S102, construct a layered skin light transmission model, and establish radiation flux equations for the epidermis, dermis and subcutaneous tissue layers based on radiation transmission theory and interlayer coupling relationship; S103, based on the radiation flux equations of the epidermis, dermis and subcutaneous tissue layers, uses an adaptive photon mapping algorithm to perform parallel Monte Carlo sampling on the epidermis, dermis and subcutaneous tissue layers respectively to obtain the radiation transmission results of each layer; the Henyey-Greenstein phase function is used for the dermis layer and the Rayleigh scattering model is used for the subcutaneous tissue layer. S104 uses a neural network accelerator to fuse the radiative transmission results of each layer to generate a skin material containing a dynamic reflectivity field. S105 binds the skin material obtained in S104 to the skeletal animation system in real time and outputs the rendered digital human.
[0025] In this embodiment of the invention, after acquiring the optical parameters of each layer of the skin in step S101, the skin can be divided into three independent media in step S102 based on the differences in optical properties of the epidermis, dermis, and subcutaneous tissue layers (which can be quantified by the parameters in step S101): the epidermis, the high-absorption ( The skin consists of a thin layer (0.05-1.5 mm); a dermis layer with strong forward scattering (g2≈0.8) and a vascular network; and a subcutaneous tissue layer with isotropic scattering (g≈0) and high refractive index (n3≈1.44). A layered skin light transmission model is established, and radiative flux equations for the epidermis, dermis, and subcutaneous tissue layers are established based on radiative transfer theory and interlayer coupling relationships. The photon tracing strategy in step S103 is implemented based on some parameters provided by the equations. Then, in step S104, the photon tracing results (radiative transfer results of each layer) are fused using a neural network accelerator to generate an optimized skin material. Finally, in step S105, the optimized skin material is bound to the skeletal animation system in real time to output the rendered digital human.
[0026] This technical solution achieves seamless integration of the entire process from optical parameter acquisition to real-time rendering through a closed-loop logic of parameter-driven modeling, physical equation construction, hardware-accelerated computation, and dynamic feedback optimization. The connections between each step are ensured by underlying technologies such as CUDA atomic operations (a mechanism that guarantees safe access to shared data by multiple threads in parallel computing) and double-buffered memory swapping, ultimately achieving an industry-breakthrough indicator of skin optical simulation error <5%.
[0027] In this embodiment of the invention, in step S101, the multispectral imaging system employs a tunable LED light source covering three bands. The visible light band (400-700 nm) is used for acquiring optical parameters of the epidermis; the near-infrared I band (701-1100 nm) can penetrate to the dermis for acquiring optical parameters of the dermis; and the near-infrared II band (1101-1500 nm) can penetrate to the subcutaneous tissue layer for acquiring optical parameters of the subcutaneous tissue layer. The imaging system's polarization module integrates linear polarizers (0°, 90°) and circular polarizers, which can separate specular reflection and diffuse reflection components. The multispectral camera is equipped with three scientific-grade CMOS sensors, each corresponding to one of the three bands, with a resolution of 2048×2048 and a frame rate of 60fps. The auxiliary sensor includes an optical coherence tomography (OCT) probe with an axial resolution of 5 μm, used for real-time measurement of skin layer thickness (epidermis 50-150 μm, dermis 1-3 mm, subcutaneous tissue layer variable). The auxiliary sensors also include a pressure sensor array: monitoring the probe contact pressure (controlled at 5-10 kPa) to avoid tissue deformation affecting the measurement. The acquisition of optical parameters of the epidermis, dermis, and subcutaneous tissue layers can be performed simultaneously in three bands.
[0028] The specific steps for acquiring optical parameters are as follows: The subject places their forearm on the measurement platform, and the OCT probe is automatically positioned. The system sequentially emits three-band polarized light and simultaneously acquires multispectral images; Real-time calculation of various optical parameters and display of 3D distribution map; When data is abnormal (e.g.) Exceeding (Range), triggering automatic retesting.
[0029] This invention achieves precise parameter separation of the epidermis, dermis, and subcutaneous tissues through the difference in penetration depth across three bands, enabling multi-layered parameter acquisition. Furthermore, during the measurement process, contact pressure control and temperature compensation ensure the reliability of in vivo measurements, achieving non-destructive in vivo measurement. Moreover, it allows for real-time output from data acquisition to parameter generation.
[0030] Using the method provided by this invention, the accuracy of skin optical parameter acquisition can be improved to medical-grade standards (error < 5%), providing reliable data input for high-fidelity digital human modeling.
[0031] In an embodiment of the present invention, in step S102, The radiation flux equation for the epidermis is: ; The radiation flux equation for the dermis is: ; The radiation flux equation for the subcutaneous tissue layer is: ; in , and , respectively, represent the radiation flux of the epidermis, dermis, and subcutaneous tissue; t represents time; , and These are the diffusion coefficients of the epidermis, dermis, and subcutaneous tissue layers, respectively. , and The values were calculated using the extinction coefficient and scattering anisotropy factor of the corresponding layers, respectively. , and These are the extinction coefficients of the epidermis, dermis, and subcutaneous tissue layers, respectively. This is the coupling coefficient between the epidermis and the dermis. , This refers to the interfacial transmittance between the epidermis and dermis. Calculated using n3; , and These are the refractive indices of the epidermis, dermis, and subcutaneous tissue layers, respectively. The coupling coefficient between the dermis and the subcutaneous tissue layer is determined by... and Calculated; Let x be the photon flux source term of the epidermis, and x be the spatial coordinate.
[0032] The radiation flux equations for the epidermis, dermis, and subcutaneous tissue layers can be established using the following method. In this invention, it should be noted that during the equation establishment process, it is necessary to consider how radiation flux is transferred from the upper layer to the lower layer in the epidermis, dermis, and subcutaneous tissue layers, and how the source term is transferred to the epidermis, thereby establishing interlayer coupling equations.
[0033] The radiative transfer equation is the fundamental equation describing the propagation of light in a medium. However, due to its complexity, it is usually simplified to the diffusion equation under the diffusion approximation. The diffusion equation assumes that after photons undergo multiple scatterings in the medium, the flux distribution tends to be isotropic, thus allowing it to be approximated using diffusion theory.
[0034] For each layer (epidermis, dermis, subcutaneous tissue), its radiation flux equation can be written separately.
[0035] In the epidermis, source terms The incident light comes from an external light source, so the equation for the epidermis is simply the standard diffusion equation plus the source term.
[0036] When light enters the dermis from the epidermis, the radiative flux of the epidermis... This will serve as a source term for the dermis. Similarly, the radiative flux of the dermis... It will serve as the source term for the subcutaneous tissue layer. Therefore, in the dermal radiation flux equation... and the radiation flux equation of subcutaneous tissue layer This demonstrates the coupling between layers.
[0037] In this embodiment of the invention, the coupling coefficient can be derived using boundary conditions. and Boundary conditions typically include flux continuity and current continuity.
[0038] At the interface between two dielectric layers, assuming there is no surface source, the boundary conditions typically require: 1. Flux continuity: The fluxes on both sides are equal, i.e. At the interface between the epidermis and the dermis.
[0039] 2. Current continuity: The current density (i.e., the derivative of the flux multiplied by the diffusion coefficient) is equal on both sides, i.e. , where n is the normal vector of the interface.
[0040] In this invention, these boundary conditions are determined by coupling coefficients. The flux of the upper layer is used as the source term of the lower layer, and the energy transfer between layers is introduced as the source term into the equation of the lower layer, instead of directly solving the boundary conditions at the interface, thus simplifying the solution of the boundary conditions.
[0041] Specifically This may include transmittance at the interface and geometric factors. For example, when light enters the dermis from the epidermis, due to the difference in refractive index, some light is reflected and some is transmitted. The transmitted portion can be used as a source term for the dermis, while the reflected portion may need to be considered back to the epidermis. In the model, the reflected portion is treated as a systematic error.
[0042] Similarly, This represents the transmittance coefficient from the dermis to the subcutaneous tissue layer. This requires calculating the transmittance T using Fresnel's formula based on the difference in refractive index between the two layers, and then determining it in conjunction with factors such as geometric diffusion. The value of .
[0043] Assuming the refractive index of the epidermis is The refractive index of the dermis is The refractive index of the subcutaneous tissue layer is Then the interfacial transmittance between the epidermis and the dermis for: , Similarly, the interfacial transmittance between the dermis and the subcutaneous tissue layer for: .
[0044] Then use calculate ,use calculate : , .
[0045] In an embodiment of the present invention, , , in, For the reduced scattering coefficient, The scattering coefficient is... This is the scattering anisotropy factor of the epidermal layer.
[0046] Similarly, the diffusion coefficients of other layers can be calculated.
[0047] In this embodiment of the invention, the radiation flux equation of the dermis layer includes ,in, It is the radiative flux of the epidermis. It is the coupling coefficient between the epidermis and the dermis, indicating that the source term of the dermis comes from the output of the epidermis.
[0048] However, the epidermis, as the outermost layer, has its source term The input source for this layer is the distribution of incident light from an external light source. Here, x represents the spatial coordinates; in three-dimensional space, it can be a vector (x, y, z); in two-dimensional space, it can be two-dimensional surface coordinates, depending on the model's dimensionality settings.
[0049] In this embodiment of the invention, in step S103, the adaptive photon mapping algorithm includes: The sampling density was automatically increased to 64 samples / mm² in the area where facial muscles were attached. The subcutaneous tissue sampling weights are dynamically adjusted based on real-time facial motion capture data.
[0050] Where Δθ is the muscle deformation angle. , .
[0051] In this embodiment, the Monte Carlo method is used to simulate light transmission, estimating illumination effects by randomly sampling photon paths. Adaptive photon mapping dynamically adjusts the sampling density based on certain conditions to improve efficiency or accuracy. For example, sampling is increased in complex regions (such as blood vessels or fibrous structures in the dermis of the skin) and decreased in relatively uniform regions.
[0052] In this invention, parallel sampling is performed due to the large computational cost of Monte Carlo sampling. Specifically, the CUDA (Compute Unified Device Architecture) framework is used to allocate thread blocks to process different photon paths.
[0053] During sampling, different scattering models were used for different skin layers. The Henyey-Greenstein phase function was used for the dermis. This phase function is commonly used in biological tissue scattering simulations, and its parameter g (scattering anisotropy factor) determines the directionality of the scattering. The dermis typically exhibits high forward scattering (g close to 1), so this parameter needs to be set correctly, and scattering directions conforming to this distribution need to be generated during sampling. The Rayleigh scattering model was used for the subcutaneous tissue layer. This model is suitable for scattering smaller particles, which is typically isotropic (g=0) or nearly isotropic. The phase function form of Rayleigh scattering differs from that of Henyey-Greenstein, requiring a different sampling method.
[0054] Adaptive photon mapping is used to dynamically adjust the sampling rate in regions with higher sampling density, and to influence computational resources and result accuracy through sampling rate adjustment. In this invention, adaptive sampling, switching between different scattering models, and ensuring proper coupling of light transmission between layers can be efficiently implemented in parallel computing, thereby maintaining computational efficiency while improving the physical accuracy of the rendering.
[0055] The radiation flux equations for the epidermis, dermis, and subcutaneous tissue layers constructed in step S102 provide a physical model and parameter constraints for Tecalo sampling, directly determining the sampling strategy and path calculation method. For example, the photon step size is determined by the total attenuation coefficient, which in turn is determined by the extinction coefficient and scattering coefficient. Furthermore, the anisotropy factor of the dermis directly determines the scattering direction distribution. Additionally, the radiation flux gradient can be calculated using the equation solutions, and then the sampling weight can be calculated using this gradient to determine different sampling densities in different gradient regions. For instance, the sampling density can be automatically increased in high-gradient regions (such as pores and wrinkles).
[0056] In this embodiment of the invention, the sampling rate can also be determined based on the optical parameters provided in step S102, for example, the sampling rate of the epidermis due to... High performance is achieved by using a low sampling rate (16 samples / pixel). The sampling direction weights of the dermis can be dynamically adjusted according to the g2 value, while the subcutaneous tissue layer, due to isotropic scattering (g≈0), can use uniform spherical sampling. Furthermore, GPU thread resources can be allocated based on the optical thickness of each layer.
[0057] In one embodiment of the present invention, in step S104, the neural network accelerator employs a multi-scale feature fusion architecture, including: The input layer is used to receive the radiation transmission results from each layer. The intermediate layer adopts a 3-level U-Net encoding and decoding structure, with each level containing 8 residual modules; The output layer is used to generate skin materials that include diffuse, specular, and subsurface scattering components.
[0058] In practical applications, the following methods can be used for implementation.
[0059] I. System Architecture and Hardware Configuration 1. The hardware platform includes: Inference accelerator: NVIDIA RTX 4090 GPU with 128 third-generation Tensor Cores; Memory optimization: 24GB GDDR6X memory, enabling NVIDIA RTX IO technology for zero-copy data transfer; Computation framework: TensorRT 8.6, with FP16 mixed precision and layer fusion optimization enabled.
[0060] 2. Data processing includes: Input data: Three-layer radiation flux field (epidermal layer) from step S103 dermis Subcutaneous tissue layer ), resolution 2048×2048, 16-bit floating-point format; preprocessing: normalization and channel splicing are performed through the CUDA kernel.
[0061] II. Neural Network Architecture Design An improved multi-scale U-Net is adopted, with the following specific structure: 1. The encoder consists of: Layer 1 (input: 2048×2048×3): 2×convolutional layers (3×3 kernels, 64 channels, ReLU); residual blocks (containing 8 residual units, channel count 64→128→256); max pooling (2×2, stride 2). Layer 2 (1024×1024×256): residual blocks (256→512 channels); dilated convolutions (dilation rate 2, preserving resolution); spatial attention module. Layer 3 (512×512×512): nonlocal neural network blocks (capturing long-range dependencies); channel attention (SENet structure).
[0062] 2. The decoder includes: Layer 3→2: transposed convolution (3×3 kernel, stride 2512→256 channels); skip connections to fuse features from encoder layer 2; dynamic convolution kernel (adjusted according to the μa1 parameter in step S101). Layer 2→1: transposed convolution (256→128 channels); introducing the inter-layer coupling coefficient β from step S102. 12 As conditional inputs, the output layer (2048×2048×6) is divided into three branches: diffuse layer: 3×3 convolutional units to Sigmoid activation units; specular layer: anisotropic filtering units to dynamic range compression units; and subsurface scattering layer: combined with the residuals of the radiation flux equation from step S102.
[0063] III. Training Strategies 1. Dataset Construction: Generate synthetic data by generating 500,000 sets of skin phototransmission data using MCML (Monte Carlo Multilayer Simulator); Real-world Data Acquisition: Collect 200 real skin samples using a hyperspectral camera, covering wavelengths from 400 to 1500 nm; Data Augmentation: Random deformation, illumination variation, and noise injection.
[0064] 2. Two-stage training optimization: First, pre-training is performed on synthetic data; then fine-tuning is performed on real test data; finally, dynamic weight adjustment is performed. Example 2
[0065] like Figure 2 As shown, another aspect of the present invention also includes a functional module architecture that is completely consistent with the aforementioned method flow. That is, the embodiments of the present invention also provide a real-time digital human generation system, including: The optical parameter acquisition module 201 is used to acquire optical parameters of the epidermis, dermis and subcutaneous tissue of the skin, including extinction coefficient, scattering anisotropy factor and scattering coefficient. The layered model construction module 202 is used to construct a layered skin light transmission model and establish radiation flux equations for the epidermis, dermis and subcutaneous tissue layers based on radiation transmission theory and interlayer coupling relationship. The sampling module 203 is used to perform parallel Monte Carlo sampling on the epidermis, dermis and subcutaneous tissue layers using an adaptive photon mapping algorithm to obtain the radiative transmission results of each layer. Rendering module 204 is used to fuse the radiation transmission results of each layer through a neural network accelerator to generate a skin material containing a dynamic reflectivity field. The dynamic binding module 205 is used to bind the skin material generated by the rendering module to the skeletal animation system in real time, and output the rendered digital human.
[0066] The sampling module includes an adaptive photon mapping algorithm unit, used for: The sampling density in the facial muscle attachment area was 64 samples / mm². Based on real-time facial motion capture of muscle deformation angles, the subcutaneous tissue sampling weight w is adjusted according to the following formula: , Where Δθ is the muscle deformation angle. , .
[0067] Furthermore, the optical parameter acquisition module integrates a three-band polarization multispectral imaging device to acquire optical parameters of the epidermis, dermis, and subcutaneous tissue layer, respectively.
[0068] This system can be implemented using the real-time digital human generation method provided in Embodiment 1 above. For the specific implementation method, please refer to the description in Embodiment 1, which will not be repeated here.
[0069] The present invention also provides a memory that stores multiple instructions for implementing the real-time digital human generation method as described in Embodiment 1.
[0070] The present invention also provides an electronic device, including a processor and a memory connected to the processor, the memory storing a plurality of instructions which can be loaded and executed by the processor to enable the processor to perform the real-time digital human generation method as described in Embodiment 1.
[0071] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.
Claims
1. A method for real-time generation of digital humans, characterized in that, include: S101, collect optical parameters of the epidermis, dermis and subcutaneous tissue of the skin, the optical parameters including extinction coefficient, scattering anisotropy factor and scattering coefficient; S102, construct a layered skin light transmission model, and establish radiation flux equations for the epidermis, dermis and subcutaneous tissue layers based on radiation transmission theory and interlayer coupling relationship; S103, based on the radiation flux equations of the epidermis, dermis and subcutaneous tissue layers, uses an adaptive photon mapping algorithm to perform parallel Monte Carlo sampling on the epidermis, dermis and subcutaneous tissue layers respectively to obtain the radiation transmission results of each layer; The Henyey-Greenstein phase function was used for the dermis, and the Rayleigh scattering model was used for the subcutaneous tissue layer. S104 uses a neural network accelerator to fuse the radiative transmission results of each layer to generate a skin material containing a dynamic reflectivity field. S105 binds the skin material obtained in S104 to the skeletal animation system in real time and outputs the rendered digital human.
2. The method for real-time generation of digital humans according to claim 1, characterized in that, Step S101 includes: Optical parameters of the epidermis, dermis, and subcutaneous tissue layers were acquired using a three-band polarization multispectral imaging device. Specifically, the optical parameters of the epidermis were acquired using the visible light band; the optical parameters of the dermis were acquired using the near-infrared I band; and the optical parameters of the subcutaneous tissue layer were acquired using the near-infrared II band.
3. The method for real-time generation of digital humans according to claim 1, characterized in that, In step S102, The radiation flux equation for the epidermis is: ; The radiation flux equation for the dermis is: ; The radiation flux equation for the subcutaneous tissue layer is: ; in , and , respectively, represent the radiation flux of the epidermis, dermis, and subcutaneous tissue; t represents time; , and These are the diffusion coefficients of the epidermis, dermis, and subcutaneous tissue layers, respectively. , and The values were calculated using the extinction coefficient and scattering anisotropy factor of the corresponding layers, respectively. , and These are the extinction coefficients of the epidermis, dermis, and subcutaneous tissue layers, respectively. This is the coupling coefficient between the epidermis and the dermis. , This refers to the interfacial transmittance between the epidermis and dermis. pass and Calculated; , and These are the refractive indices of the epidermis, dermis, and subcutaneous tissue layers, respectively. This is the coupling coefficient between the dermis and the subcutaneous tissue layer. , The transmittance of the interface between the dermis and the subcutaneous tissue layer, through and Calculated; Let x be the photon flux source term of the epidermis, and x be the spatial coordinate.
4. The method for real-time generation of digital humans according to claim 1, characterized in that, In S103, the adaptive photon mapping algorithm includes: The sampling density in the facial muscle attachment area was 64 samples / mm². Based on real-time facial motion capture of muscle deformation angles, the subcutaneous tissue sampling weight w is adjusted according to the following formula: ， Where Δθ is the muscle deformation angle. , .
5. The method for real-time generation of digital humans according to claim 1, characterized in that, In step S104, the neural network accelerator employs a multi-scale feature fusion architecture, including: The input layer is used to receive the radiation transmission results from each layer. The intermediate layer adopts a 3-level U-Net encoding and decoding structure, with each level containing 8 residual modules; The output layer is used to generate skin materials that include diffuse, specular, and subsurface scattering components.
6. A real-time digital human generation system, characterized in that, include: An optical parameter acquisition module is used to acquire optical parameters of the epidermis, dermis and subcutaneous tissue of the skin. The optical parameters include extinction coefficient, scattering anisotropy factor and scattering coefficient. The layered model construction module is used to construct a layered skin light transmission model and establish radiation flux equations for the epidermis, dermis and subcutaneous tissue layers based on radiation transmission theory and interlayer coupling relationship. The sampling module is used to perform parallel Monte Carlo sampling of the epidermis, dermis and subcutaneous tissue layers using an adaptive photon mapping algorithm to obtain the radiative transmission results of each layer. The rendering module is used to fuse the radiative transmission results of each layer through a neural network accelerator to generate a skin material containing a dynamic reflectivity field. The dynamic binding module is used to bind the skin material generated by the rendering module to the skeletal animation system in real time, and output the rendered digital human.
7. The real-time digital human generation system according to claim 6, characterized in that, The sampling module includes an adaptive photon mapping algorithm unit, used for: The sampling density in the facial muscle attachment area was 64 samples / mm². Based on real-time facial motion capture of muscle deformation angles, the subcutaneous tissue sampling weight w is adjusted according to the following formula: ， Where Δθ is the muscle deformation angle. , .
8. The real-time digital human generation system according to claim 6, characterized in that, The optical parameter acquisition module integrates a three-band polarization multispectral imaging device to acquire optical parameters of the epidermis, dermis, and subcutaneous tissue layer, respectively.
9. A memory, characterized in that, The system stores multiple instructions for implementing the real-time digital human generation method as described in any one of claims 1-5.
10. An electronic device, characterized in that, The device includes a processor and a memory connected to the processor, the memory storing a plurality of instructions that can be loaded and executed by the processor to enable the processor to perform the real-time digital human generation method as described in any one of claims 1-5.