Digital human rendering method and device, and XR device

Abstract

The application provides a digital person rendering method and device and XR equipment, and relates to the technical field of computer vision, and the method comprises the following steps: in a Unity engine, rendering an opaque part of a to-be-rendered digital person model through a basic forward rendering channel to obtain a basic rendering image; rendering a skin part of the to-be-rendered digital person model through a skin forward rendering channel to obtain a skin color; rendering a hair part of the to-be-rendered digital person model through a hair forward rendering channel to obtain a hair color; and mixing the skin color and the hair color with the basic rendering image to obtain a target rendering digital person model. The application can achieve high-fidelity effects of simulating skin and hair while adapting to the computing capacity of performance-limited platforms such as mobile platforms.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a digital human rendering method, apparatus, and XR device. Background Technology

[0002] In recent years, digital human technology has been increasingly widely used in virtual reality, augmented reality, film and television production, gaming, and human-computer interaction, and the requirements for its visual realism have reached unprecedented levels. However, existing digital human rendering technology, especially in the rendering of physical realism in key areas such as skin and hair, still faces significant technical bottlenecks, which restrict the popularization and application of high-quality, cross-platform digital humans.

[0003] To address the aforementioned lack of realism, deferred rendering solutions have been developed. However, these solutions have extremely high hardware performance requirements, limiting their operation to desktop environments and making them unsuitable for mobile platforms with power and computing limitations (such as smartphones, tablets, and XR (Extended Reality) devices). Consequently, current mobile digital human rendering solutions often suffer from oversimplification, sacrificing realism and resulting in a significant decrease in rendering quality compared to desktop versions.

[0004] Therefore, there is an urgent need for an innovative rendering method that can achieve high-fidelity simulation of skin and hair while adapting to the computing power limitations of platforms such as mobile platforms. Summary of the Invention

[0005] This invention provides a digital human rendering method, apparatus, and XR device, which can achieve high-fidelity simulation of skin and hair while adapting to the computing power of performance-limited platforms such as mobile platforms.

[0006] This invention provides a digital human rendering method, comprising:

[0007] In the Unity engine, the opaque parts of the digital human model to be rendered are rendered through the basic forward pass to obtain the basic rendered image.

[0008] The skin color is obtained by rendering the skin portion of the digital human model to be rendered through the skin forward rendering pass;

[0009] The hair portion of the digital human model to be rendered is rendered using the forward hair rendering pass to obtain the hair color;

[0010] The skin color and hair color are mixed with the base rendering image to obtain the target rendered digital human model.

[0011] According to a digital human rendering method provided by the present invention, the step of rendering the skin portion of the digital human model to be rendered through the skin forward rendering channel to obtain the skin color includes:

[0012] The normal map is separated by the skin forward rendering channel to calculate the dot product of the normal and the lighting direction.

[0013] Based on the curvature and the dot product of the normal and the illumination direction, the pre-integrated skin shading map is sampled to obtain the diffuse color of the skin;

[0014] Using a dual-highlight model, secondary highlight simulation is performed based on the surface roughness lobe wave coefficient and the deep roughness lobe wave coefficient to obtain the skin highlight color;

[0015] The skin color includes the diffuse color of the skin and the highlight color of the skin.

[0016] According to a digital human rendering method provided by the present invention, the step of sampling a pre-integrated skin shading map based on the curvature and the dot product of the normal and the illumination direction to obtain the diffuse color of the skin includes:

[0017] Based on the curvature and the dot product of the normal and the illumination direction, the pre-integrated skin shading map is sampled to obtain the basic diffuse color of the skin;

[0018] The skin's basic diffuse color is obtained by performing horizontal and vertical blurring on the separable Gaussian blur algorithm.

[0019] According to a digital human rendering method provided by the present invention, the step of rendering the hair portion of the digital human model to be rendered through the hair forward rendering channel to obtain the hair color includes:

[0020] Using the hair forward rendering channel, the lateral scattering distribution of multiple optical paths is calculated using the HairGeo hair geometry simplification model; wherein, the multiple optical paths include the reflection R path, the transmission-transmission TT path, and the transmission-reflection-transmission TRT path;

[0021] The longitudinal scattering distribution of multiple optical paths was calculated using a Gaussian model.

[0022] The colors of the multiple optical paths are calculated based on the lateral scattering distribution and the longitudinal scattering distribution of the multiple optical paths.

[0023] The hair color is calculated based on the colors of the multiple optical paths.

[0024] According to a digital human rendering method provided by the present invention, the longitudinal scattering distribution of the multiple optical paths includes the longitudinal scattering distribution of the R path, the longitudinal scattering distribution of the TT path, and the longitudinal scattering distribution of the TRT path, and the lateral scattering distribution of the multiple optical paths includes the lateral scattering distribution of the R path, the lateral scattering distribution of the TT path, and the lateral scattering distribution of the TRT path.

[0025] The calculation of the colors of the multiple optical paths based on their lateral scattering distribution and longitudinal scattering distribution includes:

[0026] Using Chiang's approximate inverse model, the energy absorption of the TT path and the TRT path were calculated;

[0027] The color of the R path is calculated based on the base hair color, the lateral scattering distribution of the R path, and the longitudinal scattering distribution of the R path.

[0028] The color of the TT path is calculated based on the base hair color, the lateral scattering distribution of the TT path, the longitudinal scattering distribution of the TT path, and the energy absorption of the TT path.

[0029] The TRT path color is calculated based on the hair base color, the TRT path lateral scattering distribution, the TRT path longitudinal scattering distribution, and the TRT path energy absorption.

[0030] The colors of the multiple optical paths include the R path color, the TT path color, and the TRT path color.

[0031] According to a digital human rendering method provided by the present invention, before mixing the skin color and hair color with the base rendering image to obtain the target rendered digital human model, the method further includes:

[0032] The eyeball portion of the digital human model to be rendered is rendered using the forward eyeball rendering channel to obtain the eyeball color;

[0033] The step of mixing the skin color and hair color with the base rendered image to obtain the target rendered digital human model includes:

[0034] The skin color, hair color, and eye color are mixed with the base rendering image to obtain the target rendered digital human model.

[0035] According to a digital human rendering method provided by the present invention, the step of rendering the eye portion of the digital human model to be rendered through the eyeball forward rendering channel to obtain the eyeball color includes:

[0036] The illumination components corresponding to the iris layer of the eyeball are calculated through the forward rendering channel of the eyeball, and the basic color of the iris is synthesized based on the illumination components; wherein, the illumination components include diffuse reflection components, subsurface scattering components and specular components;

[0037] The iris base color is modulated by applying a sub-surface ambient light occlusion map to obtain the iris layer color;

[0038] Obtain the reflected and refracted colors of the corneal layer of the eyeball and calculate the Fresnel reflectance coefficient;

[0039] Based on the Fresnel reflectance coefficient, the reflected color and the refracted color are mixed to obtain the corneal color;

[0040] The color of the eyeball includes the color of the iris and the color of the cornea.

[0041] According to a digital human rendering method provided by the present invention, the step of mixing the skin color, the hair color, and the eye color with the base rendering image to obtain a target rendered digital human model includes:

[0042] The base diffuse color of the skin is mixed with the base rendered image to obtain the first intermediate rendered image;

[0043] The iris color and the corneal color are sequentially overlaid onto the first intermediate rendering image to obtain the second intermediate rendering image;

[0044] Based on the scalp mask, the hair color is mixed with the second intermediate rendering image to obtain a third intermediate rendering image;

[0045] The diffuse color of the skin is mixed with the third intermediate rendered image to obtain the fourth intermediate rendered image;

[0046] The skin highlight color is mixed with the fourth intermediate rendering image to obtain the target rendered digital human model.

[0047] The present invention also provides a digital human rendering apparatus, comprising:

[0048] The basic rendering module is used in the Unity engine to render the opaque parts of the digital human model to be rendered through the basic forward rendering pass, and obtain the basic rendering image.

[0049] The skin rendering module is used to render the skin portion of the digital human model to be rendered through the skin forward rendering channel to obtain the skin color;

[0050] The hair rendering module is used to render the hair portion of the digital human model to be rendered through the forward hair rendering channel to obtain the hair color;

[0051] The blending module is used to blend the skin color and hair color with the base rendering image to obtain the target rendered digital human model.

[0052] The present invention also provides an XR device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the digital human rendering method as described above.

[0053] The digital human rendering method, apparatus, and XR device provided by this invention, within the Unity engine, render the opaque parts of the digital human model using a basic forward rendering channel to obtain a basic rendered image. Then, a dedicated skin forward rendering channel renders the skin color, and simultaneously, a dedicated hair forward rendering channel renders the hair color. By creating independent skin and hair forward rendering channels, relatively complex shading models designed specifically for the corresponding material physical properties can be integrated and run in these two channels respectively. This significantly improves the visual realism of the skin and hair in the digital human rendering without placing an excessive burden on overall performance, achieving a high-fidelity effect. Furthermore, the dedicated channels limit the rendering process to either the skin or hair of the digital human model, greatly reducing computational overhead. Finally, the skin and hair colors are blended with the basic rendered image to obtain the final target rendered digital human model. In summary, this invention, through the aforementioned multi-channel rendering and blending design, enables real-time rendering of digital human models with high-fidelity skin and hair details even on performance-constrained platforms. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0055] Figure 1 This is a system architecture diagram of the digital human rendering system provided by the present invention;

[0056] Figure 2 This is one of the flowcharts illustrating the digital human rendering method provided by this invention;

[0057] Figure 3 This is the second flowchart illustrating the digital human rendering method provided by this invention;

[0058] Figure 4 This is the third flowchart of the digital human rendering method provided by the present invention;

[0059] Figure 5 This is the fourth flowchart of the digital human rendering method provided by the present invention;

[0060] Figure 6 This is a schematic diagram of the structure of the digital human rendering device provided by the present invention;

[0061] Figure 7 This is a schematic diagram of the structure of the XR device provided by the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0063] This invention proposes a digital human rendering method, apparatus, and XR device, which are described below in conjunction with... Figures 1-7 Describe it.

[0064] Figure 1 This is a system architecture diagram of the digital human rendering system provided by the present invention, such as... Figure 1 As shown, the digital human rendering system may include terminal devices 101, 102, and 103 and server 104. Terminal devices 101, 102, and 103 and server 104 can be connected via a network, such as a wired or wireless communication link or fiber optic cable.

[0065] Users can interact with server 104 using terminal devices 101, 102, and 103 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to XR devices, smartphones, tablets, laptops, desktop computers, workstations, etc. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services) or as a single software program or software module. No specific limitations are made here.

[0066] Server 104 can be a server that provides various services, such as a backend server that provides support for terminal devices 101, 102, and 103.

[0067] It should be noted that the digital human rendering method provided in this application embodiment can be executed by server 104 or terminal devices 101, 102, 103. Correspondingly, the digital human rendering device can be set in server 104 or terminal devices 101, 102, 103.

[0068] Figure 2 This is one of the flowcharts illustrating the digital human rendering method provided by this invention, such as... Figure 2 As shown, the digital human rendering method includes steps S110, S120, S130 and S140.

[0069] Step S110: In the Unity engine, the opaque parts of the digital human model to be rendered are rendered through the basic forward rendering pass to obtain the basic rendering image.

[0070] In this embodiment, the digital human rendering method is applied to a digital human rendering device. The digital human rendering device may include, but is not limited to: desktop fixed terminals such as desktop computers and workstations, mobile terminals such as smartphones, tablets, and laptops, and XR (Extended Reality) devices.

[0071] XR devices refer to wearable or portable devices that integrate virtual and real environments through hardware and software technologies to achieve human-computer interaction. These can include, but are not limited to, VR (Virtual Reality) devices, AR (Augmented Reality) devices, and MR (Mixed Reality) devices. VR devices use computer technology to simulate and generate a three-dimensional virtual space, allowing users to immerse themselves in and interact with it, gaining a truly immersive experience. AR devices use technology to merge virtual information with the real world, overlaying it onto real scenes in real time to enhance sensory experience. MR devices mix the real and virtual worlds to create a new visual environment that simultaneously contains physical entities and virtual information, allowing users to interact with these physical entities and virtual information in real time.

[0072] The digital human rendering method of the present invention is applicable to application environments with extremely high requirements for high-precision facial / hair rendering, such as virtual characters, virtual anchors, virtual customer service, and XR scenes.

[0073] In the Unity engine, based on Unity's Forward Rendering Pipeline, custom shaders with multiple passes are written for the base material, skin material, and hair material. Specifically, each pass includes at least a base forward rendering pass, a skin forward rendering pass, and a hair forward rendering pass. The base forward rendering pass is used to render the opaque parts of the digitizer, the skin forward rendering pass is used to render the skin, and the hair forward rendering pass is used to render the hair.

[0074] The rendering framework described above avoids the high bandwidth and GPU (Graphics Processing Unit) performance requirements of deferred rendering, making it particularly suitable for projects that need to be deployed on mobile platforms and low-end desktop hardware. It can achieve more stable and better performance while ensuring key visual effects.

[0075] Meanwhile, the above rendering framework is scalable, and by adjusting the complexity of the shading model within the dedicated channel, a balance can be easily struck between rendering quality and performance, making it flexible and adaptable to terminal devices with different performance levels.

[0076] Furthermore, the aforementioned multi-channel design allows the development, debugging, and optimization of skin, hair, and base materials to be carried out independently without interference. This enables the parameters of a single channel to be adjusted without affecting the effects of other channels, thereby greatly improving the efficiency of development iterations and the maintainability of the project.

[0077] The opaque parts of the digital human model to be rendered include, but are not limited to, clothing, accessories, and eyes. The opaque parts of the digital human model are rendered using the basic forward rendering pass to obtain the basic rendered image.

[0078] When rendering opaque parts, you can first obtain PBR (Physically Based Rendering) material parameters. PBR material parameters include at least one of the following: base color, metallicity, roughness, specular reflection coefficient, ambient occlusion coefficient, base normal information, and detail normal information. Among them, base color represents the color of the object itself, metallicity describes the metallic properties of the material, roughness describes the roughness of the surface, specular reflection coefficient describes the basic reflection intensity of non-metallic materials, ambient occlusion coefficient describes the depth of the material, base normal information describes the large structure and medium-frequency bump details of the model, and detail normal information describes the microscopic details of the model (such as pores, fabric fibers, etc.).

[0079] Then, using the PBR lighting model, basic PBR rendering is performed based on the PBR material parameters to obtain the basic rendered image.

[0080] Step S120: Render the skin portion of the digital human model to be rendered through the skin forward rendering channel to obtain the skin color.

[0081] The skin portion of the model is rendered using the skin forward rendering pass to obtain the skin color. The specific process for obtaining the skin color is as follows:

[0082] In one embodiment, the normal map is channel-separated using the skin forward rendering channel to calculate the dot product of the normal and the lighting direction. Based on the curvature and the dot product of the normal and the lighting direction, the pre-integrated skin shading map is sampled to obtain the diffuse skin color. Using a dual specular model, a secondary specular simulation is performed based on the surface roughness lobe coefficient and the deep roughness lobe coefficient to obtain the skin specular color. The skin color includes both the diffuse skin color and the specular skin color. This embodiment is suitable for low-performance devices; the specific execution process can be referred to in the following embodiments, which will not be elaborated here.

[0083] In another embodiment, based on sampling the pre-integrated skin shading map, separable subsurface scattering is further superimposed to obtain the diffuse skin color. Specifically, the normal map is channel-separated through the skin forward rendering channel to calculate the dot product of the normal and the illumination direction; the pre-integrated skin shading map is sampled based on the curvature and the dot product of the normal and the illumination direction to obtain the basic diffuse skin color; the basic diffuse skin color is then horizontally and vertically blurred using a separable Gaussian blur algorithm to obtain the diffuse skin color. A dual specular model is used to perform secondary specular simulation based on the surface roughness lobe wave coefficient and the deep roughness lobe wave coefficient to obtain the specular skin color; wherein, the skin color includes the diffuse skin color and the specular skin color. This embodiment is suitable for high-performance devices; the specific execution process can be referred to the following embodiments, which will not be elaborated here.

[0084] Step S130: Render the hair portion of the digital human model to be rendered using the hair forward rendering channel to obtain the hair color.

[0085] The hair portion of the model is rendered using the forward rendering pass to obtain the hair color. The specific process for obtaining the hair color is as follows:

[0086] In one embodiment, the longitudinal scattering distribution of multiple optical paths is calculated using a Gaussian model through the hair forward rendering channel; the lateral scattering distribution of multiple optical paths is calculated using the HairGeo hair geometry simplification model; the energy absorption of the TT path and the energy absorption of the TRT path are obtained through a table fitting query method; the colors of multiple optical paths are calculated based on the energy absorption of the TT path, the energy absorption of the TRT path, the longitudinal scattering distribution of multiple optical paths, and the lateral scattering distribution of multiple optical paths; the colors of multiple optical paths are summed to obtain the hair color. The multiple optical paths include the R (reflection) path, the TT (transmission-transmission) path, and the TRT (transmission-reflection-transmission) path.

[0087] In another embodiment, the longitudinal scattering distribution of multiple optical paths is calculated using a Gaussian model through the hair forward rendering channel; the lateral scattering distribution of multiple optical paths is calculated using the HairGeo simplified hair geometry model; the energy absorption of the TT path and the TRT path is calculated using the Chiang approximate inverse model; the colors of multiple optical paths are calculated based on the energy absorption of the TT path, the energy absorption of the TRT path, the longitudinal scattering distribution of multiple optical paths, and the lateral scattering distribution of multiple optical paths; the colors of multiple optical paths are summed to obtain the hair color. The multiple optical paths include the reflection R path, the transmission-transmission TT path, and the transmission-reflection-transmission TRT path.

[0088] Step S140: Mix the skin color and hair color with the base rendering image to obtain the target rendering digital human model.

[0089] The skin and hair colors are blended with the base rendered image to obtain the target rendered digital human model.

[0090] In one embodiment, skin color is mixed with a base rendered image to obtain a first intermediate rendered image, and hair color is mixed with the first intermediate rendered image to obtain a target rendered digital human model.

[0091] In another embodiment, the base diffuse color of the skin is mixed with the base rendered image to obtain a first intermediate rendered image; then, based on the scalp mask, the hair color is mixed with the first intermediate rendered image to obtain a second intermediate rendered image; and finally, the skin highlight color is mixed with the second intermediate rendered image to obtain the target rendered digital human model.

[0092] In another embodiment, the base diffuse color of the skin is mixed with the base rendered image to obtain a first intermediate rendered image; then, based on the scalp mask, the hair color is mixed with the first intermediate rendered image to obtain a second intermediate rendered image; then, the skin diffuse color containing base subsurface scattering is mixed with the second intermediate rendered image to obtain a third intermediate rendered image; finally, the skin highlight color is mixed with the third intermediate rendered image to obtain the target rendered digital human model.

[0093] The digital human rendering method provided in this invention, within the Unity engine, renders the opaque parts of the digital human model using a basic forward rendering channel to obtain a basic rendered image. Then, a dedicated skin forward rendering channel is used to render the skin color, and simultaneously, a dedicated hair forward rendering channel is used to render the hair color. By creating independent skin and hair forward rendering channels, relatively complex shading models designed specifically for the corresponding material physical properties can be integrated and run in these two channels respectively. This significantly improves the visual realism of the skin and hair in the digital human rendering without placing an excessive burden on overall performance, achieving a high-fidelity effect. Furthermore, the dedicated channels limit the rendering process to either the skin or hair of the digital human model, greatly reducing computational overhead. Finally, the skin and hair colors are blended with the basic rendered image to obtain the final target rendered digital human model. In summary, this invention, through the aforementioned multi-channel rendering and blending design, enables real-time rendering of digital human models with high-fidelity skin and hair details even on performance-constrained platforms.

[0094] Based on any of the above embodiments Figure 3 This is the second flowchart illustrating the digital human rendering method provided by this invention, as shown below. Figure 3 As shown, step S120 includes: step S121, step S122 and step S123.

[0095] Step S121: Through the skin forward rendering channel, the normal map is separated into channels to calculate the dot product of the normal and the lighting direction.

[0096] A normal map consists of three channels: R (Red), G (Green), and B (Blue). The R channel corresponds to the X component of the normal vector, which is the component of the normal vector in the tangent direction; the G channel corresponds to the Y component of the normal vector, which is the component of the normal vector in the bitangent direction; and the B channel corresponds to the Z component of the normal vector, which is the component of the normal vector in the normal direction.

[0097] In the skin forward rendering channel, the normal map is separated into channels to extract the normal direction vector, i.e., the Z component of the normal vector. Then, the dot product between the normal direction vector (denoted as N) and the lighting direction vector (denoted as L) is calculated to obtain the dot product of the normal and the lighting direction, denoted as dot(N,L).

[0098] In one implementation, parallel light can be used as the main light source, and the illumination direction vector L is obtained in real time by Unity's lighting system.

[0099] The dot product of the normal and the direction of illumination, dot(N,L), represents whether the light shone onto the skin surface at a direct or grazing angle. This directly determines whether the point is in a bright area, a dark area, or on the line of demarcation between light and shadow, and affects the scattering path of the light after it enters the skin.

[0100] Step S122: Based on the curvature and the dot product of the normal and the illumination direction, sample the pre-integrated skin shading map to obtain the diffuse color of the skin.

[0101] Curvature refers to the degree of bending of a model's surface. On a model, areas with high curvature (such as the bridge of the nose, cheekbones, and ear edges) typically have thinner skin, allowing light to penetrate more easily, resulting in a smaller but more intense scattering range, appearing brighter and redder. Conversely, areas with low curvature (such as the cheeks) typically have thicker skin, resulting in a larger but weaker scattering range. Therefore, curvature is a key factor determining scattering intensity and color. Curvature can be extracted from curvature maps.

[0102] A pre-integrated skin shading map is a 2D map whose horizontal axis is the dot product of the normal and the lighting direction (usually mapped to between 0 and 1), and its vertical axis is the curvature (also usually mapped to between 0 and 1).

[0103] Based on the dot product of curvature, normal, and illumination direction, the pre-integrated skin shading map is sampled to obtain the basic diffuse color of the skin, which is then used as the final diffuse color of the skin.

[0104] The diffuse color of the skin is obtained through a single sampling, which simplifies the complex calculation of scattering integrals into a single efficient lookup operation. Compared to the Lambert diffuse reflection model, which requires multiple samplings and weight accumulation in traditional methods, this significantly reduces computational overhead. Simultaneously, this method can realistically simulate the subsurface scattering of light beneath the skin's surface, accurately reproducing the soft translucency and rosy hue unique to the skin at different curvatures (such as the tip of the nose and the auricle) and under different lighting angles, thus enhancing the realism of skin rendering.

[0105] Step S123: Using a dual-highlight model, a secondary highlight simulation is performed based on the surface roughness lobe wave coefficient and the deep roughness lobe wave coefficient to obtain the skin highlight color.

[0106] The skin color includes the diffuse color of the skin and the highlight color of the skin.

[0107] The dual specular model is a shading model used to calculate specular reflections. By blending two different specular reflection models, it achieves richer surface details. The dual specular model can be a Dual Specular model. When rendering skin using the dual specular model, the realism of the skin can be enhanced by adjusting the shape and intensity of the specular distribution.

[0108] The surface roughness lobe coefficient, which can be characterized by LobeRoughness0, defines the highlight sharpness of the grease layer and is used to simulate the sharpness of the grease layer highlights.

[0109] The deep roughness lobe coefficient, which can be characterized by LobeRoughness1, defines the roughness of different areas of the skin itself and is used to simulate the highlights of the skin's basal layer.

[0110] Using a dual-highlight model, secondary highlight simulation is performed based on the surface roughness lobe wave coefficient and the deep roughness lobe wave coefficient to obtain the skin highlight color.

[0111] This dual-highlight model decomposes highlight reflection into a sharp primary highlight term and a soft secondary highlight term, calculates the primary highlight color and secondary highlight color separately, and then performs a weighted sum to obtain the skin highlight color. The specific calculation method is as follows:

[0112] ;

[0113] in, Indicates the color of skin highlight. Indicates the direction of light incidence. Indicates the direction of light emission. This represents a single term based on microsurface specular reflection. Indicates the first preset weight. This indicates the second preset weight. Represents the lobe coefficient of surface roughness. Represents the lobe wave coefficient of deep roughness. This represents Fresnel reflectance.

[0114] Traditional single specular models or simplified BRDF versions struggle to simultaneously represent these two different specular properties. In this embodiment of the invention, a dual specular model is used for secondary specular simulation to synthesize the skin specular color. This method significantly enhances the depth and realism of the skin rendering results.

[0115] The digital human rendering method provided in this invention samples a pre-integrated skin shading map by taking the dot product of curvature, normal, and illumination direction to obtain the diffuse color of the skin. This method efficiently simulates the delicate subsurface scattering effect of the skin, enhancing its three-dimensionality and realistic translucency. Furthermore, compared to existing diffuse reflection models, it reduces computational overhead. Moreover, by utilizing a dual specular model for secondary specular simulation, it generates richly detailed specular highlights, avoiding the plastic-like appearance of traditional single specular models. Through these methods, the realism of skin rendering is significantly improved while maintaining excellent real-time performance.

[0116] Based on any of the above embodiments, step S122 includes: step S1221 and step S1222.

[0117] Step S1221: Based on the curvature and the dot product of the normal and the illumination direction, sample the pre-integrated skin shading map to obtain the basic diffuse color of the skin.

[0118] For skin rendering, current mainstream skin rendering shaders are usually based on simplified lighting models, such as the Lambert (Lambertian Reflectance) model or a simplified version of the BRDF (Bidirectional Reflectance Distribution Function). These models mainly simulate the reflection behavior of light on the skin surface, severely neglecting the complex subsurface scattering (SSS) physical processes of light in the epidermis, dermis, and subcutaneous tissue, resulting in rendered skin that appears unnaturally "plastic".

[0119] In this embodiment, based on sampling the pre-integrated skin shading map to perform preliminary diffuse reflection on the skin, a sub-surface scattering (3S) effect can be further superimposed to obtain the diffuse reflection color of the skin, which can further enhance the realism of the digital human skin rendering.

[0120] Based on the dot product of curvature and normal with the lighting direction, the pre-integrated skin shading map is sampled, and the resulting diffuse skin color is denoted as the base diffuse skin color.

[0121] After obtaining the basic diffuse color of the skin, it can be written to the RGB channel of a scattering buffer.

[0122] Step S1222: The basic diffuse color of the skin is blurred laterally and vertically using a separable Gaussian blur algorithm to obtain the diffuse color of the skin.

[0123] Gaussian blur is an image blurring algorithm that uses a normal distribution (Gaussian function) to calculate the weight of each pixel. The center pixel has the highest weight, and the weight of pixels further away decreases, which can very naturally simulate the diffusion of energy or light.

[0124] Traditional Gaussian blur algorithms are two-dimensional operations, while separable Gaussian blur algorithms decompose this two-dimensional operation into two independent one-dimensional operations: horizontal blurring and vertical blurring. This significantly reduces computational complexity and performance overhead.

[0125] In one embodiment, the basic diffuse color of the skin can be obtained by first blurring it horizontally and then blurring it vertically using a separable Gaussian blur algorithm.

[0126] In another embodiment, the basic diffuse color of the skin can be obtained by first blurring it vertically and then blurring it horizontally using a separable Gaussian blur algorithm.

[0127] The following example illustrates the process of obtaining diffuse skin color by first blurring horizontally and then blurring vertically.

[0128] A separable Gaussian blur algorithm is used to sample the scattering buffer. For each pixel, a horizontal sample is taken along its tangent direction (X-axis). The horizontal sampling offset (offset X) is scaled according to the depth value of the current pixel to simulate the stretching effect of scattering on a curved surface. The horizontal sampling results are then mixed according to their corresponding Gaussian weights to obtain the horizontally blurred result, which is output to a temporary buffer.

[0129] Then, the temporary buffer is sampled. For each pixel, a vertical sample is taken in another direction (the Y-axis), with the vertical sampling offset (offset Y) also related to the depth value. The vertical sampling results are then mixed according to their corresponding Gaussian weights to output the final bidirectional blur result, which is the final diffuse skin color.

[0130] The diffuse skin color obtained through the above method already includes the diffuse skin color of the basic subsurface scattering.

[0131] The digital human rendering method provided in this invention, after sampling the pre-integrated skin shading map to perform preliminary diffuse reflection on the skin, further implements separable subsurface scattering through a separable Gaussian blur algorithm. This simulates the scattering process of light under the skin, producing a soft, gradual halo effect, enhancing the transparency and realism of the skin material, and avoiding the plastic-like appearance of traditional models. Simultaneously, by employing a separable Gaussian blur algorithm, compared to traditional two-dimensional convolution, computational complexity and performance overhead are significantly reduced, enabling high-quality multi-sampled blur to run at real-time frame rates, making it suitable for real-time interactive applications such as games.

[0132] Based on any of the above embodiments Figure 4 This is the third flowchart of the digital human rendering method provided by the present invention, as shown below. Figure 4 As shown, step S130 includes: step S131, step S132, step S133 and step S134.

[0133] It should be noted that the execution order of steps S131 and S132 is not important.

[0134] Step S131: Using the hair forward rendering channel, the lateral scattering distribution of multiple optical paths is calculated using the HairGeo hair geometry simplification model. These multiple optical paths include a reflection (R) path, a transmission-transmission (TT) path, and a transmission-reflection-transmission (TRT) path.

[0135] For hair rendering, the industry commonly uses standard models such as Phong or Blinn. These models cannot simulate the unique anisotropic structure of hair strands, and therefore cannot accurately reproduce complex optical effects such as radial highlights, transmitted light bands, and backscattering. This results in rendered hair that looks stiff, lacks texture, and has a significant discrepancy between its gloss and physical reality. While the classic Marschner hair rendering model can solve these problems, it has high requirements for the terminal device. Therefore, this embodiment uses an improved Marschner hair rendering model, which can achieve high-fidelity rendering of hair color while maximizing performance efficiency, making it applicable to performance-constrained platforms such as mobile devices.

[0136] By using the hair forward rendering pass, the lateral scattering distribution of multiple light paths is calculated separately using the simplified HairGeo (Hair Geometry) model.

[0137] The HairGeo simplified model is an analytical approximation based on hair geometry used to calculate the scattering distribution of light across a cross-section. Specifically, it calculates the offset angle of the light path relative to the perfect specular reflection / transmission direction through geometric approximation to simulate the azimuthal offset caused by the circular cross-section of the hair, which is the source of the hair's anisotropic highlights.

[0138] By using the simplified HairGeo model, the scattering angle shift can be directly calculated through mathematical formulas to obtain the lateral scattering distribution. Compared with the accurate physical simulation or LUT (Lookup Table) lookup based on the existing Marschner model, the computational cost can be significantly reduced.

[0139] Multiple optical paths include three main paths: R (Reflection) path, TT (Transmission-Transmission) path, and TRT (Transmission-Reflection-Transmission) path.

[0140] By selecting the three main paths mentioned above and discarding other complex paths, such as the TRRT (Transmission-Reflection-Reflection-Transmission) path, computational overhead can be further reduced. Experiments show that by simplifying the calculations of complex parts such as TRRT, up to 15% of computational costs can be saved while maintaining basic energy conservation.

[0141] Correspondingly, the lateral scattering distributions of multiple optical paths include the lateral scattering distribution of the R path, the lateral scattering distribution of the TT path, and the lateral scattering distribution of the TRT path.

[0142] Step S132: Calculate the longitudinal scattering distribution of multiple optical paths using a Gaussian model.

[0143] The Gaussian model simulates the scattering of light along the length of a hair strand due to the surface roughness (i.e., the hair cuticle). The longitudinal scattering distribution of each path is approximated by a Gaussian distribution.

[0144] The longitudinal scattering distribution of multiple optical paths was calculated using the Gaussian model to obtain the longitudinal scattering distribution of path R, path TT, and path TRT.

[0145] Step S133: Calculate the colors of the multiple optical paths based on their lateral scattering distribution and longitudinal scattering distribution.

[0146] After obtaining the lateral and longitudinal scattering distributions of multiple optical paths, the colors of these paths are further calculated. The colors of the multiple optical paths include the R path color, TT path color, and TRT path color.

[0147] In another embodiment, the energy absorption of the TT path and the TRT path is obtained through a table fitting query. The table fitting query involves calculating the relative azimuth angle and roughness, converting them into horizontal and vertical coordinates, and then querying the corresponding LUT to obtain the corresponding energy absorption. Then, the R-path color is calculated based on the hair base color, the lateral scattering distribution of the R-path, and the longitudinal scattering distribution of the R-path; the TT path color is calculated based on the hair base color, the lateral scattering distribution of the TT path, the longitudinal scattering distribution of the TT path, and the energy absorption of the TT path; and the TRT path color is calculated based on the hair base color, the lateral scattering distribution of the TRT path, the longitudinal scattering distribution of the TRT path, and the energy absorption of the TRT path.

[0148] In another embodiment, the Chiang approximate back-calculation model is used to calculate the energy absorption of the TT path and the TRT path; the R path color is calculated based on the hair base color, the lateral scattering distribution of the R path, and the longitudinal scattering distribution of the R path; the TT path color is calculated based on the hair base color, the lateral scattering distribution of the TT path, the longitudinal scattering distribution of the TT path, and the energy absorption of the TT path; and the TRT path color is calculated based on the hair base color, the lateral scattering distribution of the TRT path, the longitudinal scattering distribution of the TRT path, and the energy absorption of the TRT path. The specific execution process can be found in the following embodiments, which will not be elaborated here.

[0149] Step S134: Calculate the hair color based on the colors of the multiple optical paths.

[0150] The colors of multiple optical paths include the R path color, TT path color, and TRT path color.

[0151] In one embodiment, the colors of multiple optical paths can be directly summed to obtain the hair color. That is, hair color = R path color + TT path color + TRT path color.

[0152] In another implementation, although the energy contribution of complex paths such as TRRT is much smaller than that of TRT, TT, and R paths, considering the energy conservation issue, the energy of complex paths such as TRRT can be roughly estimated and added to the most similar TRT path. Specifically, a scaling factor, a constant greater than 1, such as 1.05 or 1.1, can be preset and multiplied by the color of the TRT path. In this case, hair color = R path color + TT path color + preset scaling factor × TRT path color. This method effectively compensates for energy loss while reducing computational costs, preventing hair from darkening.

[0153] The digital human rendering method provided in this invention uses the HairGeo simplified model to handle lateral scattering and the Gaussian model to handle longitudinal scattering. At the same time, it only calculates three main optical paths (including the R path, TT path and TRT path), which can greatly reduce computation and bandwidth overhead with minimal loss of visual quality.

[0154] Based on any of the above embodiments, the longitudinal scattering distribution of the multiple optical paths includes the longitudinal scattering distribution of the R path, the longitudinal scattering distribution of the TT path, and the longitudinal scattering distribution of the TRT path; the lateral scattering distribution of the multiple optical paths includes the lateral scattering distribution of the R path, the lateral scattering distribution of the TT path, and the lateral scattering distribution of the TRT path; step S133 includes: step S1331, step S1332, step S1333, and step S1334.

[0155] Step S1331: Calculate the energy absorption of the TT path and the energy absorption of the TRT path using the Chiang approximate back-calculation model.

[0156] When light travels through the hair shaft (primarily affecting the TT and TRT paths), it is absorbed by pigments, which is the source of hair color. This process is described by the Beer-Lambert law, the core of which is an exponential function—exp(). However, frequently calculating the exp() function in the shader is quite expensive. Therefore, to reduce computation, a traditional improvement is to pre-calculate the result into a LUT, querying the absorbed color using the hair color and string length. However, this LUT query method still requires significant computational overhead.

[0157] Therefore, in this embodiment, the Chiang approximate back-calculation model is used to calculate the energy absorption of the TT path and the TRT path. Specifically, the attenuation effect of the exp() function can be approximated by a simple calculation formula, as follows:

[0158] ;

[0159] in, denoted by α, representing the single-fiber absorption coefficient, α representing the scattering albedo, β representing the azimuth roughness, and B() representing the azimuth roughness function that controls the distribution of high light scattering.

[0160] By dynamically calculating energy absorption using the approximate back-calculation formula in the Chiang approximate back-calculation model, computational overhead can be further reduced.

[0161] Step S1332: Calculate the R-path color based on the hair base color, the horizontal scattering distribution of the R-path, and the vertical scattering distribution of the R-path.

[0162] The R-path color is obtained by multiplying the base hair color, the R-path lateral scattering distribution, and the R-path longitudinal scattering distribution. That is, R-path color = base hair color × R-path longitudinal scattering distribution × R-path lateral scattering distribution.

[0163] It should be noted that the R path represents the direct reflection of light on the surface of the outermost cuticle layer of the hair. The light does not actually enter the internal medium of the hair containing pigment. Therefore, the calculation of R path color does not require energy absorption.

[0164] Step S1333: Calculate the TT path color based on the hair base color, the lateral scattering distribution of the TT path, the longitudinal scattering distribution of the TT path, and the energy absorption of the TT path.

[0165] The TT path color is obtained by multiplying the base hair color, the lateral scattering distribution of the TT path, the longitudinal scattering distribution of the TT path, and the energy absorption of the TT path. That is, TT path color = base hair color × lateral scattering distribution of the TT path × longitudinal scattering distribution of the TT path × energy absorption of the TT path.

[0166] Step S1334: Calculate the TRT path color based on the hair base color, the TRT path lateral scattering distribution, the TRT path longitudinal scattering distribution, and the TRT path energy absorption.

[0167] The colors of the multiple optical paths include the R path color, the TT path color, and the TRT path color.

[0168] The TRT path color is obtained by multiplying the base hair color, the lateral scattering distribution of the TRT path, the longitudinal scattering distribution of the TRT path, and the energy absorption of the TRT path. That is, TRT path color = base hair color × lateral scattering distribution of the TRT path × longitudinal scattering distribution of the TRT path × energy absorption of the TRT path.

[0169] The digital human rendering method provided in this invention uses the Chiang approximate back-calculation model to derive energy absorption through a single mathematical calculation. Compared to complex calculations or time-consuming LUT queries, it can efficiently calculate the energy absorption of the TT path and the TRT path, which are then used to calculate the TT path color and the TRT path color, and subsequently, the hair color. This method further reduces computational overhead.

[0170] Based on any of the above embodiments Figure 5 This is the fourth flowchart of the digital human rendering method provided by the present invention, as shown below. Figure 5 As shown, before step S140, the procedure further includes:

[0171] Step S150: Render the eye portion of the digital human model to be rendered using the eye-forward rendering channel to obtain the eye color.

[0172] The eyeball portion of the digital human model to be rendered is rendered using the forward eyeball rendering channel to obtain the eyeball color.

[0173] The rendering of the eyeball consists of two parts: the iris and the cornea.

[0174] For the iris layer, the illumination components corresponding to the iris layer of the eyeball are first calculated, and the basic iris color is synthesized based on the illumination components. The illumination components include diffuse reflection components, subsurface scattering components, and specular components. Then, the subsurface ambient light occlusion map is applied to modulate the basic iris color to obtain the iris layer color.

[0175] For the corneal layer, first obtain the reflected and refracted colors of the corneal layer in the eyeball and calculate the Fresnel reflection coefficient; then, based on the Fresnel reflection coefficient, mix the reflected and refracted colors to obtain the corneal layer color.

[0176] Finally, the colors of the iris and cornea are mixed with varying transparency to obtain the eye color.

[0177] The specific execution process can be found in the following examples, which will not be elaborated here.

[0178] At this time, step S140 includes: step S141.

[0179] Step S141: Mix the skin color, hair color, and eye color with the base rendering image to obtain the target rendering digital human model.

[0180] Skin color, hair color, and eye color are blended with the base rendered image to obtain the target rendered digital human model. Blending methods include, but are not limited to, alpha blending and additive blending; the specific blending method is determined based on the desired visual effect.

[0181] The digital human rendering method provided in this invention further creates an eyeball forward rendering channel based on the skin and hair forward rendering channels. This eyeball forward rendering channel renders the eyeball portion of the digital human model to be rendered. In this way, a computationally complex shading model specifically designed for the eyeball can be integrated and run in this eyeball forward rendering channel. This significantly improves the visual realism of the rendered eyes without placing an excessive burden on overall performance, achieving a high-fidelity effect. Since eyeball rendering is completed in this eyeball forward rendering channel, the algorithm overhead is controllable, making it suitable for applications with extremely high real-time requirements.

[0182] Based on any of the above embodiments, step S150 includes: step S151, step S152, step S153 and step S154.

[0183] Step S151: Calculate the illumination component corresponding to the iris layer of the eyeball through the forward rendering channel of the eyeball, and synthesize the basic iris color based on the illumination component; wherein, the illumination component includes diffuse reflection component, subsurface scattering component and specular component.

[0184] In this embodiment, the rendering of the eyeball includes two parts: the iris layer and the cornea layer.

[0185] First, calculate the illumination components corresponding to the iris layer of the eyeball. These illumination components include diffuse reflection, subsurface scattering, and specular highlights. The diffuse reflection component can be calculated using the Lambertian Reflection model, the subsurface scattering component using the Wrap Lighting model, and the specular highlights using the Blinn-Phong model.

[0186] Then, the diffuse reflection component, subsurface scattering component, and specular component are added together to obtain the preliminary color that has not been modulated by ambient light occlusion, namely the basic iris color.

[0187] Step S152: Apply the sub-surface ambient light occlusion map to modulate the iris base color to obtain the iris layer color.

[0188] Modulation is the process of controlling or changing the intensity, value, or properties of a modulated object through a modulation source, usually achieved through multiplication.

[0189] During modulation, the subsurface ambient occlusion map is multiplied by the iris base color to shape the internal occlusion and depth variations of the iris base color; the output is the iris layer color. Specifically, a grayscale value is sampled from the subsurface ambient occlusion map (SSAO map). This grayscale value is then used to modulate the obtained iris base color to obtain the final iris layer color. The calculation formula is: Iris layer color = Iris base color × Grayscale value.

[0190] By separating and calculating the three core physical components of diffuse reflection, subsurface scattering, and specular highlights, the interaction between light and biological tissue is accurately simulated. At the same time, the base color of the synthesized iris is modulated using a subsurface ambient occlusion map, which enhances the shadows of the concave and convex details at the microscopic level, thereby enhancing the three-dimensionality and depth of the iris.

[0191] Step S153: Obtain the reflected color and refracted color of the corneal layer of the eyeball, and calculate the Fresnel reflection coefficient.

[0192] Obtain the reflected and refracted colors corresponding to the corneal layer.

[0193] The process of obtaining the reflected color of the corneal layer is as follows: Based on the normal vector and view vector of the corneal surface, the environment cube map is sampled to obtain the environmental image color reflected by the current pixel, which is used as the reflected color.

[0194] The process of obtaining the refracted color of the cornea is as follows: Calculate the refraction direction of light as it enters the cornea from the air and then enters the aqueous humor according to Snell's law. Using this refraction direction, sample the iris color obtained from the above rendering to obtain the color of the iris and pupil as seen due to refraction, which is then used as the refracted color.

[0195] Simultaneously, the Fresnel reflection coefficient is calculated. The Schlick approximation formula can be used in the calculation, which significantly improves computational efficiency compared to the complete Fresnel formula.

[0196] Step S154: Based on the Fresnel reflection coefficient, the reflected color and the refracted color are mixed to obtain the corneal color.

[0197] The color of the eyeball includes the color of the iris and the color of the cornea.

[0198] The calculated Fresnel reflectance coefficient is used as a weight to perform linear interpolation and mixing of the reflected and refracted colors to obtain the corneal color. The formula for calculating the corneal color is: Corneal color = Fresnel reflectance coefficient × Reflected color + (1 - Fresnel reflectance coefficient) × Refracted color.

[0199] The digital human rendering method provided in this invention enhances the three-dimensionality and depth of the iris by separating, calculating, and mixing multiple lighting components, and then applying a sub-surface ambient occlusion map to the synthesized iris base color. Furthermore, by simulating reflection, refraction, and Fresnel effects in keratinocyte processing, the eyeball can be given a transparent three-dimensionality, dynamic specular changes, and a moist, biological texture, thus improving the rendering effect of the eyeball.

[0200] Based on any of the above embodiments, step S154 includes: step S1541, step S1542, step S1543, step S1544 and step S1545.

[0201] Step S1541: Mix the basic diffuse color of the skin with the basic rendering image to obtain the first intermediate rendering image.

[0202] During rendering, the base diffuse color of the skin is blended with the base rendered image to obtain the first intermediate rendered image. Alpha blending (α blending, a technique in computer graphics that achieves transparency by blending pixel color channels with alpha channels, suitable for simulating semi-transparent objects) can be used during blending.

[0203] It's important to note that during the generation of the base render image, depth testing and depth writing are simultaneously enabled, writing the depth values ​​of the base render image to the depth buffer. The purpose of this depth testing is to ensure that skin pixels are only rendered if there are no nearby opaque objects (such as clothing) in front of them, without disrupting the existing depth buffer. When rendering the base diffuse color of the skin, depth writing is disabled to preserve correct depth information for subsequent rendering of opaque objects (such as the eyeball).

[0204] Step S1542: The iris color and the corneal color are sequentially overlaid onto the first intermediate rendering image to obtain the second intermediate rendering image.

[0205] Because the eyeball is an opaque object, a blending process is not used when rendering it; instead, an overlay method is employed. Specifically, the colors of the iris and cornea are sequentially overlaid onto the first intermediate rendered image to obtain the second intermediate rendered image.

[0206] It's important to note that depth testing and depth writing are enabled during eye rendering. The purpose of depth testing is to ensure that the eyeball is only drawn on the depth layer containing the already rendered skin and eye socket, preventing rendering failures due to depth accuracy issues. The depth buffer is updated while overlaying the iris and corneal colors onto the first intermediate rendered image.

[0207] Step S1543: Based on the scalp mask, the hair color is mixed with the second intermediate rendering image to obtain a third intermediate rendering image.

[0208] The hair rendering area of ​​the second intermediate render image is determined based on the scalp mask. Then, the hair color is blended with the second intermediate render image to obtain the third intermediate render image. Alpha blending is used during the blending process.

[0209] It's important to note that when rendering hair, keep depth testing enabled and disable depth writing. The purpose of depth testing here is to ensure that hair is correctly occluded.

[0210] Step S1544: Mix the diffuse color of the skin with the third intermediate rendering image to obtain the fourth intermediate rendering image.

[0211] The diffuse color of the skin is blended with the third intermediate render image to obtain the fourth intermediate render image. Alpha blending is used during the blending process.

[0212] It should be noted that if the basic diffuse color of the skin obtained by sampling the pre-integrated skin shading map is used as the final diffuse color of the skin, that is, when separable subsurface scattering is used for further superposition, then step S1544 is not executed.

[0213] It's also important to note that when rendering skin diffuse colors including the SSS effect, depth testing should be enabled, but depth writing should be disabled. The purpose of depth testing is to prevent penetration blur. Specifically, the original depth of the current pixel in the scene is obtained, along with the original depth of the scattered light's location. The difference between these two depth values ​​is compared. If the difference is within a preset range, it indicates they belong to the same smooth surface, and the SSS effect is applied. If the difference is outside the preset range, it means this is an object edge or occlusion area; in this case, the SSS effect is not applied or is weakened to prevent color penetration. In this way, the scattering effect is only fully applied to continuous surfaces with similar depths, effectively cropping out blur that extends to other objects.

[0214] Step S1545: Mix the skin highlight color with the fourth intermediate rendering image to obtain the target rendered digital human model.

[0215] The skin highlight color is blended with the fourth intermediate rendered image to obtain the target rendered digital human model, completing the rendering process. Additive blending is used during blending. Additive blending directly adds the source color to the target color, regardless of transparency.

[0216] It's important to note that when rendering skin highlight colors, depth testing should be enabled, but depth writing should be disabled. The purpose of depth testing here is to ensure that skin highlights only appear at the pixel locations where skin exists, preventing them from incorrectly covering other objects, while maintaining high performance.

[0217] The digital human rendering method provided in this invention can construct a complex visual hierarchy of a digital human by mixing skin color, hair color, and eye color with a base rendering image in sequence, thereby obtaining a target rendering digital human model and completing the rendering process.

[0218] The digital human rendering apparatus provided by the present invention is described below. The digital human rendering apparatus described below can be referred to in correspondence with the digital human rendering method described above.

[0219] Figure 6 This is a schematic diagram of the structure of the digital human rendering device provided by the present invention, as shown below. Figure 6 As shown, the device includes a basic rendering module 610, a skin rendering module 620, a hair rendering module 630, and a blending module 640; wherein:

[0220] The basic rendering module 610 is used in the Unity engine to render the opaque parts of the digital human model to be rendered through the basic forward rendering pass, so as to obtain the basic rendering image.

[0221] The skin rendering module 620 is used to render the skin part of the digital human model to be rendered through the skin forward rendering channel to obtain the skin color;

[0222] The hair rendering module 630 is used to render the hair portion of the digital human model to be rendered through the hair forward rendering channel to obtain the hair color;

[0223] The mixing module 640 is used to mix the skin color and the hair color with the base rendering image to obtain the target rendering digital human model.

[0224] It should be noted that the digital human rendering apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above-described digital human rendering method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0225] Figure 7 An example is a schematic diagram of the physical structure of an XR device, such as... Figure 7 As shown, the XR device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a digital human rendering method. This method includes: in the Unity engine, rendering the opaque parts of the digital human model to be rendered through a base forward rendering channel to obtain a base rendering image; rendering the skin part of the digital human model to be rendered through a skin forward rendering channel to obtain a skin color; rendering the hair part of the digital human model to be rendered through a hair forward rendering channel to obtain a hair color; and mixing the skin color and the hair color with the base rendering image to obtain a target rendered digital human model.

[0226] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0227] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the digital human rendering method provided by the above methods. The method includes: in the Unity engine, rendering the opaque parts of the digital human model to be rendered through a basic forward rendering channel to obtain a basic rendering image; rendering the skin part of the digital human model to be rendered through a skin forward rendering channel to obtain a skin color; rendering the hair part of the digital human model to be rendered through a hair forward rendering channel to obtain a hair color; and mixing the skin color and the hair color with the basic rendering image to obtain a target rendered digital human model.

[0228] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the digital human rendering method provided by the above methods. The method includes: in the Unity engine, rendering the opaque parts of the digital human model to be rendered through a base forward rendering channel to obtain a base rendering image; rendering the skin part of the digital human model to be rendered through a skin forward rendering channel to obtain a skin color; rendering the hair part of the digital human model to be rendered through a hair forward rendering channel to obtain a hair color; and mixing the skin color and the hair color with the base rendering image to obtain a target rendered digital human model.

[0229] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0230] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0231] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method of digital human rendering, the method comprising: The method comprises the following steps: In the Unity engine, render the opaque part of the digital human model to be rendered through a basic forward rendering channel to obtain a basic rendering image; Render the skin part of the digital human model to be rendered through a skin forward rendering channel to obtain a skin color; Render the hair part of the digital human model to be rendered through a hair forward rendering channel to obtain a hair color; Render the eyeball part of the digital human model to be rendered through an eyeball forward rendering channel to obtain an eyeball color; Mix the skin base diffuse color in the skin color with the basic rendering image to obtain a first intermediate rendering image; Cover the iris layer color and the cornea layer color in the eyeball color to the first intermediate rendering image in sequence to obtain a second intermediate rendering image; Mix the hair color with the second intermediate rendering image according to a scalp mask to obtain a third intermediate rendering image; Mix the skin diffuse color in the skin color with the third intermediate rendering image to obtain a fourth intermediate rendering image; Mix the skin highlight color in the skin color with the fourth intermediate rendering image to obtain a target rendering digital human model.

2. The digital human rendering method of claim 1, wherein, The method comprises the following steps: Through the skin forward rendering channel, perform channel separation on a normal map to calculate the dot product of the normal and the light direction; Sample a pre-integrated skin shading map according to the curvature and the dot product of the normal and the light direction to obtain a skin diffuse color; Use a two-highlight model to perform secondary highlight simulation according to the surface roughness lobe coefficient and the deep roughness lobe coefficient to obtain a skin highlight color. The skin color comprises the skin diffuse color and the skin highlight color.

3. The digital human rendering method of claim 2, wherein, The method comprises the following steps: Sample the pre-integrated skin shading map according to the curvature and the dot product of the normal and the light direction to obtain a skin base diffuse color; Perform horizontal blur and vertical blur on the skin base diffuse color through a separable Gaussian blur algorithm to obtain the skin diffuse color.

4. The digital human rendering method of claim 1, wherein, The method comprises the following steps: Through the hair forward rendering channel, use a HairGeo hair geometry simplification model to calculate the horizontal scattering distribution of a plurality of light path paths respectively; wherein the plurality of light path paths comprise a reflection R path, a transmission-transmission TT path and a transmission-reflection-transmission TRT path; Use a Gaussian model to calculate the vertical scattering distribution of the plurality of light path paths respectively; Calculate the color of the plurality of light path paths according to the horizontal scattering distribution of the plurality of light path paths and the vertical scattering distribution of the plurality of light path paths; Calculate the hair color according to the color of the plurality of light path paths.

5. The digital human rendering method of claim 4, wherein, The longitudinal scattering distribution of the plurality of light paths includes an R path longitudinal scattering distribution, a TT path longitudinal scattering distribution, and a TRT path longitudinal scattering distribution, and the transverse scattering distribution of the plurality of light paths includes an R path transverse scattering distribution, a TT path transverse scattering distribution, and a TRT path transverse scattering distribution; The color of the plurality of light paths is calculated according to the transverse scattering distribution of the plurality of light paths and the longitudinal scattering distribution of the plurality of light paths, and includes: The TT path energy absorption and the TRT path energy absorption are calculated by using a Chiang approximation back-propagation model; The R path color is calculated according to the hair base color, the R path transverse scattering distribution, and the R path longitudinal scattering distribution; The TT path color is calculated according to the hair base color, the TT path transverse scattering distribution, the TT path longitudinal scattering distribution, and the TT path energy absorption; The TRT path color is calculated according to the hair base color, the TRT path transverse scattering distribution, the TRT path longitudinal scattering distribution, and the TRT path energy absorption; The color of the plurality of light paths includes the R path color, the TT path color, and the TRT path color.

6. The digital human rendering method of claim 1, wherein, The eye color is obtained by rendering the eye part of the digital human model to be rendered through an eye forward rendering channel, and includes: The iris layer corresponding to the eye part is calculated through the eye forward rendering channel, and the illumination component is synthesized according to the illumination component to obtain the iris base color; wherein the illumination component includes a diffuse reflection component, a subsurface scattering component, and a highlight component; The iris base color is modulated by applying a subsurface ambient light shading map to obtain the iris layer color; The reflection color and the refraction color corresponding to the cornea layer of the eye part are obtained, and the Fresnel reflection coefficient is calculated; The reflection color and the refraction color are mixed according to the Fresnel reflection coefficient to obtain the cornea layer color; The eye color includes the iris layer color and the cornea layer color.

7. A digital human rendering apparatus, characterized by, It includes: The base rendering module is used for rendering the opaque part of the digital human model to be rendered through a base forward rendering channel in the Unity engine to obtain a base rendering image; The skin rendering module is used for rendering the skin part of the digital human model to be rendered through a skin forward rendering channel to obtain a skin color; The hair rendering module is used for rendering the hair part of the digital human model to be rendered through a hair forward rendering channel to obtain a hair color; The digital human rendering device further includes a module for rendering the eye part of the digital human model to be rendered through an eye forward rendering channel to obtain an eye color; The mixing module is used for mixing the skin base diffuse reflection color in the skin color with the base rendering image to obtain a first intermediate rendering image; The iris layer color and the cornea layer color in the eye color are sequentially overlaid on the first intermediate rendering image to obtain a second intermediate rendering image; The hair color is mixed with the second intermediate rendering image according to the scalp mask to obtain a third intermediate rendering image; mixing the skin diffuse color in the skin color with the third intermediate rendering image to obtain a fourth intermediate rendering image; mixing the skin highlight color in the skin color with the fourth intermediate rendering image to obtain a target rendering digital human model.

8. An XR device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, wherein, The processor implements the digital human rendering method in any one of claims 1-6 when executing the computer program.

Images