Scheme for automatically processing digital human assets

Abstract

The invention relates to the technical field of computers, in particular to the technical fields of virtual digital humans, augmented reality, virtual reality, mixed reality, augmented reality and the like, and provides a scheme for automatically processing digital human assets. The method comprises the steps of structure identification of digital human assets, automatic rendering Shader matching and mouth shape matching. And the structure identification is used for identifying a structure file of the digital human through regular matching, wherein the structure file comprises a grid structure and a face BlendShape structure. The automatic rendering Shader matching is adapted to different Shaders according to different parts of digital human assets, for example, a skin structure is adapted to an SSS Shader, a hair structure is adapted to a double-sided transparent Shader and the like. According to mouth shape matching, standard mouth shape data which can be recognized by a driving program are automatically matched for digital human assets by recognizing key BlendShape key values of lips.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to the technical fields of virtual digital humans, augmented reality, virtual reality, mixed reality, extended reality, etc., and specifically to a solution for automatically processing digital human assets. Background Art

[0002] Lip-syncing and rendering of digital human assets is a tedious and complex task, and each step requires professionals to invest a lot of time and effort. Modelers need to fine-tune the facial structure of digital humans to ensure that the lip-syncing changes are natural and realistic; technical artists need to carefully adjust and optimize the model to ensure that its performance during the rendering process meets expectations; and Unity engineers need to accurately connect digital human assets with the driver program to achieve lip-syncing and voice synchronization. In this case, how to complete the connection between digital human assets and digital human drivers at low cost and high efficiency has become an urgent problem to be solved in the industry. Summary of the invention

[0003] The present disclosure provides a solution for automatically processing digital human assets.

[0004] According to the content of the present disclosure, a solution for automatically processing digital human assets is provided, including: structural identification of digital human assets.

[0005] According to the naming in the digital human file structure, regular matching is completed in the program to identify the digital human structure file, including the mesh structure of the digital human asset and the facial BlendShape (hereinafter referred to as "BS") structure.

[0006] Automatic rendering shader matching for digital human assets.

[0007] In graphics rendering, different parts of a digital human need to be adapted to different shaders, for example, the skin structure needs to be adapted to the SSS (subsurface scattering) shader, the hair structure needs to be adapted to the double-sided transparent shader, etc. After the mesh structure of the digital human asset is recognized, the present disclosure automatically adapts various common shaders to provide the correct rendering performance for the digital human asset.

[0008] Lip-matching of digital human assets.

[0009] In the field of digital human mouth shape, usually animators or modelers make lip shape BS for digital human assets and provide them to the program for calling. After the face BS of the digital human asset is recognized, the present invention automatically matches the digital human asset with standard lip shape data that can be recognized by the driver by recognizing the key BS key value of the lips. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] By reading the detailed description of the specific implementation below, the technical details of the present application will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the specific implementation and are not considered to be limitations of the present application. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0011] Figure 1 A flow chart of a solution for automatically processing digital human assets provided by the present application is shown.

[0012] Figure 2 A schematic diagram of Shader matching in a solution for automatically processing digital human assets provided by the present application is shown.

[0013] Figure 3 A schematic diagram of key lip shapes in a solution for automatically processing digital human assets provided by the present application is shown. DETAILED DESCRIPTION

[0014] For ease of understanding, the specific process of the embodiment of the present disclosure is described below. Figure 1 .

[0015] 101. Perform structural identification on digital human assets.

[0016] In practical applications, digital human assets are generally composed of Hair, Head, Eye, Eye Occlusion, Tear Line, Teeth, Tongue, and Cloth. In this embodiment, since the Cloth structure does not meet the conditions for standardization and there is no need for standardization, Cloth is not recognized.

[0017] In this embodiment, you need to import any digital human assets first. When importing digital human assets, the system will parse the asset naming structure for subsequent component identification and BlendShape value extraction; ensure that the folder structure of the digital human assets is clear and the naming is standardized so that the system can accurately identify each component.

[0018] During the component identification phase, the component names in the digital human assets will be matched to determine which components are included; if there are similar values ​​in the component names (such as Hair and Hair), the system should be able to identify and classify them into the same component category; when applied in the game engine, the system will perform the first regular match on all the names of the digital human asset components according to the standard values; regular expressions can be used to quickly and accurately match names that conform to specific patterns in order to identify components such as Hair, Head, Eye, etc.; after the matching is completed, the system will record the asset component with the highest matching degree to ensure that subsequent processing is based on the correct component; for the Head component, the system will further identify the BlendShape value related to the mouth shape; the BlendShape value related to the mouth shape is crucial to achieving realistic facial expressions, so identifying these values ​​can improve the expressiveness of the digital human; the automatically generated configuration file will contain the identified components and BlendShape information so that subsequent applications or game engines can accurately load and apply this data; the configuration file has a clear structure for easy parsing and use.

[0019] 102. Automatic rendering shader matching of digital human assets.

[0020] In 101, the various structures of the digital human asset have been successfully matched, and the corresponding Shader has been specified for each component. Now it is necessary to automatically match the pre-compiled Shader for the Material of each Mesh of the digital human asset based on these matching results. In this embodiment, a series of Shader codes have been pre-compiled to select the appropriate Shader for different components of the digital human asset.

[0021] like Figure 2 Specifically, the shaders matched for the digital human assets are as follows:

[0022] Head matches SSShader, which is used to simulate the texture of the skin and use subsurface scattering technology to achieve realistic rendering of semi-transparent materials such as skin, jade, and milk.

[0023] Hair matches the double-sided transparent Shader, which is used to process the hair part to ensure that the transparent effect of the hair can be correctly presented.

[0024] Eye matches SSShader to achieve realistic eye effects and help increase the vividness of the eyes.

[0025] The EyeOcclusion part uses a Shader with specular reflection and high-gloss reflection effects to highlight the realism of the eye iris.

[0026] The TearLine part uses a Shader with specular reflection and highlight reflection effects to highlight the details of the tear line.

[0027] The Teeth part uses SSShader to simulate the texture of teeth and help increase the realism of teeth.

[0028] The tongue uses an SSShader but with a lower transparency parameter to keep the tongue realistic while avoiding overly transparent effects.

[0029] 103. Lip-sync matching of digital human assets.

[0030] In this embodiment 101, the structure of the digital human has been identified.

[0031] After recognition, five BlendShapes are saved, and the corresponding lip movements are as follows:

[0032] Mouth_Pucker_Open: lower lip open;

[0033] Mouth_Top_Lip_Up: upper lip open;

[0034] Mouth_Smile: lips extend to the sides;

[0035] Mouth_Blow: lips are pursed towards the middle and front;

[0036] Mouth_Plosive: Lips are puffed;

[0037] The corresponding BlendShape effect is as follows Figure 3 shown.

[0038] In this embodiment, a set of standard configuration files has been manually created based on past experience. According to this set of standard configuration files, five vowel mouth shapes can be automatically blended according to different parameters based on the values ​​of the five recognized BlendShapes.

Claims

1. A solution for automatically processing digital human assets, characterized in that: The following steps are involved: Step S1: accurately identifying the structure files of the digital virtual human through regular matching technology. These structure files include the mesh structure Mesh Structure and the facial BlendShape structure. The mesh structure defines the basic form of the digital virtual human, while the BlendShape structure is responsible for realizing the animation effect of facial expressions. Step S2: Identify various components of the digital virtual human asset, such as hair, head, eyes, eye occlusion, tear line, teeth, tongue, etc. This step is crucial for subsequent shader matching and animation effect generation; Step S3: identifying the BlendShape values ​​related to the mouth shape, including Mouth_Pucker_Open, Mouth_Top_Lip_Up, Mouth_Smile, Mouth_Blow, Mouth_Plosive, and other key mouth shape actions. By accurately identifying these mouth shape actions, basic data can be provided for subsequent animation effect generation. Step S4: After completing the above steps, the method will automatically generate a configuration file to record all recognition results. This configuration file will serve as the basis for subsequent automatic rendering and animation effect generation; Step S5: Automatic rendering shader matching of digital virtual human assets. The method will automatically match pre-programmed shaders for each Mesh according to the various components of the digital virtual human assets. Step S6: In the process of matching the shader, the method will also adapt different shaders according to the characteristics of different parts. For example, for the skin part, the SSS shader subsurface scattering shader can be used to simulate the lighting effect of the real skin; for the transparent material part, the double-sided transparent shader can be used to achieve a more realistic transparent effect; Step S7: the method automatically matches the standard lip shape data that can be recognized by the driver according to the recognized lip key BlendShape key value, so that when generating animation effects, the lip shape movements of the digital virtual person can be driven according to these standard lip shape data; Step S8: According to the standard configuration file, the method automatically mixes different lip movements. This step achieves a smooth transition from one lip shape to another through interpolation and mixing techniques, thereby generating a more natural and realistic animation effect.

Images