A Metaverse virtual digital human production system

Abstract

The present invention discloses a metaverse virtual digital human production system. The system includes a scanning module, a BlendShape splitting module, a model Capture module, an ecological process processing module, and a physiological texture processing module. Physiological texture processing is added to fine-tune the model after processing the real-person scanning data. During the production process, due to the addition of the physiological texture processing module, three types of cameras are used, which are respectively responsible for color, optical, and high-speed photography, so as to photograph the skin texture of the displayed person, and then add the skin texture to the skin of the virtual digital human, add the corresponding physiological texture of the human body to the skin on the model surface, enable the virtual digital human to have the same fingerprint label as the real person, make the digital virtual human and the person in the display have common physiological characteristics, and have unique representativeness, avoiding a certain degree of similarity with the digital virtual humans of other people.

Description

Technical Field

[0001] The present invention belongs to the technical field related to virtual digital humans in the metaverse, and particularly relates to a metaverse virtual digital human production system. Background Art

[0002] The metaverse is essentially a process of virtualization and digitalization of the real world, which requires a large number of transformations in content production, economic systems, user experiences, and real-world content. However, the development of the metaverse is gradual and is finally formed by the continuous integration and evolution of numerous tools and platforms supported by shared infrastructure, standards, and protocols. It provides an immersive experience based on extended reality technology, generates a mirror image of the real world based on digital twin technology, builds an economic system based on blockchain technology, closely integrates the virtual world and the real world in economic systems, social systems, and identity systems, and allows each user to carry out content production and world editing. In order to better experience the virtual world, a production system is needed to produce virtual digital humans. Summary of the Invention

[0003] The purpose of the present invention is to provide a metaverse virtual digital human production system to solve the problems raised in the above background art.

[0004] To achieve the above purpose, the present invention provides the following technical solutions:

[0005] A metaverse virtual digital human production system, the system includes a scanning module, a BlendShape splitting module, a model Capture module, an ecological process processing module, and a physiological texture processing module. The scanning module scans the extreme expressions of a person, the BlendShape splitting module splits the BlendShape through AI processing, the model Capture module drives the model with high-precision results provided by some third parties, the ecological process processing module processes the real-person scanning data, that is, the reconstruction of the model and the reconstruction of the texture mapping. The physiological texture processing module, after completing the processing of the real-person scanning data, fine-tunes the surface of the virtual digital human model, adds the corresponding physiological textures of the human body to the surface skin of the model, so that the virtual digital human has the same fingerprint label as the real person.

[0006] Preferably, the scanning module scans the extreme expressions of a person. The person makes those expressions in a light field device and is photographed, and then the model cleaning work is carried out through an AI algorithm.

[0007] Preferably, the BlendShape splitting module splits BlendShape through AI, splits FACS into 60 extreme expressions, and these 60 expressions can be non-linearly combined to form any expression. It can be disassembled again, and hundreds of BlendShapes can be disassembled according to user needs. During the disassembly process, manual correction is required for accuracy. For example, a downward movement of the lips is a combination of several BlendShapes, but there is a BlendShape that is concave. Then this concave one needs to be manually corrected. Because problems will occur when these extreme expressions are combined with other expressions. When such problems occur, manual correction is required. Therefore, the automatic disassembly algorithm still needs to evolve. We can simply divide the expression texture maps into expressions and micro-expressions. Structural changes in expressions need to be done on the model. If there are some texture changes due to structural changes, such as squeezing, they need to be made into texture maps. The texture map is the WinkleMap, which is a Normal expression texture map that can be switched between different expressions. There are usually five or six Winkle Maps on a face. After scanning, the eyebrows and some eyelashes need to be removed from the map because they are noise and not needed during retopology and model repair; but these things are used during rendering. Therefore, they need to be deleted first and then added back later. In addition to the Winkle Map, there is also a BloodFlow Map (blood flow map). When your face wrinkles, due to squeezing, the blood flow map in a certain place will change, and then the color of this place will become darker. This is manifested in micro-expressions and is also included in the texture map.

[0008] Preferably, the model Capture module uses high-precision results provided by some third parties to drive the model.

[0009] Preferably, the ecological process processing module processes the real-person scanned data, which is the reconstruction of the model and the reconstruction of the texture map. It precisely corrects the scanned model and can achieve a reasonable and easily animated model reconstruction according to the user-defined standards.

[0010] Preferably, the physiological texture processing module, after completing the processing of the real-person scanned data, fine-tunes the surface of the virtual digital human model, adds the corresponding physiological textures of the human body to the surface skin of the model, so that the virtual digital human has the same fingerprint label as a real person. It uses three cameras, which are responsible for color, optical, and high-speed photography respectively, to photograph the skin texture of the displayed person, and then adds the skin texture to the skin of the virtual digital human.

[0011] Compared with the prior art, the present invention provides a metaverse virtual digital human production system, which has the following beneficial effects:

[0012] The system of the present invention includes a scanning module, a BlendShape splitting module, a model Capture module, an ecological process processing module, and a physiological texture processing module. The scanning module scans the extreme expressions of a person. The BlendShape splitting module splits the BlendShape through AI. The model Capture module uses high-precision results provided by some third parties to drive the model. The ecological process processing module processes the real-person scanning data, that is, the reconstruction of the model and the reconstruction of the texture map, and physiological texture processing. After processing the real-person scanning data, the model is fine-tuned. And during the production process, due to the addition of the physiological texture processing module, three types of cameras are used, which are respectively responsible for color, optical, and high-speed photography, so as to photograph the skin texture of the displayed person, and then add the skin texture to the skin of the virtual digital person, add the physiological texture corresponding to the human body to the surface skin of the model, make the virtual digital person have the same fingerprint label as the real person, make the digital virtual person and the person in the display have common physiological characteristics, and have its unique representativeness, avoiding a certain degree of similarity with the digital virtual persons of other people. Specific embodiments

[0013] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0014] The present invention provides a technical solution:

[0015] A metaverse virtual digital human production system, the system includes a scanning module, a BlendShape splitting module, a model Capture module, an ecological process processing module, and a physiological texture processing module. The scanning module scans the extreme expressions of a person. The BlendShape splitting module splits the BlendShape through AI. The model Capture module uses high-precision results provided by some third parties to drive the model. The ecological process processing module processes the real-person scanning data, that is, the reconstruction of the model and the reconstruction of the texture map, and physiological texture processing. After processing the real-person scanning data, the model is fine-tuned, and the physiological texture corresponding to the human body is added to the surface skin of the model, so that the virtual digital person has the same fingerprint label as the real person.

[0016] The scanning module scans the extreme expressions of a person. The person makes those expressions in the light field device and is photographed, and then the model cleaning work is carried out through an AI algorithm.

[0017] The BlendShape splitting module uses AI to split BlendShapes, splitting FACS into 60 extreme expressions. These 60 expressions can be non-linearly combined to form any expression, and they can be disassembled again to disassemble hundreds of BlendShapes according to user needs. During the disassembly process, manual correction is required for accuracy. For example, a downward movement of the lips is a combination of several BlendShapes, but there is one BlendShape that is concave, and this concave one needs to be manually corrected. Because problems will occur when these extreme expressions are combined with other expressions. When such problems occur, manual correction is required. Therefore, the automatic splitting algorithm still needs to be evolved. We can simply divide expression maps into expressions and micro-expressions. Structural changes in expressions need to be made on the model. For some texture changes due to structural changes that result in squeezing, they need to be made into maps. The map is the Winkle Map, which is a Normal expression map and can be switched between different expressions. There are often five or six Winkle Maps on one face. After scanning, the eyebrows and some eyelashes need to be removed from the map because they are noise and not needed during retopology and model repair; but these things are used during rendering. Therefore, they need to be deleted first and then added back later. In addition to the Winkle Map, there is also a Blood FlowMap (blood flow map). When your face wrinkles, due to squeezing, the blood flow map in a certain place will change, and then the color of this place will become darker. This is manifested in micro-expressions and is also included in the map.

[0018] The model Capture module uses high-precision results provided by some third parties to drive the model.

[0019] The ecological process processing module processes real-person scanned data, which is the reconstruction of the model and the reconstruction of the map. It precisely corrects the scanned model and can achieve a reasonable and easily animated model reconstruction according to the user-defined standards.

[0020] Physiological texture processing: After processing the real-person scanned data, the model is fine-tuned. The surface skin of the model is added with the corresponding physiological textures of the human body, enabling the virtual digital human to have the same fingerprint label as a real person. Three cameras are used, responsible for color, optics, and high-speed photography respectively, to photograph the skin texture of the displayed person, and then the skin texture is added to the skin of the virtual digital human. The surface skin of the model is added with the corresponding physiological textures of the human body, enabling the virtual digital human to have the same fingerprint label as a real person, making the digital virtual human and the displayed person have common physiological characteristics and unique representativeness, avoiding a certain degree of similarity with the digital virtual humans of other people.

[0021] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A metaverse virtual digital human production system, characterized in that: The system includes a scanning module, a BlendShape splitting module, a model Capture module, an ecological process processing module, and a physiological texture processing module. The scanning module scans the extreme expressions of a person. The BlendShape splitting module splits BlendShape through AI. The model Capture module uses high-precision results provided by some third parties to drive the model. The ecological process processing module processes the real-person scanning data, which is the reconstruction of the model and the reconstruction of the texture. The physiological texture processing module, after completing the processing of the real-person scanning data, fine-tunes the surface of the virtual digital human model, adds the corresponding physiological textures of the human body to the surface skin of the model, so that the virtual digital human has the same fingerprint label as a real person.

2. A metaverse virtual digital human production system according to claim 1, characterized in that: The scanning module scans the extreme expressions of a person. The person makes those expressions in a light field device and is photographed, and then the model cleaning work is carried out through an AI algorithm.

3. A metaverse virtual digital human production system according to claim 1, characterized in that: The BlendShape splitting module splits BlendShape through AI.

4. A metaverse virtual digital human production system according to claim 1, characterized in that: The ecological process processing module processes the real-person scanning data, which is the reconstruction of the model and the reconstruction of the texture. It precisely corrects the scanned model and can realize a reasonable and easy-to-animate model reconstruction according to the user's own defined standards.

5. A metaverse virtual digital human production system according to claim 1, characterized in that: The physiological texture processing module, after completing the processing of the real-person scanning data, fine-tunes the surface of the virtual digital human model, adds the corresponding physiological textures of the human body to the surface skin of the model, so that the virtual digital human has the same fingerprint label as a real person. Three cameras are used, respectively responsible for color, optics, and high-speed photography, so as to photograph the skin texture of the displayed person, and then add the skin texture to the skin of the virtual digital human.