Realistic virtual human multi-modal interaction implementation method based on UE4

Abstract

The invention discloses a realistic virtual human multi-modal interaction implementation method based on UE4. The method comprises the steps of resource manufacturing, resource assembling and functionmanufacturing. The system comprises a resource making module used for role model making, facial expression BlendShape making, skeletal skin binding, action making, map making and material adjustment;a resource assembly module which is used for carrying out scene building, light design and UI interface building; and the function making module is used for identifying voice input of the user, carrying out intelligent answering according to the input, playing voice, lip animation, expression animation and body movement and reflecting interaction multi-modality. The module specifically comprisesa voice recognition module, an intelligent question and answer module, a Chinese natural language processing module, a voice synthesis module, a lip animation module, an expression animation module and a body movement module. The system has affinity similar to that of a real person and can be accepted by a user more easily; the interactive habit of human beings is better met, and the application has a wider popularization range; therefore, the application is truly intelligent, and the response of the application is more in line with human logic.

Description

technical field [0001] The invention relates to the field of computer software, in particular to a UE4-based method and system for realizing multi-modal interaction of realistic virtual humans. Background technique [0002] The current digital virtual human technology research is very hot. As the next generation of intelligent human-computer interaction, digital virtual human has a very wide range of commercial application scenarios, such as virtual idols, virtual actors, virtual anchors, etc. will be used in games, film and television, financial services, Industries such as education and medical care are on the ground. [0003] However, most digital avatars are presented as cartoon characters, and only have simple expressions or lip expressions, which cannot be applied in most fields. Existing game engines, especially the next-generation game engines represented by UE4, have rendering effects that are difficult for human eyes to distinguish between real and fake in terms o...

AI Technical Summary

Problems solved by technology

[0003] However, most digital avatars are presented as cartoon characters, and only have simple expressions or lip expressions, which cannot be applied in most fields.

Existing game engines, especially the next-generation game engines represented by UE4, have rendering effects that are difficult for human eyes to distinguish between real and fake in terms of realistic rendering of characters and scenes, but lack of voice input, professional fields or general-purpose question-and-answer systems. Therefore, it is impossible to give an answer response that conforms to human behavior habits

Images