The application discloses a kind of
virtual image model construction method and
system based on image clone, it is related to
computer graphics field, including: from real-time audio and video
stream of real person, the
multimodal data of fusion vision, voice and action are obtained;Respectively extract
facial expression, voice and action emotional data;Adopt
dynamic time warping algorithm to calculate the similarity between each
modal emotional data, generate the
modal correlation mapping matrix of quantitative
modal synchronization relationship;Adopt
machine learning model to construct emotional
label to limb posture
expression action mapping model;According to the deviation of expression and action, the feature is adjusted back;Finally, based on the feature after adjustment, mapping matrix and mapping model, generate the
virtual image that expression, voice and action are accurately aligned on time axis.The application establishes the quantitative correlation and feedback adjustment mechanism between modal, significantly improves the real sense and coordination of
virtual image when
cloning real person in multidimensional emotional expression.