Learning system for AI-based weight map generation models for virtual human simulation

Abstract

We provide a learning system that can automatically generate a weight map for a virtual human. [Solution] The system includes a video processing unit that separates a 2D reference object from a frame of 2D video data; an object model processing unit that pre-sets a 3D object model of the reference object; a pose estimation unit that estimates 3D pose data of the reference object; an artificial intelligence model which includes a map generation model that takes 3D pose data as input and generates a weight map; a physical simulator that performs a physical simulation using the weight map and the 3D object model to generate a 3D object; a 2D object extraction unit that extracts a 2D object from the generated 3D object; and a variable adjustment unit that adjusts the internal variables of the map generation model using a loss function and adjusts them using the loss between the reference object separated from the frame and the 2D result object.

Claims

[Claim 1] A learning system for an artificial intelligence-based weight map generation model for virtual human simulation, A video processing unit that separates a 2D reference object from a frame of 2D video data, An object model processing unit that pre-sets a three-dimensional object model of a reference object included in the two-dimensional video data, A pose estimation unit that estimates the three-dimensional pose data of the reference object from the aforementioned two-dimensional reference object, An artificial intelligence model, a map generation model that takes 3D pose data as input and generates a weight map, A physics simulator that generates a 3D object by performing a physical simulation using the 3D object model that takes a 3D pose and the weight map, A 2D object extraction unit that extracts a 2D object (hereinafter referred to as a 2D result object) from the aforementioned 3D object, A learning system for an artificial intelligence-based weight map generation model for virtual human simulation, characterized by including a variable adjustment unit that adjusts the internal variables of the map generation model using a loss function and adjusts them using the loss between the two-dimensional reference object and the two-dimensional result object. [Claim 2] The aforementioned reference object includes a number of wearable objects, The three-dimensional object model includes a number of garment models corresponding to the garment object, The learning system for an artificial intelligence-based weight map generation model for virtual human simulation according to claim 1, characterized in that the variable adjustment unit calculates a loss by finding the difference between the two-dimensional garment object of the two-dimensional reference object and the corresponding garment object of the two-dimensional result object. [Claim 3] The learning system for an artificial intelligence-based weight map generation model for virtual human simulation according to claim 1, characterized in that the map generation model comprises an encoder that encodes the three-dimensional pose data to generate latent variables, and a decoder that generates the weight map using the latent variables obtained by the encoder. [Claim 4] The aforementioned map generation model consists of at least two or more components. The learning system for an artificial intelligence-based weight map generation model for virtual human simulation according to claim 2, characterized in that each map generation model is a model that generates weight maps for each of at least two or more wearable models. [Claim 5] The learning system for an artificial intelligence-based weight map generation model for virtual human simulation according to claim 2, characterized in that the wearable object is an object for an object to be worn on the body of the reference object and includes one or more of hair, clothing, and accessories. [Claim 6] The learning system for an artificial intelligence-based weight map generation model for virtual human simulation according to claim 1, characterized in that the reference object is a virtual human.

Images