Learning system for artificial intelligence-based weight map generation model for virtual human simulation

Abstract

To provide a learning system capable of automatically generating a weight map for a virtual human. [Solution] The system includes a video processing unit that separates a 2D reference object from a frame of 2D video data, an object model processing unit that pre-sets a 3D object model of the reference object, a pose estimation unit that estimates 3D pose data of the reference object, an artificial intelligence model that receives the 3D pose data as input and generates a weight map, a physics simulator that performs a physics simulation using the weight map and the 3D object model to generate a 3D object, a 2D object extraction unit that extracts a 2D object from the generated 3D object, and a variable adjustment unit that adjusts the internal variables of the map generation model using a loss function and adjusts using the loss between the reference object separated from the frame and the 2D result object.

Description

[Technical Field]

[0001] The present invention relates to a learning system for an artificial intelligence-based weight map generation model for virtual human simulation, which generates a weight generation model that generates a weight map based on a 3D pose, utilizes 2D video data of a virtual human as learning data, estimates a 3D pose from the 2D video, generates a weight map for the estimated 3D pose, performs physical simulation, and trains the weight generation model using the loss between the 2D result and the original data. [Background technology]

[0002] Physics simulation is typically applied to 3D mesh-shaped virtual humans to achieve realistic movement and interaction in a virtual environment. Physics simulation includes character movement, collision handling, natural cloth and hair movement, muscle and skin deformation, and fluid and environmental interaction.

[0003] Motion simulation expresses natural movements using joint rotation, balance maintenance, inverse kinematics, etc. Motion simulation calculates the effects of external forces to make movements more realistic.

[0004] Collision detection and response uses bounding volume hierarchies (BVH) and voxelization to calculate collisions and simulate post-collision movement (collision response). Clothes and hair are recreated using techniques such as particle-based simulation, mass-spring systems, and FEM (finite element method) based on physical attributes such as elasticity, inertia, and air resistance. Muscle and skin deformation is achieved by adding a muscle layer to skeletal animation or by combining soft body dynamics and blend shape technology to realize skin texture and elasticity. Interaction with fluids is calculated using smoothed particle hydrodynamics (SPH). Interaction with environmental elements is calculated using physical effects such as gravity, friction, and wind to enhance the sense of realism. These elements provide both performance optimization and physical realism through real-time physics engines such as Unity's PhysX and Unreal Engine's Chaos Physics, contributing to maximizing the physical realism of virtual humans in various fields such as game development, film and animation, virtual reality and augmented reality, and medical training.

[0005] Meanwhile, in 3D graphics and physics simulation, weight maps are an important tool for precisely controlling the influence of physical properties on specific regions [1, 2]. Weight maps assign weights to each point on an object to locally adjust physical effects. This allows weight maps to emphasize or attenuate physical responses only in specific regions. Weight maps are used in simulating various materials, including cloth, hair, skin, and fluids. Weight maps not only precisely represent physical properties, but also provide an efficient calculation structure.

[0006] Weight maps provide the ability to locally adjust physical properties, precisely controlling the movement and deformation of specific regions [Non-Patent Document 1]. For example, in cloth simulation, stiff, restricted-motion areas like collars can be set differently from flexible, mobile areas like sleeves and skirts. In hair simulation, the roots can be fixed, while the ends can be adjusted to move more freely. Adjusting local properties in this way allows for desired control of physical effects, more precisely embodying the physical response of each part.

[0007] Weight maps contribute to improving the efficiency of simulations. By lowering the weight of a specific area or setting it to 0, weight maps can reduce or eliminate physical calculations for that area. Therefore, weight maps can save computational resources even in complex mesh structures and large scenes. For example, by applying physical effects only to parts that move frequently without physically simulating the entire character's clothing, you can achieve realistic expression while preventing performance degradation.

[0008] Weight maps allow users to custom define the physical effects they want, allowing them to precisely manipulate the movement and deformation of an object. Graphics software provides tools that allow users to paint weights directly or define them using mathematical formulas. This allows users to precisely tune the way clothes unfold naturally or the way muscles contract in specific areas.

[0009] Weight maps are used to naturally transfer different physical attributes. For example, in a character's clothing, they can smoothly connect the physical transition between a rigid belt and a flexible groin, minimizing visual discomfort. They can also be used to set weights for a character's muscle simulation so that muscle movement is naturally transferred to the bones and skin.

[0010] Weight maps are widely used in various 3D graphics tasks, not just physics simulation. In character rigging, weight maps are used for skin binding, helping skeleton movement to be transferred naturally to the mesh. In texture painting, weight maps can be used to add or emphasize physical effects to specific parts of the texture. In rigid body and fluid simulations, weight maps can be used to adjust fluid flow or impact strength in specific areas, achieving more precise interactions.

[0011] Weight maps also play an important role in enhancing user interaction in interactive environments. For example, in virtual reality (VR) and games, weight maps can be used to dynamically change the physical response of objects in certain situations. For example, weight maps can be used to create effects such as a character's clothes flapping in the wind or hair moving more vigorously in response to a specific impact.

[0012] For example, Figure 1 is an illustrative diagram of a weight map of a 3D model according to the prior art, where (a) of Figure 1 shows the 3D model, (b) of Figure 1 shows the weight map of the skirt, and (c) of Figure 1 shows the texture of the skirt.

[0013] As shown in (a) to (c) of FIG. 1, the weight map is represented by 8-bit data, with white representing areas that move frequently and black representing areas that do not move frequently.

[0014] In particular, weight maps for objects with complex surfaces, such as a human body, hair, or clothing, are very complex. In conventional techniques, these weight maps are generated manually, i.e., the weight map is modified while manually moving the virtual human. This work takes a considerable amount of time.

[0015] Therefore, there is a need for techniques to more efficiently generate weight maps for physics simulations. [Prior art documents] [Patent documents]

[0016] [Patent Document 1] Korean Patent Registration No. 10-2416523 [Non-patent literature]

[0017] [Non-Patent Document 1] https: / / pixcap.com / glossary / weight-mapping [Non-patent document 2] https: / / www.avalab.org / avastar / features / rigging-and-weighting / [Non-patent document 3] https: / / people.csail.mit.edu / liyifei / publication / diffcloth / [Non-patent document 4] https: / / zero10.ar / blog / the-evolution-of-cloth-dynamics-for-virtual-try-on-experiences-in-ar [Non-patent document 5] Ladislav Kavan, Dan Gerszewski, Adam W. Bargteil, Peter-Pike Sloan. “Physics-Inspired Upsampling for Cloth Simulation in Games”, SIGGRAPH '11: ACM SIGGRAPH 2011 papers Article No.: 93, Pages 1-10 Summary of the Invention [Problem to be solved by the invention]

[0018] The object of the present invention is to solve the problems described above and to provide an artificial intelligence-based weight map generation model learning system for virtual human simulation, which generates a weight generation model that generates a weight map based on a 3D pose, uses 2D video data of a virtual human as learning data, estimates a 3D pose from the 2D video, generates a weight map for the estimated 3D pose, performs physical simulation, and uses the loss between the 2D result and the original data to train the weight generation model.

[0019] Another object of the present invention is to provide an AI-based weight map generation model learning system for virtual human simulation, which generates surface components of a virtual human using a weight map, defines the degree of movement of the object surface using the generated weight map, and performs physical simulation. [Means for solving the problem]

[0020] To achieve the above object, the present invention relates to a learning system for an artificial intelligence-based weight map generation model for virtual human simulation, which includes: a video processing unit that separates a 2D reference object from a frame of 2D video data; an object model processing unit that pre-sets a 3D object model of the reference object included in the 2D video data; a pose estimation unit that estimates 3D pose data of the reference object from the 2D reference object; an artificial intelligence model, a map generation model that receives the 3D pose data and generates a weight map; a physics simulator that generates a 3D object by performing a physics simulation using the 3D object model that takes a 3D pose and the weight map; a 2D object extraction unit that extracts a 2D object (hereinafter referred to as a 2D result object) from the 3D object; and a variable adjustment unit that adjusts internal variables of the map generation model using a loss function and adjusts using the loss between the 2D reference object and the 2D result object.

[0021] The reference object includes a plurality of wearing clothing objects, the 3D object model includes a plurality of wearing clothing models corresponding to the wearing clothing objects, and the variable adjustment unit calculates a loss by obtaining a difference between the 2D wearing clothing objects of the 2D reference object and the corresponding wearing clothing objects of the 2D result object.

[0022] The map generation model is composed of an encoder that encodes the 3D pose data to generate latent variables, and a decoder that generates the weight map using the latent variables obtained by the encoder.

[0023] The map generation model is configured to include at least two or more, and each map generation model is a model that generates a weight map for each of at least two or more wearing garment models.

[0024] The wearable object is an object for an object worn on the body of the reference object, and includes one or more of hair, clothing, and accessories.

[0025] The reference object is a virtual human. [Effects of the Invention]

[0026] According to the system of the present invention, an AI model for generating a weight map is created and trained, thereby automatically generating a weight map for physical simulation of a virtual human.

[0027] In addition, the system according to the present invention allows for more efficient learning by training a weight map generation model using 2D video data, thereby significantly reducing the amount of calculation required for training and improving the speed. [Brief explanation of the drawings]

[0028] [Figure 1] 1A and 1B are diagrams illustrating weight maps of a 3D model according to the prior art, showing (a) a 3D model, (b) a weight map of a skirt, and (c) a texture of the skirt. [Figure 2] 1 is a diagram illustrating an example of the configuration of an entire system for implementing the present invention; [Figure 3] 1 is a block diagram of a learning system for an artificial intelligence base weight map generation model for virtual human simulation according to a first embodiment of the present invention; [Figure 4] 10A and 10B show results of a physical simulation using a weight map according to an embodiment of the present invention, where (a) the results are obtained using a weight map that is currently being trained, and (b) the results are obtained using a trained weight map. [Figure 5] FIG. 10 is a block diagram showing the configuration of a learning system for an artificial intelligence base weight map generation model for virtual human simulation according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0029] Hereinafter, specific details for carrying out the present invention will be described with reference to the drawings.

[0030] In addition, in describing the present invention, the same parts will be given the same reference numerals and repeated description thereof will be omitted.

[0031] First, an example of the configuration of the entire system for carrying out the present invention will be described with reference to FIG.

[0032] As shown in FIG. 2(a), the learning system (hereinafter referred to as the learning method) for an artificial intelligence-based weight map generation model for virtual human simulation according to the present invention is implemented by a program system on a computer terminal 10, which learns a model for generating a weight map for virtual human simulation.

[0033] That is, the learning method is performed in a program system 30 on a computer terminal 10 such as a PC, a smartphone, or a portable PC. In particular, the learning method is configured in a program system and installed and executed on the computer terminal 10. The learning method provides a service of learning a model that generates a weight map for simulating a virtual human using hardware or software resources of the computer terminal 10.

[0034] Here, the computer terminal 10 includes a processor, a memory, etc. The processor executes a learning method and stores or retrieves virtual human data, intermediate result data, final result data, etc. from the memory. Here, the processor includes a central processing unit (CPU), a graphics processing unit (GPU), an AI accelerator, a neural processing unit, an AI chip, etc.

[0035] In another embodiment, as shown in FIG. 2(b), the learning method is implemented in a server-client system consisting of a learning client 30a on a computer terminal 10 and a learning server 30b.

[0036] Meanwhile, the learning client 30a and learning server 30b are implemented using a typical client-server configuration. That is, the overall system functions can be shared depending on the client's performance and the server's communication volume. Although the following description is of a learning system, it can be implemented in various sharing forms depending on the server-client configuration.

[0037] Meanwhile, in another embodiment, the learning method can be implemented by being configured as a program and running on a general-purpose computer, or by being configured as a single electronic circuit such as an ASIC (application-specific integrated circuit). Alternatively, it can be developed as a dedicated computer terminal that processes exclusively for learning a model that generates a weight map for virtual human simulation. Other possible forms are also possible.

[0038] Next, a learning system for an artificial intelligence base weight map generation model for virtual human simulation according to a first embodiment of the present invention will be described with reference to FIGS.

[0039] As shown in FIG. 3, first, the video processing unit 31 generates learning data using two-dimensional video data.

[0040] The 2D video data serves as basic data for extracting the movement and physical characteristics of a reference object (such as a virtual human). The 2D video data contains sufficient physical simulation phenomena (movements) for learning. That is, the 2D video data contains various angles and phenomena for physical simulation.

[0041] Two-dimensional video data is composed of a series of consecutive frames, where each frame is composed of a two-dimensional image, which represents the continuous movement (action) of a reference object (such as a virtual human).

[0042] The video processing unit 31 constructs training data from a large number of frames of two-dimensional video data. That is, one training data is generated from each frame and trained. The map generation model 34 is trained using the large number of training data constructed in this way.

[0043] The video processing unit 31 also removes the background from the frame images of the learning data to separate (extract) objects (or reference objects, object images). Reference objects include bodies (such as the body of a virtual human) and objects worn on the body (such as hair, clothing, and accessories). The reference objects are humans or animals (such as monkeys and dogs).

[0044] The video processing unit 31 can also separate wearable objects (hair, clothes, accessories, etc.) of the reference object. For example, a skirt, a coat, or hair worn on a human body can be separated into a skirt object (image), a coat object (image), or a hair object (image), respectively. A wearable object is an object worn on the body of the reference object.

[0045] The isolated reference object or wearable object is a 2D object or a 2D image. The isolated reference object or wearable object is used to train a weight map for that object.

[0046] Preferably, the video processing unit 31 selects key frames from the 2D video data and uses the selected key frames as training data. By selecting and processing only key frames containing meaningful motion and information without processing all frames of the video data, the efficiency of data processing can be improved.

[0047] Also, preferably, the two-dimensional video data is multi-point video data.

[0048] Next, the object model processing unit 32 pre-establishes a three-dimensional object model corresponding to the reference object contained in the two-dimensional video data.

[0049] The 3D object model is a predefined basic model or standard model, which is a model similar to a reference object in the 2D video data and is predefined.

[0050] A 3D object model is a 3D model created using a modeling method used in this field to represent 3D objects. A 3D object model is a model that naturally represents physical movement and interaction based on animation data and weight maps. For example, a 3D object model is composed of 3D meshes.

[0051] Furthermore, the 3D object model of the reference object is composed of a model that represents the reference object as a whole. Alternatively, the 3D object model of the reference object includes a 3D body model of the reference object and 3D models of objects (hair, clothing, accessories, etc.) worn by the body model (hereinafter referred to as wearable model). In other words, the entire model of the reference object is composed of the body model and all wearable models, or is composed of a model that integrates the body and wearables. These models are subjected to physical simulation.

[0052] The 3D object model also includes a 2D UV map that represents the texture. A UV map is a 2D map that represents the texture of a 3D model. When a 3D model is generated, the texture is created along with it. The 3D model of the 3D object model is converted into a UV map through UV mapping.

[0053] The UV map is composed of a map for the entire model of the reference object and / or a map for each wearable object model.

[0054] A weight map corresponding to the UV map of each model of the reference object is preset and initialized. The initialization values ​​are set to known or arbitrary values. The weight map consists of a two-dimensional map of the same size as the UV map.

[0055] Furthermore, the object model processing unit 32 selects a weight map of a 3D object model in order to train the map generation model 34. That is, it is possible to select a specific model from a large number of 3D object models (such as a whole model or a large number of clothing models) and set a weight map for the selected model.

[0056] Meanwhile, the object model processing unit 32 initializes the weight map of the map generation model 34 using the set weight map, UV map, etc. That is, to determine whether to generate a weight map for a certain part (model) of the reference object, the object model processing unit 32 transmits the UV map and the initial weight map to the map generation model 34.

[0057] The weight maps to be learned are weight maps of the entire model or models of clothing worn by a human (hair, clothing, accessories, etc.). In other words, weight maps of physically deformable components (object models) are selected.

[0058] A pose estimation unit 33 then estimates 3D pose data of the reference object from the frame of the 2D video or from the isolated reference object in that frame.

[0059] That is, based on the visual data of the 2D video frame, the joint positions and relationships of the reference object (such as a human) are calculated and converted into 3D coordinate information, which provides the motion data of the reference object in 3D space.

[0060] To estimate 3D pose data from 2D images (frames), we use conventional methods such as the SMPL-X deep learning model and the OpenPose library.

[0061] Preferably, if the 2D video is provided as a multi-temporal video, more precise 3D coordinates of the 3D pose can be estimated [Patent Document 1].

[0062] Next, the map generation model 34 is an artificial intelligence model that receives three-dimensional pose data and generates a weight map.

[0063] A weight map is a map that indicates the degree of change in the surface components of a reference object or a wearable object. The generated weight map can be used to define the degree of movement of the object surface and perform physical simulation.

[0064] The map generation model 34 includes an encoder 341 that encodes 3D pose data (human movement data) to generate latent variables, and a decoder 342 that generates a weight map based on the latent variables. The map generation model 34 can be implemented as an existing artificial intelligence model such as a convolutional neural network (CNN), a variational auto-encoder (VAE), an auto-encoder (AE), a graph neural network (GNN), or a transformer.

[0065] The encoder 341 encodes the structural information of human motion through the trained model and generates latent variables that can be utilized in the next stage. These latent variables are trained with information that can sufficiently contain the features of the motion.

[0066] The encoder 341 receives three-dimensional pose data (or three-dimensional data) as input and outputs latent variables (such as N-dimensional vectors) generated from the three-dimensional data. That is, the encoder 341 is configured with a neural circuit that generates latent variables from an N-dimensional coordinate set.

[0067] The decoder 342 also generates a weight map using the latent variables obtained from the encoder 341. The decoder 342 calculates appropriate weights for each vertex based on the physical attributes and movement characteristics learned by the AI ​​model. The decoder 342 then outputs a weight map that can be used for physics simulation.

[0068] The decoder 342 is configured with a neural circuit that receives latent variables (such as an N-dimensional vector) as input and generates a two-dimensional weight map.

[0069] The neural circuits of the encoder 341 and decoder 342 are composed of hidden layers of various numbers and shapes depending on the definition method of the 3D joints. Furthermore, the internal variables of the neural circuits are adjusted (optimized) through learning.

[0070] In particular, the decoder 342 starts learning using the initial weight map. That is, the decoder 342 performs learning by changing the initial weight map from latent variables with reference to the UV map transmitted from the object model processing unit 32. The UV map (texture) has coordinates that are matched one-to-one with the mesh of the 3D model that the texture covers. That is, since the UV map contains mesh position information, referring to the UV map means referring to the position of the corresponding mesh.

[0071] Meanwhile, the generated weight maps are used for physical simulation of 3D object models. The weight maps assign different physical properties to each part of the mesh. After training, the weight maps generated by the trained model contribute to the realization of the 3D model.

[0072] Next, the physics simulator 35 performs a physics simulation based on the generated weight map and the 3D object model to generate a 3D object. In particular, the physics simulation is performed by applying the estimated pose data. Through the physics simulation, a 3D object model (or a 3D object) taking a 3D pose is obtained.

[0073] The physics simulator 35 simulates the movement of a 3D object (a 3D reference object or a 3D wearable object) in a virtual environment. Here, the physics simulator 35 uses a weight map to define the degree of movement of the 3D object's surface and performs the simulation. The weight map is a map that indicates the degree of change in the surface components of the 3D object.

[0074] That is, the physical simulator 35 performs a physical simulation to obtain a three-dimensional object that assumes the estimated pose, and performs the physical simulation by applying the weight map generated by the map generation model 34.

[0075] Physics simulation is used to accurately represent various physical characteristics of virtual humans, such as their movements, clothing, skin, and muscles [3, 4, 5]. Weight maps play a key role in controlling the physical responses of specific regions.

[0076] In other words, the external surface (clothing) of the virtual human changes depending on the movement (pose data) of the 3D virtual human. Here, the weight map provides the basic information that determines how each mesh surface changes. If the color of the weight map is white, it can move more, and if it is dark, it can move less. In other words, by defining the weight map, it is possible to mimic the changes in actual clothing.

[0077] The results of the physical simulation are output as a 3D object. The result is a natural movement of the clothing worn by the 3D virtual human. Depending on the results of the physical simulation, the entire 3D object can be obtained, or the 3D clothing object can be obtained separately.

[0078] Meanwhile, the output 3D object is configured to include 3D clothing objects corresponding to the clothing models of the 3D object model. That is, if the 3D object model includes multiple clothing models, the corresponding clothing objects will also be included in the results of the physical simulation.

[0079] Next, the 2D object extraction unit 36 ​​extracts a two-dimensional object (hereinafter referred to as a result object) from the physical simulation result.

[0080] 2D rendering is performed on the results of the physical simulation to obtain a 2D image. Rendering is performed using the texture of the UV map. 3D objects are converted so that they are output on a 2D plane. Finally, visual data that can be viewed by the user is generated.

[0081] The 3D object resulting from the physical simulation is projected onto a 2D plane to obtain a 2D object (or result object). In particular, the 2D object is rendered at the same time as the original video frame.

[0082] Next, the variable adjustment unit 37 adjusts the internal variables of the map generation model 34 using a loss function, and uses the loss between the reference object in the original 2D frame and the 2D object extracted by the physical simulation (the 2D result object).

[0083] That is, a loss function is used to update the internal variables (weights of the map generation model or neural circuit that constitute the weight map generation process) and biases of the map generation model 34. The loss function is set as the difference between the 2D scene rendered using the weight map generated by the artificial intelligence model and the actual scene (e.g., actual measurements or ideal results).

[0084] The variable adjustment unit 37 uses a loss function to update the internal variables of the map generation model 34, thereby training the map generation model 34. The weight map becomes increasingly more refined through the internal variables (weights) of the artificial intelligence model (or trained map generation model) updated by the loss function. This makes the generated physical simulation results closer to reality.

[0085] In particular, the loss is calculated by finding the difference between the 2D reference object of the original 2D frame data and the 2D result object of the physical simulation. The loss can be calculated either between the whole objects or between the corresponding wearable objects.

[0086] In this way, the system of the present invention can generate precise weight maps based on 2D video, and can use these weight maps to generate 3D virtual human models that can be precisely simulated physically.

[0087] Figure 4(a) shows the results of a physics simulation using a weight map during training, while Figure 4(b) shows the results of a physics simulation using a trained weight map.

[0088] Next, a learning system for an artificial intelligence base weight map generation model for virtual human simulation according to a second embodiment of the present invention will be described with reference to FIG.

[0089] As shown in FIG. 5, the second embodiment of the present invention comprises a video processing unit 131, an object model processing unit 132, a pose estimation unit 133, a map generation module 134, a physics simulator 135, a 2D object extraction unit 136, and a variable adjustment unit 137.

[0090] The components of the second embodiment of the present invention are the same as those of the first embodiment described above. However, the map generation module 134 of the second embodiment of the present invention is different. Only the components that differ from the first embodiment will be described below. For parts that are not described, please refer to the first embodiment.

[0091] 5, the map generation module 134 is composed of a number of map generation models 340. Each map generation model 340 is identical to the map generation model 34 of the first embodiment described above. That is, unlike the first embodiment, the second embodiment of the present invention is composed of at least two map generation models 34.

[0092] Each map generation model 340 generates a weight map corresponding to a specific model of the 3D object model (an entire model or multiple wearable models). Each map generation model 340 corresponds to a specific model and generates a weight map for the corresponding model.

[0093] That is, when generating weight maps for various regions (parts) of a 3D object model, multiple sets of UV maps related to the initial weight maps are constructed, and separate learning is performed for each set.

[0094] For example, weight maps can be generated separately for a skirt model and a court model. Two map generation models are constructed, such as a map generation model for the skirt model and a map generation model for the court model. In addition, UV maps or initial weight maps corresponding to the two (two models) 3D meshes are generated separately, and each map generation model is trained and used separately.

[0095] In other words, two learning rounds are performed for each initial weight map.

[0096] In addition, when calculating a loss function for a specific map generation model, the variable adjustment unit 37 calculates the difference between the object of the original frame data corresponding to the model and the resultant object as the loss. For example, when training a model that generates a weight map for a skirt model, the difference between the skirt of the original frame data and the skirt of the simulation result is calculated as the loss.

[0097] As a result, the present invention can realize realistic and physically rich movements and interactions of virtual humans, and can be used in various application fields such as games, movies, virtual reality, and medical simulations.

[0098] The invention made by the present inventor has been specifically explained using the above-mentioned examples, but it goes without saying that the present invention is not limited to the above-mentioned examples and can be modified in various ways without departing from the spirit of the invention. [Explanation of symbols]

[0099] 10 Computer Terminals 30 Learning System 30a Learning Client 30b Learning Server 31, 131 Video processing section 32, 132 Object model processing section 33, 133 Pose estimation part 34, 340 Map generation model 35, 135 Physics Simulator 36, 136 2D object extraction section 37, 37 Variable adjustment section 134 Map Generation Module 341 Encoder 342 decoder 40 databases 80 Network

Claims

1. 1. A system for learning an artificial intelligence based weight map generation model for virtual human simulation, comprising: a video processing unit for separating a two-dimensional reference object from a frame of two-dimensional video data; an object model processing unit that pre-establishes a three-dimensional object model of a reference object included in the two-dimensional video data; a pose estimation unit for estimating three-dimensional pose data of a corresponding reference object from the two-dimensional reference object; an artificial intelligence model, a map generation model that receives three-dimensional pose data and generates a weight map; a physics simulator that performs a physics simulation using the three-dimensional object model in a three-dimensional pose and the weight map to generate a three-dimensional object; a 2D object extraction unit that extracts a 2D object (hereinafter referred to as a 2D result object) from the 3D object; A learning system for an artificial intelligence base weight map generation model for virtual human simulation, comprising: a variable adjustment unit that adjusts internal variables of the map generation model using a loss function and adjusts them using losses between the two-dimensional reference object and the two-dimensional result object.

2. the reference object includes a number of wearable objects; the three-dimensional object model includes a number of wearing clothing models corresponding to the wearing clothing object; 2. The system of claim 1, wherein the variable adjustment unit calculates a loss by obtaining a difference between the two-dimensional clothing object of the two-dimensional reference object and the corresponding clothing object of the two-dimensional result object.

3. 2. The learning system of an artificial intelligence-based weight map generation model for virtual human simulation according to claim 1, wherein the map generation model comprises an encoder that encodes the 3D pose data to generate latent variables, and a decoder that generates the weight map using the latent variables obtained by the encoder.

4. The map generation model is configured into at least two or more, 3. The learning system for an artificial intelligence-based weight map generation model for virtual human simulation according to claim 2, wherein each map generation model generates a weight map for each of at least two or more garment models.

5. 3. The learning system of an artificial intelligence base weight map generation model for virtual human simulation according to claim 2, wherein the wearable object is an object for an object worn on the body of the reference object, and includes one or more of hair, clothing, and accessories.

6. 2. The learning system of claim 1, wherein the reference object is a virtual human.

Images