Techniques for unified physics-based character control through masked motion completion

Abstract

One embodiment of a method for animating a character includes receiving one or more goals specified in one or more modalities; generating, via a trained machine learning model and based on the one or more goals, a first action for a character to perform, wherein the trained machine learning model is trained to process inputs in multiple modalities; and causing the character to perform the first action in a computer-based environment or a physical environment.

Description

[0001] Cross-references to related applications This application claims priority to U.S. Patent Application No. 18 / 983,142, filed December 16, 2024, entitled "TECHNIQUES FOR UNIFIED PHYSICS-BASED CHARACTER CONTROL THROUGH MASKED MOTION INPAINTING," which claims the benefit of U.S. Provisional Patent Application No. 63 / 647,304, filed May 14, 2024, entitled "UNIFIED PHYSICS-BASED CHARACTER CONTROL THROUGH MASKED MOTION INPAINTING." The subject matter of that related application is hereby incorporated herein by reference. Technical Field

[0002] The embodiments of this disclosure generally relate to robotics, virtual character control, artificial intelligence and machine learning, and more specifically, to techniques for unified physical-based character control via masked motion completion. Background Technology

[0003] Character animation is the process of creating a series of different poses, expressions, and / or movements of a character that can be played sequentially. Character animation can be created in various ways, including hand-drawn animation, stop-motion animation, and computer-generated animation.

[0004] One approach to creating computer-generated character animation is through a manual process where an animator uses software to design and move a three-dimensional (3D) virtual model of the character in a way that allows the character to move within a given animation sequence. For example, an animator can use software to specify the position and orientation of joints associated with the character's head, torso, arms, etc., within several keyframes of a given animation. To create a full animation, the software can use kinematic modeling to calculate the position and orientation of the same joints present within frames between keyframes. The character can then be animated to move in a way that tracks the position and orientation of joints within keyframes and in-between frames.

[0005] One drawback of the methods described above for creating computer-generated character animation is that, generally, animators need to specify the position and orientation of all the character's joints within a keyframe to create the animation. Rare (if any) conventional software programs can automatically determine physically plausible positions and orientations of the character's joints that are not specified by the animator in any keyframe. Furthermore, the kinematic modeling used to calculate the position and orientation of joints within intercalary frames does not consider the forces that cause these joints to move. This can include motor forces that move the joints, as well as collisions / contacts that change the direction of motion. Instead, kinematic modeling only calculates the motion of the joints required to move between the position and orientation within the keyframe. Because forces are not considered, the resulting animation is often physically unrealistic, negatively impacting overall visual quality.

[0006] Another approach to creating computer-generated character animation is to train machine learning models, such as artificial neural networks, to output the positions and orientations of a character's joints across multiple frames, thereby generating an animation sequence. In these types of implementations, the machine learning model is typically trained from scratch or by retraining a previously trained, reusable model to output joint positions and orientations for a specific movement, such as walking or sitting. However, a drawback of this approach is that a machine learning model trained for a specific task (such as walking or sitting) cannot be used to generate animations of a character performing different movements (such as running or climbing stairs). In some cases, a machine learning model can be trained to take a latent vector of numbers as input and output different character joint positions and orientations, not limited to any particular movement. However, the numbers in the latent vector are not easily interpreted by animators, who may struggle to choose specific values ​​corresponding to a character's specific desired movement. Therefore, these types of machine learning models cannot be effectively controlled by animators and thus have limited practicality in generating animation.

[0007] As previously explained, there is a need in the art for more efficient techniques for generating physically plausible computer-based character animations. Summary of the Invention

[0008] One embodiment of this disclosure proposes a computer-implemented method for animate a character. The method includes receiving one or more targets specified in one or more modalities. The method further includes generating a first action to be performed by the character based on one or more targets via a trained machine learning model, wherein the trained machine learning model is trained to process inputs in multiple modalities. Furthermore, the method includes causing the character to perform the first action in a computer-based or physical environment.

[0009] Another embodiment of this disclosure proposes a computer-implemented method for training a machine learning model to animate a character. The method includes: performing one or more first operations using a set of motion records to train a first untrained machine learning model, thereby generating a first trained machine learning model configured to animate a character based on motion data as input. The method further includes: performing one or more second operations using a set of motion records and the first trained machine learning model to train a second untrained machine learning model, thereby generating a second trained machine learning model configured to animate a character based on user input.

[0010] Other embodiments of this disclosure include, but are not limited to, one or more computer-readable media, which include instructions for performing one or more aspects of the disclosed technology, and one or more computing systems for performing one or more aspects of the disclosed technology.

[0011] Compared to existing technologies, at least one technical advantage of the disclosed technology is that it allows physical or virtual characters to be animated to perform different movements without specifying all joints of the character in any number of frames of the animation. The animation generated using the disclosed technology is also physically more plausible than animations achievable using kinematic models that do not consider the forces causing the character's joint movement. Furthermore, the disclosed technology allows animators to effectively control the animation by specifying joint constraints, text descriptions, and / or the objects the character interacts with. These technical advantages represent one or more technical improvements relative to existing methods. Attached Figure Description

[0012] To enable a detailed understanding of the features described above in the various embodiments, the inventive concept briefly summarized above can be described in more detail with reference to the various embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only typical embodiments of the inventive concept and should not be construed as limiting the scope in any way, and that other equally effective embodiments exist.

[0013] Figure 1 The figure illustrates a computer-based system configured to implement one or more aspects of various embodiments; Figure 2 According to various embodiments Figure 1 A more detailed illustration of the machine learning server; Figure 3 According to various embodiments Figure 1 More detailed illustrations of the computing device; Figure 4 According to various embodiments Figure 1 A more detailed illustration of the model trainer; Figure 5 According to various embodiments Figure 4 A more detailed illustration of the reinforcement learning module; Figure 6 According to various embodiments Figure 4 A more detailed illustration of the supervised imitation learning module; Figure 7 According to various embodiments Figure 4 A more detailed diagram of the partially constrained controller; Figures 8A-8C The figure illustrates example simulated terrain that can be used during training of fully constrained and partially constrained controllers according to various embodiments; Figure 9 According to various embodiments Figure 1 More detailed illustrations of the control application; Figure 10 How to use according to various embodiments Figure 1 A more detailed diagram of the controller that controls the role is provided, which includes partial constraints. Figure 11A-11C The figure illustrates example motions generated according to various embodiments by using different modal control roles; Figure 12 A flowchart illustrating the steps of a method for training a fully constrained controller and training a partially constrained controller using a trained fully constrained controller, according to various embodiments. Figure 13 A flowchart illustrating the method steps for training a controller with partial constraints according to various embodiments is provided; and Figure 14 A flowchart illustrating the steps of a method for generating animations of a character given a sparse target, according to various embodiments. Detailed Implementation

[0014] In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to those skilled in the art that the described concepts can be practiced without one or more of these specific details.

[0015] General Overview Embodiments of this disclosure provide techniques for animating a character using sparse targets. In some embodiments, sparse targets can be specified in various modalities, such as joint constraints, text descriptions, and / or objects with which the character interacts. A control application processes the target input in each modality using a corresponding modality-specific encoder to generate markers. Given the markers, a marker mask indicating which markers are associated with unspecified inputs, and the character's current state, the control application samples a prior latent distribution generated by a prior of a trained partially constrained controller to obtain a sampled latent vector. The character can be a virtual character in a computer-based environment or a physical robot in a real-world environment, and the character's current state can be received from the computer-based environment or sensed using sensors in the real-world environment. The control application inputs the sampled latent vector and the character's current state into a decoder of the partially constrained controller to generate an action. The control application can then use the action to control the character in either the computer-based or real-world environment. Control of the character can result in an updated state of the character, and the above process can be repeated to generate another action for controlling the character using the updated state of the character, the sparse target, and the partially constrained controller.

[0016] The model trainer can perform a two-stage training technique to train a partially constrained controller. In the two-stage technique, the model trainer (1) trains a fully constrained controller using reinforcement learning to predict a sequence of actions that reconstruct a reference motion in a simulation, and then (2) trains a partially constrained controller using supervised imitation learning to recover the same actions as the trained fully constrained controller for a masked target in the simulation. For each of several iterations, the reinforcement learning may include calculating a reward based on the difference between the character's state after performing an action output by the fully constrained controller and the character's ground-truth state in the reference motion, and updating the parameters of the fully constrained controller based on the reward. In some embodiments, the reward may also include one or more regularization terms related to motion, such as one or more regularization terms for reducing energy consumption, minimizing impact, and / or minimizing motor jitter. Supervised imitation learning may include repeatedly sampling motion from a reference motion and time steps within the motion, sampling a mask of a target associated with the sampled motion, simulating actions computed by a partially constrained controller to achieve the masked target, computed a true baseline action using a fully constrained controller, computed a similarity loss (e.g., L2 or Kullback-Leibler (KL) divergence loss) based on a comparison between the action and the true baseline action, and updating the parameters of the partially constrained controller based on the similarity loss. More generally, the second phase of training may include a combination of matching the true baseline action and / or maximizing the reward.

[0017] The techniques used to animate characters have many real-world applications. For example, these techniques can be used to animate characters in virtual or extended reality (XR) environments, such as gaming environments. As another example, these techniques can be used to control physical robots in real-world environments.

[0018] The examples above are not intended to be limiting in any way. Those skilled in the art will understand that, in general, the techniques described herein for animate characters can be implemented in any suitable application.

[0019] System Overview Figure 1 The figure illustrates a block diagram of a computer-based system 100 configured to implement one or more aspects of various embodiments. As shown, system 100 includes a machine learning server 110, a data storage device 120, and a computing system 140 communicating via a network 130, which may be a wide area network (WAN) (such as the Internet), a local area network (LAN), a cellular network, and / or any other suitable network.

[0020] As shown, the model trainer 116 executes on one or more processors 112 of the machine learning server 110 and is stored in the system memory 114 of the machine learning server 110. One or more processors 112 receive user input from input devices such as a keyboard or mouse. In operation, one or more processors 112 may include one or more main processors of the machine learning server 110, controlling and coordinating the operation of other system components. Specifically, one or more processors 112 may issue commands controlling the operation of one or more graphics processing units (GPUs) (not shown) and / or other parallel processing circuitry systems (e.g., parallel processing units, deep learning accelerators, etc.), which include circuitry optimized for graphics and video processing, including, for example, video output circuitry systems. One or more GPUs may deliver pixels to a display device, which may be any conventional cathode ray tube, liquid crystal display, light-emitting diode display, etc.

[0021] The system memory 114 of the machine learning server 110 stores content, such as software applications and data, for use by one or more processors 112 and one or more GPUs and / or other processing units. The system memory 114 can be any type of memory capable of storing data and software applications, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash ROM), or any suitable combination thereof. In some embodiments, a storage device (not shown) may supplement or replace the system memory 114. The storage device may include any number and type of external memory accessible to the processor 112 and / or GPU. For example, but not limited to, the storage device may include a secure digital card, external flash memory, portable optical disc read-only memory (CD-ROM), optical storage device, magnetic storage device, and / or any suitable combination thereof.

[0022] The machine learning server 110 shown herein is for illustrative purposes only and may be varied and modified without departing from the scope of this disclosure. For example, the number of processors 112, the number of GPUs and / or other processing unit types, the amount of system memory 114, and / or the number of applications included in system memory 114 may be modified as needed. Furthermore, Figure 1 The connection topology between the various units can be modified as needed. In some embodiments, any combination of one or more processors 112, system memory 114 and / or one or more GPUs may be included in and / or replaced by any type of virtual computing system, distributed computing system and / or cloud computing environment (such as public, private or hybrid cloud system) in any type of virtual computing system, distributed computing system and / or cloud computing environment (such as public, private or hybrid cloud system).

[0023] In some embodiments, the model trainer 116 is configured to train one or more machine learning models, including a partially constrained controller 151 trained to generate actions for animate a character given a sparse objective, which may be specified in one or more modalities such as joint constraints, text descriptions, and / or objects with which the character interacts. The following is in conjunction with... Figure 4 To Figure 8 and Figures 12 to 13The techniques that model trainer 116 may employ to train partially constrained controller 151 are discussed in more detail. Training data and / or trained (or deployed) machine learning models, including partially constrained controller 151, may be stored in data storage device 120. In some embodiments, data storage device 120 may include one or more of any storage devices, such as one or more fixed disk drives, one or more flash drives, optical storage devices, network attached storage devices (NAS), and / or storage area networks (SANs). Although shown as accessible via network 130, in at least one embodiment, machine learning server 110 may include data storage device 120.

[0024] Exemplarily, the data storage device 120 also stores reference motion 154. Reference motion 154 is used to train the controller 151 with partial constraints. In some embodiments, reference motion 154 includes human-recorded motion for evaluating the motion generated by the controller 151 with partial constraints. In various examples, reference motion 154 is designed from various human activities collected, for example, through motion capture technology.

[0025] As shown, a control application 146 using a trained partially constrained controller 152 is stored in memory 144 and executed on one or more processors 142 of a computer device 140. The following is in conjunction with... Figure 9-10 and Figure 14 The control application 146 is discussed in more detail. Illustratively, the control application 146 uses a partially constrained controller 152 to control the movement of the character 160 within the environment 170. In some embodiments, the partially constrained controller 152 may be a partially constrained controller 151 trained without an encoder.

[0026] The environment 170 in which the character 160 performs actions can be a computer-based environment or a physical environment. In some embodiments, a computer-based environment can be simulated in any technically feasible manner, such as using a 3D engine, a generative model (e.g., a neural network) that predicts the next state of a given action, etc. For example, in a computer-based 3D virtual environment, the character 160 can navigate digital landscapes, such as a simulated cityscape with moving traffic and pedestrians, a fantasy world with dynamic terrain and interactive elements, etc. Computer-based environments can be used for video game development, virtual reality (VR) applications, advanced artificial intelligence (AI) training simulations, etc. In a physical environment, the character 160, such as a humanoid robot, can navigate real-world scenes, such as a robot moving through a warehouse to perform logistical operations, maneuvering in a hospital to deliver supplies, operating in hazardous environments, such as nuclear facilities where the human presence is at risk, and so on.

[0027] Figure 2 According to various embodiments Figure 1 A more detailed illustration of the machine learning server 110 is provided. In some embodiments, the machine learning server 110 may include any type of computing system, including but not limited to server machines, server platforms, desktop machines, portable computers, handheld / mobile devices, digital kiosks, in-vehicle infotainment systems, and / or wearable devices. In some embodiments, the machine learning server 110 is a server machine operating in a data center or cloud computing environment that provides scalable computing resources as a service over a network.

[0028] In some embodiments, the machine learning server 110 includes, but is not limited to, one or more processors 112 and one or more memories 114 coupled to the parallel processing subsystem 212 via a memory bridge 205 and a communication path 206. The memory bridge 205 is also coupled to an I / O (input / output) bridge 207 via the communication path 206, and the I / O bridge 207 is further coupled to a switch 216.

[0029] In some embodiments, I / O bridge 207 is configured to receive user input from optional input devices 208 (such as a keyboard, mouse, or touchscreen), sensor data analysis (e.g., evaluating gestures, voice, or other information about one or more uses of the field of view or sensing field of one or more sensors), and forward the input to one or more processors 112 for processing. In some embodiments, machine learning server 110 may be a server machine in a cloud computing environment. In such embodiments, machine learning server 110 may not include input device 208, but may receive equivalent input by receiving commands (e.g., in response to one or more inputs from a remote computing device) in the form of messages sent over a network and received via network adapter 218. In some embodiments, switch 216 is configured to provide connectivity between I / O bridge 207 and other components of machine learning server 110 (such as network adapter 218 and various add-on cards 220 and 221).

[0030] In some embodiments, I / O bridge 207 is coupled to system disk 214, which may be configured to store content, applications, and data for use by one or more processors 112 and parallel processing subsystems 212. In some embodiments, system disk 214 provides non-volatile storage for applications and data and may include fixed or removable hard disk drives, flash memory devices, and CD-ROMs (disc read-only memory), DVD-ROMs (digital versatile optical disc ROMs), Blu-ray, HD-DVDs (high-definition DVDs), or other magnetic, optical, or solid-state storage devices. In some embodiments, other components, such as universal serial bus or other port connections, optical disc drives, digital versatile optical disc drives, film recording devices, etc., may also be connected to I / O bridge 207.

[0031] In some embodiments, memory bridge 205 may be a northbridge chip, and I / O bridge 207 may be a southbridge chip. Furthermore, communication paths 206 and 213, as well as other communication paths within the machine learning server 110, may be implemented using any technically suitable protocol (including but not limited to AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol known in the art).

[0032] In some embodiments, the parallel processing subsystem 212 includes a graphics subsystem that delivers pixels to an optional display device 210, which may be any conventional cathode ray tube, liquid crystal display, light-emitting diode display, etc. In such embodiments, the parallel processing subsystem 212 may include circuitry optimized for graphics and video processing, including, for example, video output circuitry. Such circuitry may be contained across one or more parallel processing units (PPUs) (also referred to herein as parallel processors) included within the parallel processing subsystem 212.

[0033] In some embodiments, the parallel processing subsystem 212 includes circuitry optimized (e.g., optimized) for general and / or computational processing. Similarly, such circuitry may be included across one or more PPUs included within the parallel processing subsystem 212, the PPUs being configured to perform such general and / or computational operations. In yet other embodiments, one or more PPUs included within the parallel processing subsystem 212 may be configured to perform graphics processing, general processing, and / or computational processing operations. System memory 114 includes at least one device driver configured to manage the processing operations of one or more PPUs within the parallel processing subsystem 212. Furthermore, system memory 114 includes a model trainer 116 (hereinafter referred to as...). Figure 4-5 and Figure 9(To be discussed in more detail). Although this paper is primarily described with respect to model trainer 116, the techniques disclosed herein can also be implemented, in whole or in part, in other software and / or hardware (such as in parallel processing subsystem 212).

[0034] In some embodiments, the parallel processing subsystem 212 can be coupled with... Figure 2 It can be integrated with one or more other components to form a single system. For example, the parallel processing subsystem 212 can be integrated with one or more processors 112 and other interconnect circuitry on a single chip to form a system-on-a-chip (SoC).

[0035] In some embodiments, one or more processors 112 include the main processor of the machine learning server 110, which controls and coordinates the operation of other system components. In some embodiments, one or more processors 112 issue commands to control the operation of the PPU. In some embodiments, the communication path 213 is a PCI Express link, in which a dedicated channel is allocated to each PPU. Other communication paths may also be used. The PPU advantageously implements a highly parallel processing architecture, and the PPU can be provided with any amount of local parallel processing memory (PP memory).

[0036] It should be understood that the system shown herein is illustrative and can be varied and modified. The connection topology, including the number and arrangement of bridges, the number of one or more processors 112, and the number of parallel processing subsystems 212, can be modified as needed. For example, in some embodiments, system memory 114 may be directly connected to one or more processors 112 (instead of via memory bridge 205), and other devices may communicate with system memory 114 via memory bridge 205 and one or more processors 112. In other embodiments, parallel processing subsystems 212 may be connected to I / O bridge 207 or directly to one or more processors 112, instead of being connected to memory bridge 205. In some other embodiments, I / O bridge 207 and memory bridge 205 may be integrated into a single chip, rather than existing as one or more discrete devices. In some embodiments, Figure 2 One or more of the components shown may be absent. For example, switch 216 can be excluded, and network adapter 218 and add-on cards 220, 221 will be directly connected to I / O bridge 207. Finally, in some embodiments, Figure 2One or more components shown can be implemented as virtualized resources in a virtual computing environment, such as a cloud computing environment. Specifically, in some embodiments, the parallel processing subsystem 212 can be implemented as a virtualized parallel processing subsystem. For example, the parallel processing subsystem 212 can be implemented as a virtual graphics processing unit (vGPU) that renders graphics on one or more virtual machines (VMs) running on one or more server machines, the GPUs and other physical resources of which are shared across one or more VMs.

[0037] Figure 3 According to various embodiments Figure 1 A more detailed illustration of the computing system 140 is provided. In some embodiments, the computing system 140 may include any type of computing system, including but not limited to server machines, server platforms, desktop machines, portable computers, handheld / mobile devices, digital kiosks, in-vehicle infotainment systems, and / or wearable devices. In some embodiments, the computing system 140 is a server machine operating in a data center or cloud computing environment that provides scalable computing resources as a service over a network.

[0038] In some embodiments, the computing system 140 includes, but is not limited to, one or more processors 142 and one or more memories 144 coupled to the parallel processing subsystem 312 via a memory bridge 305 and a communication path 306. The memory bridge 305 is also coupled to an I / O (input / output) bridge 307 via the communication path 306, and the I / O bridge 307 is further coupled to a switch 316.

[0039] In some embodiments, I / O bridge 307 is configured to receive user input from optional input devices 308 (such as a keyboard, mouse, or touchscreen), sensor data analysis (e.g., evaluating gestures, voice, or other information about one or more uses of the field of view or sensing field of one or more sensors), and forward the input to one or more processors 142 for processing. In some embodiments, computing system 140 may be a server machine in a cloud computing environment. In such embodiments, computing system 140 may not include input device 308, but may receive equivalent input by receiving commands (e.g., in response to one or more inputs from a remote computing device) in the form of messages sent over a network and received via network adapter 318. In some embodiments, switch 316 is configured to provide connectivity between I / O bridge 307 and other components of computing system 140 (such as network adapter 318 and various add-on cards 320 and 321).

[0040] In some embodiments, I / O bridge 307 is coupled to system disk 314, which may be configured to store content, applications, and data for use by one or more processors 312 and parallel processing subsystems 312. In some embodiments, system disk 314 provides non-volatile storage for applications and data and may include fixed or removable hard disk drives, flash memory devices, and CD-ROMs (optical disc read-only memory), DVD-ROMs (digital versatile optical disc ROMs), Blu-ray, HD-DVD (high-definition DVD), or other magnetic, optical, or solid-state storage devices. In some embodiments, other components, such as universal serial bus or other port connections, optical disc drives, digital versatile optical disc drives, film recording devices, etc., may also be connected to I / O bridge 307.

[0041] In some embodiments, memory bridge 305 may be a northbridge chip, and I / O bridge 307 may be a southbridge chip. Furthermore, communication paths 306 and 313, as well as other communication paths within the computing system 140, may be implemented using any technically suitable protocol (including but not limited to AGP (Accelerated Graphics Port), HyperTransport, or any other bus or point-to-point communication protocol known in the art).

[0042] In some embodiments, the parallel processing subsystem 312 includes a graphics subsystem that delivers pixels to an optional display device 310, which may be any conventional cathode ray tube, liquid crystal display, light-emitting diode display, etc. In such embodiments, the parallel processing subsystem 312 may include circuitry optimized for graphics and video processing, including, for example, video output circuitry. Such circuitry may be contained across one or more parallel processing units (PPUs) (also referred to herein as parallel processors) included within the parallel processing subsystem 312.

[0043] In some embodiments, the parallel processing subsystem 312 includes circuitry optimized (e.g., optimized) for general and / or computational processing. Similarly, such circuitry may be included across one or more PPUs included within the parallel processing subsystem 312, the PPUs being configured to perform such general and / or computational operations. In yet other embodiments, one or more PPUs included within the parallel processing subsystem 312 may be configured to perform graphics processing, general processing, and / or computational processing operations. System memory 144 includes at least one device driver configured to manage the processing operations of one or more PPUs within the parallel processing subsystem 312. Furthermore, system memory 144 includes a control application 146 (hereinafter referred to as...). Figure 9-10 and Figure 14(Discussed in more detail). Although this document is primarily described with respect to the control application 146, the techniques disclosed herein can also be implemented, in whole or in part, in other software and / or hardware, such as in the parallel processing subsystem 312.

[0044] In some embodiments, the parallel processing subsystem 312 can be coupled with... Figure 3 It can be integrated with one or more other components to form a single system. For example, the parallel processing subsystem 312 can be integrated with one or more processors 142 and other interconnect circuitry on a single chip to form a system-on-a-chip (SoC).

[0045] In some embodiments, one or more processors 142 include the main processor of the computing system 140, which controls and coordinates the operation of other system components. In some embodiments, one or more processors 142 issue commands to control the operation of the PPU. In some embodiments, communication path 313 is a PCI high-speed link, in which a dedicated channel is allocated to each PPU. Other communication paths may also be used. The PPU advantageously implements a highly parallel processing architecture, and the PPU can be provided with any amount of local parallel processing memory (PP memory).

[0046] It should be understood that the system shown herein is illustrative and can be varied and modified. The connection topology, including the number and arrangement of bridges, the number of one or more processors 312, and the number of parallel processing subsystems 312, can be modified as needed. For example, in some embodiments, system memory 144 may be directly connected to one or more processors 142 (instead of via memory bridge 305), and other devices may communicate with system memory 144 via memory bridge 305 and one or more processors 142. In other embodiments, parallel processing subsystems 312 may be connected to I / O bridge 307 or directly to one or more processors 142, instead of being connected to memory bridge 305. In some other embodiments, I / O bridge 307 and memory bridge 305 may be integrated into a single chip, rather than existing as one or more discrete devices. In some embodiments, Figure 3 One or more of the components shown may be absent. For example, switch 316 can be excluded, and network adapter 318 and add-on cards 320, 321 will be directly connected to I / O bridge 307. Finally, in some embodiments, Figure 3One or more components shown can be implemented as virtualized resources in a virtual computing environment, such as a cloud computing environment. Specifically, in some embodiments, the parallel processing subsystem 312 can be implemented as a virtualized parallel processing subsystem. For example, the parallel processing subsystem 312 can be implemented as a virtual graphics processing unit (vGPU) that renders graphics on one or more virtual machines (VMs) running on one or more server machines, the GPUs and other physical resources of which are shared across one or more VMs.

[0047] Unified Physics-Based Role Control via Mask Motion Completion Figure 4 According to various embodiments Figure 1 A more detailed illustration of the model trainer 116 is provided. As shown, the model trainer 116 includes a reinforcement learning module 402 and a supervised imitation learning module 406. In operation, the model trainer 116 receives a reference motion 154 for training a fully constrained controller 404 and a partially constrained controller 151. In some embodiments, the reference motion 154 may be a motion capture dataset, which includes captured motions of humans performing various movements. In such cases, the reference motion 154 may include the position and rotation of each of several joints in each frame of the captured motion.

[0048] Model trainer 116 performs a two-stage training technique, in which (1) reinforcement learning module 402 first uses reinforcement learning to train a fully constrained controller 404 to predict the sequence of actions that reconstruct the reference motion 154 in the simulation, and then (2) supervised imitation learning module 406 uses supervised imitation learning to train a partially constrained controller 151 to recover the same actions as the trained fully constrained controller 404 for a masked target (i.e., constraint) in the simulation. This is essentially a form of motion completion. (See below for further details.) Figure 5 In more detail, the reinforcement learning performed by the reinforcement learning module 404 may include: for each of several iterations, calculating a reward based on the difference between the character's state after the action output by the fully constrained controller 404 and the character's true baseline state in the reference motion 154, and updating the parameters of the fully constrained controller 402 based on the reward. In some embodiments, the reward may also include one or more terms independent of the reference motion, such as energy consumption, shock minimization, and / or minimum motor jitter terms. The character may be a virtual character in a computer-based environment or a physical robot in a real-world environment, and the character's current state may be received from the computer-based environment or sensed using sensors in the real-world environment. As discussed below... Figure 6In more detail, the supervised imitation learning performed by the supervised imitation learning module 406 may include: repeatedly sampling the motion from the reference motion 154 and the time steps within the motion; sampling the mask of the target associated with the sampled motion; executing in the simulation the action computed by the partially constrained controller 151 to achieve the masked target; compute the true reference action using the fully constrained controller 404; compute the similarity loss based on the comparison between the action and the true reference action; and update the parameters of the partially constrained controller 151 based on the similarity loss.

[0049] More formally, in some embodiments, the first phase of the two-phase training can follow the framework of target-conditional reinforcement learning (GCRL) to train a general motion controller, namely a fully constrained controller 404, which can be guided to perform a wide variety of tasks. During the first phase, the reinforcement learning (RL) agent performs training according to a policy. Interact with the environment (e.g., environment 170). In each step... Agent observation status and future goals Then, the agent applies the strategy. Actions in After an action is applied, the environment dynamically adjusts accordingly. Transition to a new state And the agent received a reward The agent's goal is to learn strategies that maximize the cumulative discount reward: (1) in It is a trajectory The possibility of discounting. This determines the effective period of the strategy.

[0050] The second phase of the two-stage training utilizes behavioral cloning (BC) to train teacher strategies through RL. (i.e., the fully constrained controller 406) is distilled into a more general student strategy that can be guided by multimodal inputs. (i.e., partially constrained controller 151). In some embodiments, the policy distillation process is performed using the DAgger technique. During the online distillation process, trajectories are collected by executing student policies, and then the trajectories are relabeled with actions from the teacher policy: (2) In equation (2), This represents the distribution of states and goals observed under the student's strategy.

[0051] Figure 5 According to various embodiments Figure 4A more detailed illustration of the reinforcement learning module 402 is provided. As shown, the reinforcement learning module 402 includes a scoring module 520. In operation, the reinforcement learning module 402 receives a reference motion 154 and samples a full target 504 that includes the reference motion in the reference motion 154. During each iteration of training using the full target 504, the reinforcement learning module 402 inputs the character's current state and several target poses from the full target 504 for the next frame into a fully constrained controller 404. Given such inputs, the fully constrained controller 404 outputs an action 508. The reinforcement learning module 402 simulates the character performing the action and receives an updated state of the character from the environment 170. In some embodiments, the reinforcement learning module 402 sends the action 508 to the character's controller, such as a proportional-derivative (PD) controller, which controls the movement of the character's joints within the environment 170 based on the action 508. Although the description herein focuses primarily on the controller for illustrative purposes, in some embodiments, the action may be transmitted directly to the character as, for example, a direct torque force on the character's joints.

[0052] The simulation of the character performing action 508 results in an updated state 510 for the character. The scoring module 520 uses this updated state 510 to calculate a reward 522 for updating the parameters of the fully constrained controller 404. In some embodiments, the scoring module 520 calculates the reward based on a comparison of the updated state 510 with the state in the corresponding frame of the fully constrained target 504 (and possibly one or more other items independent of the reference motion). The reinforcement learning module 402 then updates the parameters of the fully constrained controller 404 based on the calculated reward 522. In some embodiments, the reinforcement learning module 402 may use reward and backpropagation techniques to update the parameters of the fully constrained controller 404. Furthermore, the above steps can be repeated multiple training iterations until all frames of the fully constrained target 504 have been used in the training. The reinforcement learning module 402 can then sample other fully constrained targets and perform training using these fully constrained targets.

[0053] More formally, the goal of physics-based motion tracking is to generate control (such as motor actuation) that enables simulated characters to generate motion relative to kinematic targets. Very similar sports Movement can be represented as a series of postures. Each pose Encoding is performed using redundancy representations, which are derived from the roles. joint 3D Cartesian positions and their rotations Composition. To successfully track reference motion, information describing the motion it should mimic is typically provided to the controller (e.g., a fully constrained controller 404). The target pose is referred to herein as the fully constrained target. Because the future pose provides complete information about the target movement that the character should imitate.

[0054] The fully constrained controller 404 is a fully constrained motion tracking controller trained on a reference motion 154, which in some embodiments may be a large motion capture dataset. In some embodiments, the input to the fully constrained controller 404 includes a whole-body target trajectory of the desired motion. The fully constrained controller 404 can be trained to mimic a wide variety of motions, including those involving irregular terrain and object interactions. When the motion dataset only includes kinematic motion fragments, the fully constrained controller 404 is used herein. The primary purpose of the fully constrained controller 404 may be to estimate the actions (motor actuation) required to control the simulated role. Then, This provides a foundation that greatly simplifies the training process for more general controllers in subsequent stages.

[0055] In some embodiments, a fully constrained controller 404 is trained end-to-end to mimic target motion conditioned on a sequence of full-body motions and observations of the surrounding environment, such as terrain and object height maps. The terrain can include any representation of the area surrounding the character, such as a point cloud or grid of height. The training objective can be set as motion-tracking rewards and optimized using reinforcement learning. In each step, Observe the current humanoid state This includes 3D body poses and velocities normalized relative to the character's local coordinate system: (3) in This represents the quaternion difference between two rotations. In addition to the character's current state, the strategy also observes the reference motion. The next one Target posture. Each joint... The features are normalized relative to the current root and relative to the current corresponding joint: (4) Each target posture The features also change with the time from the current time step to the target pose. And this enhancement results in the following representation: To mimic movement on irregular terrain, the character's pose can be normalized relative to the height of the terrain below the character's root (e.g., pelvis). During training, a height map of the surrounding environment can be provided to a fully constrained controller 404, oriented along the facing direction of the root. The height map has a fixed resolution and records the height of nearby terrain geometry and object surfaces.

[0056] Motion tracking is a sequence modeling problem. The goal is to predict the next action based on the current character's state, the surrounding terrain, and a sequence of future target poses. In some embodiments, a transformer-based controller processes each input and design... Labeling is performed. This architectural choice allows the fully constrained controller 404 to process relevant information on the input sequence and capture dependencies between various input labels. To further enhance the learning process, a critique network (not shown) can be employed alongside the transformer-based controller. The critique network can be implemented as a fully connected network of estimated value functions. Doing so provides a learning signal to guide the controller toward the optimal action.

[0057] In some embodiments, the reward is 522 (as used herein). (This indicates that) the character is encouraged to follow a reference motion by minimizing the difference between the simulated character's state and the target motion. (5) in Indicates the various reward amounts, These are their corresponding weights. The terms in the reward function encourage the character to mimic the global joint position (gp), global joint rotation (gr), root height (rh), joint velocity (jv), joint angular velocity (jav), and energy penalty (eg) of the reference motion to encourage smoother and less jittery movement. In some embodiments, the reward may also include one or more terms that are independent of the reference motion (e.g., energy consumption, shock minimization, and / or minimum motor jitter terms).

[0058] In some embodiments, early termination can be performed during training of the fully constrained controller 404, thereby improving the success rate of rare and more complex movements. For example, in some embodiments, a movement performed on flat terrain can be terminated once any joint position deviates by a given amount, such as 0.25 meters. On irregular terrain, an episode is terminated when the joint error exceeds a given amount, such as 0.5 meters, thus providing the controller with greater flexibility to adapt the original reference movement to the new environment. In some embodiments, the model trainer 116 can also prioritize training on movements with higher failure rates. Since some movements are not expected to succeed in all scenarios (e.g., a front flip or a side flip up a flight of stairs), prioritized sampling only considers failures that occur on flat terrain. The probability of prioritizing is proportional to the probability of failure in that motion, and is limited to the minimum weight, for example, 3. e -3. This adaptive sampling strategy can help ensure that the agent collects enough data to reproduce more dynamic and challenging behaviors.

[0059] Figure 6 According to various embodiments Figure 4A more detailed illustration of the supervised imitation learning module 406 is provided. As shown, the supervised imitation learning module 406 includes a sampling module, a masking module 608, and a scoring module 620. In operation, the supervised imitation learning module 406 receives a reference motion 154, and the sampling module 602 samples a complete target 606, including the sampled reference motion, from the reference motion 154 at sampling times. During each iteration of training using the complete target 606, the masking module 608 generates a masked target 610, which includes random masking of targets in several subsequent frames of the complete target 504. In some embodiments, targets may be specified in one or more modalities, which may include one or more data of any suitable type associated with the motion of a character. For example, in some embodiments, a modality may include: the position and / or orientation of any number of joints of a character in any number of subsequent frames, including fully observed motion in which no masking is applied; a textual description of one or more tasks the character will perform in the subsequent frames; and / or objects that the character will interact with in the subsequent frames (e.g., specified as bounding boxes around the objects, point clouds, etc.). In such cases, training data for different modalities can be obtained in any technically feasible manner. For example, in some embodiments, joint positions, rotations, and their relative timings can be extracted from motion capture data, and motion mirroring (e.g., flipping from left to right) can be used as a form of data augmentation to improve generalization to new and unseen motions. As another example, a motion sequence can be decomposed into atomic behaviors, each of which can be labeled with a textual description that provides textual commands for sparse targets, or the textual description can be mirrored (e.g., "person turns left" can be converted to "person turns right"). As yet another example, objects can be randomly sampled within object categories associated with motion segments that interact with objects of different categories.

[0060] The masking module 608 samples a random mask of the target in subsequent frames. This may include randomly sampling only the mask for the last frame when using a sliding window, and sampling the mask for all frames except the last frame in previous iterations. For example, in some embodiments, the random mask may mask the position and / or orientation of a joint, text description, and / or object in zero or more of the subsequent frames, thereby producing a masked target 610. In some embodiments, sampling the random mask may include structured masking, which is described below in conjunction with... Figure 7To describe in more detail, structured masking may include sampling “time bubbles” during which constraints other than certain types of constraints (such as text or object constraints) are not used. Time bubbles can help the controller 151, which has some constraints, see sufficient examples of such constraints that remain largely fixed throughout training but can vary with a certain probability, similar to user-specified constraints. For example, in some embodiments, as described above, whether a time bubble has started and its length can be sampled at each iteration of the sliding window. In this case, the supervised imitation learning module 406 can sample the mask for the current iteration based on the mask in previous iterations and the presence of high-level constraints (such as text descriptions or objects) that might require time bubbles.

[0061] The supervised imitation learning module 406 inputs the character's current state, the full target 606, and the masked target 610 into the partially constrained controller 151. Given such input, the partially constrained controller 151 outputs an action 612. The supervised imitation learning module 406 simulates the character performing the action and receives an updated state of the character from the environment 170. In some embodiments, the supervised imitation learning module 406 sends the action 612 to the character's controller, such as a PD controller, which controls the movement of the character's joints within the environment 170 based on the action 612. The simulation of the character performing the action 612 results in an updated state 614 for the character.

[0062] The supervised imitation learning module 406 also inputs the character's current state and the number of next frames corresponding to the masked target 610 for the full target 606 into the fully constrained controller 404. Given such input, the fully constrained controller 404 outputs a true baseline action 620. The scoring module 620 calculates a similarity loss 622 based on a comparison between the action 612 output by the partially constrained controller 151 and the true baseline action 616 output by the fully constrained controller 404. The supervised imitation learning module 406 then updates the parameters of the partially constrained controller 151 based on the similarity loss 622. In some embodiments, the supervised imitation learning module 406 can use the similarity loss 622 and backpropagation techniques to update the parameters of the partially constrained controller 151, and can repeat the above steps for multiple iterations until all frames using the full target 606 have been used in training. The sampling module 602 of the supervised imitation learning module 406 can then sample other full targets and perform training using those full targets. In some embodiments where the partially constrained controller 151 is a variational autoencoder (VAE), a Kullback-Leibler (KL) schedule can be performed during training, which uses the KL divergence loss between the encoder and the prior, starting with low KL coefficients and increasing the KL coefficients over time, as discussed in more detail below.

[0063] Partially observable targets (i.e., sparse targets) are also represented in this paper as Such partial targets specify only some elements of the desired motion. To train a versatile partially constrained controller 151 that can be guided by partial targets, model trainer 116 trains the partially constrained controller 152 on random masked observations of the target motion. These masked observations are obtained using a random masking function. The partially constrained controller 151 can also be guided by different control inputs. The versatility of the partially constrained controller 151 stems from a masked training scheme. During training, the task of the partially constrained controller 151 is to reconstruct the target's whole-body motion given random masked inputs. This allows the partially constrained controller 151 to generate whole-body motion based on arbitrary partial constraints.

[0064] More specifically, in some embodiments, once the fully constrained controller 404 has been trained, the fully constrained controller 404 is used to train the partially constrained controller 151 (referred to herein as...). (Representation). Given partial constraints (such as the target position of a joint, text commands, or object positions), the controller 151 of the partial constraints... This will generate various full-body movements that satisfy these constraints. Training for a fully constrained controller Predicted actions Model the distribution while observing only a portion of the constraints. Then, some constraints provide users with a general and convenient interface for guidance. Perform new tasks without requiring task-specific training.

[0065] The goal is to generate motion that conforms to constraints specified by partial objectives, similar to the task of motion completion. As described, some example objectives include: (1) any joint-any time: the model should support target position and rotation conditions for any joint at any future time step; (2) text-to-motion: the model should support high-level text commands, making the character's direction of movement more intuitive and expressive; and (3) object: the model should support object-based objectives, such as interaction with furniture, when available. To generate the desired behavior, the partially constrained controller 151 can support conditioned on one or more of the above objectives simultaneously. For example, path tracking can be achieved with the arm raised by conditioned on the target root trajectory and the text command "walk while raising your hand". This flexibility allows for the generation of a variety of complex and expressive motions from a concise partial specification.

[0066] For training This method uses random masking to procedurally extract flexible targets from reference motion (e.g., motion capture data). In some embodiments, random masking may include the structured masking described herein. During training, Training is performed to mimic the original, fully (unmasked) target motion by predicting the actions of a fully constrained controller that observes the full target motion of a real baseline. Partial objectives are undefined problems because there may be multiple plausible motions that can satisfy a given set of partial objectives. For example, when conditioned on reaching the target position within 1 second, there are numerous possible motions to achieve this objective. To address this ambiguity, in some embodiments, the partially constrained controller 151, The model is constructed as a conditional variational autoencoder (C-VAE). In this case, the C-VAE model makes... Instead of simply generating a single deterministic behavior, C-VAE models can model the distribution of different behaviors that satisfy a specific set of constraints. By sampling from the learned distribution, C-VAE models can generate a variety of realistic and physically plausible motions that conform to a specified partial objective, while still allowing for natural variation and adaptability to different situations.

[0067] In some embodiments, various training strategies can be used to improve the stability and effectiveness of the partially constrained controller 151. In such cases, strategies may include structured masking, KL scheduling, incidental latent noise, and / or observation history. Furthermore, during the distillation process, [the following can be observed / retrieved]... and Sampling is performed on deterministic actions to reduce randomness during data collection. Early termination can also be applied during distillation to prevent [further issues]. Enter State not observed during training. Due to... Training also continued even after the program was terminated early. May not be during training Provide appropriate actions in the areas experienced.

[0068] In some embodiments using structured masking, the masking performed by masking module 608 randomly removes individual target joints, text descriptions, and scene information. Such structured masking can lead to improved robustness to potential user inputs. More specifically, structured masking may include randomly removing individual target joints, text descriptions, and scene information (where applicable) from the model's input targets. To better ensure temporally consistent behavior, a temporally structured masking scheme can be used. In this case, instead of randomly resampling the mask at each step, a mask randomly sampled in one time step has the opportunity to be repeated in multiple subsequent time steps. Randomly resampling the mask at each step can reduce the ambiguity encountered by the model during training. Therefore, the resulting model generalizes poorly. This is because different joints may be visible on different frames, and cross-frame information provides a less ambiguous description of the requested motion. By using a temporally consistent sampling scheme, joints can be observed across multiple consecutive frames while other joints are consistently hidden. To ensure the model supports high-level objectives, such as text commands and interactions with target objects, all future poses can be masked. This structured sampling mechanism helps to ensure During training, the system encounters and learns to handle a range of different masking patterns. This results in improved robustness to potential user inputs.

[0069] In some embodiments using KL scheduling, the KL coefficients can be initialized with low values ​​(such as 0.0001) and linearly increased to higher values ​​(such as 0.01) during training. Starting with low KL coefficients allows the partially constrained controller 151 to more closely mimic... Increasing the coefficients then encourages the model to impose more structure into the learned latent space, making it easier to sample from the prior at runtime.

[0070] In some embodiments using sporadic latent noise, latent values ​​can be sampled during training via a reparameterization technique. To further encourage more temporally consistent behavior, the "noise" parameter... This can remain constant throughout the entire training round. Therefore, in each period τ In China, according to The latent variables are sampled, and the noise... It remains constant throughout the round.

[0071] In some embodiments using observation history, when conditional on a text command, the controller 151 with partial constraints can be configured. Providing past poses, as experience has shown, helps generate long, coherent movements that align with the intent of a given text command. In such cases, several (e.g., 5) observations, subsampled from observations over past time steps (e.g., 40 past time steps), can be provided to the prior of the partially constrained controller 151.

[0072] Figure 7 According to various embodiments Figure 4 A more detailed illustration of the partially constrained controller 151 is shown. As illustrated, the partially constrained controller 151 is a VAE, which includes encoder 708 and modality-specific encoder 710. i(Hereinafter referred to as modality-specific encoder 710, and modality-specific decoder 710, respectively), prior 714, and decoder 720. Although this document primarily describes VAEs as a reference example, in some embodiments, the partially constrained controllers 151 and 152 can have any technically feasible architecture. For example, in some embodiments, the partially constrained controllers 151 and 152 can be another type of generative model, such as a diffusion model, a feedforward model, etc. The modality-specific encoder 710 is a neural network trained to tokenize (encode) different input modalities (such as joint constraints, text descriptions, and / or objects) in a sparse target 704 derived from a reference motion 702 (which can be sampled from reference motion 154). In some embodiments, the modality-specific encoder 710 may also encode other inputs (such as current character state, surrounding terrain, and past poses) into the partially constrained controllers 151. In some embodiments, the output of the modality-specific encoder 710 may be a sequence of tags, each tag representing a different input to the model, and the tags may also have the same tag dimension. In some embodiments, the tags may include tags for different character poses. A tag mask is used to prevent the prior 714 from processing unspecified inputs, such as joint constraints, text descriptions, and / or objects specified in the sparse target 704. In some embodiments, the tag mask informs the transformer attention mechanism which tags to ignore. The prior 714 is a neural network that takes the tags and the tag mask 712 as inputs and outputs a base distribution. In some embodiments, the prior 714 may be a transformer-based neural network. The encoder 708, used only during training, is a neural network that takes the full target pose 706 and the sparse target 704 as inputs and outputs the residuals to the base distribution output by the prior 714. In some embodiments, the encoder 708 may be a fully connected neural network. Using the encoder 708 during training prevents the output of the prior 714 from collapsing to the mean after training. The partially constrained controller 151 adds the residual to the latent distribution to generate a prior distribution 716. The partially constrained controller 152 samples the prior distribution 716 to obtain a sampled latent vector 718. The sampled latent vector 718 essentially represents the solution to be applied. The partially constrained controller 151 inputs the sampled latent vector 718 and the character's current state 728 into a decoder 720, which outputs an action 722. In some embodiments, the decoder 720 may be a fully connected neural network.

[0073] The model trainer 116 uses actions 722 to control the character within the environment 170. As described above, in some embodiments, the model trainer 116 can send the actions 722 to the character's controller, such as a PD controller, which controls the movement of the character's joints within the environment 170 based on the actions 722. The model trainer 116 then receives the updated state 726 of the character from the environment 170 and can repeat the above process to generate another action for controlling the character in subsequent time steps, and so on.

[0074] More formally, in some embodiments, the partially constrained controller 151 includes learnable priors 714, which are described herein by Indicates; encoder 708, here by The text indicates that the decoder 720 is represented in this document. Indicates. Encoder Based on the desired reference motion, the latent distribution is output given a fully observable future target pose. Then, the decoder... The simulation generates actions based on latent values ​​sampled from the encoder distribution. The last component is the learned prior. Given only partially observed constraints, a prior is trained to match the encoder distribution. The learnable prior allows the partially constrained controller 151 to generate natural motion at runtime based on simple, user-defined partial constraints, without requiring the user to specify the full target trajectory the character must follow. The encoder is used only for training and is not utilized at runtime.

[0075] In some embodiments, prior 714 can be modeled as a delay. A Gaussian distribution with mean ... and diagonal standard deviation matrix , (6) In some embodiments, the encoder 708 can be modeled as a priori residuals. (7) This design helps ensure that the embeddings from encoder 708 (which have access to the full observations of the target motion) remain close to the priors that only receive partial observations. During training, the embeddings from encoder 708 are used to input latent variables. Sampling is performed. All components can be trained using an objective (i.e., a similarity loss) that makes the components... Maximize the log-likelihood of the predicted action and minimize the KL divergence between encoder 708 and prior 714: (8) in By using random masking functions Built for use with the original, fully observed target: In the above formula, Interacting with the environment, and The target action is labeled at each time step. In some embodiments, other similarity losses, such as L2 loss, may be used. More generally, in some embodiments, the training of the second phase may include a combination of matching the real baseline action and / or maximizing the reward via reinforcement learning. During inference, encoder 708 is discarded, and latent values ​​are sampled only from prior 714.

[0076] In some embodiments, to provide a unified architecture capable of handling multimodal inputs, a transformer-encoder can be used to process prior information. Modeling is performed. This is done by enabling variable-length input labels for the observable at each time step. For example, for each input modality (target pose)... Object bounding box Topographic elevation map Current posture ,text and historical posture The encoder 708 can have a unique encoder shared by all inputs across the same modality. When the input is masked, a transformer masking mechanism can be used to exclude the corresponding marker. In some embodiments, the output of the transformer is provided to two fully connected layers to output the mean and log-standard deviation of the prior distribution. Since the encoder 708 always observes the complete target frame as input, a natural architecture for the encoder 708 is a fully connected model, as the input to the encoder 708 is always a fixed size. More generally, in some embodiments, any technically feasible encoder 708, such as a transformer or other architecture, can be used. In addition to masking applied to keyframes, the encoder also observes the complete future pose. This indicates which joints are visible a priori. Furthermore, encoder 708 observes the current pose. and topographic elevation map Similar to the prior, two fully connected output heads can be used to output the encoder's residual mean and logstd (log-standard deviation). Similarly, the decoder 720 can also be modeled as a fully connected network. The decoder 720 observes the current state. Sampled latent values and topographic elevation map Then, decoder 720 outputs deterministic actions. .

[0077] In some embodiments, The input modalities that can be received as input can be represented as follows. The goal is to provide a sufficiently rich representation that is also computationally efficient and facilitates generalization to new tasks. To represent keyframes, future keyframes with partially observable joints can first be normalized to the current pose (Equation (3)). Unobserved joints can then be zeroed out and a mask can be appended to the target frame. The time frame is next to the time frame. Observations of pose at previous time steps can be represented in a similar manner, but all joints are observed, and no masking is applied. In some embodiments, each object can be represented using the positions of the eight corners of a bounding box, normalized to the character's local coordinate system; as a point cloud; or in any other technically feasible manner. To identify different types of objects, an index representing the object type (e.g., chair, sofa, stool) can be used. To represent text, each text command can be encoded using an embedding, which can be trained on video language pairs to better capture temporal relationships. By leveraging spatiotemporal information in the video during training, the embedding can encode the temporal aspects of language that are crucial for describing motion, making it well-suited for representing text commands to be translated into character animation.

[0078] Figures 8A-8C The figure illustrates exemplary simulated terrain that can be used during training of a fully constrained controller 404 and a partially constrained controller 151, according to various embodiments. (As described above...) Figure 4 In some embodiments, different simulated terrains can be used during training, allowing the trained, partially constrained controller 151 to control a character performing tasks on different terrains. In such cases, characters can be generated in random terrains during different training rounds, and the reference motion and the position of masked targets in different frames can be normalized relative to the terrain used. For example, in some embodiments, random terrain can be random locations on terrain encompassing different regions. Figure 8A The figure illustrates region 800, which includes flat terrain. Flat terrain allows controllers (such as fully constrained controller 404 and partially constrained controller 151) to generate the original reference motion in an environment that best represents how reference motion should be recorded. Flat terrain is a simple environment where it is assumed that most of the training data is recorded on flat ground, and the controller can primarily focus on mimicking reference motion. Flat terrain can serve as a baseline for evaluating the controller's ability to mimic motion in a simple, unobstructed environment.

[0079] Figure 8BThe diagram illustrates a second region 812 with rugged terrain and a third region 810 with stairs and ramps. The rugged terrain, stairs, and ramps allow the controller to learn robust movement skills on different ground geometries. More generally, in some embodiments, irregular terrain regions can include a wide variety of irregular terrain features, such as stairs, rugged gravel terrain, and ramps (smooth and rugged). When the agent is mimicking movement that does not involve object interaction, the agent can be induced at any random location within both flat and irregular terrain regions. Such setups expose the controller to different terrain conditions, allowing the controller to learn robust movement skills capable of adapting to different types of terrain.

[0080] Figure 8C The diagram illustrates a fourth region 820 for interacting with objects, such as object 822. The fourth region 820 allows the controller to practice interacting with objects in a clean and reproducible setting without interference from irregular terrain features. In some embodiments, the object interaction region may include various objects placed on a flat surface, such as chairs, tables, and sofas. In such cases, the character can only be initialized in this region when it is mimicking movements involving object interaction.

[0081] Figure 9 According to various embodiments Figure 1A more detailed illustration of the control application 146 is provided. As shown, the control application 146 includes a partially constrained controller 152. In operation, the control application 146 receives a sparse target 902. The sparse target 902 may specify one or more tasks to be performed by a character using any number of modalities that the partially constrained controller 152 can be trained to process. For example, in some embodiments, the sparse target may include the position and / or orientation of any number of joints of the character in any number of frames of animation, including fully observed motion without masking; a textual description of one or more tasks to be performed by the character; and / or objects that the character will interact with (e.g., bounding boxes, point clouds, etc., specified as objects). In some embodiments, the sparse target 902 may be specified by the user in any technically feasible manner, such as via a graphical user interface (GUI). The control application 146 inputs the sparse target 902 and the character's current state into the partially constrained controller 152 to generate an action 904. The control application 146 then uses the generated action 904 to control the character within the environment 170. Although this document describes primarily a controller 152 that controls a partially constrained character within the same environment 170 for illustrative purposes, in some embodiments, the control application 146 may use the partially constrained controller 52 to control the character in an environment different from the one used during training. For example, in some embodiments, the control application 146 may transmit an action 904 to the character's controller, such as a PD controller, which controls the movement of the character's joints within environment 170 (or another environment) based on the action 904. The control application 146 receives an updated state 906 of the character from environment 170 (or another environment), which may be used with sparse targets 902 to generate further actions for controlling the character.

[0082] Figure 10 How to use according to various embodiments Figure 1 The controller 152, which contains partial constraints, controls the role in a more detailed manner. As shown, given a sparse target 1002 (which can be similar to the combination above), Figure 9When the sparse target 1002 described is used as input, the control application 146 encodes the sparse target 1002 using a modality-specific encoder 710 to generate a marker, and the control application 146 concatenates the marker with a marker mask to generate a marker and a marker mask 1004. In some embodiments, the control application 146 inputs different modalities of the sparse target 1002 into the corresponding modality-specific encoder, which outputs a marker. In some embodiments, the modality-specific encoder 710 may also encode other inputs, such as the current character state, surrounding terrain, and past poses, into the partially constrained controller 152. Furthermore, a transformer attention mechanism can be used to mask unused markers via a marker mask. As described, the marker mask prevents the transformer from processing unspecified inputs, such as keyframes without any target joints, or sequences without text or object conditionalization.

[0083] Control application 146 inputs a tag and tag mask 1004, along with the character's current state 1012, into prior 714, which outputs a prior distribution 1006. Control application 146 then samples the prior distribution 1006 to obtain a latent vector 1008. Subsequently, control application 146 inputs the sampled latent vector 1008 and the character's current state into decoder 720, which outputs an action 1010. Control application 146 uses action 1010 to control the character within environment 170. As described, in some embodiments, control application 146 may transmit action 1010 to the character's controller, such as a PD controller, which controls the character's joints to move within environment 170 based on the action. Control application 146 then receives the updated state of the character from environment 170 and may repeat the above process to generate another action for controlling the character at subsequent time steps, and so on. It should be noted that the partially constrained controller 152 does not include the encoder 708 of the partially constrained controller 151, because the encoder 708 can be discarded after the partially constrained controller 150 is trained.

[0084] Figure 11A-11C The figures illustrate exemplary motion generated according to various embodiments by using different modal control roles. For example... Figure 11A As shown, when a sparse target is specified as the path 1102 that character 1100 will follow, the partially constrained controller 152 can be used to generate the motion of character 1100 on the path 1112 that it will follow on terrain including stairs and slopes. In this example, path 1102 includes the coordinates that character 1100's head will follow. Figure 11BAs shown, when a sparse target is specified as the path 1112 that character 1110 is to follow, along with the text "a person raises their hands and walks forward," the partially constrained controller 152 can be used to generate the motion of character 1110 raising their hands as they walk forward along path 1112 on rugged terrain. Figure 11C As shown, when a sparse target is specified as the bounding box of an object, such as armchair 1122, a partially constrained controller 152 can be used to generate motions for a character 1120 to approach and interact with armchair 1122, such as sitting on armchair 1122.

[0085] Figure 12 Flowcharts illustrating method steps for training a fully constrained controller and training a partially constrained controller using the trained fully constrained controller, according to different embodiments, are provided. Although combined... Figure 1-7 and Figure 9-10 The method steps have been described, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this invention.

[0086] As shown, method 1200 begins at step 1202, where model trainer 116 receives reference motion 154 for training fully constrained controller 404 and partially constrained controller 151. As described, in some embodiments, reference motion 154 may be a motion capture dataset comprising captured human motion performing various movements. In such cases, reference motion 154 may include the position and rotation of each joint in each frame of the captured motion.

[0087] In step 1204, the model trainer 116 uses reinforcement learning to train the fully constrained controller 404 to predict sequences of actions that reconstruct the reference motion in the simulation. For each of several iterations, the reinforcement learning may include calculating a reward based on the difference between the character's state after performing an action output by the fully constrained controller 404 and the character's true baseline state in the reference motion, and updating the parameters of the fully constrained controller 404 based on the reward. In some embodiments, the reward may also include one or more terms independent of the reference motion (e.g., energy consumption, shock minimization, and / or minimum motor jitter terms).

[0088] In step 1206, model trainer 116 uses supervised imitation learning to train partially constrained controller 151 to recover the same action as the trained fully constrained controller 404 for a masked target in the simulation. In some embodiments, model trainer 116 may train partially constrained controller 151 by: repeatedly sampling motion from reference motion 154 and time steps within the motion; sampling a mask of a target associated with the sampled motion; executing the action computed by partially constrained controller 151 to achieve the masked target in the simulation; computed a true baseline action using fully constrained controller 404; computed a similarity loss based on a comparison between the action and the true baseline action; and updated the parameters of partially constrained controller 152 based on the similarity loss, as combined below. Figure 13 This will be discussed in more detail.

[0089] Figure 13 A flowchart illustrating the method steps for training a controller with partial constraints in step 1206 of method 1200 according to different embodiments is provided. Although combined with… Figure 1-7 and Figure 9-10 The method steps have been described, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this invention.

[0090] As shown, in step 1302, the model trainer 116 samples the motion based on the reference motion 154 and the time step within the motion.

[0091] In step 1304, the model trainer 116 samples a mask of a target associated with the sampled motion and time step. The target may include the position and / or orientation of any number of joints of a character in any number of frames of the animation, a textual description of one or more tasks to be performed by the character, and / or an object that the character will interact with. The mask randomly masks a portion of the target from one or more frames. In some embodiments, sampling the mask may include sampling a time bubble, during which no constraints other than certain types of constraints (such as text or object constraints) are used. For example, in some embodiments, during training, whether the time bubble starts and the length of the time bubble may be sampled in each iteration of the sliding window, as described above. Figure 6 As stated above.

[0092] In step 1306, the model trainer 116 causes the action for realizing the masked target, calculated by the partially constrained controller 151, to be executed in the simulation. In some embodiments, the partially constrained controller 151 may input the current state of the masked target and the character into a partially constrained controller 152, which outputs an action for the character that can be simulated within the environment 170.

[0093] In step 1308, the model trainer 116 calculates the true baseline motion using the fully constrained controller 404. In some embodiments, the model trainer 116 inputs the character's complete target and current state into the fully constrained controller 404, which outputs the true baseline motion.

[0094] In step 1310, the model trainer 116 calculates a similarity loss based on a comparison between the action and the true baseline action. In some embodiments, the similarity loss may be an objective that maximizes the log-likelihood of the action predicted by the fully constrained controller 106 and minimizes the KL divergence between the encoder 708 of the partially constrained controller 151 and the prior 714, as described above. Figure 7 More generally, any technically feasible loss can be used in some embodiments. For example, when the distribution is a delta function, mean squared error loss can be used in some embodiments. As another example, for a more general function, KL divergence can be used in some embodiments. In some embodiments, in addition to binary cross-entropy loss relative to the true baseline action, a combined reward for tracking the original motion and an additional regularization term can also be used.

[0095] In step 1312, model trainer 116 updates the parameters of the partially constrained controller 151 based on similarity loss (and optional reward, as described above). In some embodiments, model trainer 116 may use similarity loss and backpropagation techniques to update the parameters of encoder 708, prior 714, and decoder 720 in the partially constrained controller 151. In some embodiments where the partially constrained controller 151 is a VAE, KL scheduling may be performed during training, starting with low KL coefficients and increasing the KL coefficients over time, as described above. Figure 6-7 As stated above.

[0096] In step 1314, if model trainer 116 determines to continue training, method 1200 returns to step 1302, where model trainer 118 samples another motion from the reference motion and the time step within the motion. For example, training may terminate after a certain number of training iterations or if the similarity loss does not improve significantly after several training iterations.

[0097] Figure 14 A flowchart illustrating method steps for generating character animations given a sparse target, according to various embodiments, is provided. Although combined... Figure 1-7 and Figure 9-10 The method steps have been described, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this invention.

[0098] As shown, method 1400 begins at step 1402, where the control application 146 receives a sparse target. The sparse target can be specified by the user in any technically feasible manner, such as via a GUI. As described, the sparse target can specify one or more tasks to be performed by the character using any number of modalities that the partial constraint controller 152 can be trained to process. For example, in some embodiments, the sparse target may include: the position and / or orientation of any number of joints of the character in any number of frames of the animation, including fully observed motion without masking; a textual description of one or more tasks to be performed by the character; and / or an object that the character will interact with. In some embodiments where the sparse target includes an object, the object can be specified in any technically feasible manner, such as using a point cloud or bounding box calculated from an image of the object or specified by the user.

[0099] In step 1404, the control application 146 encodes the sparse target using the modality-specific encoder 710 to generate tags and concatenates the tags with a tag mask. In some embodiments, the control application 146 inputs different modalities from the sparse target into the corresponding modality-specific encoder 710, which outputs tags. In such cases, a converter attention mechanism can be used to mask unused tags via the tag mask, thereby preventing the converter from processing unspecified inputs, as described above. Figure 10 In some embodiments, the modality-specific encoder 710 may also encode other inputs, such as current character state, surrounding terrain, and past poses, into the partially constrained controller 152.

[0100] In step 1406, the control application 146 predicts the distribution based on the states of the tag, tag mask, and role, using the prior 714 of the partially constrained controller 152. In some embodiments, the control application 146 inputs the states of the tag, tag mask, and role into the prior 714, which outputs the distribution.

[0101] In step 1408, the control application 146 samples the distribution to obtain a latent vector. In some embodiments, the control application 146 may sample the distribution by sampling random noise and then using a reparameterization technique to obtain a latent vector.

[0102] In step 1410, the control application 146 generates an action based on the latent vector and the character's state. In some embodiments, the control application 146 inputs the latent vector and the character's current state into the decoder 720, and the decoder 720 outputs the action.

[0103] In step 1412, the control application 146 uses the generated actions to control the character within the environment 170. In some embodiments, the control application 146 transmits the actions to the character's controller, such as a PD controller, or directly to the character, thereby controlling the movement of the character's joints within the environment 170 based on the actions.

[0104] In step 1414, the control application 146 receives the character's state from the environment 170. The state may include updated joint positions and orientations after the character performs an action.

[0105] In step 1416, if the control application 146 determines to continue controlling the role, method 1400 returns to step 1406, where the control application 146 again uses prior 714 to predict the distribution based on the tag, tag mask, and the role's state.

[0106] In summary, techniques for animating characters using sparse targets are disclosed. In some embodiments, sparse targets can be specified in various modalities, such as joint constraints, text descriptions, and / or objects that the character interacts with. A control application processes the target input in each modality using a corresponding modality-specific encoder to generate markers. Given the markers, a marker mask indicating which markers are associated with unspecified inputs, and the character's current state, the control application samples a prior latent distribution generated by a prior of a trained partially constrained controller to obtain a sampled latent vector. The character can be a virtual character in a computer-based environment or a physical robot in a real-world environment, and the character's current state can be received from the computer-based environment or sensed using sensors in the real-world environment. The control application inputs the sampled latent vector and the character's current state into a decoder of the partially constrained controller to generate an action. The control application can then use the action to control the character in either the computer-based or real-world environment. Control of the character can result in an updated state of the character, and the above process can be repeated to generate another action for controlling the character using the updated state of the character, the sparse target, and the partially constrained controller.

[0107] The model trainer can perform a two-stage training technique to train a partially constrained controller. In the two-stage technique, the model trainer (1) trains a fully constrained controller using reinforcement learning to predict a sequence of actions that reconstruct a reference motion in a simulation, and then (2) trains a partially constrained controller using supervised imitation learning to recover the same actions as the trained fully constrained controller for a masked target in the simulation. For each of several iterations, the reinforcement learning may include calculating a reward based on the difference between the character's state after performing an action output by the fully constrained controller and the character's true baseline state in the reference motion, and updating the parameters of the fully constrained controller based on the reward. In some embodiments, the reward may also include one or more regularization terms related to motion, such as one or more regularization terms for reducing energy consumption, minimizing impact, and / or minimizing motor jitter. Supervised imitation learning may include repeatedly sampling motion from a reference motion and time steps within the motion, sampling a mask of a target associated with the sampled motion, simulating actions computed by a partially constrained controller to achieve the masked target, computed a true baseline action using a fully constrained controller, computed a similarity loss (e.g., L2 or KL divergence loss) based on a comparison between the action and the true baseline action, and updating the parameters of the partially constrained controller based on the similarity loss. More generally, the second phase of training may include a combination of matching the true baseline action and / or maximizing the reward.

[0108] Compared to existing technologies, at least one technical advantage of the disclosed technology is that it allows for the animation of physical or virtual characters to perform different movements without specifying all joints of the character in any number of frames of the animation. The animation generated using the disclosed technology is also physically more plausible than animations achievable using kinematic models that do not consider the forces causing the character's joint movement. Furthermore, the disclosed technology allows animators to effectively control the animation by specifying joint constraints, text descriptions, and / or the objects the character interacts with. These technical advantages represent one or more technical improvements relative to existing methods.

[0109] 1. In some embodiments, a computer-implemented method for animate a character includes: receiving one or more targets specified in one or more modalities; generating a first action to be performed by the character via a trained machine learning model and based on the one or more targets, wherein the trained machine learning model is trained to handle inputs from multiple modalities; and causing the character to perform the first action in a computer-based environment or a physical environment.

[0110] 2. The computer-implemented method according to Clause 1, wherein generating the first action comprises: encoding the one or more targets to generate one or more markers; sampling a prior distribution based on the state of the character, the one or more markers, and one or more masks associated with the one or more markers to generate a latent vector; and processing the latent vector and the state of the character using a decoder included in the trained machine learning model to generate the first action.

[0111] 3. The computer-implemented method according to Clause 1 or 2, wherein sampling the prior distribution comprises: processing the state of the role, the one or more tags, and the one or more masks using priors included in the trained machine learning model to generate a latent distribution; and sampling the latent vectors from the latent distribution.

[0112] 4. The computer-implemented method according to any one of clauses 1-3 further includes: training a first machine learning model to obtain the trained machine learning model, wherein the first machine learning model includes an encoder.

[0113] 5. A computer-implemented method according to any one of clauses 1-4, wherein sampling the prior distribution includes sampling random noise and performing one or more reparameterization operations on the random noise to generate the latent vector.

[0114] 6. A computer-implemented method according to any one of clauses 1-5, wherein the one or more targets include at least one of: a set of constraints, text descriptions, or objects that the character needs to interact with, associated with a subset of joints belonging to the character in one or more frames.

[0115] 7. The computer-implemented method according to any one of clauses 1-6, wherein the trained machine learning model includes at least one of a trained variational autoencoder (VAE) or a trained generative model.

[0116] 8. The computer-implemented method according to any one of clauses 1-7, further comprising: generating a second action to be performed by the character after the first action via the trained machine learning model and based on the one or more objectives; and causing the character to perform the second action in the computer-based environment or physical environment.

[0117] 9. The computer-implemented method according to any one of claims 1-8, further comprising: training a first machine learning model to generate the trained machine learning model based on a loss, said loss being a measure of a comparison between actions generated by the first machine learning model and actions generated by a second machine learning model, wherein reinforcement learning is used to train the second machine learning model to reproduce one or more movements from a set of motion records.

[0118] 10. A computer-implemented method according to any one of claims 1-9, wherein the first machine learning model is trained using one or more motions sampled from a set of motion records, and the one or more motions are masked based on one or more sampled masks.

[0119] 11. In some embodiments, one or more non-transitory computer-readable media store instructions that, when executed by at least one processor, cause the at least one processor to perform the following steps: receiving one or more targets specified in one or more modalities; generating a first action to be performed by a character based on the one or more targets via a trained machine learning model, wherein the trained machine learning model is trained to handle inputs from multiple modalities; and causing the character to perform the first action in a computer-based environment or a physical environment.

[0120] 12. One or more non-transitory computer-readable media as described in Clause 11, wherein generating the first action comprises: encoding the one or more targets to generate one or more markers; sampling a prior distribution based on the state of the character, the one or more markers, and one or more masks associated with the one or more markers to generate a latent vector; and processing the latent vector and the state of the character using a decoder included in the trained machine learning model to generate the first action.

[0121] 13. One or more non-transitory computer-readable media according to clause 11 or 12, wherein the prior distribution is generated by a prior including a transformer-based neural network, and the decoder includes a fully connected neural network.

[0122] 14. One or more non-transitory computer-readable media according to any one of clauses 11-13, wherein the one or more targets include at least one of: a set of constraints, text descriptions, or objects that the character is to interact with, associated with a subset of joints belonging to the character, of one or more frames.

[0123] 15. One or more non-transitory computer-readable media according to any one of clauses 11-14, wherein the instructions, when executed by the at least one processor, further cause the at least one processor to perform the step of training a first machine learning model to generate the trained machine learning model based on a loss, the loss being a measure of a comparison between actions generated by the first machine learning model and actions generated by a second machine learning model, wherein reinforcement learning is used to train the second machine learning model to reproduce one or more movements from a set of motion records.

[0124] 16. One or more non-transitory computer-readable media according to any one of clauses 11-15, wherein the first machine learning model is trained using one or more motions sampled from a set of motion records, and the one or more motions are masked based on one or more sampled masks.

[0125] 17. One or more non-transitory computer-readable media according to any one of clauses 11-16, wherein training the first machine learning model further includes increasing the value of the Kullback-Leibler (KL) coefficients during successive iterations of the training.

[0126] 18. One or more non-transitory computer-readable media according to any one of clauses 11-17, wherein the role includes a virtual character or a physical robot.

[0127] 19. One or more non-transitory computer-readable media according to any one of clauses 11-18, wherein the environment is at least one of a simulated environment, an extended reality (XR) environment, a gaming environment, or a physical environment.

[0128] 20. In some embodiments, a system includes: one or more memories storing instructions; and one or more processors coupled to the one or more memories and configured, upon execution of the instructions, to: receive one or more targets specified in one or more modalities; generate a first action to be performed by a character based on the one or more targets via a trained machine learning model, wherein the trained machine learning model is trained to handle inputs from multiple modalities; and cause the character to perform the first action in a computer-based environment or a physical environment.

[0129] Any element of any claim referenced in any claim and / or any combination of any element described in this application in any manner falls within the intended scope of this disclosure and protection.

[0130] Various embodiments have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

[0131] Aspects of this embodiment can be implemented as a system, method, or computer program product. Therefore, aspects of this disclosure can take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects (generally referred to herein collectively as a "module" or "system"). Furthermore, aspects of this disclosure can take the form of a computer program product embodied in one or more computer-readable media on which computer-readable program code is implemented.

[0132] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) will include the following: electrical connections having one or more wires, portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the context of this document, a computer-readable storage medium may be any tangible medium that may contain or store programs for use by or associated with an instruction execution system, apparatus, or device.

[0133] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine. When executed via a processor of a computer or other programmable data processing apparatus, the instructions enable the implementation of the functions / actions specified in the flowchart and / or one or more block diagram blocks. Such processors can be, but are not limited to, general-purpose processors, special-purpose processors, application-specific processors, or field-programmable gate arrays.

[0134] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, comprising one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in some alternative implementations, the functions indicated in the blocks may not appear in the order shown in the figures. For example, depending on the functions involved, two blocks shown successively may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. It will also be noted that each block in the block diagram and / or flowchart illustrations, and combinations of blocks in the block diagram and / or flowchart illustrations, can be implemented by a system based on dedicated hardware that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0135] While the foregoing pertains to embodiments of this disclosure, other and additional embodiments of this disclosure may be designed without departing from the basic scope of this disclosure, the scope of which is determined by the appended claims.

Claims

1. A computer-implemented method for animate a character, the method comprising: Receive one or more targets specified in one or more modalities; A first action to be performed by a character is generated via a trained machine learning model and based on one or more objectives, wherein the trained machine learning model is trained to handle inputs from multiple modalities; as well as The character performs the first action in a computer-based or physical environment.

2. The computer-implemented method according to claim 1, wherein, Generating the first action includes: Encode the one or more targets to generate one or more tags; The prior distribution is sampled based on the role's state, the one or more markers, and one or more masks associated with the one or more markers to generate a latent vector; and The latent vector and the state of the role are processed using a decoder included in the trained machine learning model to generate the first action.

3. The computer-implemented method according to claim 2, wherein, Sampling the prior distribution includes: The priors included in the trained machine learning model are used to process the state of the role, the one or more tags, and the one or more masks to generate a latent distribution; and The latent vectors are sampled from the latent distribution.

4. The computer-implemented method according to claim 2 further includes: A first machine learning model is trained to obtain the trained machine learning model, wherein the first machine learning model includes an encoder.

5. The computer-implemented method according to claim 2, wherein, Sampling the prior distribution includes sampling random noise and performing one or more reparameterization operations on the random noise to generate the latent vector.

6. The computer-implemented method according to claim 1, wherein, The one or more targets include at least one of the following: a set of constraints, text descriptions, or objects that the character needs to interact with, associated with a subset of joints belonging to the character in one or more frames.

7. The computer-implemented method according to claim 1, wherein, The trained machine learning model includes at least one of a trained variational autoencoder (VAE) or a trained generative model.

8. The computer-implemented method according to claim 1, further comprising: The second action to be performed by the character after the first action is generated using the trained machine learning model and based on one or more objectives; as well as The character performs the second action in the computer-based or physical environment.

9. The computer-implemented method according to claim 1, further comprising: A first machine learning model is trained to generate the trained machine learning model based on a loss, which is a measure of the comparison between actions generated by the first machine learning model and actions generated by a second machine learning model, wherein reinforcement learning is used to train the second machine learning model to reproduce one or more movements from a set of motion records.

10. The computer-implemented method according to claim 9, wherein, The first machine learning model is trained using one or more motions sampled from a set of motion records, and the one or more motions are masked based on one or more sampled masks.

11. One or more non-transitory computer-readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform the following steps: Receive one or more targets specified in one or more modalities; The first action to be performed by the character is generated using a trained machine learning model based on one or more objectives, wherein... The trained machine learning model is trained to handle inputs from multiple modalities; as well as The character performs the first action in a computer-based or physical environment.

12. One or more non-transitory computer-readable media according to claim 11, wherein, Generating the first action includes: Encode the one or more targets to generate one or more tags; The prior distribution is sampled based on the role's state, the one or more markers, and one or more masks associated with the one or more markers to generate a latent vector; and The latent vector and the state of the role are processed using a decoder included in the trained machine learning model to generate the first action.

13. One or more non-transitory computer-readable media according to claim 12, wherein, The prior distribution is generated by a prior including a transformer-based neural network, and the decoder includes a fully connected neural network.

14. One or more non-transitory computer-readable media according to claim 11, wherein, The one or more targets include at least one of the following: a set of constraints, text descriptions, or objects that the character needs to interact with, associated with a subset of joints belonging to the character in one or more frames.

15. One or more non-transitory computer-readable media according to claim 11, wherein, When executed by the at least one processor, the instructions also cause the at least one processor to perform the step of training a first machine learning model to generate the trained machine learning model based on a loss, the loss being a measure of a comparison between actions generated by the first machine learning model and actions generated by a second machine learning model, wherein reinforcement learning is used to train the second machine learning model to reproduce one or more movements from a set of motion records.

16. One or more non-transitory computer-readable media according to claim 15, wherein, The first machine learning model is trained using one or more motions sampled from a set of motion records, and the one or more motions are masked based on one or more sampled masks.

17. One or more non-transitory computer-readable media according to claim 15, wherein, Training the first machine learning model also includes increasing the value of the Kullback-Leibler (KL) coefficient during successive iterations of the training.

18. One or more non-transitory computer-readable media according to claim 11, wherein, The roles include virtual characters or physical robots.

19. One or more non-transitory computer-readable media according to claim 11, wherein, The environment is at least one of a simulation environment, an extended reality (XR) environment, a gaming environment, or a physical environment.

20. A system comprising: One or more memories, wherein the one or more memories store instructions; as well as One or more processors, said one or more processors coupled to said one or more memories, and configured, when executing said instructions, to: Receive one or more targets specified in one or more modalities; A first action to be performed by a character is generated via a trained machine learning model and based on one or more objectives, wherein the trained machine learning model is trained to handle inputs from multiple modalities; as well as The character performs the first action in a computer-based or physical environment.

Images