Creating variants of animated avatar models using low-resolution cages

Abstract

Several implementations relate to methods, systems, and computer-readable media for creating variants of template avatars. In some implementations, the method includes the steps of: obtaining a template avatar containing template geometry obtained from the mesh of the template avatar; generating a template cage associated with the template avatar as a low-resolution approximation wrapped around the template geometry; creating a target cage from the template cage by modifying the template cage based on user input; and morphing the template geometry with the target cage to generate a target avatar, which is a variant of the template avatar. The method may further include the steps of adjusting the rigging and skinning of the target avatar to enable animation of the target avatar. Using these techniques makes creating variants of template avatars more efficient and less labor-intensive.

Description

[Technical Field]

[0001] Cross-reference of related applications This application claims priority to U.S. Provisional Application No. 63 / 465,621, filed on 11 May 2023, entitled "CREATION OF VARIANTS OF AN ANIMATED AVATAR MODEL USING LOW-RESOLUTION CAGES," the contents of which are incorporated in their entirety herein.

[0002] The implementations generally relate to computer graphics, and more specifically, to methods, systems, and computer-readable media for creating and animating variants of template avatars, though not exclusively. [Background technology]

[0003] Creating visually compelling animated avatars is a time-consuming process that requires advanced expertise in 3D modeling, character rigging, and animation. In the context of building 3D animable avatars, enabling user-generated content (UGC) or supporting developers with minimal 3D character creation experience can be challenging.

[0004] Based on the above, several implementation forms were devised.

[0005] The background information provided herein is for the purpose of presenting the context of this disclosure. The research of the currently named inventors to the extent described in this background section, as well as aspects of the main text of the specification that might otherwise not qualify as prior art at the time of filing, are not expressly or implicitly recognized as prior art to the prior disclosure. [Overview of the Initiative] [Means for solving the problem]

[0006] Embodiments of the present application relate to creating high-quality variants of avatars. For example, various techniques are used to create high-quality avatars while minimizing human labor. These techniques include obtaining information about a template avatar that includes template geometry, generating a template cage that approximates the template avatar, creating a target cage from the template cage based on user input, and morphing the template geometry with the target cage to generate a target avatar.

[0007] One or more computer systems can be configured to perform certain operations or actions by having software, firmware, hardware, or combinations thereof installed on the system that cause the system to perform actions during operation. One or more programs can be configured to perform certain operations or actions by including instructions that cause an action to be performed by a data processing device.

[0008] According to one aspect, a computer-implemented method for creating a variant of a template avatar is provided. The method includes obtaining a template avatar that includes template geometry obtained from a mesh of the template avatar, generating a template cage associated with the template avatar as a low-resolution approximation wrapped around the template geometry, creating a target cage from the template cage by modifying the template cage based on input from a user, and morphing the template geometry with the target cage to generate a target avatar that is a variant of the template avatar.

[0009] In this specification, various embodiments of the computer-implemented method are described.

[0010] In some implementations, the computer-implemented method further includes adjusting the rigging and skinning of the target avatar to enable animation of the target avatar.

[0011] In some implementations, the template avatar further includes a template head of the template avatar, the target avatar includes a target head of the target avatar, and the step of adjusting the rigging and skinning includes one or more of the steps of determining the pose of the target head based on a particular pose of the template head and determining the expression of the target head based on a particular expression of the template head.

[0012] In some implementations, the step of adjusting the rigging and skinning of the target avatar includes converting the mesh of the template avatar into a planar panel mesh, deforming the planar panel mesh into a deformed neutral based on the neutral pose of the target avatar, performing retargeting on the deformed neutral to obtain the deformed rig of the template avatar, stitching the deformed rig to the shape of the target avatar to generate a stitched rig having rigging and skinning, and performing skin diffusion on the stitched rig to obtain the target avatar after stitching.

[0013] In some implementations, the computer-implemented method further includes defining a spatial deformation function that maps points in the 3D world coordinates of the planar panel mesh to 3D points in the deformed neutral, and using the spatial deformation function to deform the planar panel mesh into the deformed neutral.

[0014] In some implementations, a template avatar is associated with multiple poses encoded via a facial action coding system (FACS), and the steps of performing retargeting include performing a shape resolution operation using a spatial deformation function to map each pose of the multiple poses to generate a set of deformed pose shapes, and performing a joint resolution operation which includes using a set of deformed neutral and deformed pose shapes that serve as ground truth shapes to construct a linearly blended skinned rig for the target avatar.

[0015] In some implementations, the step of morphing the template geometry of a template avatar with a target cage to generate a target avatar includes the step of using at least one surface-based deformation technique.

[0016] In some implementations, the step of using at least one surface-based deformation technique includes the steps of performing a wrap deformation to provide a wrapped deformation version of the template avatar, and selecting a sparse subset of deltas based on the wrapped deformation version of the template avatar.

[0017] In some implementations, at least one surface-based deformation technique involves variational optimization.

[0018] In some implementations, variational optimization involves radial basis function optimization to find the displacement field, and then applying the displacement field to a template avatar to generate the target avatar.

[0019] In some implementations, the computer implementation method further includes the step of performing at least one of hidden surface tracking or Laplacian fitting.

[0020] In some implementations, the steps of performing hidden surface tracking include generating a first hidden surface based on a template cage, adding embedding equivalents of the vertices of the template avatar to the first hidden surface, generating a second hidden surface based on a target cage, and projecting the vertices of the target avatar to the corresponding equivalents in the second hidden surface based on the first hidden surface and the embedding equivalents of the first hidden surface.

[0021] In some implementations, Laplacian fitting involves solving a Poisson problem to reconstruct a target avatar based on a modified Laplacian designed to reproduce the regular geometric shape of the target cage by generating a surface of the target avatar that satisfies delta-fit constraints.

[0022] In another embodiment, a non-temporary computer-readable medium is provided. The non-temporary computer-readable medium stores instructions causing the processing device to perform actions in response to execution by the processing device, including: obtaining a template avatar including template geometry obtained from the mesh of a template avatar; generating a template cage associated with the template avatar as a low-resolution approximation wrapped around the template geometry; creating a target cage from the template cage by modifying the template cage based on user input; and morphing the template geometry with the target cage to generate a target avatar which is a variant of the template avatar.

[0023] This specification describes various implementations of non-temporary computer-readable media.

[0024] In some implementations, the operation further involves adjusting the rigging and skinning of the target avatar to enable the animation of the target avatar.

[0025] In some implementations, adjusting the rigging and skinning of the target avatar includes converting the template avatar's mesh to a planar panel mesh, deforming the planar panel mesh into a deformed neutral based on the target avatar's neutral pose, performing retargeting on the deformed neutral to obtain the deformed rig of the template avatar, stitching the deformed rig to the shape of the target avatar to generate a stitched rig with rigging and skinning, and performing skin diffusion on the stitched rig after stitching to obtain the target avatar.

[0026] In some implementations, morphing the template geometry of a template avatar with a target cage to generate a target avatar involves using at least one surface-based deformation technique.

[0027] In another embodiment, a system is disclosed comprising a memory storing instructions and a processing device coupled to the memory, wherein the processing device is configured to access the memory, and when an instruction is executed by the processing device, it causes the processing device to perform actions including: obtaining a template avatar including template geometry obtained from a mesh of a template avatar; generating a template cage associated with the template avatar as a low-resolution approximation wrapped around the template geometry; creating a target cage from the template cage by modifying the template cage based on user input; and morphing the template geometry with the target cage to generate a target avatar which is a variant of the template avatar.

[0028] This specification describes various implementations of the system.

[0029] In some implementations, the operation further involves adjusting the rigging and skinning of the target avatar to enable the animation of the target avatar.

[0030] In some implementations, adjusting the rigging and skinning of the target avatar includes converting the template avatar's mesh to a planar panel mesh, deforming the planar panel mesh into a deformed neutral based on the target avatar's neutral pose, performing retargeting on the deformed neutral to obtain the deformed rig of the template avatar, stitching the deformed rig to the shape of the target avatar to generate a stitched rig with rigging and skinning, and performing skin diffusion on the stitched rig after stitching to obtain the target avatar.

[0031] In yet another aspect, details of the systems, methods, and non-temporary computer-readable media, features, and implementations may be omitted and / or modified from some or more of the individual components or features and combined to form additional aspects including additional components or features and / or other modifications, all of which are within the scope of the disclosure. [Brief explanation of the drawing]

[0032] [Figure 1] This is a diagram illustrating an exemplary system architecture for creating and animating variants of a template avatar using several implementation methods. [Figure 2] This is a flowchart illustrating exemplary methods for creating and animating variants of a template avatar using several implementation forms. [Figure 3] This flowchart illustrates exemplary methods for adjusting the rigging and skinning of a target avatar using several implementation configurations. [Figure 4]This figure shows an example workflow for creating a head cage and the resulting animable head variant, using several implementation methods. [Figure 5] This figure shows examples of planar panel meshes and rigs in several implementation forms. [Figure 6] This figure shows examples of automated processing pipelines for deforming the geometry of a planar face panel, using several implementation configurations. [Figure 7] This figure shows further details of the automated processing pipeline in several implementation forms. [Figure 8] This figure shows further details of the automated processing pipeline in several implementation forms. [Figure 9] This figure shows examples of shape transfer using existing dynamic heads, based on several implementation configurations. [Figure 10] This figure shows examples of two-step methods, including shape transfer and linear blend skinning (LBS) rig resolution, in several implementation forms. [Figure 11] This figure shows an example function that performs shape transfer via spatial deformation, with several implementation variations. [Figure 12] This figure shows a rig transfer technique that automatically transfers rigs, skinning, and poses to a target, given a correspondence between neutral representations in several implementation forms. [Figure 13] This block diagram shows exemplary computing devices in several implementation forms. [Modes for carrying out the invention]

[0033] In the following detailed description, references are made to the accompanying drawings that form part of this specification. In the drawings, unless otherwise indicated by different context, similar symbols typically identify similar components. The exemplary implementations described in the detailed description, drawings, and claims are not intended to be limiting. Other implementations may be used and other modifications may be made without departing from the gist or scope of the subject matter presented herein. The aspects of this disclosure described herein and shown in the figures may be arranged, substituted, combined, and separated in a wide variety of different configurations, all of which are contemplated herein.

[0034] References in this specification to “several implementations,” “implementation forms,” and “exemplary implementation forms” indicate that the described implementation forms may include certain features, structures, or characteristics, but not all implementation forms necessarily include those features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same implementation form. Moreover, if certain features, structures, or characteristics are described in relation to an implementation form, such features, structures, or characteristics may also be present in relation to other implementation forms, whether explicitly described or not.

[0035] This disclosure is particularly directed toward a technique for deforming a “template” avatar (or “avatar rig”) into a target shape and automatically transferring the complex rigging elements of the template avatar to the target shape. This disclosure provides a method for adapting an existing avatar rig to generate a new variant. This adaptation can be achieved by an intuitive workflow that deforms the existing “template” avatar rig into a target shape and automatically transfers the complex rigging elements to the target avatar.

[0036] The implementation uses a low-resolution cage wrapped around the geometry of the template avatar to infer the surface correspondence between the existing template avatar rig and the new target variant of the template avatar rig and to establish a deformation field. To create the new avatar variant, the proposed workflow includes two stages. Stage 1 may be a manual stage. For example, in the manual stage, the creator may create and / or sculpt a cage (e.g., via some digital content creation (DCC) tool or procedurally via scripting) to define the new rough shape of the part. This is the only manual step that should be performed by the creator and requires significantly less expertise than the other methods.

[0037] Stage 2 may be an automated stage. For example, in the automated stage, automated techniques may morph the underlying geometry and transfer facial expressions. The implementation may further adapt joints and skinning to create a target face rig optimized for runtime performance on mobile devices.

[0038] Automation reduces the time required to create character rigs and poses from long periods (such as a month) to much shorter periods (such as seconds). A workflow may exist in which the head is morphed via cage sculpting, which automatically generates the identity of a head posed to match the facial expression. In such a workflow, the user sculpts an existing cage to the shape they desire. The automated technique then adapts the joints and skinning, and morphs the original geometry to the morphed target.

[0039] Once the user sculpts the cage to the intended shape, the following key steps of an automated technique are performed to create an animated avatar variant. In the cage morphing process, the technique morphs the underlying avatar portion so that its rough shape matches the changes in the cage shape. Several methods have been considered. For example, the methods include spatial deformation, wrap deformation, and surface deformation via a radial basis function (RBF). In some implementations, surface deformation can provide an effective way to create animated avatar variants.

[0040] Current formulations of surface deformation include three terms in the optimization framework: variational optimization, implicit surface tracking, and Laplacian fitting. Variational optimization imposes physical constraints to ensure reasonable results. Implicit surface tracking extends implicit skinning techniques to surface-based deformation. Laplacian fitting can be a variation of the Laplacian surface editing technique used to reproduce regular geometric primitive properties.

[0041] Cage morphing can be followed by pose / expression transfer. This technique adjusts joints and skinning to adapt the expressions and poses of the initial geometry to plausible expressions / poses for the cage-morphed geometry. A key feature is the use of spatial deformers (e.g., radial basis functions (RBFs)) calculated from the cage morphing step to compute target geometry vertices for each expression and body pose.

[0042] After pose / expression transfer, rig creation may follow. For linear blend skinning (LBS) rigs, the computer target vertices from the pose / expression transfer may be used as constraints to a known solver for adjusting joint deformation and skinning. For blend shape rigs, the implementation may involve calculating vertex deltas for each pose / expression.

[0043] In some implementations, examples of animated avatar rigs and their corresponding cages may exist. Specifically, as shown in the examples, an avatar rig can be an animable / animated template avatar represented by a mesh. Multiple low-resolution cages are created and wrapped around parts of the avatar rig (e.g., head, torso, hands, feet, etc.). In other contexts, cages may be used to wrap clothing and facial accessories around an avatar, and the implementations disclosed herein use cages to generate new variants of a template avatar.

[0044] One aspect of the example may include a pose / expression transfer technique. The technique adjusts joints and skinning to adapt the expressions and poses of the initial geometry (e.g., head geometry) to plausible expressions / poses for the cage-morphed geometry (e.g., head variant geometry). One aspect of this feature is the use of spatial deformers (e.g., RBFs) calculated from the cage-morphing step to compute target geometry vertices for each expression and body pose.

[0045] Another aspect is rig creation. For linear blend skinning (LBS) rigs, automated techniques can use calculated target vertices from pose / expression transfers as constraints to known solvers for adjusting joint deformations and skinning. For blend shape rigs, automated techniques can calculate vertex deltas for each pose / expression. Further details of pose / expression transfers (e.g., shape resolution) and rig creation (e.g., LBS rig resolution) in various implementation forms are provided herein.

[0046] Depending on the implementation, a platform is provided in which each user can have an expressive and communicative avatar. The platform may have a wide variety of avatars with static (e.g., non-animable) heads that the user community uses to express themselves in virtual experiences.

[0047] The ability of one or more avatars on a media platform to display animable facial expressions presents the challenge of converting one or more of the static heads of one or more avatars into versions that support high-quality facial expressions. This presents a scalability challenge, as one or more static heads are typically constructed as a combination of static decals and simple head shape meshes. The avatar's face, mouth, eyes, and other facial features are generally not represented by geometry and are therefore not easily animated. An exemplary media platform may have approximately 600 static decals and approximately 50 head shapes. Such an example corresponds to the possibility of 30,000 different heads that could be made animable.

[0048] Separating a static head into decals and head shapes allows for the separation of rigged, skinned, and animated faces from their head shapes. Artists (and other creators) can generate rigged, animable faces on planar panel meshes, efficiently simplifying the creation of geometry, rigs, and animations and bringing it closer to a two-dimensional (2D) problem. The implementations of the automated face stitching techniques described herein allow the pipeline to morph and stitch these artist-generated planar face panels into arbitrary head shapes. This morphing and stitching enables the large-scale generation of animated face and head shape combinations.

[0049] The implementation may provide a workflow that transforms one or more avatars with static heads into avatars with animable facial expressions. According to some implementations, exemplary static heads may exist on the media platform. Various head shapes and static decals may be provided.

[0050] Each specific static head may be the result of texturing a selected head shape (e.g., one of approximately 50 available shapes) with a defined UV mapping that has static face decals (e.g., approximately 600 decals are available) live at runtime. Separating the head shape from the static decal combinatorially expands the number of unique heads, resulting in a greater variety of static head avatars for the user to choose from.

[0051] Manually converting a static head to a head with an animable facial expression can be a time-consuming process. Such a manual conversion may involve modeling the geometry with animable parts like a static face decal, creating a joint rig, skinning the vertices, and finally posing the geometry to the expression. Performing manual conversions individually for every combination of head shape and face decal in an avatar / head catalog is not scalable. Therefore, the implementation disclosed in this specification employs a combination approach that separates the creation of the animable face rig from the final head shape.

[0052] Using an automated processing pipeline to stitch animable faces and their rigs into a head shape mesh can simplify the creation of animable face meshes and rigs, which are primarily flat, square face surfaces. For example, several implementations may use an example of a planar panel face mesh and rig. There may be front view, side view, rig joints, and Face Action Coding System (FACS) poses that, when blended, create expressions. In one example, a small number of FACS poses (e.g., 6 poses) may be used, but for use in a rig-targeting solver, many more (such as between 85 and 120 poses) can be defined (again, as an example).

[0053] The techniques discussed herein eliminate the need to handle the curvature of any particular head shape during rigging and animation. These planar panel face rigs can be 2.5-dimensional (2.5D) because there is some depth in the off-surface components for the shape of the mouse bag and the teeth / tongue area inside, as well as for the eye area and other facial features. The animation used for the eyes, eyebrows, and nose of the face can be primarily 2D. This less complex approach greatly simplifies the rigging, skinning, and animation processes. By design, the main faces of planar panel face meshes and rigs can be square such that the xy coordinates of the vertices are precisely uv coordinates in UV space.

[0054] With the planar panel face mesh and rig created, the next step is an automated method for stitching the planar panel face rig onto an existing head shape. The automated processing pipeline stitches the planar panel face mesh and rig onto a head shape mesh defined by UV mapping, which is similar to how actual textures are applied to a mesh. Given the planar face mesh and target face shape, the automated pipeline generates a fully rigged, animable, stitched head.

[0055] In some implementations, the automated processing pipeline deforms the geometry of the planar face panel. The planar face panel is retargeted to a head shape using acquired UV mapping, resulting in a fully rigged, animated, stitched head with facial expressions.

[0056] Several technical challenges in this processing pipeline require techniques to address, which may include: The planar panel is deformed onto an arbitrarily shaped surface of the head shape mesh. If the planar panel is very thin, this is a simple mapping via UV coordinates and signed normal distance from the face surface. However, the planar panel is a 2.5D planar panel. Also, the planar panel may have an inner mouth pouch with areas for teeth and tongue. The challenge lies in determining how these areas deform when the face surface is stitched onto a curved surface.

[0057] The rig of a planar face panel should be retargeted to improve the appearance of the resulting facial expressions on the stitched head. Since it is useful to be able to stitch a planar face panel to any arbitrary head shape, the implementation may have a general methodology that can take into account a wide range of curvature differences and anisotropic scaling. This general methodology may enable the use of a wide variety of head shapes.

[0058] In some implementations, the first part of the automated processing pipeline deforms the planar face panel in a neutral pose into a head shape and then retargets the rig. Retargeting the planar face panel rig into a head shape can be done in two steps, specifically a shape resolution step and a joint resolution step. The second part of the automated processing pipeline stitches the deformed face rig into a head shape to generate a stitched head rig. The automated processing pipeline may also apply some skin diffusion around the edges where the deformed face rig is stitched to obtain a smooth reduction of deformation in the final result of the pipeline.

[0059] Exemplary major weighting problems may include the following: One example is weight gaps. If weights do not extend evenly to the boundary, they can "bleed" around internal fixed weights, causing undesirable deformation. The bottom corners of a panel may be fully weighted to "pin" the heads to these corners.

[0060] Maintaining this results in a "pinned pool" of weights. Weight gaps can be corrected by applying narrowband diffusion centered on the stitch boundary and blending over a specified geodesic distance from the boundary, which may require a specified "blending mask". There may also be issues with internal overweighting. In some implementations, the weight of the mouth pouch is often too high for head shapes that do not leave much space between the skin surface and the inside of the mouth, leading to crashes if the internal weights are not adjusted as part of the diffusion process.

[0061] Overweighting of the mouth pouch can be corrected by diffusing the limiting weighting from the surface to the interior through scalar extension using the vector heat method on the mesh's point cloud. This avoids the "bleed" problem by diffusing the weights at R3. Similarly, these new weights are then extended to the lower teeth and tongue, keeping their weights proportional to the surrounding mesh weights. For example, a scalar extension of the mouth pouch may exist with respect to the tongue and lower teeth. There may also be overweighting that is corrected.

[0062] Depending on the implementation, various examples of stitching results are possible. For example, various planar face panels may be stitched onto one of several head shapes, and animation may be performed accordingly.

[0063] Accordingly, the implementations disclosed herein provide a method for stitching rigged, skinned, and posed 2.5D planar panels onto a mesh surface. Such a method is analogous to rendering textures onto a mesh via UV mapping. Fine-tuning and consideration of numerous details in a particular set of planar panels and head shapes achieves high-quality results.

[0064] Consistent geometry can be maintained across all or nearly all planar face panels. Specifically, forcing similar signed distances for 2.5D parts (e.g., eyes, eyebrows, mouth pouches, teeth, and tongue) improves the quality of the results. Forcing these distances makes the results more predictable, leading to a reduction in crashes in the final stitched asset.

[0065] In the deformation neutral step, applying subdivision surface smoothing to the head shape when deforming the planar face panel helps create higher quality stitching results. This technique removes faceting present in the original head shape and reduces associated artifacts in the final stitched asset. Defining vertices that do not move between the neutral pose and any specific FACS pose is important for creating visually appealing and intuitive retargeting.

[0066] Regarding symmetry, aside from small numerical asymmetries and the usual problems of intended asymmetric designs, unintended asymmetries introduced between both shape and joint solutions are managed by the implementation. Asymmetries introduced by joint solutions (e.g., joint stationary pose / vertex weights) generally stem from asymmetries in the input from shape solutions. However, since these asymmetries may include intended design asymmetries, modifying the shape solution output may not be sufficient to guarantee a symmetric joint solution result. Therefore, a similar re-establishment of intended symmetry is applied after the solution.

[0067] Skin diffusion, and especially the subsequent normalization (including clipping small weights and limiting the number of joints / vertices), can amplify small asymmetries in the process. Ensuring that the results of both resolution operations (shape resolution and joint resolution) are symmetrical when the result is intended to be symmetrical mitigates the problem of amplified asymmetry. However, to guarantee a proper result, the weights still need to be symmetrized after diffusion.

[0068] Regarding smoothing, a method for diffusing weights can be the graph Laplacian. This method is very fast, but the results depend only on the mesh topology. Using the cotangent Laplacian addresses the topology problem but leaves the continuity problem. Solving the polyharmonic equations using the cotangent Laplacian solves the previous problem. This method uses the Laplacian energy (EΔ 2 Rather than minimizing the Hessian energy (EH), 2 By minimizing ), a better "form" result can be obtained.

[0069] Debugging can be helpful in separating the step of finding the shape through shape resolution from the step of finding the final linear blend-skinned rig through joint resolution. Being able to visualize each individual step of the process that produces the final stitched result allows the implementation form to narrow down which step in the process is failing or producing an unacceptable result.

[0070] Figure 1 - System Architecture Figure 1 is a diagram of an exemplary system architecture for creating and animating variants of a template avatar in several implementation forms. Figure 1 and other diagrams use similar reference numbers to indicate similar elements. Letters following a reference number, such as "110," indicate that the text specifically refers to the element with that particular reference number. A reference number in text without following letters refers to one or all of the elements in the diagram that have that reference number (for example, "110" in text refers to reference numbers "110a," "110b," and / or "110n" in the diagram).

[0071] The system architecture 100 (also referred to herein as the “System”) includes an online virtual experience server 102, a data store 120, client devices 110a, 110b, and 110n (collectively referred to herein as “Client Devices” 110), and developer devices 130a and 130n (collectively referred to herein as “Developer Devices” 130). The virtual experience server 102, the data store 120, the client devices 110, and the developer devices 130 are connected via a network 122. In some implementations, client devices 110 and developer devices 130 may refer to the same or the same type of device.

[0072] The online virtual experience server 102 may include, among other things, a virtual experience engine 104, one or more virtual experiences 106, and a graphics engine 108. In some implementations, the graphics engine 108 may be a system, application, or module that enables the online virtual experience server 102 to provide graphics and animation capabilities. In some implementations, the graphics engine 108 may perform one or more of the operations described below in relation to the flowcharts shown in Figures 2 and 3. The client device 110 may include a virtual experience application 112 and an input / output (I / O) interface 114 (e.g., an input / output device). The input / output device may include one or more of the following: a microphone, speaker, headphones, display device, mouse, keyboard, game controller, touchscreen, virtual reality console, etc.

[0073] The developer device 130 may include a virtual experience application 132 and an input / output (I / O) interface 134 (for example, an input / output device). The input / output device may include one or more of the following: a microphone, speaker, headphones, display device, mouse, keyboard, game controller, touchscreen, virtual reality console, etc.

[0074] System architecture 100 is provided for illustrative purposes. In different implementations, system architecture 100 may include the same, fewer, more, or different elements configured in the same or different ways as shown in Figure 1.

[0075] In some implementations, network 122 may include public networks (e.g., the Internet), private networks (e.g., local area networks (LANs) or wide area networks (WANs)), wired networks (e.g., Ethernet networks), wireless networks (e.g., 802.11 networks, Wi-Fi® networks, or wireless LANs (WLANs)), cellular networks (e.g., 5G networks, Long-Term Evolution (LTE) networks, etc.), routers, hubs, switches, server computers, or combinations thereof.

[0076] In some implementations, the datastore 120 may be non-temporary computer-readable memory (e.g., random-access memory), a cache, a drive (e.g., a hard drive), a flash drive, a database system, or another type of component or device capable of storing data. The datastore 120 may also include multiple storage components (e.g., multiple drives or multiple databases) that may span multiple computing devices (e.g., multiple server computers). In some implementations, the datastore 120 may include cloud-based storage.

[0077] In some implementations, the online virtual experience server 102 may include a server having one or more computing devices (e.g., a cloud computing system, a rack-mount server, a server computer, a cluster of physical servers, etc.). In some implementations, the online virtual experience server 102 may be an independent system, include multiple servers, or be part of another system or server.

[0078] In some implementations, the online virtual experience server 102 may include one or more computing devices (such as rack-mount servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, and desktop computers), data stores (e.g., hard disks, memory, and databases), networks, software components, and / or hardware components that can be used to run on the online virtual experience server 102 and provide users with access to the online virtual experience server 102. The online virtual experience server 102 may also include a website (e.g., a web page) or application backend software that can be used to provide users with access to content provided by the online virtual experience server 102. For example, a user may access the online virtual experience server 102 using a virtual experience application 112 on a client device 110.

[0079] In some implementations, virtual experience session data is generated via the online virtual experience server 102, the virtual experience application 112, and / or the virtual experience application 132, and stored in the data store 120. With permission from the virtual experience participant, the virtual experience session data may include relevant metadata, such as a virtual experience identifier, device data associated with the participant, participant demographic information, a virtual experience session identifier, chat transcripts, session start time, session end time, and session duration for each participant, the relative position of the participant avatar in the virtual experience environment, purchases made by one or more participants within the virtual experience, and accessories used by the participant.

[0080] In some implementations, the online virtual experience server 102 may be a type of social network providing connections between users, or a type of user-generated content system enabling users (e.g., end-users or consumers) to communicate with other users on the online virtual experience server 102, where communication may include voice chat (e.g., synchronous and / or asynchronous voice communication), video chat (e.g., synchronous and / or asynchronous video communication), or text chat (e.g., 1:1 and / or N:N synchronous and / or asynchronous text-based communication). Records of some or all of the communications may be stored in the data store 120 or within the virtual experience 106. The data store 120 may be used to store chat transcripts (text, audio, images, etc.) exchanged between participants.

[0081] In some implementations, the chat transcript is generated by the virtual experience application 112 and / or the virtual experience application 132 and / or stored in the data store 120. The chat transcript may include chat content and associated metadata, such as the text content of the chat with corresponding senders and receivers for each message, message formatting (e.g., bold, italics, loudness, etc.), message timestamps, the relative positions of participant avatars in the virtual experience environment, and accessories used by the virtual experience participants. In some implementations, the chat transcript may include multilingual content, and messages in different languages ​​from different sessions of the virtual experience may be stored in the data store 120.

[0082] In some implementations, chat transcripts can be stored in the form of conversation between participants based on timestamps. In some implementations, chat transcripts can be stored based on the message sender.

[0083] In some implementations of this disclosure, “User” may be represented as a single individual. However, other implementations of this disclosure may include the fact that “User” (e.g., a creator user) is an entity controlled by a set of users or an automated source. For example, a set of individual users united as a community or group in a user-generated content system may be considered a “User.”

[0084] In some implementations, the online virtual experience server 102 may be a virtual game server. For example, the game server may provide a single-player or multiplayer game to a community of users accessing it as a “system” in this specification, and may include the online game server 102, a data store 120, and clients, or allow interaction with the virtual experience using client devices 110 via a network 122. In some implementations, the virtual experience (including virtual domains or virtual worlds, virtual games, and other computer-simulated environments) may be, for example, a two-dimensional (2D) virtual experience, a three-dimensional (3D) virtual experience (e.g., a 3D user-generated virtual experience), a virtual reality (VR) experience, or an augmented reality (AR) experience. In some implementations, users may participate in interactions with other users (such as gameplay). In some implementations, the virtual experience may be experienced in real time together with other users of the virtual experience.

[0085] In some implementations, virtual experience engagement may refer to interactions between one or more participants using a client device (e.g., 110) within a virtual experience (e.g., 106), or the presentation of interactions on the display of the client device 110 or other output device (e.g., 114). For example, virtual experience engagement may include interactions with one or more participants within a virtual experience, or the presentation of interactions on the display of the client device.

[0086] In some implementations, the virtual experience 106 may include electronic files that can be executed or loaded using software, firmware, or hardware configured to present virtual experience content (e.g., digital media items) to entities. In some implementations, a virtual experience application 112 may be executed, and the virtual experience 106 may be rendered in relation to the virtual experience engine 104. In some implementations, the virtual experience 106 may have a common set of rules or common goals, and the environment of the virtual experience 106 may share a common set of rules or common goals. In some implementations, different virtual experiences may have different rules or goals from one another.

[0087] In some implementations, a virtual experience may have one or more environments (hereinafter also referred to as “virtual experience environments” or “virtual environments”) to which multiple environments can be linked. An example of an environment may be a three-dimensional (3D) environment. One or more environments of virtual experience 106 may be collectively referred to herein as a “world,” or “virtual experience world,” or “game world,” or “virtual world,” or “universe.” An example of a world may be a 3D world of virtual experience 106. For example, a user may build a virtual environment linked to another virtual environment created by another user. A character in a virtual experience may cross virtual boundaries to enter an adjacent virtual environment.

[0088] Note that 3D environments or 3D worlds use graphics that employ a 3D representation of geometric data representing the virtual experience content (or, regardless of whether a 3D representation of geometric data is used, at least present the virtual experience content in a way that it appears as 3D content). 2D environments or 2D worlds use graphics that employ a 2D representation of geometric data representing the virtual experience content.

[0089] In some implementations, the online virtual experience server 102 can host one or more virtual experiences 106 and enable users to interact with the virtual experiences 106 using a virtual experience application 112 on a client device 110. Users of the online virtual experience server 102 can play, create, interact with, or build virtual experiences 106, communicate with other users, and / or create and build objects of the virtual experience 106 (for example, also referred to herein as “items,” “virtual experience objects,” or “virtual experience items”).

[0090] For example, when generating user-generated virtual items, a user may create, among other things, a character, decorations for a character, one or more virtual environments for an interactive virtual experience, or a build structure used within the virtual experience 106. In some implementations, a user may buy, sell, or exchange virtual experience objects, such as in-platform currency (e.g., virtual currency), with other users of the online virtual experience server 102. In some implementations, the online virtual experience server 102 may transmit virtual experience content to a virtual experience application (e.g., 112). In some implementations, virtual experience content (also referred to herein as “content”) may refer to any data or software instructions associated with the online virtual experience server 102 or the virtual experience application (e.g., virtual experience objects, virtual experiences, user information, videos, images, commands, media items, etc.). In some implementations, a virtual experience object (for example, also referred to herein as an “item,” “object,” “virtual object,” or “virtual experience item”) may refer to an object used, created, shared, or otherwise depicted in a virtual experience 106 on an online virtual experience server 102 or a virtual experience application 112 on a client device 110. For example, a virtual experience object may include body parts, models, characters, accessories, tools, weapons, clothing, buildings, vehicles, currency, plants, animals, and components of the aforementioned (e.g., windows of a building).

[0091] It should be noted that the online virtual experience server 102 hosting the virtual experience 106 is provided for illustrative purposes only. In some implementations, the online virtual experience server 102 may host one or more media items that may contain communication messages from one user to one or more other users. With the user's permission and explicit consent, the online virtual experience server 102 may analyze chat transcript data to improve the virtual experience platform. Media items may include, but are not limited to, digital video, digital movies, digital photographs, digital music, audio content, melodies, website content, social media updates, ebooks, e-magazines, digital newspapers, digital audiobooks, e-journals, weblogs, real simple syndication (RSS) feeds, e-comic books, and software applications. In some implementations, media items may be electronic files that can be executed or loaded using software, firmware, or hardware configured to present digital media items to entities.

[0092] In some implementations, a virtual experience 106 may be associated with a specific user or group of users (e.g., a private virtual experience), or access to the online virtual experience server 102 may be made widely available to users (e.g., a public virtual experience). In some implementations, if the online virtual experience server 102 associates one or more virtual experiences 106 with a specific user or group of users, the online virtual experience server 102 may associate a specific user with a virtual experience 106 using user account information (e.g., a user account identifier such as a username and password).

[0093] In some implementations, the online virtual experience server 102 or client device 110 may include a virtual experience engine 104 or a virtual experience application 112. In some implementations, the virtual experience engine 104 may be used for developing or running a virtual experience 106. For example, among its many features, the virtual experience engine 104 may include a rendering engine ("renderer") for 2D, 3D, VR, or AR graphics, a physics engine, a collision detection engine (and collision response), a sound engine, scripting capabilities, an animation engine, an artificial intelligence engine, networking capabilities, streaming capabilities, memory management capabilities, threading capabilities, scene graph capabilities, or video support for cinematics. The components of the virtual experience engine 104 may generate commands (e.g., rendering commands, collision commands, physics commands, etc.) that help compute and render the virtual experience. In some implementations, the virtual experience application 112 of the client device 110 may operate independently, in cooperation with the virtual experience engine 104 of the online virtual experience server 102, or in a combination of both.

[0094] In some implementations, both the online virtual experience server 102 and the client device 110 may run virtual experience engines (104 and 112, respectively). The online virtual experience server 102 using virtual experience engine 104 may run some or all of the virtual experience engine functions (e.g., generating physical commands, rendering commands, etc.), or it may offload some or all of the virtual experience engine functions to the virtual experience engine 104 of the client device 110. In some implementations, each virtual experience 106 may have a different ratio between the virtual experience engine functions run on the online virtual experience server 102 and the virtual experience engine functions run on the client device 110. For example, the virtual experience engine 104 of the online virtual experience server 102 may be used to generate physical commands if a collision exists between at least two virtual experience objects, while additional virtual experience engine functions (e.g., generating rendering commands) may be offloaded to the client device 110. In some implementations, the ratio of virtual experience engine functions executed by the online virtual experience server 102 and the client device 110 can be changed (for example, dynamically) based on virtual experience engagement conditions. For example, if the number of users involved in a particular virtual experience 106 exceeds a threshold number, the online virtual experience server 102 may execute one or more virtual experience engine functions previously executed by the client device 110.

[0095] For example, a user may be playing a virtual experience 106 on a client device 110 and may send control commands (e.g., user input such as right, left, up, down, user selection, or character position and speed information) to the online virtual experience server 102. After receiving control commands from the client device 110, the online virtual experience server 102 may send experience commands (e.g., position and speed information of characters participating in the group experience, or commands such as rendering commands or collision commands) to the client device 110 based on the control commands. For example, the online virtual experience server 102 may perform one or more logical operations on the control commands (e.g., using the virtual experience engine 104) to generate experience commands for the client device 110. In another example, the online virtual experience server 102 may pass one or more control commands from one client device 110 to other client devices participating in the virtual experience 106 (e.g., from client device 110a to client device 110b). The client device 110 can use experience commands to render a virtual experience for presentation on the client device 110's display.

[0096] In some implementations, control commands may refer to commands that indicate actions of the user's character within the virtual experience. For example, control commands may include user inputs to control actions within the experience, such as right, left, up, down, user selection, gyroscope position and orientation data, and force sensor data. Control commands may also include character position and velocity information. In some implementations, control commands are sent directly to the online virtual experience server 102. In other implementations, control commands may be sent from one client device 110 to another client device (e.g., from client device 110b to client device 110n), and the other client device generates experience commands using the local virtual experience engine 104. Control commands may include commands to play voice communication messages or other sounds from another user, such as voice communication or other sounds generated using the audio spatialization techniques described herein, in an audio device (e.g., speaker, headphones, etc.).

[0097] In some implementations, an experience command may refer to a command that enables the client device 110 to render a virtual experience, such as a multiplayer virtual experience. An experience command may include one or more of the following: user input (e.g., control commands), character position and velocity information, or commands (e.g., physics commands, rendering commands, collision commands, etc.).

[0098] In some implementations, a character (or virtual experience object in general) is composed of components, one or more of which can be selected by the user and automatically combined to assist the user during editing.

[0099] In some implementations, a character is implemented as a 3D model, which includes a surface representation (also known as skin or mesh) used to depict the character, and a hierarchical set of interconnected bones (also known as skeleton or rig). The rig can be used to animate the character and simulate movement and actions performed by the character. The 3D model can be represented as a data structure, and one or more parameters of the data structure can be modified to change various characteristics of the character, such as dimensions (height, width, waist circumference, etc.), body shape, movement patterns, number / type of body parts, proportions (e.g., shoulder-to-hip ratio), head size, etc.

[0100] One or more characters (also referred to herein as “avatars” or “models”) may be associated with a user, and the user may control the characters to facilitate user interaction with the virtual experience 106.

[0101] In some implementations, a character may include components such as body parts (e.g., hair, arms, legs, etc.) and accessories (e.g., T-shirts, glasses, decorative images, tools, etc.). In some implementations, customizable character body parts include, among other things, head type, body part type (arms, legs, torso, and hands), face type, hair type, and skin type. In some implementations, customizable accessories include clothing (e.g., shirts, trousers, hats, shoes, glasses, etc.), weapons, or other tools.

[0102] In some implementations, for certain asset types, such as shirts and trousers, an online virtual experience platform may provide users with access to simplified 3D virtual object models represented by meshes with a low polygon count, for example, between approximately 20 and 30 polygons.

[0103] In some implementations, the user may also control the character's scale (e.g., height, width, or depth) or the scale of the character's components. In some implementations, the user may also control the character's proportions (e.g., blocky, anatomical, etc.). In some implementations, the character may not include character virtual experience objects (e.g., body parts), but it should be noted that the user may still control the character (without character virtual experience objects) to facilitate user interaction with the virtual experience (e.g., a puzzle game where there are no rendered character game objects, but the user still controls the character to control actions within the game).

[0104] In some implementations, components such as body parts may be primitive geometric shapes such as blocks, cylinders, or spheres, or other primitive geometric shapes such as wedges, tori, tubes, or channels. In some implementations, the creator module may expose the user's character for viewing or use by other users of the online virtual experience server 102. In some implementations, creating, modifying, or customizing characters, other virtual experience objects, virtual experience 106, or the virtual experience environment may be done by the user using an I / O interface (e.g., a developer interface), with or without scripts (or with or without an application programming interface (API)). Note that for illustrative purposes, the character is described as having a humanoid form. Further note that the character may have any form, such as a vehicle, animal, inanimate object, or other creative form.

[0105] In some implementations, the online virtual experience server 102 may store user-created characters in the data store 120. In some implementations, the online virtual experience server 102 maintains a character catalog and a virtual experience catalog that can be presented to the user. In some implementations, the virtual experience catalog includes images of virtual experiences stored on the online virtual experience server 102. In addition, the user may select a character (e.g., a character created by the user or another user) from the character catalog to participate in a selected virtual experience. The character catalog includes images of characters stored on the online virtual experience server 102. In some implementations, one or more characters in the character catalog may be created or customized by the user. In some implementations, the selected character may have character settings that define one or more of the character's components.

[0106] In some implementations, a user's character may include the configuration of its components, and the configuration and appearance of these components, as well as the character's appearance in general, may be defined by character settings. In some implementations, the user's character settings may be selected by the user, at least partially. In other implementations, the user may select a character using default character settings or other user-selected character settings. For example, the user may select a default character from a character catalog with predefined character settings, and the user may further customize the default character by changing some of the character settings (e.g., adding a shirt with a customized logo). Character settings may be associated with a specific character by the online virtual experience server 102.

[0107] In some implementations, the client device 110 may include computing devices such as personal computers (PCs), mobile devices (e.g., laptops, mobile phones, smartphones, tablet computers, or notebook computers), network-connected televisions, and game consoles. In some implementations, the client device 110 may also be referred to as a “user device.” In some implementations, one or more client devices 110 may connect to the online virtual experience server 102 at any given moment. Note that the number of client devices 110 is provided as an example. In some implementations, any number of client devices 110 may be used.

[0108] In several implementations, each client device 110 may contain an instance of the virtual experience application 112. In one implementation, the virtual experience application 112 may enable the user to use and interact with the online virtual experience server 120, such as controlling a virtual character in a virtual experience hosted by the online virtual experience server 102, or viewing or updating content such as the virtual experience 106, images, video items, web pages, and documents. In one example, the virtual experience application may be a web application (e.g., an application that works in conjunction with a web browser) that can access, search, present, or navigate content served by a web server (e.g., virtual characters in a virtual environment). In another example, the virtual experience application may be a native application (e.g., a mobile application, app, virtual experience program, or game program) that is installed and runs locally on the client device 110 and enables the user to interact with the online virtual experience server 102. The virtual experience application may render, display, or present content (e.g., web pages, media viewers) to the user. In one implementation, a virtual experience application may also include an embedded media player (e.g., Flash® or HTML player) embedded within a web page.

[0109] In aspects of this disclosure, the virtual experience application may be an online virtual experience server application for users to build, create, edit, upload content to the online virtual experience server 120, and interact with the online virtual experience server 102 (for example, to participate in a virtual experience 106 hosted by the online virtual experience server 102). Thus, the virtual experience application may be provided to the client device 110 by the online virtual experience server 102. In another example, the virtual experience application may be an application downloaded from a server.

[0110] In several implementations, each developer device 130 may contain an instance of the virtual experience application 132. In one implementation, the virtual experience application 132 may enable a developer user to use and interact with the online virtual experience server 120, such as controlling a virtual character in a virtual experience hosted by the online virtual experience server 102, or viewing or updating content such as the virtual experience 106, images, video items, web pages, and documents. In one example, the virtual experience application may be a web application (e.g., an application that works in conjunction with a web browser) that can access, search, present, or navigate content served by a web server (e.g., virtual characters in a virtual environment). In another example, the virtual experience application may be a native application (e.g., a mobile application, app, virtual experience program, or game program) that is installed and runs locally on the developer device 130 and enables the user to interact with the online virtual experience server 102. The virtual experience application may render, display, or present content (e.g., web pages, media viewers) to the user. In one implementation, a virtual experience application may also include an embedded media player (e.g., Flash or HTML player) embedded within a web page.

[0111] In aspects of this disclosure, the virtual experience application 132 may be an online virtual experience server application for users to build, create, edit, upload content to the online virtual experience server 120, and interact with the online virtual experience server 102 (for example, providing and / or participating in a virtual experience 106 hosted by the online virtual experience server 102). Thus, the virtual experience application may be provided to the client device 110 by the online virtual experience server 102. In another example, the virtual experience application 132 may be an application downloaded from the server. The game application 132 may be configured to interact with the online virtual experience server 102 and obtain access to user credentials, user currency, etc., relating to one or more virtual experiences 106 developed, hosted, or provided by a virtual experience developer.

[0112] In some implementations, a user may log in to the online virtual experience server 102 via a virtual experience application. A user may access a user account by providing user account information (e.g., username and password), and the user account is associated with one or more characters available to participate in one or more virtual experiences 106 on the online virtual experience server 102. In some implementations, with appropriate credentials, a virtual experience developer may gain access to virtual experience virtual objects such as in-platform currency (e.g., virtual currency), avatars, special abilities, and accessories that are owned by or associated with other users.

[0113] In general, functions described in one implementation as being performed by the online virtual experience server 102 can also be performed by the client device 110 or the server in other implementations, where appropriate. In addition, functions belonging to a particular component can be performed by different components or multiple components working together. The online virtual experience server 102 can also be accessed as a service provided to other systems or devices via an appropriate application programming interface (API), and is therefore not limited to use in a website.

[0114] Figure 2 - Creating and animating variants of a template avatar. Figure 2 is a flowchart of an exemplary method 200 for creating and animating variants of a template avatar in several implementation forms. Method 200 can begin in block 210.

[0115] In block 210, a template avatar, including template geometry, is obtained. For example, template geometry may be obtained from the mesh of the template avatar. Information about the template avatar may be obtained from a virtual experience within a virtual environment. Block 220 may follow block 210.

[0116] In block 220, a template cage associated with the template avatar is generated. The template cage can be generated by wrapping the cage around the template geometry. The template cage can be a low-resolution cage (the cage has a lower resolution than the template geometry). The template cage can provide a method for inferring the surface correspondence between the existing template rig and the new target variant and establishing the deformation field. Block 230 may follow block 220.

[0117] In block 230, a target cage is created from a template cage based on user input. For example, the user sculpts the template cage (e.g., via some digital content creation (DCC) tool, or procedurally via scripting). This cage creation is the only manual part of the process. Block 240 may follow block 230.

[0118] In block 240, the geometry of the template avatar can be morphed with the target cage to generate the target avatar. Automated techniques morph the underlying geometry and transfer the facial expressions. Morphing works by morphing the underlying avatar parts so that their rough shape matches the changes in the cage shape. Several techniques exist, including spatial deformation via radial basis functions (RBFs) or wrap deformation.

[0119] However, alternative techniques may include surface deformation, which may involve an optimization framework with variational optimization, implicit surface tracking, and Laplacian fitting. Variational optimization imposes physical constraints to ensure reasonable results. Implicit surface tracking is a technique that extends implicit skinning techniques to surface-based deformation. Laplacian fitting is a variation of the Laplacian surface editing technique used to reproduce regular geometric primitive properties. Additional details regarding morphing are discussed herein. Block 240 may be followed by Block 250.

[0120] In block 250, the rigging and skinning of the target avatar may be adjusted. This action may include pose / expression transfer. The rigging and skinning adjustments adjust the joints and skinning to adapt the expressions and poses of the initial geometry to plausible expressions / poses for the cage-morphed geometry.

[0121] A key feature may be the use of spatial deformations (e.g., RBF) calculated from cage morphing steps to compute target geometry vertices for each facial expression and body pose. For linear blend skinning rigs, the target vertices computed from pose / expression transfers can be used as constraints to known solvers for adjusting joint deformations and skinning. For blend shape rigs, implementations may compute vertex deltas for each pose / expression. Block 250 may be followed by Block 260.

[0122] In block 260, the target avatar may be provided to a three-dimensional (3D) environment (such as a virtual game environment). For example, information about the generated mesh may be provided to the 3D environment for avatar rendering, avatar animation, or other applications. After block 260, the 3D environment may continue to use information about the target avatar to display or animate the target avatar.

[0123] Figure 3 - Adjusting the rigging and skinning of the target avatar Figure 3 is a flowchart of an exemplary method 300 for adjusting the rigging and skinning of a target avatar in several implementation forms. Method 300 may begin in block 310. Method 300 may correspond to block 250 in Figure 2 and provides more detail on how the rigging and skinning of the target avatar may be performed.

[0124] In block 310, the mesh may be converted to a flat panel mesh. The conversion involves using an automated processing pipeline to stitch the animable face and the animable face rig into a head shape mesh. The creation of the animable face mesh and rig may be simplified to be a primarily planar square face surface that can contain multiple views. Using this technique eliminates the need to handle the curvature of any particular head shape during rigging and animation. These planar panel face rigs may be 2.5-dimensional (2.5D) as there is some depth in the components that are off-surface for the mouth pouch and the teeth / tongue area inside, as well as for the eye area and other facial features. Block 320 may follow block 310.

[0125] In block 320, the planar panel mesh can be deformed into a deformed neutral. This operation takes the head shape and the planar face panel rig in a neutral pose and performs UV mapping on the deformed neutral. Thus, in block 320, the first part of the automated processing pipeline deforms the planar face panel in a neutral pose into a head shape and then retargets the rig. Such a deformed neutral can be a generic model of a head that has no rig, no skinning, and no pose. Block 330 may follow block 320.

[0126] In block 330, retargeting may be performed on a deformed neutral to obtain a deformed rig. For example, retargeting may involve performing shape resolution on a deformed neutral rig. Such shape resolution may involve using a Face Action Coding System (FACS) pose via UV mapping, which is a 3D modeling process that projects the surface of a 3D model onto a 2D image for texture mapping. The result of shape resolution may be a deformed pose shape without rigging or skinning. A planar face panel rig may also provide a joint hierarchy and initial skinning weights.

[0127] Joint resolution may follow shape resolution. Joint resolution obtains a deformed pose shape (without rigging or skinning) along with the joint hierarchy and initial skinning weights. Joint resolution manages joint deformations and skinning weights via Smooth Skinning Decomposition with Rigid Bones (SSDR). The deformed neutral and each pose shape serve as ground truth shapes to optimize joint deformations for all poses and vertex skinning weights in order to build a linearly blended skinned rig. The result of joint resolution may be a deformed face rig. Block 340 may follow block 330.

[0128] In block 340, the deformed rig can be stitched to generate a stitched rig. The head shape and deformed face rig are stitched into a stitched head with rigging and skinning. Zero skinning weights may be present in the non-panel areas of the head. Block 350 may follow block 340.

[0129] In block 350, skin diffusion may be performed on the stitched rig. Such skin diffusion may result in a final resulting head, which may be a stitched head with rigging and skinning. The automated processing pipeline may apply some skin diffusion around the stitched edges of the deformed face rig to obtain a smooth reduction of deformation. After block 350, the target avatar may be ready and provided for use in block 260.

[0130] Figure 4 - Workflow for creating a head cage and the resulting animable head variant. Figure 4 shows examples of workflow 400 for creating a head cage and the resulting animable head variant, in several implementation forms. While the workflow in Figure 4 is adapted for use with avatar heads, it may be recognized that some implementation forms use similar techniques for other parts of the avatar. For example, there may be existing template heads 402 and existing template cages 404 that are used as templates for new avatars, particularly for the heads of new avatars.

[0131] Figure 4 shows several examples of user-sculpted target cages 406. For example, the sculpted target cages 406 show several cages corresponding to avatar heads, with the shape of the face and facial features (eyes, mouth, nose, ears, etc.) having varying sizes and shapes. Sculpting can be done using DCC tools or languages. Figure 4 shows that each of the user-sculpted target cages 406 is supplied as input to an automated algorithm 408.

[0132] The automated algorithm 408 generates a corresponding animable head variant 410. Such animable head variant 410 has a shape similar to the target cage 406 sculpted by the user. The animable head variant 410 has more detail by identifying relevant information from the existing template head 402 and incorporating that information into the animable head variant 410.

[0133] As a related example, Figure 4 also shows sequences that help illustrate how several implementations work. For example, Figure 4 shows that there may be a template 420, a user input 422, and a cage morph output 424. The template 420 may include, for example, a template head (corresponding to an existing template head) and a template cage 432 (corresponding to an existing template cage 404). The user input 422 may include a target cage 434. The cage morph output 424 may include a target head 436.

[0134] Specifically, an animable existing template head 402 may be graphically represented by a mesh, and a low-resolution existing template cage 404 may be created / sculpted for the existing template head 402, or otherwise provided, such as by wrapping it over the mesh of the existing template head 402.

[0135] An implementation of workflow 400 may include at least two steps. In the first step, the user may sculpt a first existing template cage 404 into one or more target cages 406. In this first step, the user may sculpt (e.g., create) these target cages 406 procedurally via scripting or by other means using any suitable graphical tool to define a new rough shape of the part (e.g., one of the low-resolution target cages 406 for the existing template cage 402). This first step may require significantly less technical expertise from the user / creator and may therefore be user-friendly and easy to perform.

[0136] In the second step, one or more automated algorithms 408 adapt the joints and skinning and morph the original geometry of an existing template head 402 to a morphed target to result in one or more head variants 410. The second step results in an animable head having parts that move correctly when animated (e.g., mouth, lips, eyes, etc.).

[0137] For example, in the second step, the automated algorithm 408 morphs the underlying geometry of the existing template head 402 and transfers the facial expressions of the existing template head 402. The automated algorithm 408 may further adapt joints and skinning to create a target face rig optimized for runtime performance on mobile devices.

[0138] Automation using the automated algorithm 408 in this way can reduce the time required to create character rigs and poses from about a month to a few seconds. In Figure 4, workflow 400 morphs the existing template head 402 through sculpting the existing template cage 404, which generates a visually changing head identity (e.g., head variant 410) that can be automatically posed to match the facial expression.

[0139] Further details are provided regarding various implementation forms of the second step, which include an automated algorithm 408 that converts a user-sculpted target cage 406 into an animable head variant 410.

[0140] Once the user has sculpted the target cage 406 into its intended shape (for example, during the first step described above), the following actions of the automated algorithm 408 may be performed on the target cage 406 to create an animated avatar variant.

[0141] The automated algorithm 408 may include a cage morphing technique. This technique morphs an underlying avatar part (e.g., an existing template head 402) so that the rough shape of the avatar part matches the shape changes provided by the target cage 406. Multiple methods are possible. For example, possible methods include spatial deformation, wrap deformation, or surface deformation using radial basis functions (RBFs). Depending on the various implementations, the automated algorithm 408 uses surface deformation to perform cage morphing. In some implementations, three aspects of cage morphing may be involved, such as variational optimization, implicit surface tracking, and Laplacian fitting.

[0142] Variational optimization is a technique that imposes physical constraints to ensure reasonable results. Implicit surface tracking is a technique that extends implicit skinning techniques to surface-based deformations. Laplacian fitting is a variation of the Laplacian surface editing technique used to reproduce regular geometric primitive properties. These various techniques and additional details about their use in cage morphing are presented below. The result of this cage morphing step is to generate a new target head for pose / expression transfer, which is the next step in the process, and this next step uses this mesh (from the cage morphing step) to fit a dynamic head rig.

[0143] The process begins with one of several predefined templates, which include a template head (the mesh to be rendered), a template cage (a low-resolution approximation of the template head), skinning (joints and weights for the template head), and a Face Action Coding System (FACS) shape (animated poses of joints representing different expressions / phonemes). The result of the entire process is to adapt this known good template into a fully functional, dynamic head rig that matches user-specified modifications.

[0144] The way users specify the changes they want to make to the template head is by providing a modified version of the template cage called a target cage. In this way, users do not need to directly modify the much more complex template head or its skinning. Users also do not need to update the associated joint poses for the FACS shape.

[0145] Similarly, a known, high-quality dynamic head rig (template head rig) exists to start with. The implementation can be constrained to the extent that changes to the template head rig are feasible. Such constraints can help ensure that the result is a working dynamic head.

[0146] Conceptually, the result of the cage morphing step is to apply a deformation to the template head mesh that is similar to the deformation resulting from moving the template cage to the target cage. One way to view this problem is as a so-called spatial deformation. From this perspective, the problem is modeled by assuming some function that deforms space so that points located at the vertices of the template cage are transformed into points located at the corresponding vertices of the target cage. This deformation function also smoothly deforms the space between each of these points.

[0147] In this way, the deformation function deforms points located in the template head to new points that define the target head shape. Many methods exist for defining such spatial deformation functions. A direct approach to such spatial deformation functions is through radial basis function (RBF) interpolation.

[0148] Another common method for performing spatial deformations is through generalized centroid coordinates (most commonly mean and harmonic coordinates). While not exactly the same as globally supported spatial deformations, this method shares many similarities. The difficulty with these methods lies in the requirements they impose on the shape defining the spatial domain (e.g., requiring a closed mesh that is typically convex). These methods also involve computational and numerical challenges.

[0149] Another approach to address these issues is to "bind" the vertices of the template head to the surface of the template cage. When the template cage is deformed into the target cage (by simple linear shape interpolation), the vertices of the template mesh maintain their relative offset from the cage surface as the deformation progresses. This technique can be very effective, and variations of wrap deformation can be used as part of surface-based deformations used in implementations.

[0150] The first aspect of wrap deformation is that the template head must have suitable surface points for binding onto the template cage. In some implementations, the cage is not a closed surface. The cage contains holes around the eyes, mouth, and neck. The template head is a closed mesh and contains geometry in those areas. The only surface points on the template cage to which such areas can be bound are at the geometric boundaries of the holes. This situation results in less-than-ideal binding for several reasons, including sensitivity to non-normal offset vectors and twists in the local coordinate frame. One solution is to fill the holes in the cage, but this leads to a second limitation.

[0151] A second such limitation is that the normal depth of the offset from the template head to the template cage becomes problematic in some areas of the shape. The closer the template head vertices are to the surface of the template cage, the more meaningful the binding to the surface becomes. As the position of the template head vertices increases in normal distance from the template cage, it becomes unclear which point on the surface of the template cage they should be bound to. Also, deformation of the template cage surface is less relevant to the position of the bound template head vertices. For example, it may be problematic to determine which point on the surface of the template cage the vertices at the back of the mouth should be bound to.

[0152] Another problem encountered when using wrap deformation without modification is that such techniques involve binding the template mesh to any target cage shape provided by the user, even if that shape is impractical or unsuitable for the rig transfer stage. Implementations benefit from being able to limit the degree of deformation to the template mesh, which may allow for finding solutions in the rig transfer operation. In standard wrap deformation, there are no inherent elements that can provide this deformation limiting function. Generally, the techniques presented herein are used in situations where the template mesh and target mesh have similar resolutions.

[0153] Surface-based deformation, also known as "shape-recognition" deformation, is a deformation that takes into account the inherent differential geometric properties of a mesh to generate deformations that minimize local distortion of the shape. Several implementations utilize such deformations. This technique allows implementations to achieve results that are constrained to a feasible range of shapes that are physically reasonable and can be resolved in the pose / expression transfer process.

[0154] A preliminary step in surface-based deformation begins with variations of wrap deformation. Instead of averaging the local coordinate frame bound to the template cage, some implementations instead bind directly to a frame obtained from the nearest triangle in the loop subdivision of the template cage.

[0155] To ensure that this subdivided surface does not contain geometric artifacts such as folds, some implementations optimize the cage topology by first performing a greedy flip optimization that maximizes the minimum angle of triangles and preserves feature edges. A similar process is performed on the target cage, and the bindings to the template cage subdivide are transferred to the target cage subdivide. Evaluating those bindings in the target cage subdivide results in a lapped deformed version of the template head.

[0156] From the wrapped template head, some implementations select a sparse subset of deltas (displacement vectors between corresponding vertex positions) based on specific criteria aimed at evenly distributing the selection and highlighting parts of the mesh that characterize the shape's features. For example, delta selection may be based on cage points, curvature points, and UV boundary points. In this way, some implementations use the wrap deformation as a guide for surface-based deformation. Some implementations take steps to match the deformation defined by the wrap deformation, but only to the extent that the implementation can maintain the intrinsic geometric properties of the original template head shape.

[0157] Based on the techniques discussed above, the delta selection does not include any vertices that are considered to have low-quality binding in the wrap deformer. These vertices are those that coincide with holes in the template cage or vertices that have a normal offset that is too large to be considered reliable (e.g., inside the mouth cavity or orbit).

[0158] Euler-Lagrange equations that minimize thin-shell energy -k s Δd+k b Δ 2 d=0 (Equation 1) is the linear system A=-k s L+k b (LM -1 It is expressed by L)(Equation 2). Adding a weighted delta constraint, which is a soft constraint on the partial differential equation (PDE),

number

number

[0159] In these equations, the stretching stiffness coefficient and the bending stiffness coefficient are respectively represented by k s and k b The Laplacian is represented by Δ, and the bi-Laplacian is represented by Δ 2 (L and LM in Equation 2 -1 L). Thus, the implementation form minimizes stretching / bending with respect to displacement (the difference vector between two positions) rather than minimizing their energy at the point positions. The former is related to "smoothing" the deformation, and the latter is related to "smoothing" the surface itself (thus managing details). The solved displacement is represented by d, and the right side with a value of 0 enforces the minimization of Equation 1. The system of equations is solved in the sense of the least squares method according to the weighted Dirichlet boundary conditions provided by the delta constraints (Equation 4).

[0160] As described above, none of the constraints include deltas from regions such as inside the mouth. The solution of Equation 4 may include the displacements of these regions that match the template head surface characteristics given the provided constraints.

[0161] Applying the solved displacement field to the template head generates a target head mesh, such that the template mesh with the solved displacement is used to generate the target head mesh. The target head can then have several properties. First, the target head is deformed to match the exemplary deformation provided by the mapping from template cage to target cage. Second, the target head minimizes deformation energy so that the target head is locally similar to the template head everywhere (i.e., the target head retains the characteristics of the template head). Third, the target head does not contain large deformations that may exist in the example target cage. Fourth, the target head represents a shape from which a solution can be found that fits template skinning / animation.

[0162] At this point, if the intent of the deformation described by the target cage is to define the reshaping of the template mesh while preserving the surface features present in that mesh, the process is complete. However, if the user intends for the target cage to define the modeling of new surface features that do not exist in the template mesh, the user may feel that the current result is too restrictive. By design, the target mesh at this point is unlikely to have maintained its local offset from the target cage as the target mesh did using standard wrap deformation. Therefore, it is generally impossible to model new features using the target cage. Further deformation steps may be required to capture some of these new modeled features.

[0163] To recover some of the target cage modeling features lost in surface-based deformation, the technique can extend implicit skinning techniques by tracking a template head mesh embedded within an implicit surface function defined by the template cage.

[0164] An implicit surface is a shape defined by a scalar field. An implicit surface exists at all points where the function (equivalent) has a constant value (most commonly zero). This set is called the zero-level set of implicit functions. All other points in space have a non-zero equivalent. The sign of the equivalent indicates whether such a point with a non-zero equivalent lies inside or outside the surface. Often, mesh reconstruction techniques such as marching cubes or dual contouring are used to generate a mesh representing the zero-level set.

[0165] This implementation does not use this hidden surface in isolation. Rather, the implementation focuses on where the vertices of the template head are located relative to this set of levels. By recording the equivalents of the template head vertices in the hidden surface field of the template cage, the implementation effectively embeds the template head within this field. The implementation can similarly generate a hidden surface field from the target cage and use this target hidden surface to project the vertices of the target head back toward these embedded equivalents.

[0166] The implicit surface has many useful properties, including modeling self-contact, defining a continuous gradient at all points in space (not just on the surface of the implicit surface), and defining a smooth interface. However, there is no known direct way for points embedded within the implicit surface function to be deformed in response to deformations of the implicit field.

[0167] Other methods may be used to help points move along with the field. In certain techniques that perform implicit skinning, linear blend skinning (LBS) is used to provide this assistance or "tracking." The tracked points are then projected back along the gradient of the implicit field to their embedding equivalents. Several implementations employ variations of this technique. However, these implementations use the surface-based modifications described herein to provide this tracking.

[0168] By using surface-based tracking methods, the implementation does not simply replace one predeformer with another. Instead, the implementation acquires additional deformation properties that the implementation would not otherwise possess by using relatively arbitrary deformation techniques such as linear blend skinning (LBS).

[0169] Surface-based deformation minimizes deformation energy, especially in elongation. Hidden surface projection is constrained in the gradient direction. Therefore, the implementation yields tangential skin-slip deformation, which is desirable but difficult to model with techniques such as LBS. In short, surface-based tracking of a mesh provides a very high-quality and effective starting point for initiating projection toward the embedding equivalent.

[0170] Additionally, the delta constraint used in surface-based deformation is directly related to the deformation of the template cage to the target cage, and the implicit surface function is directly related to both the template and target cages. Therefore, the pre-deformation of the tracking is directly correlated with the deformation of the implicit surface. This method provides far less arbitrary source of tracking within the field than alternative methods.

[0171] The implementation involves obtaining an implicit scalar field defining the implicit surface by sampling the template and target cage mesh (both position and normal), and then applying Hermitian radial basis functions to these samples such that the value of the implicit scalar field is zero at the sample points (i.e., on the surface) and its gradient aligns with the surface normal at these samples according to Equation 5.

number

[0172] The final projection onto the embedded equivalent is:

number

[0173] A natural extension to this method is to combine the original tracking technique of LBS tracking with the surface-based methods used by implementations. Instead of obtaining delta constraints for thin-shell optimization (a type of radial basis function optimization) from a wrap-like deformer, some implementations may instead obtain delta constraints from LBS deformations from the skeleton that deforms the body. This approach offers the advantages of skin-slip implementations obtained from surface-based methods, in the context of more appropriate skeletal skinning deformations often used to deform character bodies.

[0174] Therefore, implicit surface tracking may include generating a first implicit surface based on a template cage, adding embedded equivalents of the template avatar's vertices to the first implicit surface, generating a second implicit surface based on a target cage, and projecting the target avatar's vertices toward the corresponding equivalents in the second implicit surface based on the first implicit surface and the embedded equivalents. In other words, the vertices are aligned so that their equivalents match in both the source implicit function and the target implicit function. Some implementations may use implicit functions from which values ​​can be taken anywhere in the 3D field, rather than simply contouring uniform equivalents that generate a set of levels.

[0175] Another condition arising from surface-based thin-shell optimization (a type of radial basis function optimization) is that minimizing bending energy can sometimes lead to the implementation form producing a surface that is too smooth. This situation can occur when the intended surface includes shapes similar to regular geometric primitives such as cylinders, spheres, and planes. By minimizing bending energy, the surface may overshoot the intended shape and even become wavy. For target shapes that are generally more organic in nature, these deformations fit well and are actually desirable. However, when a regular geometric shape is the intended result, the implementation form must provide additional post-processing deformations.

[0176] To achieve this, implementations include variations of the well-known Laplacian surface editing technique called Laplacian fitting. This technique involves solving a Poisson problem in which the implementation reconstructs the surface according to a modified Laplacian designed to reproduce the regular geometric shape of the target cage. For example, a distorted flat disk may exist which can be corrected by Laplacian fitting. Such corrections can be seen in both isometric and cross-sectional views. Some implementations may modify the Laplacian used to reflect surface features to be reproduced (such as folds and flat areas) that would otherwise be smoothed by optimization.

[0177] The implementation form is Δx i =δ i By constraining the so-called delta coordinates in (Equation 7) to values ​​that reflect an appropriate shape, and solving the coordinate function (x in Equation 7) that satisfies the Poisson problem, we force appropriate geometric properties using Laplacian fitting.

[0178] This method involves the implementation having several variants of the target head that exhibit these appropriate geometric properties. However, if the implementation has such a version of the target head, the implementation does not resolve that version of the target head in the first place. However, the implementation has something close to this version of the target head in the form of a binding already created for the wrap deformer.

[0179] The implementation previously evaluated these bindings to deform a template mesh into a target mesh and established delta constraints for thin-shell optimization (a type of radial basis function optimization). This evaluation preserved the relative offset of the surface from the bound cage refinement surface. However, if the implementation evaluates these bindings without preserving the offset, the implementation effectively shrink-wraps the head to the cage refinement surface with the local curvature gradient that the implementation is looking for.

[0180] Solving Equation 7 using these values ​​only reconstructs the shrink-wrapped surface, and therefore this is insufficient. Instead, the implementation form is:

number

number

[0181] Finally, the implementation is obtained by finding x that satisfies these constraints in Equation 10, and the coordinate function

number

[0182] Post-processing can be applied by either hidden surface tracking or Laplacian fitting, depending on the user's intent.

[0183] A template head mesh may contain multiple connected components. In addition to the skin, there may be any number of components representing head features such as eyeballs, teeth, and tongue. These components are often treated differently from the skin (or differently from each other). These components follow the deformations applied to the skin to fit the new shape, but the implementation cannot distort their methods in an unnatural way.

[0184] For example, there may be eyeball components in the template head. If the area around the orbit is enlarged and shifted laterally in the target head, the eyeball components should be enlarged and moved accordingly. Otherwise, the eyeball components will no longer fit within the orbit.

[0185] However, if the implementation simply deforms the eyeball component in the same way as the surrounding skin mesh, the eyeball component will no longer remain spherical (assuming the eyeball component was spherical to begin with and not a stylized shape). Such a method will not work for animations where the eyeball rotates around its center when the avatar turns in a different direction.

[0186] With teeth (often modeled as a single unit for upper teeth and a second unit for lower teeth), a slightly different situation arises. Issues with teeth may not be relevant to animation. Instead, issues with modeling teeth may be more related to expectations about how reasonably hard materials like teeth can be reshaped.

[0187] The implementation handles the problem of reshaping these additional components so that it can first determine how the components will deform to conform to the skin shape. Then, the implementation adapts a rigid affine transformation to the deformed shape and applies the rigid affine transformation instead of the non-rigid deformation (thus "de-deforming" the components).

[0188] Rigid body transformation fitting can be calculated using the Procrustes method, which finds the optimal rotation through singular value decomposition (SVD) of the weighted cross-covariance matrix of vertex positions (static and deformed shapes). Depending on the specifications of the components, the size can be adjusted as either non-uniform scaling for semi-rigid deformations (e.g., teeth) or uniform scaling for fully rigid deformations (e.g., spherical eyeballs).

[0189] Contact with the skin surface is maintained by applying RBF-based spatial deformation, defined by the non-deformable delta of the (semi)rigid component, to the local contact interface region of the skin that maintains contact.

[0190] Figure 5 - Exemplary planar panel mesh and rig Figure 5 shows examples of planar panel meshes and rigs 500 in several implementation forms. Figure 5 shows the use of an automated processing pipeline to stitch an animable face and its rig into a head shape mesh. The creation of the animable face mesh and rig can be simplified to mainly a flat, square face surface, as shown in Figure 5.

[0191] Figure 5 shows a front view 510, a side view 520, a rig joint 530, and a Face Action Coding System (FACS) pose 540, which, when blended, create a facial expression. Six FACS poses 540 are shown in Figure 5, but these are merely examples, and around 85 to 120 poses (again, as an example) can be defined for use in the rig targeting solver.

[0192] Using the technique shown in Figure 5, it is not necessary to handle the curvature of any particular head shape during rigging and animation. These planar panel face rigs in Figure 5 can be 2.5-dimensional (2.5D) because there is some depth in the off-surface components for the mouth pouch and the teeth / tongue area inside, as well as for the eye area and other facial features. 2.5D perspective refers to gameplay or motion in video games or virtual reality environments that are simulated and rendered in a 3D digital environment, often limited to a two-dimensional plane with little or no access to three dimensions in a space that otherwise appears three-dimensional.

[0193] The animations used for the eyes, eyebrows, and nose of a face can be 2D, primarily to simplify the rigging, skinning, and animation processes. By design, the main surface of a planar panel face mesh and rig is square so that the x and y coordinates of the vertices precisely match the corresponding uv coordinates in UV space.

[0194] With the planar panel face mesh and rig created, the next step is an automated method for stitching the planar panel face rig onto an existing head shape. The automated processing pipeline stitches the planar panel face mesh and rig onto a head shape mesh defined by UV mapping, which is similar to how actual textures are applied to a mesh. Given the planar panel face mesh and target face shape, the automated pipeline generates a fully rigged, animable, stitched head.

[0195] Figure 6 - Automated processing pipeline for deforming the geometry of a planar face panel Figure 6 shows examples of automated processing pipelines for deforming the geometry of a planar face panel 600 in several implementation configurations. For example, the planar face panel 610 is retargeted to the head shape 620 using acquired UV mapping. The retargeting results in a fully rigged, animated, stitched head with facial expressions 630.

[0196] Some technical challenges related to this processing pipeline and technique may include: Firstly, the planar panel is deformed into an arbitrary shape surface of the head shape mesh. If the planar panel is very thin, this is a simple mapping via UV coordinates and signed normal distance from the face surface.

[0197] However, the planar panel is a 2.5D planar panel and may have an inner mouth pouch with areas for teeth and tongue. The challenge lies in determining how these areas deform when the face surface is stitched onto a curved surface. To solve this problem, the planar face panel rig is retargeted so that the resulting facial expression looks good (i.e., matches the user's intent) on the stitched head. Since the implementation should be designed so that the planar face panel can be stitched onto any arbitrary head shape, the implementation can use a general methodology that can take into account a wide range of curvature differences and anisotropic scaling. For example, the implementation can be designed to work with a wide variety of head shapes and poses.

[0198] Figure 7 - Further details of the automated processing pipeline. Figure 7 shows further details of the automated processing pipeline 700 in several implementation forms. In Figure 7, the first part of the automated processing pipeline deforms the planar face panel rig 720 in a neutral pose into a head shape 710, and then retargets the rig. For example, the automated processing pipeline 700 starts with the head shape 710 and the planar face panel rig 720. The head shape 710 and the planar face panel rig 720 are provided to a deformed neutral process 730 using UV mapping. The result of the deformed neutral process 730 is a deformed neutral 740. The deformed neutral 740 has no rig, skinning, or pose.

[0199] The deformed neutral 740 can be found as follows: Given a head shape mesh 710 and a planar face panel mesh and rig 712, this step involves finding the vertices of the planar face panel mesh and rig 712 in a neutral pose.

number

number

[0200] Therefore, the implementation involves a spatial deformation function f that maps the 3D of the planar face panel 712 in world coordinates to a 3D point in the deformed space 740. deform :D→R 3 It is designed to find the implementation form f for all vertices i. deform :(b (i) )=c (i) It should be constrained to have f. deformThere are many different options that can satisfy the above constraints, including radial basis functions, mean coordinates, harmonic coordinates, and Green's coordinates.

[0201] However, for the problems discussed herein, the implementation can find this mapping using a UV map and a signed distance from the main face surface. The planar face panel is in xy coordinate (b x ,b y ) is directly mapped to UV coordinates and constructed such that the z coordinate is equal to zero for points on the main face surface. Specifically, given UV coordinates u,v ∈ [0,1], c surface =g map (u,v) is the 3D coordinate c on the surface of the head shape. surface =[c x ,c y ,c z Let this be a function that returns ]T.

[0202] Also, gradient head Let (u,v) return the normal vector on the head surface with respect to UV coordinates. Function g for u,v∈[0,1] map Please note that the image is a golden surface on head shape 710. Next, the spatial deformation is f deform (b) = g map (b x ,b y )+s·bx·normal head (b x ,b y ) is given by, where s is a scalar coefficient that takes into account the global scale difference between the planar face panel and the face portion of the head shape.

[0203] In other words, for a point s in the input space, the implementation first involves the head surface c having the same UV coordinates. surface Find the point shown above, and along the surface normal of the head, measure the original signed distance b on the planar face panel. z Move it by the scale factor of f. deformUsing this method, the implementation can calculate a neutral 740 that is deformed from a planar face panel neutral.

[0204] The deformed neutral 740 is provided as input to the retargeting process 750. In addition, joint and skinning information 742, including the joint hierarchy and initial skinning weights, is provided to the retargeting process 750. The retargeting process 750 includes performing a shape resolution operation 752 on the deformed neutral 740. In the shape resolution operation 752, the mapping f calculated in the previous section is used to map all vertices in all FACS poses of the planar face panel to the deformed space. deform This can exist. This results in a set of 754 deformed pose shapes, one for each FACS pose.

[0205] Each target pose shape within the deformed neutral 740 and the set of deformed pose shapes 754 serves as a ground truth shape to optimize joint transformations for all pose and vertex skinning weights in order to construct a linear blend skinning rig. The implementation reuses the joint structure of the original planar face panel rig 720 and uses an iterative method to find the corresponding joint transformations and vertex skinning weights.

[0206] The implementation involves first resolving the shape of each FACS pose in the shape resolution step, and then using those shapes as constraints in the joint resolution operation 756 to optimize joint transformation and skinning. This method effectively decouples how a FACS pose looks from the underlying deformer type and parameters that transform the neutral into the pose. Specifically, for a linear blend skinning rig, joint resolution can effectively find the best combination of rotation and translation to transform the neutral into the pose shape.

[0207] The joint resolution operation 756 manages joint transformations and skinning weights via a smooth skinning decomposition (SSDR) using rigid bones. The joint resolution operation may include using a set of deformed neutrals and deformed pose shapes to serve as ground truth shapes for constructing a linear blend skinning rig for the target avatar. Thus, the joint resolution operation 756 generates a deformed face rig 760 in its output. The deformed face rig 760 can serve as input to the continuation of the automated processing pipeline presented in Figure 8.

[0208] Figure 8 - Further details of the automated processing pipeline Figure 8 still shows further details of the automated processing pipeline 800 in several implementation forms. In Figure 8, the second part of the automated processing pipeline performs a stitching operation 830 to stitch the deformed face rig 820 to the head shape 810 in order to generate a stitched head rig 840. The deformed face rig 820 may correspond to the deformed face rig 760 generated by processing the pipeline 700 discussed in Figure 7. The automated processing pipeline applies some skin diffusion 850 around the stitched edges of the deformed face rig in order to obtain a smooth reduction of deformation in the final result 860.

[0209] To complete the pipeline, the pipeline then stitches the target face rig into the UV map region of the head shape to obtain a complete head. To obtain aesthetically pleasing deformations in the head shape, it may be necessary to diffuse the skin weights around and within the boundaries of the deformed face panels. Such diffusion may be particularly relevant to head shapes with jaws. For example, if the skin weights are not properly adjusted, there may be cases where the lower teeth protrude through the jaw.

[0210] Additional details regarding specific aspects of stitching and skin diffusion in various implementations are provided here. To complete the automated processing pipeline, the target face rig is stitched into the UV map region of the head shape to obtain a complete head. Skin weights are also diffused around and within the boundaries of the deformed face panel to obtain a good-looking deformation of the head shape, especially the head shape with a jaw. Otherwise, if the skin weights are not properly adjusted, the lower teeth may pierce the jaw.

[0211] Since skinning weights exist only at the resolved face panel vertices, the automated processing pipeline blends those weights outward beyond the stitching boundary to ensure that deformations at the original face panel boundary propagate naturally to the surrounding head mesh.

[0212] Ideally, diffusion would involve simply smoothing the weights outward (e.g., by the neighborhood or geodetic distance of a certain number of faces) while constraining that the face panels remain unchanged. However, such a simple rule does not work globally. Instead, skin diffusion would have to depend on the head shape. However, this technique is complex because the original weights were designed to deform the face panels in isolation, making it very difficult for an artist to blend them with a single, yet-unstitched, head shape, and probably even more so for an entire range of different shapes. Therefore, automated processing pipelines use a head shape-dependent "blending mask".

[0213] Figure 9 - Shape transfer using existing dynamic heads Figure 9 shows examples of shape transfer using an existing dynamic head 900 in several implementation forms. For example, a template head 910 and a target head 920 may exist. The template head 910 and the target head 920 may have the same topology and vertex correspondence.

[0214] Each of the template head 910 and target head 920 can originally be represented to have a neutral expression. Various mappings exist to map the neutral template to a pose. These mappings provide pose 940, which is a variant of template head 910. Since a correspondence exists between template head 910 and target head 920, it is possible to generate pose 950, which is similar to pose 940, in the context of target head 920.

[0215] Creating rigs, skinning, and poses can be difficult and time-consuming. These techniques can leverage combinatorial theory of facial components. The problem faced is how, given a template head 910 and a target head 920, the implementation can transfer the rig and acquire facial expressions on the target. Existing dynamic heads can be used to solve this problem.

[0216] The implementation begins with a template natural and a target natural. The template natural head is associated with various poses. Using this information, the target natural head can be used as a basis for generating a target head pose that is similar to the pose of the template natural head. Such target head poses can be identified based on the same topology and vertex correspondence, as will be described in more detail herein.

[0217] Figure 10 - Two-step method including shape transfer and linear blend skinning (LBS) rig resolution Figure 10 shows examples of a two-step method involving shape transfer and linear blend skinning (LBS) rig resolution 1000 in several implementation forms. For example, Figure 10 shows a face with a neutral pose 1010 and various poses 1012 of that face. The neutral pose 1010 and the various poses 1012 are supplied for a shape transfer operation 1020. The shape transfer operation 1020 results in a transferred shape 1030 and a transferred pose 1032. For example, these may correspond to a template head and a target head.

[0218] A target pose 1040 may also exist. The transferred shape 1030, the transferred pose 1032, and the target pose 1040 are supplied to the LBS rig resolution operation 1050. The LBS rig resolution operation 1050 generates a pose with the transferred rig 1060.

[0219] Shape transfer 1020 may include capturing a mapping that moves template neutral vertices to target neutral vertices. Thus, there may be n template vertices and n corresponding target vertices. Shape transfer 1020 may use the corresponding spatial deformation method.

[0220] The spatial deformation method is y i =f(x i Each template vertex x i Target vertex y i Function f:R that maps to 3 →R 3 Find it. Radial basis functions (RBFs) can be used (w j , c j One such function (parameterized by A and b), where y = f(x) = SUM j w j ||xc j ||+Ax+b. f(x) is not only the template vertex xi, but also R 3Note that this is defined for all x within the given range. Having a correspondence between vertices means that the implementation has a correspondence between triangles between the template face and the target face. For example, there may be gradient transfer techniques for calculating the FACS shape. Techniques may also exist that include neural Jacobian fields or as-rigid-as-possible (ARAP++) methods.

[0221] Linear Blend Skinning (LBS) rig resolution operation 1050 takes the skinning weights from the original (source) template, the joint hierarchy from the original (source) template, and the template / target vertex position pairs for all poses as given. LBS rig resolution operation 1050 may use the optimization results for the transfer of all joints for all poses and for the updated skinning weights.

[0222] Figure 11 - Function for shape transfer via spatial deformation Figure 11 shows an exemplary function that performs shape transfer via spatial deformation 1100 in several implementation forms. For example, there may be a function f1110 (described herein) that deforms space to map a first face 1112 to a target face 1114. In 1120, f is applied to all vertices of the template geometry in all poses to obtain the corresponding target shape.

[0223] For example, template pose 1122 maps to target pose 1124, template pose 1126 maps to target pose 1128, template pose 1130 maps to target pose 1132, template pose 1134 maps to target pose 1136, and template pose 1138 maps to target pose 1140. Such mappings provide poses similar to the template face poses in the context of the target face.

[0224] Figure 12 - Rig transfer technology Figure 12 illustrates a rig transfer technique that automatically transfers the rig, skinning, and pose to the target 1200, given a correspondence between neutral representations, in several implementation forms. For example, Figure 12 shows a template face 1210, a template pose 1212, and a template cage 1214. A correspondence 1216 may exist between the template cage 1214 and the target cage 1218. Such a correspondence provides the generation of the target face 1220 using the target cage 1218 based on the correspondence.

[0225] Figure 12 illustrates an additional embodiment using the correspondence between templates and targets. For example, there is a template face 1230 with a neutral pose. The template face 1230 is associated with several template poses 1232 having various facial expressions. A vertex correspondence 1234 exists between the template face 1230 in the neutral pose and the target face 1236 in the neutral pose. Thus, it is possible to perform the resulting rig transfer 1238 operation on several target poses 1240. The template poses 1232 and target poses 1240 have similar facial expressions, but the poses resemble the template face 1230 and target face 1236, respectively.

[0226] For example, rig transfer starts with a template (as the source) associated with a neutral pose that has been rigged and posed into several poses. Cage morphing or UV mapping provides a version of the template neutral that has been morphed relative to the target neutral. A function f is identified that can precisely map each vertex of the source neutral to the corresponding vertex of the target neutral. Thus, the function f is R 3 →R 3 This is a mapping of f and R. 3If defined throughout, the implementation can transform vertices of any pose in the source to vertices of the target. Some implementations may use radial basis functions (RBFs) with a linear kernel for the function f.

[0227] Joint resolution can be performed such that the RBF function maps each vertex of the template neutral face to the corresponding vertex of the target neutral face. Thus, once a function f is defined, the function can be appropriately applied to generate the target pose 1240 by performing a rig transfer 1238 based on the template face 1230 and the target face 1236, based on the vertex correspondence relationship 1234 and the template pose 1232.

[0228] Figure 13 - Exemplary Computing Device Figure 13 is a block diagram showing exemplary computing device 1300 in several implementation configurations.

[0229] Figure 13 is a block diagram showing an exemplary computing device 1300 that may be used to implement one or more features described herein. In one example, device 1300 may be used to implement a computer device (e.g., 102 and / or 110 in Figure 1) and to perform an implementation of a suitable method described herein. Computing device 1300 can be any suitable computer system, server, or other electronic or hardware device. For example, computing device 1300 may be a mainframe computer, desktop computer, workstation, portable computer, or electronic device (portable device, mobile device, cell phone, smartphone, tablet computer, television, TV set-top box, personal digital assistant (PDA), media player, game device, wearable device, etc.). In some implementations, device 1300 includes a processor 1302, memory 1304, an input / output (I / O) interface 1306, and an audio / video input / output device 1314.

[0230] The processor 1302 may be one or more processors and / or processing circuits that execute program code and control the basic operation of device 1300. “Processor” includes any suitable hardware and / or software system, mechanism, or component that processes data, signals, or other information. A processor may include a system having a general-purpose central processing unit (CPU), multiple processing units, dedicated circuits for achieving a function, or other systems. Processing does not need to be limited to a specific geographical location or have temporal limitations. For example, a processor may perform its functions in “real-time,” “offline,” “batch mode,” etc. Parts of the processing may be performed by different (or the same) processing systems at different times and in different locations. A computer may be any processor that communicates with memory.

[0231] Memory 1304 is typically provided within device 1300 for access by processor 1302 and may be any suitable processor-readable storage medium located separately from and / or integrated with processor 1302, suitable for storing instructions for execution by the processor, such as random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), or flash memory. Memory 1304 can store software running on server device 1300 by processor 1320, including the operating system 1308 and one or more applications 1310, such as an avatar generation application 1312. In some implementations, application 1310 may include instructions that enable processor 1302 to perform (or control) some or all of the functions described herein, such as the methods described with respect to Figures 2 and 3.

[0232] For example, application 1310 may include an avatar generation application 1312 that can generate avatars within an online virtual experience server (e.g., 102) as described herein. The elements of the software in memory 1304 can, alternatively, be stored on any other suitable storage location or computer-readable medium. In addition, memory 1304 (and / or other connected storage devices) can store instructions and data used in the functions described herein. Memory 1304, and any other type of storage (magnetic disks, optical disks, magnetic tapes, or other tangible media) can be considered “storage” or “storage devices.”

[0233] The I / O interface 1306 can provide functionality that enables the server device 1300 to interface with other systems and devices. For example, network communication devices, storage devices (e.g., memory and / or datastore 120), and input / output devices can communicate via interface 1306. In some implementations, the I / O interface can be connected to an interface device that includes input devices (such as keyboards, pointing devices, touchscreens, microphones, cameras, scanners, etc.) and / or output devices (such as display devices, speaker devices, printers, motors, etc.).

[0234] The audio / video input / output device 1314 may include a user input device (e.g., a mouse) that can be used to receive user input, a display device (e.g., a screen, monitor) that can be used to provide graphical and / or visual output, and / or a combined input and display device.

[0235] For ease of explanation, Figure 13 shows one block for each of the following: processor 1302, memory 1304, I / O interface 1306, and software blocks for the operating system 1308 and virtual experience application 1310. These blocks may represent one or more processors or processing circuits, operating systems, memory, I / O interfaces, applications, and / or software engines. In other implementations, device 1300 may not have all of the illustrated components and / or may have other components, including other types of elements, instead of or in addition to those illustrated herein. The online virtual experience server 102 is described herein as performing the operations described in several implementations, but any suitable component or combination of components of the online virtual experience server 102 or a similar system, or any suitable processor associated with such a system, may perform the operations described.

[0236] User devices can also implement and / or be used with the functions described herein. An exemplary user device may be a computer device including several components similar to device 1300, for example, a processor 1302, memory 1304, and / or an I / O interface 1306. An operating system, software, and applications suitable for the client device may be provided in memory and used by the processor. The I / O interface for the client device may be connected to network communication devices, as well as input and output devices, for example, a microphone for capturing sound, a camera for capturing images or video, a mouse for capturing user input, a gesture device for recognizing user gestures, a touchscreen for detecting user input, an audio speaker device for outputting sound, a display device for outputting images or video, or other output devices. The display device within the audio / video input / output device 1314 may be connected to (or included within) device 1300 for displaying pre-processed and post-processed images, as described herein, and such display device may include any suitable display device, e.g., LCD, LED, or plasma display screen, CRT, television, monitor, touchscreen, 3-D display screen, projector, or other visual display device. Some implementations may provide an audio output device, e.g., text-to-speech output or synthesis.

[0237] One or more methods described herein (e.g., methods 200 and / or 300) can be implemented by computer program instructions or code that can be executed on a computer. For example, the code can be implemented by one or more digital processors (e.g., microprocessors or other processing circuits) and can be stored on computer program products including non-temporary computer-readable media (e.g., storage media), such as magnetic storage media, optical storage media, electromagnetic storage media, or semiconductor storage media, including semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random-access memory (RAM), read-only memory (ROM), flash memory, rigid magnetic disks, optical disks, solid-state memory drives, etc. Program instructions can also be included in or provided as electronic signals, for example, in the form of software as a service (SaaS) delivered from a server (e.g., a distributed system and / or a cloud computing system). Alternatively, one or more methods can be implemented in hardware (e.g., logic gates) or in a combination of hardware and software. Exemplary hardware may include programmable processors (e.g., field-programmable gate arrays (FPGAs), composite programmable logic devices), general-purpose processors, graphics processors, and application-specific integrated circuits (ASICs). One or more methods may run as part of or as a component of an application running on a system, or as an application or software running in conjunction with other applications and operating systems.

[0238] One or more methods described herein can be performed as standalone programs that can run on any type of computing device, as programs that run on a web browser, or as mobile applications ("apps") that run on mobile computing devices (e.g., mobile phones, smartphones, tablet computers, wearable devices (watches, armbands, jewelry, hats, goggles, glasses, etc.), laptop computers, etc.). In one example, a client / server architecture can be used, for example, where a mobile computing device (as a client device) sends user input data to a server device and receives final output data for output (e.g., for display) from the server. In another example, all computations can be performed within a mobile app (and / or other app) on a mobile computing device. In yet another example, computations can be divided between a mobile computing device and one or more server devices.

[0239] While this specification has described specific implementations, these are merely illustrative and not restrictive. Concepts presented in the examples may apply to other examples and implementations.

[0240] The functional blocks, operations, functions, methods, devices, and systems described herein may be integrated or divided into different combinations of systems, devices, and functional blocks, as is known to those skilled in the art. Any suitable programming language and programming technique may be used to implement routines in a particular implementation. Various programming techniques, e.g., procedural or object-oriented, may be used. Routines may be executed on a single processing device or on multiple processors. Steps, operations, or calculations may be presented in a particular order, but the order may be modified in different particular implementations. In some implementations, multiple steps or operations shown herein as sequential may be executed simultaneously. [Explanation of symbols]

[0241] 100 System Architectures 102 Online virtual experience servers, online game servers 104 Virtual Experience Engine 106 Virtual Experiences 108 Graphics Engines 110, 110a, 110b, 110n client devices 112 Virtual Experience Applications 114 Input / Output (I / O) Interfaces 120 datastores 122 Network 130, 130a, 130n Developer Devices 132 Virtual experience applications, game applications 134 Input / Output (I / O) Interfaces 400 Workflows 402 Existing template head, Animated existing template head 404 Existing template cage, low-resolution existing template cage 406 User-sculpted target cages, target cages 408 Automated algorithm 410 Animatable head variants, head variants 420 Template 422 User input 424 Cage morph output 430 Template head 432 Template cage 434 Target cage 436 Target head 500 Planar panel mesh and rig 510 Front view 520 Side view 530 Rig joint 540 Facial Action Coding System (FACS) pose, FACS pose 600 Planar face panel 610 Planar face panel 620 Head shape 630 Expression 700 Automatic processing pipeline, pipeline 710 Head shape 712 Planar face panel mesh and rig 720 Planar face panel rig 730 Deformation neutral process 740 Deformed neutral, deformation neutral 742 Joint and skinning information 750 Retargeting process 752 Shape resolution operation 754 Set of deformed pose shapes 756 Joint resolution operation 760 Deformed face rig 800 Automatic processing pipeline 810 Head shape 820 Deformed face rig 830 Stitch operation 840 Stitched head rig 850 Skin diffusion 860 Final result 900 Dynamic head 910 Template Head 920 Target Head 940 Pose 950 Pose 1000 Shape Transfer and Linear Blend Skinning (LBS) Rig Solving 1010 Neutral Pose 1012 Pose 1020 Shape Transfer 1030 Transferred Shape 1032 Transferred Pose 1040 Target Pose 1050 LBS Rig Solving Operation, Linear Blend Skinning (LBS) Rig Solving Operation 1060 Transferred Rig 1100 Spatial Deformation 1110 Function f 1112 First Face 1114 Target Face 1122 Template Pose 1124 Target Pose 1126 Template Pose 1128 Target Pose 1130 Template Pose 1132 Target Pose 1134 Template Pose 1136 Target Pose 1138 Template Pose 1140 Target Pose 1200 Target 1210 Template Face 1212 Template Pose 1214 Template Cage 1216 Correspondence 1218 Target Cage 1230 Template Face 1232 Template Pose 1234 Vertex Correspondence 1236 Target Face 1238 Rig Transfer 1240 Target Pose 1300 Computing devices, devices, server devices 1302 Processors 1304 memory 1306 Input / Output (I / O) Interface, I / O Interface, Interface 1308 Operating Systems 1310 Applications, Virtual Experience Applications 1312 Avatar Generation Application 1314 Audio / Video Input / Output Devices

Claims

1. A computer implementation method for creating a variant of a template avatar, wherein the method is A step of obtaining the template avatar, which includes the template geometry obtained from the mesh of the template avatar, The steps include generating a template cage associated with the template avatar as a low-resolution approximation that wraps around the template geometry, The steps include creating a target cage from the template cage by modifying the template cage based on user input, To generate a target avatar which is a variant of the template avatar, the steps include morphing the template geometry with the target cage and A computer implementation method comprising the above.

2. The computer implementation method according to claim 1, further comprising the step of adjusting the rigging and skinning of the target avatar in order to enable the animation of the target avatar.

3. The template avatar further includes a template head of the template avatar, and the target avatar comprises a target head of the target avatar, and the step of adjusting the rigging and skinning is, The steps of determining the pose of the target head based on a specific pose of the template head, and The step of determining the facial expression of the target head based on a specific facial expression of the template head. The computer implementation method according to claim 2, comprising one or more of the above.

4. The step of adjusting the rigging and skinning of the target avatar is The steps include converting the mesh of the template avatar into a planar panel mesh, The steps include: deforming the planar panel mesh to a deformed neutral state based on the neutral pose of the target avatar; The steps include: performing retargeting on the modified neutral in order to obtain the modified rig of the template avatar; To generate a stitched rig having rigging and skinning, the steps include stitching the deformed rig onto the shape of the target avatar, After stitching, the steps include performing skin diffusion on the stitched rig in order to obtain the target avatar. The computer implementation method according to claim 2, comprising:

5. The aforementioned computer implementation method The steps include defining a spatial deformation function that maps points in the 3D world coordinates of the planar panel mesh to 3D points in the deformed neutral, The steps of using the spatial deformation function to deform the planar panel mesh into the deformed neutral shape and The computer implementation method according to claim 4, further comprising:

6. The step of associating the template avatar with multiple poses encoded via a facial action coding system (FACS) and performing retargeting is as follows: To generate a set of deformed pose shapes, the steps include: performing a shape resolution operation using the spatial deformation function to map each of the multiple poses; The steps include performing a joint resolution operation, which involves using a set of deformed neutral and deformed pose shapes that serve as ground truth shapes to construct a linearly blended skinned rig for the target avatar. The computer implementation method according to claim 5, comprising:

7. The computer implementation method according to claim 1, wherein the step of morphing the template geometry of the template avatar with the target cage in order to generate the target avatar comprises the step of using at least one surface-based deformation technique.

8. The computer implementation method according to claim 7, wherein the step of using at least one surface-based deformation technique comprises the steps of performing a wrap deformation to provide a wrap deformation version of the template avatar, and selecting a sparse subset of deltas based on the wrap deformation version of the template avatar.

9. The computer implementation method according to claim 7, wherein the at least one surface-based deformation technique comprises variational optimization.

10. The computer implementation method according to claim 9, wherein the variational optimization includes radial basis function optimization for finding a displacement field, and the displacement field is applied to the template avatar to generate the target avatar.

11. The computer implementation method according to claim 10, further comprising the step of performing at least one of hidden surface tracking or Laplacian fitting.

12. The step of performing the aforementioned shadow surface tracing is, A step of generating a first hidden surface based on the template cage, The steps include adding the embedding equivalents of the vertices of the template avatar to the first hidden surface, A step of generating a second hidden surface based on the target cage, The steps of projecting the vertices of the target avatar toward the corresponding equivalents in the second shadow surface based on the first shadow surface and the embedded equivalents of the first shadow surface. The computer implementation method according to claim 11, comprising:

13. The computer implementation method according to claim 11, wherein the Laplacian fitting comprises solving a Poisson problem to reconstruct the target avatar based on a modified Laplacian designed to reproduce the regular geometric shape of the target cage by creating a surface for the target avatar that satisfies delta-fit constraints.

14. In response to execution by the processing device, the processing device is instructed to: Obtaining the template avatar, which includes the template geometry obtained from the mesh of the template avatar, To generate a template cage associated with the template avatar as a low-resolution approximation that wraps around the template geometry, By modifying the template cage based on user input, a target cage is created from the template cage, To generate a target avatar which is a variant of the template avatar, the template geometry is morphed with the target cage. A non-temporary, computer-readable medium containing instructions that cause an action to be performed.

15. The non-temporary computer-readable medium according to claim 14, further comprising adjusting the rigging and skinning of the target avatar to enable the animation of the target avatar.

16. Adjusting the rigging and skinning of the target avatar, Converting the mesh of the template avatar into a planar panel mesh, Based on the neutral pose of the target avatar, the planar panel mesh is deformed to a deformed neutral state. In order to obtain the deformed rig of the template avatar, retargeting is performed on the deformed neutral, To generate a stitched rig with rigging and skinning, the deformed rig is stitched onto the shape of the target avatar, After stitching, skin diffusion is performed on the stitched rig in order to obtain the target avatar. A non-temporary computer-readable medium according to claim 15, comprising:

17. A non-temporary computer-readable medium according to claim 14, wherein generating the target avatar involves morphing the template geometry of the template avatar with the target cage using at least one surface-based deformation technique.

18. The memory where the instructions are stored, A processing device coupled to the memory, configured to access the memory and execute the instructions, and A system comprising the above, wherein the instruction is directed to the processing device, Obtaining the template avatar, which includes the template geometry obtained from the mesh of the template avatar, To generate a template cage associated with the template avatar as a low-resolution approximation that wraps around the template geometry, By modifying the template cage based on user input, a target cage is created from the template cage, To generate a target avatar which is a variant of the template avatar, the template geometry is morphed with the target cage. A system that performs actions including those mentioned above.

19. The system according to claim 18, further comprising adjusting the rigging and skinning of the target avatar to enable the animation of the target avatar.

20. Adjusting the rigging and skinning of the target avatar, Converting the mesh of the template avatar into a planar panel mesh, Based on the neutral pose of the target avatar, the planar panel mesh is deformed to a deformed neutral state. In order to obtain the deformed rig of the template avatar, retargeting is performed on the deformed neutral, To generate a stitched rig with rigging and skinning, the deformed rig is stitched onto the shape of the target avatar, After stitching, skin diffusion is performed on the stitched rig in order to obtain the target avatar. The system according to claim 19, comprising:

Images