Inferred Skeleton Structures for Practical 3D Assets
By inferring skeleton structures between joints in a graph representation and combining them with avatar data models, the method addresses the challenge of creating practical 3D assets with accurate animation, improving user experience and reducing computational load in online virtual experience platforms.
Patent Information
- Application Number
- JP2024521295
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2022-09-08
- Filing Date
- 2022-10-13
- Publication Date
- 2025-04-01
- Estimated Expiration
- 2042-10-13
AI Technical Summary
Existing online virtual experience platforms face challenges in creating practical 3D assets with accurate skeleton structures, especially when combining body parts from different parent models, which can result in inconsistent seams and animated joints, leading to poor animation and user experience.
The method involves receiving an avatar data model with body parts from different parent models, creating a graph representation to determine joint locations, inferring a skeleton structure between joints, and combining these elements to create a practical 3D asset with a defined skeleton structure suitable for animation.
This approach enables the creation of practical 3D assets with correctly aligned joints and skeleton structures, improving animation quality, reducing computational load, and enhancing user experience across various virtual environments.
Smart Images

Figure 0007654905000001 
Figure 0007654905000002 
Figure 0007654905000003
Abstract
Description
[Technical Field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is an international application claiming priority under 35 U.S.C. §119(e) to U.S. Provisional Patent Application No. 63 / 255,489, filed October 14, 2021, and U.S. Provisional Patent Application No. 63 / 404,871, filed September 8, 2022, both of which are incorporated herein by reference in their entireties.
[0002] FIELD Embodiments relate generally to online virtual experience platforms, and more particularly to methods, systems, and computer-readable media for creating inferred skeletal structures for practical 3D assets. [Background technology]
[0003] Online platforms, such as virtual experience platforms and online gaming platforms, may include avatars and avatar editing suites for use by users and developers.
[0004] The background art discussion provided herein is intended to provide a context for the present disclosure. The work of the presently named inventors, and aspects of the discussion that may not qualify as prior art at the time of filing, to the extent described in this background art section, are not admitted expressly or implicitly as prior art to the present disclosure. Summary of the Invention [Means for solving the problem]
[0005] Aspects of the present disclosure are directed to methods, systems, and computer-readable media for creating inferred skeletal structures for practical 3D assets.
[0006] In one aspect, a computer-implemented method is described that includes receiving an avatar data model for a three-dimensional (3D) avatar, the avatar data model including at least a first body part from a first parent model and a second body part from a second parent model, the first parent model and the second parent model being different 3D models; creating a graph representation for the avatar data model, the graph representation starting at a selected root point for the avatar data model and extending to an end portion of the avatar data model; and traversing the graph representation to determine a plurality of joint locations, the joint locations being locations where joints will be located. The method may include steps of: representing each of the joint locations, and determining joint locations based on intersections of different body parts, descriptions of different body parts, or connections between different body parts; inferring a skeletal structure between the plurality of joint locations such that, for every joint, the skeleton portion spans between a first joint and a second joint and the skeleton portion rotates with movement of one or both of the first joint and the second joint; and combining the joints at each of the plurality of joint locations, the inferred skeletal structure, and the avatar data model to create a practical 3D asset.
[0007] Various embodiments of computer-implemented methods are described.
[0008] In some embodiments, the computer-implemented method further includes animating the practical 3D asset within the virtual environment of the virtual experience.
[0009] In some embodiments, animating the practical 3D asset includes utilizing the inferred skeletal structure for skin deformation during animation of the practical 3D asset.
[0010] In some embodiments, the graphical representation includes multiple nodes representing individual body parts of the avatar data model.
[0011] In some embodiments, the graph representation further includes a plurality of edges, each edge of the plurality of edges spanning between at least two nodes, and each edge of the plurality of edges representing a joint location.
[0012] In some embodiments, each joint combined in a plurality of joint locations comprises either an animated joint or a fixed joint.
[0013] In some embodiments, the computer-implemented method further includes skinning the practical 3D asset based at least on the inferred skeletal structure.
[0014] In some embodiments, traversing the graph representation comprises one or more of a depth-first search or a breadth-first search.
[0015] In some embodiments, the graph representation comprises a weighted graph with weights assigned to edges based on the scale of individual body parts of the avatar data model.
[0016] In some embodiments, one or more of the first parent model or the second parent model is user-generated content of the virtual experience platform.
[0017] In another aspect, a system is described that may include a memory storing instructions and a processing device coupled to the memory and operable to access the memory, the instructions, when executed by the processing device, causing the processing device to: receive an avatar data model for a three-dimensional (3D) avatar, the avatar data model including at least a first body part from a first parent model and a second body part from a second parent model, the first parent model and the second parent model being different 3D models; create a graphical representation for the avatar data model, the graphical representation starting at a selected root point for the avatar data model and extending to an end portion of the avatar data model; and traverse the graphical representation to determine a plurality of joint locations. the joint locations each represent a respective location where a joint will be placed, and the joint locations are determined based on intersections of different body parts, descriptions of the different body parts, or connections between different body parts; inferring a skeletal structure between the plurality of joint locations such that, for every joint, a skeleton portion spans between a first joint and a second joint and the skeleton portion rotates with movement of one or both of the first joint and the second joint; and combining the joints at each of the plurality of joint locations, the inferred skeletal structure, and the avatar data model to create a practical 3D asset.
[0018] Various embodiments of the system are described.
[0019] In some embodiments, the operations further include animating the practical 3D asset within the virtual environment of the virtual experience.
[0020] In some embodiments, animating the practical 3D asset includes utilizing the inferred skeletal structure for skin deformation during animation of the practical 3D asset.
[0021] In some embodiments, the graphical representation includes multiple nodes representing individual body parts of the avatar data model.
[0022] In some embodiments, the graph representation further includes a plurality of edges, each edge of the plurality of edges spanning between at least two nodes, and each edge of the plurality of edges representing a joint location.
[0023] In some embodiments, each joint combined in a plurality of joint locations comprises either an animated joint or a fixed joint.
[0024] In some embodiments, the operations further include skinning the practical 3D asset based at least on the inferred skeletal structure.
[0025] In some embodiments, traversing the graph representation comprises one or more of a depth-first search or a breadth-first search.
[0026] In some embodiments, the graph representation comprises a weighted graph with weights assigned to edges based on the scale of individual body parts of the avatar data model.
[0027] In another aspect, a non-transitory computer-readable medium is described that may store instructions that, when executed by a processing device, cause the processing device to: receive an avatar data model for a three-dimensional (3D) avatar, the avatar data model including at least a first body part from a first parent model and a second body part from a second parent model, the first parent model and the second parent model being different 3D models; create a graphical representation for the avatar data model, the graphical representation starting at a selected root point for the avatar data model and extending to an end portion of the avatar data model; and traverse the graphical representation to determine a plurality of joint locations, the joint locations being selected from the plurality of joint locations. each represent a respective location where a joint will be placed, and the joint locations are determined based on intersections of different body parts, descriptions of different body parts, or connections between different body parts; inferring a skeletal structure between the plurality of joint locations such that, for every joint, a skeleton portion spans between a first joint and a second joint and the skeleton portion rotates with movement of one or both of the first joint and the second joint; and combining the joints at each of the plurality of joint locations, the inferred skeletal structure, and the avatar data model to create a practical 3D asset.
[0028] Various embodiments of non-transitory computer readable media are described.
[0029] According to yet other aspects, portions, features, and implementation details of the systems, methods, and non-transitory computer-readable media may be combined to form additional aspects, including some aspects that omit and / or modify some or portions of the individual components or features, include additional components or features, and / or other variations, all such variations being within the scope of the present disclosure. [Brief explanation of the drawings]
[0030] [Figure 1] FIG. 1 is a diagram of an exemplary network environment, according to some embodiments. [Figure 2] FIG. 1 illustrates an exemplary progression from an input avatar data model to a functional 3D asset, according to some embodiments. [Figure 3] 1 is a flowchart of an exemplary method for generating a practical 3D asset from an input avatar data model, according to some embodiments. [Figure 4] FIG. 1 is a schematic diagram of an exemplary visualization of a base avatar data model, according to some embodiments. [Figure 5] FIG. 1 is a schematic diagram of an exemplary customized avatar data model, according to some embodiments. [Figure 6] FIG. 1 is a diagram of an exemplary graphical representation of an avatar data model, according to some embodiments. [Figure 7] FIG. 10 is a schematic diagram of an exemplary inferred skeletal structure based on an input customized avatar data model, according to some embodiments. [Figure 8] 1 is a flowchart of a method for generating a functional 3D asset from an input customized avatar data model, according to some embodiments. [Figure 9] FIG. 1 is a block diagram illustrating an exemplary computing device, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION
[0031] In the following detailed description, reference is made to the accompanying drawings, which form a part of this application. In the drawings, like symbols typically identify like components unless the context dictates otherwise. The exemplary embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. The aspects of the present disclosure, as generally described and illustrated in the figures herein, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein.
[0032] References herein to "some embodiments," "one embodiment," "exemplary embodiment," etc. indicate that the described embodiments may include a particular feature, structure, or characteristic, but that not every embodiment necessarily includes that particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with one embodiment, such feature, structure, or characteristic may also be implemented in connection with other embodiments, whether or not explicitly described.
[0033] Various embodiments are described herein in the context of 3D avatars used within a 3D virtual experience or environment. Some embodiments of the techniques described herein may be applied to various types of 3D environments, such as virtual reality (VR) conferences, 3D sessions (e.g., online lectures or other types of presentations involving 3D avatars), virtual concerts, augmented reality (AR) sessions, or within other types of 3D environments that may include one or more users represented within the 3D environment by one or more 3D avatars.
[0034] In some aspects, systems and methods are provided for manipulating 3D assets and creating new practical assets. For example, practical 3D assets are 3D assets that are one or more of: easy to animate with low computational load, suitable for visual presentation in a virtual environment on any type of client device, suitable for multiple different forms of animation, suitable for different skinning methods, suitable for different skinning transformations, and / or suitable for animation on a variety of client devices. Online platforms, such as online virtual experience platforms, generally provide the ability to create, edit, store, and potentially manipulate virtual items, virtual avatars, and other practical 3D assets to be used within the virtual experience.
[0035] For example, the virtual experience platform may include user-generated content or developer-generated content (each referred to herein as "UGC"). UGC may be stored and implemented through the virtual experience platform, for example, by allowing users to search for and interact with various virtual elements to create avatars and other items. Users may select and rearrange various virtual elements from various virtual avatars and 3D models to create new models and avatars. Avatar body creators can create bodies with geometry of any desired / customized shape and size and publish the bodies in a body library hosted by the virtual experience platform.
[0036] At runtime during a virtual experience or other 3D session, a user accesses a body library to select a particular body (including various parts such as the head, torso, limbs, etc.) and to rearrange the body (or parts thereof). According to embodiments described herein, the virtual experience platform may take as input an overall model of the body (or parts thereof) and infer a skeletal structure that allows for appropriate movement (e.g., joint movement, rotation, etc.). In this way, many different avatar body parts may be rearranged to create new avatar bodies without compromising the user experience.
[0037] For example, user-generated content may include a first data model and a second data model. A user may create a new avatar data model by mixing one or more body parts from the first data model and the second data model (taking different body parts from the first and second data models, and possibly from additional data models). Thus, because various data models for an avatar may have body parts of different types, scales, sizes, and structures, it follows that the new avatar data model may not have joints, seams, connection points, connections, and / or other parts that match properly for practical 3D animation using the new avatar data model.
[0038] For example, the new avatar data model may have inconsistent seams that cause improper skinning deformations during animation that detract from the user experience. In another example, the new avatar data model may have inconsistent animated joints that cause the skeletal structure to improperly characterize skinning deformations that detract from the user experience. In another example, the new avatar data model may have too many or too few animated joints to achieve the desired motion in animation for a compelling experience. Furthermore, the new avatar data model may have other inconsistent, improper, or other elements that detract from a compelling user experience and smooth animation. In a different example, two or more of the above deficiencies may be present in the new avatar data model.
[0039] According to some embodiments, the new avatar data model may be analyzed to determine the different body parts, and based on the analysis, a graph representation may be created to represent the different body parts. The graph representation may be traversed using a search algorithm to accurately determine new joint locations that more closely match the different, mixed body parts and / or elements. Animated joints or fixed joints (depending on the parent body part model) may be placed at the determined joint locations. Skeleton structures may be inferred to exist between the joints, such that each joint has at least one connected skeletal member. The new joints and inferred skeletal structures may be combined with the new avatar data model to create a practical 3D asset that includes the correct number of joints and properly aligned elements so that animations may be appealing and smooth.
[0040] In this way, embodiments provide the benefit of reducing the computational resources associated with a user attempting to manually align new joints and skeletal structures in a 3D avatar model. Embodiments also provide the technical benefit of automating much of the 3D model creation process so that users may focus on creating their own avatars rather than relying on complex computer animation. Embodiments also provide the technical benefits of an improved user experience, greater cross-platform compatibility of new avatar models, greater compatibility of new avatar models across multiple virtual experiences, a lower barrier to entry for inexperienced users to 3D modeling, improved user engagement metrics, and other technical benefits.
[0041] Figure 1: System architecture
[0042] FIG. 1 illustrates an exemplary network environment 100 according to some embodiments of the present disclosure. FIG. 1 and other figures use similar reference numbers to identify similar elements. A letter following a reference number, such as "110a," indicates that the text refers specifically to the element with that particular reference number. A reference number in text without a following letter, such as "110," refers to any or all of the elements in the figure bearing that reference number (e.g., "110" in the text refers to reference numbers "110a," "110b," and / or "110n" in the figures).
[0043] The network environment 100 (also referred to herein as a "platform") includes an online virtual experience server 102, a data store 108, and client devices 110a-n (referred to as client devices 110), all coupled via a network 122.
[0044] The online virtual experience server 102 may include, among other things, a virtual experience engine 104, one or more virtual experiences 105, and a modeling component 130. The online virtual experience server 102, in some embodiments, may be configured to provide the virtual experiences 105 to one or more client devices 110 and, via the modeling component 130, to provide automatic generation of an inferred skeletal structure.
[0045] The data store 108 is shown coupled to the online virtual experience server 102, but in some embodiments may be provided as part of the online virtual experience server 102. The data store, in some embodiments, may be configured to store advertising data, user data, engagement data, and / or other contextual data related to the modeling component 130. All data is stored with specific user permissions and in accordance with applicable regulations.
[0046] Client devices 110 (e.g., 110a, 110b, 110n) may include virtual experience applications 112 (e.g., 112a, 112b, 112n) and I / O interfaces 114 (e.g., 114a, 114b, 114n) for interacting with online virtual experience server 102 and for viewing a graphical user interface (GUI), for example, through a computer monitor, television, tablet / laptop screen, or other display (e.g., a wearable display such as virtual reality goggles or other wearable device) (not shown). In some embodiments, client devices 110 may be configured to run and display virtual experiences, which may include customized avatar data models and practical 3D assets, as described herein.
[0047] Network environment 100 is provided for illustrative purposes. In some embodiments, network environment 100 may include the same, fewer, more, or different elements, configured in the same or different manner as shown in FIG.
[0048] In some embodiments, the network 122 may include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or a wide area network (WAN)), a wired network (e.g., an Ethernet network), a wireless network (e.g., an 802.11 network, a Wi-Fi network, or a wireless LAN (WLAN)), a cellular network (e.g., a Long Term Evolution (LTE) network), a router, a hub, a switch, a server computer, or a combination thereof.
[0049] In some embodiments, the data store 108 may be a non-transitory computer-readable memory (e.g., random access memory), a cache, a drive (e.g., a hard drive), a flash drive, a database system, or another type of component or device capable of storing data. The data store 108 may also include multiple storage components (e.g., multiple drives or multiple databases) that may be spread across multiple computing devices (e.g., multiple server computers).
[0050] In some embodiments, the online virtual experience server 102 may include a server having one or more computing devices (e.g., a cloud computing system, a rack-mounted server, a server computer, a cluster of physical servers, a virtual server, etc.). In some embodiments, the server may be included in the online virtual experience server 102, may be a separate system, or may be part of another system or platform. In some embodiments, the online virtual experience server 102 may be a single server, or any combination of multiple servers, load balancers, network devices, and other components. The online virtual experience server 102 may also be implemented on a physical server, although in some embodiments, it may utilize virtualization technology. Other variations of the online virtual experience server 102 may also be applicable.
[0051] In some embodiments, the online virtual experience server 102 may include one or more computing devices (such as rack-mounted servers, router computers, server computers, personal computers, mainframe computers, laptop computers, tablet computers, desktop computers, etc.), data stores (e.g., hard disks, memory, databases), networks, software components, and / or hardware components that may be used to perform operations on the online virtual experience server 102 and to provide users (e.g., via client devices 110) with access to the online virtual experience server 102.
[0052] The online virtual experience server 102 may also include a website (e.g., one or more web pages) or application backend software that may be used to provide users with access to content provided by the online virtual experience server 102. For example, users (or developers) may each access the online virtual experience server 102 using a virtual experience application 112 on a client device 110.
[0053] In some embodiments, the online virtual experience server 102 may include digital asset and digital virtual experience creation services. For example, the platform may provide an administrator interface that enables design, modification, personalized tailoring, and other modification functions. In some embodiments, the virtual experience may be, for example, a two-dimensional (2D) game, a three-dimensional (3D) game, a virtual reality (VR) game, or an augmented reality (AR) game. In some embodiments, virtual experience creators and / or developers may search for virtual experiences, combine parts of virtual experiences, tailor virtual experiences for specific activities (e.g., group virtual experiences), and other features offered through the virtual experience server 102.
[0054] In some embodiments, the online virtual experience platform 102 or the client device 110 may include a virtual experience engine 104 or a virtual experience application 112. In some embodiments, the virtual experience engine 104 may be used for the development or execution of the virtual experience 105. For example, the virtual experience engine 104 may include, among other features, a rendering engine (“renderer”) for 2D, 3D, VR, or AR graphics, a physics engine, a collision detection engine (and collision response), a sound engine, scripting capabilities, a haptics engine, an artificial intelligence engine, networking capabilities, streaming capabilities, memory management capabilities, threading capabilities, scene graph capabilities, or video support for cinematics. Components of the virtual experience engine 104 may generate commands (e.g., rendering commands, collision commands, physics commands, etc.) that aid in the computation and rendering of the virtual experience.
[0055] The online virtual experience server 102 using the virtual experience engine 104 may perform some or all of the virtual experience engine functions (e.g., generate physics commands, rendering commands, etc.) or may offload some or all of the virtual experience engine functions to the virtual experience engine 104 (not shown) of the client device 110. In some embodiments, each virtual experience 105 may have a different ratio between the virtual experience engine functions that run on the online virtual experience server 102 and the virtual experience engine functions that run on the client device 110.
[0056] In some embodiments, virtual experience instructions may refer to instructions that enable client device 110 to render gameplay, graphics, and other features of a virtual experience. The instructions may include one or more of user input (e.g., physical object positioning), character position and velocity information, or commands (e.g., physics commands, rendering commands, collision commands, etc.).
[0057] In some embodiments, client devices 110 may each include a computing device such as a personal computer (PC), a mobile device (e.g., a laptop, a mobile phone, a smartphone, a tablet computer, or a netbook computer), a network-connected television, a gaming console, etc. In some embodiments, client devices 110 may also be referred to as "user devices." In some embodiments, one or more client devices 110 may connect to online virtual experience server 102 at any given moment. It may be noted that the number of client devices 110 is provided by way of example, not limitation. In some embodiments, any number of client devices 110 may be used.
[0058] In some embodiments, one or more of the client devices 110 may include an instance of a virtual experience application 112. The virtual experience application 112 may be rendered for interaction at the client device 110. During user interaction within a virtual experience or another GUI of the online platform 100, the user may create an avatar that includes different body parts from different libraries, from different parent models, and / or from other user-generated content. The modeling component 130 may take as input a data model representing the newly created avatar, read all the individual part information, and infer a skeletal structure that may be used to create a functional 3D asset with an internal skeletal structure that enables animation within the virtual experience.
[0059] In some embodiments, the inferred skeletal structure refers to virtual bones suspended between joints that characterize the motion of the virtual bones so that they rotate, move, or possibly pivot on the joints.
[0060] In some embodiments, joints may include both fixed and animated joints. Other forms of joints may be applicable. Some embodiments include joints that pivot on a single axis, joints that pivot on one or more axes, joints that pivot on two or more axes, joints that pivot on multiple axes, joints that pivot only in a single plane of motion, joints that pivot in one or more planes of motion, joints that pivot in two or more planes of motion, and / or joints that remain fixed in all planes but rotate along a single axis of rotation.
[0061] In some embodiments, joints can be dynamic and physically simulated. For example, joints can be simulated joints based on ball-and-socket, hinge, and / or rod. These and other simulated joints allow implicitly combined skinned character models to be fully or partially physically simulated, where body parts can move independently in response to physics simulation (e.g., a character can be a single rigid body or multiple rigid bodies). Additionally, in some embodiments, platform 100 and modeling component 130 implicitly support complete bodies or body parts being simulated as deformable soft bodies. For example, as long as there is a skeletal structure with skinning nodes, shared nodes can be linked together in separate parts into a single combined skinned entity inferred based on the physical joints used by the simulator, or they can be linked more directly and explicitly using explicit skeletal graph links that combine multiple skeleton nodes together explicitly specified by the user.
[0062] The inferred skeletal structure may be integrated with the joints such that each joint includes at least one skeletal structure and / or portion thereof (e.g., an individual virtual bone) connected to it. In some embodiments, a joint may include one or more virtual bones connected to it. In some embodiments, a joint may include two or more virtual bones connected to it. In some embodiments, a joint may be connected to another joint and at least one virtual bone. In some embodiments, a joint may be connected to one or more joints and at least one virtual bone.
[0063] The inferred skeletal structure (e.g., all virtual bones) and joints may be combined with the avatar data model to generate a practical 3D asset. The combining may be accomplished by the modeling component 130 or another software component. In some embodiments, the combining may be performed on the client device 110 (e.g., through the VE application 112 or a local instance of the modeling component 130).
[0064] The practical 3D assets generated through platform 100 may be stored in any suitable storage device, including a distributed storage device, a hardware storage device, a logical storage device, etc., and in some embodiments may be transmitted to client device 110. Data store 108 may store and index different practical 3D assets generated through use of platform 100, so that client device 110 may easily search for and acquire practical 3D assets. The acquired practical 3D assets may be manipulated to further create new and unique avatars, and may later be processed through methods described herein to create new, unique 3D assets for use in animation, gameplay, or other uses in platform 100 or other platforms.
[0065] Figure 2: Example generation of 3D assets 2 shows an exemplary progression from an input avatar data model to a working 3D asset, according to some embodiments. As shown, the customized avatar data model 210 may include a body cage or mesh 212 that defines the exterior of the model.
[0066] A customized avatar data model may be created by a user of online virtual experience platform 100 from two or more different parent models. For example, a user may select arms, legs, head, torso, etc. from different models and combine them to arrive at customized avatar data model 210.
[0067] Online virtual experience platform 100 may receive customized avatar data model 210 and perform a first sequence of actions 202 to generate an inferred skeletal structure 220. Inferred skeletal structure 220 may include a plurality of joints 222 (only a subset of which are labeled in FIG. 2 ) and a plurality of virtual bones 224 (only a subset of which are labeled in FIG. 2 ). In some embodiments, modeling component 130 performs first sequence of actions 202. First sequence of actions 202 may also be distributed among several different instances of modeling component 130 and / or other software components of online virtual experience platform 100.
[0068] The online virtual experience platform may perform a second sequence of actions 204 to combine the joints 222, the virtual bones 224, and the customized avatar data model 210 to obtain a functional 3D asset 230.
[0069] The practical 3D asset 230 may be a 3D asset with a defined skeletal structure by which the modeling component 130 (or another software component) may perform skinning transformations for an animation sequence based on the defined skeletal structure. For example, the cutaway 232 allows a view below the body cage or mesh 212 to see the underlying skeletal structure and joints as positioned in the practical 3D asset 230.
[0070] Below, a more detailed description of the method for generating a practical 3D asset is presented with reference to FIG.
[0071] Figure 3: An exemplary method for generating a 3D asset using an inferred skeletal structure. FIG. 3 is a flowchart of an example method 300 for generating a functional 3D asset from an input avatar data model, according to some embodiments.
[0072] In some embodiments, method 300 may be implemented on, for example, the server 102 described with reference to FIG. 1 . In some embodiments, part or all of method 300 may be implemented on one or more client devices 110, as shown in FIG. 1 , on one or more developer devices (not shown), or on one or more server devices 102, and / or on a combination of developer devices, server devices, and client devices. In the described example, the implementing system includes one or more digital processors or processing circuits (“processors”) and one or more storage devices (e.g., data store 108 or other storage). In some embodiments, different components of one or more servers and / or clients may perform different blocks or other portions of method 300. In some examples, a first device is described as performing blocks of method 300. Some embodiments cause one or more blocks of method 300 to be performed by one or more other devices (e.g., other client devices or server devices) that may send results or data to the first device.
[0073] In some embodiments, method 300, or portions thereof, can be initiated automatically by a system. In some embodiments, the implementing system is a first device. For example, the method (or portions thereof) can be executed periodically or based on one or more specific events or conditions, such as upon user request, upon a change in avatar dimensions, upon a change in avatar body piece, the expiration of a predetermined time period since the last execution of method 300 for a particular avatar model or user, and / or the occurrence of one or more other conditions, which can be specified in settings read by the method.
[0074] Method 300 may begin at block 302. In block 302, a body mesh is constructed, for example, in a neutral bind pose. The neutral bind pose is the base (or rest) pose of an avatar. For example, when the avatar's skeleton is moved after skinning, the skeleton's action deforms the skin. A pose that does not cause deformation to the skin is the neutral bind pose. Thus, in some embodiments, the body mesh is first constructed in the neutral bind pose, which can simplify subsequent motion.
[0075] In some embodiments, the body mesh may be constructed in an alternative, but reduced or minimally deformed, neutral rest pose. In these and other embodiments, the neutral pose may be chosen based on the likelihood that a skinned version of the avatar will have minimal deformation in this pose. For example, Figure 4 shows an alternative neutral pose, while Figure 5 shows a neutral bind pose. Block 302 may be followed by block 304.
[0076] At block 304, joints may be placed and a skeleton may be inferred based on the placement of the joints to obtain a skeletonized mesh. For example, the joints may be placed based on one or more of intersections of different body parts, descriptions of different body parts, or connections between different body parts.
[0077] The intersection of different body parts may include an intersection between a first body part (such as an upper arm) and a second body part (such as a lower arm), with the joint located at or proximal to the intersection.
[0078] The descriptions of the different body parts may include text or computer-coded descriptions of a first body part (e.g., an upper leg) and a second body part (e.g., a lower leg), and a joint is located between the first body part and the second body part based on the descriptions. For example, the descriptions may also include a distal end of each body part, if appropriate.
[0079] Connections between different body parts may include hard-coded descriptions of connections that may be replaced with joints.
[0080] The skeletonized structure may be inferred based on joint locations. For example, when a joint may be located between the upper arm and the lower arm, an inference of virtual bones extending from this joint onto other body parts may be made. The inference is intelligent inference based on the established body structure. For example, a basic body with an established structure may be humanoid with approximately 15 parts. For example, FIG. 4 shows a basic body and these exemplary 15 body parts. Block 304 may be followed by block 306.
[0081] At block 306, the skeletonized body mesh with its joints and associated virtual bones may be skinned to obtain a 3D model or practical 3D asset. Skinning may include any suitable skinning method. For example, the skinning method may include, but is not limited to, linear blend skinning, inverse linear blend skinning, skeleton-subspace deformation, or other methods.
[0082] In some embodiments, a weight map may be created for each placed joint, which may control how much each vertex will move based on the underlying skeletal structure.
[0083] In some embodiments, skinning may be performed based on a CPU skinning or a graphics processing unit (GPU) skinning process. For example, CPU skinning may rely on a typical available hardware processor for skinning transformations. For example, GPU skinning may rely on a typical available GPU for skinning transformations. Block 306 may be followed by block 308.
[0084] At block 308, the 3D model or functional 3D asset may be animated; for example, the 3D model may be utilized to render an animated avatar. For example, the animation may, in some embodiments, be rendered within a virtual environment within a virtual experience hosted on virtual experience platform 100. The animation may be characterized by an inferred skeletal structure such that skinning deformations are smooth, appealing, and artifacts are reduced, for example, compared to an inconsistent skeleton.
[0085] The basic avatar data model and body parts are described below with reference to FIG.
[0086] Figure 4: Basic avatar data model 4 is a schematic diagram of an example visualization of a base avatar data model, according to some embodiments. The example visualization comprises a body cage 400 and several labeled body parts.
[0087] More specifically, the 15 body parts shown in FIG. 4 are (for a humanoid mannequin) a head, torso, buttocks, right foot, left foot, left lower leg, right lower leg, left upper leg, right upper leg, left hand, right hand, left lower arm, right lower arm, left upper arm, and right upper arm. The number of parts in any body shape may be more or less than the 15 body parts shown. For example, a "one-armed" avatar character may have 12 body parts (as opposed to 15) due to the exclusion of the hand, lower arm, and upper arm. Furthermore, other body shapes may involve fewer or more body parts depending on factors such as body geometry, desired resolution, processing power, and the type of avatar character (e.g., animal, alien, monster, etc.).
[0088] Each of the 15 groups / portions in Figure 4 includes feature points that define that portion of the avatar body. Such groups of feature points can then be mapped to a corresponding underlying skeletal structure for inference. For example, feature points in body cage 400 that define the left / right lower arm and left / right upper arm may be used to characterize the positioning of the joint associated with the elbow. Similarly, feature points in left / right upper leg and left / right lower leg may be used to characterize the positioning of the joint associated with the knee.
[0089] Furthermore, this separation into multiple groups (such as those shown in Figure 4) allows for the inference of skeletal structures, which can be used to accurately skin and animate 3D models.
[0090] A customized avatar data model based on two or more different parent models is described below with reference to FIG.
[0091] Figure 5: Customized avatar data model 5 is a schematic diagram of an example visualization of a customized avatar data model, according to some embodiments. The example visualization comprises a body cage 500 with a head 502, neck 506, torso and legs 504 from a first parent model, and arms 508, 510, hands 509, 511 from a second parent model.
[0092] Because each of the depicted arms 508, 510 is from a different parent model than the head, neck, torso, and legs, the existing skeletal structure in the model does not allow for proper skinning. Therefore, any skinning transformation based on such mismatched parts may appear foreign or unnatural to the user. However, based on the techniques described herein, a new skeletal structure may be inferred by first generating graph representations of the different body parts. Figure 6 shows an illustration of an example graph representation.
[0093] Figure 6: An example graph representation of a customized avatar data model 6 is a diagram of an example graph representation 600 of an avatar data model, according to some embodiments. As shown, graph 600 includes a number of nodes labeled according to the corresponding body parts. In some embodiments, each graph representation will have a number of nodes equal to the number of body parts. In this example, 15 body parts are analyzed from customized model 500.
[0094] Each body part may have at least one associated edge that spans to another body part. The edges in the graph representation 600 are labeled 602, 604, 606, 608, 610, 612, 614, 616, 618, 620, 622, 624, 626, and 628. In this example, each edge corresponds to a specific joint placement. Thus, each edge 602-628 may have a joint associated with it. Similarly, each node that includes two or more edges may have one or more virtual bones inferred between them. In this manner, joints may be positioned in the edges 602-628, and virtual bones may be positioned to approximately align with non-distal nodes. Furthermore, virtual bones may be scaled based on the customized avatar data model and associated body part. For example, FIG. 7 shows an exemplary inferred skeletal structure based on the customized data model 500.
[0095] Figure 7: An example inferred skeleton structure 7 is a schematic diagram of an example inferred skeletal structure based on an input customized avatar data model, according to some embodiments. The inferred skeletal structure 700 may include one or more distal joints 702 associated with distal portions of the avatar data model 500 and one or more internal joints 704 associated with non-distal portions of the avatar data model 500.
[0096] As further shown, an elongated neck bone 706 may be inferred in this example to match the elongated neck portion 506 of the input model 500 (shown in the adjacent figure for clarity). Additionally, elongated lower arm bones 708 and 710 may be inferred in this example to match the elongated lower arms 508, 510 from the second parent model.
[0097] Thus, although the avatar data model 500 is created using different body parts from different parent models, the inferred skeletal structure 700 may be an appropriate skeletal structure to achieve smooth animation along with accurate skinning and skinning deformations during animation.
[0098] Below, a detailed description of an exemplary method for generating a functional 3D asset from an input customized avatar data model is presented with reference to FIG.
[0099] Figure 8: Method for generating practical 3D assets from customized input models FIG. 8 is a flowchart of an example method 800 for generating a functional 3D asset from an input customized avatar data model, according to some embodiments.
[0100] In some embodiments, method 800 may be implemented on, for example, the server 102 described with reference to FIG. 1 . In some embodiments, part or all of method 800 may be implemented on one or more client devices 110, as shown in FIG. 1 , on one or more developer devices (not shown), or on one or more server devices 102, and / or on a combination of developer devices, server devices, and client devices. In the described example, an implementing system includes one or more digital processors or processing circuits (“processors”) and one or more storage devices (e.g., data store 108 or other storage). In some embodiments, different components of one or more servers and / or clients may perform different blocks or other portions of method 800. In some examples, a first device is described as performing blocks of method 800. Some embodiments cause one or more blocks of method 800 to be performed by one or more other devices (e.g., other client devices or server devices) that may send results or data to the first device.
[0101] In some embodiments, method 800, or portions thereof, can be initiated automatically by a system. In some embodiments, the implementing system is a first device. For example, the method (or portions thereof) can be executed periodically or based on one or more specific events or conditions, such as upon user request, upon a change in avatar dimensions, upon a change in avatar body piece, the expiration of a predetermined time period since the last execution of method 800 for a particular avatar model or user, and / or the occurrence of one or more other conditions, which can be specified in settings read by the method.
[0102] Method 800 may begin at block 802. At block 802, an avatar data model for a three-dimensional (3D) avatar may be received. The avatar data model may include at least a first body part from a first parent model and a second body part from a second parent model.
[0103] In this example, the first parent model and the second parent model are different 3D models. Note that in some embodiments, the first parent model and the second parent model may be the same 3D model with different scales (e.g., longer arms or a longer neck, but from the same model). In some embodiments, the avatar data model may have parts from several different 3D models.
[0104] The upper limit on the number of different models may be chosen or selected based on the embodiment. For example, if using avatars with a humanoid appearance, an upper limit of 15 different parent models may be selected. Avatars with different appearances (e.g., alien, robot, animal, etc.) may have different upper limits on the number of body parts. Block 802 may be followed by block 804.
[0105] A graphical representation may be created for the avatar data model at block 804. For example, the graphical representation may start at a selected root point for the avatar data model (e.g., head, left foot, right hand, etc.) and extend to an end portion of the avatar data model (e.g., left / right foot, left / right hand, head, etc.).
[0106] In some embodiments, the graph representation is a weighted graph with weights assigned to edges based on the scale of individual body parts of the avatar data model. In this example, the scale of individual body parts may help inform the inference of the underlying skeletal structure. Block 804 may be followed by block 806.
[0107] At block 806, the graph representation may be traversed to determine a plurality of joint locations. In some embodiments, traversing the graph representation includes one or more of a depth-first search or a breadth-first search. Other search or tree traversal methods may also be applicable.
[0108] In some embodiments, the joint location represents a location where a joint will be placed. Additionally, the joint location may be determined based on an intersection of different body parts, a description of different body parts, or a connection between different body parts. Block 806 may be followed by block 808.
[0109] In block 808, a skeleton structure may be inferred between multiple joint locations. The inference may be for different joints such that a skeleton portion spans between a first joint and a second joint, and such that the skeleton portion rotates with movement of one or both of the first joint and the second joint. In some embodiments, the skeleton portion is a virtual bone that spans between at least two joints. Block 808 may be followed by block 810.
[0110] In block 810, the joints at each of the plurality of joint locations, the inferred skeletal structure, and the avatar data model are combined to create a practical 3D asset. For example, the data representation of the inferred skeletal structure and joints may be combined with the avatar data model so that the entire model may be used in animation. Block 810 may be followed by block 812.
[0111] At block 812, the practical 3D assets may be rendered within the virtual environment. The practical 3D assets may be animated within the virtual environment, for example, an avatar may move (e.g., walk, run, jump, etc.) or an avatar body part may move (e.g., wave, kick, raise arms, turn head, etc.). Additionally, the practical 3D assets may be stored in online platform 100 in some embodiments for use by multiple users.
[0112] As described above, embodiments relate to methods, systems, and computer-readable media for inferring skeletal structure to create practical 3D assets. A user may create an avatar using different parts from different parent models. At creation time, a data model for the avatar is examined to determine joint placement and infer the overall skeletal structure. Different skinning techniques may then be used to deform the avatar skin during animation based on the joint placement and movement. The practical 3D assets may be stored, presented, and rendered through an online virtual experience platform accessible by multiple users.
[0113] Figure 9: Computing Device Below, a more detailed description of various computing devices that may be used to implement the different devices and / or components shown in FIG. 1 is provided with reference to FIG.
[0114] 9 is a block diagram of an exemplary computing device 900 that may be used to implement one or more features described herein, according to some embodiments. In one example, device 900 may be used to implement a computer device (e.g., 102, 110 of FIG. 1 ) and perform appropriate operations described herein. Computing device 900 can be any suitable computer system, server, or other electronic or hardware device. For example, computing device 900 can be a mainframe computer, desktop computer, workstation, portable computer, or electronic device (portable device, mobile device, cell phone, smartphone, tablet computer, television, TV set-top box, personal digital assistant (PDA), media player, gaming device, wearable device, etc.). In some embodiments, device 900 includes a processor 902, memory 904, input / output (I / O) interfaces 906, and audio / video input / output devices 914 (e.g., a display screen, a touch screen, display goggles or glasses, audio speakers, headphones, a microphone, etc.).
[0115] Processor 902 can be one or more processors and / or processing circuits for executing program code and controlling basic operations of device 900. A "processor" includes any suitable hardware and / or software system, mechanism, or component that processes data, signals, or other information. A processor may include a system with a general-purpose central processing unit (CPU), multiple processing units, dedicated circuits for achieving functions, or other systems. Processing need not be limited to a particular geographic location or to having time limitations. For example, a processor may perform its functions in "real time," "offline," "batch mode," etc. Portions of processing may be performed by different (or the same) processing systems, at different times, and in different locations. A computer may be any processor in communication with memory.
[0116] Memory 904 is typically provided in device 900 for access by processor 902 and may be any suitable processor-readable storage medium, e.g., random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory, etc., suitable for storing instructions for execution by the processor, located separately from and / or integrated with processor 902. Memory 904 may store software run on server device 900 by processor 902, including an operating system 908, software applications 910, and associated data 912. In some embodiments, applications 910 may include instructions that enable processor 902 to perform the functions described herein. Software applications 910 may include some or all of the functionality needed to model components and to infer 3D skeletal structures for practical 3D assets. In some embodiments, one or more portions of the software application 910 may be implemented in special-purpose hardware, such as an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), a machine learning processor, etc. In some embodiments, one or more portions of the software application 910 may be implemented in a general-purpose processor, such as a central processing unit (CPU) or a graphics processing unit (GPU). In various embodiments, any suitable combination of special-purpose and / or general-purpose processing hardware may be used to implement the software application 910.
[0117] For example, software applications 910 stored in memory 904 may include instructions for retrieving user data, instructions for displaying / presenting an avatar with an inferred skeletal structure, and / or other functions or software, such as modeling component 130, VE engine 104, and / or VE application 112. Any of the software in memory 904 can alternatively be stored on any other suitable storage location or computer-readable medium. Additionally, memory 904 (and / or other connected storage devices) can store instructions and data used in the features described herein. Memory 904 and any other type of storage (such as magnetic disk, optical disk, magnetic tape, or other tangible medium) can be considered “storage” or “storage device.”
[0118] The I / O interface 906 may provide functionality to allow the server device 900 to interface with other systems and devices. For example, network communication devices, storage devices (e.g., memory and / or data store 106), and input / output devices may communicate via the interface 906. In some embodiments, the I / O interface may connect to interface devices, including input devices (keyboards, pointing devices, touchscreens, microphones, cameras, scanners, etc.) and / or output devices (display devices, speaker devices, printers, monitors, etc.).
[0119] For ease of illustration, FIG. 9 shows one block for each of the processor 902, memory 904, I / O interface 906, software blocks 908 and 910, and database 912. These blocks may represent one or more processors or processing circuits, operating systems, memories, I / O interfaces, applications, and / or software modules. In other embodiments, the device 900 may have less than all of the components shown and / or may have other elements, including other types of elements, instead of or in addition to those shown herein. Although the online server 102 has been described as performing the operations described in some embodiments herein, any suitable component or combination of components of the online server 102 or a similar system, or any suitable processor or processors associated with such a system, may perform the described operations.
[0120] User devices can also implement and / or be used in conjunction with the features described herein. An exemplary user device can be a computing device that includes several components similar to device 900, e.g., processor 902, memory 904, and I / O interface 906. Operating systems, software, and applications suitable for the client device can be provided in the memory and used by the processor. The I / O interface for the client device can be connected to a network communication device and to input and output devices, e.g., a microphone for capturing sound, a camera for capturing images or video, an audio speaker device for outputting sound, a display device for outputting images or video, or other output devices. A display device in audio / video input / output device(s) 914 can be connected to (or included within) device 900, e.g., to display image pre-processing and post-processing as described herein, where such a display device can include any suitable display device, e.g., an LCD, LED, or plasma display screen, a CRT, a television, a monitor, a touchscreen, a 3D display screen, a projector, or other visual display device. Some embodiments may provide an audio output device, for example, text-to-speech output or synthesis.
[0121] Methods, blocks, and / or operations described herein may, where appropriate, be performed in an order different from that illustrated or described, and / or may be performed concurrently (partially or completely) with other blocks or operations. Some blocks or operations may be performed on one portion of data and then performed again later, e.g., on another portion of data. Not all of the blocks and operations described need be performed in various embodiments. In some embodiments, blocks and operations may be performed multiple times in a method, in a different order, and / or at different times.
[0122] In some embodiments, some or all of the methods may be implemented on a system, such as one or more client devices. In some embodiments, one or more methods described herein may be implemented, for example, on a server system and / or on both a server system and a client system. In some embodiments, different components of one or more servers and / or clients may perform different blocks, operations, or other portions of a method.
[0123] One or more methods described herein (e.g., methods 300 and / or 800) can be implemented by computer program instructions or code that can be executed on a computer. For example, the code can be implemented by one or more digital processors (e.g., microprocessors or other processing circuits) and stored on a computer program product that includes a non-transitory computer-readable medium (e.g., storage medium), e.g., a magnetic, optical, electromagnetic, or semiconductor storage medium, including semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random access memory (RAM), read-only memory (ROM), flash memory, rigid magnetic disks, optical disks, solid-state memory drives, etc. The program instructions can also be contained in and provided as electronic signals, e.g., in the form of software as a service (SaaS) delivered from a server (e.g., a distributed system and / or a cloud computing system). Alternatively, one or more methods can be implemented in hardware (such as logic gates) or a combination of hardware and software. Exemplary hardware may be a programmable processor (e.g., a field programmable gate array (FPGA), complex programmable logic device), general-purpose processor, graphics processor, application-specific integrated circuit (ASIC), etc. One or more methods may be implemented as part of or a component of an application running on the system, or as an application or software running in conjunction with other applications and the operating system.
[0124] One or more methods described herein can be implemented in a standalone program, which can be executed on any type of computing device, a program running on a web browser, a mobile application ("app") running on a mobile computing device (e.g., a cell phone, a smartphone, a tablet computer, a wearable device (such as a watch, an armband, jewelry, headwear, goggles, glasses, etc.), a laptop computer, etc.). In one example, a client / server architecture can be used, e.g., the mobile computing device (as a client device) sends user input data to a server device and receives live feedback data for output (e.g., for display) from the server. In another example, computation can be split between the mobile computing device and one or more server devices.
[0125] In situations where some embodiments described herein may obtain or use user data (e.g., user demographics, user behavior data, user context data, user preferences for advertising, etc.), the user is provided with options to control whether and how such information is collected, stored, or used. That is, the embodiments described herein collect, store, and / or use user information upon receipt of explicit user authorization and in accordance with applicable regulations.
[0126] The user is given control over whether the program or feature collects user information about that particular user or other users associated with the program or feature. Each user about whom information is to be collected is presented with options (e.g., via a user interface) to allow the user to exercise control over information collection associated with that user to grant permission or authorization for whether information is to be collected and what portion of the information is to be collected. Additionally, some data may be modified in one or more ways before being stored or used so that personally identifiable information is removed. As one example, a user's identifying information may be modified (e.g., by substitution with a pseudonym, numeric values, etc.) so that personally identifiable information cannot be determined. In another example, a user's geographic location may be generalized to a larger region (e.g., city, zip code, state, country, etc.).
[0127] Although this description has been set forth in terms of specific embodiments thereof, these specific embodiments are illustrative only and not limiting, and concepts illustrated in the examples may be applied to other examples and embodiments.
[0128] It should be noted that the functional blocks, operations, features, methods, devices, and systems described in this disclosure may be combined or divided into different combinations of systems, devices, and functional blocks, as will be known to those skilled in the art. Any suitable programming language and programming techniques may be used to implement the routines of a particular embodiment. Different programming techniques, e.g., procedural or object-oriented, may be employed. The routines may be executed on a single processing device or multiple processors. While steps, operations, or computations may be presented in a particular order, the order may be changed in different particular embodiments. In some embodiments, multiple steps or operations shown as sequential in this specification may be performed simultaneously. [Explanation of symbols]
[0129] 100 network environment, online virtual experience platform 102 Online Virtual Experience Server 104 Virtual Experience (VE) Engine 105 Virtual Experience 108 Datastores 110, 110a, 110b, 110n client devices 112 Virtual Experience (VE) Applications 112a, 112b, 112n Virtual Experience Applications 114, 114a, 114b, 114n Input / Output (I / O) Interfaces 122 Network 130 Modeling Components 202 First Sequence of Actions 204 Second Sequence of Actions 210 Customized Avatar Data Model 212 Body cage or mesh 220,700 Inferred Skeleton Structure 222 Joint 224 Virtual Bones 230 Practical 3D Assets 232 Cutaway 400 Body Cage 500 Body Cage, Customized Data Model, Avatar Data Model, Input Model 502 Head 504 Torso and Legs 506 Neck, elongated neck 508, 510 Arms, elongated lower arms 509, 511 hands 600 Graphical Representation 602, 604, 606, 608, 610, 612, 614, 616, 618, 620, 622, 624, 626, 628 Edge 702 Distal Joint 704 Internal Joint 706 Long Neck Bone 708, 710 Long and slender lower arm bone 900 Exemplary Computing Device 902 processor 904 memory 906 Input / Output (I / O) Interface 910 Software Applications 914 Audio / Video Input / Output Devices
Claims
1. 1. A computer-implemented method comprising: receiving an avatar data model for a three-dimensional (3D) avatar, the avatar data model including at least a first body part from a first parent model and a second body part from a second parent model, the first parent model and the second parent model being different 3D models; creating a graphical representation for the avatar data model, the graphical representation starting at a selected root point for the avatar data model and extending to a terminal portion of the avatar data model; traversing the graph representation to determine a plurality of joint locations, each of the plurality of joint locations representing a respective location where a joint will be placed, the joint locations being determined based on intersections of different body parts, descriptions of different body parts, or connections between different body parts; inferring a skeleton structure between the plurality of joint locations such that, for every joint, a skeleton portion spans between a first joint and a second joint and such that a skeleton portion rotates with movement of one or both of the first joint and the second joint; combining the joints at each of the plurality of joint locations, the inferred skeletal structure, and the avatar data model to create a practical 3D asset; 4. A computer-implemented method comprising:
2. animating said functional 3D assets within a virtual environment of a virtual experience. The computer-implemented method of claim 1 , further comprising:
3. animating the functional 3D asset, utilizing the inferred skeletal structure for skin deformation during animation of the practical 3D asset.
3. The computer-implemented method of claim 2, comprising:
4. The computer-implemented method of claim 1 , wherein the graphical representation includes a plurality of nodes representing individual body parts of the avatar data model.
5. 5. The computer-implemented method of claim 4, wherein the graph representation further comprises a plurality of edges, each edge of the plurality of edges spanning between at least two nodes, and each edge of the plurality of edges representing a joint location.
6. The computer-implemented method of claim 1 , wherein each joint combined at the plurality of joint locations comprises either an animated joint or a fixed joint.
7. skinning the practical 3D asset based at least on the inferred skeletal structure; The computer-implemented method of claim 1 , further comprising:
8. traversing the graph representation, performing one or more of a depth-first search or a breadth-first search; 2. The computer-implemented method of claim 1, comprising:
9. The computer-implemented method of claim 1 , wherein the graph representation comprises a weighted graph having weights assigned to edges based on a scale of individual body parts of the avatar data model.
10. The computer-implemented method of claim 1 , wherein one or more of the first parent model or the second parent model is user-generated content of a virtual experience platform.
11. 1. A system comprising: A memory storing instructions; a processing device coupled to the memory and operable to access the memory; the instructions, when executed by the processing device, cause the processing device to: receiving an avatar data model for a three-dimensional (3D) avatar, the avatar data model including at least a first body part from a first parent model and a second body part from a second parent model, the first parent model and the second parent model being different 3D models; creating a graphical representation for the avatar data model, the graphical representation starting at a selected root point for the avatar data model and extending to a terminal portion of the avatar data model; traversing the graph representation to determine a plurality of joint locations, each of the plurality of joint locations representing a respective location where a joint will be placed, the joint locations being determined based on intersections of different body parts, descriptions of different body parts, or connections between different body parts; inferring a skeleton structure between the plurality of joint locations such that, for every joint, a skeleton portion spans between a first joint and a second joint and such that a skeleton portion rotates with movement of one or both of the first joint and the second joint; combining the joints at each of the plurality of joint locations, the inferred skeletal structure, and the avatar data model to create a practical 3D asset; A system that causes a system to perform an operation including:
12. The operation, animating said functional 3D assets within a virtual environment of a virtual experience. The system of claim 11 further comprising:
13. animating said functional 3D assets, Utilizing the inferred skeletal structure for skin deformation during animation of the practical 3D asset. The system of claim 12, comprising:
14. The system of claim 11 , wherein the graphical representation includes a plurality of nodes representing individual body parts of the avatar data model.
15. 15. The system of claim 14, wherein the graph representation further comprises a plurality of edges, each edge of the plurality of edges spanning between at least two nodes, and each edge of the plurality of edges representing a joint location.
16. The system of claim 11 , wherein each joint combined at the plurality of joint locations comprises either an animated joint or a fixed joint.
17. The operation, Skinning the practical 3D asset based at least on the inferred skeletal structure. The system of claim 11 further comprising:
18. traversing the graph representation Performing one or more of a depth-first search or a breadth-first search The system of claim 11 , comprising:
19. The system of claim 11 , wherein the graph representation comprises a weighted graph having weights assigned to edges based on a scale of individual body parts of the avatar data model.
20. A non-transitory computer readable medium having instructions stored thereon, the instructions, in response to execution by a processing device, causing the processing device to: receiving an avatar data model for a three-dimensional (3D) avatar, the avatar data model including at least a first body part from a first parent model and a second body part from a second parent model, the first parent model and the second parent model being different 3D models; creating a graphical representation for the avatar data model, the graphical representation starting at a selected root point for the avatar data model and extending to a terminal portion of the avatar data model; traversing the graph representation to determine a plurality of joint locations, each of the plurality of joint locations representing a respective location where a joint will be placed, the joint locations being determined based on intersections of different body parts, descriptions of different body parts, or connections between different body parts; inferring a skeleton structure between the plurality of joint locations such that, for every joint, a skeleton portion spans between a first joint and a second joint and such that a skeleton portion rotates with movement of one or both of the first joint and the second joint; combining the joints at each of the plurality of joint locations, the inferred skeletal structure, and the avatar data model to create a practical 3D asset; A non-transitory computer-readable medium for causing a computer to perform operations including:
Citation Information
Patent Citations
Method for communicating and generating computer graphics, animation data, and recording media
CN1171580A
Method for transmitting, receiving and generating computer graphics animation data, and recording medium
JP1998040419A
Method of structuring handwritten line graphic, method of identifying part of the structure, and recording medium with program about those methods recorded thereon
JP2001155170A
Animation generating system
JP2002319035A
Method for controlling movement, robot and program
JP2012157964A