Articulated-object generation using kinematic-graph grammar

Abstract

A computer-implemented method includes obtaining a first kinematic graph that represents a first articulated object. The method includes generating a set of primitive structures by decomposing the first kinematic graph. The method includes generating a graph hierarchy by enumerating one or more valid combinations of the set of primitives. The method includes determining the production rules based on the graph hierarchy. The method includes determining kinematic graph grammar based on the set of primitive structures and the production rules. The method includes determining the continuous parameters for the discrete kinematic graph structure based on the production rules using a generative flow network. The method includes generating a second kinematic graph to represent a second articulated object based on the kinematic graph grammar and the continuous parameters.

Description

ARTICULATED-OBJECT GENERATION USING KINEMATIC-GRAPH GRAMMARCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is a non-provisional application that claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 63 / 712,297, filed on October 25, 2024, the contents of which are hereby incorporated by reference herein in their entirety.TECHNICAL FIELD

[0002] Embodiments relate generally to online virtual experience platforms, and more particularly but not exclusively, to methods, systems, and computer readable media for articulated-object generation using kinematic-graph grammar.BACKGROUND

[0003] Online platforms, such as virtual experience platforms including online gaming platforms, may include various virtual objects. In some platforms, virtual objects may be static or articulated.

[0004] Existing methods use training datasets with manually labeled shape parts and mechanical joints to generate articulated objects.

[0005] The background description provided herein is for the purpose of presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.SUMMARY

[0006] According to one aspect of the present disclosure, a computer-implemented method is provided. The computer-implemented method includes obtaining, by a processor, a first kinematic graph that represents a first articulated object that includes a first plurality of parts interconnected by one or more first joints. The first kinematic graph includes a plurality of nodes corresponding to the first plurality of parts and one or more edges corresponding to theone or more first joints. The computer-implemented method includes generating, by the processor, a set of primitive structures by decomposing the first kinematic graph. Each primitive structure in the set of primitive structures includes a respective terminal node corresponding to a respective node of the plurality of nodes and at least one respective nonterminal node connected to the respective terminal node. A non-terminal node corresponds to a node-edge connection. The computer-implemented method includes generating, by the processor, a graph hierarchy by enumerating one or more valid combinations of the set of primitives. The computer-implemented method includes determining, by the processor, production rules based on the graph hierarchy. The computer-implemented method includes determining, by the processor, kinematic graph grammar for generating discrete kinematic graph structures based on the set of primitive structures and the production rules. The computer-implemented method includes determining, by the processor, continuous parameters based on the production rules using a generative flow network. The continuous parameters include a plurality7of bounding boxes each corresponding to a respective node of the plurality of nodes and one or more joint parameters each corresponding to a respective edge of the one or more edges. The computer-implemented method includes generating, by the processor, a second kinematic graph, different from the first kinematic graph, to represent a second articulated object, different from the first articulated object, based on the kinematic graph grammar and the continuous parameters. The second articulated object includes a second plurality of parts interconnected by one or more second joints. The computer-implemented method includes animating, by the processor, the second plurality of parts interconnected by the one or more second joints of the second articulated object based on the second kinematic graph.

[0007] In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes determining, by the processor, the one or more edges based on the first kinematic graph. In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes segmenting, by the processor, each of the one or more edges into two respective parts. In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes generating, by the processor, a non-terminal node at a respective segmentation point of each of the two respective parts for each of the one or more edges.

[0008] In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating, by the processor, a first level of the graph hierarchy based on the set of primitives. In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating, by the processor, a second level of the graph hierarchy by attaching each primitive in the set of primitives to all other primitives in the set of primitives to form a first set of graphs.

[0009] In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating, by the processor, a third level of the graph hierarchy by attaching each primitive in the set of primitives to each graph in the first set of graphs to form a second set of graphs.

[0010] In some implementations, determining the production rules based on the graph hierarchy includes determining, by the processor, the production rules for each graph-hierarchy level pair that convert a set of graphs from level t to another graph in level t +1.

[0011] In some implementations, determining the continuous parameters based on the production rules using the generative flow network includes determining, by the processor, the plurality of bounding boxes each corresponding to the respective node of the plurality of nodes based on a respective center position and a respective spatial extent of the respective node based on the first kinematic graph. In some implementations, determining the continuous parameters based on the production rules using the generative flow network includes determining, by the processor, the one or more joint parameters each corresponding to the respective edge of the one or more edges based on a respective joint axis and a respective joint position of a corresponding edge based on the first kinematic graph.

[0012] In some implementations, the first kinematic graph further includes a type set that associates each node of the plurality of nodes with a node type and each edge of the one or more edges with an edge type, respectively. In some implementations, the continuous parameters include a plurality of bounding boxes for the plurality of nodes and one or more joint parameters for the one or more edges.

[0013] In some implementations, the node type is determined based on a respective geometry label of corresponding to the first articulated objected in a training dataset.

[0014] In some implementations, the edge type comprises revolute, prismatic, screw, or free.

[0015] In some implementations, the production rules define local connectivity constraints as only those node-type connections identified in a training data set.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable medium with instructions stored thereon that, when executed by one or more hardware processors, cause the one or more hardware processors to perform or control performance of operations. The operations include obtaining, by a processor, a first kinematic graph that represents a first articulated object that includes a first plurality of parts interconnected by one or more first joints. The first kinematic graph includes a plurality of nodes corresponding to the first plurality of parts and one or more edges corresponding to the one or more first joints. The operations include generating, by the processor, a set of primitive structures by decomposing the first kinematic graph. Each primitive structure in the set of primitive structures includes a respective terminal node corresponding to a respective node of the plurality of nodes and at least one respective non-terminal node connected to the respective terminal node. A non-terminal node corresponds to a node-edge connection. The operations include generating, by the processor, a graph hierarchy by enumerating one or more valid combinations of the set of primitives. The operations include determining, by the processor, production rules based on the graph hierarchy. The operations include determining, by the processor, kinematic graph grammar for generating discrete kinematic graph structures based on the set of primitive structures and the production rules. The operations include determining, by the processor, continuous parameters based on the production rules using a generative flow network. The continuous parameters include a plurality of bounding boxes each corresponding to a respective node of the plurality of nodes and one or more joint parameters each corresponding to a respective edge of the one or more edges. The operations include generating, by the processor, a second kinematic graph, different from the first kinematic graph, to represent a second articulated object, different from the first articulated object, based on the kinematic graph grammar and the continuous parameters. The second articulated object includes a second plurality of parts interconnected by one or more second joints. The operations include animating, by the processor, the second plurality of parts interconnected by the one or more second joints of the second articulated object based on the second kinematic graph.

[0017] In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes determining, by the processor, the one or more edges based on the first kinematic graph. In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes segmenting, by the processor, each of the one or more edges into two respective parts. In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes generating, by the processor, a non-terminal node at a respective segmentation point of each of the two respective parts for each of the one or more edges.

[0018] In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating, by the processor, a first level of the graph hierarchy based on the set of primitives. In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating, by the processor, a second level of the graph hierarchy by attaching each primitive in the set of primitives to all other primitives in the set of primitives to form a first set of graphs.

[0019] In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating, by the processor, a third level of the graph hierarchy by attaching each primitive in the set of primitives to each graph in the first set of graphs to form a second set of graphs.

[0020] In some implementations, determining the production rules based on the graph hierarchy includes determining, by the processor, the production rules for each graph-hierarchy level pair that convert a set of graphs from level t to another graph in level t +1.

[0021] In some implementations, determining the continuous parameters based on the production rules using the generative flow network includes determining, by the processor, the plurality of bounding boxes each corresponding to the respective node of the plurality of nodes based on a respective center position and a respective spatial extent of the respective node based on the first kinematic graph. In some implementations, determining the continuous parameters based on the production rules using the generative flow network includes determining, by the processor, the one or more joint parameters each corresponding to the respective edge of the one or more edges based on a respective joint axis and a respective joint position of a corresponding edge based on the first kinematic graph.

[0022] In some implementations, the first kinematic graph further includes a type set that associates each node of the plurality’ of nodes with a node type and each edge of the one or more edges with an edge type, respectively. In some implementations, the continuous parameters include a plurality of bounding boxes for the plurality of nodes and one or more joint parameters for the one or more edges.

[0023] In some implementations, the node type is determined based on a respective geometry’ label of corresponding to the first articulated objected in a training dataset.

[0024] In some implementations, the edge type comprises revolute, prismatic, screw, or free.

[0025] In some implementations, the production rules define local connectivity’ constraints as only those node-type connections identified in a training data set.

[0026] According to a further aspect of the present disclosure, a computing device is provided. The computing device includes one or more hardware processors. The computing device includes a non-transitoiy computer readable medium coupled to the one or more hardware processors, with instructions stored thereon, that when executed by the one or more hardware processors, cause the one or more hardware processors to perform or control performance of operations. The operations include obtaining, by a processor, a first kinematic graph that represents a first articulated object that includes a first plurality' of parts interconnected by one or more first joints. The first kinematic graph includes a plurality of nodes corresponding to the first plurality of parts and one or more edges corresponding to the one or more first joints. The operations include generating, by the processor, a set of primitive structures by decomposing the first kinematic graph. Each primitive structure in the set of primitive structures includes a respective terminal node corresponding to a respective node of the plurality of nodes and at least one respective non-terminal node connected to the respective terminal node. A non-terminal node corresponds to a node-edge connection. The operations include generating, by the processor, a graph hierarchy by enumerating one or more valid combinations of the set of primitives. The operations include determining, by the processor, production rules based on the graph hierarchy. The operations include determining, by the processor, kinematic graph grammar for generating discrete kinematic graph structures based on the set of primitive structures and the production rules. The operations include determining, by the processor, continuous parameters based on the production rules using a generative flownetwork. The continuous parameters include a plurality of bounding boxes each corresponding to a respective node of the plurality of nodes and one or more joint parameters each corresponding to a respective edge of the one or more edges. The operations include generating, by the processor, a second kinematic graph, different from the first kinematic graph, to represent a second articulated object, different from the first articulated object, based on the kinematic graph grammar and the continuous parameters. The second articulated object includes a second plurality of parts interconnected by one or more second joints. The operations include animating, by the processor, the second plurality of parts interconnected by the one or more second joints of the second articulated object based on the second kinematic graph.

[0027] In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes determining, by the processor, the one or more edges based on the first kinematic graph. In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes segmenting, by the processor, each of the one or more edges into two respective parts. In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes generating, by the processor, a non-terminal node at a respective segmentation point of each of the two respective parts for each of the one or more edges.

[0028] In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating, by the processor, a first level of the graph hierarchy based on the set of primitives. In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating, by the processor, a second level of the graph hierarchy by attaching each primitive in the set of primitives to all other primitives in the set of primitives to form a first set of graphs.

[0029] In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating, by the processor, a third level of the graph hierarchy by attaching each primitive in the set of primitives to each graph in the first set of graphs to form a second set of graphs.

[0030] In some implementations, determining the production rules based on the graph hierarchy includes determining, by the processor, the production rules for each graph-hierarchy level pair that convert a set of graphs from level t to another graph in level t +1.

[0031] In some implementations, determining the continuous parameters based on the production rules using the generative flow network includes determining, by the processor, the plurality of bounding boxes each corresponding to the respective node of the plurality of nodes based on a respective center position and a respective spatial extent of the respective node based on the first kinematic graph. In some implementations, determining the continuous parameters based on the production rules using the generative flow network includes determining, by the processor, the one or more joint parameters each corresponding to the respective edge of the one or more edges based on a respective joint axis and a respective joint position of a corresponding edge based on the first kinematic graph.

[0032] In some implementations, the first kinematic graph further includes a type set that associates each node of the plurality of nodes with a node type and each edge of the one or more edges with an edge type, respectively. In some implementations, the continuous parameters include a plurality of bounding boxes for the plurality of nodes and one or more joint parameters for the one or more edges.

[0033] In some implementations, the node type is determined based on a respective geometry label of corresponding to the first articulated objected in a training dataset.

[0034] In some implementations, the edge type comprises revolute, prismatic, screw, or free.

[0035] In some implementations, the production rules define local connectivity constraints as only those node-ty pe connections identified in a training data set.

[0036] According to yet another aspect, portions, features, and implementation details of the systems, methods, and non-transitory computer-readable media may be combined to form additional aspects, including some aspects which omit and / or modify some or portions of individual components or features, include additional components or features, and / or other modifications; and all such modifications are within the scope of this disclosure.BRIEF DESCRIPTION OF DRAWINGS

[0037] FIG. 1 is a diagram of an example network environment, in accordance with some implementations.

[0038] FIG. 2 is a diagram of a first example articulated obj ect and its corresponding typed graph, in accordance with some implementations.

[0039] FIG. 3 is a diagram of a second example articulated object and its corresponding typed graph, in accordance with some implementations.

[0040] FIG. 4A is a diagram of example kinematic graph grammar for generating an articulated object, in accordance with some implementations.

[0041] FIG. 4B is a diagram of an example production sequence for generating an articulated object using the kinematic graph grammar of FIG. 4A, in accordance with some implementations.

[0042] FIG. 5 is a diagram of an example decomposition of kinematic graphs into corresponding sets of primitives, in accordance with some implementations.

[0043] FIG. 6 is a diagram of an example graph hierarchy enumerating possible combinations of primitives to generate various articulated objects, in accordance with some implementations.

[0044] FIG. 7 is a flowchart of an example method of articulated-object generation, in accordance with some implementations.

[0045] FIG. 8 is a block diagram illustrating an example computing device, in accordance with some implementations.DETAILED DESCRIPTION

[0046] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative implementations described in the detailed description, drawings, and claims are not meant to be limiting. Other implementations may be utilized, and other changes may be made, without departing from thespirit or scope of the subject matter presented herein. Aspects of the present disclosure, as generally described herein, and illustrated in the Figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein.

[0047] References in the specification to “some implementations ’ “an implementation,” “an example implementation,” etc. indicate that the implementation described may include a particular feature, structure, or characteristic, but every implementation may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same implementation. Further, when a particular feature, structure, or characteristic is described in connection with an implementation, such feature, structure, or characteristic may be effected in connection with other implementations whether or not explicitly described.

[0048] V arious embodiments are described herein in the context of 3D avatars that are used in a 3D virtual experience or environment. Some implementations of the techniques described herein may be applied to various types of 3D environments, such as a virtual reality (VR) conference, a 3D session (e.g., an online lecture or other type of presentation involving 3D avatars), a virtual concert, an augmented reality (AR) session, an online game, or in other types of 3D environments that may include one or more users that are represented in the 3D environment by one or more 3D avatars.

[0049] Some articulated-shape generation models rely on training datasets with manually labeled shape parts and mechanical joints, which are tedious to obtain and sparse in nature. Consequently, these methods can easily overfit to the limited available training data, making it difficult to generate diverse and high-quality shapes that fall outside the training distribution.

[0050] To address these and other challenges, the present disclosure proposes a data-efficient generative model for articulated objects that generalizes beyond the training dataset with a few7dozen training samples. The generative model described herein may encode the spatial and relational information between parts of a three-dimensional (3D) articulated object. Once a kinematic graph representation is established, a machine learning model that learns both the “grammar” to generate such graphs and the generative process to synthesize new graphs using the learned grammars may be trained. Then, continuous parameters are generated basedon the discrete graphs produced by the learned kinematic graph grammar using a generative-flow model.

[0051] Additional details of the present technique(s) are provided below with reference to FIGs. 1-8.

[0052] FIG. 1 is a diagram of an example system architecture 100 that includes a virtual experience platform that can support the generation and presentation of articulated objects, in accordance with some implementations. In the example of FIG. 1, the 3D environment platform will be described in the context of a virtual experience server 102 purely for purposes of explanation, and various other implementations can provide other types of 3D environment platforms, such as online meeting platforms, virtual reality (VR) or augmented reality (AR) platforms, gaming platforms, or other types of platforms that can provide 3D content. The description provided herein for the virtual experience ser er 102 and other elements of the system architecture 100 can be adapted to be operable with such other types of 3D environment platforms. References to games and related gaming functionality throughout this description are for purposes of illustrating various example features, and such features can be adapted for other ty pes of virtual experiences that may not necessarily involve games.

[0053] Virtual experience platforms (also referred to as ‘'user-generated content platforms” or “user-generated content systems”) offer a variety of ways for users to interact with one another, such as while the users are playing an electronic virtual experience. For example, users of a virtual experience platform may work together towards a common goal, share various virtual gaming items, send electronic messages to one another, and so forth. Users of a virtual experience platform may play virtual experiences using characters, such as the 3D avatars, which the users can navigate through a 3D world rendered in the electronic virtual experience.

[0054] A virtual experience platform may also enable users of the platform to create and animate avatars, as well as enabling the users to create other graphical objects to place in the 3D world. For example, users of the virtual experience platform may be allowed to create, design, and customize the avatar, and to create other 3D objects for presentation in the 3D world.

[0055] FIG. 1 is a diagram of an example environment (including system 100) to generate 3D articulated objects to be rendered on a computing device, in accordance with some implementations. FIG. 1 and other figures use like reference numerals to identity like elements.-liA leter after a reference numeral, such as “110,’' indicates that the text refers specifically to the element having that particular reference numeral. A reference numeral in the text without a following leter, such as ‘"110,” refers to any or all of the elements in the figures bearing that reference numeral (e.g. “110” in the text refers to reference numerals “110a,” “110b,” and / or “1 lOn” in the figures).

[0056] The system architecture 100 (also referred to as “system” herein) includes online virtual experience sen- er 102, data store 120, client devices 110a, 110b, and 11 On (generally referred to as “client device(s) 110” herein), content management server 140, and developer devices 130a and 130n (generally referred to as “developer device(s) 130” herein). Virtual experience sen- er 102, content management server 140, data store 120, client devices 110, and developer devices 130 are coupled via network 122. In some implementations, client devices 110 and developer device(s) 130 may refer to the same or same ty pe of device.

[0057] Online virtual experience server 102 can include a virtual experience engine 104, one or more virtual expert ence(s) 106, and graphics engine 108. A client device 110 can include a virtual experience application 112, and input / output (I / O) interfaces 114 (e.g., input / output devices). The input / output devices can include one or more of a microphone, speakers, headphones, display device, mouse, keyboard, game controller, touchscreen, virtual reality consoles, etc. The input / output devices can also include accessory devices that are connected to the client device by means of a cable (wired) or that are wirelessly connected.

[0058] Content management server 140 can include a graphics engine 144, and a classification controller 146. In some implementations, the content management server 140 may include a plurality of servers. In some implementations, the plurality of servers may be arranged in a hierarchy (e.g., based on respective prioritization values assigned to content sources).

[0059] Graphics engine 144 may be utilized for the rendering of one or more objects (e.g., 3D objects associated with the virtual environment). Classification controller 146 may be utilized to classify assets such as 3D objects and for the detection of inauthentic digital assets, etc. Data store 148 may be utilized to store a search index, model information, etc.

[0060] A developer device 130 can include a virtual experience application 132 and input / output (I / O) interfaces 134 (e.g., input / output devices). The input / output devices caninclude one or more of a microphone, speakers, headphones, display device, mouse, keyboard, game controller, touchscreen, virtual reality consoles, etc.

[0061] System architecture 100 is provided for illustration. In different implementations, the system architecture 100 may include the same, fewer, more, or different elements configured in the same or different manner as that shown in FIG. 1.

[0062] In some implementations, network 122 may include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), a wired network (e.g., ethemet network), a wireless network (e.g., an 802.11 network, a Wi-Fi® network, or wireless LAN (WLAN)), a cellular network (e.g., a 5G network, a long term evolution (LTE) network, etc.), routers, hubs, switches, server computers, or a combination thereof.

[0063] In some implementations, the data store 120 may be a non-transitory computer readable memory (e.g., random access memory), a cache, a drive (e.g., a hard drive), a flash drive, a database system, a cloud storage system, or another type of component or device capable of storing data. The data store 120 may also include multiple storage components (e.g., multiple drives or multiple databases) that may also span multiple computing devices (e.g., multiple server computers).

[0064] In some implementations, the online virtual experience ser er 102 can include a server having one or more computing devices (e.g., a cloud computing system, a rackmount server, a server computer, cluster of physical servers, etc.). In some implementations, the online virtual experience server 102 may be an independent system, may include multiple servers, or be part of another system or server.

[0065] In some implementations, the online virtual experience server 102 may include one or more computing devices (such as a rackmount server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a distributed computing system, a cloud computing system, etc.), data stores (e.g., hard disks, memories, databases), networks, software components, and / or hardware components that may be used to perform operations on the online virtual experience server 102 and to provide a user with access to online virtual experience server 102. The online virtual experience server 102 may also include a website (e.g., a web page) or application back-end software that may be used to provide a user with access to content provided by online virtualexperience server 102. For example, users may access online virtual experience server 102 using the virtual experience application 112 on client devices 110.

[0066] In some implementations, online virtual experience server 102 may be a type of social network providing connections between users or a type of user-generated content system that allows users (e.g., end-users or consumers) to communicate with other users on the online virtual experience server 102, where the communication may include voice chat (e.g., synchronous and / or asynchronous voice communication), video chat (e.g., synchronous and / or asynchronous video communication), or text chat (e.g., synchronous and / or asynchronous textbased communication). In some implementations of the disclosure, a “user” may be represented as a single individual. However, other implementations of the disclosure encompass a “user” (e.g., creating user) being an entity controlled by a set of users or an automated source. For example, a set of individual users federated as a community or group in a user-generated content system may be considered a “user.” In some contexts, a “user” may be a system administrator, a developer, a content provider, or other type of entity that may have privileges / capabilities that are different from those of an end user.

[0067] In some implementations, online virtual experience server 102 may be an online gaming server. For example, the virtual experience server 102 may provide single-player or multiplayer games to a community of users that may access or interact with games using client devices 110 via network 122. In some implementations, games (also referred to as “video game,” “online game,” or “virtual game” herein) may be two-dimensional (2D) games, three-dimensional (3D) games (e.g., 3D user-generated games), virtual reality (VR) games, or augmented reality (AR) games, for example. In some implementations, users may participate in gameplay with other users. In some implementations, a game may be played in real-time with other users of the game.

[0068] In some implementations, gameplay may refer to the interaction of one or more players using client devices (e.g., 110) within a game (e.g., game that is part of virtual experience 106) or the presentation of the interaction on a display or other output device (e.g., 114) of a client device 110.

[0069] In some implementations, a virtual experience 106 can include an electronic file that can be executed or loaded using software, firmware or hardware configured to present the game content (e.g., digital media item) to an entity7. In some implementations, a virtualexperience application 112 may be executed and a virtual experience 106 executed in connection with a virtual experience engine 104. In some implementations, a virtual experience 106 may have a common set of rules or common goal, and the environment of a virtual experience 106 shares the common set of rules or common goal. In some implementations, different games may have different rules or goals from one another.

[0070] In some implementations, virtual experience(s) may have one or more environments (also referred to as “gaming environments” or “virtual environments” herein) where multiple environments may be linked. An example of an environment may be a three-dimensional (3D) environment. The one or more environments of a virtual experience application 112 may be collectively referred to a “world” or “gaming world” or “virtual world” or “universe” herein. An example of a world may be a 3D world of a virtual experience 106. For example, a user may build a virtual environment that is linked to another virtual environment created by another user. A character of the virtual game may cross the virtual border to enter the adjacent virtual environment.

[0071] It may be noted that 3D environments or 3D worlds use graphics that use a three-dimensional representation of geometric data representative of game content (or at least present game content to appear as 3D content whether or not 3D representation of geometric data is used). 2D environments or 2D worlds use graphics that use two-dimensional representation of geometric data representative of game content.

[0072] In some implementations, the online virtual experience server 102 can host one or more virtual experiences 106 and can permit users to interact with the virtual experiences 106 using a virtual experience application 112 of client devices 110. Users of the online virtual experience server 102 may play, create, interact with, or build virtual experiences 106, communicate with other users, and / or create and build objects (e.g., also referred to as “item(s)” or “game objects” or “virtual game item(s)” herein) of virtual experiences 106. For example, in generating user-generated virtual items, users may create characters, decoration for the characters, one or more virtual environments for an interactive game, or build structures used in a game. In some implementations, users may buy, sell, or trade virtual game objects, such as in-platform currency (e.g.. virtual currency), with other users of the online virtual experience server 102. In some implementations, online virtual experience sen7er 102 may transmit game content to virtual experience applications (e.g., 112). In some implementations, game content (also referred to as “content” herein) may refer to any data or software instructions (e.g., gameobjects, game, user information, video, images, commands, media item, etc.) associated with online virtual experience server 102 or virtual experience applications. In some implementations, game objects (e.g., also referred to as ‘'item(s)” or ‘'objects” or ‘'virtual objects” or “virtual game item(s)” herein) may refer to objects that are used, created, shared or otherwise depicted in virtual experiences 106 of the online virtual experience server 102 or virtual experience applications 112 of the client devices 110. For example, game objects may include a part, model, character, accessories, tools, weapons, clothing, buildings, vehicles, currency, flora, fauna, components of the aforementioned (e.g., windows of a building), and so forth.

[0073] It may be noted that the online virtual experience server 102 hosting virtual experiences 106, is provided for purposes of illustration, rather than limitation. In some implementations, online virtual experience server 102 may host one or more media items that can include communication messages from one user to one or more other users. Media items can include, but are not limited to, digital video, digital movies, digital photos, digital music, audio content, melodies, website content, social media updates, electronic books, electronic magazines, digital newspapers, digital audio books, electronic journals, web blogs, real simple syndication (RSS) feeds, electronic comic books, software applications, etc. In some implementations, a media item may be an electronic file that can be executed or loaded using software, firmware or hardware configured to present the digital media item to an entity.

[0074] In some implementations, a virtual application 112 / 132 may be associated with a particular user or a particular group of users (e.g., a private game) or made widely available to users with access to the online virtual experience server 102 (e.g., a public game). In some implementations, where online virtual experience server 102 associates one or more virtual experiences 106 with a specific user or group of users, online virtual experience server 102 may associate the specific user(s) with a virtual experience 106 using user account information (e.g., a user account identifier such as username and password).

[0075] In some implementations, online virtual experience server 102 or client devices 110 may include a virtual experience engine 104 or virtual experience application 112. In some implementations, virtual experience engine 104 may be used for the development or execution of virtual experiences 106. For example, virtual experience engine 104 may include a rendering engine (“renderer”) for 2D, 3D, VR, or AR graphics, a physics engine, a collision detection engine (and collision response), sound engine, scripting functionality’, animation engine,artificial intelligence engine, networking functionality, streaming functionality, memory management functionality, threading functionality, scene graph functionality, or video support for cinematics, among other features. The components of the virtual experience engine 104 may generate commands that help compute and render the game (e.g., rendering commands, collision commands, physics commands, etc.) In some implementations, virtual experience applications 112 of client devices 110 may work independently, in collaboration with virtual experience engine 104 of online virtual experience server 102, or a combination of both.

[0076] In some implementations, both the online virtual experience server 102 and client devices 110 may execute a virtual experience engine and a virtual experience application (104 and 112, respectively). The online virtual experience server 102 using virtual experience engine 104 may perform some or all the virtual experience engine functions (e.g., generate physics commands, rendering commands, etc.), or offload some or all the virtual experience engine functions to virtual experience engine 104 of client device 110. In some implementations, each virtual application 112 / 132 may have a different ratio between the virtual experience engine functions that are performed on the online virtual experience server 102 and the virtual experience engine functions that are performed on the client devices 110. For example, the virtual experience engine 104 of the online virtual experience server 102 may be used to generate physics commands in cases where there is a collision between at least two virtual application objects, while the additional virtual experience engine functionality (e.g.. generate rendering commands) may be offloaded to the client device 110. In some implementations, the ratio of virtual experience engine functions performed on the online virtual experience server 102 and client device 110 may be changed (e.g., dynamically) based on gameplay conditions. For example, if the number of users participating in gameplay of a particular virtual application 106 exceeds a threshold number, the online virtual experience server 102 may perform one or more virtual experience engine functions that were previously performed by the client devices 110.

[0077] For example, users may be playing a virtual application 112 on client devices 110, and may send control instructions (e.g., user inputs, such as right, left, up, down, user election, or character position and velocity information, etc.) to the online virtual experience server 102. Subsequent to receiving control instructions from the client devices 110. the online virtual experience server 102 may send gameplay instructions (e.g., position and velocity information of the characters participating in the group gameplay or commands, such as renderingcommands. collision commands, etc.) to the client devices 110 based on control instructions. For instance, the online virtual experience server 102 may perform one or more logical operations (e.g., using virtual experience engine 104) on the control instructions to generate gameplay instruct! on(s) for the client devices 110. In other instances, online virtual experience server 102 may pass one or more or the control instructions from one client device 110 to other client devices (e.g., from client device 110a to client device 110b) participating in the virtual application 112. The client devices 110 may use the gameplay instructions and render the gameplay for presentation on the displays of client devices 110.

[0078] In some implementations, the control instructions may refer to instructions that are indicative of in-game actions of a user’s character. For example, control instructions may include user input to control the in-game action, such as right, left, up, down, user selection, g roscope position and orientation data, force sensor data, etc. The control instructions may include character position and velocity information. In some implementations, the control instructions are sent directly to the online virtual experience server 102. In other implementations, the control instructions may be sent from a client device 110 to another client device (e.g., from client device 110b to client device HOn), where the other client device generates gameplay instructions using the local virtual experience engine 104. The control instructions may include instructions to play a voice communication message or other sounds from another user on an audio device (e.g., speakers, headphones, etc.), for example voice communications or other sounds generated using the audio spatialization techniques as described herein.

[0079] In some implementations, gameplay instructions may refer to instructions that allow a client device 110 to render gameplay of a game, such as a multiplayer game. The gameplay instructions may include one or more of user input (e.g., control instructions), character position and velocity information, or commands (e.g., physics commands, rendering commands, collision commands, etc ).

[0080] In some implementations, the online virtual experience server 102 may store characters created by users in the data store 120. In some implementations, the online virtual experience server 102 maintains a character catalog and game catalog that may be presented to users. In some implementations, the game catalog includes images of virtual experiences stored on the online virtual experience server 102. In addition, a user may select a character (e.g., a character created by the user or other user) from the character catalog to participate in thechosen game. The character catalog includes images of characters stored on the online virtual experience server 102. In some implementations, one or more of the characters in the character catalog may have been created or customized by the user. In some implementations, the chosen character may have character settings defining one or more of the components of the character.

[0081] In some implementations, a user’s character can include a configuration of components, where the configuration and appearance of components and more generally the appearance of the character may be defined by character settings. In some implementations, the character settings of a user’s character may at least in part be chosen by the user. In other implementations, a user may choose a character with default character settings or character setting chosen by other users. For example, a user may choose a default character from a character catalog that has predefined character settings, and the user may further customize the default character by changing some of the character settings (e.g., adding a shirt with a customized logo). The character settings may be associated with a particular character by the online virtual experience server 102.

[0082] In some implementations, the virtual experience platform may support three-dimensional (3D) objects that are represented by a 3D model and includes a surface representation used to draw the character or object (also known as a skin or mesh) and a hierarchical set of interconnected bones (also known as a skeleton or rig). The ng may be utilized to animate the object and to simulate motion of the object. The 3D model may be represented as a data structure, and one or more parameters of the data structure may be modified to change various properties of the character (e.g., dimensions (height, width, girth, etc.); shape; movement style; number / type of parts; proportion, etc.).

[0083] In some implementations, the 3D model may include a 3D mesh. The 3D mesh may define a three-dimensional structure of the virtual 3D object. In some implementations, the 3D mesh may also define one or more surfaces of the 3D object. In some implementations, the 3D object may be a virtual avatar (e.g., a virtual character such as a humanoid character, an animalcharacter, a robot-character, etc.).

[0084] In some implementations, the mesh may be received (imported) in a FBX file format. The mesh file includes data that provides dimensional data about polygons that comprise the virtual 3D object and UV map data that describes how to attach portions of texture to various polygons that comprise the 3D object. In some implementations, the 3D object maycorrespond to an articulated object (e.g., such as a swivel chair, a lamp with a hinged shade or arms, a rubbish bin, a cabinet, etc.).

[0085] In some implementations, a platform may enable users to submit (upload) candidate 3D objects for utilization on the platform. A virtual experience development environment (developer tool) may be provided by the platform, in accordance with some implementations. The virtual experience development environment may provide a user interface that enables a developer user to design and / or create virtual experiences (e.g. games). The virtual experience development environment may be a client-based tool (e.g., downloaded and installed on a client device, and operated from the client device), a server-based tool (e.g., installed and executed at a server that is remote from the client device, and accessed and operated by the client device), or a combination of both client-based and sen ice-based elements.

[0086] The virtual experience development environment may be operated by a developer of a virtual experience (e.g., a game developer or any other person who seeks to create a virtual experience that may be published by an online virtual experience platform and utilized by others). The user interface of the virtual experience development environment may be rendered on a display screen of a client device (e.g., such as a developer device 130 described with reference to FIG. 1), so as to enable the creator / developer to interact with the development environment using actions such as typing, highlighting, selecting, drag and drop, clicking, and so forth via a mouse, keyboard, or other input device configured to communicate with the user interface. The user interface may include a menu bar, a tool bar, a w orkspace pane, and a plurality of secondary panes. Depending on the particular implementation, the user interface may include alternative or additional elements, arrangements, operational features, etc. of the virtual experience development environment than what is shown and described herein.

[0087] A developer user (creator) may utilize the virtual experience development environment to create virtual experiences. As part of the development process, the developer / creator may upload various types of digital content such as object files (meshes), image files, audio files, short videos, etc., to enhance the virtual experience.

[0088] In implementations where the 3D object is an accessory, data indicative of use of the object in a virtual experience may also be received. For example, a "shoe" object may include annotations indicating that the object can be depicted as being worn on the feet of avirtual humanoid character, while a ‘‘shirt'’ object may include annotations that it may be depicted as being worn on the torso of a virtual humanoid character.

[0089] In some implementations, the 3D model may further include texture information associated with the 3D object. For example, texture information may indicate color and / or pattern of an outer surface of the 3D object. The texture information may enable varying degrees of transparency, reflectiveness, degrees of diffusiveness, material properties, and refractory behavior of the textures and meshes associated with the 3D object. Examples of textures include plastic, cloth, grass, a pane of light blue glass, ice, water, concrete, brick, carpet, wood, etc.

[0090] In some implementations, the client device(s) 110 may each include computing devices such as personal computers (PCs), mobile devices (e.g., laptops, mobile phones, smart phones, tablet computers, or netbook computers), network-connected televisions, gaming consoles, etc. In some implementations, a client device 110 may also be referred to as a “client device.” In some implementations, one or more client devices 110 may connect to the online virtual experience server 102 at any given moment. It may be noted that the number of client devices 110 is provided as illustration. In some implementations, any number of client devices 110 may be used.

[0091] In some implementations, each client device 110 may include an instance of the virtual experience application 112, respectively. In one implementation, the virtual experience application 112 may permit users to use and interact with online virtual experience server 102, such as control a virtual character in a virtual game hosted by online virtual experience server 102, or view or upload content, such as virtual experiences 106, images, video items, web pages, documents, and so forth. In one example, the virtual experience application may be a web application (e.g., an application that operates in conjunction with a w eb browser) that can access, retrieve, present, or navigate content (e.g., virtual character in a virtual environment, etc.) served by a w eb server. In another example, the virtual experience application may be a native application (e.g., a mobile application, app, or a gaming program) that is installed and executes local to client device 110 and allows users to interact with online virtual experience server 102. The virtual experience application may render, display, or present the content (e.g., a web page, a media viewer) to a user. In an implementation, the virtual experience application may also include an embedded media player (e.g., a Flash® player) that is embedded in a web page.

[0092] In some implementations, the virtual experience application may include an audio engine 116 that is installed on the client device, and which enables the playback of sounds on the client device. In some implementations, audio engine 116 may act cooperatively with an audio engine that is installed on the sound server.

[0093] According to aspects of the disclosure, the virtual experience application may be an online virtual experience server application for users to build, create, edit, and upload content to the online virtual experience server 102 as well as interact with online virtual experience server 102 (e.g., participate in virtual experiences 106 hosted by online virtual experience server 102). As such, the virtual experience application may be provided to the client device(s) 110 by the online virtual experience server 102. In another example, the virtual experience application may be an application that is downloaded from a server.

[0094] In some implementations, each developer device 130 may include an instance of the virtual experience application 132, respectively. In one implementation, the virtual experience application 132 may permit a developer user(s) to use and interact with online virtual experience sen- er 102, such as control a virtual character in a virtual game hosted by online virtual experience ser er 102, or view or upload content, such as virtual experiences 106, images, video items, web pages, documents, and so forth. In one example, the virtual experience application may be a web application (e.g., an application that operates in conjunction with a web browser) that can access, retrieve, present, or navigate content (e.g., virtual character in a virtual environment, etc.) served by a web server. In another example, the virtual experience application may be a native application (e.g., a mobile application, app, or a virtual experience program) that is installed and executes local to developer device 130 and allows users to interact with online virtual experience server 102. The virtual experience application may render, display, or present the content (e.g., a web page, a media viewer) to a user. In an implementation, the virtual experience application may also include an embedded media player (e.g., a Flash® player) that is embedded in a web page.

[0095] According to aspects of the disclosure, the virtual experience application 132 may be an online virtual experience server application for users to build, create, edit, upload content to the online virtual experience server 102 as well as interact with online virtual experience server 102 (e.g., provide and / or play virtual experiences 106 hosted by online virtual experience server 102). As such, the virtual experience application may be provided to the client device(s) 110 by the online virtual experience server 102. In another example, the virtualexperience application 132 may be an application that is downloaded from a sen- er. Virtual experience application 132 may be configured to interact with online virtual experience server 102 and obtain access to user credentials, user currency, etc. for one or more virtual applications 112 / 132 developed, hosted, or provided by a virtual experience application developer.

[0096] In some implementations, a user may login to online virtual experience server 102 via the virtual experience application. The user may access a user account by providing user account information (e.g., username and password) where the user account is associated with one or more characters available to participate in one or more virtual experiences 106 of online virtual experience server 102. In some implementations, with appropriate credentials, a virtual experience application developer may obtain access to virtual experience application objects, such as in-platform currency (e.g., virtual currency), avatars, special powers, accessories, which are owned by or associated with other users.

[0097] In general, functions described in one implementation as being performed by the online virtual experience server 102 can also be performed by the client device(s) 110, a server, and / or other device(s) in the environment of FIG. 1 in other implementations if appropriate. In addition, the functionality attributed to a particular component can be performed by different or multiple components operating together. The online virtual experience server 102 can also be accessed as a sendee provided to other systems or devices through appropriate application programming interfaces (APIs) and thus is not limited to use in w-ebsites.

[0098] In some implementations, online virtual experience server 102 may include a graphics engine 108. In some implementations, the graphics engine 108 may be a system, application, or module that permits the online virtual experience server 102 to provide graphics and animation capability. In some implementations, the graphics engine 108, and / or content management ser er 140 may perform one or more of the operations described below in connection with the flowcharts and workflows shown in FIG. 7 or otherwise described herein.

[0099] FIG. 2 is a diagram of a first example articulated object 200 and its corresponding typed graph 201, in accordance with some implementations. Referring to FIG. 2, the first articulated object 200 in this non-limiting example is a chair with an upper portion 210 (e.g., seat, arms, and back), a base 220, casters 230, and wheels 240. As used herein, a typed graph may also be referred to as a kinematic graph.

[0100] As shown on the right side of FIG. 2, the first articulated object 200 may be represented as a typed graph 201 G = (V, E. T). where V denotes terminal nodes 202 (dashed circles) that each correspond to a different part geometry of the first articulated object 200 (a chair in this example), E denotes edges 204 (squares) that each correspond to a different mechanical joint of the first articulated object 200, and T is the discrete type set that associates each terminal node and edge with a type and denotes the continuous parameters. For the edge type, joint types associated with four mechanical joints may be used. These edge types may include, for example, revolute, prismatic, screw, and free joints.

[0101] The node type for the first articulated object 200 may be determined based on part geometry labels accessible from the input training data. For instance, a terminal node 202 for the chair depicted in FIG. 2 may have four possible types: upper portion 210 (e.g., first terminal node 202a), base 220 (e.g., second terminal node 202b), caster 230 (e.g., third terminal node 202c), and wheel 240 (e.g., fourth terminal node 202d).

[0102] Each of the chair’s part geometries may have its own terminal node 202 in the typed graph 201. For instance, the chair has one upper portion 210 and one base 220; therefore, the typed graph 201 depicted in FIG. 2 has one first terminal node 202a and one second terminal node 202b. The chair’s base 220 has five legs each with a corresponding caster 230 coupled therewith; thus, the typed graph 201 depicted in FIG. 2 has five third terminal nodes 202c, each respectively corresponding to one of the five casters 230. Each caster 230 has two wheels 240 coupled therewith; therefore, the ty ped graph 201 depicted in FIG. 2 has ten fourth terminal nodes 202d, each respectively corresponding to one of the ten wheels 240.

[0103] In the typed graph 201, a first edge 204a represents a first joint that couples the upper portion 210 to the base 220, a second edge 204b represents a second joint that couples the base 220 to a corresponding caster 230, and a third edge 204c represents a third joint that couples a wheel 240 to its corresponding caster 230.

[0104] FIG. 3 is a diagram of a second example articulated obj ect 300 and its corresponding typed graph 301, in accordance with some implementations.

[0105] As shown on the left side of FIG. 3, the second articulated object 300 in this example is a floor lamp. The lamp includes a shade 310, a first arm 320, a second arm 330, and a base 340. The typed graph 301, which includes terminal nodes and edges corresponding to the various components of the lamp, is shown on the right side of FIG. 3.

[0106] The node types for the lamp may be determined based on part geometry labels accessible from the input training data. For instance, a terminal node 302 for the lamp can have four possible types: shade 310 (e.g., first terminal node 302a), first arm 320 (e.g., second terminal node 302b), second arm 330 (e.g., third terminal node 302c), and base 340 (e.g., fourth terminal node 302d).

[0107] In the typed graph 301, a first edge 304a represents a first joint that couples the shade 310 and the first arm 320, a second edge 304b represents a second joint that couples the first arm 320 and the second arm 330, and a third edge 304c represents a third joint that couples the second arm 330 and the base 340. Here again, for the edge typejoint types associated with four mechanical joints may be used. These edge types may include, for example, revolute, prismatic, screw, and free joints.

[0108] FIG. 4A is a diagram of example kinematic graph grammar 400 for generating an articulated object using graph grammar, in accordance with some implementations. In the nonlimiting example of FIG. 4A, the kinematic graph grammar 400 may be used to generate a rubbish bin with two wheels and a hinged lid, which is depicted in FIG. 4B.

[0109] Kinematic graph grammar is a procedural modeling tool that may be used to generate kinematic graphs with all the discrete parameters except the continuous parameters. In shape synthesis, the kinematic graphs are generated to locally resemble the input training examples. Kinematic graph grammar is both data-efficient and provides the property' of local connectivity. Local connectivity represents that two kinematic terminal nodes of different types can be connected if such a connection exists in the original training dataset. For example, a rubbish bin’s wheel can be connected to its base but not its lid.

[0110] Referring to FIG. 4A. the kinematic graph grammar 400 Q = (Z. N, J3) in the present example includes a set of terminal nodes (X) 402, a set of non-terminal nodes (J?) 406, and a set of production rules (?) 408. The terminal nodes 402 represent the typed nodes of kinematic graphs, while the non-terminal nodes 406 represent external nodes, which are absent from the final graph. Kinematic graph grammar 400 generates graphs starting from a single external node and iteratively applying each production rule from the set of production rules (?) 408.[OHl] The set of production rules (?) 408 may include rules in the form of left-hand side (LHS) —> right-hand side (RHS), where both LHS and RHS are subgraphs. In the non-limitingexample of FIG. 4A, the set of production rules (?) 408 includes a total of four production rules pl, p2, p3, and p4.

[0112] At each production rule iteration, a subgraph of the current graph that matches the LHS of a production rule is replaced by the RHS of the corresponding rule. This process continues until the graph does not include any non-terminal nodes. An example of a production sequence that generates a tree-structured kinematic graph using the kinematic graph grammar 400 and its set of production rules (?) 408 is described below with reference to FIG. 4B.

[0113] FIG. 4B is a diagram of an example production sequence 401 for generating an articulated object using the kinematic graph grammar 400 of FIG. 4A, in accordance with some implementations.

[0114] Referring to FIG. 4B. the iterative process of generating a tree-structured kinematic graph corresponding to a rubbish bin 450, which is made up of a lid 410, a bin 420, a right wheel 430, and a left wheel 440, using the kinematic graph grammar 400 of FIG. 4 A, is shown.

[0115] The iterative process begins with the first production rule pl. Applying the first production rule pl. the non-terminal node 406 (LHS of pl in FIG. 4A) is replaced with a first terminal node 402a (corresponding to the bin 420) and three non-terminal nodes 406 coupled thereto (RHS of pl in FIG. 4A). In the second, third, and fourth production rules p2, p3, and p4, the three non-terminal nodes 406 are replaced by different terminal nodes and corresponding edges to generate the kinematic graph corresponding to the rubbish bin 450.

[0116] For example, applying the second production rule p2, one of the non-terminal node 406 (LHS of p2 in FIG. 4A) is replaced with a second terminal node 402b (corresponding to the lid 410) and a first edge 404a (RHS of p2 in FIG. 4A). The first edge 404a represents the mechanical joint that couples the lid 410 and the bin 420.

[0117] Applying the third production rule p3, another non-terminal node 406 (LHS of p3 in FIG. 4A) is replaced with a third terminal node 402c (corresponding to the right wheel 430) and a second edge 404b (RHS of p3 in FIG. 4A). The second edge 404b represents the mechanical joint that couples the right wheel 430 and the bin 420.

[0118] Then, to apply the fourth production rule p4, the remaining non-terminal node 406 (LHS of p4 in FIG. 4A) is replaced with a fourth terminal node 402d (corresponding to the leftwheel 440) and a third edge 404c (RHS of p4 in FIG. 4A). The third edge 404c represents the mechanical joint that couples the left wheel 440 and the bin 420.

[0119] In this way, the kinematic graph for the rubbish bin 450 may be generated using the kinematic graph grammar 400 of FIG. 4A. However, different kinematic graph grammar containing different numbers of production rules may be used to generate kinematic graphs for different types of articulated objects without departing from the scope of the present disclosure.

[0120] FIG. 5 is a diagram of an example decomposition 500 of kinematic graphs into corresponding sets of primitives, in accordance with some implementations.

[0121] Referring to FIG. 5, learning kinematic graph grammar may be the reverse process of generating the graph grammar. For instance, given an input training dataset of one or more kinematic graphs (e.g.. first input kinematic graph 502a, second input kinematic graph 502b, etc.), the present technique(s) derive a set of production rules that may generate all the kinematic graphs in the input set while satisfying the local connectivity constraint. Satisfying the local connectivity constraint enables each generated shape to resemble the local geometry of an input training dataset.

[0122] To learn kinematic graph grammar with a local connectivity constraint, the present technique(s) may use a bottom-up method. For instance, the input training dataset of one or more kinematic graphs are each decomposed into a corresponding set of primitives by breaking all the edges of the input kinematic graphs and creating two external nodes for each edge. For example, the first input kinematic graph 502a may be decomposed into first set of primitives 504a, and the second input kinematic graph 502b may be decomposed into a second set of primitives 504b.

[0123] Based on decomposed primitives, a graph hierarchy may be generated by enumerating all possible ways to combine primitives, as described below with reference to FIG. 6.

[0124] FIG. 6 is a diagram of an example graph hierarchy 600 enumerating possible combinations of primitives to generate various articulated objects, in accordance with some implementations.

[0125] Referring to FIG. 6, the graph hierarchy 600 may be generated by enumerating all possible ways to combine decomposed primitives. Specifically, the graph hierarchy 600 is made up of levels, with each level containing a set of graphs. Level 0 contains a complete set of all primitives. Level t+1 contains all unique graphs formed by attaching each primitive to all the graphs in level t. This graph hierarchy may be a directed acyclic graph, where the leaves are complete graphs and the intermediate nodes are incomplete graphs containing external nodes. The set of production rules for the graph grammar are obtained between each graphhierarchy level pair, where each rule converts a graph from level t to another graph in level t+1.

[0126] For instance, the graph hierarchy 600 includes level 0 602a, level 1 602b, level 2 602c. and level 3 602d. Level 0602a includes a set of primitives 604a. Level 1 602b includes a first set of graphs 604b formed by attaching each primitive in the set of primitives 604a. Level 2 602c includes a second set of graphs 604c formed by attaching each primitive in the set of primitives to the first set of graphs 604b in level 1. Level 3 602d includes a third set of graphs 604d formed by attaching the primitives to the second set of graphs 604c in level 2.

[0127] In the non-limiting example of FIG. 6, the graph hierarchy 600 includes a first graph-hierarchy level pair 606a (e.g., level 0602a and level 1 602b), a second graph-hierarchy level pair 606b (e.g., level 1 602b and level 2602c), and a third graph-hierarchy level pair 606c (e.g., level 2 602c and level 3 602d).

[0128] Thus, in the present example, the set of production rules for the graph grammar are obtained by converting 1) the primitives 604a in level 0602a to the first set of graphs 604b in level 1, 2) the first set of graphs 604b in level I 602b to the second set of graphs 604c in level 2602c, and 3) the second set of graphs 604b in level 2 602c to the third set of graphs 604d in level 3 602d. Using the generated set of production rules, various articulated objects, such as different types of cabinets, rubbish bins, etc., may be generated.

[0129] There are two possible challenges in achieving a complete kinematic graph generative model: (1) building the graph hierarchy has combinatorial complexity, making it infeasible to enumerate all possible graphs as the hierarchy size grows exponentially; and (2) the graph grammar deals with discrete graph structures, not the continuous parameters associated with the graph, such as bounding boxes for nodes and joint parameters for edges.

[0130] To address these challenges, the present disclosure uses a generative flow network. The present generative flow network is a trained stochastic policy or generative model thatsamples graphs through a sequence of production steps, with probabilities proportional to a reward function that is non-negative and integrable. For instance, the reward function may be defined to mimic the training dataset so that a trained generative flow network will generate new graphs that follow the training dataset distribution but extrapolate beyond it. Training the present generative flow network may not use the full construction of the graph hierarchy but instead uses sampled generation sequences from the root to the leaves within the hierarchy. This approach eliminates or otherwise reduces use of the explicit construction of the entire graph hierarchy. Additionally, the present generative flow network is trained to infer the continuous parameters associated with graph structures. These continuous parameters are the bounding boxes for nodes and joint axes and positions for edges in the graph. Together with the learning of the graph grammar, the present generative flow network serves as an end-to-end solution for generating diverse kinematic graphs.

[0131] FIG. 7 is a flowchart of an example method 700 of articulated-shape generation, in accordance with some implementations.

[0132] In some implementations, method 700 can be implemented, for example, on an online virtual experience server 102 (e.g., by a virtual-experience coordinator) described with reference to FIG. 1. In some implementations, some or all of the method 700 can be implemented on one or more client devices 90 as shown in FIG. 1, on one or more developer devices 130, or on one or more online virtual experience server(s) 102, and / or on a combination of developer device(s), server device(s) and client device(s), or on other devices that may be usable in the environment 100 of FIG. 1. In described examples, the implementing system includes one or more digital processors or processing circuitry ("‘processors’"), and one or more storage devices (e.g., a data store 1 8 or other storage). In some implementations, different components of one or more servers and / or clients can perform different blocks or other parts of the method 700. In some examples, a first device is described as performing blocks of method 700. Some implementations can have one or more blocks of method 700 performed by one or more other devices (e.g., other client devices or server devices) that can send results or data to the first device.

[0133] In some implementations, method 700, or portions of the methods, can be initiated automatically by a system. In some implementations, the implementing system is a first device. For example, the method (or portions thereof) can be periodically performed, or performed based on one or more particular events or conditions (e.g., upon a user request and / or one ormore other conditions occurring which can be specified in settings read by the methods). In the method 100 of FIG. 7 and / or in other methods described herein, some operations may be optional, omitted, modified, combined, supplemented with other operations, performed in a different order than depicted, performed in parallel or in sequence, etc.

[0134] Referring to FIG. 7, method 700 may begin at block 702. At block 702, a first kinematic graph that represents a first articulated object that includes a first plurality of parts interconnected by one or more first joints may be obtained. The first kinematic graph includes a plurality of nodes corresponding to the first plurality7of parts and one or more edges corresponding to the one or more first joints.

[0135] In some implementations, the first kinematic graph further includes a type set that associates each node of the plurality' of nodes with a node type and each edge of the one or more edges with an edge type, respectively. In some implementations, the node type is determined based on a respective geometry label of corresponding to the first articulated objected in a training dataset. In some implementations, the edge type comprises revolute, prismatic, screw, or free. Block 702 may be followed by block 704.

[0136] At block 704, a set of primitive structures may be generated by decomposing the first kinematic graph. Each primitive structure in the set of primitive structures includes a respective terminal node corresponding to a respective node of the plurality' of nodes and at least one respective non-terminal node connected to the respective terminal node. A nonterminal node corresponds to a node-edge connection.

[0137] In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes determining the one or more edges based on the first kinematic graph. In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes segmenting each of the one or more edges into two respective parts. In some implementations, generating the set of primitive structures by decomposing the first kinematic graph includes generating a non-terminal node at a respective segmentation point of each of the two respective parts for each of the one or more edges. Block 704 may be followed by block 706.

[0138] At block 706, a graph hierarchy may be generated by enumerating one or more valid combinations of the set of primitives.

[0139] In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating a first level of the graph hierarchy based on the set of primitives. In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating a second level of the graph hierarchy by attaching each primitive in the set of primitives to all other primitives in the set of primitives to form a first set of graphs.

[0140] In some implementations, generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives includes generating a third level of the graph hierarchy by attaching each primitive in the set of primitives to each graph in the first set of graphs to form a second set of graphs. Block 706 may be followed by block 708.

[0141] At block 708, production rules may be determined based on the graph hierarchy.

[0142] In some implementations, determining the production rules based on the graph hierarchy includes determining the production rules for each graph-hierarchy level pair that convert a set of graphs from level t to another graph in level t +1.

[0143] In some implementations, the production rules define local connectivity constraints as only those node-type connections identified in a training data set. Block 708 may be followed by block 710.

[0144] At block 710, kinematic graph grammar for generating discrete kinematic graph structures based on the set of primitive structures and the production rules may be determined. Block 710 may be followed by block 712.

[0145] At block 712, continuous parameters may be determined based on the production rules using a generative flow network. The continuous parameters include a plurality of bounding boxes each corresponding to a respective node of the plurality7of nodes and one or more joint parameters each corresponding to a respective edge of the one or more edges.

[0146] In some implementations, determining the continuous parameters based on the production rules using the generative flow network includes determining the plurality of bounding boxes each corresponding to the respective node of the plurality of nodes based on a respective center position and a respective spatial extent of the respective node based on the first kinematic graph. In some implementations, determining the continuous parameters basedon the production rules using the generative flow network includes determining the one or more joint parameters each corresponding to the respective edge of the one or more edges based on a respective joint axis and a respective joint position of a corresponding edge based on the first kinematic graph.

[0147] In some implementations, the continuous parameters include a plurality of bounding boxes for the plurality of nodes and one or more joint parameters for the one or more edges. Block 712 may be followed by block 714.

[0148] At block 714. a second kinematic graph, different from the first kinematic graph, to represent a second articulated object, different from the first articulated object, may be generated based on the kinematic graph grammar and the continuous parameters. The second articulated object includes a second plurality of parts interconnected by one or more second joints. Block 714 may be followed by block 716.

[0149] At block 716, the second plurality of parts interconnected by the one or more second joints of the second articulated object may be animated based on the second kinematic graph.

[0150] By way of example and not limitation, using operations 702-716, a first example articulated object 200 (e.g., an office chair) and its corresponding typed graph 201 (e.g., a first kinematic graph) may be used to generate kinematic graph grammar, which is used to generate typed graph 301 (e.g., a second kinematic graph) corresponding to a second example articulated object 300 (a floor lamp).

[0151] Hereinafter, a more detailed description of various computing devices that may be used to implement different devices and / or components illustrated in FIG. 1 is provided with reference to FIG. 8.

[0152] FIG. 8 is a block diagram of an example computing device 800, which may be used to implement one or more features described herein, in accordance with some implementations. In one example, computing device 800 may be used to implement a computer device, (e.g., 102, 110, or 130 ofFIG. 1). and perform appropriate operations as described herein. Computing device 800 can be any suitable computer system, server, or other electronic or hardware device. For example, the computing device 800 can be a mainframe computer, desktop computer, workstation, portable computer, or electronic device (portable device, mobile device, cell phone, smart phone, tablet computer, television. TV set top box, personal digital assistant(PDA), media player, game device, wearable device, etc ). In some implementations, device 800 includes a processor 802, a memory 804, input / output (I / O) interface 806, and audio / video input / output devices 814 (e.g., display screen, touchscreen, display goggles or glasses, audio speakers, headphones, microphone, etc.).

[0153] Processor 802 can be one or more processors and / or processing circuits to execute program code and control basic operations of the computing device 800. A ‘‘processor” includes any suitable hardware and / or software system, mechanism or component that processes data, signals or other information. A processor may include a system with a general-purpose central processing unit (CPU), multiple processing units, dedicated circuitry for achieving functionality, or other systems. Processing need not be limited to a particular geographic location or have temporal limitations. For example, a processor may perform its functions in “real-time,” “offline,” in a “batch mode,” etc. Portions of processing may be performed at different times and at different locations, by different (or the same) processing systems. A computer may be any processor in communication with a memory.

[0154] Memory 804 is typically provided in computing device 800 for access by the processor 802, and may be any suitable processor-readable storage medium (e.g., random access memory’ (RAM), read-only memory (ROM), electrical erasable read-only memory (EEPROM), flash memory, etc.), suitable for storing instructions for execution by the processor, and located separate from processor 802 and / or integrated therewith. Memory 804 can store software operating on the computing device 800 by the processor 802, including an operating system 808, software application 810, and associated database 812. In some implementations, the software application 810 can include instructions that enable processor 802 to perform or control performance of the functions / operations described herein. Software application 810 may include some or all of the functionality usable to generate a set of production rules, kinematic graphs, kinematic graph grammar, articulated objects, etc. In some implementations, one or more portions of software application 810 may be implemented in dedicated hardware such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), a machine learning processor, etc. In some implementations, one or more portions of software application 810 may be implemented in general purpose processors, such as a central processing unit (CPU) or a graphics processing unit (GPU). In various implementations, suitable combinations ofdedicated and / or general purpose processing hardware may be used to implement software application 810.

[0155] For example, software application 810 stored in memory 804 can include instructions for obtaining kinematic graphs and / or production rules for generating articulated virtual objects, and / or other functionality or software such as the virtual experience engine 104, graphics engine 108, 144. The software application 810 and / or other executable computer-readable instructions stored in memory 8904 can also be used to implement the virtual experience application 112, the virtual experience engine 104, the virtual experience 106, and / or other components depicted in and / or otherwise usable for the environment of FIG. 1. Any of software in memory 804 can alternatively or additionally be stored on any other suitable storage location or computer-readable medium. In addition, memory 804 (and / or other connected storage device(s)) can store instructions and data used in the features described herein. Memory' 804 and any other ty pe of storage (magnetic disk, optical disk, magnetic tape, or other tangible media) can be considered ‘“storage” or “storage devices.”

[0156] I / O interface 806 (which can correspond to the I / O interface 114 of FIG. 1) can provide functions to enable interfacing the computing device 800 with other systems and devices. For example, network communication devices, storage devices (e.g., memory and / or data store 120), and input / output devices can communicate via interface 806. In some implementations, the I / O interface can connect to interface devices including input devices (keyboard, pointing device, touchscreen, microphone, camera, scanner, etc.) and / or output devices (display device, speaker devices, printer, motor, etc.).

[0157] For ease of illustration, FIG. 8 shows one block for each of processor 802, memory 804, I / O interface 806, operating system 808, software application 810, and database 812. These blocks may represent one or more processors or processing circuitries, operating systems, memories. I / O interfaces, applications, and / or software modules. In other implementations, computing device 800 may not have all of the components shown and / or may have other elements including other types of elements instead of, or in addition to, those shown herein. While the online virtual experience server 102 are described as performing operations as described in some implementations herein, any suitable component or combination of components of online virtual experience server 102, or similar system, or any suitable processor or processors associated with such a system, may perform the operations described.

[0158] A user device can also implement and / or be used with features described herein. Example user devices can be computer devices including some similar components as the computing device 800 (e.g., processor(s) 802, memory 804, and I / O interface 806). An operating system, software and applications suitable for the client device can be provided in memory and used by the processor. The I / O interface for a client device can be connected to network communication devices, as well as to input and output devices (e.g., a microphone for capturing sound, a camera for capturing images or video, audio speaker devices for outputting sound, a display device for outputting images or video, or other output devices). A display device within the audio / video input / output devices 814, for example, can be connected to (or included in) the computing device 800 to display images pre- and post-processing as described herein, where such display device can include any suitable display device (e.g., an LCD, LED, or plasma display screen, CRT, television, monitor, touchscreen, 3-D display screen, projector, or other visual display device). Some implementations can provide an audio output device (e.g., voice output or synthesis that speaks text).

[0159] The methods, blocks, and / or operations described herein can be performed in a different order than shown or described, and / or performed simultaneously (partially or completely) with other blocks or operations, where appropriate. Some blocks or operations can be performed for one portion of data and later performed again (e.g., for another portion of data). Not all of the described blocks and operations need be performed in various implementations. In some implementations, blocks and operations can be performed multiple times, in a different order, and / or at different times in the methods.

[0160] In some implementations, some or all of the methods can be implemented on a system such as one or more client devices. In some implementations, one or more methods described herein can be implemented, for example, on a server system, and / or on both a server system and a client system. In some implementations, different components of one or more servers and / or clients can perform different blocks, operations, or other parts of the methods.

[0161] One or more methods described herein (e.g., method 700) can be implemented by computer program instructions or code, which can be executed on a computer. For example, the code can be implemented by one or more digital processors (e.g., microprocessors or other processing circuitry), and can be stored on a computer program product including a non-transitory computer readable medium (e.g., storage medium), for example, a magnetic, optical, electromagnetic, or semiconductor storage medium, including semiconductor or solid statememory, magnetic tape, a removable computer diskette, a random access memory' (RAM), a read-only memory (ROM), flash memory, a rigid magnetic disk, an optical disk, a solid-state memory drive, etc. The program instructions can also be contained in, and provided as, an electronic signal, for example in the form of software as a service (SaaS) delivered from a sen?er (e.g., a distributed system and / or a cloud computing system). Alternatively, one or more methods can be implemented in hardware (logic gates, etc.), or in a combination of hardware and software. Example hardware can be programmable processors (e.g. field-programmable gate array (FPGA), complex programmable logic device), general purpose processors, graphics processors, application specific integrated circuits (ASICs), and the like). One or more methods can be performed as part of or component of an application running on the system, or as an application or software running in conjunction with other applications and operating system.

[0162] One or more methods described herein can be run in a standalone program that can be run on any ty pe of computing device, a program run on a web browser, a mobile application (“app”) executing on a mobile computing device (e.g., cell phone, smart phone, tablet computer, wearable device (wristwatch, armband, jewelry, headwear, goggles, glasses, etc.), laptop computer, etc.). In one example, a client / server architecture can be used, e.g., a mobile computing device (as a client device) sends user input data to a server device and receives from the server the live feedback data for output (e.g., for display). In another example, computations can be split between the mobile computing device and one or more server devices.

[0163] Although the description has been described with respect to particular implementations thereof, these particular implementations are merely illustrative, and not restrictive. Concepts illustrated in the examples may be applied to other examples and implementations.

Claims

CLAIMSWhat is claimed is:

1. A computer-implemented method, comprising:obtaining, by a processor, a first kinematic graph that represents a first articulated object that includes a first plurality of parts interconnected by one or more first joints, wherein the first kinematic graph includes a plurality' of nodes corresponding to the first plurality of parts and one or more edges corresponding to the one or more first joints;generating, by the processor, a set of primitive structures by decomposing the first kinematic graph, wherein each primitive structure in the set of primitive structures includes a respective terminal node corresponding to a respective node of the plurality of nodes and at least one respective non-terminal node connected to the respective terminal node, wherein a non-terminal node corresponds to a node-edge connection;generating, by the processor, a graph hierarchy by enumerating one or more valid combinations of the set of primitives;determining, by the processor, production rules based on the graph hierarchy; determining, by the processor, kinematic graph grammar for generating discrete kinematic graph structures based on the set of primitive structures and the production rules;determining, by the processor, continuous parameters based on the production rules using a generative flow' network, wherein the continuous parameters include a plurality of bounding boxes each corresponding to a respective node of the plurality of nodes and one or more joint parameters each corresponding to a respective edge of the one or more edges; generating, by the processor, a second kinematic graph, different from the first kinematic graph, to represent a second articulated obj ect, different from the first articulated object, based on the kinematic graph grammar and the continuous parameters, wherein the second articulated object includes a second plurality of parts interconnected by one or more second joints; andanimating, by the processor, the second plurality' of parts interconnected by the one or more second joints of the second articulated object based on the second kinematic graph.

2. The computer-implemented method of claim 1, wherein generating the set of primitive structures by decomposing the first kinematic graph comprises:determining, by the processor, the one or more edges based on the first kinematic graph;segmenting, by the processor, each of the one or more edges into two respective parts; andgenerating, by the processor, a non-terminal node at a respective segmentation point of each of the two respective parts for each of the one or more edges.

3. The computer-implemented method of claim 1 , wherein generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives comprises:generating, by the processor, a first level of the graph hierarchy based on the set of primitives; andgenerating, by the processor, a second level of the graph hierarchy by attaching each primitive in the set of primitives to all other primitives in the set of primitives to form a first set of graphs.

4. The computer-implemented method of claim 3, wherein generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives comprises:generating, by the processor, a third level of the graph hierarchy by attaching each primitive in the set of primitives to each graph in the first set of graphs to form a second set of graphs.

5. The computer-implemented method of claim 1, wherein determining the production rules based on the graph hierarchy comprises:determining, by the processor, the production rules for each graph-hierarchy level pair that convert a set of graphs from level t to another graph in level t +1.

6. The computer-implemented method of claim 1, wherein determining the continuous parameters based on the production rules using the generative flow network comprises: determining, by the processor, the plurality of bounding boxes each corresponding to the respective node of the plurality of nodes based on a respective center position and a respective spatial extent of the respective node based on the first kinematic graph; and determining, by the processor, the one or more joint parameters each corresponding to the respective edge of the one or more edges based on a respective joint axis and a respective joint position of a corresponding edge based on the first kinematic graph.

7. The computer-implemented method of claim 1, wherein:the first kinematic graph further includes a type set that associates each node of the plurality of nodes with a node type and each edge of the one or more edges with an edge ty pe, respectively, andthe continuous parameters comprise a plurality of bounding boxes for the plurality of nodes and one or more joint parameters for the one or more edges.

8. The computer-implemented method of claim 7, wherein the node ty pe is determined based on a respective geometry' label of corresponding to the first articulated objected in a training dataset.

9. The computer-implemented method of claim 7, wherein the edge t pe comprises revolute, prismatic, screw, or free.

10. The computer-implemented method of claim 1, wherein the production rules define local connectivity constraints as only those node-type connections identified in a training data set.

11. A non-transitory computer-readable medium with instructions stored thereon that, when executed by one or more hardware processors, cause the one or more hardware processors to perform or control performance of operations comprising:obtaining a first kinematic graph that represents a first articulated object that includes a plurality of parts interconnected by one or more joints, yvherein the first kinematic graph includes a plurality of nodes corresponding to the plurality of parts and one or more edges corresponding to the one or more joints;generating a set of primitive structures by decomposing the first kinematic graph, wherein each primitive structure in the set of primitive structures includes a respective terminal node corresponding to a respective node of the plurality of nodes and at least one respective non-terminal node connected to the respective terminal node, wherein a nonterminal node corresponds to a node-edge connection;generating a graph hierarchy by enumerating one or more valid combinations of the set of primitives;determining production rules based on the graph hierarchy;determining kinematic graph grammar for generating discrete kinematic graph structures based on the set of primitive structures and the production rules;determining continuous parameters based on the production rules using a generative flow network, wherein the continuous parameters include a plurality of bounding boxes each corresponding to a respective node of the plurality' of nodes and one or more joint parameters each corresponding to a respective edge of the one or more edges; andgenerating a second kinematic graph, different from the first kinematic graph, to represent a second articulated object, different from the first articulated object, based on the kinematic graph grammar and the continuous parameters.

12. The non-transitory computer-readable medium of claim 11, wherein generating the set of primitive structures by decomposing the first kinematic graph comprises:determining the one or more edges based on the first kinematic graph; segmenting each of the one or more edges into two respective parts; and generating a non-terminal node at a respective segmentation point of each of the two respective parts for each of the one or more edges.

13. The non-transitory computer-readable medium of claim 11, wherein generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives comprises:generating a first level of the graph hierarchy based on the set of primitives; and generating a second level of the graph hierarchy by attaching each primitive in the set of primitives to all other primitives in the set of primitives to form a first set of graphs.

14. The non-transitory computer-readable medium of claim 13, wherein generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives comprises:generating a third level of the graph hierarchy by attaching each primitive in the set of primitives to each graph in the first set of graphs to form a second set of graphs.

15. The non-transitory computer-readable medium of claim 11, wherein determining the production rules based on the graph hierarchy comprises:determining the production rules for each graph-hierarchy level pair that convert a set of graphs from level t to another graph in level t +1.

16. The non-transitory computer-readable medium of claim 11, wherein determining the continuous parameters based on the production rules using the generative flow network comprises:determining the plurality of bounding boxes each corresponding to the respective node of the plurality of nodes based on a respective center position and a respective spatial extent of the respective node based on the first kinematic graph; anddetermining the one or more joint parameters each corresponding to the respective edge of the one or more edges based on a respective joint axis and a respective joint position of a corresponding edge based on the first kinematic graph.

17. A computing device, comprising:one or more hardware processors; anda non-transitory computer readable medium coupled to the one or more hardware processors, with instructions stored thereon, that when executed by the one or more hardware processors, cause the one or more hardware processors to perform or control performance of operations comprising:obtaining a first kinematic graph that represents a first articulated object that includes a plurality' of parts interconnected by one or more joints, wherein the first kinematic graph includes a plurality of nodes corresponding to the plurality of parts and one or more edges corresponding to the one or more joints;generating a set of primitive structures by decomposing the first kinematic graph, wherein each primitive structure in the set of primitive structures includes a respective terminal node corresponding to a respective node of the plurality of nodes and at least one respective non-terminal node connected to the respective terminal node, wherein a nonterminal node corresponds to a node-edge connection;generating a graph hierarchy by enumerating one or more valid combinations of the set of primitives;determining production rules based on the graph hierarchy;determining kinematic graph grammar for generating discrete kinematic graph structures based on the set of primitive structures and the production rules;determining continuous parameters based on the production rules using a generative flow network, wherein the continuous parameters include a plurality of bounding boxes eachcorresponding to a respective node of the plurality of nodes and one or more joint parameters each corresponding to a respective edge of the one or more edges; andgenerating a second kinematic graph, different from the first kinematic graph, to represent a second articulated object, different from the first articulated object, based on the kinematic graph grammar and the continuous parameters.

18. The computing device of claim 17, wherein generating the set of primitive structures by decomposing the first kinematic graph comprises:determining the one or more edges based on the first kinematic graph; segmenting each of the one or more edges into two respective parts; and generating a non-terminal node at a respective segmentation point of each of the two respective parts for each of the one or more edges.

19. The computing device of claim 17, wherein generating the graph hierarchy by enumerating one or more valid combinations of the set of primitives comprises:generating a first level of the graph hierarchy based on the set of primitives; and generating a second level of the graph hierarchy by attaching each primitive in the set of primitives to all other primitives in the set of primitives to form a first set of graphs.

20. The computing device of claim 17, wherein determining the continuous parameters based on the production rules using the generative flow network comprises:determining the plurality of bounding boxes each corresponding to the respective node of the plurality of nodes based on a respective center position and a respective spatial extent of the respective node based on the first kinematic graph; anddetermining the one or more joint parameters each corresponding to the respective edge of the one or more edges based on a respective joint axis and a respective joint position of a corresponding edge based on the first kinematic graph.

Images