Geometry representation agnostic learning

Abstract

Three-dimensional (3D) shapes can be represented in a variety of different ways. When applying machine learning to such shape representations, machine learning (ML) techniques may be adapted to specific representations. A common geometry representation is defined. Representation agnostic learning may be performed for an input representation representing a 3D shape, using the common representation to represent the 3D shape. The technique may include converting the input representation into a corresponding common representation, performing a ML task on the corresponding common representation to generate a predicted output in the common representation, and converting the predicted output in the common representation into the input representation. The input representation may be a triangle mesh, an oriented point cloud, a signed distance function, or a voxel representation. The ML task may be performed using a graph neural network.

Description

Attorney Docket No. RC-0395-01-WOGEOMETRY REPRESENTATION AGNOSTIC LEARNINGCROSS-REFERENCE TO RELATED APPLICATIQN(S)

[0001] This application is a non-provisional application that claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 63 / 709,288, filed on October 18, 2024, the contents of which are hereby incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] Embodiments relate generally to online virtual experience platforms, and more particularly but not exclusively, to methods, systems, and computer readable media that train a machine learning model using different types of input geometry representations by converting each type of input geometry representation into a common geometry representation, which enables geometry representation agnostic learning.BACKGROUND

[0003] Online platforms, such as virtual experience platforms including online gaming platforms, can express three-dimensional (3D) shapes in a variety of differing geometry representations, each one offering its own unique structure and affordances. Examples of such heterogeneous shape representations include meshes, point clouds, voxels, signed distance functions, non-uniform rational B-splines (NURBS), etc.

[0004] The background description provided herein is for the purpose of presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present disclosure.SUMMARY

[0005] According to one aspect of the present disclosure, a computer-implemented method is provided. The computer-implemented method includes generating, by a processor, a first common geometry representation of a three-dimensional shape based on an input geometry representation of the three-dimensional shape. The input geometry representation includes a first shape representation of a first type. The first common geometry representation includes aAttorney Docket No. RC-0395-01-WO second shape representation of a second type, different from the first type. The generating the first common geometry representation include sampling, by the processor, a first plurality’ of points having a first distribution characteristic from a surface of the three-dimensional shape defined by' the input geometry representation. The generating the first common geometry representation include sampling, by the processor, a second plurality’ of points having a second distribution characteristic from an interior volume of the three-dimensional shape defined by the input geometry representation. The generating the first common geometry representation include defining, by the processor, a connectivity graph that includes connections between individual points from the first plurality' of points and the second plurality’ of points. The computer-implemented method includes inputting, by the processor, the first common geometry representation into a machine learning model that is included in a geometry representation-agnostic learning system of a computing device. The computer-implemented method includes generating, by the processor, a second common geometry representation of the three-dimensional shape using the machine learning model. The second common geometry representation includes a third shape representation of the second type. The computer- implemented method includes generating, by the processor, an output geometry representation by mapping the second common geometry’ representation to the input geometry’ representation via interpolation. The output geometry’ representation includes a fourth shape representation of the first type. The computer-implemented method includes training, by the processor, the machine learning model based on one or more of the input geometry’ representation, the first common geometry representation, the second common geometry representation, or the output geometry’ representation of the three-dimensional shape to improve an ability of the computing device to accurately identify geometry' of the three-dimensional shape, perform segmentation of the geometry’ of the three-dimensional shape, and estimate j oint location for auto-rigging of the three-dimensional shape.

[0006] In some implementations, the second common geometry representation includes first geometry information contained in the first common geometry representation and one or more of geometry identification of the three-dimensional shape, geometry’ segmentation of the geometry' of the three-dimensional shape, or joint location for auto-rigging of the three- dimensional shape. In some implementations, the output geometry’ representation includes second geometry information contained in the input geometry' representation and one or more of the geometry identification of the three-dimensional shape, the geometry segmentation ofAttorney Docket No. RC-0395-01-WO the geometry of the three-dimensional shape, or the joint location for auto-rigging of the three- dimensional shape.

[0007] In some implementations, the first shape representation of the first type is a mesh. In some implementations, sampling the first plurality of points includes sampling, by the processor, one or more points from within a triangle in the mesh for inclusion in the first plurality of points. In some implementations, the one or more points are weighted by an area of the triangle in the mesh.

[0008] In some implementations, the first shape representation of the first type is a point cloud. In some implementations, sampling the first plurality of points includes determining, by the processor, one or more k-nearest neighbors in the point cloud from the interior volume of the three-dimensional shape defined by the input geometry representation. In some implementations, sampling the first plurality of points includes determining, by the processor, a one-ring neighborhood of points for each point in the first plurality of points based on the one or more k-nearest neighbors. In some implementations, sampling the first plurality of points includes determining, by the processor, a Delaunay triangulation based on the one-ring neighborhood of points for each point in the first plurality of points. In some implementations, sampling the first plurality of points includes sampling, by the processor, the Delaunay triangulation determined for each point in the first plurality of points.

[0009] In some implementations, the first shape representation of the first type is a signed- distance function that includes a gradient. In some implementations, sampling the first plurality of points include sampling, by the processor, each point in the first plurality of points volumetrically within a three-dimensional bounding box around a 0-isosurface. In some implementations, sampling the first plurality of points include projecting, by the processor, each point in the first plurality of points onto the 0-isosurface by translating each point in a direction of the gradient by an absolute value of its signed distance.

[0010] In some implementations, the first shape representation of the first type is a voxel. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, determining, by the processor, a surface corresponding to a boundary of a voxel grid of a respective point. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, determining, by the processor, a smooth surface of the three-dimensional shape by surface regularization of the surfaceAttorney Docket No. RC-0395-01-WO corresponding to the boundary' of the voxel grid of the respective point. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, defining, by the processor, a quadrilateral face for the respective point based on the smooth surface of the three-dimensional shape. In some implementations, sampling the first plurality' of points includes, for each point in the first plurality' of points, sampling, by the processor, the quadrilateral face for the respective point.

[0011] In some implementations, sampling the second plurality of points includes sampling, by the processor, the second plurality' of points within a three-dimensional bounding box of the three-dimensional shape. In some implementations, sampling the second plurality of points includes in response to determining that one or more points from the second plurality of points are located outside the three-dimensional bounding box of the three-dimensional shape, discarding, by the processor, the one or more points located outside the three- dimensional bounding box of the three-dimensional shape.

[0012] In some implementations, sampling the first plurality of points and the second plurality of points includes determining, by the processor, a Poisson disk radius based on a point from the first plurality' of points and each point in the second plurality of points. In some implementations, sampling the first plurality of points and the second plurality’ of points includes adjusting, by the processor, the Poisson disk radius based on a target number of points and a surface area of the three-dimensional shape to control sampling resolution.

[0013] In some implementations, defining the connectivity graph includes determining, by the processor, a Delaunay tetrahedralization based on the first plurality of points and the second plurality of points. In some implementations, defining the connectivity7graph includes retaining, by the processor, tetrahedron edges whose centroids lie within the three-dimensional shape based on the Delaunay tetrahedralization. In some implementations, defining the connectivity graph includes determining, by the processor, pairwise connections between individual points from the first plurality of points and the second plurality of points based on the retained tetrahedron edges.

[0014] In some implementations, generating the first common geometry representation further includes associating, by the processor, each point from the first plurality' of points with a feature vector. In some implementations, the feature vector for a point in the first pluralityAttomey Docket No. RC-0395-01-WO of points is a six-dimensional vector that includes a three-dimensional coordinate of the point and a three-dimensional normal vector.

[0015] According to another aspect of the present disclosure, a non-transitory computer- readable medium of a virtual-experience coordinator is provided. The non-transitory computer-readable medium includes instructions stored thereon that, when executed by one or more hardware processors, cause the one or more hardware processors to perform or control performance of operations.

[0016] The operations include generating a first common geometry’ representation of a three-dimensional shape based on an input geometry representation of the three-dimensional shape. The input geometry representation includes a first shape representation of a first ty pe. The first common geometry’ representation includes a second shape representation of a second type, different from the first ty pe. The generating the first common geometry representation includes sampling a first plurality of points having a first distribution characteristic from a surface of the three-dimensional shape defined by the input geometry representation. The generating the first common geometry’ representation includes sampling a second plurality' of points having a second distribution characteristic from an interior volume of the three- dimensional shape defined by the input geometry representation. The generating the first common geometry representation include defining a connectivity graph that includes connections between individual points from the first plurality of points and the second plurality of points. The operations include inputting the first common geometry’ representation into a machine learning model that is included in a geometry representation-agnostic learning system of a computing device. The operations include generating a second common geometry representation of the three-dimensional shape using the machine learning model. The second common geometry’ representation includes a third shape representation of the second type. The operations include generating an output geometry' representation by mapping the second common geometry representation to the input geometry representation via interpolation. The output geometry representation includes a fourth shape representation of the first type. The operations include training the machine learning model based on one or more of the input geometry’ representation, the first common geometry’ representation, the second common geometry representation, or the output geometry representation of the three-dimensional shape to improve an ability of the computing device to accurately identify geometry of the three-Attorney Docket No. RC-0395-01-WO dimensional shape, perform segmentation of the geometry of the three-dimensional shape, and estimate joint location for auto-rigging of the three-dimensional shape.

[0017] In some implementations, the second common geometry representation includes first geometry information contained in the first common geometry representation and one or more of geometry identification of the three-dimensional shape, geometry’ segmentation of the geometry of the three-dimensional shape, or joint location for auto-rigging of the three- dimensional shape. In some implementations, the output geometry' representation includes second geometry' information contained in the input geometry' representation and one or more of the geometry identification of the three-dimensional shape, the geometry segmentation of the geometry of the three-dimensional shape, or the joint location for auto-rigging of the three- dimensional shape.

[0018] In some implementations, the first shape representation of the first type is a mesh. In some implementations, sampling the first plurality of points includes sampling one or more points from within a triangle in the mesh for inclusion in the first plurality of points. In some implementations, the one or more points are weighted by an area of the triangle in the mesh.

[0019] In some implementations, the first shape representation of the first type is a point cloud. In some implementations, sampling the first plurality' of points includes determining one or more k-nearest neighbors in the point cloud from the interior volume of the three- dimensional shape defined by the input geometry' representation. In some implementations, sampling the first plurality of points includes determining a one-ring neighborhood of points for each point in the first plurality of points based on the one or more k-nearest neighbors. In some implementations, sampling the first plurality of points includes determining a Delaunay triangulation based on the one-ring neighborhood of points for each point in the first plurality of points. In some implementations, sampling the first plurality of points includes sampling the Delaunay triangulation determined for each point in the first plurality of points.

[0020] In some implementations, the first shape representation of the first type is a signed- distance function that includes a gradient. In some implementations, sampling the first plurality of points includes sampling each point in the first plurality of points volumetrically within a three-dimensional bounding box around a O-isosurface. In some implementations, sampling the first plurality' of points include projecting each point in the first plurality of points onto theAttorney Docket No. RC-0395-01-WOO-isosurface by translating each point in a direction of the gradient by an absolute value of its signed distance.

[0021] In some implementations, the first shape representation of the first type is a voxel. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, determining a surface corresponding to a boundary of a voxel grid of a respective point. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, determining a smooth surface of the three-dimensional shape by surface regularization of the surface corresponding to the boundary of the voxel grid of the respective point. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, defining a quadrilateral face for the respective point based on the smooth surface of the three-dimensional shape. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, sampling the quadrilateral face for the respective point.

[0022] In some implementations, sampling the second plurality of points includes sampling the second plurality of points within a three-dimensional bounding box of the three- dimensional shape. In some implementations, sampling the second plurality of points includes in response to determining that one or more points from the second plurality of points are located outside the three-dimensional bounding box of the three-dimensional shape, discarding the one or more points located outside the three-dimensional bounding box of the three- dimensional shape.

[0023] In some implementations, sampling the first plurality of points and the second plurality of points includes determining a Poisson disk radius based on a point from the first plurality of points and each point in the second plurality of points. In some implementations, sampling the first plurality of points and the second plurality of points includes adjusting the Poisson disk radius based on a target number of points and a surface area of the three- dimensional shape to control sampling resolution.

[0024] In some implementations, defining the connectivity graph includes determining a Delaunay tetrahedralization based on the first plurality of points and the second plurality of points. In some implementations, defining the connectivity graph includes retaining tetrahedron edges whose centroids lie within the three-dimensional shape based on the Delaunay tetrahedralization. In some implementations, defining the connectivity graphAttorney Docket No. RC-0395-01-WO includes determining pairwise connections between individual points from the first plurality' of points and the second plurality of points based on the retained tetrahedron edges.

[0025] In some implementations, generating the first common geometry representation further includes associating each point from the first plurality- of points yvith a feature vector. In some implementations, the feature vector for a point in the first plurality’ of points is a sixdimensional vector that includes a three-dimensional coordinate of the point and a three- dimensional normal vector.

[0026] According to a further aspect of the present disclosure, a computing device of a virtual-experience coordinator is provided. The computing device includes one or more hardyvare processors. The computing device may include a non-transitory computer readable medium coupled to the one or more hardware processors, with instructions stored thereon, that when executed by the one or more hardware processors, cause the one or more hardware processors to perform or control performance of operations. The operations include generating a first common geometry representation of a three-dimensional shape based on an input geometry representation of the three-dimensional shape. The input geometry representation includes a first shape representation of a first ty pe. The first common geometry representation includes a second shape representation of a second type, different from the first type. The generating the first common geometry representation includes sampling a first plurality of points having a first distribution characteristic from a surface of the three-dimensional shape defined by the input geometry' representation. The generating the first common geometry' representation includes sampling a second plurality of points having a second distribution characteristic from an interior volume of the three-dimensional shape defined by the input geometry representation. The generating the first common geometry representation include defining a connectivity7graph that includes connections betyveen individual points from the first plurality of points and the second plurality of points. The operations include inputting the first common geometry representation into a machine learning model that is included in a geometry representation-agnostic learning system of a computing device. The operations include generating a second common geometry / representation of the three-dimensional shape using the machine learning model. The second common geometry' representation includes a third shape representation of the second type. The operations include generating an output geometry representation by mapping the second common geometry representation to the input geometry representation via interpolation. The output geometry representation includes a fourth shapeAttorney Docket No. RC-0395-01-WO representation of the first type. The operations include training the machine learning model based on one or more of the input geometry representation, the first common geometry representation, the second common geometry representation, or the output geometry representation of the three-dimensional shape to improve an ability of the computing device to accurately identify geometry7of the three-dimensional shape, perform segmentation of the geometry of the three-dimensional shape, and estimate joint location for auto-rigging of the three-dimensional shape.

[0027] In some implementations, the second common geometry' representation includes first geometry information contained in the first common geometry representation and one or more of geometry identification of the three-dimensional shape, geometry' segmentation of the geometry of the three-dimensional shape, or joint location for auto-rigging of the three- dimensional shape. In some implementations, the output geometry’ representation includes second geometry information contained in the input geometry' representation and one or more of the geometry identification of the three-dimensional shape, the geometry segmentation of the geometry of the three-dimensional shape, or the joint location for auto-rigging of the three- dimensional shape.

[0028] In some implementations, the first shape representation of the first type is a mesh. In some implementations, sampling the first plurality of points includes sampling one or more points from within a triangle in the mesh for inclusion in the first plurality of points. In some implementations, the one or more points are weighted by an area of the triangle in the mesh.

[0029] In some implementations, the first shape representation of the first type is a point cloud. In some implementations, sampling the first plurality' of points includes determining one or more k-nearest neighbors in the point cloud from the interior volume of the three- dimensional shape defined by the input geometry' representation. In some implementations, sampling the first plurality of points includes determining a one-ring neighborhood of points for each point in the first plurality of points based on the one or more k-nearest neighbors. In some implementations, sampling the first plurality of points includes determining a Delaunay triangulation based on the one-ring neighborhood of points for each point in the first plurality of points. In some implementations, sampling the first plurality of points includes sampling the Delaunay triangulation determined for each point in the first plurality of points.Attorney Docket No. RC-0395-01-WO

[0030] In some implementations, the first shape representation of the first type is a signed- distance function that includes a gradient. In some implementations, sampling the first plurality of points includes sampling each point in the first plurality of points volumetrically within a three-dimensional bounding box around a 0-isosurface. In some implementations, sampling the first plurality of points include projecting each point in the first plurality of points onto the 0-isosurface by translating each point in a direction of the gradient by an absolute value of its signed distance.

[0031] In some implementations, the first shape representation of the first ty pe is a voxel. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, determining a surface corresponding to a boundary of a voxel grid of a respective point. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, determining a smooth surface of the three-dimensional shape by surface regularization of the surface corresponding to the boundary of the voxel grid of the respective point. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, defining a quadrilateral face for the respective point based on the smooth surface of the three-dimensional shape. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, sampling the quadrilateral face for the respective point.

[0032] In some implementations, sampling the second plurality of points includes sampling the second plurality of points within a three-dimensional bounding box of the three- dimensional shape. In some implementations, sampling the second plurality of points includes in response to determining that one or more points from the second plurality of points are located outside the three-dimensional bounding box of the three-dimensional shape, discarding the one or more points located outside the three-dimensional bounding box of the three- dimensional shape.

[0033] In some implementations, sampling the first plurality of points and the second plurality of points includes determining a Poisson disk radius based on a point from the first plurality7of points and each point in the second plurality of points. In some implementations, sampling the first plurality of points and the second plurality of points includes adjusting the Poisson disk radius based on a target number of points and a surface area of the three- dimensional shape to control sampling resolution.Attorney Docket No. RC-0395-01-WO

[0034] In some implementations, defining the connectivity graph includes determining a Delaunay tetrahedralization based on the first plurality of points and the second plurality of points. In some implementations, defining the connectivity graph includes retaining tetrahedron edges whose centroids lie within the three-dimensional shape based on the Delaunay tetrahedralization. In some implementations, defining the connectivity' graph includes determining pairwise connections between individual points from the first plurality of points and the second plurality of points based on the retained tetrahedron edges.

[0035] In some implementations, generating the first common geometry' representation further includes associating each point from the first plurality of points with a feature vector. In some implementations, the feature vector for a point in the first plurality of points is a sixdimensional vector that includes a three-dimensional coordinate of the point and a three- dimensional normal vector.

[0036] According to yet another aspect, portions, features, and implementation details of the systems, methods, and non-transitory computer-readable media may be combined to form additional aspects, including some aspects which omit and / or modify some or portions of individual components or features, include additional components or features, and / or other modifications; and all such modifications are within the scope of this disclosure.BRIEF DESCRIPTION OF DRAWINGS

[0037] FIG. 1 is a diagram of an example network environment, in accordance with some implementations.

[0038] FIG. 2 is a diagram of an example geometry representation-agnostic learning system, in accordance with some implementations.

[0039] FIG. 3 is a diagram of example surface points of an input geometry representation sampled by the system of FIG. 2, in accordance with some implementations.

[0040] FIG. 4 is a diagram of example interior points of an input geometry representation sampled by the system of FIG. 2, in accordance with some implementations.Attorney Docket No. RC-0395-01-WO

[0041] FIG. 5 is a diagram of an example tetrahedralization of sampled surface and interior points performed by the system of FIG. 2, in accordance with some implementations.

[0042] FIG. 6 is a diagram of a connectivity graph generated based on the sampled surface and interior points and tetrahedralization by the system of FIG. 2, in accordance with some implementations.

[0043] FIG. 7 is a flowchart of an example method of geometry representation-agnostic learning, in accordance with some implementations.

[0044] FIG. 8 is a block diagram illustrating an example computing device, in accordance with some implementations.DETAILED DESCRIPTION

[0045] In the following detailed description, reference is made to the accompanying drawings, which form a part hereof. In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The illustrative implementations described in the detailed descnption, drawings, and claims are not meant to be limiting. Other implementations may be utilized, and other changes may be made, without departing from the spirit or scope of the subject matter presented herein. Aspects of the present disclosure, as generally described herein, and illustrated in the Figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein.

[0046] References in the specification to “some implementations,” “an implementation,” “an example implementation,” etc. indicate that the implementation described may include a particular feature, structure, or characteristic, but every implementation may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same implementation. Further, when a particular feature, structure, or characteristic is described in connection with an implementation, such feature, structure, or characteristic may be effected in connection with other implementations whether or not explicitly described.

[0047] Various embodiments are described herein in the context of 3D shapes (e.g., 3D avatars, objects, etc.) that are used in a 3D virtual experience or environment. SomeAttorney Docket No. RC-0395-01-WO implementations of the techniques described herein may be applied to various ty pes of 3D environments, such as a virtual reality (VR) conference, a 3D session (e.g., an online lecture or other type of presentation involving 3D avatars), a virtual concert, an augmented reality (AR) session, an online game, or in other types of 3D environments that may include one or more users that are represented in the 3D environment by one or more 3D avatars.

[0048] 3D shapes can be expressed in a variety of differing geometry representations, each offering its own unique structure and affordances. Examples of such heterogeneous shape representations include meshes, point clouds, voxels, signed distance functions, NURBS, etc.

[0049] In the context of applying machine learning to such geometric data, model architectures have been developed to accommodate the structure of a single representation. Despite these architectural advancements, the advancements tend to be coupled to one type of representation, which limits the ability to leam geometric characteristics for rendering due to a scarcity of 3D data.

[0050] To address these and other challenges, the present disclosure provides geometry representation-agnostic learning that enables a machine learning model to be trained and tested using input data corresponding to different types of geometry presentations. The present technique(s) convert different types of input geometry representations (e.g., mesh, point cloud, voxel, signed distance function, NURBS, etc.) to a common geometry representation, yvhich is then used to train and test a machine learning model.

[0051] Upon conversion to the common geometry representation, the underlying 3D shape data appears to correspond to the same underlying data distribution, regardless of the ty pe of geometry' representation used as the input. The common geometry' representation described herein is compatible with a yvide range of neural network architectures and is not limited to a graph neural network. The common geometry representation has a defined geometric structure (e.g., as compared to an encoded embedding).

[0052] Using the common geometry representation, the machine learning model of the present disclosure may leam and be trained to identify various information used to render the corresponding 3D shape. This information may include the underlying geometry’ of the 3D shape, 3D geometry segmentation labels (e.g., head, torso, bicep, forearm, hand, thigh, calf, foot, etc.), and joint location for auto-rigging, etc. The machine learning model may output the common representation along with this additional information.Attorney Docket No. RC-0395-01-WO

[0053] An output geometry' representation (which is the same or similar type of geometry representation as the input geometry representation) may be generated by mapping the output common geometry representation to the input geometry representation via interpolation. That is, the output geometry representation may include the geometry information included in the input geometry' representation 201 and the above-mentioned additional information (e.g., the underlying geometry of the 3D shape, 3D geometry segmentation labels, joint location for autorigging, etc.). Using the geometry information and the additional information in the output geometry' representation, the 3D shape may be rendered and / or animated.

[0054] FIG. 1 is a diagram of an example system architecture 100 that includes a virtual experience platform that can support construction and presentation of 3D objects, in accordance with some implementations. In the example of FIG. 1 , the 3D environment platform will be described in the context of a virtual experience server 102 purely for purposes of explanation, and various other implementations can provide other types of 3D environment platforms, such as online meeting platforms, virtual reality (VR) or augmented reality (AR) platforms, or other types of platforms that can provide 3D content. The description provided herein for the virtual experience server 102 and other elements of the system architecture 100 can be adapted to be operable with such other ty pes of 3D environment platforms.

[0055] Virtual experience platforms (also referred to as “user-generated content platforms” or “user-generated content systems”) offer a variety of ways for users to interact with one another, such as while the users are playing an electronic virtual experience. For example, users of a virtual experience platform may work together towards a common goal, share various virtual gaming items, send electronic messages to one another, and so forth. Users of a virtual experience platform may play virtual experiences using characters, such as the 3D avatars, which the users can navigate through a 3D world rendered in the electronic virtual experience.

[0056] A virtual experience platform may also enable users of the platform to create and animate avatars, as well as enabling the users to create other graphical objects to place in the 3D world. For example, users of the virtual experience platform may be allowed to create, design, and customize the avatar, and to create other 3D objects for presentation in the 3D world.

[0057] FIG. 1 is a diagram of an example environment (having a system architecture 100) to enable geometry representation-agnostic learning, in accordance with someAttorney Docket No. RC-0395-01-WO implementations. FIG. 1 and other figures use like reference numerals to identify like elements. A letter after a reference numeral, such as “110," indicates that the text refers specifically to the element having that particular reference numeral. A reference numeral in the text without a following letter, such as "110," refers to any or all of the elements in the figures bearing that reference numeral (e.g. " 110" in the text refers to reference numerals “110a," “110b," and / or “1 lOn” in the figures).

[0058] The system architecture 100 (also referred to as “system” herein) includes online virtual experience server 102, data store 120, client devices 110a, 110b, and HOn (generally referred to as “client device(s) 110” herein), content management server 140, and developer devices 130a and 130n (generally referred to as “developer device(s) 130” herein). Virtual experience server 102, content management server 140, data store 120, client devices 1 10, and developer devices 130 are coupled via network 122. In some implementations, client devices 110 and developer device(s) 130 may refer to the same or same type of device.

[0059] Online virtual experience server 102 can include a virtual experience engine 104, one or more virtual expert ence(s) 106, and graphics engine 108. A client device 110 can include a virtual experience application 112, and input / output (I / O) interfaces 114 (e.g., input / output devices). The input / output devices can include one or more of a microphone, speakers, headphones, display device, mouse, keyboard, game controller, touchscreen, virtual reality consoles, etc. The input / output devices can also include accessory devices that are connected to the client device by means of a cable (wired) or that are wirelessly connected.

[0060] Content management server 140 can include a graphics engine 144 and a classification controller 146. In some implementations, the content management server 140 may include a plurality' of servers. In some implementations, the plurality of servers may be arranged in a hierarchy (e.g., based on respective prioritization values assigned to content sources).

[0061] Graphics engine 144 may be utilized for the rendering of one or more objects (e.g., 3D objects associated with the virtual environment). Classification controller 146 may be utilized to classify assets such as 3D objects and for the detection of inauthentic digital assets, etc. Data store 148 may be utilized to store a search index, model information, etc.

[0062] A developer device 130 can include a virtual experience application 132 and input / output (I / O) interfaces 134 (e.g., input / output devices). The input / output devices canAttorney Docket No. RC-0395-01-WO include one or more of a microphone, speakers, headphones, display device, mouse, keyboard, game controller, touchscreen, virtual reality consoles, etc.

[0063] System architecture 100 is provided for illustration. In different implementations, the system architecture 100 may include the same, fewer, more, or different elements configured in the same or different manner as that shown in FIG. 1.

[0064] In some implementations, network 122 may include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), a wired network (e.g., ethemet network), a wireless network (e.g., an 802.11 network, a Wi-Fi® network, or wireless LAN (WLAN)), a cellular network (e.g., a 5G network, a long term evolution (LTE) network, etc.), routers, hubs, switches, server computers, or a combination thereof.

[0065] In some implementations, the data store 120 may be a non-transitory computer readable memory (e.g., random access memory), a cache, a drive (e.g., a hard drive), a flash drive, a database system, a cloud storage system, or another type of component or device capable of storing data. The data store 120 may also include multiple storage components (e.g., multiple drives or multiple databases) that may also span multiple computing devices (e.g., multiple server computers).

[0066] In some implementations, the online virtual experience ser er 102 can include a server having one or more computing devices (e.g., a cloud computing system, a rackmount server, a server computer, cluster of physical servers, etc.). In some implementations, the online virtual experience server 102 may be an independent system, may include multiple servers, or be part of another system or server.

[0067] In some implementations, the online virtual experience server 102 may include one or more computing devices (such as a rackmount server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a distributed computing system, a cloud computing system, etc.), data stores (e.g., hard disks, memories, databases), networks, software components, and / or hardware components that may be used to perform operations on the online virtual experience server 102 and to provide a user with access to online virtual experience server 102. The online virtual experience serv er 102 may also include a website (e.g., a web page) or application back-end software that may be used to provide a user with access to content provided by online virtualAttorney Docket No. RC-0395-01-WO experience server 102. For example, users may access online virtual experience server 102 using the virtual experience application 112 on client devices 110.

[0068] In some implementations, online virtual experience server 102 may be a type of social network providing connections between users or a type of user-generated content system that allows users (e.g., end-users or consumers) to communicate with other users on the online virtual experience server 102, where the communication may include voice chat (e.g., synchronous and / or asynchronous voice communication), video chat (e.g., synchronous and / or asynchronous video communication), or text chat (e.g., synchronous and / or asynchronous textbased communication). In some implementations of the disclosure, a “user” may be represented as a single individual. However, other implementations of the disclosure encompass a “user” (e.g., creating user) being an entity controlled by a set of users or an automated source. For example, a set of individual users federated as a community or group in a user-generated content system may be considered a “user.” In some contexts, a “user” may be a system administrator, a developer, a content provider, or other type of entity that may have privileges / capabilities that are different from those of an end user.

[0069] In some implementations, online virtual experience server 102 may be an online gaming server. For example, the virtual experience server 102 may provide single-player or multiplayer games to a community of users that may access or interact with games using client devices 110 via network 122. In some implementations, games (also referred to as “video game,” “online game,” or “virtual game” herein) may be two-dimensional (2D) games, three- dimensional (3D) games (e.g., 3D user-generated games), virtual reality (VR) games, or augmented reality (AR) games, for example. In some implementations, users may participate in gameplay with other users. In some implementations, a game may be played in real-time with other users of the game.

[0070] In some implementations, gameplay may refer to the interaction of one or more players using client devices (e.g., 110) within a game (e.g., game that is part of virtual experience 106) or the presentation of the interaction on a display or other output device (e.g., 114) of a client device 110.

[0071] In some implementations, a virtual experience 106 can include an electronic file that can be executed or loaded using software, firmware or hardware configured to present the game content (e.g., digital media item) to an entity7. In some implementations, a virtualAttorney Docket No. RC-0395-01-WO experience application 112 may be executed and a virtual experience 106 executed in connection with a virtual experience engine 104. In some implementations, a virtual experience 106 (e.g., a game) may have a common set of rules or common goal, and the environment of a virtual experience 106 shares the common set of rules or common goal. In some implementations, different games may have different rules or goals from one another.

[0072] In some implementations, virtual experience(s) may have one or more environments (also referred to as “gaming environments” or “virtual environments” herein) where multiple environments may be linked. An example of an environment may be a three- dimensional (3D) environment. The one or more environments of a virtual experience application 112 may be collectively referred to a “world” or “gaming world” or “virtual world” or “universe” herein. An example of a world may be a 3D world of a virtual experience 106. For example, a user may build a virtual environment that is linked to another virtual environment created by another user. A character of the virtual game may cross the virtual border to enter the adjacent virtual environment.

[0073] It may be noted that 3D environments or 3D worlds use graphics that use a three- dimensional representation of geometric data representative of game content (or at least present game content to appear as 3D content whether or not 3D representation of geometric data is used). 2D environments or 2D worlds use graphics that use two-dimensional representation of geometric data representative of game content.

[0074] In some implementations, the online virtual experience server 102 can host one or more virtual experiences 106 and can permit users to interact with the virtual experiences 106 using a virtual experience application 112 of client devices 110. Users of the online virtual experience server 102 may play, create, interact with, or build virtual experiences 106, communicate with other users, and / or create and build objects (e.g., also referred to as “item(s)” or “game objects” or “virtual game item(s)” herein) of virtual experiences 106. For example, in generating user-generated virtual items, users may create characters, decoration for the characters, one or more virtual environments for an interactive game, or build structures used in a game. In some implementations, users may buy, sell, or trade virtual game objects, such as in-platform currency (e.g.. virtual currency), with other users of the online virtual experience server 102. In some implementations, online virtual experience sen7er 102 may transmit game content to virtual experience applications (e.g., 112). In some implementations, game content (also referred to as “content” herein) may refer to any data or software instructions (e.g., gameAttorney Docket No. RC-0395-01-WO objects, game, user information, video, images, commands, media item, etc.) associated with online virtual experience server 102 or virtual experience applications. In some implementations, game objects (e.g., also referred to as ‘"item(s)” or ‘"objects” or ‘"virtual objects” or “virtual game item(s)” herein) may refer to objects that are used, created, shared or otherwise depicted in virtual experiences 106 of the online virtual experience server 102 or virtual experience applications 112 of the client devices 110. For example, game objects may include a part, model, character, accessories, tools, weapons, clothing, buildings, vehicles, currency, flora, fauna, components of the aforementioned (e.g., windows of a building), and so forth.

[0075] It may be noted that the online virtual experience server 102 hosting virtual experiences 106, is provided for purposes of illustration, rather than limitation. In some implementations, online virtual experience serv er 102 may host one or more media items that can include communication messages from one user to one or more other users. Media items can include, but are not limited to, digital video, digital movies, digital photos, digital music, audio content, melodies, website content, social media updates, electronic books, electronic magazines, digital newspapers, digital audio books, electronic journals, web blogs, real simple syndication (RSS) feeds, electronic comic books, software applications, etc. In some implementations, a media item may be an electronic file that can be executed or loaded using software, firmware or hardware configured to present the digital media item to an entity.

[0076] In some implementations, a virtual application 112 / 132 may be associated with a particular user or a particular group of users (e.g., a private game) or made widely available to users with access to the online virtual experience server 102 (e.g., a public game). In some implementations, where online virtual experience server 102 associates one or more virtual experiences 106 with a specific user or group of users, online virtual experience server 102 may associate the specific user(s) with a virtual experience 106 using user account information (e.g., a user account identifier such as username and password).

[0077] In some implementations, online virtual experience server 102 or client devices 110 may include a virtual experience engine 104 or virtual experience application 112. In some implementations, virtual experience engine 104 may be used for the development or execution of virtual experiences 106. For example, virtual experience engine 104 may include a rendering engine (“renderer”) for 2D, 3D, VR, or AR graphics, a physics engine, a collision detection engine (and collision response), sound engine, scripting functionality’, animation engine,Attorney Docket No. RC-0395-01-WO artificial intelligence engine, networking functionality, streaming functionality, memory management functionality, threading functionality, scene graph functionality, or video support for cinematics, among other features. The components of the virtual experience engine 104 may generate commands that help compute and render the game (e.g., rendering commands, collision commands, physics commands, etc.) In some implementations, virtual experience applications 112 of client devices 110 may work independently, in collaboration with virtual experience engine 104 of online virtual experience server 102, or a combination of both.

[0078] In some implementations, both the online virtual experience server 102 and client devices 110 may execute a virtual experience engine and a virtual experience application (104 and 112, respectively). The online virtual experience server 102 using virtual experience engine 104 may perform some or all the virtual experience engine functions (e.g., generate physics commands, rendering commands, etc.), or offload some or all the virtual experience engine functions to virtual experience engine 104 of client device 110. In some implementations, each virtual application 112 / 132 may have a different ratio between the virtual experience engine functions that are performed on the online virtual experience server 102 and the virtual experience engine functions that are performed on the client devices 110. For example, the virtual experience engine 104 of the online virtual experience serv er 102 may be used to generate physics commands in cases where there is a collision between at least two virtual application objects, while the additional virtual experience engine functionality (e.g.. generate rendering commands) may be offloaded to the client device 1 10. In some implementations, the ratio of virtual experience engine functions performed on the online virtual experience server 102 and client device 110 may be changed (e.g., dynamically) based on gameplay conditions. For example, if the number of users participating in gameplay of a particular virtual application 106 exceeds a threshold number, the online virtual experience server 102 may perform one or more virtual experience engine functions that were previously performed by the client devices 110.

[0079] For example, users may be playing a virtual application 1 12 on client devices 1 10, and may send control instructions (e.g., user inputs, such as right, left, up, down, user election, or character position and velocity information, etc.) to the online virtual experience server 102. Subsequent to receiving control instructions from the client dev ices 110. the online virtual experience serv er 102 may send gameplay instructions (e.g., position and velocity information of the characters participating in the group gameplay or commands, such as renderingAttorney Docket No. RC-0395-01-WO commands, collision commands, etc.) to the client devices 110 based on control instructions. For instance, the online virtual experience server 102 may perform one or more logical operations (e.g., using virtual experience engine 104) on the control instructions to generate gameplay instruct! on(s) for the client devices 110. In other instances, online virtual experience server 102 may pass one or more or the control instructions from one client device 110 to other client devices (e.g., from client device 110a to client device 110b) participating in the virtual application 112. The client devices 110 may use the gameplay instructions and render the gameplay for presentation on the displays of client devices 110.

[0080] In some implementations, the control instructions may refer to instructions that are indicative of in-game actions of a user’s character. For example, control instructions may include user input to control the in-game action, such as right, left, up, down, user selection, g roscope position and orientation data, force sensor data, etc. The control instructions may include character position and velocity information. In some implementations, the control instructions are sent directly to the online virtual experience server 102. In other implementations, the control instructions may be sent from a client device 110 to another client device (e.g., from client device 110b to client device HOn), where the other client device generates gameplay instructions using a local virtual experience engine. The control instructions may include instructions to play a voice communication message or other sounds from another user on an audio device (e.g., speakers, headphones, etc.), for example voice communications or other sounds generated using the audio spatialization techniques as described herein.

[0081] In some implementations, gameplay instructions may refer to instructions that allow a client device 1 10 to render gameplay of a game, such as a multiplayer game. The gameplay instructions may include one or more of user input (e.g., control instructions), character position and velocity information, or commands (e.g., physics commands, rendering commands, collision commands, etc ).

[0082] In some implementations, the online virtual experience serv er 102 may store characters created by users in the data store 120. In some implementations, the online virtual experience server 102 maintains a character catalog and game catalog that may be presented to users. In some implementations, the game catalog includes images of virtual experiences stored on the online virtual experience server 102. In addition, a user may select a character (e.g., a character created by the user or other user) from the character catalog to participate in theAttorney Docket No. RC-0395-01-WO chosen game. The character catalog includes images of characters stored on the online virtual experience server 102. In some implementations, one or more of the characters in the character catalog may have been created or customized by the user. In some implementations, the chosen character may have character settings defining one or more of the components of the character.

[0083] In some implementations, a user’s character can include a configuration of components, where the configuration and appearance of components and more generally the appearance of the character may be defined by character settings. In some implementations, the character settings of a user’s character may at least in part be chosen by the user. In other implementations, a user may choose a character with default character settings or character setting chosen by other users. For example, a user may choose a default character from a character catalog that has predefined character settings, and the user may further customize the default character by changing some of the character settings (e.g., adding a shirt with a customized logo). The character settings may be associated with a particular character by the online virtual experience server 102.

[0084] In some implementations, the virtual experience platform may support three- dimensional (3D) objects that are represented by a 3D model and includes a surface representation used to draw the character or object (also known as a skin or mesh) and a hierarchical set of interconnected bones (also known as a skeleton or rig). The ng may be utilized to animate the object and to simulate motion of the object. The 3D model may be represented as a data structure, and one or more parameters of the data structure may be modified to change various properties of the character, e.g., dimensions (height, width, girth, etc.); shape; movement style; number / type of parts; proportion, etc.

[0085] In some implementations, the 3D model may include a 3D mesh. The 3D mesh may define a three-dimensional structure of the unauthenticated virtual 3D object. In some implementations, the 3D mesh may also define one or more surfaces of the 3D object. In some implementations, the 3D object may be a virtual avatar, e.g., a virtual character such as a humanoid character, an animal-character, a robot-character, etc.

[0086] In some implementations, the mesh may be received (imported) in a FBX file format. The mesh file includes data that provides dimensional data about polygons that comprise the virtual 3D object and UV map data that describes how to attach portions of texture to various polygons that comprise the 3D object. In some implementations, the 3D object mayAttorney Docket No. RC-0395-01-WO correspond to an accessory' (e.g., a hat, a weapon, a piece of clothing, etc. worn by a virtual avatar or otherwise depicted with reference to a virtual avatar).

[0087] In some implementations, a platform may enable users to submit (upload) candidate 3D objects for utilization on the platform. A virtual experience development environment (developer tool) may be provided by the platform, in accordance with some implementations. The virtual experience development environment may provide a user interface that enables a developer user to design and / or create virtual experiences (e.g. games). The virtual experience development environment may be a client-based tool (e.g., downloaded and installed on a client device, and operated from the client device), a server-based tool (e.g., installed and executed at a server that is remote from the client device, and accessed and operated by the client device), or a combination of both client-based and service-based elements.

[0088] The virtual experience development environment may be operated by a developer of a virtual experience (e.g., a game developer or any other person who seeks to create a virtual experience that may be published by an online virtual experience platform and utilized by others). The user interface of the virtual experience development environment may be rendered on a display screen of a client device (e.g., such as a developer device 130 described with reference to FIG. 1), so as to enable the creator / developer to interact with the development environment using actions such as ty ping, highlighting, selecting, drag and drop, clicking, and so forth via a mouse, keyboard, or other input device configured to communicate w ith the user interface. The user interface may include a menu bar, a tool bar, a w orkspace pane, and a plurality of secondary panes. Depending on the particular implementation, the user interface may include alternative or additional elements, arrangements, operational features, etc. of the virtual experience development environment than what is shown and described herein.

[0089] A developer user (creator) may utilize the virtual experience development environment to create virtual experiences. As part of the development process, the developer / creator may upload various types of digital content such as object files (meshes), image files, audio files, short videos, etc., to enhance the virtual experience.

[0090] In implementations where the 3D object is an accessory, data indicative of use of the object in a virtual experience may also be received. For example, a "shoe" object may include annotations indicating that the object can be depicted as being worn on the feet of aAttorney Docket No. RC-0395-01-WO virtual humanoid character, while a ‘‘shirt'’ object may include annotations that it may be depicted as being worn on the torso of a virtual humanoid character.

[0091] In some implementations, the 3D model may further include texture information associated with the 3D object. For example, texture information may indicate color and / or pattern of an outer surface of the 3D object. The texture information may enable varying degrees of transparency, reflectiveness, degrees of diffusiveness, material properties, and refractory behavior of the textures and meshes associated with the 3D object. Examples of textures include plastic, cloth, grass, a pane of light blue glass, ice, water, concrete, brick, carpet, wood, etc.

[0092] In some implementations, the client device(s) 110 may each include computing devices such as personal computers (PCs), mobile devices (e.g., laptops, mobile phones, smart phones, tablet computers, or netbook computers), network-connected televisions, gaming consoles, etc. In some implementations, a client device 110 may also be referred to as a “client device.” In some implementations, one or more client devices 110 may connect to the online virtual experience server 102 at any given moment. It may be noted that the number of client devices 110 is provided as illustration. In some implementations, any number of client devices 110 may be used.

[0093] In some implementations, each client device 110 may include an instance of the virtual experience application 112, respectively. In one implementation, the virtual experience application 112 may permit users to use and interact with online virtual experience server 102, such as control a virtual character in a virtual game hosted by online virtual experience server 102, or view or upload content, such as virtual experiences 106, images, video items, web pages, documents, and so forth. In one example, the virtual experience application may be a web application (e.g., an application that operates in conjunction with a w eb browser) that can access, retrieve, present, or navigate content (e.g., virtual character in a virtual environment, etc.) served by a w eb server. In another example, the virtual experience application may be a native application (e.g., a mobile application, app, or a gaming program) that is installed and executes local to client device 110 and allows users to interact with online virtual experience server 102. The virtual experience application may render, display, or present the content (e.g., a web page, a media viewer) to a user. In an implementation, the virtual experience application may also include an embedded media player (e.g., a Flash® player) that is embedded in a web page.Attorney Docket No. RC-0395-01-WO

[0094] In some implementations, the virtual experience application may include an audio engine 116 that is installed on the client device, and which enables the playback of sounds on the client device. In some implementations, audio engine 116 may act cooperatively with graphics engine 144 that is installed on the sound sen- er.

[0095] According to aspects of the disclosure, the virtual experience application may be an online virtual experience server application for users to build, create, edit, and upload content to the online virtual experience server 102, as well as interact with online virtual experience server 102 (e.g., participate in virtual experiences 106 hosted by online virtual experience server 102). As such, the virtual experience application may be provided to the client device(s) 110 by the online virtual experience server 102. In another example, the virtual experience application may be an application that is downloaded from a server.

[0096] In some implementations, each developer device 130 may include an instance of the virtual experience application 132, respectively. In one implementation, the virtual experience application 132 may permit a developer user(s) to use and interact with online virtual experience sen- er 102, such as control a virtual character in a virtual game hosted by online virtual experience ser er 102, or view or upload content, such as virtual experiences 106, images, video items, web pages, documents, and so forth. In one example, the virtual experience application may be a web application (e.g., an application that operates in conjunction with a web browser) that can access, retrieve, present, or navigate content (e.g., virtual character in a virtual environment, etc.) served by a web server. In another example, the virtual experience application may be a native application (e.g., a mobile application, app, or a virtual experience program) that is installed and executes local to developer device 130 and allows users to interact with online virtual experience server 102. The virtual experience application may render, display, or present the content (e.g., a web page, a media viewer) to a user. In an implementation, the virtual experience application may also include an embedded media player (e.g., a Flash® player) that is embedded in a web page.

[0097] According to aspects of the disclosure, the virtual experience application 132 may be an online virtual experience server application for users to build, create, edit, upload content to the online virtual experience server 102 as well as interact with online virtual experience server 102 (e.g., provide and / or play virtual experiences 106 hosted by online virtual experience server 102). As such, the virtual experience application may be provided to the client device(s) 110 by the online virtual experience server 102. In another example, the virtualAttorney Docket No. RC-0395-01-WO experience application 132 may be an application that is downloaded from a sen- er. Virtual experience application 132 may be configured to interact with online virtual experience server 102 and obtain access to user credentials, user currency, etc. for one or more virtual applications 132 developed, hosted, or provided by a virtual experience application developer.

[0098] In some implementations, a user may login to online virtual experience server 102 via the virtual experience application. The user may access a user account by providing user account information (e.g., username and password) where the user account is associated with one or more characters available to participate in one or more virtual experiences 106 of online virtual experience server 102. In some implementations, with appropriate credentials, a virtual experience application developer may obtain access to virtual experience application objects, such as in-platform currency (e g., virtual currency), avatars, special powers, accessories, which are owned by or associated with other users.

[0099] In general, functions described in one implementation as being performed by the online virtual experience server 102 can also be performed by the client device(s) 110, a server, and / or other device(s) usable in the system architecture 100 of FIG. 1 in other implementations if appropriate. In addition, the functionality attributed to a particular component can be performed by different or multiple components operating together. The online virtual experience server 102 can also be accessed as a service provided to other systems or devices through appropriate application programming interfaces (APIs) and thus is not limited to use in w ebsites.

[0100] In some implementations, online virtual experience server 102 may include a graphics engine 108. In some implementations, the graphics engine 108 may be a system, application, or module that permits the online virtual experience sen' er 102 to provide graphics and animation capability . In some implementations, the graphics engine 108, and / or content management server 140 may perform one or more of the operations described below with reference to FIGs. 2-7 or otherwise described herein.

[0101] FIG. 2 is a diagram of an example geometry representation-agnostic learning system 200, in accordance with some implementations. The geometry representation-agnostic learning system 200 may include, for example, a first common geometry engine 202, a machine learning model 204, and a second common geometry engine 206. The geometry representation-agnostic learning system 200 may reside in, for example, virtual experienceAttorney Docket No. RC-0395-01-WO server 102, virtual experience engine 104, graphics engine 108, client device 110, virtual experience application 112, developer device 130, virtual experience application 132. or any other device or component of FIG. 1.

[0102] The technique(s) described below with reference to FIG. 2 enable geometry representation-agnostic learning training and testing of a machine learning model 204 on more than one type of geometry representation. Various example operations that enable geometry representation-agnostic learning will now be described.

[0103] Referring to FIG. 2, an input geometry’ representation 201 of a 3D shape (e.g.. an avatar, object, etc.) may be input into the first common geometry engine 202, which converts the input geometry representation 201 into a first common geometry representation 203. The first common geometry’ representation 203 may be realized as a point set with a connectivity graph.

[0104] To convert the input geometry' representation 201 to the first common geometry representation 203, the first common geometry' engine 202 may, for example, 1) sample surface points of the 3D shape defined by input geometry representation 201, 2) sample interior points of the 3D shape, 3) define connectivity between the surface points and the interior points, and 4) compute pointwise features of the sampled points.

[0105] The operation(s) for sampling surface points may differ depending on the representation type of the input geometry representation 201.

[0106] For instance, in some implementations, the input geometry' representation 201 may be a triangle mesh-type representation. To perform surface-point sampling of a triangle-mesh representation of the 3D shape, the first common geometry engine 202 may randomly sample one or more points from within a triangle (or another type of polygon when a different polygon mesh-type representation is used) in the mesh for inclusion in a first plurality' of points.

[0107] In some implementations, the input geometry’ representation 201 may be a point cloud-type representation. To perform surface-point sampling of a point-cloud representation, for each point of the 3D shape, the first common geometry engine 202 may determine one or more k-nearest neighbors in the point cloud from an interior volume of the 3D shape defined by the input geometry representation 201. Based on the one or more k-nearest neighbors, the first common geometry’ engine 202 may determine a one-ring neighborhood of points for eachAttorney Docket No. RC-0395-01-WO point in the first plurality of points. The first common geometry engine 202 may determine a Delaunay triangulation based on the one-ring neighborhood of points for each point in the first plurality of points. The first common geometry engine 202 may sample the Delaunay triangulation determined for each point in the first plurality of points.

[0108] In some implementations, the input geometry representation 201 may be a signed distance function (SDF)-type representation. To perform surface-point sampling of an SDF representation of the 3D shape, the first common geometry engine 202 may sample each point in the first plurality' of points volumetrically within a 3D bounding box around a 0-isosurface. The first common geometry engine 202 may project each point in the first plurality of points onto the O-isosurface by translating each point in a direction of a gradient of the SDF by an absolute value of its signed distance.

[0109] In some implementations, the input geometry representation 201 may be a voxeltype representation. To perform surface-point sampling of a voxel representation of the 3D shape, the first common geometry engine 202 may determine a surface corresponding to a boundary of a voxel grid of each point in the first plurality' of points, respectively. The first common geometry engine 202 may determine a smooth surface of the 3D shape by surface regularization of the surface corresponding to the boundary of the voxel grid of each point in the first plurality of points. The first common geometry engine 202 may define a quadrilateral face for each point in the first plurality' of points based on the smooth surface of the 3D shape. The first common geometry engine 202 may sample the quadrilateral face for each point in the first plurality of points.

[0110] Once the surface points are sampled for the input geometry representation 201 using one of the above-described techniques, the first common geometry' engine 202 may apply weighted-sample elimination to subsample the sampled surface points until a desired distribution characteristic (e.g., white noise characteristic, blue noise characteristic, etc.) is achieved. In some implementations, the first common geometry engine 202 may apply density- weighted sample elimination to achieve adaptive sampling. The first common geometry' engine 202 may determine a Poisson disk radius based on a point from sampled surface points (e.g., the first plurality of points). Then, the first common geometry engine 202 may adjust the Poisson disk radius based on a target number of points and a surface area of the 3D shape to control sampling resolution.Attorney Docket No. RC-0395-01-WO

[0111] The first common geometry engine 202 may sample the interior points (e.g., a second plurality of points) of the 3D shape. To sample the interior points of the 3D shape, the first common geometry engine 202 may randomly sample points within a 3D bounding box of the 3D shape. Similar to surface-point sampling, the first common geometry engine 202 may apply weighted-sample elimination to subsample point sets of the sampled interior points until a desired distribution characteristic (e g., white noise characteristic, blue noise characteristic, etc.) is achieved. The first common geometry engine 202 may determine a Poisson disk radius based on a point from the sampled interior points. Then, the first common geometry engine 202 may adjust the Poisson disk radius based on a target number of points and a surface area of the 3D shape to control sampling resolution.

[0112] Any points located outside the 3D bounding box of the 3D shape may be discarded by the first common geometry engine 202. The operation(s) for discarding points located outside of the 3D bounding box may differ depending on the representation type of the input geometry representation 201.

[0113] For example, when the input geometry representation 201 is either a triangle meshtype representation or a point cloud-ty pe representation, the first common geometry engine 202 may perform inside-outside segmentation using generalized winding numbers. A winding number at a point p measures the infinitesimal angle subtended by the curve at p and sums up these angles over the curve. For a closed curve in the plane, this yields a piecewise constant function whose value at each point indicates how many times the curve “winds” around it. Hence, a winding number may be a topological quantity' (though it is often defined through a geometric formula). A generalized winding number considers open curves in addition to closed curves.

[0114] When the input geometry representation 201 is an SDF-type representation, the first common geometry engine 202 may perform inside-outside segmentation based on the sign of a point. SDF represents the surface of a 3D shape by a continuous volumetric field: the magnitude of a point in the field represents the distance to the surface boundary' and its sign indicates whether the region in which the point is located is inside (-) or outside (+) of the shape. Hence, its representation implicitly encodes the shape’s boundary as the zero-level-set of the learned function while explicitly representing the classification of space as being part of the shapes’ interior or not.Attorney Docket No. RC-0395-01-WO

[0115] When the input geometry representation 201 is a voxel - pe representation, the first common geometry engine 202 may perform inside-outside segmentation based on voxel occupancy.

[0116] Once the surface points and the interior points have been sampled, the first common geometry engine 202 may define a connectivity’ graph that includes connections between individual points from the surface and interior of the 3D shape. For example, the first common geometry engine 202 may determine a Delaunay tetrahedralization based on the sampled surface and interior points. The Delaunay tetrahedralization of a point set may be determined using incremental insertions whose output satisfies the Delaunay criterion: the interior of the circumsphere of any tetrahedron does not contain any point of the input point set.

[0117] Based on the Delaunay tetrahedralization, the first common geometry engine 202 may retain tetrahedron edges whose centroids lie within the 3D shape. For example, the first common geometry engine 202 may discard tetrahedron edges whose centroids lie outside of the 3D shape. The operation(s) for discarding tetrahedron edges whose centroids lie of the 3D bounding box may differ depending on the representation type of the input geometry representation 201.

[0118] For example, when the input geometry representation 201 is either a triangle meshtype representation or a point cloud-type representation, the first common geometry engine 202 may perform inside-outside segmentation using generalized winding numbers to determine tetrahedron edges with centroids located outside of the surface of the 3D shape. When the input geometry representation 201 is an SDF-type representation, the first common geometry engine 202 may perform inside-outside segmentation based on the sign of a point to determine tetrahedron edges with centroids located outside of the surface of the 3D shape. When the input geometry representation 201 is a voxel-type representation, the first common geometry engine 202 may perform inside-outside segmentation based on voxel occupancy to determine tetrahedron edges with centroids located outside of the surface of the 3D shape.

[0119] Then, the first common geometry' engine 202 may determine pairwise connections between individual points from the sampled surface points and the sampled interior points based on the retained tetrahedron edges. In some implementations, the pointwise features are a 6-dimensional vector that may include an xyz-coordinate and a 3D normal vector. For interior points, a zero vector may be used.Attorney Docket No. RC-0395-01-WO

[0120] The first common geometry engine 202 may generate the first common geometry representation 203 based on the sampled surface points, the sampled interior points, the connectivity graph, and the pointwise features. The first common geometry representation 203 may be input into the machine learning model 204.

[0121] Using the first common geometry representation 203, the machine learning model 204 may learn and be trained to identify various information used to render the corresponding 3D shape. This information may include the underlying geometry of the 3D shape, 3D geometry' segmentation labels (e.g., head, torso, bicep, forearm, hand, thigh, calf, foot, etc.), and joint location for auto-rigging, etc. The machine learning model 204 may output a second common geometry representation 205 that includes the geometry information included in the first common geometry representation 203 and the additional information mentioned above.

[0122] The second common geometry' representation 205 may be input into the second common geometry engine 206. The second common geometry engine 206 may generate an output geometry representation 207, which is the same or similar type of geometry representation as the input geometry representation 201, by mapping the second common geometry representation 205 to the input geometry representation 201 via interpolation. For example, the output geometry representation 207 may include the geometry information included in the input geometry representation 201 and the above-mentioned additional information (e.g., the underlying geometry' of the 3D shape, 3D geometry segmentation labels, joint location for auto-rigging, etc.). Using the geometry' information and the additional information in the output geometry representation 207, the 3D shape may be rendered and / or animated.

[0123] Using the above-described technique(s), the machine learning model 204 of the present disclosure may be trained on an aggregation of datasets that correspond to different types of input geometry’ representations, thereby increasing the total scale of data that is available for training. Training the machine learning model 204 on data corresponding to different ty pes of input geometry' representations may improve the ability of the geometry representation-agnostic learning system 200 (e.g., a computing device) to accurately identify the geometry of a 3D shape corresponding to the input geometry representation 201, perform segmentation of the geometry of the 3D shape to generate geometry segmentation labels, and estimate j oint location for auto-rigging of the three-dimensional shape, etc.Attorney Docket No. RC-0395-01-WO

[0124] FIG. 3 is a diagram of example surface points 300 of an input geometry representation sampled by the first common geometry engine 202 of FIG. 2, in accordance with some implementations.

[0125] Referring to FIG. 3, surface points 300 of a mesh-type geometry representation are depicted. A visualization of sampled surface points of other types of geometry representations may appear the same or different than those depicted in FIG. 3.

[0126] FIG. 4 is a diagram of example interior points 400 of an input geometry representation sampled by the first common geometry engine 202 of FIG. 2, in accordance with some implementations.

[0127] Referring to FIG. 4, interior points 400 of a mesh-type geometry representation are depicted. A visualization of sampled interior points of other types of geometry representations may appear the same or different than those depicted in FIG. 4.

[0128] FIG. 5 is a diagram of an example tetrahedralization 500 of sampled surface and interior points performed by the first common geometry engine 202 of FIG. 2, in accordance with some implementations.

[0129] Referring to FIG. 5, a cross-sectional view of a 3D shape is depicted to illustrate the interior tetrahedrons formed from the sampled surface and interior points. The tetrahedron edges whose centroids fall outside of the 3D shape have been removed.

[0130] FIG. 6 is a diagram of a connectivity graph 600 generated based on the sampled surface and interior points and tetrahedralization by the first common geometry engine 202 of FIG. 2, in accordance with some implementations.

[0131] Referring to FIG. 6, a connectivity graph 600 of a mesh-type geometry representation is depicted. A visualization of a connectivity graph for other types of geometry representations may appear the same or different than the one depicted in FIG. 6.

[0132] FIG. 7 is a flowchart of an example method 700 of articulated-shape generation, in accordance with some implementations.

[0133] In some implementations, method 700 can be implemented, for example, on an online virtual experience server 102 (e.g., by a virtual-experience coordinator) described withAttorney Docket No. RC-0395-01-WO reference to FIG. 1 and which implements the system 200 of FIG. 2. In some implementations, some or all of the method 700 can be implemented on one or more client devices 110 as shown in FIG. 1, on one or more developer devices 130, or on one or more online virtual experience server(s) 102, and / or on a combination of developer device(s), server device(s) and client device(s), and / or on other device(s) usable in the system architecture 100 of FIG. 1. In described examples, an implementing system may include one or more digital processors or processing circuitry ('“processors”), and one or more storage devices (e.g., a data store 108 or other storage). In some implementations, different components of one or more servers and / or clients can perform different blocks or other parts of the method 700. In some examples, a first device is described as performing blocks of method 700. Some implementations can have one or more blocks of method 700 performed by one or more other devices (e.g., other client devices or server devices) that can send results or data to the first device.

[0134] In some implementations, method 700, or portions of the methods, can be initiated automatically by a system. In some implementations, the implementing system is a first device. For example, the method (or portions thereof) can be periodically or otherwise repeatedly performed or performed based on one or more particular events or conditions (e.g., upon a user request and / or one or more other conditions occurring which can be specified in settings read by the method(s) / computing device). In the method 700 of FIG. 7 and in other methods / operations described herein, some operations may be optional, modified, omitted, combined, supplemented with other operations, performed in a different order than as shown (e.g., sequentially, in parallel, etc.), and so forth.

[0135] Referring to FIG. 7, method 700 may begin at block 702. At block 702, a first common geometry representation of a three-dimensional shape may be generated based on an input geometry representation of the three-dimensional shape. The input geometry representation includes a first shape representation of a first type. The first common geometry representation includes a second shape representation of a second t pe, different from the first type.

[0136] In some implementations, the generating the first common geometry representation includes sampling a first plurality of points having a first distribution characteristic from a surface of the three-dimensional shape defined by the input geometry representation.Attorney Docket No. RC-0395-01-WO

[0137] In some implementations, the first shape representation of the first type is a mesh. In some implementations, sampling the first plurality of points includes sampling one or more points from within a triangle in the mesh for inclusion in the first plurality of points. In some implementations, the one or more points are weighted by an area of the triangle in the mesh.

[0138] In some implementations, the first shape representation of the first type is a point cloud. In some implementations, sampling the first plurality of points includes determining one or more k-nearest neighbors in the point cloud from an interior volume of the three- dimensional shape defined by the input geometry representation. In some implementations, sampling the first plurality of points includes determining a one-ring neighborhood of points for each point in the first plurality of points based on the one or more k-nearest neighbors. In some implementations, sampling the first plurality of points includes determining a Delaunay triangulation based on the one-ring neighborhood of points for each point in the first plurality of points. In some implementations, sampling the first plurality of points includes sampling the Delaunay triangulation determined for each point in the first plurality of points.

[0139] In some implementations, the first shape representation of the first type is a signed- distance function that includes a gradient. In some implementations, sampling the first plurality of points includes sampling each point in the first plurality of points volumetrically within a three-dimensional bounding box around a 0-isosurface. In some implementations, sampling the first plurality of points include projecting each point in the first plurality of points onto the 0-isosurface by translating each point in a direction of the gradient by an absolute value of its signed distance.

[0140] In some implementations, the first shape representation of the first type is a voxel. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, determining, by the processor, a surface corresponding to a boundary of a voxel grid of a respective point. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, determining, by the processor, a smooth surface of the three-dimensional shape by surface regularization of the surface corresponding to the boundary of the voxel grid of the respective point. In some implementations, sampling the first plurality of points includes, for each point in the first plurality of points, defining, by the processor, a quadrilateral face for the respective point based on the smooth surface of the three-dimensional shape. In some implementations, sampling theAttorney Docket No. RC-0395-01-WO first plurality of points includes, for each point in the first plurality of points, sampling, by the processor, the quadrilateral face for the respective point.

[0141] In some implementations, the generating the first common geometry' representation includes sampling a second plurality of points having a second distribution characteristic from an interior volume of the three-dimensional shape defined by the input geometry representation.

[0142] In some implementations, sampling the second plurality' of points includes sampling the second plurality’ of points within a three-dimensional bounding box of the three- dimensional shape. In some implementations, sampling the second plurality of points includes in response to determining that one or more points from the second plurality of points are located outside the three-dimensional bounding box of the three-dimensional shape, discarding the one or more points located outside the three-dimensional bounding box of the three- dimensional shape.

[0143] In some implementations, sampling the first plurality’ of points and the second plurality of points includes determining a Poisson disk radius based on a point from the first plurality of points and each point in the second plurality of points. In some implementations, sampling the first plurality of points and the second plurality of points includes adjusting the Poisson disk radius based on a target number of points and a surface area of the three- dimensional shape to control sampling resolution.

[0144] In some implementations, the generating the first common geometry representation include defining a connectivity graph that includes connections between individual points from the first plurality' of points and the second plurality' of points.

[0145] In some implementations, defining the connectivity graph includes determining a Delaunay tetrahedralization based on the first plurality’ of points and the second plurality of points. In some implementations, defining the connectivity’ graph includes retaining tetrahedron edges whose centroids he within the three-dimensional shape based on the Delaunay tetrahedralization. In some implementations, defining the connectivity graph includes determining pairwise connections between individual points from the first plurality of points and the second plurality7of points based on the retained tetrahedron edges.Attorney Docket No. RC-0395-01-WO

[0146] In some implementations, generating the first common geometry representation further includes associating each point from the first plurality of points with a feature vector. In some implementations, the feature vector for a point in the first plurality of points is a sixdimensional vector that includes a three-dimensional coordinate of the point and a three- dimensional normal vector. Block 702 may be followed by block 704.

[0147] At block 704, the first common geometry representation is input into a machine learning model that is included in a geometry representation-agnostic learning system (e g., the system 200 of FIG. 2) of a computing device. Block 704 may be followed by block 706.

[0148] At block 706, a second common geometry representation of the three-dimensional shape is generated using the machine learning model. In some implementations, the second common geometry' representation includes first geometry information contained in the first common geometry representation and one or more of geometry' identification of the three- dimensional shape, geometry’ segmentation of the geometry of the three-dimensional shape, or joint location for auto-rigging of the three-dimensional shape. Block 706 may be followed by block 708.

[0149] At block 708, an output geometry representation is generated by mapping the second common geometry' representation to the input geometry representation via interpolation. In some implementations, the output geometry representation includes second geometry information contained in the input geometry' representation and one or more of the geometry identification of the three-dimensional shape, the geometry segmentation of the geometry of the three-dimensional shape, or the joint location for auto-rigging of the three- dimensional shape. Block 708 may be followed by block 710.

[0150] At block 710, the machine learning model is trained based on one or more of the input geometry representation, the first common geometry- representation, the second common geometry’ representation, or the output geometry’ representation of the three-dimensional shape to improve an ability’ of the computing device to accurately identify geometry’ of the three- dimensional shape, perform segmentation of the geometry of the three-dimensional shape, and estimate joint location for auto-rigging of the three-dimensional shape, etc.

[0151] Hereinafter, a more detailed description of various computing devices that may be used to implement different devices and / or components illustrated in FIG. 1 is provided with reference to FIG. 8.Attorney Docket No. RC-0395-01-WO

[0152] FIG. 8 is a block diagram of an example computing device 800 which may be used to implement one or more features described herein, in accordance with some implementations. In one example, device 800 may be used to implement a computing device (e.g., 102, 1 10, etc. of FIG. 1), and perform appropriate operations as described herein. Computing device 800 can be any suitable computer system, server, or other electronic or hardware device. For example, the computing device 800 can be a mainframe computer, desktop computer, workstation, portable computer, or electronic device (portable device, mobile device, cell phone, smart phone, tablet computer, television, TV set top box, personal digital assistant (PDA), media player, game device, wearable device, etc.). In some implementations, device 800 includes a processor 802, a memoir 804, input / output (I / O) interface 806, and audio / video input / output devices 814 (e.g., display screen, touchscreen, display goggles or glasses, audio speakers, headphones, microphone, etc.).

[0153] Processor 802 can be one or more processors and / or processing circuits to execute program code and control basic operations of the device 800. A 'processor” includes any suitable hardware and / or software system, mechanism or component that processes data, signals or other information. A processor may include a system with a general-purpose central processing unit (CPU), multiple processing units, dedicated circuitry for achieving functionality, or other systems. Processing need not be limited to a particular geographic location or have temporal limitations. For example, a processor may perform its functions in “real-time,” “offline,” in a “batch mode,” etc. Portions of processing may be performed at different times and at different locations, by different (or the same) processing systems. A computer may be any processor in communication with a memory.

[0154] Memory 804 is typically provided in device 800 for access by the processor 802 and may be any suitable processor-readable storage medium, e.g., random access memory (RAM), read-only memory (ROM), electrical erasable read-only memory (EEPROM), flash memory, etc., suitable for storing instructions for execution by the processor, and located separate from processor 802 and / or integrated therewith. Memory 804 can store software operating on the computing device 800 by the processor 802, including an operating system 808, software application 810, and associated database 812. In some implementations, the software application 810 can include instructions that enable processor 802 to perform the functions described herein. Software application 810 may include some or all of the functionality used to perform geometry-representation agnostic learning. In someAttorney Docket No. RC-0395-01-WO implementations, one or more portions of software application 810 may be implemented in dedicated hardware such as an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), a machine learning processor, etc. In some implementations, one or more portions of software application 810 may be implemented in general purpose processors, such as a central processing unit (CPU) or a graphics processing unit (GPU). In various implementations, suitable combinations of dedicated and / or general purpose processing hardware may be used to implement software application 810.

[0155] For example, software application 810 stored in memory 804 may include instructions for performing geometry-representation agnostic learning and / or other functionality or software such as the virtual experience engine 104 and / or virtual experience application 112. The software application 810 and / or other executable computer-readable instructions stored in memory 804 can also be used to implement the virtual experience application 112, the virtual experience engine 104, the virtual experience 106, and / or other components depicted in and / or otherwise usable for the system architecture 100 of FIG. 1 . Any of software in memory 804 can alternatively be stored on any other suitable storage location or computer-readable medium. In addition, memory7804 (and / or other connected storage device(s)) can store instructions and data used in the features described herein. Memory 804 and any other type of storage (magnetic disk, optical disk, magnetic tape, or other tangible media) can be considered "storage" or "storage devices."

[0156] I / O interface 806 (which can correspond to the I / O interface 114 of FIG. 1) can provide functions to enable interfacing the computing device 800 with other systems and devices. For example, network communication devices, storage devices (e.g., memory' and / or data store 120), and input / output devices can communicate via interface 806. In some implementations, the I / O interface can connect to interface devices including input devices (keyboard, pointing device, touchscreen, microphone, camera, scanner, etc.) and / or output devices (display device, speaker devices, printer, motor, etc.).

[0157] For ease of illustration, FIG. 8 shows one block for each of processor 802, memory 804, I / O interface 806, operating system 808, software application 810, and database 812. These blocks may represent one or more processors or processing circuitries, operating systems, memories, I / O interfaces, applications, and / or software modules. In other implementations, device 800 may not have all of the components shown and / or may have otherAttorney Docket No. RC-0395-01-WO elements including other ty pes of elements instead of, or in addition to, those shown herein. While the online virtual experience server 102 are described as performing operations as described in some implementations herein, any suitable component or combination of components of online virtual experience server 102, or similar system, or any suitable processor or processors associated with such a system, may perform the operations described.

[0158] A user device can also implement and / or be used with features described herein. Example user devices can be computer devices including some similar components as the device 800 (e.g., processor(s) 802, memory7804, and I / O interface 806). An operating system, software and applications suitable for the client device can be provided in memory and used by the processor. The I / O interface for a client device can be connected to network communication devices, as w ell as to input and output devices (e.g., a microphone for capturing sound, a camera for capturing images or video, audio speaker devices for outputting sound, a display device for outputting images or video, or other output devices). A display device within the audio / video input / output devices 814, for example, can be connected to (or included in) the device 800 to display images pre- and post-processing as described herein, where such display device can include any suitable display device (e.g., an LCD, LED, or plasma display screen, CRT, television, monitor, touchscreen, 3-D display screen, projector, or other visual display device). Some implementations can provide an audio output device (e.g., voice output or synthesis that speaks text).

[0159] The methods, blocks, and / or operations described herein can be performed in a different order than shown or described, and / or performed simultaneously (partially or completely) with other blocks or operations, where appropriate. Some blocks or operations can be performed for one portion of data and later performed again (e.g., for another portion of data). Not all of the described blocks and operations need be performed in various implementations. In some implementations, blocks and operations can be performed multiple times, in a different order, and / or at different times in the methods.

[0160] In some implementations, some or all of the methods can be implemented on a system such as one or more client devices. In some implementations, one or more methods described herein can be implemented, for example, on a server system, and / or on both a server system and a client system. In some implementations, different components of one or more servers and / or clients can perform different blocks, operations, or other parts of the methods.Attorney Docket No. RC-0395-01-WO

[0161] One or more methods described herein (e.g., method 700) can be implemented by computer program instructions or code, which can be executed on a computer. For example, the code can be implemented by one or more digital processors (e.g., microprocessors or other processing circuitry), and can be stored on a computer program product including a non- transitory computer readable medium (e.g., storage medium), for example, a magnetic, optical, electromagnetic, or semiconductor storage medium, including semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memorv (RAM), a read-only memory (ROM), flash memory, a rigid magnetic disk, an optical disk, a solid-state memory drive, etc. The program instructions can also be contained in, and provided as, an electronic signal, for example in the form of software as a service (SaaS) delivered from a server (e.g., a distributed system and / or a cloud computing system). Alternatively, one or more methods can be implemented in hardware (logic gates, etc.), or in a combination of hardware and software. Example hardware can be programmable processors (e.g., field-programmable gate array (FPGA), complex programmable logic device), general purpose processors, graphics processors, application specific integrated circuits (ASICs), and the like. One or more methods can be performed as part of or component of an application running on the system, or as an application or software running in conjunction with other applications and operating system.

[0162] One or more methods described herein can be run in a standalone program that can be run on any type of computing device, a program run on a web browser, a mobile application C’app”) executing on a mobile computing device (e.g., cell phone, smart phone, tablet computer, wearable device (wristwatch, armband, jewelry, headwear, goggles, glasses, etc.), laptop computer, etc.). In one example, a client / server architecture can be used, e.g., a mobile computing device (as a client device) sends user input data to a server device and receives from the server the live feedback data for output (e.g., for display). In another example, computations can be split between the mobile computing device and one or more serv er devices.

[0163] Although the description has been described with respect to particular implementations thereof, these particular implementations are merely illustrative, and not restrictive. Concepts illustrated in the examples may be applied to other examples and implementations.

[0164] Note that the functional blocks, operations, features, methods, devices, and systems described in the present disclosure may be integrated or divided into different combinations of systems, devices, and functional blocks. Any suitable programming language andAttorney Docket No. RC-0395-01-WO programming techniques may be used to implement the routines of particular implementations. Different programming techniques may be employed (e.g., procedural or object-oriented). The routines may execute on a single processing device or multiple processors. Although the steps, operations, or computations may be presented in a specific order, the order may be changed in different particular implementations. In some implementations, multiple steps or operations shown as sequential in this specification may be performed at the same time.

Claims

Attomey Docket No. RC-0395-01-WOCLAIMSWhat is claimed is:

1. A computer-implemented method, comprising: generating, by a processor, a first common geometry representation of a three- dimensional shape based on an input geometry representation of the three-dimensional shape, wherein the input geometry' representation includes a first shape representation of a first ty pe, and wherein the first common geometry representation includes a second shape representation of a second type, different from the first type, wherein generating the first common geometry representation comprises: sampling, by the processor, a first plurality' of points having a first distribution characteristic from a surface of the three-dimensional shape defined by the input geometry representation; sampling, by the processor, a second plurality of points having a second distribution characteristic from an interior volume of the three-dimensional shape defined by the input geometry representation; and defining, by the processor, a connectivity graph that includes connections between individual points from the first plurality of points and the second plurality' of points; inputting, by the processor, the first common geometry representation into a machine learning model that is included in a geometry representation-agnostic learning system of a computing device; generating, by the processor, a second common geometry representation of the three- dimensional shape using the machine learning model, wherein the second common geometry representation includes a third shape representation of the second ty pe; generating, by the processor, an output geometry representation by mapping the second common geometry' representation to the input geometry representation via interpolation, wherein the output geometry' representation includes a fourth shape representation of the first type; and training, by the processor, the machine learning model based on one or more of the input geometry representation, the first common geometry' representation, the second common geometry' representation, or the output geometry' representation of the three-dimensional shape to improve an ability' of the computing device to accurately identify geometry' of the three-Attorney Docket No. RC-0395-01-WO dimensional shape, perform segmentation of the geometry of the three-dimensional shape, and estimate joint location for auto-rigging of the three-dimensional shape.

2. The computer-implemented method of claim 1, wherein: the second common geometry representation includes first geometry' information contained in the first common geometry representation and one or more of geometry identification of the three-dimensional shape, geometry segmentation of the geometry’ of the three-dimensional shape, or joint location for auto-rigging of the three-dimensional shape, and the output geometry’ representation includes second geometry information contained in the input geometry representation and one or more of the geometry identification of the three- dimensional shape, the geometry segmentation of the geometry of the three-dimensional shape, or the joint location for auto-rigging of the three-dimensional shape.

3. The computer-implemented method of claim 1, wherein the first shape representation of the first type is a mesh, and wherein sampling the first plurality’ of points comprises: sampling, by the processor, one or more points from within a triangle in the mesh for inclusion in the first plurality of points, wherein the one or more points are weighted by an area of the triangle in the mesh.

4. The computer-implemented method of claim 1, wherein the first shape representation of the first type is a point cloud, and wherein sampling the first plurality of points comprises: determining, by the processor, one or more k-nearest neighbors in the point cloud from the interior volume of the three-dimensional shape defined by the input geometry representation; determining, by the processor, a one-ring neighborhood of points for each point in the first plurality’ of points based on the one or more k-nearest neighbors; determining, by the processor, a Delaunay triangulation based on the one-ring neighborhood of points for each point in the first plurality of points; and sampling, by the processor, the Delaunay triangulation determined for each point in the first plurality of points.

5. The computer-implemented method of claim 1, wherein the first shape representation of the first type is a signed-distance function that includes a gradient, and wherein sampling the first plurality of points comprises:Attorney Docket No. RC-0395-01-WO sampling, by the processor, each point in the first plurality7of points volumetrically within a three-dimensional bounding box around a O-isosurface; and projecting, by the processor, each point in the first plurality of points onto the 0- isosurface by translating each point in a direction of the gradient by7an absolute value of its signed distance.

6. The computer-implemented method of claim 1, wherein the first shape representation of the first type is a voxel, and wherein sampling the first plurality of points comprises: for each point in the first plurality7of points, determining, by the processor, a surface corresponding to a boundary7of a voxel grid of a respective point; determining, by the processor, a smooth surface of the three-dimensional shape by surface regularization of the surface corresponding to the boundary7of the voxel grid of the respective point; defining, by the processor, a quadrilateral face for the respective point based on the smooth surface of the three-dimensional shape; and sampling, by the processor, the quadrilateral face for the respective point.

7. The computer-implemented method of claim 1, wherein sampling the second plurality of points comprises: sampling, by the processor, the second plurality7of points within a three-dimensional bounding box of the three-dimensional shape; and in response to determining that one or more points from the second plurality of points are located outside the three-dimensional bounding box of the three-dimensional shape, discarding, by the processor, the one or more points located outside the three-dimensional bounding box of the three-dimensional shape.

8. The computer-implemented method of claim 1, wherein sampling the first plurality7of points and the second plurality of points comprises: determining, by the processor, a Poisson disk radius based on a point from the first plurality7of points and each point in the second plurality of points; and adjusting, by the processor, the Poisson disk radius based on a target number of points and a surface area of the three-dimensional shape to control sampling resolution.Attorney Docket No. RC-0395-01-WO9. The computer-implemented method of claim 1. wherein defining the connectivity graph comprises: determining, by the processor, a Delaunay tetrahedralization based on the first plurality of points and the second plurality of points; retaining, by the processor, tetrahedron edges whose centroids lie within the three- dimensional shape based on the Delaunay tetrahedralization; and determining, by the processor, pairwise connections between individual points from the first plurality of points and the second plurality of points based on the retained tetrahedron edges.

10. The computer-implemented method of claim 1, wherein generating the first common geometry representation further comprises: associating, by the processor, each point from the first plurality of points w ith a feature vector, wherein the feature vector for a point in the first plurality of points is a six-dimensional vector that includes a three-dimensional coordinate of the point and a three-dimensional normal vector.

11. A non-transitory computer-readable medium with instructions stored thereon that, when executed by one or more hardware processors, cause the one or more hardware processors to perform or control performance of operations comprising: generating a first common geometry representation of a three-dimensional shape based on an input geometry representation of the three-dimensional shape, wherein the input geometry representation includes a first shape representation of a first type, and wherein the first common geometry representation includes a second shape representation of a second type, different from the first type, wherein generating the first common geometry representation comprises: sampling a first plurality of points having a first distribution characteristic from a surface of the three-dimensional shape defined by the input geometry representation; sampling a second plurality of points having a second distribution characteristic from an interior volume of the three-dimensional shape defined by the input geometry representation; and defining a connectivity graph that includes connections between individual points from the first plurality of points and the second plurality of points;Attorney Docket No. RC-0395-01-WO inputting the first common geometry representation into a machine learning model that is included in a geometry’ representation-agnostic learning system of a computing device; generating a second common geometry representation of the three-dimensional shape using the machine learning model, wherein the second common geometry representation includes a third shape representation of the second type; generating an output geometry representation by mapping the second common geometry representation to the input geometry representation via interpolation, wherein the output geometry representation includes a fourth shape representation of the first type: and training the machine learning model based on one or more of the input geometry representation, the first common geometry' representation, the second common geometry representation, or the output geometry representation of the three-dimensional shape to improve an ability of the computing device to accurately identify geometry of the three- dimensional shape, perform segmentation of the geometry' of the three-dimensional shape, and estimate j oint location for auto-rigging of the three-dimensional shape.

12. The non-transitory computer-readable medium of claim 11, wherein: the second common geometry representation includes first geometry information contained in the first common geometry' representation and one or more of geometry’ identification of the three-dimensional shape, geometry segmentation of the geometry' of the three-dimensional shape, or joint location for auto-rigging of the three-dimensional shape, and the output geometry representation includes second geometry information contained in the input geometry' representation and one or more of the geometry’ identification of the three- dimensional shape, the geometry segmentation of the geometry' of the three-dimensional shape, or the joint location for auto-rigging of the three-dimensional shape.

13. The non-transitory computer-readable medium of claim 1 1, wherein the first shape representation of the first ty pe is a mesh, and wherein sampling the first plurality’ of points comprises: sampling one or more points from within a triangle in the mesh for inclusion in the first plurality of points, wherein the one or more points are weighted by an area of the triangle in the mesh.Attorney Docket No. RC-0395-01-WO14. The non-transitory computer-readable medium of claim 11, wherein the first shape representation of the first type is a point cloud, and wherein sampling the first plurality of points comprises: determining one or more k-nearest neighbors in the point cloud from the interior volume of the three-dimensional shape defined by the input geometry' representation; determining a one-ring neighborhood of points for each point in the first plurality of points based on the one or more k-nearest neighbors; determining a Delaunay triangulation based on the one-ring neighborhood of points for each point in the first plurality7of points; and sampling the Delaunay triangulation determined for each point in the first plurality' of points.

15. The non-transitory computer-readable medium of claim 11, wherein the first shape representation of the first ty pe is a signed-distance function that includes a gradient, and wherein sampling the first plurality' of points comprises: sampling each point in the first plurality of points volumetrically within a three- dimensional bounding box around a O-isosurface; and projecting each point in the first plurality' of points onto the O-isosurface by translating each point in a direction of the gradient by an absolute value of its signed distance.

16. The non-transitory computer-readable medium of claim 1 1, wherein the first shape representation of the first ty pe is a voxel, and wherein sampling the first plurality of points comprises: for each point in the first plurality of points determining a surface corresponding to a boundary of a voxel grid of a respective point; determining a smooth surface of the three-dimensional shape by surface regularization of the surface corresponding to the boundary of the voxel grid of the respective point; defining a quadrilateral face for the respective point based on the smooth surface of the three-dimensional shape; and sampling the quadrilateral face for the respective point.Attorney Docket No. RC-0395-01-WO17. The non-transitory computer-readable medium of claim 11, wherein sampling the second plurality of points comprises: sampling the second plurality of points within a three-dimensional bounding box of the three-dimensional shape; and in response to determining that one or more points from the second plurality7of points are located outside the three-dimensional bounding box of the three-dimensional shape, discarding the one or more points located outside the three-dimensional bounding box of the three-dimensional shape.

18. The non-transitory computer-readable medium of claim 11, wherein sampling the first plurality of points and the second plurality of points comprises: determining a Poisson disk radius based on a point from the first plurality of points and each point in the second plurality of points; and adjusting the Poisson disk radius based on a target number of points and a surface area of the three-dimensional shape to control sampling resolution.

19. The non-transitory computer-readable medium of claim 11, wherein defining the connectivity7graph comprises: determining a Delaunay tetrahedralization based on the first plurality of points and the second plurality of points; retaining tetrahedron edges whose centroids lie within the three-dimensional shape based on the Delaunay tetrahedralization; and determining pairwise connections between individual points from the first plurality7of points and the second plurality of points based on the retained tetrahedron edges.

20. A computing device, comprising: one or more hardware processors; and a non-transitory computer readable medium coupled to the one or more hardware processors, with instructions stored thereon, that when executed by the one or more hardware processors, cause the one or more hardware processors to perform or control performance of operations comprising: generating a first common geometry representation of a three-dimensional shape based on an input geometry7representation of the three-dimensional shape, wherein the input geometry representation includes a first shape representation of a first type, and wherein theAttorney Docket No. RC-0395-01-WO first common geometry representation includes a second shape representation of a second ty pe, different from the first type, wherein generating the first common geometry representation comprises: sampling a first plurality of points having a first distribution characteristic from a surface of the three-dimensional shape defined by the input geometry7representation; sampling a second plurality of points having a second distribution characteristic from an interior volume of the three-dimensional shape defined by the input geometry representation; and defining a connectivity' graph that includes connections between individual points from the first plurality of points and the second plurality of points; inputting the first common geometry representation into a machine learning model that is included in a geometry7representation-agnostic learning system of a computing device; generating a second common geometry representation of the three-dimensional shape using the machine learning model, wherein the second common geometry representation includes a third shape representation of the second type; generating an output geometry7representation by mapping the second common geometry' representation to the input geometry representation via interpolation, wherein the output geometry representation includes a fourth shape representation of the first ty pe; and training the machine learning model based on one or more of the input geometry representation, the first common geometry representation, the second common geometry representation, or the output geometry representation of the three-dimensional shape to improve an ability7of the computing device to accurately identify geometry of the three- dimensional shape, perform segmentation of the geometry of the three-dimensional shape, and estimate joint location for auto-rigging of the three-dimensional shape.

Images