Generating surfaces with arbitrary topology using signed distance fields.

Abstract

Various embodiments described herein include methods, devices, and systems for generating an object mesh. In some embodiments, the method includes obtaining a trained classifier and input observations of a 3D object. The method further includes generating a tripolar signed distance field from the input observations using the trained classifier. The method also includes generating an output mesh of the 3D object from the tripolar signed distance field, and generating a representation of the 3D object from the output mesh.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of and claims priority to U.S. patent application Ser. No. 17 / 713,129, filed April 4, 2022, entitled "Generating Surfaces with Arbitrary Topology Using Signed Distance Fields," which is incorporated by reference in its entirety.

[0002] This relates generally to object surface generation, and in particular to using signed distance fields for object mesh generation. [Background technology]

[0003] Geometric three-dimensional (3D) representation of objects is important for many tasks in computer vision and computer graphics. Over the past few years, various approaches to 3D representation have been explored, including meshes, voxels, and point clouds. However, most approaches to generating 3D representations are limited to closed surfaces because they require the partitioning of the space into an inner and outer part. Summary of the Invention [Problem to be solved by the invention]

[0004] The choice of representation for 3D shapes and surfaces is critical for effective 3D learning. In recent years, the advent of neural implicit functions (NIFs) has led to advances in learning-based 3D reconstruction and modeling. Traditional NIF methods are built on signed distance functions (SDFs). These methods train deep neural networks to classify consecutive 3D locations as inside or outside a surface via occupancy prediction or regression of the SDF. However, they can only model closed surfaces, which support input-output tests for level surface extraction. With unsigned distance functions (UDFs), it is possible to learn open surfaces from point clouds. However, instantiating unsigned distance fields into explicit meshes is computationally challenging and prone to artifacts. [Means for solving the problem]

[0005] In some embodiments of the present disclosure, a new learnable implicit representation called tripolar signed distance function (3PSDF) is used to represent highly complex geometric shapes, including both closed and open surfaces, with high fidelity. 3PSDF inherits the advantages of SDF, making the learning component an easy classification task to train. 3PSDF is also compatible with classical isosurface extraction techniques, e.g., marching cubes algorithm, to achieve efficient field-to-mesh transformation. An important aspect of the 3PSDF approach is the inclusion of non-directional signs (e.g., NULL signs) in the traditional binary signed distance function. For example, points with non-directional signs are assigned a not-a-number (NaN) value to prevent the formation of a decision boundary between NaN values ​​and their neighbors. By appropriately distributing the non-directional labels over the space, surfaces with arbitrary topology can be cast. As an example, a neural network is trained to classify consecutive points into three categories: inside, outside, and non-directional. The resulting labels can be converted back to the 3PSDF using a simple mapping function to obtain the mesh result.

[0006] Thus, in one aspect, some embodiments include a method executed on a computing system having a memory and one or more processors, the method including: (i) obtaining a trained classifier, (ii) obtaining input observations of a 3D object, (iii) generating a tripolar signed distance field from the input observations using the trained classifier, (iv) generating an output mesh of the 3D object from the tripolar signed distance field, and (v) generating a representation of the 3D object from the output mesh.

[0007] In other aspects, some embodiments include a computing system having one or more processors, a memory, and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for (i) obtaining a trained classifier, (ii) obtaining input observations of a 3D object, (iii) generating a tripolar signed distance field from the input observations using the trained classifier, (iv) generating an output mesh of the 3D object from the tripolar signed distance field, and (v) generating a representation of the 3D object from the output mesh.

[0008] Thus, systems and devices are provided with methods for object surface generation, thereby increasing the effectiveness, efficiency, and user satisfaction of such systems and devices. Such systems, devices, and methods, optionally, complement or replace conventional systems, devices, and methods for object surface generation.

[0009] For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the following drawings, in which like reference numerals refer to corresponding parts throughout. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example architecture for object mesh generation, according to some embodiments. [Diagram 2] FIG. 2 illustrates an example architecture for classifier training, according to some embodiments. [Diagram 3] FIG. 2 illustrates an example signed distance field unit cell in accordance with some embodiments. [Figure 4A] FIG. 1 illustrates an exemplary application of a signed distance field in accordance with some embodiments. [Figure 4B] FIG. 1 illustrates an exemplary application of a signed distance field in accordance with some embodiments. [Figure 4C] FIG. 1 illustrates an exemplary application of a signed distance field in accordance with some embodiments. [Figure 4D] FIG. 1 illustrates an exemplary application of a signed distance field in accordance with some embodiments. [Figure 5A] FIG. 2 illustrates an example input observation and output mesh, in accordance with some embodiments. [Figure 5B] FIG. 2 illustrates an example input observation and output mesh, in accordance with some embodiments. [Figure 6A] FIG. 2 illustrates an example input observation and output mesh, in accordance with some embodiments. [Figure 6B] FIG. 2 illustrates an example input observation and output mesh, in accordance with some embodiments. [Figure 6C] FIG. 2 illustrates an example input observation and output mesh, in accordance with some embodiments. [Figure 7] FIG. 1 is a block diagram illustrating an exemplary computing system, according to some embodiments. [Figure 8A] 1 is a flowchart illustrating an example method for generating an output mesh of a 3D object, in accordance with some embodiments. [Figure 8B]1 is a flowchart illustrating an example method for generating an output mesh of a 3D object, in accordance with some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Reference will now be made in detail to the embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it will be apparent to those skilled in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments.

[0012] One approach for generating an output mesh of a 3D object uses Neural Distance Fields (NDF), a neural network-based model that predicts the Unsigned Distance Field (UDF) of an arbitrary 3D shape using a sparse point cloud as input observations. For a point P, NDF directly regresses the distance to the surface with a learned function whose zero level set represents the surface.

[0013] However, training with UDFs, as required by NDF methods, is difficult. Some UDF-based training requires extracting a point cloud or mesh, or directly rendering the implicit surface onto the image, which requires finding its zero level set. Traditional methods of finding the zero level set by detecting the transition from inside to outside and vice versa (e.g., marching cubes and volume rendering) are not compatible with UDFs. Alternatively, NDF methods may infer a continuous unsigned distance field of the target object. Then, based on the UDF, the NDF method may compute a gradient flow of the inferred UDF. Furthermore, dense 3D sampling points may be generated randomly in the 3D space. By using the gradient flow, the sample points can be pushed onto the surface. Finally, the NDF method may use a ball-turning algorithm to convert the dense point cloud into a mesh.

[0014] Thus, there are challenges when using NDF techniques. For example, converting the unsigned distance field generated by the NDF to a mesh is a time-consuming process that can introduce numerous artifacts. To convert the UDF to a mesh, the NDF technique may require the generation of a dense point cloud and leverage the gradient field of the UDF to repeatedly push the points onto the target surface. Such a process is vulnerable to complex gradient landscapes, e.g., parts with many details, and can get stuck in local minima. In addition, the reconstruction of a mesh from the UDF may require the use of a ball-turning (BP) algorithm, which has several shortcomings. For example, the BP algorithm is sensitive to the input ball radius. A slightly larger or smaller radius can result in imperfect meshing results. In addition, the BP algorithm tends to generate self-intersecting and disconnected surface patches with inconsistent normals, even for surfaces of moderate complexity. Furthermore, the BP algorithm is time-consuming, especially when dealing with dense point clouds.

[0015] Another challenge with the NDF approach is that learning UDFs is a regression task, rather than a classification task (as for SDFs), making training more complex. Another challenge with the NDF approach is that it only applies to the task of point cloud completion (e.g., the input for NDF is limited to point clouds). Therefore, many 3D learning tasks, such as single or multi-view 3D reconstruction, cannot be performed with the NDF approach.

[0016] Another approach for generating an output mesh of a 3D object uses a deep implicit surface point prediction network. This approach presents a class of implicit representations called closest surface point (CSP) representations. Instead of directly regressing the unsigned distance field of the target surface, CSP infers the closest point on the target shape to the query point. After obtaining the closest surface point, the unsigned distance to the surface can be calculated as the distance between the query point and the surface point. With additional information such as the closest surface point, this approach simplifies the calculation of differential geometric properties such as tangent planes and surface normals. The CSP approach may use a volume encoder and a shape decoder. For example, the point cloud is input to a volume encoder to obtain a feature volume. The shape decoder is tuned based on the feature volume to obtain the closest surface point of the query point.

[0017] Unlike NDF, which converts UDF to densely sampled points using gradient fields and then converts the points to a mesh using the BPA algorithm, the CSP method leverages a multi-resolution surface extraction scheme and can directly use the marching cubes algorithm with a small positive threshold to obtain the final mesh. Although it requires fewer steps than the NDF method and produces a smoother surface, the CSP method also cannot obtain an open surface with a single-layer mesh. The CSP method has similar limitations to the NDF method. For example, learning is a regression task rather than a classification task (as for SDF), making training more complex. In addition, the CSP method also applies only to the task of point cloud completion.

[0018] The embodiments described herein address the limitations and challenges presented by the NDF and CSP approaches. For example, the three-polar signed distance function (3PSDF) implicit representation can model complex 3D shapes, including both closed and open surfaces, without lengthy or complicated point-to-mesh conversion. In addition, the 3PSDF approach can directly generate single-layer open surfaces in mesh form by applying the marching cubes algorithm (e.g., in a similar manner to converting a signed distance field (SDF) to a mesh). By extracting single-layer open surfaces, the 3PSDF does not introduce artifacts during the field-to-mesh conversion. In addition, the 3PSDF approach can use classification network learning, for example, the 3PSDF approach infers which category a query point belongs to (e.g., inside, outside, or null). This can significantly reduce the learning difficulty of the 3PSDF approach compared to the NDF or CSP approaches. Furthermore, the 3PSDF approach can handle broader and much more difficult tasks, such as 3D shape reconstruction from a single image.

[0019] According to some embodiments, the 3PSDF approach can represent highly complex 3D shapes, including both closed and open surfaces, with high fidelity. In addition, the 3PSDF approach inherits the advantages of the SDF approach, such as making learning a classification task (easy to train) and being compatible with isosurface extraction techniques (e.g., marching cubes algorithm) to achieve efficient field-to-mesh conversion. An exemplary learning method for the 3PSDF approach is based on a classification neural network that classifies query points into three categories: inside, outside, and null. The obtained labels can be returned to the 3PSDF using a mapping function to obtain mesh results. Therefore, the 3PSDF approach can be used in tasks related to 3D modeling and reconstruction, such as shape auto-encoding, point cloud completion, single-view 3D shape reconstruction, and multi-view 3D shape reconstruction.

[0020] Referring now to the drawings, Figure 1 illustrates an architecture 100 for object mesh generation according to some embodiments. The architecture 100 includes a trained classifier 106 and a field-to-mesh transformer 110. According to some embodiments, the trained classifier 106 takes the sampling point template 102 and input data 104 (e.g., input observations) and generates a signed distance field 108. The field-to-mesh transformer 110 takes the signed distance field 108 and generates an object mesh 112. In some embodiments, the trained classifier 106 includes multiple perceptual layers. In some embodiments, the trained classifier 106 includes multiple 1D convolutional layers.

[0021] For example, the architecture 100 takes regular sampling points along with other input observations used at training time and infers a tripolar signed distance value for each query sampling point. The regular sampling points along with the tripolar signed distance values ​​at each point location form a generated tripolar signed distance field 108, which can be converted to an output mesh using conventional field-to-mesh techniques such as the Marching Cubes (MC) algorithm.

[0022] In some embodiments, once the classifier is trained, each query point can be labeled with a prediction. For example, to extract an isosurface, the inferred discrete labels are converted to a 3PSDF representation (e.g., points with labels 0, 1, and 2 are assigned SDF values ​​as -1, 1, and NaN, respectively). The reconstructed surface can then be extracted as a zero-level surface.

[0023] In some embodiments, the isosurface represented by the 3PSDF is extracted directly using the marching cubes algorithm. During extraction, null values ​​prevent the MC algorithm from extracting valid isosurfaces at locations that do not contain shapes. In some embodiments, in the vicinity of the target surface, isosurface extraction is performed in the same way as for conventional signed distance fields. In some embodiments, after the MC calculation, NaN vertices and faces generated by the null cubes are removed, and the remaining vertices and faces serve as the meshing result.

[0024] FIG. 2 illustrates an architecture 200 for classifier training, according to some embodiments. According to some embodiments, the architecture 200 includes an octree builder 204, a signed distance value builder 206, and a classifier 210. In the example of FIG. 2, the octree builder 204 takes shape data 202 and generates a cell grid 205 of the shape data 202. The signed distance value builder 206 takes the cell grid 205 of the shape data 202 and generates sampling points 207. The classifier 210 is trained using the sampling points 207 and input data 208. In some embodiments, the classifier 210 is a ternary classifier.

[0025] In some embodiments, the shape data 202 includes a plurality of 3D shapes. In some embodiments, the octree builder 204 generates adaptive octree cells of the 3D shapes for use in computing the 3-pole signed distance field. In some embodiments, the corner points of the octree cells are used as input sampling points 207. In some embodiments, the input data 208 includes images and point clouds. For example, in the case of single-view 3D reconstruction, the sampling points 207 include 3D sampling points with ground truth 3PSDF values, and the input data 208 includes corresponding single-view images. In some embodiments, the input data 208 includes sparse point clouds. In this manner, the classifier 210 is trained to classify, for each input sampling point, the category to which it belongs (e.g., inside, outside, or NaN).

[0026] Although the introduction of null codes provides the flexibility to eliminate undesirable surfaces, NaN values ​​may not be used for some calculations, including computing gradients required to update deep neural networks. To circumvent this issue, instead of applying a network to directly regress the signed distance values, the training of the 3PSDF is formulated as a classification problem according to some embodiments.

[0027] In some embodiments, the 3D function is approximated with a neural network that infers pointwise labels, i.e., {in, out, null}. The label semantics can be expressed using discrete numbers without loss of generality. In some embodiments, the classifier 210 uses a mapping function o:R 3 → We learn {0,1,NaN}, where the labels {0,1,NaN} represent inside, outside, and null, respectively.

[0028] In some embodiments, the classifier 210 (e.g., a network) is trained based on input for a downstream task (e.g., 3D reconstruction) based on observations of objects (e.g., point clouds or images). Thus, the classifier 210 is trained based on the sampling points 207 (e.g., points p∈R 3 In addition to the coordinates of (p,x), the mapping function also takes as input input data 208 (e.g., observations of object x∈X). In some embodiments, the mapping function is a neural network Φ θ It is parameterized by

number

[0029] In some embodiments, the classifier 210 is trained using a batch of point samples and a neural network Φ θ( p,x) parameters θ. For example, for the i-th sample in the training batch, N points are sampled, where p ij ∈R 3, j = 1, …, N. The mini-batch loss L B is shown in Equation 2 below.

number

[0030] In some embodiments, each training sample is embedded with a 512-dimensional latent code, and the classifier 210 is trained to reconstruct the 3D shape from the embedding.

[0031] In some embodiments, the 3PSDF calculation is performed cell-by-cell. Thus, in some scenarios, it is preferable to reduce or minimize the geometric details and open surfaces of individual cells. Thus, in some embodiments, the octree builder 204 only subdivides a cell if it intersects with the input shape. As the subdivision depth increases, the surface complexity contained in each local cell decreases, which can improve the approximation accuracy. Regions that do not contain shapes are not further divided, so a trade-off is obtained between computational complexity and reconstruction accuracy.

[0032] 3 is a diagram illustrating signed distance field unit cells 300, 320, and 340, according to some embodiments. 2D signed distance field unit cell 300 includes points 302, 304, 306, and 308. Points 302 and 306 have corresponding values ​​of 0.5 and 0.2, respectively, and thus map to the outside of the surface, according to some embodiments. Points 304 and 308 have corresponding values ​​of -0.3 and -0.6, respectively, and thus map to the inside of the surface, according to some embodiments. 2D signed distance field unit cell 320 includes surface 322 extracted from points 302, 304, 306, and 308. 2D signed distance field unit cell 340 is the same as 2D signed distance field unit cell 300, except that point 306 in 2D signed distance field unit cell 300 is replaced with point 342 in 2D signed distance field unit cell 340. Point 342 has a value of NaN that is mapped to a null code according to some embodiments. Including point 342 in the 2D signed distance field unit cell 340 prevents extraction of a surface according to some embodiments (e.g., if any point in the unit cell has a NaN value, then no surface is extracted).

[0033] The watertight 3D shape can be implicitly represented by a signed distance function. 3 Given p, a deep neural network is applied to calculate the occupancy of p as f(p):R 3 →Predict as [0,1] or set the SDF to f(p):R 3 →R. In this way, the formation of closed surfaces occurs when both positive and negative signs are present in the space.

[0034] For signed distance functions with the ability to represent open surfaces, a third non-directional pole (e.g., a null sign) is incorporated into the traditional signed distance field. For example, no isosurfaces are formed at the bisectors of either the positive / null or negative / null pairs. In this way, the null sign acts as a surface remover that prevents undesired surfaces, thus allowing the flexibility to cast any topology, including those with open surfaces. Incorporating a non-directional pole allows for the 3D point p∈R to be represented in a 3D space. 3 For p, in addition to the continuous signed distance, p can also be mapped to a null value, i.e., Ψ(p):R 3 →{R,nan}. Thus, given an input surface S, a mapping function Ψ can be learned to minimize the inter-surface distance, as shown in Equation 3 below.

number

[0035] 4A-4D are diagrams illustrating the application of an exemplary signed distance field, according to some embodiments. FIG. 4A shows a space 402 with a 2D shape 404 (e.g., a unicorn's head). As shown in FIG. 4A, the shape 404 is open (e.g., has an open surface) at the bottom. FIG. 4B shows an SDF applied to the space 402, where a diagonal line 410 represents an interior position and a horizontal line 412 represents an exterior position. Since an SDF can only represent closed surfaces, the shape 404 is closed in FIG. 4B by adding a line 406 at the bottom.

[0036] Figure 4C shows an octree-based subdivision applied to space 402 resulting in unit cell 420. Figure 4C further shows a 3PSDF applied to space 402, with diagonal lines 424 representing interior positions, horizontal lines 422 representing exterior positions, and empty unit cells (e.g., no lines) representing null (non-directional) positions.

[0037] FIG. 4D shows a unit cell 426 (from FIG. 4C) with an open surface (e.g., a partial line) 432. In some embodiments, applying the 3PSDF to the unit cell 426 results in approximation errors because no surfaces are formed. In some embodiments, the approximation errors resulting from the open surfaces in the unit cell are reduced or minimized by applying a finer spatial resolution (e.g., continuing to subdivide the space 402 until the unit cell has no open surfaces). FIG. 4D further shows a unit cell 428 (from FIG. 4C) with a complex (e.g., nonlinear) surface 434. According to some embodiments, the unit cell 428 includes a surface normal 438 that is used to determine the inner and outer positions.

[0038] For non-tight surfaces (e.g., without a closed boundary), the input / output test of the query point is insufficient. In some embodiments, the surface normal is used to determine the sign of the distance of the query point. In some embodiments, the space is decomposed into a grid of local cells. As shown in FIG. 4C, each cell 420 (C i ), if it does not contain a surface, the cell space is set as a null region, and C i Any sample point p inside i has a NaN distance to the target surface S as shown in Equation 4 below. Ψ(p i ,S)=P i ∈C i And C i If ∩S=φ, then nan Equation 4: NaN distance

[0039] A local cell C surrounding a surface patch Si i If p is a query point, i ∈C i Given p on Si, i The closest point q of i is determined. First, q i The surface normal n(q i ) as the vector

number

number

number

[0040] In some scenarios, a finer decomposition of the 3D space results in cells containing geometric shapes being distributed around the surface of the object, while null cells occupy most of the space. This differs from traditional signed distance fields (e.g., as shown in Figure 4B), where the entire space is filled with distances of either positive or negative sign. Thus, 3PSDFs may better reflect the nature of 3D surfaces of arbitrary topology in terms of the high sparsity of surface occupancy.

[0041] 5A-5B are diagrams illustrating example input observations and output meshes, according to some embodiments. In particular, FIGS. 5A-5B show a visual comparison of point cloud completion on non-watertight data with open surfaces. FIG. 5A shows an input observation 502 (e.g., a point cloud for an automobile) and a corresponding 3PSDF mesh 504 generated from the input observation. The 3PSDF mesh 504 includes an outer surface 506 and an inner surface 508. FIG. 5A further shows a ground truth mesh 510 (e.g., the mesh from which the input observation 502 was generated) having an outer surface 512 and an inner surface 514. FIG. 5B shows an input observation 520 (e.g., a point cloud for a convertible car), a corresponding 3PSDF mesh 522 generated from the input observation, and a ground truth mesh 524 (e.g., the mesh from which the input observation 520 was generated).

[0042] 6A-6C are diagrams illustrating example input observations and output meshes, according to some embodiments. FIG. 6A illustrates an image 602 of an airplane (e.g., a single image input observation), a corresponding 3PSDF mesh 604, and a ground truth mesh 606. In particular, FIG. 6A illustrates a different perspective of the airplane in image 602 using 3PSDF mesh 604. FIG. 6B illustrates an image 608 of a car, a corresponding 3PSDF mesh 610, and a ground truth mesh 612. In particular, FIG. 6B illustrates a different perspective of the car in image 608 using 3PSDF mesh 610. FIG. 6C illustrates an image 614 of a chair, a corresponding 3PSDF mesh 616, and a ground truth mesh 618. In particular, FIG. 6C illustrates a different perspective of the chair using 3PSDF mesh 616.

[0043] 7 is a block diagram illustrating a computing system 700, according to some embodiments. Various examples of computing systems 700 include desktop computers, laptop computers, tablet computers, server systems, and other computing devices having a processor capable of executing a classifier 722. The computing system 700 typically includes one or more processing units / cores (CPUs) 702 for executing modules, programs, and / or instructions stored in memory 714, thereby performing processing operations, one or more network or other communication interfaces 704, the memory 714, and one or more communication buses 712 for interconnecting these components. The communication bus 712 may include circuitry for interconnecting and controlling communications between the system components.

[0044] Computing system 700 optionally includes a user interface 706 comprising a display device 708 and one or more input devices or mechanisms 710. In some embodiments, the input devices include a keyboard and / or a mouse. In some embodiments, the input devices include a "soft" keyboard that is optionally displayed on the display device 708 and allows a user to "press keys" that are displayed on the display 708. In some embodiments, the display 708 and input devices 710 include a touch screen display (also referred to as a touch sensitive display).

[0045] In some embodiments, memory 714 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices. In some embodiments, memory 714 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some embodiments, memory 714 includes one or more storage devices located remotely from CPU 702. Memory 714, or alternatively a non-volatile memory device within memory 714, is or includes a non-transitory computer-readable storage medium. In some embodiments, memory 714, or the computer-readable storage medium of memory 714, stores the following programs, modules, and data structures, or a subset thereof: An operating system 716 that handles various basic system services and includes instructions for performing hardware-dependent tasks; A communications module 718 used to connect the computing system 700 to one or more communications network interfaces 704 (wired or wireless) and other computers and devices via one or more communications networks, such as the Internet, other wide area networks, local area networks, metropolitan area networks, etc.; one or more applications 720 (e.g., word processors, media players, web browsers, and communications platforms) that perform a particular task or set of tasks for a user; A classifier 722 (e.g., classifier 106) for determining a surface function of the input data. In some embodiments, the classifier 722 includes one or more of the following neural networks: A mesh generator 724 (e.g., the field-to-mesh converter 110) for generating an object mesh from the signed distance field. In some embodiments, the mesh generator 724 includes a marching cubes algorithm and / or a marching tetrahedra algorithm, a sampling point generator 726 (e.g., the octree builder 204 and the signed distance value builder 206) for generating sampling points for use with the input data; and One or more databases 730 for use by the application 720, the classifier 722, the mesh generator 724, and / or the sampling point generator 726. According to some embodiments, the one or more databases 730 include training data 732, sampling point templates 734, and / or input observations 736.

[0046] Each of the above identified executable modules, applications, or sets of procedures may be stored in one or more of the memory devices mentioned above and correspond to a set of instructions for performing the functions described above. The above identified modules or programs (i.e., sets of instructions) need not be implemented as separate software programs, procedures, or modules, and thus various subsets of these modules may be combined or otherwise rearranged in various embodiments. In some embodiments, memory 714 stores a subset of the above identified modules and data structures. Additionally, memory 714 may store additional modules or data structures not described above.

[0047] 7 illustrates a computing system 700, which is intended more as a functional description of various features that may be present than as a structural schematic of the embodiments described herein. In practice, and as will be recognized by those skilled in the art, items shown separately may be combined and some items may be separated.

[0048] 8A-8B are flow charts illustrating a method 800 for generating an output mesh of a 3D object, according to some embodiments. The method 800 is performed on a computing system (e.g., computing system 700) having one or more processors and memory. In some embodiments, the memory stores one or more programs configured to be executed by the one or more processors.

[0049] The computing system obtains 802 a trained classifier (e.g., trained classifier 106). In some embodiments, the trained classifier is obtained through a training process (e.g., as shown in FIG. 2).

[0050] In some embodiments, a classifier is trained 804 using the set of input sampling points and corresponding input training observations. For example, the classifier 210 is trained using the sampling points 207 and the input data 208. In some embodiments, the classifier 210 is the trained classifier 106 after the training process has been performed.

[0051] In some embodiments, the set of input sampling points is generated by applying an octree construction to the input shape (806). For example, the sampling points 207 are generated via the octree constructor 204 and the signed distance value constructor 206.

[0052] In some embodiments, a classifier is trained to learn a respective surface function for the set of input shapes 808. For example, the classifier 210 is trained to learn a surface function for the shape data 202 (e.g., the mapping function of Equation 1).

[0053] In some embodiments, the classifier (e.g., classifier 722) is a classification neural network (e.g., neural network Φ θ ) (810). In some embodiments, the classifier includes multiple perceptual layers. In some embodiments, the classifier includes multiple 1D convolutional layers.

[0054] A computing system obtains (812) an input observation of a 3D object (e.g., input data 104). In some embodiments, the input observation includes one or more open surfaces (814) (e.g., as shown in FIG. 6A). In some embodiments, the input observation is (816) point cloud data (e.g., point cloud data 502 or 520). In some embodiments, the input observation is (818) an image (e.g., image 602, 608, or 614).

[0055] The computing system uses the trained classifier to generate a three-polar signed distance field (e.g., signed distance field 108) from the input observations (820). In some embodiments, the computing system obtains a sampling point template (e.g., sampling point template 102), and the three-polar signed distance field is generated (822) using the sampling point template. In some embodiments, the three-polar signed distance field includes a three-polar signed distance value for each sampling point in the sampling point template (824).

[0056] In some embodiments, the computing system assigns (826) to each point of the plurality of points a value indicating whether the point is inside the surface (e.g., as shown in FIG. 4C ), outside the surface, or undefined.

[0057] The computing system generates 828 an output mesh of the 3D object from the tripolar signed distance field (e.g., using the mesh transformer 724). In some embodiments, the output mesh is generated 834 from the tripolar signed distance field using a marching cubes algorithm.

[0058] In some embodiments, the computing system extracts (830) a surface only between a set of inner and outer values. For example, the system does not extract a surface between signed points and null / NaN points, as shown in Figure 3. In some embodiments, the computing system generates (832) one or more open surfaces for the 3D object (e.g., as shown in Figures 5A-5B).

[0059] The computing system generates (836) a representation of the 3D object from the output mesh (e.g., displayed on the display 708). In some embodiments, the computing system generates (838) a 2D view of the 3D object on a display device (e.g., a perspective view as shown in FIGS. 6A-6C). In some embodiments, the computing system generates (840) a representation of the 3D object in a synthetic reality environment.

[0060] With these principles in mind, we now turn to specific embodiments.

[0061] (A1) In one aspect, some embodiments include a method (e.g., method 800) performed on a computing system (e.g., computing system 700) having a memory and one or more processors. The method includes (i) obtaining a trained classifier (e.g., classifier 106), (ii) obtaining input observations of a 3D object (e.g., input data 104), (iii) generating a three-polar signed distance field (e.g., signed distance field 108) from the input observations using the trained classifier, (iv) generating an output mesh of the 3D object (e.g., object mesh 112) from the three-polar signed distance field, and (v) generating a representation of the 3D object from the output mesh.

[0062] (A2) In some embodiments of A1, the method further includes obtaining a sampling point template (e.g., sampling point template 102), and the tripolar signed distance field is generated using the sampling point template. In some embodiments, the sampling point template includes a regular set (e.g., a grid) of sampling points for the input observations.

[0063] (A3) In some embodiments of A1 or A2, the tripolar signed distance field includes a tripolar signed distance value (eg, 1, -1, or NaN) for each sampling point in the sampling point template.

[0064] (A4) In some embodiments of any of A1-A3, generating the tripolar signed distance field from the input observations includes assigning to each point of the plurality of points a value indicating whether the point is inside the surface, outside the surface, or undefined. For example, a negative value corresponds to a point inside the surface, a positive value corresponds to a point outside the surface, and a not-a-number (NaN) or null value corresponds to an undefined point.

[0065] (A5) In some embodiments of A4, generating the output mesh includes extracting a surface from between a set of only inner and outer values ​​(eg, as shown in FIG. 3).

[0066] (A6) In some embodiments of any of A1-A5, the input observation includes one or more open surfaces (eg, as shown in FIGS. 5A-5B).

[0067] (A7) In some embodiments of any of A1-A6, generating an output mesh of the 3D object from the tripolar signed distance field includes generating one or more open surfaces for the 3D object.

[0068] (A8) In some embodiments of any of A1-A7, the input observations are point cloud data (eg, point cloud data 502 or 520).

[0069] (A9) In some embodiments of any of A1-A7, the input observation is an image (eg, a 2D image such as those shown in FIGS. 6A-6C).

[0070] (A10) In some embodiments of any of A1-A9, the output mesh is generated from the tripolar signed distance field using a marching cubes algorithm. In some embodiments, the output mesh is generated using a marching tetrahedra algorithm. In some embodiments, the output mesh is generated using an isosurface extraction technique.

[0071] (A11) In some embodiments of any of A1-A10, the classifier is trained using a set of input sampling points and corresponding input training observations (e.g., labeled input training observations). For example, FIG. 2 shows a training architecture 200 for training a classifier 210.

[0072] (A12) In some embodiments of A11, the set of input sampling points is generated by applying an octree construction to the input shape. For example, the input sampling points are corner points of octree cells. In some embodiments, applying the octree construction includes subdividing local cells only if they intersect with a surface of the input shape. In some embodiments, the set of input sampling points includes a regular set of sampling points (e.g., not specific to the input shape).

[0073] (A13) In some embodiments of any of A1-A12, a classifier is trained to learn respective surface functions for a set of input shapes.

[0074] (A14) In some embodiments of any of A1-A13, the classifier comprises a classification neural network.

[0075] (A15) In some embodiments of any of A1-A14, generating a representation of the 3D object includes generating a 2D view of the 3D object on a display device (eg, display 708).

[0076] (A16) In some embodiments of any of A1-A15, generating a representation of the 3D object includes generating a representation of the 3D object in an artificial reality environment (eg, a virtual reality or an augmented reality environment).

[0077] In other aspects, some embodiments include a computing system including one or more processors and a memory coupled to the one or more processors, the memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described herein (e.g., A1-A16 above).

[0078] In yet another aspect, some embodiments include a non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors of a computing system, the one or more programs including instructions for performing any of the methods described herein (e.g., A1-A16 above).

[0079] As will be apparent to those skilled in the art, many modifications and variations of this disclosure can be made without departing from the spirit and scope of the disclosure. The specific embodiments described herein are offered by way of example only, and the disclosure should be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled.

[0080] It will also be understood that, although terms such as first, second, etc. are used herein in some instances to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first object can be referred to as a second object, and similarly, a second object can be referred to as a first object, without departing from the scope of various described embodiments. Although a first object and a second object are both objects, they are not the same object unless expressly stated as such.

[0081] The terms used in the description of the various embodiments described herein are intended to describe only certain embodiments and are not intended to be limiting. As used in the description of the various described embodiments and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "includes", "including", "comprises" and / or "comprising", as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0082] The foregoing description has been described with reference to specific embodiments for purposes of illustration. However, the above exemplary description is not intended to be exhaustive or to limit the claims to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments have been selected to best explain the principles underlying the claims and their practical application, thereby enabling others skilled in the art to best utilize the embodiments with various modifications suited to the particular use contemplated. [Explanation of symbols]

[0083] 100 Architecture 102 Sampling Point Template 104 Input Data 106 trained classifiers 108 Signed Distance Fields 110 Field to Mesh Transducer 112 Object Mesh 200 Architecture 202 Shape Data 204 Octree constructor 205 Cell Grid 206 Signed Distance Value Constructor 207 Sampling Points 208 Input Data 210 Classifier 300, 320, 340 2D signed distance field unit cells 302, 304, 306, 308, 342 points 322 Surface 402 Space 404 Shape 406 line 410, 424 diagonal 412, 422 horizontal line 420, 426, 428 unit cells 432 Surface 434 Surface 438 Surface Normal 502 Input Observations 504, 604, 616 3PSDF mesh 506, 512 External surface 508, 514 Inside 602, 608, 614 Images 606, 612, 618 Ground Truth Meshes 700 Computing System 704 Communication Interface 706 User Interface 708 Displays, display devices 710 Input Devices or Mechanisms 712 Communication Bus 714 Memory 716 Operating Systems 718 Communication Module 720 Applications 722 Classifier 724 Mesh Generator 726 Sampling Point Generator 730 Database 732 Training Data 734 Sampling Point Template 736 Input Observations

Claims

1. 1. A method executed on a computing system having a memory and one or more processors, the method comprising: obtaining a trained classifier; obtaining an input observation of a 3D object; generating a tripolar signed distance field from the input observations using the trained classifier; generating an output mesh of the 3D object from the tripolar signed distance field; generating a representation of the 3D object from the output mesh; A method comprising:

2. obtaining a sampling point template; the tripolar signed distance field is generated using the sampling point template; The method of claim 1.

3. The method of claim 2 , wherein the tripolar signed distance field includes a tripolar signed distance value for each sampling point in the sampling point template.

4. 4. The method of claim 1, wherein generating the tripolar signed distance field from the input observations comprises assigning to each point of a plurality of points a value indicating whether the point is inside a surface, outside the surface, or undefined.

5. The method of claim 4 , wherein generating the output mesh comprises extracting surfaces only between a set of inner and outer values.

6. The method of claim 1 , wherein the input observations include one or more open surfaces.

7. 7. The method of claim 1, wherein generating the output mesh of the 3D object from the tripolar signed distance field comprises generating one or more open surfaces for the 3D object.

8. The method of claim 1 , wherein the input observations are point cloud data.

9. The method of claim 1 , wherein the input observation is an image.

10. 10. The method of claim 1, wherein the output mesh is generated from the tripolar signed distance field using a marching cubes algorithm.

11. 11. The method of claim 1, wherein the classifier is trained using a set of input sampling points and corresponding input training observations.

12. The method of claim 11 , wherein the set of input sampling points is generated by applying an octree construction to an input shape.

13. The method of claim 1 , wherein the classifier is trained to learn respective surface functions for a set of input shapes.

14. 14. The method of claim 1, wherein the classifier comprises a classification neural network.

15. 15. The method of claim 1, wherein generating the representation of the 3D object comprises generating a 2D view of the 3D object on a display device.

16. 16. The method of claim 1, wherein generating the representation of the 3D object comprises generating the representation of the 3D object in a synthetic reality environment.

17. one or more processors; Memory, One or more programs stored in the memory and configured for execution by the one or more processors, the one or more programs comprising: obtaining a trained classifier; obtaining an input observation of a 3D object; generating a tripolar signed distance field from the input observations using the trained classifier; generating an output mesh of the 3D object from the tripolar signed distance field; generating a representation of the 3D object from the output mesh; one or more programs containing instructions for A computing system comprising:

18. 20. The computing system of claim 17, wherein generating the output mesh of the 3D object from the tripolar signed distance field comprises generating one or more open surfaces for the 3D object.

19. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by a computing device having one or more processors, a memory, and a display, the one or more programs comprising: obtaining a trained classifier; obtaining an input observation of a 3D object; generating a tripolar signed distance field from the input observations using the trained classifier; generating an output mesh of the 3D object from the tripolar signed distance field; generating a representation of the 3D object from the output mesh; A non-transitory computer-readable storage medium comprising instructions for:

20. 20. The non-transitory computer-readable storage medium of claim 19, wherein generating the output mesh of the 3D object from the tripolar signed distance field comprises generating one or more open surfaces for the 3D object.

Images