Generating a surface with an arbitrary topology using a signed distance field

Abstract

Various embodiments described herein include methods, devices, and systems for generating an object mesh. In some embodiments, the method includes obtaining a trained classifier and input observations of a 3D object. The method further includes generating a tripolar signed distance field from the input observations using the trained classifier. The method also includes generating an output mesh of the 3D object from the tripolar signed distance field, and generating a representation of the 3D object from the output mesh.

Description

[Technical Field]

[0001] Cross-reference of related applications This application is a continuation of U.S. Patent Application No. 17 / 713,129, filed on April 4, 2022, entitled “Generating a Surface Having an Arbitrary Topology Using a Referenced Distance Field,” claiming its priority, and the entirety of which is incorporated herein by reference.

[0002] This generally relates to object surface generation, and more specifically to the use of signed distance fields for object mesh generation. [Background technology]

[0003] The geometric three-dimensional (3D) representation of objects is crucial for many tasks in the fields of computer vision and computer graphics. Over the past few years, various methods for 3D representation, including meshes, voxels, and point clouds, have been studied. However, many methods for generating 3D representations are limited to closed surfaces because they require the space to be divided into an inner and outer portion. [Overview of the project] [Problems that the invention aims to solve]

[0004] The choice of 3D shape and surface representation is crucial for effective 3D learning. In recent years, the emergence of neural implicit functions (NIFs) has advanced learning-based 3D reconstruction and modeling. Traditional NIF methods are built on signed distance functions (SDFs). These methods train deep neural networks to classify continuous 3D locations as inside or outside a surface via occupancy prediction or SDF regression. However, they can only model closed surfaces that support input / output testing for level surface extraction. Using unsigned distance functions (UDFs), it is possible to learn open surfaces from point clouds. However, instantiating unsigned distance fields into explicit meshes is computationally difficult and prone to artifacts. [Means for solving the problem]

[0005] In some embodiments of this disclosure, a novel learnable implicit representation called a tripolar signed distance function (3PSDF) is used to represent very complex geometric shapes, including both closed and open surfaces, with high fidelity. 3PSDF inherits the advantages of SDF and makes the learning components easier to train for classification tasks. 3PSDF is also compatible with classical isosurface extraction techniques, such as the marching cube algorithm, to achieve efficient field-to-mesh conversion. A key aspect of the 3PSDF method is the inclusion of non-directional codes (e.g., NULL codes) in the conventional binary signed distance function. For example, points with non-directional codes are assigned Not-A-Number (NaN) values ​​to prevent the formation of decision boundaries between NaN values ​​and their neighbors. By appropriately distributing the non-directional labels in space, surfaces with arbitrary topologies can be cast. As an example, a neural network is trained to classify continuous points into three categories: inside, outside, and non-directional. The resulting labels can be converted back to a 3PSDF using a simple mapping function to obtain a mesh result.

[0006] Accordingly, in one embodiment, some embodiments include a method that is performed on a computing system having memory and one or more processors. The method includes (i) obtaining a trained classifier, (ii) obtaining input observations of a 3D object, (iii) using the trained classifier to generate a tripolar signed distance field from the input observations, (iv) generating an output mesh of a 3D object from the tripolar signed distance field, and (v) generating a representation of the 3D object from the output mesh.

[0007] In other embodiments, some embodiments include a computing system having one or more processors, memory, and one or more programs stored in memory and configured to be executed by one or more processors. The one or more programs include instructions for the steps of (i) acquiring a trained classifier, (ii) acquiring input observations of a 3D object, (iii) using the trained classifier to generate a tripolar signed distance field from the input observations, (iv) generating an output mesh of a 3D object from the tripolar signed distance field, and (v) generating a representation of the 3D object from the output mesh.

[0008] Therefore, the systems and devices provide methods for generating object surfaces, thereby enhancing the effectiveness, efficiency, and user satisfaction of such systems and devices. Such systems, devices, and methods optionally complement or replace conventional systems, devices, and methods for generating object surfaces.

[0009] For a better understanding of the various embodiments described, the following detailed description should be consulted in conjunction with the following drawings, and the same reference numbers throughout the drawings refer to the corresponding parts. [Brief explanation of the drawing]

[0010] [Figure 1] FIG. [Figure 1] is a diagram showing an exemplary architecture for object mesh generation according to some embodiments. [Figure 2] FIG. is a diagram showing an exemplary architecture for classifier training according to some embodiments. [Figure 3] FIG. [Figure 2] is a diagram showing an exemplary signed distance field unit cell according to some embodiments. [Figure 4A] FIG. is a diagram showing an exemplary application of a signed distance field according to some embodiments. [Figure 4B] FIG. [Figure 3] is a diagram showing an exemplary application of a signed distance field according to some embodiments. [[ID=1'4]] [Figure 4C] FIG. is a diagram showing an exemplary application of a signed distance field according to some embodiments. [Figure 4D] FIG. [Figure 4A] is a diagram showing an exemplary application of a signed distance field according to some embodiments. [Figure 5A] FIG. is a diagram showing exemplary input observations and an output mesh according to some embodiments. [Figure 5B] FIG. [Figure 4B] is a diagram showing exemplary input observations and an output mesh according to some embodiments. [Figure 6A] FIG. is a diagram showing exemplary input observations and an output mesh according to some embodiments. [Figure 6B] FIG. [Figure 4C] is a diagram showing exemplary input observations and an output mesh according to some embodiments. [Figure 6C] FIG. is a diagram showing exemplary input observations and an output mesh according to some embodiments. [Figure 7] FIG. [Figure 4D] is a block diagram showing an exemplary computing system according to some embodiments. [Figure 8A] FIG. is a flowchart showing an exemplary method for generating an output mesh of a 3D object according to some embodiments. [Figure 8B]This flowchart shows one exemplary method for generating an output mesh of a 3D object according to several embodiments. [Modes for carrying out the invention]

[0011] Here, examples of embodiments are shown in the accompanying drawings and are referred to in detail. The following detailed description includes numerous specific details to provide a complete understanding of the various described embodiments. However, it will be apparent to those skilled in the art that the various described embodiments can be carried out without these specific details. In other cases, well-known methods, procedures, components, circuits, and networks are not described in detail so as not to unnecessarily obscure the aspects of the embodiments.

[0012] One method for generating output meshes for 3D objects is to use a neural distance field (NDF). An NDF is a neural network-based model that predicts an unsigned distance field (UDF) for an arbitrary 3D shape using a sparse point cloud as input observations. For a point P, the NDF directly regresses the distance to the surface using a trained function whose zero-level set represents the surface.

[0013] However, learning using UDFs, as required by NDF methods, is difficult. Some UDF-based learning requires extracting point clouds or meshes, or rendering implicit surfaces directly onto an image, which necessitates finding their zero-level set. Conventional methods for finding the zero-level set by detecting transitions from inside to outside and vice versa (e.g., marching cubes and volume rendering) are incompatible with UDFs. As an alternative, NDF methods can infer a continuous unsigned distance field of the target object. Then, based on the UDF, NDF methods can compute the gradient flow of the inferred UDF. Furthermore, dense 3D sampling points may be randomly generated in 3D space. Using gradient flow, the sample points can be pushed onto the surface. Finally, NDF methods can use a ball-orbiting algorithm to convert the dense point cloud into a mesh.

[0014] Therefore, there are challenges when using NDF methods. For example, converting an unsigned distance field generated by an NDF into a mesh is a time-consuming process that can introduce numerous artifacts. To convert a UDF into a mesh, NDF methods may require the generation of a dense point cloud, leveraging the UDF's gradient field to repeatedly push points onto a target surface. Such a process is vulnerable to landscapes with complex gradients, such as parts with many details, and can get stuck at local minimums. In addition, reconstructing a mesh from a UDF may require the use of the Ball Orbit (BP) algorithm, which has several drawbacks. For example, the BP algorithm is sensitive to the input ball radius; a slightly larger or smaller radius can result in imperfect interlocking results. Furthermore, the BP algorithm tends to generate self-intersecting and cut face patches with inconsistent normals, even on surfaces of moderate complexity. Moreover, the BP algorithm is time-consuming, especially when dealing with dense point clouds.

[0015] Another challenge with using NDF methods is that training UDFs is a regression task, rather than a classification task (like SDFs), making training more complex. Another challenge with NDF methods is that they are only applicable to point cloud completion tasks (e.g., NDF inputs are limited to point clouds). Therefore, many 3D learning tasks, such as single or multi-view 3D reconstruction, cannot be performed using NDF methods.

[0016] Another method for generating output meshes for 3D objects employs deep implicit surface point prediction networks. This method presents a class of implicit representations called nearest surface point (CSP) representations. Instead of directly regressing the unsigned distance field of the target surface, the CSP infers the nearest point on the target geometry to the query point. After obtaining the nearest surface point, the unsigned distance to the surface can be calculated as the distance between the query point and the surface point. Additional information, such as the nearest surface point, simplifies the calculation of differential geometric properties such as tangent planes and surface normals. The CSP method can use volume encoders and shape decoders. For example, a point cloud is the input to a volume encoder to obtain a feature volume. The shape decoder is tuned based on the feature volume to obtain the nearest surface point to the query point.

[0017] Unlike NDF, which uses a gradient field to transform a UDF into densely sampled points and then uses the BPA algorithm to transform the points into a mesh, the CSP method leverages a multi-resolution surface extraction scheme and can directly use a marching cubes algorithm with a small positive threshold to obtain the final mesh. While it requires fewer steps than the NDF method and produces a smoother surface, the CSP method also cannot obtain an open surface with a single-layer mesh. The CSP method shares similar limitations with the NDF method. For example, training involves regression tasks rather than classification tasks (like those for SDF), making training more complex. Furthermore, the CSP method is also only applicable to point cloud completion tasks.

[0018] The embodiments described herein address the limitations and challenges presented by NDF and CSP methods. For example, the three-pole signed distance function (3PSDF) implicit representation can model complex 3D shapes, including both closed and open surfaces, without long or complex point-to-mesh transformations. In addition, the 3PSDF method can directly generate single-layer open surfaces in mesh form by applying a marching cube algorithm (for example, in a similar manner to when transforming a signed distance field (SDF) into a mesh). By extracting single-layer open surfaces, the 3PSDF does not introduce artifacts during the field-to-mesh transformation. Furthermore, the 3PSDF method can utilize classification network learning; for example, the 3PSDF method infers which category a query point belongs to (e.g., inside, outside, or null). This significantly reduces the learning difficulty of the 3PSDF method compared to the NDF or CSP methods. Moreover, the 3PSDF method can handle a broader and far more difficult range of tasks, such as 3D shape reconstruction from a single image.

[0019] According to several embodiments, the 3PSDF method can represent highly complex 3D shapes, including both closed and open surfaces, with high fidelity. In addition, the 3PSDF method inherits the advantages of the SDF method, such as making learning a (trainable) classification task and being compatible with isosurface extraction techniques (e.g., the marching cube algorithm) to achieve efficient field-to-mesh conversion. An exemplary learning method for the 3PSDF method is based on a classification neural network that classifies query points into three categories: inside, outside, and null. The resulting labels can be returned to the 3PSDF using a mapping function to obtain a mesh result. Therefore, the 3PSDF method can be used for tasks related to 3D modeling and reconstruction, such as automatic shape coding, point cloud completion, single-view 3D shape reconstruction, and multi-view 3D shape reconstruction.

[0020] Referring here to the drawings, Figure 1 shows an architecture 100 for object mesh generation according to several embodiments. The architecture 100 includes a trained classifier 106 and a field-to-mesh converter 110. According to some embodiments, the trained classifier 106 takes a sampling point template 102 and input data 104 (e.g., input observations) and generates a signed distance field 108. The field-to-mesh converter 110 takes the signed distance field 108 and generates an object mesh 112. In some embodiments, the trained classifier 106 includes multiple perceptual layers. In some embodiments, the trained classifier 106 includes multiple 1D convolutional layers.

[0021] For example, architecture 100 incorporates regular sampling points along with other input observations used during training time and estimates a tripolar signed distance value for each query sampling point. The regular sampling points, along with the tripolar signed distance value at each point location, form a generated tripolar signed distance field 108, which can be converted into an output mesh using conventional field-to-mesh techniques such as the Marching Cube (MC) algorithm.

[0022] In some embodiments, once the classifier is trained, each query point can be labeled with a prediction. For example, to extract isosurfaces, the inferred discrete labels are converted to a 3PSDF representation (e.g., points with labels 0, 1, and 2 are assigned SDF values ​​of -1, 1, and NaN, respectively). The reconstructed surfaces can then be extracted as zero-level surfaces.

[0023] In some embodiments, the isosurfaces represented in the 3PSDF are extracted directly using a marching cube algorithm. During extraction, null values ​​prevent the MC algorithm from extracting valid isosurfaces at locations that do not contain geometry. In some embodiments, near the target surface, isosurface extraction is performed in the same manner as a conventional signed distance field. In some embodiments, after the MC calculation, NaN vertices and faces generated by the null cube are removed, and the remaining vertices and faces serve as the mesh result.

[0024] Figure 2 shows an architecture 200 for training a classifier according to several embodiments. According to some embodiments, the architecture 200 includes an octree constructor 204, a signed distance value constructor 206, and a classifier 210. In the example in Figure 2, the octree constructor 204 takes shape data 202 and generates a cell grid 205 of the shape data 202. The signed distance value constructor 206 takes the cell grid 205 of the shape data 202 and generates sampling points 207. The classifier 210 is trained with the sampling points 207 and input data 208. In some embodiments, the classifier 210 is a ternary classifier.

[0025] In some embodiments, the shape data 202 includes a plurality of 3D shapes. In some embodiments, the octree builder 204 generates adaptive octree cells of the 3D shapes for use when calculating a tripolar signed distance field. In some embodiments, the corner points of the octree cells are used as input sampling points 207. In some embodiments, the input data 208 includes an image and a point cloud. For example, in the case of a single-view 3D reconstruction, the sampling points 207 include a 3D sampling point with a ground truth 3PSDF value, and the input data 208 includes the corresponding single-view image. In some embodiments, the input data 208 includes a sparse point cloud. In this way, the classifier 210 is trained to classify each input sampling point into the category to which it belongs (e.g., inside, outside, or NaN).

[0026] The introduction of null codes provides flexibility in eliminating undesirable surfaces, but NaN values ​​may not be used for some calculations, including calculating the gradients necessary to update the deep neural network. To circumvent this problem, instead of applying the network to directly regress signed distance values, the learning of the 3PSDF is formulated as a classification problem in several embodiments.

[0027] In some embodiments, the 3D function is approximated by a neural network that infers point-by-point labels, i.e., {in,out,null}. The label semantics can be expressed using discrete numbers without loss of generality. In some embodiments, the classifier 210 uses a mapping function o:R 3 →Learn {0,1,NaN}, where the labels {0,1,NaN} represent the inside, outside, and null, respectively.

[0028] In some embodiments, the classifier 210 (e.g., a network) is adjusted based on the input for a downstream task (e.g., 3D reconstruction) based on observations of an object (e.g., a point cloud or image). Thus, the classifier 210 is adjusted based on the sampling points 207 (e.g., point p∈R). 3 In addition to the coordinates of (p,x), input data 208 (e.g., observations of object x∈X) are also taken as input. In some embodiments, the mapping function is a neural network Φ that takes pairs (p,x) as input and outputs three-class labels, as shown in Equation 1 below. θ It is parameterized by [the specified method].

number

[0029] In some embodiments, the classifier 210 is trained using a batch of point samples, and the neural network Φ θ( Learn the parameter θ of p(x). For example, for the i-th sample in the training batch, N points are sampled, where p ij ∈R 3, where j = 1, …, N. The mini-batch loss L B is shown in Equation 2 below.

Equation

[0030] In some embodiments, each training sample is embedded with a 512-dimensional latent code, and the classifier 210 is trained to reconstruct a 3D shape from the embedding.

[0031] In some embodiments, the 3PSDF calculation is performed cell by cell. Thus, in some scenarios, it is preferable to reduce or minimize the geometric details and open surfaces of individual cells. Thus, in some embodiments, the octree builder 204 subdivides a cell only when the cell intersects the input shape. As the subdivision depth increases, the complexity of the surface contained in each local cell decreases, and the approximation accuracy can be improved. Since regions without a shape are not further divided, a trade-off is achieved between the computational complexity and the reconstruction accuracy.

[0032] Figure 3 shows signed distance field unit cells 300, 320, and 340 according to several embodiments. 2D signed distance field unit cell 300 includes points 302, 304, 306, and 308. Points 302 and 306 have corresponding values ​​of 0.5 and 0.2, respectively, and are therefore mapped to the outside of the surface according to some embodiments. Points 304 and 308 have corresponding values ​​of -0.3 and -0.6, respectively, and are therefore mapped to the inside of the surface according to some embodiments. 2D signed distance field unit cell 320 includes a surface 322 extracted from points 302, 304, 306, and 308. 2D signed distance field unit cell 340 is identical to 2D signed distance field unit cell 300, except that point 306 in 2D signed distance field unit cell 300 is replaced by point 342 in 2D signed distance field unit cell 340. Point 342 has a NaN value, which is mapped to a null sign, according to some embodiments. Including point 342 in the 2D signed distance field unit cell 340 prevents surface extraction according to some embodiments (for example, if any point in the unit cell has a NaN value, the surface is not extracted).

[0033] A watertight 3D shape can be implicitly represented by a signed distance function. 3D query point p∈R 3 Given a value, a deep neural network is applied to calculate the occupancy of p as f(p):R 3 →Predict as [0,1] or use SDF as f(p):R 3 →This can be directly regressed as R. In this way, the formation of a closed surface occurs when both positive and negative signs are present in space.

[0034] For signed distance functions that have the ability to represent open surfaces, a third non-directional pole (e.g., a null sign) is incorporated into the conventional signed distance field. For example, no equisurfaces are formed on the bisectors of either positive / null or negative / null pairs. In this way, the null sign acts as a surface remover that prevents undesirable surfaces, and therefore allows for flexible casting of any topology, including those with open surfaces. Incorporating a non-directional pole allows for 3D point p∈R 3 This means that, in addition to continuous signed distances, p can also be mapped to null values, i.e., Ψ(p):R 3 →{R, nan}. Therefore, given an input surface S, the mapping function Ψ can be learned to minimize the inter-surface distance, as shown in Equation 3 below.

number

[0035] Figures 4A to 4D illustrate the application of exemplary signed distance fields according to several embodiments. Figure 4A shows a space 402 having a 2D shape 404 (e.g., the head of a unicorn). As shown in Figure 4A, the shape 404 is open at the bottom (e.g., has an open surface). Figure 4B shows an SDF applied to the space 402, where the diagonal line 410 represents the inner position and the horizontal line 412 represents the outer position. Since an SDF can only represent a closed surface, the shape 404 is closed in Figure 4B by adding a line 406 to the bottom.

[0036] Figure 4C shows an octree-based subdivision applied to space 402, resulting in unit cell 420. Figure 4C further shows a 3PSDF applied to space 402, where diagonal lines 424 represent inner positions, horizontal lines 422 represent outer positions, and empty unit cells (e.g., no lines) represent null (non-directional) positions.

[0037] Figure 4D shows a unit cell 426 (from Figure 4C) having an open surface (e.g., a partial line) 432. In some embodiments, since no surface is formed, applying 3PSDF to the unit cell 426 results in approximation errors. In some embodiments, the approximation errors arising from the open surface in the unit cell are reduced or minimized by applying finer spatial resolution (e.g., continuing to subdivide the space 402 until the unit cell no longer has an open surface). Figure 4D further shows a unit cell 428 (from Figure 4C) having a complex (e.g., nonlinear) surface 434. According to some embodiments, the unit cell 428 includes a surface normal 438 used to determine the internal and external positions.

[0038] For non-watertight surfaces (e.g., without closed boundaries), input / output testing of query points is insufficient. In some embodiments, surface normals are used to determine the sign of the distance to the query point. In some embodiments, the space is decomposed into a grid of local cells. As shown in Figure 4C, each cell 420(C i Regarding ), if it does not include the surface, the cell space is set as a null region, C i Any sample point p inside i The distance to the target surface S is NaN, as shown in Equation 4 below. Ψ(p i ,S)=P i ∈C i and C i If ∩S=φ, then nanan Equation 4: NaN distance

[0039] Local cell C surrounding the surface patch Si i In this case, query point p i ∈C i Given, p on Si, i The nearest point q i This is determined. First, q i Surface normal n(q) i Set it as follows: Vector

number

number

number

[0040] In some scenarios, a finer decomposition of 3D space results in cells containing geometric shapes being distributed around the surface in question, while null cells occupy the majority of the space. This differs from conventional signed distance fields (such as those shown in Figure 4B), where the entire space is filled with either positive or negative distances. Thus, 3PSDFs can better reflect the properties of 3D surfaces of arbitrary topologies due to their high sparsity of surface occupancy.

[0041] Figures 5A and 5B show exemplary input observations and output meshes according to several embodiments. In particular, Figures 5A and 5B show a visual comparison of point cloud completion on non-watertight data with open surfaces. Figure 5A shows an input observation 502 (e.g., a point cloud for an automobile) and a corresponding 3PSDF mesh 504 generated from the input observation. The 3PSDF mesh 504 includes an outer surface 506 and an inner surface 508. Figure 5A further shows a ground truth mesh 510 (e.g., the mesh from which input observation 502 was generated) with an outer surface 512 and an inner surface 514. Figure 5B shows an input observation 520 (e.g., a point cloud for a convertible car), a corresponding 3PSDF mesh 522 generated from the input observation, and a ground truth mesh 524 (e.g., the mesh from which input observation 520 was generated).

[0042] Figures 6A to 6C show exemplary input observation and output meshes according to several embodiments. Figure 6A shows an image 602 of an aircraft (e.g., a single image input observation), a corresponding 3PSDF mesh 604, and a ground truth mesh 606. In particular, Figure 6A shows different viewpoints of the aircraft in image 602 using the 3PSDF mesh 604. Figure 6B shows an image 608 of a car, a corresponding 3PSDF mesh 610, and a ground truth mesh 612. In particular, Figure 6B shows different viewpoints of the car in image 608 using the 3PSDF mesh 610. Figure 6C shows an image 614 of a chair, a corresponding 3PSDF mesh 616, and a ground truth mesh 618. In particular, Figure 6C shows different viewpoints of the chair using the 3PSDF mesh 616.

[0043] Figure 7 is a block diagram showing a computing system 700 in several embodiments. Various examples of the computing system 700 include desktop computers, laptop computers, tablet computers, server systems, and other computing devices having processors capable of running a classifier 722. The computing system 700 typically includes one or more processing units / cores (CPUs) 702 for executing modules, programs, and / or instructions stored in memory 714 and thereby performing processing operations, one or more network or other communication interfaces 704, memory 714, and one or more communication buses 712 for interconnecting these components. The communication buses 712 may include circuits for interconnecting and controlling communication between system components.

[0044] The computing system 700 optionally includes a user interface 706 comprising a display device 708 and one or more input devices or mechanisms 710. In some embodiments, the input devices include a keyboard and / or a mouse. In some embodiments, the input devices include a "soft" keyboard that is displayed on the display device 708 as needed, allowing the user to "press keys" displayed on the display 708. In some embodiments, the display 708 and the input devices 710 include a touchscreen display (also called a touch-sensitive display).

[0045] In some embodiments, memory 714 includes high-speed random-access memory such as DRAM, SRAM, DDR RAM, or other random-access solid-state memory devices. In some embodiments, memory 714 includes non-volatile memory such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. In some embodiments, memory 714 includes one or more storage devices located remotely from the CPU 702. Memory 714, or alternatively, the non-volatile memory devices within memory 714, is or includes a non-temporary computer-readable storage medium. In some embodiments, memory 714, or the computer-readable storage medium of memory 714, stores the following programs, modules, and data structures or subsets thereof, i.e., • Operating System 716, which handles various basic system services and includes procedures for performing hardware-dependent tasks. • A communication module 718 is used to connect the computing system 700 to other computers and devices via one or more communication network interfaces 704 (wired or wireless) and one or more communication networks such as the Internet, other wide area networks, local area networks, and metropolitan area networks. • One or more applications 720 that perform specific tasks or sets of tasks for the user (e.g., word processors, media players, web browsers, and communication platforms), • A classifier 722 (e.g., classifier 106) for determining the surface function of the input data. In some embodiments, the classifier 722 includes one or more neural networks, • A mesh generator 724 (e.g., field-to-mesh converter 110) for generating an object mesh from a signed distance field. In some embodiments, the mesh generator 724 includes a marching cube algorithm and / or a marching tetrahedras algorithm. • A sampling point generator 726 (e.g., an octree builder 204 and a signed distance value builder 206) for generating sampling points to be used with the input data, and One or more databases 730 used by application 720, classifier 722, mesh generator 724, and / or sampling point generator 726. According to some embodiments, one or more databases 730 include training data 732, sampling point templates 734, and / or input observations 736.

[0046] Each of the above-specified sets of executable modules, applications, or procedures may be stored in one or more of the aforementioned memory devices and correspond to a set of instructions for performing the above-specified function. The modules or programs (i.e., sets of instructions) specified above do not need to be implemented as separate software programs, procedures, or modules, and therefore various subsets of these modules may be combined or otherwise rearranged in various embodiments. In some embodiments, memory 714 stores a subset of the modules and data structures specified above. Furthermore, memory 714 may store additional modules or data structures not described above.

[0047] Figure 7 shows the computing system 700, but Figure 7 is intended not as a schematic diagram of the structure of the embodiments described herein, but rather as a functional description of the various features that may exist. In practice, as will also be recognized by those skilled in the art, items shown separately can be combined, and some items can be separated.

[0048] Figures 8A and 8B are flowcharts of Method 800 for generating output meshes of 3D objects according to several embodiments. Method 800 is performed on a computing system (e.g., computing system 700) having one or more processors and memory. In some embodiments, the memory stores one or more programs configured to be executed by one or more processors.

[0049] The computing system acquires a trained classifier (e.g., trained classifier 106) (802). In some embodiments, the trained classifier is acquired through a training process (e.g., as shown in Figure 2).

[0050] In some embodiments, the classifier is trained using a set of input sampling points and corresponding input training observations (804). For example, classifier 210 is trained using sampling point 207 and input data 208. In some embodiments, classifier 210 is the trained classifier 106 after the training process has been performed.

[0051] In some embodiments, the set of input sampling points is generated by applying an octree construction to the input shape (806). For example, sampling point 207 is generated via an octree constructor 204 and a signed distance value constructor 206.

[0052] In some embodiments, the classifier is trained to learn the respective surface function for a set of input shapes (808). For example, classifier 210 is trained to learn the surface function of shape data 202 (e.g., the mapping function of Equation 1).

[0053] In some embodiments, the classifier (e.g., classifier 722) is a classification neural network (e.g., neural network Φ) θ ) including (810). In some embodiments, the classifier includes multiple perceptual layers. In some embodiments, the classifier includes multiple 1D convolutional layers.

[0054] The computing system acquires input observations of a 3D object (e.g., input data 104) (812). In some embodiments, the input observations include one or more open surfaces (e.g., as shown in Figure 6A) (814). In some embodiments, the input observations are point cloud data (e.g., point cloud data 502 or 520) (816). In some embodiments, the input observations are images (e.g., images 602, 608, or 614) (818).

[0055] The computing system uses a trained classifier to generate a tripolar signed distance field (e.g., signed distance field 108) from the input observations (820). In some embodiments, the computing system obtains a sampling point template (e.g., sampling point template 102), and the tripolar signed distance field is generated using the sampling point template (822). In some embodiments, the tripolar signed distance field includes a tripolar signed distance value for each sampling point in the sampling point template (824).

[0056] In some embodiments, the computing system assigns a value to each of a plurality of points indicating whether the point is inside the surface, outside the surface, or undefined (as shown, for example, in Figure 4C) (826).

[0057] The computing system generates an output mesh of a 3D object from a tripolar signed distance field (for example, using a mesh converter 724) (828). In some embodiments, the output mesh is generated from the tripolar signed distance field using a marching cube algorithm (834).

[0058] In some embodiments, the computing system extracts surfaces only from the set of inside and outside values ​​(830). For example, the system does not extract surfaces between signed points and null / NaN points, as shown in Figure 3. In some embodiments, the computing system generates one or more open surfaces for a 3D object (for example, as shown in Figures 5A-5B) (832).

[0059] The computing system generates a representation of the 3D object from the output mesh (for example, displayed on display 708) (836). In some embodiments, the computing system generates a 2D view of the 3D object on the display device (for example, an oblique view as shown in Figures 6A to 6C) (838). In some embodiments, the computing system generates a representation of the 3D object in an artificial reality environment (840).

[0060] In light of these principles, we now turn to a specific embodiment.

[0061] (A1) In one embodiment, several embodiments include a method (e.g., method 800) that is performed on a computing system (e.g., computing system 700) having memory and one or more processors. The method includes (i) acquiring a trained classifier (e.g., classifier 106), (ii) acquiring input observations of a 3D object (e.g., input data 104), (iii) using the trained classifier to generate a tripolar signed distance field (e.g., signed distance field 108) from the input observations, (iv) generating an output mesh of a 3D object (e.g., object mesh 112) from the tripolar signed distance field, and (v) generating a representation of the 3D object from the output mesh.

[0062] (A2) In some embodiments of A1, the method further includes the step of obtaining a sampling point template (e.g., sampling point template 102), and the tripolar signed distance field is generated using the sampling point template. In some embodiments, the sampling point template includes a regular set of sampling points (e.g., a grid) for input observations.

[0063] (A3) In some embodiments of A1 or A2, the tripolar signed distance field includes a tripolar signed distance value (e.g., 1, -1, or NaN) for each sampling point in the sampling point template.

[0064] (A4) In some embodiments of A1 to A3, the step of generating a tripolar signed distance field from input observations includes assigning a value to each of a plurality of points indicating whether the point is inside, outside, or undefined on the surface. For example, negative values ​​correspond to points inside the surface, positive values ​​correspond to points outside the surface, and NaN or null values ​​correspond to undefined points.

[0065] (A5) In some embodiments of A4, the step of generating the output mesh includes the step of extracting a surface from a set of only inside values ​​and only outside values ​​(for example, as shown in Figure 3).

[0066] (A6) In some embodiments of A1 to A5, the input observation includes one or more open surfaces (as shown, for example, in Figures 5A to 5B).

[0067] (A7) In some embodiments of A1 to A6, the step of generating an output mesh of a 3D object from a tripolar signed distance field includes the step of generating one or more open surfaces for the 3D object.

[0068] (A8) In some embodiments of A1 to A7, the input observation is point cloud data (e.g., point cloud data 502 or 520).

[0069] (A9) In some embodiments of A1 to A7, the input observation is an image (for example, a 2D image as shown in Figures 6A to 6C).

[0070] (A10) In some embodiments of A1 to A9, the output mesh is generated from a tripolar signed distance field using a marching cube algorithm. In some embodiments, the output mesh is generated using a marching tetrahedras algorithm. In some embodiments, the output mesh is generated using an isosurface extraction technique.

[0071] (A11) In some embodiments of A1 to A10, the classifier is trained using a set of input sampling points and corresponding input training observations (e.g., labeled input training observations). For example, Figure 2 shows a training architecture 200 for training a classifier 210.

[0072] (A12) In some embodiments of A11, the set of input sampling points is generated by applying an octree structure to the input shape. For example, the input sampling points are the corner points of the octree cells. In some embodiments, applying the octree structure includes subdividing local cells only when they intersect with the surface of the input shape. In some embodiments, the set of input sampling points includes a regular set of sampling points (e.g., not specific to the input shape).

[0073] (A13) In some embodiments of A1 to A12, the classifier is trained to learn the respective surface functions for a set of input shapes.

[0074] (A14) In some embodiments of A1 to A13, the classifier includes a classification neural network.

[0075] (A15) In some embodiments of A1 to A14, the step of generating a display of a 3D object includes the step of generating a 2D view of the 3D object on a display device (e.g., display 708).

[0076] (A16) In some embodiments of A1 to A15, the step of generating a representation of a 3D object includes the step of generating a representation of a 3D object in an artificial reality environment (e.g., a virtual reality or augmented reality environment).

[0077] In other embodiments, some embodiments include a computing system comprising one or more processors and memory coupled to one or more processors, the memory storing one or more programs configured to be executed by one or more processors, and the one or more programs containing instructions for executing any of the methods described herein (e.g., A1 to A16 above).

[0078] In yet another embodiment, some embodiments include a non-temporary computer-readable storage medium that stores one or more programs for execution by one or more processors of a computing system, the one or more programs including instructions for executing any of the methods described herein (e.g., A1 to A16 above).

[0079] As will be apparent to those skilled in the art, many modifications and variations of this disclosure can be made without departing from the spirit and scope of this disclosure. The specific embodiments described herein are provided for illustrative purposes only, and this disclosure should be limited only by the terms of the appended claims, together with the entire scope of equivalents to which the appended claims are granted.

[0080] While terms such as "first," "second," etc., are used herein in some instances to describe various elements, it should be understood that these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first object can be called a second object without departing from the scope of the various described embodiments, and similarly, a second object can be called a first object. Both the first and second objects are objects, but unless explicitly stated otherwise, they are not the same object.

[0081] The terms used in the description of the various embodiments described herein are intended to describe only specific embodiments and are not intended to limit them. As used in the various described embodiments and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural form unless the context clearly indicates otherwise. The terms “and / or” as used herein will be understood to refer to and include one or any possible combination of the related enumerated items. The terms “includes,” “including,” “comprises,” and / or “comprising,” as used herein, identify the presence of the described features, integers, steps, actions, elements, and / or components, but will not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0082] The above description is provided with reference to specific embodiments for illustrative purposes. However, the above exemplary description is not intended to be exhaustive or to limit the claims to the exact form disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments have been selected to best illustrate the principles underlying the claims and their practical applications, thereby enabling other persons skilled in the art to best utilize the embodiments with various modifications suited to their specific intended use. [Explanation of Symbols]

[0083] 100 Architectures 102 Sampling Point Template 104 Input Data 106 pre-trained classifiers 108 Signed distance field 110 Field-to-Mesh Converters 112 Object Meshes 200 Architectures 202 Shape Data 204 Octree constructor 205 Cell Grid 206 Signed Distance Value Builder 207 sampling points 208 Input Data 210 Classifier 300, 320, 340 2D signed distance field unit cells 302, 304, 306, 308, 342 points 322 Surface 402 Space 404 Shape 406 line 410, 424 diagonals 412, 422 horizontal line 420, 426, 428 unit cells 432 Surface 434 Surface 438 Surface Normal 502 Input observation 504, 604, 616 3PSDF mesh 506, 512 External surface 508, 514 Inside Images 602, 608, 614 606, 612, 618 Ground Truth Mesh 700 Computing Systems 704 Communication Interface 706 User Interface 708 displays, display devices 710 Input device or mechanism 712 Communications Bus 714 memory 716 Operating Systems 718 Communication Module 720 applications 722 classifier 724 Mesh Generator 726 Sampling Point Generator 730 databases 732 training data 734 Sampling Point Template 736 Input observations

Claims

1. A computing system having memory and one or more processors, a method performed by one or more processors, wherein the method is Steps to obtain a trained classifier, Steps to obtain input observations of a 3D object, A step of generating a tripolar signed distance field from the input observations using the trained classifier, the step of assigning a value to each of a plurality of points indicating whether the point is inside the surface, outside the surface, or undefined; A step of generating an output mesh of the 3D object from the three-pole signed distance field, comprising the step of extracting surfaces only from between sets of inside and outside values, The steps include generating a display of the 3D object from the output mesh and Methods that include...

2. This further includes the step of obtaining a sampling point template, The three-pole signed distance field is generated using the sampling point template. The method according to claim 1.

3. The method according to claim 2, wherein the tripolar signed distance field includes a tripolar signed distance value for each sampling point in the sampling point template.

4. The method according to claim 1, wherein the input observation includes one or more open surfaces.

5. The method according to claim 1, wherein the step of generating the output mesh of the 3D object from the three-pole signed distance field includes the step of generating one or more open surfaces for the 3D object.

6. The method according to claim 1, wherein the input observation is point cloud data.

7. The method according to claim 1, wherein the input observation is an image.

8. The method according to claim 1, wherein the output mesh is generated from the tripolar signed distance field using a marching cube algorithm.

9. The method according to claim 1, wherein the classifier is trained using a set of input sampling points and corresponding input training observations.

10. The method according to claim 9, wherein the set of input sampling points is generated by applying an octvine structure to the input shape.

11. The method according to claim 1, wherein the classifier is trained to learn each surface function for a set of input shapes.

12. The method according to claim 1, wherein the classifier includes a classification neural network.

13. The method according to claim 1, wherein the step of generating the display of the 3D object includes the step of generating a 2D view of the 3D object on a display device.

14. The method according to claim 1, wherein the step of generating the representation of the 3D object includes the step of generating the representation of the 3D object in an artificial reality environment.

15. An apparatus configured to perform the method described in any one of claims 1 to 14.

16. A computer program for causing a computer to perform the method described in any one of claims 1 to 14.

Images