Low-poly mesh generation for 3D models

Abstract

In the method, a visual hull is generated based on an intersection of first 3D primitives of a plurality of first silhouettes with a bounding box of the 3D model. The first silhouettes are generated by projecting the 3D model onto a plane perpendicular to several selected view directions of the 3D model. Each of the first 3D primitives is obtained by extruding a connected loop of the respective first silhouette along one of several selected view directions associated with the respective first silhouette. A cutout mesh is generated based on subtraction of a second 3D primitive derived from a positive portion of the 3D model to cut out redundant structures from the visual hull. The positive portion is obtained based on a fitting plane that slices the 3D model. A low-poly mesh sequence is generated based on progressive simplification of the cutout mesh.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Patent Application No. 17 / 737,951, "LOW-POLY MESH GENERATION FOR THREE-DIMENSIONAL MODELS," filed May 5, 2022. The entire disclosure of this application is incorporated herein by reference.

[0002] This disclosure describes embodiments generally related to computational methodologies involving mesh geometry models. [Background technology]

[0003] The description of the background art provided herein is intended to generally present the context of the present disclosure. To the extent that it is provided in this Background section, the inventors' work described herein, as well as aspects of the description that may not be admitted as prior art at the time of filing, are not admitted expressly or impliedly as prior art to the present disclosure.

[0004] Three-dimensional (3D) models, such as building models, are an important asset that constitutes the metaverse for virtual reality. Building modelers typically focus on creating a good-looking appearance rather than other aspects such as clean topology and connected geometry. Highly detailed building models can have complex topology and geometry characteristics such as separated components, open boundaries, non-manifold edges, and / or self-intersections. However, rendering detailed building models at all times can be expensive, and some devices, such as low-end platforms, may have more limited computational resources that limit the generation of detailed meshes. To improve runtime performance, level-of-detail (LOD) techniques can be used. For example, instead of using highly detailed (or high-poly) 3D models, low element count (low-poly) meshes can be rendered in distant views. Thus, an effective and robust solution for generating low-poly meshes is needed. Summary of the Invention [Problem to be solved by the invention]

[0005] Aspects of the present disclosure include methods, apparatus, and non-transitory computer-readable storage media for computational methodologies. In some examples, an apparatus for generating a mesh geometry model includes a processing circuit. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, a method for generating a low-poly mesh sequence for a three-dimensional (3D) model is provided. In the method, a visual hull can be generated based on an intersection of a first 3D primitive of a plurality of first silhouettes with a bounding box of the 3D model. The plurality of first silhouettes can be generated by projecting the 3D model onto a plane perpendicular to several selected view directions of the 3D model. Each of the first 3D primitives can be obtained by extruding a connected loop of the respective first silhouette along one of several selected view directions associated with the respective first silhouette. A cutout mesh can be generated based on subtraction of a second 3D primitive derived from a positive portion of the 3D model to cut out redundant structures from the visual hull. The positive portion can be obtained based on a plurality of fitting planes slicing the 3D model. The low-poly mesh sequence can be generated based on a progressive simplification of the cutout mesh.

[0007] In the method, a plurality of fitting planes may be formed, and each region of the 3D model may be included in a respective one of the plurality of fitting planes. A plurality of candidate view directions may be determined. Each of the plurality of candidate view directions may be parallel to a respective pair of fitting planes and may be associated with a respective weight value. The respective weight value may be equal to a sum of areas of the regions of the 3D model included in the respective pair of fitting planes. A number of selected view directions may be determined from the plurality of candidate view directions associated with the top k weight values, where k may be a positive integer.

[0008] In some embodiments, each of the plurality of first silhouettes of the 3D model can be simplified through at least one of 2D simplification or shape size filtering. One or more hollow loops can be extracted from each of the plurality of first silhouettes. One or more connected loops can be generated for each of the plurality of first silhouettes by subtracting the one or more hollow loops from a 2D bounding box of each of the plurality of first silhouettes. The first 3D primitive can be formed by extruding one or more connected loops of the plurality of first silhouettes along a number of selected view directions.

[0009] In the method, a first virtual volume intersection may be formed by performing a Boolean intersection operation on a first one of the first 3D primitives and a bounding box of the 3D model to form a virtual volume intersection. A first visual disparity improvement may be determined based on the first virtual volume intersection and the 3D model. In response to the visual disparity improvement being greater than a threshold, a second virtual volume intersection may be formed by performing a Boolean intersection operation on a second one of the first 3D primitives and the first virtual volume intersection. A second visual disparity improvement may be determined based on the second virtual volume intersection and the 3D model.

[0010] To determine a first visual disparity improvement based on the first virtual volume intersection and the 3D model, an initial visual disparity can be determined based on an average pixel-wise disparity between a bounding box of the 3D model and the 3D model. The first visual disparity can be determined based on an average pixel-wise disparity between the first virtual volume intersection and the 3D model. The first visual disparity improvement can then be determined by subtracting the first visual disparity from the initial visual disparity.

[0011] In some embodiments, in response to one of (i) the nth visual disparity improvement being less than a threshold, and (ii) n being equal to an upper limit N, the nth virtual hull intersection may be determined as the virtual hull intersection. N may be a positive integer. The nth virtual hull intersection may be formed by performing a Boolean intersection operation on the nth one of the first 3D primitives and the (n-1)th virtual hull intersection.

[0012] In some embodiments, the 3D model may be sliced ​​by a plurality of fitting planes to form the cutout mesh. Each of the plurality of fitting planes may slice the 3D model into a positive portion and a negative portion. The positive portion of the 3D model may be located on a positive side of each of the plurality of fitting planes. Each of the positive portions of the 3D model may be projected onto a corresponding fitting plane of the plurality of fitting planes to obtain a respective second silhouette. An enlarged bounding square of each second silhouette may be obtained on the corresponding fitting plane such that the respective second silhouette is included in the enlarged bounding square. A Boolean subtraction operation may be operated to subtract the respective second silhouette from the enlarged bounding square to obtain a boundary loop from the respective second silhouette. A second 3D primitive of one of the second 3D primitives may be formed by extruding the boundary loop of the respective second silhouette in a normal direction from a positive side of the fitting plane corresponding to the respective second silhouette.

[0013] The method may form a first provisional segmented mesh by performing a Boolean subtraction operation on a first one of the second 3D primitives and the visual hull intersection to form a segmented mesh. A first visual disparity improvement may be determined based on the first provisional segmented mesh and the 3D model. In response to the visual disparity improvement being greater than a threshold, a second provisional segmented mesh may be formed by performing a Boolean subtraction operation on a second one of the second 3D primitives and the first provisional segmented mesh. A second visual disparity improvement may be determined based on the second provisional segmented mesh and the 3D model.

[0014] In some embodiments, in response to one of (i) the nth visual disparity improvement being less than a threshold and (ii) n being equal to an upper limit N, the segmentation mesh may be determined to be the nth provisional segmentation mesh, where N may be a positive integer, and the nth provisional segmentation mesh may be formed by performing a Boolean intersection operation on the nth one of the second 3D primitives and the (n-1)th provisional segmentation mesh.

[0015] To generate the low-poly mesh sequence, edge erasure and edge reversal operations can be performed progressively on the segmented meshes to generate the low-poly mesh sequence. Each of the low-poly meshes in the low-poly mesh sequence can include a respective number of triangles that is less than a user-selected value T. T can be a positive integer, and the number of triangles in each of the low-poly meshes can be less than the number of triangles in the segmented meshes.

[0016] In the method, the low-poly meshes in the sequence of low-poly meshes can be ranked based on the number of faces and visual disparity of the low-poly meshes to generate a Pareto set.

[0017] According to another aspect of the present disclosure, an apparatus is provided having a processing circuit that can be configured to perform any one or combination of the methods for generating a mesh geometry model.

[0018] Aspects of the present disclosure provide a non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform any one or combination of methods for generating a mesh geometry model.

[0019] Further features, nature and various advantages of the disclosed subject matter may become more apparent from the following detailed description and the accompanying drawings. [Brief description of the drawings]

[0020] [Figure 1] FIG. 2 illustrates an example of a high-poly mesh and corresponding low-poly counterparts of multiple high-poly meshes. [Diagram 2] FIG. 1 shows an example of a comparison of low-poly meshes generated using different methods. [Diagram 3] 1A-1C illustrate example outputs at various stages for generating a low-poly mesh. [Figure 4] FIG. 2 illustrates an exemplary view hull intersection. [Diagram 5] FIG. 1 illustrates an exemplary silhouette decomposition. [Figure 6] FIG. 13 illustrates an example of primitive generation based on connected loops of an input mesh. [Figure 7] FIG. 1 illustrates an exemplary process for mesh cutting. [Figure 8] FIG. 1 illustrates an exemplary process for forming a segmented mesh. [Figure 9] FIG. 2 illustrates an exemplary Pareto set for ranking low-poly meshes in a sequence of low-poly meshes. [Figure 10]FIG. 13 illustrates an example of the impact of stopping criteria on segmentation mesh detail according to one embodiment. [Figure 11A] FIG. 1 illustrates a comparison of different methods for generating a first low-poly mesh according to one embodiment. [Figure 11B] 11A-11C illustrate a comparison of different methods for generating a second low-poly mesh according to one embodiment. [Figure 12A] 11A-11C illustrate a comparison of different methods for generating a third low-poly mesh according to one embodiment. [Figure 12B] FIG. 11 illustrates a comparison of different methods for generating a fourth low-poly mesh according to one embodiment. [Figure 12C] FIG. 11 illustrates a comparison of different methods for generating a fifth low-poly mesh according to one embodiment. [Figure 12D] FIG. 11 illustrates a comparison of different methods for generating a sixth low-poly mesh according to one embodiment. [Figure 13] FIG. 13 illustrates a comparison of different methods for generating a seventh low-poly mesh according to one embodiment. [Figure 14] 1 is a flowchart outlining an example process for generating a low-poly mesh according to some embodiments of the present disclosure. [Figure 15] FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] 1 illustrates a number of exemplary building models. The building models can be used in a variety of environments, such as virtual reality and gaming applications. As illustrated in FIG. 1, the number of exemplary building models (e.g., 102 and 106) can have complex topology and geometry characteristics.

[0022] Rendering detailed building models all the time can be expensive, so LOD techniques can be used extensively to maximize runtime performance, as mentioned above. For example, instead of using highly detailed (or high-poly) 3D models, low-element-count (low-poly) meshes can be rendered in the distance. This requires that the low-poly meshes have a reasonably low element count while still maintaining as much of the appearance of the high-poly model as possible.

[0023] To create a low-poly mesh from an input high-poly model, different methods can be used, such as mesh simplification, reconstructing a simple mesh from point clouds, voxelizing a high-poly mesh to remove small-scale details, and 3D reconstruction from images using machine learning. However, with the increasing use of applications on low-end platforms such as mobile devices, the above-mentioned methods may not be sufficient. For example, limited computational resources on mobile platforms may limit a building mesh (or building model) to, for example, several thousand triangles, which is already considered a low-poly mesh by traditional LOD generation approaches, but the budget for the coarsest LOD is only a few hundred elements. To generate a low-poly mesh, none of the related methods were found to work well for a sample of a building model, such as the building model shown in Figure 1. Moreover, the standard process for creating such low-poly meshes is labor-intensive and involves tedious trial and error in the current gaming industry. For example, the conventional wisdom among game modelers is to manually create a low-poly mesh for a given 3D building model to achieve a more ideal balance between a low element count and visual similarity. However, this process can take hours and involve tedious trial and error. Thus, generating low-poly meshes for building models used by mobile applications is a challenging task, and an effective and robust solution for it remains elusive.

[0024] The present disclosure includes a method for effectively generating a low-poly mesh. The low-poly mesh can be used as the coarsest mesh in an LOD hierarchy. A visual metric can first be defined to qualitatively measure the visual difference between the low-poly mesh and the high-poly input mesh. The generation of the low-poly mesh can include three stages.

[0025] In the first stage, a coarse hull intersection can be constructed. In one example, a coarse hull intersection can be constructed by intersecting a small set of greedily selected 3D primitives by minimizing a visual metric. These 3D primitives can be generated by computing the silhouettes of the input mesh from several view directions, decomposing the silhouettes into 2D solids and holes, and extruding the 2D shapes. The result of the first stage can be represented as a hull intersection. A hull intersection can capture the silhouette of the input mesh, but it may miss important concave features of the input mesh.

[0026] The second stage aims to segment a segmentation mesh from the visual hull intersection. In one example, the segmentation is performed by subtracting the redundant volume to restore the concave features of the input mesh. A greedy strategy can be deployed to select the segmentation primitives by minimizing the visual metric between the segmentation mesh and the input mesh. When all 3D primitives for constructing the visual hull intersection and the redundant volume are watertight and precise arithmetic is adopted for the calculation, the generated segmentation mesh can be watertight and self-intersection-free.

[0027] A third stage can generate a mesh sequence from which low-poly meshes are selected. In one example, the mesh sequence is generated by progressively performing edge erasure and / or edge reversal on the segmented meshes.

[0028] In the third stage, high quality low poly meshes can occur indefinitely in a mesh sequence. Thus, a history of simplified meshes can be kept and the simplified meshes can be ordered into a Pareto set. The simplified meshes can be ordered by two objectives: number of triangles and visual closeness to the input mesh. An ideal mesh can be selected from the simplified meshes as the final result. The selection can be made by a user, such as a game modeler, or it can be selected automatically according to one or more criteria.

[0029] The effectiveness and performance of the related methods and low-poly mesh generation disclosed herein have been empirically compared based on a dataset containing 100 building models with various styles that were manually created by artists and used by real-world games. The low-poly mesh generation disclosed herein can show significant improvements, such as achieving low element counts and / or maintaining high visual appearance of the results.

[0030] Aspects of the present disclosure include a method that can reduce the number of polygons in a 3D model. The method can be used to automate the formation of a low-poly mesh (or low-poly mesh) by converting a high-poly 3D model (e.g., a building model) into a simple and visually preserving low-poly mesh. The method can include three stages. First, a watertight and self-collision-free volume intersection can be generated, e.g., via Boolean intersection 3D extrusion of the silhouette of the input. Second, salient but redundant structures can be cut out of the volume intersection, e.g., by Boolean subtraction of 3D primitives derived from a portion of the input. Third, a sequence of progressively simplified low-poly meshes can be generated from the cutout mesh. In some embodiments, a Pareto set can be extracted for a user to select from.

[0031] The stage of low-poly mesh formation can be driven by visual metrics with the goal of maintaining visual similarity with the input mesh. Furthermore, aspects of the present disclosure have been tested based on a dataset including 100 building models with different styles, most of which are used in popular digital games. The improvement in robustness and quality of the low-poly meshes is shown by comparison with other low-poly meshing techniques. An exemplary dataset including 100 building models (or input meshes) and corresponding low-poly meshes generated by the methods described herein is shown in FIG. 1. For example, a first building model 102 can have a corresponding low-poly mesh 104, and a second building model 106 can have a corresponding low-poly mesh 108.

[0032] Mesh simplification includes a large group of methods that either directly remesh the raw input through progressively error-guided element removal operations, such as edge erasure, or segment the mesh into patches that can later be retriangulated into a simplified mesh. These local remeshing operators are guided by a global, uniform error metric, which works well when dealing with medium to high-poly meshes. However, these local operators do not work well on extremely coarse low-poly meshes, and overly aggressive local remeshing can result in noticeable and detrimental visual artifacts. Figure 2 shows a mesh with these artifacts generated using edge erasure guided by a quadratic error metric (QEM) or the mesh reduction module within the commercial software Simplygon. As shown in Figure 2, a simplified mesh 204 can be generated by applying QEM to a building model 202. The building model 202 can serve as the input mesh and can be a manifold model with 39620 triangles and 1894 separated components. FIG. 2 also shows a simplified mesh 206 generated by applying Simplygon to the building model 202. As shown in FIG. 2, QEM or Simplygon can apply an overly aggressive remeshing operator that can result in noticeable artifacts with 1000 triangles. FIG. 2 further shows a low-poly mesh 208 generated by the method of the present disclosure. The low-poly mesh 208 can more faithfully represent the overall structure of the building model 202 using only 128 triangles.

[0033] Another type of approach is to voxelize the raw input and then apply feature-guided retriangulation to generate low-poly output by assuming that the input mesh is consistently oriented, i.e., has an inherent inside / outside definition. However, building models in games can be non-manifold, non-watertight, and self-intersecting, so face orientations often have unclear inside and outside labeling, even with mesh repair. PolyFit is yet another approach to generate low-poly meshes. Given a point cloud, PolyFit first computes a set of planes, builds a binary tree for all planes to obtain subfaces, and then selects faces for the final output by solving a mixed integer programming problem. PolyFit has successfully dealt with many dirty building models, but some candidate subfaces can be prohibitively large for models of practical complexity in practice, resulting in unmanageably large mixed integer programs.

[0034] Visual hulls can be used to generate 3D models. An object has a unique well-defined visual hull, which is the maximal shape that has the same silhouette as the object in any view direction. Visual hulls were originally used for 3D reconstruction from images or videos by considering the image boundaries as silhouettes. The advantage is that visual hulls can reconstruct a watertight, topologically simple mesh from an arbitrary dirty input (e.g., an input mesh with complex topology and / or geometric properties). In standard applications of visual hulls, users typically prefer more views to capture as much detail as possible. In theory, an infinite number of views are needed to obtain an accurate visual hull from a general 3D model. To construct an accurate visual hull using the viewpoints inside the convex hull, the computation time is O(n 5 ) silhouettes can be used. Visual hulls also have drawbacks, such as not representing concave features well. To address this issue, visual hulls can be constructed piece by piece via Boolean operations.

[0035] Aspects of the present disclosure further include an overview of a visual metric and an example pipeline (or method) for measuring the quality of a low-poly mesh. An example pipeline of the present disclosure is shown in FIG. 3. FIG. 3 illustrates three stages of low-poly mesh generation. As shown in FIG. 3, given a topologically and geometrically dirty input mesh 302, a small set of greedily selected primitives can be used to construct a visual hull intersection 304 such that the small set of greedily selected primitives can reduce the visual difference between the visual hull intersection 304 and the input mesh 302. In such a small set, some concave features may be erroneously flattened. To fix the issue of missing concave features, a segmented mesh 306 can be generated by segmenting the concave features. The segmentation can be performed using a set of segmented primitives. The segmented mesh 306 can be further simplified into a set of final output low-poly mesh candidates, such as low-poly mesh 308.

[0036] Non-orientable and non-watertight high-poly 3D building model M i Given this, one goal is to create a low-poly mesh M that can satisfy three qualitative requirements: o The goal is to generate a mesh called M for use as the coarsest mesh in the LOD hierarchy. o The visual appearance of M from an arbitrary distant viewpoint i The visual appearance of a M should resemble that of a 3D object. To maximize the rendering effect, the number of geometric elements (e.g., faces) should be as small as possible. Furthermore, the number of elements can be made user-controllable. Finally, o needs to be watertight to allow automated downstream mesh editing operations.

[0037] Visual metric is M i and M. o Given a view direction d, we can render the mesh into image space via a perspective projection, which is achieved by the operator Rn It can be shown as R n( Let M,d) be the three Cartesian components of the surface normal vector that can be rendered into the frame buffer, and the visual disparity can be defined as the average per-pixel distance in Equation 1: d n (M i ,M o ,d)=∥R n (M i ,d)-R n (M o ,d)∥ / N Equation (1) Here, N may be the number of pixels. The visual appearance metric may be further defined as the insignificant visual difference across all directions in Equation 2.

number

[0038] Low Poly Mesh M o To generate M i From the volume intersection M v can be computed, which can be topologically consistent and geometrically clean while capturing most of the silhouette information. M i Instead of directly simplifying the mesh itself, v is friendly (or compatible) with existing remeshing operators and is geometrically narrower than its convex hull, M i The range of M that is important for visual appearance i The silhouette of the object can be maintained. Then, the cutout mesh M c To obtain M v It can be subtracted from M c can enhance the visual appearance with prominent concave features. Finally, the low-poly output M o To generate the mesh, c can be simplified.

[0039] The generation of low-poly meshes can include generating a hull intersection. This stage aims to generate a topologically simple and geometrically clean hull intersection while capturing the visual appearance of salient structures. However, as shown in FIG. 4, generating an accurate hull intersection 404 based on an input mesh 402 may result in many small features and details. Instead, a simplified hull intersection 406 can be generated. A simplified hull intersection can be generated by carefully selecting and limiting the number of views of the hull intersection. For example, a simplified hull intersection can be generated based on the following Algorithm 1. As shown in FIG. 4, for a given input mesh 402, an accurate hull intersection 404 can be approximated by intersecting silhouettes from 13 view directions, resulting in 277k faces. In contrast, the hull intersection 406 generated by Algorithm 1 has only 368 faces obtained by three primitives.

[0040] In Algorithm 1, given M i For M, one silhouette can be generated for each of the top k view directions. Unlike related hull intersection construction methods, a self-intersection free simplification can be performed in 2D space for each silhouette. A hull intersection can then be constructed through the intersection of the extruded silhouettes. Furthermore, each silhouette can be decomposed into connected 2D loops, and the extruded silhouettes can be shown as primitives. Instead of considering a set of extruded silhouettes, a larger set of primitives can allow for finer and more complex control of the hull intersection. Specifically, M v M i can be initialized as the bounding box of M v A greedy algorithm can be applied to iteratively intersect P with the next best primitive P. An example of each step can be written as follows:

[0041] [Table 1]

[0042] Constructing a volume intersection can include initializing a primitive set. For example, in line 1 of Algorithm 1, we initialize the primitive set

number

number

[0043] Constructing the view volume intersection may further include determining a set of view directions. For example, in line 2 of Algorithm 1, a set of view directions may be extracted, and the top k view directions may be called the view direction set

number

number

number

number

number

number

number

[0044]

number

number

number

[0045] The construction of the visual hull may further include the computation of silhouettes. A silhouette may be computed for each direction in the determined set of view directions. For example, in line 4 of Algorithm 1, a silhouette S along a direction d may be generated. To compute a silhouette from a view direction d, M i All faces of can be projected onto a plane perpendicular to the view direction d. Then, a 2D Boolean union can be used to obtain the corresponding silhouette shape, where the vertex coordinates can be represented by integers for fast and robust computation. The generated silhouette shape can be guaranteed to be free of self-intersections.

[0046] The construction of the volume intersection may further include simplification of one or more of the silhouettes. For example, in line 5 of Algorithm 1, silhouette S may be simplified. Even with fewer view directions, silhouette simplification may be necessary because the volume intersection generated based on a small number of view directions may still contain a large number of small details due to the complex silhouette. Therefore, each silhouette may be simplified through 2D simplification and shape size filtering for further complexity reduction. In other embodiments, one or more silhouettes may be selectively simplified. The selective simplification may be performed based on the number of small details. The 2D simplification of the present disclosure may be a global approach implemented using rational arithmetic to generate silhouettes without self-intersections. The simplification may be a squared distance criterion.

number

[0047] Constructing the volume intersection may further include generating primitives.

number

[0048] FIG. 6 shows an example of primitives generated from an input mesh. As shown in FIG. 6, a plurality of loops 604-607 can be extracted from an input mesh 602. Furthermore, each of the loops can be extruded along a respective view direction to form a primitive. For example, a primitive 616 can be formed based on the loop 605, a primitive 618 can be formed based on the loop 606, a primitive 614 can be formed based on the loop 607, and a primitive 612 can be formed based on the loop 605.

[0049] In line 7 of Algorithm 1, we add primitives (e.g., Extrusion(L)) formed based on the loops extruded along the view direction to the primitive set

number

[0050] Between lines 9 and 19 of Algorithm 1, we determine the stopping criterion (e.g., a user-specified upper bound N or a user-specified threshold ε τ ) until M v We can iteratively perform Boolean intersection to intersect,p,with the suboptimal primitive,P,. As shown in line 9, when,n,is smaller than,N,, a user-specified upper bound, the suboptimal visual,disparity improvement,Δτ, best can be assigned an initial value such as 0, and the next best primitive P best can be assigned the empty set in line 10.

[0051] The construction of a volume intersection is a Boolean intersection that can be greedily selected.

number

number

number

[0052] In line 16, the suboptimal visual difference improvement Δτ best still subject to a user-specified threshold ε τ If M is larger than M, this means there is still room for further improvement. Therefore, in line 17, we select the best tentative mesh intersection (M v ,P best ) to M v can be assigned to the next best primitive P best is a primitive set

number

[0053] In line 18 of Algorithm 1, we can add one iteration count n to the initial visual disparity τ to the suboptimal visual disparity improvement Δτ best can be reduced by M v is the best tentative mesh intersection (M v ,P best ), which is updated by the input mesh M i can have less visual difference than

[0054] Line 19 of Algorithm 1 shows an example of a stopping criterion. Stopping criteria that can be applied to Algorithm 1 for constructing a visual hull intersection can include a visual dissimilarity threshold and a number of selected primitives. First, the improvement in visual dissimilarity Δτ p is the user-specified threshold ε τ If k is less than , this means there is little room for further improvement, and construction stops. Second, construction stops when the number of selected primitives (or iteration count) n reaches a user-specified upper limit N. The parameter k is

number

number

[0055] Algorithm 1 uses d to select a primitive. nNote that we apply a slightly different version of visual difference than Algorithm 1, because it focuses on generating similar silhouettes and not on the interior geometry. In fact, the mesh is flattened to a 2D plane, removing all geometric features inside the silhouette. Therefore, we apply another operator R, which can render the mesh M into a stencil buffer, perform binary masking of occluded pixels, and discard normal information. s (M, d) can be provided. Correspondingly, the visual silhouette difference d can be calculated in Equation 4 as follows: s We can define the visual silhouette metric τ s can be defined.

number

number

[0056] The generation of the low-poly mesh may further include generating a segmentation mesh. The segmentation mesh is a mesh that is generated by dividing the intersection volume M v The visual hull intersection M v can be generated based on Algorithm 1, for example. In some embodiments, the visual hull intersection can inherit the restriction of capturing only the silhouette while ignoring other features. As shown in FIG. 7, the input mesh M i The visual hull intersection M generated from 702 v 710 is the input mesh M i To overcome this limitation, we cut out redundant volume blocks to create a volume of M c and M. i By further reducing the visual difference between v Cut out mesh M c can be refined as follows.

[0057] A segmentation mesh can be generated by progressively subtracting multiple primitives from the visual hull. For example, the segmentation mesh can be generated based on Algorithm 2.

[0058] [Table 2]

[0059] Algorithm 2 may include the following steps: First, in the second line of Algorithm 2, the fitting plane set generated above is

number

number

number

[0060] Next, in line 7 of Algorithm 2, the positive part

number

number

number

[0061] Where:

number

number

number

number

number

number

number

number

number

number

number

number

number

[0062] The overall structure of Algorithm 2 can be similar to the overall structure of Algorithm 1. The segmentation mesh M c is a candidate segmentation primitive.

number

number

[0063] The generation of the low-poly mesh may further include generating a low-poly mesh based on the segmentation mesh. For example, the low-poly mesh may be generated based on the segmentation mesh M generated according to Algorithm 2. c After generating the volume intersection and segmentation mesh (e.g., M v and M. cAfter forming the input mesh M, the resulting segmented mesh may largely maintain the visual appearance of the input, but may have more triangles than desired. The desired number of triangles may be determined, for example, by a user. Thus, for the input mesh M i While maintaining low visual difference from the low-poly mesh M o To obtain the mesh, we can simplify the cutout mesh. For example, M c can be progressively remeshed through edge erasure and / or edge reversal operators. Thus, the first low-poly mesh is reconstructed into a segmented mesh M c A second low-poly mesh can be generated by performing a first edge erasing and / or edge reversing operation on the first low-poly mesh. Further, a second low-poly mesh can be generated by performing a second edge erasing and / or edge reversing operation on the first low-poly mesh. Each of the edge erasing and edge reversing operators can reduce two triangles. Each of the edge erasing and edge reversing operators reduces the respective low-poly mesh M o The operator may be run until no more triangles can be removed. The operator may be run in other embodiments until a certain number of triangles are removed or remain. Furthermore, only meshes with ≦T number of triangles may be stored. T may be set to the maximum number of elements a user can tolerate. Thus, by progressively running the edge erase and edge reversal operators, a low-poly mesh sequence may be formed. The low-poly mesh sequence may be displayed for user selection. The sequence may be displayed, for example, in a graphical user interface.

[0064] The sequence of low-poly meshes can be further ranked in some embodiments. For example, a Pareto efficiency concept can be applied to rank the low-poly meshes in the sequence of low-poly meshes. The ranking can be done by the number of faces and / or the visual difference τ nIn some embodiments, two metrics can be kept in the Pareto set, as shown in FIG. o Since selecting M is often subjective in practice, the meshes in the Pareto set can be visualized through the Pareto set shown in Figure 9. Thus, a user such as a game modeler can select M as the output. o In other embodiments, the user can manually select M from a low-poly mesh sequence without sorting. o You can also choose directly. n Note that it is also possible to automatically export meshes from the Pareto set just prior to the crystallization increment.

[0065] For comparison purposes, we implemented Algorithms 1 and 2 in C++ using GLSL shaders for metric computation, Eigen for linear algebra routines, CGAL for rational number computation, the Clipper library for silhouette computation, and mesh placement encapsulated in libigl for 3D exact Boolean operations.

[0066] In this implementation, a GLSL shader was used to calculate the visual difference between any two meshes. GLSL is a C-like language, tailored for use with graphics, and contains useful features specifically targeted at vector and matrix manipulation. Given a view direction, the camera is set to be 3l away from the meshes, where l is the maximum length of the bounding boxes of both meshes. Each mesh is rendered into a 128x128 framebuffer and the d s and d n Calculated τ n and τ s is S 2 We evaluate the model by repeating the process for C uniformly distributed view directions. We assume that the building model always faces upwards, so S 2is a hemisphere. We also evaluated the number of view directions and its effect on the corresponding visual metrics. The metric values ​​converge when C ≥ 250. To be conservative, we set C to 10 in the experiments described below. 3 was set to.

[0067] The third stage of the pipeline (e.g., M o During the remeshing of M c Edge elimination and edge inversion are performed iteratively while preserving the topology of θ. In edge elimination, QEM is employed to rank all edges and add virtual vertical planes with small weights to each edge to combat coplanar degeneracy, which is a common numerical instability in QEM. If any adjacent triangle of an edge is θ π If the dihedral angle of the edge is greater than θ 2π If θ is greater than θ, edge inversion is performed. π is a threshold close to π, and θ 2π is a threshold close to 2π.

[0068] Regarding hyperparameters in the implementation of this disclosure, the following default parameter settings were used: α=175°, which is the M i β=1°, which is the threshold for two directions to be considered overlapping; k=50 for selecting the set of view directions for the generation of hull intersections and the slice planes during the generation of segmented meshes; θ for edge flipping during the formation of low-poly meshes π = 175° and θ 2π = 355°. The stopping criterion for silhouette simplification applied during the generation of the volume intersection and segmentation meshes, ε d was set to 1% of the maximum diagonal length of the bounding square of all silhouettes. The filtering process discards all loops with a surface area less than 1% of the maximum area of ​​the bounding square of all silhouettes.

[0069] ε τ and N are both M v , M c, and indirectly M o As shown in Figure 10, a smaller ε τ The stopping criterion for is now more detailed and c where ( , ) is the (ε τ , τ n ) is shown. τ and N may not be very intuitive for end users to adjust. Therefore, empirically, ε τ and N by default ε τ =1×10 -3 and N = 20. In practice, a single parameter can be exposed to the user, which can be, for example, the maximum number of triangles T of the output low-poly mesh. In all experiments in this disclosure, T = 600.

[0070] It should be noted that although various parameters and settings are described above for an example implementation, other settings and parameters may be utilized.

[0071] For comparison purposes, experiments were performed based on the implementation described above. The experiments were performed on a workstation with a 32-core Intel processor clocked at 3.5 GHz and 64 Gb of memory, using OpenMP to parallelize silhouette-related calculations and candidate selection during volume intersection and segmentation mesh generation.

[0072] For the dataset, we collected 100 building models with various styles that commonly appear in digital images or environments such as games. Examples of building models are shown in FIG. 1. The models in the dataset can have complex geometry and topology, with 39% being non-manifold and 88% being non-watertight. Of these models, the average number of triangles, intersecting face pairs, and separated components is 20k, 35.6k, and 685, respectively. For models that are manifold, the average number of genus and holes is 136 and 640, respectively, where genus can indicate the number of holes in the surface. Table 1 provides exemplary statistics of the input models that appear in FIG. 2, FIG. 3, FIG. 6, FIG. 11A, FIG. 11B, FIG. 12A, FIG. 12B, FIG. 12C, FIG. 12D, and FIG. 13. FIG. 1 shows examples of both building models (input meshes) and low-poly meshes that can be generated based on the method of the present disclosure for the entire dataset. Further, a comparison of results generated by related methods with results generated by the methods of the present disclosure is shown in Figures 10, 11A, 11B, 12A, 12B, 12C, 12D, and 13, as well as Tables 1 and 2.

[0073] [Table 3]

[0074] As shown in Table 1, the number of vertices, N V , face N F , genus N G , intersecting surface pair N S , component N C , and hole N H , and whether it is a manifold or not M an The statistics of the input models shown in Fig. 2, Fig. 3, Fig. 6, Fig. 11A, Fig. 11B, Fig. 12A, Fig. 12B, Fig. 12C, Fig. 12D, and Fig. 13 are summarized, including: Note that for non-manifold meshes, genus and holes are not well-defined.

[0075] To evaluate the effectiveness of the mesh cutout in preserving the visual appearance, we performed a mesh cut of all 100 models. vand M. c For both τ n The average τ n (M v ) and τ n (M c ) are 0.066 and 0.056, which indicates that mesh cropping can reduce the visual disparity by 15.2%, and is important for preserving the input normals (or appearance).

[0076] We compared it with commercial software such as InstaLOD and Simplygon, which can automatically generate simplified meshes and are state-of-the-art commercial solutions used by game studios. Results from InstaLOD and Simplygon are shown in Table 2 and Figures 11A and 11B. Results are consistent with the InstaLOD optimization (InstaLOD 1 ), InstaLOD remeshing (InstaLOD 2 ), Simplygon decrease (Simplygon 1 ), and Simplygon remeshing (Simplygon 2 It was generated using various modules provided by InstaLOD and Simplygon, including the number of faces, τ s , τ n The results of the low-poly meshes generated by the disclosed method are compared with those of InstaLOD and Simplygon in terms of the mean and standard deviation of the simplification ratio. By default, the meshes from the Pareto front were manually selected as the final result (or final low-poly mesh). Simplygon 2 Since is the best performing of the four modules provided by both InstaLOD and Simplygon, to be fair, we will use Simplygon 2 The results of our method are shown in Fig. 1 from a Pareto front with a face count that matches the number of faces of the mesh by S) were extracted. As shown in Table 2, the method of the present disclosure produces results with smaller element counts and better visual appearance preservation. Figures 11A and 11B compare the four methods of the two software with the method of the present disclosure based on two sample models (or input meshes), where (·,·) are the (N F , τ n ) is shown.

[0077] Furthermore, we compare it with PolyFit, an open-source, state-of-the-art polygon mesh reconstruction method. To adopt PolyFit for low-poly meshing, we limit the number of point samples to 1M and the diagonal length of its bounding box to 1×10. -4 Each 3D model was uniformly sampled into a point cloud using the point cloud library by setting as the sampling interval. A batch process of the dataset was performed with the CGAL implementation of PolyFit with default parameter settings. PolyFit was unable to produce any results for nine models because not enough primitives were found. The results successfully processed by PolyFit were summarized and compared with the results of the method of this disclosure by matching the face count of the results generated by PolyFit (Ours P 12A, 12B, 12C, and 12D show an exemplary comparison between the results of the disclosed method and PolyFit, where (·,·) is the number of faces (τ n ) As shown in Table 2 and Figures 12A, 12B, 12C, and 12D, the results of the disclosed method can maintain more visual appearance than PolyFit with the same number of faces.

[0078] Further, a comparison was made with an alternative pipeline. To handle models with many open boundaries, an alternative pipeline for generating a simplified mesh is to first apply mesh repair and then run a remeshing algorithm. The results of the disclosed method were further compared with those generated by a two-stage alternative pipeline, for example, where a watertight mesh was first generated and then QEM-guided mesh simplification was applied using meshlab. Figure 13 shows the comparison results of the watertight mesh, mesh simplification without topology preservation, mesh simplification with topology preservation, and the results of the disclosed method, where (·,·) are the number of faces (τ n ) As demonstrated in Fig. 13, although the mesh repair approach can repair the mesh to some extent, it is still difficult for the local remeshing operator to produce satisfactory results when the desired number of elements is low.

[0079] [Table 4]

[0080] As shown in Table 2, the percentage of watertight results, W, and the average number of faces

number

number

number

number

number

number

number

[0081] In this disclosure, an approach is provided for generating low-poly representations of commonly used 3D models. The approach can be applied to digital games, including, for example, building models. A feature of this disclosure is that it relies on a view volume intersection to generate a proxy mesh whose topology is simple. An exemplary algorithm is provided for constructing and cutting out the view volume intersection using primitives selected for their structural simplicity. However, in other embodiments, modifications to one or more of the algorithms can be made. For example, planar slicing during the subtraction step can produce unsatisfactory results for models with smooth concave features. In this case, other proxy representations such as voxelization can be used if the input is accompanied by consistent inside-outside labeling. When the desired number of triangles in the low-poly mesh is greater than 1K, traditional remeshing methods such as QEM-guided edge erasure and commercial solutions may already be satisfactory. The method of this disclosure can be designed to generate the coarsest level of detail in some embodiments.

[0082] FIG. 14 shows a flowchart outlining an example process (1400) for generating a low-poly mesh sequence according to some embodiments of the present disclosure.

[0083] As shown in FIG. 14, the process (1400) can start at (S1401) and proceed to (S1410). At (S1410), a visual hull intersection can be generated. The visual hull intersection can be generated based on an intersection of first 3D primitives of the plurality of first silhouettes with a bounding box of the 3D model. The plurality of first silhouettes can be generated by projecting the 3D model onto a plane perpendicular to several selected view directions of the 3D model. Each of the first 3D primitives can be obtained by extruding a connected loop of the respective first silhouette along one view direction of several selected view directions associated with the respective first silhouette.

[0084] At (S1420), a segmentation mesh can be generated. The segmentation mesh can be generated based on subtraction of a second 3D primitive derived from a positive portion of the 3D model to segment redundant structures from the volume intersection. The positive portion can be obtained based on a number of fitting planes that slice the 3D model.

[0085] In (S1430), a low-poly mesh sequence can be generated. The low-poly mesh sequence can be generated based on progressive simplification of the segmentation meshes.

[0086] In the process (1400), a plurality of fitting planes may be formed, and each region of the 3D model may be included in a respective one of the plurality of fitting planes. A plurality of candidate view directions may be determined. Each of the plurality of candidate view directions may be parallel to a respective pair of fitting planes and may be associated with a respective weight value. The respective weight value may be equal to a sum of areas of the regions of the 3D model included in the respective pair of fitting planes. A number of selected view directions may be determined from the plurality of candidate view directions associated with the top k weight values, where k may be a positive integer.

[0087] In some embodiments, each of the plurality of first silhouettes of the 3D model can be simplified through at least one of 2D simplification or shape size filtering. One or more hollow loops can be extracted from each of the plurality of first silhouettes. One or more connected loops can be generated for each of the plurality of first silhouettes by subtracting the one or more hollow loops from a 2D bounding box of each of the plurality of first silhouettes. The first 3D primitive can be formed by extruding one or more connected loops of the plurality of first silhouettes along a number of selected view directions.

[0088] In the process (1400), a first virtual volume intersection may be formed by performing a Boolean intersection operation on a first one of the first 3D primitives and a bounding box of the 3D model to form a virtual volume intersection. A first visual disparity improvement may be determined based on the first virtual volume intersection and the 3D model. In response to the visual disparity improvement being greater than a threshold, a second virtual volume intersection may be formed by performing a Boolean intersection operation on a second one of the first 3D primitives and the first virtual volume intersection. A second visual disparity improvement may be determined based on the second virtual volume intersection and the 3D model.

[0089] To determine a first visual disparity improvement based on the first virtual volume intersection and the 3D model, an initial visual disparity can be determined based on an average pixel-wise disparity between a bounding box of the 3D model and the 3D model. The first visual disparity can be determined based on an average pixel-wise disparity between the first virtual volume intersection and the 3D model. The first visual disparity improvement can then be determined by subtracting the first visual disparity from the initial visual disparity.

[0090] In some embodiments, in response to one of (i) the nth visual disparity improvement being less than a threshold, and (ii) n being equal to an upper limit N, the nth virtual hull intersection may be determined as the virtual hull intersection. N may be a positive integer. The nth virtual hull intersection may be formed by performing a Boolean intersection operation on the nth one of the first 3D primitives and the (n-1)th virtual hull intersection.

[0091] In some embodiments, the 3D model may be sliced ​​by a plurality of fitting planes to form the cutout mesh. Each of the plurality of fitting planes may slice the 3D model into a positive portion and a negative portion. The positive portion of the 3D model may be located on a positive side of each of the plurality of fitting planes. Each of the positive portions of the 3D model may be projected onto a corresponding fitting plane of the plurality of fitting planes to obtain a respective second silhouette. An enlarged bounding square of each second silhouette may be obtained on the corresponding fitting plane such that the respective second silhouette is included in the enlarged bounding square. A Boolean subtraction operation may be operated to subtract the respective second silhouette from the enlarged bounding square to obtain a boundary loop from the respective second silhouette. A second 3D primitive of one of the second 3D primitives may be formed by extruding the boundary loop of the respective second silhouette in a normal direction from a positive side of the fitting plane corresponding to the respective second silhouette.

[0092] In the process (1400), a first provisional segmentation mesh may be formed by performing a Boolean subtraction operation on a first one of the second 3D primitives and the visual hull intersection to form a segmentation mesh. A first visual disparity improvement may be determined based on the first provisional segmentation mesh and the 3D model. In response to the visual disparity improvement being greater than a threshold, a second provisional segmentation mesh may be formed by performing a Boolean subtraction operation on a second one of the second 3D primitives and the first provisional segmentation mesh. A second visual disparity improvement may be determined based on the second provisional segmentation mesh and the 3D model.

[0093] In some embodiments, in response to one of (i) the nth visual disparity improvement being less than a threshold and (ii) n being equal to an upper limit N, the segmentation mesh may be determined to be the nth provisional segmentation mesh, where N may be a positive integer, and the nth provisional segmentation mesh may be formed by performing a Boolean intersection operation on the nth one of the second 3D primitives and the (n-1)th provisional segmentation mesh.

[0094] To generate the low-poly mesh sequence, edge erasure and edge reversal operations can be performed progressively on the segmented meshes to generate the low-poly mesh sequence. Each of the low-poly meshes in the low-poly mesh sequence can include a respective number of triangles that is less than a user-selected value T. T can be a positive integer, and the number of triangles in each of the low-poly meshes can be less than the number of triangles in the segmented meshes.

[0095] In the process (1400), the low-poly meshes in the sequence of low-poly meshes may be ranked based on the number of faces and visual differences of the low-poly meshes to generate a Pareto set.

[0096] The techniques described above can be implemented as computer software using computer readable instructions and physically stored on one or more computer readable media. For example, Figure 15 illustrates a computer system (1500) suitable for implementing certain embodiments of the disclosed subject matter.

[0097] Computer software may be coded using any suitable machine or computer language, which may be processed by assembly, compilation, linking or similar mechanisms to generate code comprising instructions that may be executed directly, interpreted, executed by microcode, or otherwise by one or more computer central processing units (CPUs), graphics processing units (GPUs), or other processing circuitry.

[0098] The instructions may be executed on various types of computers or computer components including, for example, personal computers, tablet computers, servers, smart phones, gaming consoles, Internet of Things devices, and the like.

[0099] 15 for computer system (1500) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. The arrangement of components should not be interpreted as having any dependency or requirement regarding any one or combination of components illustrated in the exemplary embodiment of computer system (1500).

[0100] The computer system (1500) may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, via tactile input (keystrokes, swipes, data glove movements, etc.), audio input (voice, clapping, etc.), visual input (gestures, etc.), olfactory input (not shown). The human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (e.g., voice, music, ambient sounds), images (e.g., scanned images, photographic images, still images obtained from a camera, etc.), and video (two-dimensional video, three-dimensional video including stereoscopic video, etc.).

[0101] The input human interface devices may include one or more of a keyboard (1501), a mouse (1502), a trackpad (1503), a touch screen (1510), a data glove (not shown), a joystick (1505), a microphone (1506), a scanner (1507), a camera (1508), etc. (only one of each is shown).

[0102] The computer system (1500) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the senses of a human user, for example, through haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touch screen (1510), data gloves (not shown), or joystick (1505), although there may also be haptic feedback devices that do not function as input devices), audio output devices (such as speakers (1509), headphones (not shown)), visual output devices (such as screens (1510), including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touch screen input capability, each with or without haptic feedback capability, some of which may be capable of outputting two-dimensional visual output, or output in more than three dimensions by means of stereoscopic output, such as virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0103] The computer system (1500) may also include human accessible storage devices and associated media for such storage devices, such as optical media including CD / DVD ROM / RW (1520) with CD / DVD or similar media (1521), thumb drives (1522), removable hard drives or solid state drives (1523), legacy magnetic media such as tapes or floppy disks (not shown), dedicated ROM / ASIC / PLD based devices such as security dongles (not shown), etc.

[0104] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0105] The computer system (1500) may also include an interface (1554) to one or more communication networks (1555). The networks may be, for example, wireless, wired, optical. The networks may further be local, wide area, metropolitan, vehicular and industrial, real-time, delay tolerant, etc. Examples of networks include local area networks such as Ethernet, wireless LAN, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area digital networks including cable TV, satellite TV and terrestrial broadcast TV, vehicular and industrial including CANBus, etc. Certain networks generally require an external network interface adapter attached to a particular general-purpose data port (e.g., a USB port of the computer system (1500)) or peripheral bus (1549), while other networks are generally integrated into the core of the computer system (1500) by connection to a system bus described below (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1500) may communicate with other entities. Such communications may be unidirectional, receive only (e.g., television broadcast), unidirectional transmit only (e.g., a CANbus to a particular CANbus device), or bidirectional, for example, to other computer systems using local or wide area digital networks. Specific protocols and protocol stacks may be used with each of these networks and network interfaces, as described above.

[0106] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be connected to the core (1540) of the computer system (1500).

[0107] The cores (1540) may include processing circuitry such as one or more central processing units (CPUs) (1541), graphics processing units (GPUs) (1542), dedicated programmable processing units in the form of field programmable gate areas (FPGAs) (1543), hardware accelerators for specific tasks (1544), graphics adapters (1550), etc. These devices may be connected via a system bus (1548) along with read only memory (ROM) (1545), random access memory (1546), internal mass storage (1547) such as internal hard drives, SSDs, etc. that are not accessible to the user. In some computer systems, the system bus (1548) may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus (1548) or via a peripheral bus (1549). In one example, a screen (1510) may be connected to the graphics adapter (1550). Architectures for peripheral buses include PCI, USB, etc.

[0108] The CPU (1541), GPU (1542), FPGA (1543), and accelerator (1544) can execute certain instructions that, in combination, can constitute the aforementioned computer code. This computer code can be stored in ROM (1545) or RAM (1546). Transient data can also be stored in RAM (1546), while permanent data can be stored, for example, in internal mass storage (1547). Fast storage and retrieval from any of the memory devices can be enabled by the use of cache memory, which can be closely associated with one or more of the CPU (1541), GPU (1542), mass storage (1547), ROM (1545), RAM (1546), etc.

[0109] The computer-readable medium can bear computer code for performing various computer-implemented operations. The medium and computer code can be those specially designed and constructed for the purposes of the present disclosure, or they can be of the available kind well known to those skilled in the computer software arts.

[0110] By way of example and not limitation, a computer system (1500) having an architecture, and in particular a core (1540), may provide functionality as a result of a processor (one or more) (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media may be user-accessible mass storage as described above, as well as media associated with specific storage of the core (1540) that is non-transitory in nature, such as the core internal mass storage (1547) or ROM (1545). Software implementing various embodiments of the present disclosure may be stored in such devices and executed by the core (1540). The computer-readable media may include one or more memory devices or chips, depending on the particular needs. The software may cause the core (1540), and in particular the processors (including a CPU, GPU, FPGA, etc.) therein, to perform certain processes or certain portions of certain processes described herein, including defining data structures stored in RAM (1546) and modifying such data structures according to the software-defined processes. Additionally or alternatively, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator (1544)) that may operate in place of or in conjunction with software to perform certain processes or certain portions of certain processes described herein. References to software may encompass logic, and vice versa, where appropriate. References to computer-readable media may encompass circuitry (such as integrated circuits (ICs)) that stores software for execution, circuitry that embodies logic for execution, or both, where appropriate. The present disclosure encompasses any suitable combination of hardware and software.

[0111] While this disclosure describes several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure. [Explanation of symbols]

[0112] 102 First Building Model 104, 108, 208, 308 low poly meshes 106 Second Building Model 202 Building Model 204, 206 Simplified mesh 302, 402, 602, 702 input meshes 304, 404, 406, 710, 802 Volume Intersection 306, 712 Cutting mesh 502 silhouettes 506, 508 cw loop 604~607 Loop 612, 614, 616, 618, 708 Primitives 804 First temporary cut mesh 806 Second temporary cut mesh 808 Final cut mesh 810 First Primitive 812 Second Primitive 814 The Third Primitive 1500 Computer System 1501 Keyboard 1502 Mouse 1503 Trackpad 1505 Joystick 1506 Mike 1507 Scanner 1508 Camera 1509 Speaker 1510 Touch Screen 1520 CD / DVD ROM / RW 1521 CD / DVD or similar media 1522 Thumb Drive 1523 Removable Hard Drive or Solid State Drive 1540 Core 1541 Central Processing Unit (CPU) 1542 Graphics Processing Unit (GPU) 1543 Field Programmable Gate Area (FPGA) 1544 Hardware Accelerator 1545 Read-Only Memory (ROM) 1546 Random Access Memory 1547 Internal Mass Storage 1548 System Bus 1549 Peripheral Bus 1550 Graphics Adapter 1554 Interface 1555 Communication Network

Claims

1. 1. A method for generating a low-poly mesh sequence for a three-dimensional (3D) model, the method comprising: generating a view volume intersection based on an intersection of first 3D primitives of a plurality of first silhouettes with a bounding box of the 3D model, the plurality of first silhouettes being generated by projecting the 3D model onto a plane perpendicular to several selected view directions of the 3D model, each of the first 3D primitives being obtained by extruding a connected loop of the respective first silhouette along one of the several selected view directions associated with the respective first silhouette; forming a segmentation mesh based on subtraction of a second 3D primitive derived from a positive portion of the 3D model to segment redundant structures from the visual hull, the positive portion being obtained based on a number of fitting planes slicing the 3D model; generating the low-poly mesh sequence based on a progressive simplification of the segmented meshes; The method includes:

2. The step of generating the visual hull comprises: forming a plurality of fitting planes, each region of the 3D model being included in a respective one of the plurality of fitting planes; determining a plurality of candidate view directions, each of the plurality of candidate view directions being parallel to a respective pair of fitting planes and associated with a respective weight value, the respective weight value being equal to a sum of areas of the regions of the 3D model that are contained in the respective pair of fitting planes; determining the number of selected view directions from the plurality of candidate view directions associated with top k weight values, where k is a positive integer; The method of claim 1, further comprising:

3. The step of generating the visual hull comprises: simplifying each of the plurality of first silhouettes of the 3D model through at least one of 2D simplification or shape size filtering; Extracting one or more hollow loops from each of the plurality of first silhouettes; generating one or more connected loops for each of the plurality of first silhouettes by subtracting the one or more hollow loops from a 2D bounding box of each of the plurality of first silhouettes; forming the first 3D primitive by extruding the one or more connected loops of the plurality of first silhouettes along the number of selected view directions; The method of claim 1, further comprising:

4. The step of generating the visual hull comprises: forming a first virtual volume intersection by performing a Boolean intersection operation on a first one of the first 3D primitives and the bounding box of the 3D model; determining a first visual disparity improvement based on the first virtual volume intersection and the 3D model; forming a second virtual volume intersection by performing the Boolean intersection operation on a second one of the first 3D primitives and the first virtual volume intersection in response to the visual disparity improvement being greater than a threshold; determining a second visual disparity improvement based on the second virtual volume intersection and the 3D model; The method of claim 1, further comprising:

5. The step of determining the first visual disparity improvement based on the first virtual volume intersection and the 3D model comprises: determining an initial visual disparity based on an average pixel-by-pixel disparity between the bounding box of the 3D model and the 3D model; determining a first visual disparity based on an average pixel-by-pixel disparity between the first virtual volume intersection and the 3D model; determining the first visual disparity improvement by subtracting the first visual disparity from the initial visual disparity; 5. The method of claim 4, further comprising:

6. The step of generating the visual hull comprises: determining an nth virtual hull intersection as the virtual hull intersection in response to one of (i) the nth visual disparity improvement being less than the threshold, and (ii) the nth visual disparity improvement being equal to an upper limit N, where N is a positive integer, the nth virtual hull intersection being formed by performing the Boolean intersection operation on the nth one of the first 3D primitives and a (n-1)th virtual hull intersection.

5. The method of claim 4, further comprising:

7. The step of forming a segmented mesh comprises: slicing the 3D model with the plurality of fitting planes, each of the plurality of fitting planes slicing the 3D model into a positive portion and a negative portion, the positive portion of the 3D model being located on the positive side of the respective one of the plurality of fitting planes; projecting each of the positive portions of the 3D model onto a corresponding one of the plurality of fitting planes to obtain a respective second silhouette; obtaining an enlarged bounding square of each of the second silhouettes on the corresponding fitting plane such that the respective second silhouettes are contained within the enlarged bounding square; performing a Boolean subtraction operation to subtract the respective second silhouette from the augmented bounding square to obtain a bounding loop from the respective second silhouette; forming a second 3D primitive of one of the second 3D primitives by extruding the boundary loop of the respective second silhouette in a normal direction from the positive side of the fitting plane corresponding to the respective second silhouette; The method of claim 1, further comprising:

8. The step of forming the segmented mesh comprises: forming a first provisional segmented mesh by performing a Boolean subtraction operation on a first one of the second 3D primitives and the visual hull intersection; determining a first visual disparity improvement based on the first provisional segmentation mesh and the 3D model; forming a second provisional segmented mesh by performing the Boolean subtraction operation on a second one of the second 3D primitives and the first provisional segmented mesh in response to the visual disparity improvement being greater than a threshold; determining a second visual disparity improvement based on the second provisional segmentation mesh and the 3D model; The method of claim 1, further comprising:

9. The step of forming the segmented mesh comprises: determining, in response to one of (i) the nth visual disparity improvement being less than the threshold, and (ii) the nth visual disparity improvement being equal to an upper limit N, that the segmentation mesh is an nth provisional segmentation mesh, where N is a positive integer, and the nth provisional segmentation mesh is formed by performing a Boolean intersection operation on the nth one of the second 3D primitives and the (n-1)th provisional segmentation mesh.

9. The method of claim 8, further comprising:

10. The step of generating the low-poly mesh sequence comprises: performing progressive edge erasure and edge reversal operations on the segmentation meshes to generate the sequence of low-poly meshes, each of the low-poly meshes in the sequence of low-poly meshes containing a respective number of triangles that is less than a user-selected value T, where T is a positive integer, and the number of triangles in each of the low-poly meshes is less than the number of triangles in the segmentation mesh; The method of claim 1, further comprising:

11. ranking the low-poly meshes in the sequence of low-poly meshes based on the number of faces and visual differences of the low-poly meshes to generate a Pareto set. The method of claim 1, further comprising:

12. 1. An apparatus for generating a low-poly mesh sequence for a three-dimensional (3D) model, comprising: generating a view volume intersection based on an intersection of first 3D primitives of a plurality of first silhouettes with a bounding box of the 3D model, the plurality of first silhouettes being generated by projecting the 3D model onto a plane perpendicular to a number of selected view directions of the 3D model, each of the first 3D primitives being obtained by extruding a connected loop of the respective first silhouette along one of the number of selected view directions associated with the respective first silhouette; forming a segmentation mesh based on subtraction of a second 3D primitive derived from a positive portion of the 3D model to segment redundant structures from the visual volume, the positive portion being obtained based on a number of fitting planes slicing the 3D model; generating the low-poly mesh sequence based on a progressive simplification of the segmented meshes; A processing circuit configured as follows: An apparatus comprising:

13. The processing circuitry includes: forming a plurality of fitting planes, each region of the 3D model being included in a respective one of the plurality of fitting planes; determining a plurality of candidate view directions, each of the plurality of candidate view directions being parallel to a respective pair of fitting planes and associated with a respective weight value, the respective weight value being equal to a sum of areas of the regions of the 3D model that are contained in the respective pair of fitting planes; determining the number of selected view directions from the plurality of candidate view directions associated with top k weight values, where k is a positive integer; The apparatus of claim 12, configured to:

14. The processing circuitry includes: simplifying each of the plurality of first silhouettes of the 3D model through at least one of 2D simplification or shape size filtering; Extracting one or more hollow loops from each of the plurality of first silhouettes; generating one or more connected loops for each of the plurality of first silhouettes by subtracting the one or more hollow loops from a 2D bounding box of each of the plurality of first silhouettes; forming the first 3D primitive by extruding the one or more connected loops of the plurality of first silhouettes along the number of selected view directions. The apparatus of claim 12, configured to:

15. The processing circuitry includes: forming a first virtual volume intersection by performing a Boolean intersection operation on a first one of the first 3D primitives of the 3D model and the bounding box; determining a first visual disparity improvement based on the first virtual volume intersection and the 3D model; forming a second virtual volume intersection by performing the Boolean intersection operation on a second one of the first 3D primitives and the first virtual volume intersection in response to the visual disparity improvement being greater than a threshold; determining a second visual disparity improvement based on the second virtual volume intersection and the 3D model; The apparatus of claim 12, configured to:

16. The processing circuitry includes: determining an initial visual disparity based on an average pixel-by-pixel disparity between the bounding box of the 3D model and the 3D model; determining a first visual disparity based on an average pixel-by-pixel disparity between the first virtual volume intersection and the 3D model; determining the first visual difference improvement by subtracting the first visual difference from the initial visual difference; The apparatus of claim 15, configured to:

17. The processing circuitry includes: in response to one of (i) the nth visual disparity improvement being less than the threshold, and (ii) the nth visual disparity improvement being equal to an upper limit N, determining an nth virtual hull intersection as the virtual hull intersection, N being a positive integer, the nth virtual hull intersection being formed by performing the Boolean intersection operation on the nth one of the first 3D primitives and a (n-1)th virtual hull intersection; The apparatus of claim 15, configured to:

18. The processing circuitry includes: slicing the 3D model with the plurality of fitting planes, each of the plurality of fitting planes slicing the 3D model into a positive portion and a negative portion, the positive portion of the 3D model being located on the positive side of each of the plurality of fitting planes; projecting each of the positive portions of the 3D model onto a corresponding one of the plurality of fitting planes to obtain a respective second silhouette; obtaining an expanded bounding square of each of the second silhouettes on the corresponding fitting plane such that the each of the second silhouettes is contained within the expanded bounding square; performing a Boolean subtraction operation to subtract the respective second silhouette from the augmented bounding square to obtain a bounding loop from the respective second silhouette; forming a second 3D primitive of one of the second 3D primitives by extruding the boundary loop of the respective second silhouette in a normal direction from the positive side of the fitting plane corresponding to the respective second silhouette; The apparatus of claim 12, configured to:

19. The processing circuitry includes: forming a first provisional segmented mesh by performing a Boolean subtraction operation on a first one of the second 3D primitives and the visual hull intersection; determining a first visual disparity improvement based on the first provisional segmentation mesh and the 3D model; forming a second provisional segmented mesh by performing the Boolean subtraction operation on a second one of the second 3D primitives and the first provisional segmented mesh in response to the visual disparity improvement being greater than a threshold; determining a second visual disparity improvement based on the second provisional segmented mesh and the 3D model; The apparatus of claim 12, configured to:

20. The processing circuitry includes: in response to one of (i) the nth visual disparity improvement being less than the threshold, and (ii) the nth visual disparity improvement being equal to an upper limit N, determining that the segmentation mesh is an nth provisional segmentation mesh, where N is a positive integer, and the nth provisional segmentation mesh is formed by performing a Boolean intersection operation on the nth one of the second 3D primitives and the (n-1)th provisional segmentation mesh; 20. The apparatus of claim 19, configured to:

Images