Occluder Generation for Structures in Digital Applications

Abstract

A method is performed in a computing system for automatically generating an occluder, the method including receiving an input model of a visual three-dimensional structure, the input model including a plurality of faces. The method generates an initial occluder by simplifying the input model into a plurality of candidate patches in a patch-based coarse mesh. The method includes comparing a first two-dimensional area occluded by the input model of the visual three-dimensional structure to a second two-dimensional area occluded by the initial occluder along a first viewing direction, and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder. The method includes removing the plurality of faces from the initial occluder while maintaining the first quality metric above a first threshold to form an occluder for the visual three-dimensional structure.

Description

[Technical Field]

[0001]

[0001] Priority and Related Applications This application is a continuation of U.S. patent application Ser. No. 17 / 866,413, filed July 15, 2022, entitled "Occluder Generation for Construction in Digital Applications," which claims priority to U.S. provisional application Ser. No. 63 / 302,916, filed January 25, 2022, entitled "Occluder Generation for Construction in Digital Games," each of which is incorporated herein by reference in its entirety.

[0002]

[0002] Technical field The disclosed implementations relate generally to the field of computer technology, and more particularly to occlusion culling in computer graphics. [Background technology]

[0003]

[0003] Modern game engines use occlusion culling techniques to remove occluded objects from the rendering pipeline. Objects are occluded for a particular viewing direction if there is another object in front of them, along that particular viewing direction. Summary of the Invention

[0004]

[0004] By culling obscured objects during application execution using an approximated coarse mesh (occluders) rather than a finer rendering mesh (visual mesh), graphical processing unit (GPU) bandwidth is conserved and rendering costs are reduced due to a reduction in the number of draw calls. A draw call instructs the GPU to prepare drawing resources based on information about textures, states, shaders, rendering objects, buffers, etc.

[0005] The quality of the occluder mesh affects the effectiveness and accuracy of culling. Low-poly meshes (e.g., polygon meshes with a relatively small number of polygons) reduce the cost of culling tests. The culling test checks whether a particular face is completely occluded by other faces. If so, that particular face is discarded from the rendering pipeline, saving the cost of an additional rendering step. In some embodiments, a low-poly mesh may contain approximately 200–300 polygons. Furthermore, occluders ideally should be conservative and completely within the volume of the visual mesh. Non-conservative occluders, then, can incorrectly cull visible objects (e.g., objects outside the volume of the visual mesh) and cause severe visual artifacts, as explained in the context of Figures 1B and 1C below.

[0006]

[0006] It is a part of the purpose of the present application to address the above challenges by presenting a set of solutions for automating the generation of occluders for three-dimensional structures.

[0007] According to one aspect of the present application, a method is performed in a computing system for automatically generating occluders for a visual three-dimensional structure. The method includes the following steps: receiving an input model of a visual three-dimensional structure, the input model including a plurality of faces; generating an initial occluder by simplifying the input model into a plurality of candidate patches in a patch-based coarse mesh, the initial occluder blocking objects behind the three-dimensional structure along a first viewing direction; comparing a first two-dimensional area occluded by an input model of the visual three-dimensional structure with a second two-dimensional area occluded by an initial occluder along a first viewing direction, and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder; and removing faces from the initial occluder while maintaining a first quality metric above a first threshold to form an occluder for the visual three-dimensional structure, the occluder preventing objects behind the three-dimensional structure from being rendered in the application along the first viewing direction.

[0008] According to another aspect of the present application, a computing system is provided for automatically generating occluders for a visual three-dimensional structure. The computing system includes one or more processors; a memory; and a plurality of programs stored in the memory. The plurality of programs, when executed by the one or more processors, cause the computing system to perform one or more operations, the one or more operations including: receiving an input model of a visual three-dimensional structure, the input model including a plurality of faces; generating an initial occluder by simplifying the input model into a plurality of candidate patches in a patch-based coarse mesh, the initial occluder blocking objects behind the visual three-dimensional structure along a first viewing direction; comparing a first two-dimensional area occluded by an input model of the visual three-dimensional structure with a second two-dimensional area occluded by an initial occluder along a first viewing direction, and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder; and removing faces from the initial occluder while maintaining a first quality metric above a first threshold to form an occluder for the visual three-dimensional structure, the occluder preventing objects behind the visual three-dimensional structure from being rendered in the application along the first viewing direction.

[0009] According to yet another aspect of the present application, a non-transitory computer-readable storage medium stores, in association with a computing system having one or more processors, a plurality of programs for automatically generating occluders for a visual three-dimensional structure, the plurality of programs, when executed by the one or more processors, causing the computing system to perform one or more operations, the one or more operations including: receiving an input model of a visual three-dimensional structure, the input model including a plurality of faces; generating an initial occluder by simplifying the input model into a plurality of candidate patches in a patch-based coarse mesh, the initial occluder blocking objects behind the visual three-dimensional structure along a first viewing direction; comparing a first two-dimensional area occluded by an input model of the visual three-dimensional structure with a second two-dimensional area occluded by an initial occluder along a first viewing direction, and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder; and removing faces from the initial occluder while maintaining a first quality metric above a first threshold to form an occluder for the visual three-dimensional structure, the occluder preventing objects behind the visual three-dimensional structure from being rendered in the application along the first viewing direction. [Brief explanation of the drawings]

[0010] The foregoing and additional implementation forms of the present invention will be more clearly understood as a result of the following detailed description of various aspects of the invention taken together with the drawings in which like reference numerals refer to corresponding parts throughout the several views of the drawing. [Figure 1A]

[0011] 1A-1D show how non-conservative occluders according to some implementations of the present application cull visible objects. [Figure 1B] 1A-1D illustrate how a non-conservative occluder according to some implementations of the present application culls out visible objects. [Figure 1C] 1A-1D illustrate how a non-conservative occluder according to some implementations of the present application culls out visible objects. [Figure 1D] 1A-1D illustrate how a non-conservative occluder according to some implementations of the present application culls out visible objects. [Figure 2A]

[0012] 2B-2E illustrate how various meshes are derived from the input model shown in FIG. 2A, according to some implementations of the present application. [Figure 2B]

[0012] Figures 2B-2E illustrate how various meshes are derived from the input model shown in Figure 2A, according to some implementations of the present application. [Figure 2C]

[0012] Figures 2B-2E illustrate how various meshes are derived from the input model shown in Figure 2A, according to some implementations of the present application. [Figure 2D]

[0012] Figures 2B-2E illustrate how various meshes are derived from the input model shown in Figure 2A, according to some implementations of the present application. [Figure 2E]

[0012] Figures 2B-2E illustrate how various meshes are derived from the input model shown in Figure 2A, according to some implementations of the present application. [Figure 3A]

[0013] 3A-3E show an input model, the results of patch-based simplification of the input model, the results of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification, according to some implementations of the present application. [Figure 3B]

[0013] Figures 3A-3E show an input model, the results of patch-based simplification of the input model, the results of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification, according to some implementations of the present application. [Figure 3C]

[0013] Figures 3A-3E show an input model, the results of patch-based simplification of the input model, the results of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification, according to some implementations of the present application. [Figure 3D]

[0013] Figures 3A-3E show an input model, the results of patch-based simplification of the input model, the results of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification, according to some implementations of the present application. [Figure 3E]

[0013] Figures 3A-3E show an input model, the results of patch-based simplification of the input model, the results of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification, according to some implementations of the present application. [Figure 4A]

[0014] 4A and 4B show flat and curved patches obtained using different types of grouping of input models according to some implementations of the present application. [Figure 4B] 4A and 4B show flat and curved patches obtained using different types of grouping of input models according to some implementations of the present application. [Figure 5]

[0015] FIG. 5 illustrates a verification process for checking whether a hole corresponds to a decorator, according to some implementations of the present application. [Figure 6]

[0016] Figure 6 shows an example of voxel-based mesh simplification according to some implementations of the present application. [Figure 7A]

[0017] 7A and 7B compare different view sampling methods according to some implementations of the present application. [Figure 7B] 7A and 7B compare different view sampling methods according to some implementations of the present application. [Figure 8A]

[0018] 8A-8D show examples of occlusion measurement acceleration according to some implementations of the present application. [Figure 8B] 8A-8D show examples of occlusion measurement acceleration according to some implementations of the present application. [Figure 8C]8A-8D show examples of occlusion measurement acceleration according to some implementations of the present application. [Figure 8D] 8A-8D show examples of occlusion measurement acceleration according to some implementations of the present application. [Figure 9A]

[0019] 9A and 9B show the standard deviation of recall and precision for different numbers of quadrilaterals per view and different sampling distances according to some implementations of the present application. [Figure 9B] 9A and 9B show the standard deviation of recall and precision for different numbers of quadrilaterals per view and different sampling distances according to some implementations of the present application. [Figure 10]

[0020] FIG. 10 shows a dataset of 77 building models used in a game according to some implementations of the present application. [Figure 11]

[0021] FIG. 11 shows the average recall and precision for various test buildings according to some implementations of the present application. [Figure 12]

[0022] FIG. 12 illustrates an example building and corresponding occluders generated in accordance with some implementations of the present application. [Figure 13]

[0023] FIG. 13 shows an example of closing a seam according to some implementations of the present application. [Figure 14A]

[0024] 14A and 14B show examples of some possible splitting planes and their corresponding slice areas for a heterogeneous building model, according to some implementations of the present application. [Figure 14B] 14A and 14B show examples of some possible splitting planes and their corresponding slice areas for a heterogeneous building model, according to some implementations of the present application. [Figure 15]

[0025] FIG. 15 illustrates a method for automatically generating occluders for visual 3D structures according to some implementations of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0011]

[0026] The following description of implementations refers to the accompanying drawings to illustrate specific implementations that may be implemented by the present application. Terms of direction used in the present application, such as "upper," "lower," "front," "rear," "left," "right," "inner," "outer," and "side," are used only to refer to the directions in the accompanying drawings. Therefore, the terms of direction used are not intended to limit the present application, but are used only to explain and understand the present application. In the drawings, units having similar structures are represented by the same reference numerals.

[0012]

[0027] 1A shows an actual view 100 from a position in front of a structure 102. The actual view 100 corresponds to the ground truth of what a player at that location would observe. A first structure 104 and a second structure 106 are visible in the actual view 100 from a position in front of the structure 102 along this viewing direction.

[0013]

[0028] 1B shows a view 108 from the same position in front of the structure 102 when a first non-conservative occluder is used. Examples of first non-conservative occluders include occluders created by Silvennoinen. A second structure 106 that would be visible from a position in front of the structure 102 along this line of sight in the actual view 100 is no longer visible in the view 108 because the first non-conservative occluder extends beyond the volume of the visual mesh, culling out the second structure 106 that would have been visible as shown in the actual view 100. Instead, a region 110 lacking the second structure 106 is shown in the view 108.

[0014]

[0029] 1C shows a view 112 from the same position in front of the structure 102 when a second, non-conservative occluder is used. Examples of second, non-conservative occluders include occluders generated by Simplygons. The second structure 106 that would be visible from a position in front of the structure 102 along this viewing direction in the actual view 100 is no longer visible in the view 112 because the second, non-conservative occluder extends beyond the volume of the visual mesh, culling out the second structure 106 that would have been visible as shown in the actual view 100. Instead, a region 114 devoid of the second structure 106 is shown in the view 112.

[0015]

[0030] 1D shows a view 116 from the same position in front of structure 102 when an occluder according to some embodiments of the present invention is used. A second structure 106, which would be visible from a position in front of structure 102 along this line of sight in the actual field of view 100, is again visible in field of view 116.

[0016]

[0031] Non-conservative occluders can erroneously cull visible objects, causing severe visual artifacts, as shown in Figures 1B and 1C. As disclosed in detail below, in accordance with some embodiments, we qualitatively measure the accuracy of an occluder across the 3D domain using two metrics: precision and recall: Precision measures the likelihood that an object blocked by an occluder is also blocked by the original model.

[0017] Recall measures the likelihood that an object blocked by the original model is also blocked by the occluder. The disclosed embodiments relate to automatically generating occluders for structures (e.g., building models) from game assets. Game artists manually create building models to maximize visual realism, and building models typically include numerous disconnected sections, large open doors and windows, and interior structures, as shown in FIG. 2A. FIG. 2A shows an input model 200 with an open door 202 and disconnected sections 204a and 204b. In some embodiments, model 200 may be an input mesh with 7,888 faces, 521 components, 562 boundary loops, and 9,034 intersecting triangle pairs. In some implementations, each boundary loop is a hole. Because artists only focus on the exterior of the building, building models are typically non-manifold, non-watertight, and / or self-intersecting, making them unsuitable for traditional mesh processing algorithms. In some embodiments, non-manifold geometry includes any edge shared by more than two faces. In some embodiments, at least one edge in a non-watertight model is a boundary edge. In other words, a non-watertight mesh has holes. In some embodiments, a model that is self-intersecting means that the mesh has two faces that intersect with each other.

[0018]

[0032] While it may be possible to manually adjust the occlusions of handcrafted buildings to achieve a good balance between precision and recall, such a process can be time-consuming. Furthermore, handcrafted occluders may still contain faces that do not contribute to recall or that violate conservative constraints. Such faces are sometimes called "wasted faces."

[0019]

[0033] One strategy for automatic occluder generation is First, the input mesh is voxelized, Extracting an isosurface (or isosurface) from the voxelized input mesh (e.g., an output isosurface), The output isosurface is then simplified, The idea is to insert an axis-aligned box or cross section to form the occluder. In some embodiments, the axis-aligned box (AAB) is simply a rectangular prism, with each face of the prism perpendicular to one of the basis vectors. Using such a strategy can cause some important features, such as thin walls, not to be captured at a reasonable resolution, as shown in FIG. 2B . FIG. 2B shows an isosurface 206 corresponding to a winding number of 0.5. Compared to the input model 200, the isosurface 206 lacks a thin wall 208. Furthermore, nested open structures within a building model can result in ill-defined orientations, which can prevent the isosurface from being accurately extracted. For example, an edge (or face) within an open structure may have a front direction that is perpendicular to one edge of the open structure and parallel to another edge of the open structure, making the orientation of the open structure ill-defined. For example, two connected faces may have opposite normal directions.

[0020]

[0034] 2C shows a mesh 212 obtained by applying Sirvenoinen's method to the isosurface 206. The resulting mesh 212 is obtained by inserting a cutting plane. In some embodiments, the resulting mesh 212 may contain 150 faces.

[0021]

[0035] The second strategy involves gradually removing faces from the input mesh through error-guided element-removal operations. In some embodiments, the second strategy involves operations such as edge fracturing, which can produce results with large gaps or portions outside the visual mesh. Such a resulting mesh may violate conservativity, as in the example shown in FIG. 2D . FIG. 2D shows an occluder 214 generated by a simpligon. Portion 216 extends beyond the volume of the input model 200, and portion 218 covers the open door 202 in the input model 200. Thus, the occluder 214 violates conservativity. In some embodiments, the occluder 214 may contain 231 faces.

[0022]

[0036] The Simpligon and Silvenoinen method assumes that buildings are viewed from a distance and does not consider concave and / or interior building structures, therefore, such output occluders fail to provide accurate occlusion when game characters enter these areas.

[0023]

[0037] Rather than relying on one strategy and hoping it will be generally sufficient to handle all building models with various styles, the disclosed embodiments first generate two coarse meshes from the input mesh using two different methods. The two coarse meshes populate a large solution space with a set of candidate faces. Figure 2E shows an occluder generated using the disclosed methods and systems. In some embodiments, the occluder 214 may contain 231 faces.

[0024]

[0038] The candidate occluder face set is evaluated using precision and recall metrics with respect to the input model on a 3D evaluation domain. The methods and systems described herein use a metric-based algorithm to select the best face set from within a solution space (e.g., candidate face set) with a high occlusion rate while maintaining as much conservatism as possible. The methods and systems also incorporate one or more strategies to accelerate metric computation at runtime.

[0025]

[0039] In some embodiments, the methods and systems described herein are validated using 77 building models with various styles. In some embodiments, the methods described herein generate occluders with a low face count of 260, while achieving an average precision of 99.4% and an average recall of 78.0% from all possible observer positions, including far-away, close-up, and walk-in views. These results reflect an increase of 3.7% and 2.9% in terms of precision and recall, respectively, compared to occluders generated by Simplgon, while using 50 fewer faces on average. Results using Silvenoinen's method yield an average recall of only 39.7%.

[0026]

[0040] Below we summarize the relevant techniques for occlusion culling, mesh simplification, and occluder generation.

[0027]

[0041] Occlusion Culling: For static scenes, it is possible to pre-compute and store a set of possible visibility points for a single viewpoint or region of viewpoints. For buildings with accessible interiors, cell-and-portal models break the interior down into rooms (cells) connected by doors or windows (doorways, gates, portals). However, pre-computing and storing visibility data for complex scenes in large open worlds can be costly. To avoid excessive pre-computation and storage, view-dependent subsets of the input mesh can be maintained as virtual occluders at runtime.

[0028]

[0042] Mesh Simplification: Software rasterization can be used to render the coarse mesh (occluders) to a depth buffer, which is then used to cull obscured objects at a very early stage. Generating an approximate coarse mesh from a fine mesh can be achieved by collapsing edges that meet certain conditions or by minimizing certain metrics (e.g., Quadric Error Metrics (QEM)). Other metrics, such as surface visibility metrics, can be defined and added to collapsing conditions for specific applications. Other techniques involve satisfying hard constraints during remeshing. For example, progressive hulls can be introduced to ensure that all vertices are outside the input mesh, or a coarse mesh can be generated while maintaining strict nesting. Unfortunately, these methods do not work well in topologically inconsistent cases. For example, a topologically inconsistent case can involve edges (or faces) in an open structure where two connected faces have opposite normal directions, as mentioned above.

[0029]

[0043] Occluder Generation: Conservative mesh simplification can be used to generate occluders for terrain patches in games while assuming a clean topology. If the topology can be made consistent, it is possible to generate occluders using existing mesh simplification methods. Voxelization can be used for this purpose. For example, a set of planes inside a voxelized input model is selected to form an occluder with bounded occlusion error. However, using a voxelized mesh as input can introduce large occlusion errors during voxelization. Buildings with nested structures are non-orientable and cannot be voxelized, while thin walls cannot be captured with reasonable resolution. In the gaming industry, collision meshes are used as starting meshes for simplification, avoiding the need to manually create occluders for every building. However, collision meshes are typically larger than the input mesh, severely violating conservativeness.

[0030]

[0044] Illustrative Embodiments

[0045] In some embodiments, an input building model (e.g., input model 200) is represented as a triangle / polygon soup, containing hundreds of (potentially self-intersecting) disconnected components, nested structures, and thin features. Due to topological complexity, neither voxelization nor traditional mesh simplification may work well on their own. The methods and systems described herein recognize that some disconnected components contain many large patches that are useful candidates for forming the final occluder mesh, while other large bulky features can be adequately captured by voxelization. Therefore, the methods and systems described herein use a hybrid approach, as shown in FIG. 3D , that combines the output from two mesh simplification strategies (patch-based simplification as shown in FIG. 3B and voxelization-based simplification as shown in FIG. 3C ), to compile a large set of face candidates into an initial occluder mesh. occluder (shown in Figure 3D). The two evaluation metrics (precision P and recall R) are occluder It is formulated in terms of M occluder Using metric-guided mesh simplification, a high-quality final occluder is extracted from the initial occluder, as shown in Figure 3E. Metric-guided mesh simplification terminates when a user-specified quality limit is reached. Figure 3A shows an input model 300. Figure 3B shows the result 302 of patch-based simplification of the input model 300. Figure 3C shows the result 304 of voxelization-based simplification of the input model 300. Figure 3D shows the combination 306 of the patch-based simplification result 302 and the voxelization-based simplification result 304. Figure 3E shows the final occluder 308 after metric-guided mesh simplification.

[0031]

[0046] Initial occluder generation:

[0047] Two hybrid methods are used to form the initial set of face candidates.

[0032]

[0048] Patch-based mesh simplification:

[0049] The first approach is to generate a coarse mesh by performing one of the following steps:

[0050] 1. Planar patch grouping: Group pairs of faces into planar patches if the dihedral angle of one shared edge is less than a threshold. In some embodiments, the threshold is 1×10 -2 Less than 5 x 10 -3 Less than 1 x 10 -3 In some embodiments, the threshold is 1×10 -3 is.

[0033]

[0051] 2. Curved Patch Grouping: The dihedral angle of one shared edge is less than the user-specified threshold ε a Group the planar patches generated from the last step into curved patches if they are less than a user-specified threshold ε a is smaller than the threshold used for planar patch grouping. Using planar patches may result in occluders with fewer triangles, while using curved patches captures more detailed information. Combining these two patch sets results in a larger candidate set from which the final occluder is selected. An example is shown in Figures 4A and 4B. Figure 4A shows planar patches 400 obtained using planar patch grouping of the input model 300. Figure 4B shows curved patches obtained using curved patch grouping of the input model 300.

[0034]

[0052] 3. Simplify: In some embodiments, each curved patch is simplified using QEM-guided mesh simplification. Other methods, such as variational shape approximation (VSA), can also be used. In some embodiments, the boundary of each 2D projected patch of each planar patch is simplified using the Ramer-Douglas-Peucker algorithm. Many simplification methods exist, and the main idea is to remove vertices that have little impact on the shape. After simplification, the boundary is retriangulated into a triangular mesh using constrained Delaunay triangulation. While other methods are also used, Delaunay triangulation is one of the most common and useful methods.

[0035]

[0053] 4.Hole filling: There are holes in the triangle mesh from step 3, but M Input If does not contain such a hole, the hole in the triangle mesh is filled. The hole is filled with multiple triangle faces according to the edge number of the hole.

[0036]

[0054] 5. Reduction: The mesh simplification process then sorts all planar and curved patches according to their area, and patches (faces) are sorted according to the user-specified number N P are added to the final mesh until a curved patch is selected over a planar patch. In some embodiments, there is no preference to select a curved patch over a planar patch, and only the area of ​​the patch is used in the selection process.

[0037]

[0055] Hole filling:

[0056] In some embodiments, hole filling is a mesh repair technique. However, in building models, some holes correspond to building decorations such as windows and doors and need to be left open for the preservation of the occluders.

[0038]

[0057] Figure 5 illustrates the verification process for checking whether a hole corresponds to a decorator. Figure 5 shows an input mesh 500 (e.g., M input ) The verification process begins by triangulating the hole into multiple faces. The hole itself is defined by its boundary edges and does not have any faces. There are multiple faces associated with a hole only if the hole is to be filled (e.g., to fill the hole). For example, Figure 5 shows a planar patch 504 with a hole 506 extracted from an input mesh 500. s A set of test line segments with (L-S) is sampled uniformly along the normal direction inside each face. For example, Figure 5 shows a test line segment 508 placed inside a hole 506 in a planar patch 504. M input As long as there is at least one segment that does not intersect with input If all segments are in contact with the input mesh, M input There are no openings in the patch, and the holes can be reliably filled. After hole filling, the method and system meshes and simplifies the patch to further reduce the number of faces. For example, Figure 5 shows the output mesh 510 after hole filling and remeshing.

[0039]

[0058] Voxelization-based mesh simplification:

[0059] The second approach, voxelization-based mesh simplification, generates candidate faces and includes one or more of the following steps:

[0060] 1. Voxelization: Input model M input into voxels, where each voxel has an edge length of l / 64, where l is M inputis the diagonal length of the bounding box of voxels. A 3D winding number is calculated for each voxel and an isosurface mesh is extracted (e.g., using a marching cubes algorithm to extract an isosurface mesh corresponding to a particular winding number (e.g., 0.5, 1, 1.5)). A smaller winding number means that the voxel mesh is larger than the input mesh, which in some embodiments may result in a higher R but a lower P.

[0040]

[0061] 2. Remesh: The isosurfaces obtained in step (1) can be applied to a coarse mesh M, for example, using a QEM-guided method. coarse Other methods for simplification, such as variational shape approximation (VSA), can also be used.

[0041]

[0062] 3. Conservative Enforcement: The simplified mesh M obtained in step (2) coarse is projected back onto the isosurface from step (1) to enforce conservatism.

[0042]

[0063] Conservative Enforcement

[0064] QEM-based simplification may not be conservative, and the resulting coarse mesh M coarse is the input mesh M input Such a coarse mesh may occlude objects that should actually be visible, causing false negatives. An optimization algorithm is used to remove the overhanging parts (e.g., the part of the input mesh M input (extending beyond the input mesh M input Push back inward.

[0043]

[0065] The optimization algorithm begins by computing a signed distance field φ, which is M coarse Represents the signed distance between and the isosurface:

[0066]

[0044]

number

[0067] where x is M coarse is a vector corresponding to the vertex position of Mcoarse Any point on

[0045]

[0068] M coarse Each edge of the

[0046]

number

[0047]

[0069] where p0 and p1 are the end points of the edge and r is the edge length before conservative enforcement.

[0048]

[0070] In some embodiments, M coarse is a triangular or polygonal mesh. To formulate unilateral constraints in the above optimization problem, the method and system described herein coarse and the signed distance field φ. coarse A collision occurs if any point on the face of has a signed value greater than 0. The method and system described herein checks whether a collision occurs at each step during optimization. If a collision occurs, the following soft SDF penalty energy is added to the objective function:

[0071]

[0049]

number

[0072] Putting these together to replace the hard constraints in Equation 1, the optimization is reformulated into the following unconstrained form:

[0073]

[0050]

number

[0074] In some embodiments, the method is essentially a penalty method that handles hard constraints with automatic parameter tuning. SDF Although no weighting is introduced for E, if the same consecutive collisions occur repeatedly, more E SDF The terms are added together, essentially increasing their weight. M coarse Since typically has fewer than 100 vertices, in some embodiments, the Quasi-Newton method (QNM) is used to solve the optimization problem. Newton's method is another way to solve optimization problems.

[0051]

[0075] FIG. 6 shows an example of voxel-based mesh simplification. An input model 600 is provided, from which an isosurface 602 is derived. The isosurface 602 is simplified into a simplified mesh 606. Portions of the simplified mesh 606 are non-conservative (e.g., extend beyond the volume of the input model 600). The final mesh 608 enforces conservatism by pushing faces 610 back into the final mesh 608. The methods and systems described herein push the coarse mesh into the building, improving the fit, such as the sharp corners highlighted in face 610 in FIG. 6.

[0052]

[0076] Occlusion Evaluation

[0077] Input model M input Occluder M occluder The two metrics, precision P and recall R, evaluate the quality of the occluder and guide further mesh simplification procedures.

[0053]

[0078] FIG. 7A illustrates an evaluation scheme in which both the occluder and the structure (e.g., building 700) are located at origin 701, while the observer (represented by viewpoint 702) is located outside the structure and faces the origin. The evaluation method described herein allows a player to navigate through a structure (e.g., building 704 with interior walls, caves, tunnels, and structures) and consider camera views from the interior or inside 708, as well as recessed areas of such structures, as shown in FIG. 7B. For example, FIG. 7B illustrates sampling across a 3D space including point 706. At each point, including point 706, all camera views (e.g., four directions: up, down, left, and right) are evaluated, including the interior and recessed areas. Thus, the evaluation method involves calculating precision P and recall R across the 3D evaluation space. The player can look into all directions of the surrounding space from any accessible position in 3D space. The evaluation domain is defined as:

[0054]

number

[0079] B' is M input The expanded volume B((1+ε padding ) and M input The overall precision P and recall R are given as:

[0080]

[0055]

number

[0081] The above integral is approximated numerically by uniformly dividing B' into volume blocks of interval Δx. M input All outer volume blocks are marked as valid. The overall precision, P, and recall, R, are calculated as follows:

[0082]

[0083]

[0056]

number

[0084] where N is the number of valid blocks and ΔV=Δx 3 is a block volume.

[0057]

[0085] Px and Rx denote the precision and recall at block center x.

[0058]

[0086] To calculate Px and Rx at a fixed camera position (e.g., block center x), the entire view direction space is discretized into six view frustums along the ±X, Y, and Z axes, each with a 90° view angle. The block center is the location of the fixed camera position in the evaluation.

[0059]

[0087] In some embodiments, M occluder and M input The occlusion calculation is reduced from 3D space to 2D screen space by comparing the 2D area occluded by each of the 3D planes. Monte Carlo sampling is used to approximate the Px and Rx of each viewing frustum.

[0060]

[0088] N occluders rasterized in evaluation space (e.g., 2D screen space, a 2D plane in 3D virtual space). quad For each of the randomly sampled, axis-aligned rectangles or quads, there are three possibilities:

[0089] (1) True positive: The quad is M occluder and M input The number of pixels in such a quad is N t It is expressed as;

[0090] (2) False positive: The quad is M input Although it is completely covered by occluder The number of pixels in such a quad is P f It is expressed as;

[0091] (3) False negative: The quad is M occluder Although the pixels are completely covered by input Within such a quad, M input The number of pixels not covered by f It is expressed as;

[0092] The discretized Px and Rx are calculated as follows:

[0093]

[0061]

number

[0094] Here the summation is over all six directions ±X,Y,Z.

[0062]

[0095] Metric-guided occluder simplification

[0096] The results from the patch-based simplification method (e.g., Results 302) and the voxel-based simplification method (e.g., Results 304) are combined to generate a high-quality set of face candidates (e.g., Combined 306). The final step involves using a metric-guided face reduction algorithm to cull the face subset and preserve its occlusion quality when forming the final occluder. A naive approach for this purpose is to greedily remove faces that result in the smallest recall reduction ΔR. However, in some embodiments, this is computationally too expensive. A combined mesh M with m faces is then generated. combined (For example, for the combined example 306), when calculating recall using n view position samples,

[0063]

number

[0064]

[0097] Strategies that (1) reduce neighboring views and (2) skip unnecessary evaluations further accelerate metric-guided occluder simplification.

[0065]

[0098] Reducing the sampled field of view

[0099] If two views are close to each other, the difference in their occlusion results will be small, so in some embodiments the number of view samples is reduced by merging neighbors.

[0066]

[0100] For example, in some embodiments, the method includes uniformly dividing domain B into N equal-sized blocks (single-sized blocks 800), as shown in FIG. 8A. In other words, the empty space is discretized into a large number of single-sized blocks 800. The method replaces adjacent 2x2x2 blocks with larger blocks (e.g., larger block 802 and largest block 804) in an octree-like manner, as shown in FIG. 8B, until no further merging occurs. Each block has six view directions along ±X, Y, and Z. If the center of any block is inside a building, no merging occurs. Equation 5 is reformulated as follows:

[0101]

[0067]

number

[0102] where ΔV i and x i are the block volume and center position of the ith sample, respectively.

[0068]

[0103] Sample skip processing:

[0104] If a structure (e.g., a building) is outside the viewing frustum of the viewing position, N t ,P f ,N f will be 0. Thus, before simplification, in some embodiments, one or more views (e.g., majority views, all views) are tested, and if a view cannot see the structure at all, it is marked to be skipped. Figure 8C shows the views marked to be skipped (e.g., view 806) in gray because the gray views cannot see the building.

[0069]

[0105] In some embodiments, if a face 808 is removed, the metric values ​​measured from viewpoints that cannot see the face will not change, as shown in Figure 8D. Thus, the evaluation domain is separated into two parts by the orientation of the face. As shown in Figure 8D, the metric values ​​are updated for viewpoint positions 810 that are on the positive side of the face 808 (e.g., on the positive side of the face normal).

[0070]

[0106] Example Implementation

[0107] In some embodiments, the method and system are implemented in C++ using CGAL and libigl. Fast parallel software rasterization is used on the CPU of Unreal Engine 4 to evaluate occluders and calculate metrics. Specifically, the input model and occluders are rasterized into the depth buffer. A large number of quads are then randomly generated on the depth buffer, and precision and recall are calculated using Equation 6. To avoid excessive empty pixels in the depth buffer, the minimum length of the bounding box of the input model is set to ε padding In some embodiments, the method was performed on a computer equipped with an AMD Ryzen Threadripper 3970X 32-core processor @ 3.69 GHz with 256 GB RAM.

[0071]

[0108] Ablation study

[0109] Metric Discretization Precision

[0110] Number of Screen Quads N quad This may affect the accuracy of the approximate metric calculation in some embodiments. Using an example construction model (shown in Figure 2A), the metric is calculated as: 100; 1,000; 5,000; 10,000; 20,000; 30,000; and 100,000 pieces Each metric is calculated using 100 quads per view. For each number of quads, the metric is calculated 10 times using a different set of randomly selected quads. Figure 9A shows the standard deviation of the 10 metrics. Plot 902 shows the standard deviation of recall at different numbers of quads per view. The other plot in Figure 9A shows the standard deviation of precision. Throughout the remaining experiments, 5,000 quads per view may be used to balance computational cost and accuracy.

[0072]

[0111] Sampling Distance

[0112] The visibility sample interval Δx can affect the accuracy of the metric approximation calculation. Halving the sampling distance increases the computational cost by a factor of eight. input The maximum diagonal length of the bounding box of 8%, 4%, 2%, 1%, 0.5% As above, the metrics were calculated 10 times under each sampling distance. Figure 9B shows the standard deviation for both recall and precision. Figure 9B shows a plot 904 of the standard deviation of recall at different sampling distances. The standard deviations for recall at 4% and 2% are very close, 1.9 x 10, respectively. -3 and 1.8 × 10 -3 Therefore, 4% was used as the sampling distance throughout the rest of the experiment.

[0073]

[0113] Sample Reduction

[0114] Sample reduction and / or sample skipping can accelerate metric calculation and, consequently, mesh simplification guided by the metric. Across all 77 models, the results show that the total number of evaluation tests was reduced to 16.3% of the number of tests without evaluation acceleration techniques. After skipping sample views that could not see the model and / or removed faces, the ratio was further reduced to 13.1% and 5.9%. While there was no visible difference between the reduction results with and without sample reduction, with fluctuations of 0.5% and 0.2% in precision and recall, respectively, sample reduction can save an average of 83.7% of samples. Overall, computation time was sped up by 5.56x using sample reduction. Sample skipping did not affect the accuracy of the evaluation results.

[0074]

[0115] Example

[0116] The methods and systems described herein were evaluated using a dataset of 77 building models used in games, as shown in Figure 10. Each building has hundreds of components and thousands of intersecting triangle pairs, making it extremely difficult, if not impossible, to process using traditional mesh simplification methods. Five thousand randomly generated quads were rasterized with respect to the depth buffer to gather information for precision and recall calculations to evaluate the generated occluders. s (LS) = 1% input It is set as the maximum diagonal length of the bounding box of l. s (LS) is defined as the test line segment line for the "fill in the gap" procedure as described above. All hyper-parameters are listed in Table 1. Prior to the method and system described herein, hyper-parameters refer to parameters that were typically manually adjusted by technical artists designing video game graphics based on their experience. There is no theoretical explanation for why a parameter should assume its specific value.

[0075]

[0117] Table 1. Hyperparameters

[0076] [Table 1]

[0118] On average, the occluders generated using the methods and systems described herein have 260 faces with a recall of 78.0% and a precision of 99.4%, as shown in Figure 11 and Table 2. Figure 12 also shows 10 example buildings and corresponding occluders generated by the simpligons and the methods and systems described herein.

[0077]

[0119] Conservative Enforcement: Figure 11 shows that conservative enforcement can improve precision from 94.6% (Figure 11(a)) to 95.7% (Figure 11(b)). However, it can be seen that along with the improved precision comes a decrease in recall. When creating a given coarse mesh strictly within the input mesh, there are some areas that are not occupied by the coarse mesh. In some embodiments, a higher precision may be preferable, even at the expense of a loss in recall, because incorrect culling can lead to severe visual artifacts. The precision of the voxel-based result may be lower than that of the patch-based solution because the generation of the isosurface is based on the winding number for interior-exterior segmentation, which can lead to precision errors.

[0078]

[0120] Combined patch / voxel-based results: Figure 11 shows that when using the patch-based (Figure 11(a)) and voxel-based (Figure 11(c)) methods as the face candidate set, the generated occluders have recall rates of 41.2% and 78.4%, respectively, and face counts of 500 and 53, respectively.

[0079]

[0121] The combined candidate set in Figure 11(d) yields a significantly higher recall of 86.0%. An example of combining patch-based and voxel-based results can also be seen in Figures 3A-3E.

[0080]

[0122] Timing: Across 77 test building models, the method and system described herein took an average of 155 seconds (Table 2), while patch-based and voxel-based mesh simplification took 5 and 24 seconds, respectively. Metric-guided mesh simplification took 126 seconds to further reduce the number of faces from 553 to 260.

[0081]

[0123] Table 2. Average number of faces and computation time for 77 test buildings using the illustrated method, simpligons, and planar occluders (*for one-sided rendering, numbers need to be multiplied by 2)

[0082] [Table 2]

[0124] Comparison with Simplgon: Using Simplegon's visibility-driven mesh simplification pipeline, the number of target triangles is set to 300, the number of occluder faces commonly used in mobile games. Simplegon preserves silhouettes at a moderately fast speed (3 seconds per model) with a recall rate of 74.3%, which is 3.7% lower than the recall achieved using the method and system described herein. Furthermore, Simplegon fails to maintain conservatism (96.2% precision compared to the 99.4% precision achieved using the method and system described herein). More importantly, the standard deviation of the precision achieved using the method and system described herein is only 0.6, while Simplegon's standard deviation is 4.3%, indicating that the method and system described herein has more stable performance. Lower precision is more detrimental than lower recall. With low recall, the system must render more occluded objects, which hinders rendering efficiency. However, as shown in Figures 1B and 1C, low precision can lead to erroneous culling of objects even when they are visible to the player. For the first model in Figure 12, even though the occluder generated by the simpligon can capture the overall input shape, its recall is only 66.9% due to small cracks between the cut walls, which significantly weakens its occlusion power.

[0083]

[0125] Comparison with planar occluders: To compare with the planar occluder, the input mesh is first voxelized to generate an isosurface. The Silvenoinen method assumes that the building model is viewed from a distance and generates only one plane for each viewing direction. After selecting a set of planes, the Silvenoinen method greedily removes the smallest triangles and does not minimize the loss of recall. The method and system described here optimizes by more accurately pruning faces, evaluating occlusion whenever a face is discarded. Even with this optimization, the output planar occluder achieves only an average recall of 39.7% and a precision of 92.3%. One reason for this is that the test model contains thin walls and nested structures, which are difficult to accurately voxelize. Therefore, the output isosurface captures only a small portion of the input mesh, resulting in a large precision error, which can also be observed in our voxel-based results (only 41.2% recall). In terms of computation time, Sirvenoinen's method took 267 seconds per model, which is slower than the methods and systems described herein, but Sirvenoinen's occlusion metrics were only half of those obtained using the methods and systems described herein. Sirvenoinen's method also assumed double-sided rendering. For a culling method with single-sided rendering, the number of faces in the occluder would need to be doubled. For a good comparison, Sirvenoinen's target output face count was set to 150, and the number of faces was doubled at runtime because the game engine (based on Unreal Engine 4) used in some embodiments of the methods and systems described herein uses single-sided rendering.

[0084]

[0126] The method and system described in this paper combine patch-based and voxel-based face generation techniques. The best face subset is selected to form the occluder based on a novel evaluation metric. Furthermore, two evaluation metrics are introduced across the 3D domain to measure the quality of the occluder, and several strategies for accelerating the evaluation procedure are also described. Testing of the method and system described in this paper has been performed using 77 building models in Unreal Engine 4, highlighting the method's ability to generate occluders with higher precision and recall.

[0085]

[0127] The methods and systems described herein are not limited to manifold and watertight inputs. Using heuristic, problem-specific techniques, including hyper-parameters, is another approach. The methods and systems described herein are not limited to utilizing patch-based and voxel-based mesh simplification tools; they can fuse results from additional, different mesh simplification techniques to improve the quality of the face candidate set. Furthermore, metric-guided occluder simplification can involve additional operations, such as edge collapse and vertex removal, instead of using face reduction alone (although at the cost of higher computational load).

[0086]

[0128] In some implementations, seam-closing is used. As shown in Figure 13, when small, narrow patches are removed from the input mesh, seams can occur, which can significantly reduce precision. To close these seams, the boundaries of all patches are first segmented into a number of straight line segments of length l. i (L-I) and l j (L.J.), Direction d i and d j Suppose there are two segments with each having a different patch. If the two segments are of similar length and parallel,

[0087]

number

[0088]

number

[0089]

[0129] Seam closing is performed by adding a quad Q formed by the four vertices of the two segments. Note that a candidate will be considered a real seam only if the gap between the pair is closed in the input mesh, typically by several intersecting narrow patches, e.g., pillars or rounded corners. To prevent improper seam closing, a number of test line segments of length ls (e.g., ls between 50 mm and 200 mm, ls is e.g., 100 mm) are uniformly sampled inside Q along the normal direction of Q (Fig. 13(c)). M input As long as there is at least one segment that does not intersect with input This means that there may be a hollow structure in Q, and Q is discarded.

[0090]

[0130] Figure 13(a) shows the input mesh with two walls and one pillar. Figure 13(b) shows the paired segments after patch removal. Figure 13(c) shows the line segments evenly spaced inside the quads formed by the segment pairs. Figure 13(d) shows the mesh after seam closing.

[0091]

[0131] Heterogeneous buildings are buildings that are only partially accessible. In some embodiments, the lower part of a heterogeneous building has an interior structure that the player can walk through, while the upper part is a large, closed structure that can only be viewed from the outside. In some embodiments, such heterogeneous building structures can be handled by first splitting the building vertically into two components, applying coarse mesh generation (e.g., patch-based simplification) to only the lower part and mesh simplification with conservative constraints to the upper part, and then combining the two results as a final coarse mesh.

[0092]

[0132] In some embodiments, a heterogeneous building has multiple large horizontal patches (usually floors and ceilings) that face downwards and can split the building into two components. Our splitting plane is selected from these candidates. For a candidate plane P, the generated 2D segment soups are +η and P -η The comparison is made by slicing the building using P +η and P -η are two planes parallel to P with a perpendicular offset ±η (e.g., in some embodiments, η≈1×10 -3 (The candidate plane is the plane where there is a significant area difference between two segments, for example, the ratio between their areas ε a is the threshold (e.g., ε a =10), it is selected as the final splitting plane. In some embodiments, the plane with the largest difference between the first area and the second area is selected as the final splitting plane.

[0093]

[0133] For a given 2D segment soup, its area is calculated by first obtaining a closed 2D triangle mesh, which can be calculated using TriWild, and then trivially summing the areas of all triangles. Figures 14A and 14B show examples of several candidate splitting planes and their corresponding slice areas for a heterogeneous building model, according to some implementations of the present application. Figure 14A shows the input building and five potential transparent splitting planes. Potential transparent splitting planes are candidate planes for splitting the building into two or more parts. Figure 14B shows the upper and lower slices of each splitting plane. 2D intersection segments and interior triangles are shown. In some embodiments, the method selects splitting plane (4) because the area of ​​the lower slice is significantly smaller than the area of ​​the upper slice.

[0094]

[0134] FIG. 15 illustrates a process performed by a computing system to automatically generate occluders for visual three-dimensional structures, according to some implementations of the present application.

[0095]

[0135] First, a computing system receives (1510) an input model of a visual three-dimensional structure, the input model including a plurality of faces. Then, the computing system generates (1520) an initial occluder by simplifying the input model into a plurality of candidate patches in a patch-based coarse mesh. In some embodiments, the visual three-dimensional structure corresponds to a visual structure in a video game. In some embodiments, the computing system automatically generates the occluder before a user initializes a video game to be played. In some embodiments, the computing system automatically generates the occluder during the production phase of the video game. In some embodiments, after a user initializes the video game, the video game displays structures and images that are not obstructed by the occluder. In some embodiments, the visual three-dimensional structure is mapped with texture and color before being displayed (e.g., during the production phase of the video game, before a user initializes the video game). In some embodiments, the coarse mesh is an approximate mesh generated from a fine rendering mesh (e.g., input mesh, visual mesh). In some embodiments, the coarse mesh has not undergone refinement (e.g., has not undergone metric-based simplification, has not yet been checked for maintainability). The initial occluder occludes objects behind the visual three-dimensional structure along a first viewing direction. In some embodiments, the occluder is automatically generated during the production stage of the video game, but graphics associated with the visual three-dimensional structure are not rendered. The computing system compares a first two-dimensional area occluded by an input model of the visual three-dimensional structure with a second two-dimensional area occluded by the initial occluder along the first viewing direction and determines a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder (1530). The computing system removes faces from the initial occluder while maintaining the first quality metric above a first threshold to form an occluder for the visual three-dimensional structure (1540).The occluder blocks objects behind the visual three-dimensional structure from being rendered in the application along the first viewing direction. In some embodiments, the automatically generated occluder is the final occluder used in the video game application (e.g., the initial occluder is not used by the video game application at runtime).

[0096]

[0136] In some implementations, the computing system voxelizes the input model to obtain a plurality of voxels for generating a voxelization-based coarse mesh (1550) and combines the patch-based coarse mesh and the voxelization-based coarse mesh to form an initial occluder (1560). In some implementations, the computing system determines a second quality metric based on a second number of pixels blocked by both the initial occluder and the input model (1570). Simplifying the input model into a plurality of candidate patches includes merging a first number of faces of the plurality of faces into a plurality of candidate patches, the plurality of candidate patches meeting a threshold requirement for the first quality metric or the second quality metric. In some implementations, merging the first number of faces into a plurality of candidate patches by the computing system includes combining faces that fall within a first degree of coplanarity into candidate planar patches of the plurality of candidate patches (1580).

[0097]

[0137] In some implementations, generating the voxelization-based coarse mesh by the computing system includes: voxelizing a bounding box of the input model to obtain a plurality of voxels; calculating a winding number for each of the plurality of voxels; extracting an isosurface based on the winding number; and simplifying the isosurface to obtain the voxelization-based coarse mesh (1590).

[0098]

[0138] Although specific implementations have been described above, it will be understood that the invention is not intended to be limited to these specific implementations. Rather, the invention encompasses alternatives, modifications, and equivalents that are within the spirit and scope of the appended claims. Numerous specific details have been set forth to provide a thorough understanding of the subject matter presented herein. However, it will be apparent to those skilled in the art that the subject matter may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the implementations.

[0099]

[0139] Although terms such as "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first ranking criterion could be referred to as a second ranking criterion, and similarly, a second ranking criterion could be referred to as a first ranking criterion, without departing from the scope of this application. The first ranking criterion and the second ranking criterion are both ranking criteria, but they are not the same ranking criterion.

[0100]

[0140] The terminology used in the description of the present invention herein is for the purpose of describing particular implementations only and is not intended to limit the present invention. As used in the specification of the present invention and the appended claims, "a," "an," or "the" is intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that, as used herein, the terms "include," "including," "comprises," and / or "comprising" specify the presence of stated features, processes, elements, and / or components, but do not exclude the presence or addition of one or more other features, processes, elements, components, and / or groups thereof.

[0101]

[0141] As used herein, the term "if" may be interpreted to mean "if" or "upon" or "in response to determining" or "in accordance with a determination that" or "in response to detecting" the stated condition precedent is true, depending on the context. Similarly, the phrases "if it is determined that the stated condition precedent is true," "if the stated condition precedent is true," and "if the stated condition precedent is true" may be interpreted to mean "upon determining," "in response to determining," "in accordance with determining," "upon detecting," or "in response to detecting" the stated condition precedent is true, depending on the context.

[0102]

[0142] While some of the various figures depict some logical stages in a particular order, stages that are not order-dependent may be rearranged and other stages may be combined or separated. While some rearrangements or other groupings are specifically mentioned, others will be apparent to those skilled in the art, and thus this does not present an exhaustive list of alternatives. Furthermore, it should be recognized that these stages may be implemented in hardware, firmware, software, or any combination thereof.

[0103]

[0143] The foregoing description has been described with reference to specific implementations for purposes of explanation. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The implementations have been chosen and described to best explain the principles of the invention and its practical application, so as to enable those skilled in the art to best utilize the invention and its various embodiments, with various modifications suited to the particular use contemplated. The implementations encompass alternatives, modifications, and equivalents within the spirit and scope of the appended claims. Numerous specific details have been set forth to provide a thorough understanding of the subject matter presented herein. However, it will be apparent to those skilled in the art that the subject matter may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the implementations.

Claims

1. 1. A method for automatically generating occluders for a visual three-dimensional structure, comprising: receiving an input model of the visual three-dimensional structure, the input model including a plurality of faces; generating an initial occluder by simplifying the input model into a plurality of candidate patches in a patch-based coarse mesh, the initial occluder blocking objects behind the three-dimensional structure along a first viewing direction; comparing a first two-dimensional area occluded by an input model of the visual three-dimensional structure with a second two-dimensional area occluded by the initial occluder along the first viewing direction, and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder; and removing faces from the initial occluder while maintaining the first quality metric above a first threshold to form an occluder for the visual three-dimensional structure, the occluder blocking objects behind the visual three-dimensional structure from being rendered in the application along the first viewing direction; A method comprising:

2. 10. The method of claim 1, further comprising: voxelizing the input model to obtain a plurality of voxels for generating a voxelization-based coarse mesh; and combining the patch-based coarse mesh and the voxelization-based coarse mesh to form the initial occluder; A method comprising:

3. 3. The method of claim 2, further comprising determining a second quality metric based on a second number of pixels blocked by both the initial occluder and the input model, and wherein simplifying the input model into a plurality of candidate patches comprises merging a first number of faces of the plurality of faces into the plurality of candidate patches, the plurality of candidate patches meeting a threshold requirement for the first quality metric or the second quality metric.

4. 4. The method of claim 3, wherein merging the first number of faces into the plurality of candidate patches includes combining faces that fall within a first degree of coplanarity into candidate planar patches of the plurality of candidate patches.

5. 5. The method of claim 4, wherein: responsive to determining that a shared edge of two candidate planar patches is less than a threshold, grouping the two candidate planar patches into a curved patch; The method further comprises:

6. The method of claim 3 , wherein the first quality metric or the second quality metric is a user-specified value.

7. 3. The method of claim 2, wherein generating a voxelization-based coarse mesh comprises: voxelizing a bounding box of the input model to obtain the plurality of voxels; calculating a winding number for each of the plurality of voxels; Extracting an isosurface based on the winding number; simplifying the isosurface to obtain a voxelization-based coarse mesh; A method comprising:

8. 8. The method of claim 7, wherein: extracting the isosurface includes using a marching cubes algorithm; The method, wherein simplifying the isosurface includes generating a simplified mesh using a quadratic error method (QEM) guided method.

9. 9. The method of claim 8, including performing conservative enforcement on the simplified mesh to constrain the simplified mesh within the input model.

10. 10. The method of claim 9, wherein performing conservative enforcement comprises: determining positive and negative distance fields from said isosurfaces; determining each edge in the simplified mesh in terms of spring energy; and applying a penalty energy if the simplified mesh exceeds the input model.

11. 10. The method of claim 1, wherein removing the plurality of faces comprises: discretizing a space in and around the input model into a plurality of equally sized blocks, each of the equally sized discretized blocks having a plurality of viewing directions; merging a portion of the plurality of equal-sized blocks into a plurality of larger blocks; removing a view direction from each of the plurality of larger blocks such that the input model is outside a view frustum of the view direction.

12. 12. The method of claim 11, wherein the plurality of viewing directions includes six viewing directions along ±X, Y, and Z for each larger block in the plurality of larger blocks and for each block in the plurality of equal-sized blocks.

13. The method of claim 1 further comprising filling holes in the patch-based coarse mesh.

14. 14. The method of claim 13, wherein a hole is filled if a test segment oriented perpendicular to a quad covering a gap in the patch-based coarse mesh intersects with the input model.

15. 15. The method of claim 14, wherein the visual three-dimensional structure is a building, and windows of the building are not blocked by the occluder.

16. 10. The method of claim 1, wherein the visual three-dimensional structure comprises a building having non-manifold, non-watertight, and / or self-intersecting characteristics, and the building comprises an interior structure having a defined viewing direction within it.

17. 17. The method of claim 16, further comprising determining a plane dividing the building into a first portion and a second portion, the second portion including the interior structure.

18. 18. The method of claim 17, wherein the step of determining the plane comprises: For each of the candidate planes: a first area of ​​the building extracted by an upper slice offset from each of the plurality of candidate planes; a second area of ​​the building extracted from each of the plurality of candidate planes by a bottom slice offset; determining selecting a plane from the plurality of candidate planes that has the largest difference between the first area and the second area; A method comprising:

19. 10. The method of claim 1, wherein the input model is represented as a mesh comprising a polygon soup, the mesh comprising hundreds of disconnected components and nested structures.

20. The method of claim 1 , wherein the number of faces in the occluder is less than 10% of the number of faces in the plurality of faces of the input model.

21. A computing device comprising a memory for storing a program and one or more processors, the program, when executed by the one or more processors, causing the computing device to perform a method for automatically generating occluders for visual three-dimensional structures according to any one of claims 1-20.

22. 21. A non-transitory computer-readable storage medium storing a program associated with a computing device including one or more processors, the program, when executed by the one or more processors, causing the computing device to perform a method for automatically generating occluders for visual three-dimensional structures according to any one of claims 1-20.

23. A computer program product causing a computer processor to carry out a method according to any one of claims 1 to 20.

Images