Occluder Generation for Structures in Digital Applications

Abstract

A method is performed in a computing system for automatically generating an occluder, the method including receiving an input model of a visual three-dimensional structure, the input model including a plurality of faces. The method generates an initial occluder by simplifying the input model into a plurality of candidate patches in a patch-based coarse mesh. The method includes comparing a first two-dimensional area occluded by the input model of the visual three-dimensional structure to a second two-dimensional area occluded by the initial occluder along a first viewing direction, and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder. The method includes removing the plurality of faces from the initial occluder while maintaining the first quality metric above a first threshold to form an occluder for the visual three-dimensional structure.

Description

[Technical field]

[0001]

[0001] Priority and Related Applications This application is a continuation of U.S. patent application Ser. No. 17 / 866,413, filed on July 15, 2022, entitled "Occluder Generation for Structure in Digital Applications," which claims priority to U.S. Provisional Application No. 63 / 302,916, filed on January 25, 2022, entitled "Occluder Generation for Construction in Digital Games," each of which is incorporated by reference in its entirety.

[0002]

[0002] TECHNICAL FIELD The disclosed implementations relate generally to the field of computer technology, and more particularly to occlusion culling in computer graphics. [Background technology]

[0003]

[0003] Modern game engines use occlusion culling techniques to remove occluded objects from the rendering pipeline. Objects are occluded for a particular viewing direction if there is another object in front of them, along that particular viewing direction. Summary of the Invention

[0004]

[0004] By culling out obscured objects during application execution using an approximated coarse mesh (occluder) rather than a delicate rendering mesh (visual mesh), graphical processing unit (GPU) bandwidth is conserved and rendering costs are reduced due to a reduction in the number of draw calls. A draw call instructs the GPU to prepare drawing resources based on information about textures, states, shaders, rendering objects, buffers, etc.

[0005]

[0005] The quality of the occluder mesh affects the effectiveness and accuracy of culling. A low-poly mesh (e.g., a polygon mesh with a relatively small number of polygons) reduces the cost of the culling test. The culling test checks whether a particular face is completely occluded by other faces. If so, the particular face is discarded from the rendering pipeline to save the cost of further rendering steps. In some embodiments, a low-poly mesh may contain about 200-300 polygons. Furthermore, occluders should ideally be conservative and completely within the volume of the visual mesh. A non-conservative occluder may then erroneously cull out visible objects (e.g., objects outside the volume of the visual mesh), causing severe visual artifacts, as explained in the context of Figures 1B and 1C below.

[0006]

[0006] It is a part of the objective of the present application to address the above challenges by presenting a set of solutions for automating the generation of occluders for three-dimensional structures.

[0007] According to one aspect of the present application, a method is performed in a computing system for automatically generating occluders for a visual three-dimensional structure. The method includes the following steps: receiving an input model of a visual three-dimensional structure, the input model including a plurality of faces; generating an initial occluder by simplifying the input model into a number of candidate patches in a patch-based coarse mesh, the initial occluder blocking objects behind the three-dimensional structure along a first viewing direction; comparing a first two-dimensional area occluded by an input model of the visual three-dimensional structure to a second two-dimensional area occluded by an initial occluder along a first viewing direction and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder; and removing faces from the initial occluder while maintaining a first quality metric above a first threshold to form an occluder for the visual three-dimensional structure, the occluder preventing objects behind the three-dimensional structure from being rendered in the application along the first viewing direction.

[0008]

[0008] According to another aspect of the present application, a computing system is provided for automatically generating occluders for visual three-dimensional structures. The computing system includes one or more processors; a memory; and a number of programs stored in the memory. The number of programs, when executed by the one or more processors, cause the computing system to perform one or more operations, the one or more operations including: receiving an input model of a visual three-dimensional structure, the input model including a plurality of faces; generating an initial occluder by simplifying the input model into a number of candidate patches in a patch-based coarse mesh, the initial occluder blocking objects behind a visual three-dimensional structure along a first viewing direction; comparing a first two-dimensional area occluded by an input model of the visual three-dimensional structure to a second two-dimensional area occluded by an initial occluder along a first viewing direction and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder; and removing faces from the initial occluder while maintaining a first quality metric above a first threshold to form an occluder for the visual three-dimensional structure, the occluder preventing objects behind the visual three-dimensional structure from being rendered in the application along a first viewing direction.

[0009]

[0009] According to yet another aspect of the present application, a non-transitory computer readable storage medium stores, in association with a computing system having one or more processors, a plurality of programs for automatically generating occluders for a visual three-dimensional structure. The plurality of programs, when executed by the one or more processors, cause the computing system to perform one or more operations, the one or more operations including: receiving an input model of a visual three-dimensional structure, the input model including a plurality of faces; generating an initial occluder by simplifying the input model into a number of candidate patches in a patch-based coarse mesh, the initial occluder blocking objects behind a visual three-dimensional structure along a first viewing direction; comparing a first two-dimensional area occluded by an input model of the visual three-dimensional structure to a second two-dimensional area occluded by an initial occluder along a first viewing direction and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder; and removing faces from the initial occluder while maintaining a first quality metric above a first threshold to form an occluder for the visual three-dimensional structure, the occluder preventing objects behind the visual three-dimensional structure from being rendered in the application along a first viewing direction. [Brief description of the drawings]

[0010]

[0010] The foregoing and additional implementation forms of the present invention will be more clearly understood as a result of the following detailed description of the various aspects of the invention taken in conjunction with the drawings in which like reference numerals refer to corresponding parts throughout the several views. [Figure 1A]

[0011] 1A-1D show how non-conservative occluders, according to some implementations of the present application, can cull visible objects. [Figure 1B] 1A-1D show how a non-conservative occluder according to some implementations of the present application culls out visible objects. [Figure 1C] 1A-1D show how a non-conservative occluder according to some implementations of the present application culls out visible objects. [Figure 1D] 1A-1D show how a non-conservative occluder according to some implementations of the present application culls out visible objects. [Figure 2A]

[0012] 2B-2E show how various meshes are derived from the input model shown in FIG. 2A, according to some implementations of the present application. [Figure 2B]

[0012] Figures 2B-2E show how various meshes are derived from the input model shown in Figure 2A, according to some implementations of the present application. [Figure 2C]

[0012] Figures 2B-2E show how various meshes are derived from the input model shown in Figure 2A, according to some implementations of the present application. [Figure 2D]

[0012] Figures 2B-2E show how various meshes are derived from the input model shown in Figure 2A, according to some implementations of the present application. [Figure 2E]

[0012] Figures 2B-2E show how various meshes are derived from the input model shown in Figure 2A, according to some implementations of the present application. [Figure 3A]

[0013] 3A-3E show an input model, the result of patch-based simplification of the input model, the result of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification, in accordance with some implementations of the present application. [Figure 3B]

[0013] Figures 3A-3E show an input model, the results of patch-based simplification of the input model, the results of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification in accordance with some implementations of the present application. [Figure 3C]

[0013] Figures 3A-3E show an input model, the results of patch-based simplification of the input model, the results of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification in accordance with some implementations of the present application. [Figure 3D]

[0013] Figures 3A-3E show an input model, the results of patch-based simplification of the input model, the results of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification in accordance with some implementations of the present application. [Figure 3E]

[0013] Figures 3A-3E show an input model, the results of patch-based simplification of the input model, the results of voxelization-based simplification of the input model, the combined results, and the final occluder after metric-guided mesh simplification in accordance with some implementations of the present application. [Figure 4A]

[0014] 4A and 4B show flat and curved patches obtained using different types of grouping of the input model according to some implementations of the present application. [Figure 4B] 4A and 4B show flat and curved patches obtained using different types of grouping of the input model according to some implementations of the present application. [Diagram 5]

[0015] FIG. 5 illustrates a verification process for checking whether a hole corresponds to a decorator, according to some implementations of the present application. [Figure 6]

[0016] Figure 6 shows an example of voxel-based mesh simplification according to some implementations of the present application. [Figure 7A]

[0017] 7A and 7B compare different view sampling methods according to some implementations of the present application. [Figure 7B] 7A and 7B compare different view sampling methods according to some implementations of the present application. [Figure 8A]

[0018] 8A-8D show examples of occlusion measurement acceleration according to some implementations of the present application. [Figure 8B] 8A-8D show examples of occlusion measurement acceleration according to some implementations of the present application. [Figure 8C]8A-8D show examples of occlusion measurement acceleration according to some implementations of the present application. [Figure 8D] 8A-8D show examples of occlusion measurement acceleration according to some implementations of the present application. [Figure 9A]

[0019] 9A and 9B show the standard deviation of recall and precision for different numbers of quadrilaterals per view and different sampling distances according to some implementations of the present application. [Figure 9B] 9A and 9B show standard deviations of recall and precision for different numbers of quadrilaterals per view and different sampling distances according to some implementations of the present application. [Figure 10]

[0020] FIG. 10 shows a dataset of 77 building models used in the game according to some implementations of the present application. [Figure 11]

[0021] FIG. 11 shows the average recall and precision rates for various test buildings according to some implementations of the present application. [Figure 12]

[0022] FIG. 12 illustrates an example building and corresponding occluders generated in accordance with some implementations of the present application. [Figure 13]

[0023] FIG. 13 shows an example of closing a seam according to some implementations of the present application. [Figure 14A]

[0024] 14A and 14B show examples of several possible splitting planes and their corresponding slice areas for a heterogeneous building model, according to some implementations of the present application. [Figure 14B] 14A and 14B show examples of several candidate splitting planes and their corresponding slice areas for a heterogeneous building model in accordance with some implementations of the present application. [Figure 15]

[0025] FIG. 15 illustrates a method for automatically generating occluders for visual three-dimensional structures according to some implementations of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011]

[0026] The following description of implementations refers to the accompanying drawings to illustrate specific implementations that may be implemented by the present application. Directional terms such as "upper", "lower", "front", "rear", "left", "right", "inner", "outer", and "side" used in the present application are only used as directional references in the accompanying drawings. Therefore, the directional terms used are only used to explain and understand the present application, not to limit the present application. In the drawings, units having similar structures are represented by the same reference numerals.

[0012]

[0027] 1A shows an actual view 100 from a position in front of a structure 102. The actual view 100 corresponds to the ground truth of what a player at that location would observe. A first structure 104 and a second structure 106 are visible in the actual view 100 from a position in front of the structure 102 along this viewing direction.

[0013]

[0028] 1B shows a view 108 from the same position in front of the structure 102 when a first non-conservative occluder is used. Examples of first non-conservative occluders include occluders created by Silvennoinen. A second structure 106 that would be visible from a position in front of the structure 102 along this line of sight in the actual view 100 is no longer visible in the view 108 because the first non-conservative occluder extends beyond the volume of the visual mesh, culling the second structure 106 that would have been visible as shown in the actual view 100. Instead, a region 110 that is devoid of the second structure 106 is shown in the view 108.

[0014]

[0029] 1C shows a view 112 from the same position in front of the structure 102 when a second non-conservative occluder is used. An example of a second non-conservative occluder includes an occluder generated by a Simplygon. The second structure 106 that would be visible from a position in front of the structure 102 along this line of sight in the actual view 100 is no longer visible in the view 112 because the second non-conservative occluder extends beyond the volume of the visual mesh, culling the second structure 106 that would have been visible as shown in the actual view 100. Instead, a region 114 that is devoid of the second structure 106 is shown in the view 112.

[0015]

[0030] 1D shows a view 116 from the same position in front of the structure 102 when an occluder according to some embodiments of the present invention is used. The second structure 106, which in the actual field of view 100 would be visible from a position in front of the structure 102 along this line of sight, is again visible in the field of view 116.

[0016]

[0031] A non-conservative occluder can erroneously cull visible objects, causing severe visual artifacts, as shown in Figures 1B and 1C. As disclosed in detail below, in accordance with some embodiments, we qualitatively measure the accuracy of an occluder across the 3D domain using two metrics: precision and recall: Precision measures the likelihood that an object blocked by an occluder is also blocked by the original model.

[0017] Recall measures the likelihood that an object blocked by the original model is also blocked by the occluder. The embodiments disclosed herein relate to automatically generating occluders for structures (e.g., building models) from game assets. Game artists manually create building models to maximize visual realism, and building models typically include many disconnected parts, large open doors and windows, and interior structures, as shown in FIG. 2A. FIG. 2A shows an input model 200 with an open door 202, disconnected parts 204a, 204b. In some embodiments, the model 200 may be an input mesh with 7,888 faces, 521 components, 562 boundary loops, and 9,034 intersecting triangle pairs. In some embodiments, each boundary loop is a hole. Because artists are only focused on the exterior of the building, the building models are typically non-manifold, non-watertight, and / or self-intersecting, making them unacceptable to conventional mesh processing algorithms. In some embodiments, the non-manifold geometry includes any edge shared by more than two faces. In some embodiments, at least one edge in the non-tight model is a boundary edge. In other words, the non-tight mesh has holes. In some embodiments, a model that is self-intersecting means that the mesh has two faces that intersect with each other.

[0018]

[0032] While it may be possible to manually tune the occlusion of hand-crafted buildings to achieve a good balance between precision and recall, such a process can be time-consuming. Furthermore, hand-crafted occluders may still contain faces that do not contribute to recall or that violate conservative constraints. Such faces are sometimes called "wasted faces".

[0019]

[0033] One strategy for automatic occluder generation is First, the input mesh is voxelized, Extract an isosurface (or isosurface) from the voxelized input mesh (e.g., an output isosurface), The output isosurface is then simplified, The idea is to insert an axis-aligned box or cross section to form the occluder. In some embodiments, the axis-aligned box (AAB) is simply a rectangular prism, each of whose faces is perpendicular to one of the basis vectors. Using such a strategy may cause some important features, e.g., thin walls, not to be captured at a reasonable resolution, as shown in FIG. 2B. FIG. 2B shows an isosurface 206 corresponding to a winding number of 0.5. Compared to the input model 200, the isosurface 206 is missing a thin wall 208. Furthermore, nested open structures in a building model may result in ill-defined orientations, which may cause the isosurface to not be accurately extracted. For example, an edge (or face) in an open structure may have a front direction that is perpendicular to one edge of the open structure and parallel to another edge of the open structure, making the orientation of the open structure ill-defined. For example, two connected faces may have opposite normal directions.

[0020]

[0034] 2C shows a mesh 212 obtained by applying Sirvenoinen's method to the isosurface 206. The resulting mesh 212 is obtained by inserting a cutting plane. In some embodiments, the resulting mesh 212 may contain 150 faces.

[0021]

[0035] The second strategy involves gradually removing faces from the input mesh by error-guided element-removal operations. The second strategy, in some embodiments, involves operations such as edge fracturing, which may produce results with large gaps and parts outside the visual mesh. Such a resulting mesh may violate conservativity, as in the example shown in FIG. 2D. FIG. 2D shows an occluder 214 generated by a simpligon. Part 216 extends beyond the volume of the input model 200, and part 218 covers the open door 202 in the input model 200. Thus, the occluder 214 violates conservativity. In some embodiments, the occluder 214 may include 231 faces.

[0022]

[0036] The Simplgon and Silvenoinen method assumes that buildings are viewed from a distance and does not consider concave and / or interior building structures, therefore such output occluders fail to provide accurate occlusion when game characters enter these areas.

[0023]

[0037] Rather than relying on one strategy and hoping that it will be generally sufficient to handle all building models with various styles, the embodiments disclosed herein first involve generating two coarse meshes from the input mesh using two different methods. The two coarse meshes populate the candidate face set as a large solution space. Figure 2E shows an occluder generated using the methods and systems disclosed herein. In some embodiments, the occluder 214 may contain 231 faces.

[0024]

[0038] The candidate face set for the occluder is evaluated using precision and recall metrics with respect to the input model on the 3D evaluation domain. The methods and systems described herein use a metric-based algorithm to select the best face set from within the solution space (e.g., the candidate face set) with a high occlusion rate while maintaining as much conservativeness as possible. The methods and systems also include incorporating one or more strategies to accelerate the metric computation at run time.

[0025]

[0039] In some embodiments, the methods and systems described herein are validated using 77 building models with various styles. In some embodiments, the methods described herein generate occluders with a low face count of 260, while achieving an average precision of 99.4% and an average recall of 78.0% from all possible observer positions, including far-away, close-up, and walk-in views. These results reflect an increase of 3.7% and 2.9% in terms of precision and recall, respectively, compared to occluders generated by Simplgon, while using 50 fewer faces on average. Results using Silvenoinen's method yield an average recall of only 39.7%.

[0026]

[0040] Below we summarize the relevant techniques of occlusion culling, mesh simplification, and occluder generation.

[0027]

[0041] Occlusion Culling: For static scenes, it is possible to pre-compute and store a set of possible visibility for a single viewpoint or region of viewpoints. For buildings with accessible interiors, cell-and-portals break the interior down into rooms (cells) connected by doors or windows (doorways, gates, portals). However, pre-computing and storing visibility data for complex scenes in large open worlds can be costly. To avoid excessive pre-computation and storage, view-dependent subsets of the input meshes can be maintained as virtual occluders at runtime.

[0028]

[0042] Mesh Simplification: Software rasterization can be used to render the coarse mesh (occluders) into a depth buffer, which is used to cull occluding objects at a very early stage. Generating an approximate coarse mesh from a fine mesh can be achieved by collapsing edges that satisfy certain conditions or by minimizing certain metrics (e.g. Quadratic Error Metrics (QEM)). Other metrics, such as surface visibility metrics, can be defined and added to the collapsing conditions for specific applications. Other techniques involve satisfying hard constraints during remeshing. For example, progressive hulls are introduced to ensure that all vertices are outside the input mesh, or a coarse mesh is generated while maintaining strict nesting. Unfortunately, these methods do not work well in topologically inconsistent cases. For example, a topologically inconsistent case may involve edges (or faces) in an open structure where two connected faces have opposite normal directions, as mentioned before.

[0029]

[0043] Occluder Generation: Conservative mesh simplification may be used to generate occluders for terrain patches in games while assuming a clean topology. If the topology can be made consistent, it is possible to generate occluders using existing mesh simplification methods. For this purpose, voxelization may be used. For example, a set of planes inside a voxelized input model is selected to form an occluder with bounded occlusion error. However, using a voxelized mesh as input may introduce large occlusion errors during voxelization. Also, buildings with nested structures are non-orientable and cannot be voxelized, while thin walls cannot be captured with a reasonable resolution. In the game industry, collision meshes are used as starting meshes for simplification, trying to avoid creating occluders manually for every building. However, collision meshes are typically larger than the input mesh, which would severely violate conservativeness.

[0030]

[0044] Exemplary embodiments

[0045] In some embodiments, the input building model (e.g., input model 200) is represented as a triangle / polygon soup, containing hundreds of (potentially self-intersecting) disconnected components, nested structures, and thin features. Due to the topological complexity, neither voxelization nor traditional mesh simplification may work well by themselves. The methods and systems described herein recognize that some disconnected components contain many large patches that are useful candidates for forming the final occluder mesh, while other large bulk features can be adequately captured by voxelization. Thus, the methods and systems described herein use a hybrid approach as shown in FIG. 3D that combines the output from two mesh simplification strategies (patch-based simplification as shown in FIG. 3B and voxelization-based simplification as shown in FIG. 3C) to derive a large set of face candidates into an initial occluder mesh M. occluder (shown in Figure 3D). The two evaluation metrics (precision P and recall R) are occluder It is formulated in terms of M occluder Using the metric-guided mesh simplification, a high-quality final occluder is extracted from the initial occluder, as shown in FIG. 3E. The metric-guided mesh simplification is terminated when a user-specified quality limit is reached. FIG. 3A shows an input model 300. FIG. 3B shows a result 302 of a patch-based simplification of the input model 300. FIG. 3C shows a result 304 of a voxelization-based simplification of the input model 300. FIG. 3D shows a combination 306 of the patch-based simplification result 302 and the voxelization-based simplification result 304. FIG. 3E shows a final occluder 308 after metric-guided mesh simplification.

[0031]

[0046] Initial occluder generation:

[0047] Two hybrid methods are used to form the initial set of face candidates.

[0032]

[0048] Patch-based Mesh Simplification:

[0049] The first approach is to generate a coarse mesh by performing one of the following steps:

[0050] 1. Planar patch grouping: Group pairs of faces into planar patches if the dihedral angle of one shared edge is less than a threshold. In some embodiments, the threshold is 1×10 -2 Less than 5×10 -3 Less than 1×10 -3 In some embodiments, the threshold is 1×10 -3 It is.

[0033]

[0051] 2. Curved Patch Grouping: The dihedral angle of one shared edge is less than the user-specified threshold ε a Group the planar patches generated from the last step into curved patches if they are less than a user-specified threshold ε a is smaller than the threshold used for planar patch grouping. Using planar patches may result in occluders with fewer triangles, while using curved patches captures more detailed information. Combining these two sets of patches results in a large candidate set from which the final occluder is selected. An example is shown in Figures 4A and 4B. Figure 4A shows a planar patch 400 obtained using planar patch grouping of the input model 300. Figure 4B shows a curved patch obtained using curved patch grouping of the input model 300.

[0034]

[0052] 3. Simplification: In some embodiments, each curved patch is simplified by QEM-guided mesh simplification. It is also possible to use other methods such as variational shape approximation (VSA). In some embodiments, the boundary of each 2D projected patch of each planar patch is simplified using the Ramer-Douglas-Peucker algorithm. There are many simplification methods, the main idea is to remove vertices that have less impact on the shape. After simplification, the boundary is retriangulated into a triangle mesh using constrained Delaunay triangulation. Although other methods are also used, Delaunay triangulation is one of the most common and useful methods.

[0035]

[0053] 4.Hole filling: There are holes in the triangle mesh from step 3, but M Input If does not contain such a hole, the hole in the triangular mesh is filled. The hole is filled with multiple triangle faces according to the edge number of the hole.

[0036]

[0054] 5. Reduction: The mesh simplification process then sorts all planar and curved patches according to their area, and the patches (faces) are sorted according to the user-specified number of faces N P are added to the final mesh until it reaches: In some embodiments, there is no preference to select curved patches over planar patches, and only the area of ​​the patch is used in the selection process.

[0037]

[0055] Hole filling:

[0056] In some embodiments, hole filling is a mesh repair technique. However, in building models, some holes correspond to building decorations such as windows and doors and need to be left open for the preservation of the occluders.

[0038]

[0057] FIG. 5 illustrates the verification process for checking whether a hole corresponds to a decorator. FIG. 5 illustrates an input mesh 500 (e.g., M input ) is shown. The verification process begins by triangulating the hole into multiple faces. The hole itself is defined by its boundary edges and does not have any faces. There are multiple faces associated with a hole only if the hole is to be filled (e.g., to fill the hole). For example, FIG. 5 shows a planar patch 504 with a hole 506 that has been extracted from an input mesh 500. s A set of test line segments with L−S are sampled uniformly inside each face along the normal direction. For example, FIG. 5 shows a test line segment 508 located inside a hole 506 in a planar patch 504. M input As long as there is at least one segment that does not intersect with input If all segments are in contact with the input mesh, M input There are no openings in the mesh, and the holes can be reliably filled. After filling the holes, the method and system meshes and simplifies the patch to further reduce the number of faces. For example, Figure 5 shows the output mesh 510 after hole filling and remeshing.

[0039]

[0058] Voxelization-based mesh simplification:

[0059] A second approach, voxelization-based mesh simplification, generates candidate faces and includes one or more of the following steps:

[0060] 1. Voxelization: Input model M input into voxels, where each voxel has an edge length of l / 64, where l is M inputwhere R is the diagonal length of the bounding box of voxel. A 3D winding number is computed for each voxel and an isosurface mesh is extracted (e.g., a marching cubes algorithm is used to extract an isosurface mesh corresponding to a particular winding number (e.g., 0.5, 1, 1.5)). A smaller winding number means that the voxel mesh is larger than the input mesh, which in some embodiments may result in a higher R but a lower P.

[0040]

[0061] 2. Remesh: The isosurfaces obtained in step (1) are then scaled on a coarse mesh M, for example using a QEM-guided method. coarse It is also possible to use other methods for the simplification, such as variational shape approximation (VSA).

[0041]

[0062] 3. Conservative Enforcement: The simplified mesh M obtained in step (2) coarse is projected back onto the isosurface from step (1) to enforce conservatism.

[0042]

[0063] Conservative Enforcement

[0064] QEM-based simplification may not be conservative and the resulting coarse mesh M coarse is the input mesh M input Such a coarse mesh may occlude objects that should actually be visible, causing false negatives. An optimization algorithm is used to remove the overhanging parts (e.g., the overhanging parts of the input mesh M input (the part extending beyond the input mesh M input Push back inward.

[0043]

[0065] The optimization algorithm begins by computing a signed distance field φ, which is then denoted M coarse Represents the signed distance between and the isosurface:

[0066]

[0044]

number

[0067] where x is M coarse is a vector corresponding to the vertex position of Mcoarse Any point on

[0045]

[0068] M coarse Each edge of the

[0046]

number

[0047]

[0069] where p0 and p1 are the end points of the edge and r is the edge length before conservative enforcement.

[0048]

[0070] In some embodiments, M coarse is a triangular or polygonal mesh. To formulate unilateral constraints in the above optimization problem, the method and system described herein coarse and the signed distance field φ. coarse A collision occurs if any point on the face of has a signed value greater than 0. The method and system described herein checks whether a collision occurs at each step during optimization. If a collision occurs, the following soft SDF penalty energy is added to the objective function:

[0071]

[0049]

number

[0072] Putting these together to replace the hard constraints in Eq. 1, the optimization is reformulated into the following unconstrained form:

[0073]

[0050]

number

[0074] In some embodiments, the method is essentially a penalty method that uses automatic parameter tuning to handle hard constraints. SDF No weighting is introduced for E, but if the same consecutive collisions occur repeatedly, more E SDF The terms are added together, essentially increasing their weight. coarse Since typically has fewer than 100 vertices, in some embodiments, the Quasi-Newton method (QNM) is used to solve the optimization problem. Newton's method is another way to solve optimization problems.

[0051]

[0075] FIG. 6 shows an example of voxel-based mesh simplification. An input model 600 is provided from which an isosurface 602 is derived. The isosurface 602 is simplified to a simplified mesh 606. Parts of the simplified mesh 606 are non-conservative (e.g., extend beyond the volume of the input model 600). The final mesh 608 enforces conservatism by pushing faces 610 back into the final mesh 608. Like the sharp corners highlighted in face 610 in FIG. 6, the methods and systems described herein push the coarse mesh into the building to improve the fit.

[0052]

[0076] Occlusion Evaluation

[0077] Input model M input Occluder M occluder The two metrics, precision,P,and recall,R,evaluate the quality of the occluders and guide further mesh simplification steps.

[0053]

[0078] FIG. 7A shows an evaluation scheme in which both the occluder and the structure (e.g., building 700) are located at the origin 701, while the observer (represented by viewpoint 702) is located outside the building of the structure and faces the origin. The evaluation method described herein allows the player to walk through the structure (e.g., building 704 with interior walls, caves, tunnels, and structures) and consider camera views from the interior or inside 708, as well as recessed areas of such structures, as shown in FIG. 7B. For example, FIG. 7B shows sampling over a 3D space including point 706. At each point including point 706, all camera views (e.g., four directions: up, down, left, and right) are evaluated, including the interior and recessed areas. Thus, the evaluation method involves calculating precision P and recall R over the 3D evaluation space. The player can look into all directions of the surrounding space at any accessible position in the 3D space. The evaluation domain is defined as:

[0054]

number

[0079] B' is M input The expanded volume B((1+ε padding ) and M input The overall precision P and recall R are given by:

[0080]

[0055]

number

[0081] The above integral is approximated numerically by uniformly dividing B' into volume blocks of interval Δx. M input All outer volume blocks are marked as valid. The overall precision, P, and recall, R, are calculated as follows:

[0082]

[0083]

[0056]

number

[0084] where N is the number of valid blocks and ΔV=Δx 3 is a block volume.

[0057]

[0085] Px and Rx denote the precision and recall at block center x.

[0058]

[0086] To compute Px and Rx at a fixed camera position (e.g., block center x), the entire view direction space is discretized into six view frustums along the ±X, Y, and Z axes, each with a 90° view angle. The block center is the location of the fixed camera position in the evaluation.

[0059]

[0087] In some embodiments, M occluder and M. input The occlusion calculation is reduced from 3D space to 2D screen space by comparing the 2D area occluded by each of the 3D surfaces. Monte Carlo sampling is used to approximate the Px and Rx of each viewing frustum.

[0060]

[0088] N occluders are rasterized in the evaluation space (e.g., 2D screen space, a 2D plane in a 3D virtual space) as occluders. quad For each randomly sampled, axis-aligned rectangle or quad, there are three possibilities:

[0089] (1) True positive: The quad is M occluder and M. input The number of pixels in such a quad is N t It is represented as;

[0090] (2) False positive: The quad is M input It is completely covered by occluder The number of pixels in such a quad is P f It is represented as;

[0091] (3) False negative: The quad is M occluder Although some pixels are completely covered by M input In such a quad, M input The number of pixels not covered by is N f It is represented as;

[0092] The discretized Px and Rx are calculated as follows:

[0093]

[0061]

number

[0094] Here the summation is over all six directions ±X,Y,Z.

[0062]

[0095] Metric-Guided Occluder Simplification

[0096] The results from the patch-based simplification method (e.g., Results 302) and the voxel-based simplification method (e.g., Results 304) are combined to generate a high-quality face candidate set (e.g., Combined Example 306). The final step involves using a metric-guided face reduction algorithm to cull the face subset and preserve its occlusion quality in forming the final occluder. A naive approach for this purpose is to greedily remove the faces that result in the smallest recall reduction ΔR. However, in some embodiments, this is computationally too expensive. Given a combined mesh M with m faces, combined (For example, for the combined example 306), when calculating the recall using n viewing position samples,

[0063]

number

[0064]

[0097] The strategies of (1) pruning neighboring views and (2) skipping unnecessary evaluations further accelerate metric-guided occluder simplification.

[0065]

[0098] Reducing sampled visibility

[0099] If two views are close to each other, the difference between their occlusion results will be small, so in some embodiments the number of view samples is reduced by merging neighbors.

[0066]

[0100] For example, in some embodiments, the method includes uniformly dividing the domain B into N equal-sized blocks (single-sized blocks 800), as shown in FIG. 8A. In other words, the empty space is discretized into a number of single-sized blocks 800. The method replaces 2×2×2 adjacent blocks with larger blocks (e.g., larger block 802 and largest block 804) in an octree-like manner, as shown in FIG. 8B, until no further merging can be performed. For each block, there are six viewing directions along ±X, Y, and Z. If the center of any block is inside the building, no merging can occur. Equation 5 is reformulated as follows:

[0101]

[0067]

number

[0102] Here, ΔV i and x i are the block volume and center position of the i-th sample, respectively.

[0068]

[0103] Sample skip processing:

[0104] If a structure (e.g., a building) is outside the viewing frustum of the viewing position, N t ,P f ,N f would be 0. Thus, before simplification, in some embodiments, one or more views (e.g., majority views, all views) are tested and marked to be skipped if the view cannot see the structure at all. Figure 8C shows the views marked to be skipped (e.g., view 806) in gray because the gray views cannot see the building.

[0069]

[0105] In some embodiments, as shown in Figure 8D, if a face 808 is removed, the metric values ​​measured from viewpoints that cannot see the face will not change. Thus, the evaluation domain is separated into two parts by the orientation of the face. As shown in Figure 8D, the metric values ​​are updated for viewpoint positions 810 that are on the positive side of the face 808 (e.g., on the positive side of the face normal).

[0070]

[0106] Example Implementation

[0107] In some embodiments, the method and system are implemented in C++ using CGAL and libigl. Fast parallel software rasterization on the CPU of Unreal Engine 4 is used to evaluate the occluders and compute the metrics. Specifically, the input model and occluders are rasterized into the depth buffer. Then, a large number of quads are randomly generated on the depth buffer, and precision and recall are computed using Equation 6. To avoid excessive empty pixels in the depth buffer, the minimum length of the bounding box of the input model is set to ε padding In some embodiments, the method was performed on a computer equipped with an AMD Ryzen Threadripper 3970X 32-core processor @ 3.69 GHz with 256 GB RAM.

[0071]

[0108] Ablation study

[0109] Metric Discretization Precision

[0110] Number of screen quads N quad may affect the accuracy of the approximate metric calculation in some embodiments. Using the example construction model (shown in FIG. 2A), the metric is calculated as: 100; 1,000; 5,000; 10,000; 20,000; 30,000; and 100,000 pieces Each metric is calculated using 10 quads per view. For each number of quads, the metric is calculated 10 times with a different set of randomly selected quads. FIG. 9A shows the standard deviation of the 10 metrics. Plot 902 shows the standard deviation of recall at different numbers of quads per view. The other plot in FIG. 9A shows the standard deviation of precision. Throughout the remaining experiments, 5,000 quads per view may be used to balance computational cost and precision.

[0072]

[0111] Sampling Distance

[0112] The visibility sample interval Δx can affect the accuracy of the metric approximation calculation. Halving the sampling distance increases the computational cost by a factor of eight. Experiments show that M input The maximum diagonal length of the bounding box of 8%, 4%, 2%, 1%, 0.5% As above, the metrics were calculated 10 times under each sampling distance. Figure 9B shows the standard deviation for both recall and precision. Figure 9B shows a plot 904 for the standard deviation of recall at different sampling distances. The standard deviations for recall at 4% and 2% are very close, 1.9×10, respectively. -3 and 1.8 x 10 -3 Therefore, 4% was used as the sampling distance throughout the rest of the experiment.

[0073]

[0113] Sample Reduction

[0114] Sample reduction and / or sample skipping can accelerate the metric calculation and, consequently, the mesh simplification induced based on the metric. Through testing all 77 models, the results show that the total number of evaluation tests is reduced to 16.3% of the number of tests without using the evaluation acceleration technique. After skipping sample views that cannot see the model and / or just remove faces, the ratio is further reduced to 13.1% and 5.9%. There is no visible difference between the reduction results with and without sample reduction, with a fluctuation of 0.5% and 0.2% in precision and recall, respectively, but sample reduction can save 83.7% samples on average. Overall, the calculation time can be speeded up by 5.56 times by using sample reduction. Sample skipping does not affect the accuracy of the evaluation results.

[0074]

[0115] Working Example

[0116] The methods and systems described herein were evaluated using a dataset of 77 building models used in games, as shown in Figure 10. Each building has hundreds of components and thousands of intersecting triangle pairs, making it extremely difficult, if not impossible, to process using traditional mesh simplification methods. 5000 randomly generated quads were rasterized with respect to the depth buffer to gather information for calculating precision and recall to evaluate the generated occluders. s (LS)=1% is M input It is set as the maximum diagonal length of the bounding box of l s (LS) is defined as above as the test line segment line for the "fill in the gap" procedure. All hyper-parameters are listed in Table 1. The hyper-parameters refer to parameters that are typically manually adjusted by technical artists designing video game graphics based on their experience prior to the method and system described in this application. There is no theoretical explanation as to why the parameters should assume their particular values.

[0075]

[0117] Table 1. Hyper-parameters

[0076] [Table 1]

[0118] On average, the occluders generated using the methods and systems described herein have 260 faces with a recall of 78.0% and a precision of 99.4%, as shown in Figure 11 and Table 2. Figure 12 also shows the simpligons and 10 example buildings and corresponding occluders generated by the methods and systems described herein.

[0077]

[0119] Conservative Enforcement: Fig. 11 shows that conservative enforcement can improve precision from 94.6% (Fig. 11(a)) to 95.7% (Fig. 11(b)). However, it can be seen that along with the improved precision comes a loss in recall. If we create a given coarse mesh strictly within the input mesh, there will be some areas that are not occupied by the coarse mesh. In some embodiments, a higher precision may be preferred even with a loss in recall, since incorrect culling can lead to severe visual artifacts. The precision of the voxel-based result may be lower than that of the patch-based solution, since the generation of the isosurface is based on the number of windings for the interior-exterior segmentation, which may lead to precision errors.

[0078]

[0120] Combined patch / voxel based results: Figure 11 shows that using the patch-based (Figure 11(a)) and voxel-based (Figure 11(c)) methods as the face candidate set, the generated occluders have recall rates of 41.2% and 78.4%, respectively, and face counts of 500 and 53, respectively.

[0079]

[0121] The combined candidate set in Figure 11(d) yields a significantly higher recall of 86.0%. An example of combining patch-based and voxel-based results can also be seen in Figures 3A-3E.

[0080]

[0122] Timing: Across the 77 test building models, the method and system described herein took an average of 155 seconds (Table 2), while patch-based and voxel-based mesh simplification took 5 and 24 seconds, respectively. Metric-guided mesh simplification took 126 seconds to further reduce the number of faces from 553 to 260.

[0081]

[0123] Table 2. Average number of faces and computation times for 77 test buildings using the illustrated method, simpligon, and planar occluder (*for one-sided rendering, numbers need to be multiplied by 2)

[0082] [Table 2]

[0124] Comparison with Simplgon: Using the visibility-driven mesh simplification pipeline in SimplGon, the number of target triangles is set to 300, which is the number of occluder faces commonly used in mobile games. SimplGon saves silhouettes at a moderately fast speed (3 seconds per model) with a recall of 74.3%, which is 3.7% lower than the recall obtained using the method and system described in this application. Moreover, SimplGon fails to maintain conservatism (96.2% precision compared to 99.4% precision obtained using the method and system described in this application). More importantly, the standard deviation of the precision obtained using the method and system described in this application is only 0.6, while SimplGon's standard deviation is 4.3%, indicating that the method and system described in this application has a more stable performance. A lower precision is more harmful than a lower recall. With a low recall, the system has to render more hidden objects, which hinders rendering efficiency. However, as shown in Figure 1B and Figure 1C, low precision may lead to erroneous culling of objects even when they are visible to the player. For the first model in Figure 12, even though the occluder generated by the simpligon can capture the overall input shape, its recall is only 66.9% due to the small cracks between the cut walls, which significantly weakens its occlusion power.

[0083]

[0125] Comparison with planar occluders: To compare with the planar occluder, the input mesh is first voxelized to generate an isosurface. In the Silvenoinen method, we assume that the building model is viewed from a distance and generate only one plane for each view direction. After selecting a set of planes, the Silvenoinen method greedily removes the triangles with the smallest area and does not minimize the loss of recall. The method and system described here optimizes by evaluating occlusion whenever a face is discarded and pruning faces more accurately. Even with such optimization, the output planar occluder can only achieve an average recall of 39.7% and a precision of 92.3%. One of the reasons is that the test model contains thin walls and nested structures, which are difficult to voxelize accurately. Thus, the output isosurface captures only a small portion of the input mesh with a large precision error, which can also be observed in our voxel-based results (only 41.2% recall is obtained). In terms of computation time, the Silvenoinen method took 267 seconds per model, which is slower than the methods and systems described herein, but the Silvenoinen occlusion metrics are only half of those obtained using the methods and systems described herein. The Silvenoinen method also assumes double-sided rendering. For a culling method with single-sided rendering, the number of faces of the occluder would need to be doubled. For a good comparison, the target output face count of Silvenoinen is set to 150, and the number of faces is doubled at run-time because the game engine used in some embodiments of the methods and systems described herein (based on Unreal Engine 4) uses single-sided rendering.

[0084]

[0126] The method and system described herein combine patch-based and voxel-based face generation techniques. The best face subset is selected to form the occluder based on a novel evaluation metric. Furthermore, two evaluation metrics are introduced across the 3D domain to measure the quality of the occluder, and some strategies to accelerate the evaluation procedure are also described. Testing of the method and system described herein has been performed using 77 building models in Unreal Engine 4, highlighting the ability of the method to generate occluders with higher precision and recall.

[0085]

[0127] The methods and systems described herein are not limited to inputs that are manifold and watertight. Using heuristic problem-specific techniques including hyper-parameters is another approach. The methods and systems described herein are not limited to utilizing patch-based and voxel-based mesh simplification tools, but can fuse results from additional different mesh simplification techniques to improve the quality of the face candidate set. Furthermore, metric-guided occluder simplification may include additional operations such as edge collapse and vertex removal instead of using face reduction alone (although at the cost of higher computational load).

[0086]

[0128] In some implementations, seam-closing is used. As shown in Figure 13, when small narrow patches are removed from the input mesh, seams can occur, which can significantly reduce the precision. To close these seams, the boundaries of all patches are first segmented into a number of straight line segments of length l i (L i) and l j (L.J.), Direction d i and d j Suppose there are two segments with each having the same length. If the two segments of different patches are parallel and of similar length,

[0087]

number

[0088]

number

[0089]

[0129] Seam closing is performed by adding a quad Q formed by the four vertices of the two segments. Note that a candidate will be considered as a real seam only if the gap between the pair is closed in the input mesh, typically by some intersecting narrow patches, e.g., by pillars or rounded corners. To prevent inappropriate seam closing, a number of test line segments of length ls (e.g., ls is between 50 mm and 200 mm, ls is e.g., 100 mm) are uniformly sampled inside Q along the normal direction of Q (Fig. 13(c)). M input As long as there is at least one segment that does not intersect with input This means that there may be a hollow structure in Q, so Q is discarded.

[0090]

[0130] Figure 13(a) shows the input mesh with two walls and one pillar. Figure 13(b) shows the paired segments after patch removal. Figure 13(c) shows the line segments evenly spaced inside the quads formed by the segment pairs. Figure 13(d) shows the mesh after seam closing.

[0091]

[0131] Heterogeneous buildings are buildings that are only partially accessible. In some embodiments, the lower part of a heterogeneous building has an interior structure and can be walked inside by the player, while the upper part is a large closed structure that can only be viewed from the outside. In some embodiments, such heterogeneous building structures can be handled by first splitting the building vertically into two components, applying a coarse mesh generation (e.g., patch-based simplification) only to the lower part and a mesh simplification with conservative constraints to the upper part, and then combining the two results as a final coarse mesh.

[0092]

[0132] In some embodiments, a heterogeneous building has multiple large horizontal patches (usually floors and ceilings) that are oriented downwards and can split the building into two components. Our splitting plane is selected from these candidates. For a candidate plane P, the generated 2D segment soups are +η and P -η The comparison is done by slicing the building using P +η and P -η are two planes parallel to P with a perpendicular offset ±η (e.g., in some embodiments, η≈1×10 -3 A candidate plane is one that is chosen when there is a significant area difference between two segment soups, e.g., the ratio between their areas ε a is the threshold (eg,ε a = 10), it is selected as the final splitting plane. In some embodiments, the plane with the largest difference between the first area and the second area is selected as the final splitting plane.

[0093]

[0133] For a given 2D segment soup, its area is calculated by first obtaining a closed 2D triangle mesh, which can be calculated using TriWild, and then trivially summing the areas of all triangles. Figures 14A and 14B show examples of some candidate splitting planes and their corresponding slice areas for a heterogeneous building model, according to some implementations of the present application. Figure 14A shows an input building and five potential transparent splitting planes. Potential transparent splitting planes are candidate planes for splitting the building into two or more parts. Figure 14B shows upper and lower slices of each splitting plane. 2D intersection segments and interior triangles are shown. In some embodiments, the method selects splitting plane (4) because the area of ​​the lower slice is significantly smaller than the upper slice.

[0094]

[0134] FIG. 15 illustrates a process performed by a computing system to automatically generate occluders for visual three-dimensional structures according to some implementations of the present application.

[0095]

[0135] First, a computing system receives (1510) an input model of a visual three-dimensional structure, the input model including a plurality of faces. The computing system then generates (1520) an initial occluder by simplifying the input model into a plurality of candidate patches in a patch-based coarse mesh. In some embodiments, the visual three-dimensional structure corresponds to a visual structure in a video game. In some embodiments, the computing system automatically generates the occluder before a user initializes a video game to be played. In some embodiments, the computing system automatically generates the occluder during a production phase of the video game. In some embodiments, after a user initializes the video game, the video game displays structures and images that are not obscured by the occluders. In some embodiments, the visual three-dimensional structure is mapped with texture and color before being displayed (e.g., during the production phase of the video game, before a user initializes the video game). In some embodiments, the coarse mesh is an approximation mesh generated from a fine rendering mesh (e.g., input mesh, visual mesh). In some embodiments, the coarse mesh has not undergone refinement (e.g., no metric-based simplification, no maintainability check yet). The initial occluder occludes objects behind the visual three-dimensional structure along a first viewing direction. In some embodiments, the occluder is automatically generated during a production stage of the video game, but graphics associated with the visual three-dimensional structure are not rendered. The computing system compares a first two-dimensional area occluded by an input model of the visual three-dimensional structure to a second two-dimensional area occluded by the initial occluder along the first viewing direction and determines a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder (1530). The computing system removes faces from the initial occluder while maintaining the first quality metric above a first threshold to form an occluder for the visual three-dimensional structure (1540).The occluder blocks objects behind the visual three-dimensional structure from being rendered in the application along a first viewing direction. In some embodiments, the automatically generated occluder is the final occluder used in the video game application (e.g., the initial occluder is not used by the video game application at run time).

[0096]

[0136] In some implementations, the computing system voxelizes the input model to obtain a plurality of voxels for generating a voxelization-based coarse mesh (1550), and combines the patch-based coarse mesh and the voxelization-based coarse mesh to form an initial occluder (1560). In some implementations, the computing system determines a second quality metric based on a second number of pixels blocked by both the initial occluder and the input model (1570). Simplifying the input model into a plurality of candidate patches includes merging a first number of faces of the plurality of faces into a plurality of candidate patches, the plurality of candidate patches meeting a threshold requirement for the first quality metric or the second quality metric. In some implementations, merging the first number of faces into a plurality of candidate patches includes combining faces that fall within a first degree of coplanarity into a candidate planar patch of the plurality of candidate patches (1580).

[0097]

[0137] In some implementations, generating the voxelization-based coarse mesh by the computational system includes: voxelizing a bounding box of the input model to obtain a plurality of voxels; calculating a winding number for each of the plurality of voxels; extracting an isosurface based on the winding numbers; and simplifying the isosurface to obtain the voxelization-based coarse mesh (1590).

[0098]

[0138] Although specific implementations have been described above, it will be understood that it is not intended to limit the invention to these specific implementations. Rather, the invention encompasses alternatives, modifications, and equivalents that are within the spirit and scope of the appended claims. Numerous specific details have been described to provide a thorough understanding of the subject matter presented herein. However, it will be apparent to those skilled in the art that the subject matter may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the implementations.

[0099]

[0139] Although terms such as first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another element. For example, a first ranking criterion may be referred to as a second ranking criterion, and similarly, a second ranking criterion may be referred to as a first ranking criterion, without departing from the scope of the present application. A first ranking criterion and a second ranking criterion are both ranking criteria, but are not the same ranking criterion.

[0100]

[0140] The terms used in the description of the invention herein are for the purpose of describing particular implementations only and are not intended to limit the invention. As used in the specification of the invention and the appended claims, "a", "an", "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "comprise", "including", "comprising", and / or "comprising" as used herein specify the presence of stated features, processes, elements, and / or components, but do not exclude the presence or addition of one or more other features, processes, elements, components, and / or groups thereof.

[0101]

[0141] As used herein, the term "if" may be interpreted to mean "when" or "upon" or "in response to determining that" or "pursuant to a determination that" or "in response to detecting that" the stated condition precedent is true, depending on the context. Similarly, the phrases "if it is determined that [the stated condition precedent is true]," "if [the stated condition precedent is true]," and "if [the stated condition precedent is true]" may be interpreted to mean "upon determining," "in response to determining," "pursuant to determining," "upon detecting," or "in response to detecting" that the stated condition precedent is true, depending on the context.

[0102]

[0142] Although some of the various figures depict some logical stages in a particular order, stages that are not order dependent may be rearranged and other stages may be combined or separated. While some rearrangements or other groupings are specifically mentioned, others will be apparent to those of ordinary skill in the art, and thus this is not intended to present an exhaustive list of alternatives. Furthermore, it should be recognized that these stages may be implemented in hardware, firmware, software, or any combination thereof.

[0103]

[0143] The foregoing description has been described with reference to specific implementations for purposes of explanation. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The implementations have been chosen and described to best explain the principles of the invention and its practical application, thereby enabling those skilled in the art to best utilize the invention and its various embodiments, with various modifications suited to the particular use envisioned. The implementations encompass alternatives, modifications and equivalents within the spirit and scope of the appended claims. Numerous specific details have been described to provide a thorough understanding of the subject matter presented herein. However, it will be apparent to those skilled in the art that the subject matter may be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the implementations.

Claims

1. 1. A method for automatically generating occluders for a visual three-dimensional structure, comprising: receiving an input model of the visual three-dimensional structure, the input model including a plurality of faces; generating an initial occluder by simplifying the input model into a number of candidate patches in a patch-based coarse mesh, the initial occluder blocking objects behind the three-dimensional structure along a first viewing direction; comparing a first two-dimensional area occluded by an input model of the visual three-dimensional structure to a second two-dimensional area occluded by the initial occluder along the first viewing direction and determining a first quality metric based on a first number of pixels blocked by the input model that are also blocked by the initial occluder; and removing faces from the initial occluder while maintaining the first quality metric above a first threshold to form an occluder for the visual three-dimensional structure, the occluder blocking objects behind the visual three-dimensional structure from being rendered in an application along the first viewing direction; The method includes:

2. 13. The method of claim 1 further comprising: voxelizing the input model to obtain a number of voxels for generating a voxelization-based coarse mesh; and combining the patch-based coarse mesh and the voxelization-based coarse mesh to form the initial occluder; The method includes:

3. 3. The method of claim 2, further comprising determining a second quality metric based on a second number of pixels blocked by both the initial occluder and the input model, and wherein simplifying the input model into a plurality of candidate patches comprises merging a first number of faces of the plurality of faces into the plurality of candidate patches, the plurality of candidate patches meeting a threshold requirement for the first quality metric or the second quality metric.

4. 4. The method of claim 3, wherein merging the first number of faces into the plurality of candidate patches includes combining faces that fall within a first degree of coplanarity into a candidate planar patch of the plurality of candidate patches.

5. 5. The method of claim 4, in response to determining that a shared edge of two candidate planar patches is less than a threshold, grouping the two candidate planar patches into a curved patch; The method further comprises:

6. The method of claim 3 , wherein the first quality metric or the second quality metric is a user-specified value.

7. 3. The method of claim 2, wherein generating a voxelization-based coarse mesh comprises: voxelizing a bounding box of the input model to obtain the plurality of voxels; calculating a winding number for each of the plurality of voxels; Extracting an isosurface based on the winding number; Simplifying the isosurface to obtain a voxelization-based coarse mesh; The method includes:

8. 8. The method of claim 7, Extracting the isosurface includes using a marching cubes algorithm; The method, wherein simplifying the isosurface includes generating a simplified mesh using a quadratic error method (QEM) guided method.

9. 9. The method of claim 8, including performing conservative constraints on the simplified mesh to constrain the simplified mesh within the input model.

10. 10. The method of claim 9, wherein implementing conservative enforcement comprises: determining a positive and negative distance field from said isosurface; determining each edge in the simplified mesh in terms of a spring energy; and applying a penalty energy if the simplified mesh exceeds the input model.

11. 2. The method of claim 1, wherein removing the plurality of faces comprises: discretizing a space in and around the input model into a plurality of equally sized blocks, each of the equally sized discretized blocks having a plurality of viewing directions; merging a portion of the plurality of equal sized blocks into a plurality of larger blocks; removing a view direction from each of the plurality of larger blocks such that the input model is outside a view frustum of the view direction.

12. 12. The method of claim 11, wherein the plurality of viewing directions includes six viewing directions along ±X, Y, Z for each larger block in the plurality of larger blocks and for each block in the plurality of equal-sized blocks.

13. The method of claim 1 further comprising the step of filling holes in the coarse mesh of the patch base.

14. 14. The method of claim 13, wherein a hole is filled if a test segment oriented perpendicular to a quad that covers a gap in the patch-based coarse mesh intersects with the input model.

15. 15. The method of claim 14, wherein the visual three-dimensional structure is a building, and windows of the building are not blocked by the occluder.

16. 2. The method of claim 1, wherein the visual three-dimensional structures include buildings having non-manifold, non-watertight, and / or self-intersecting characteristics, the buildings including interior structures having defined viewing directions within.

17. 20. The method of claim 16, further comprising determining a plane dividing the building into a first portion and a second portion, the second portion including the interior structure.

18. 20. The method of claim 17, wherein the step of determining the plane comprises: For each of the candidate planes: a first area of ​​the building retrieved by an upper slice offset from each of the plurality of candidate planes; a second area of ​​the building retrieved by a bottom slice offset from each of the plurality of candidate planes; determining selecting, from among the plurality of candidate planes, a plane having the largest difference between the first area and the second area; The method includes:

19. 2. The method of claim 1, wherein the input model is represented as a mesh comprising a polygon soup, the mesh including hundreds of disconnected components and nested structures.

20. The method of claim 1 , wherein the number of faces in the occluder is less than 10% of the number of faces in the input model.

21. A computing device comprising a memory for storing a program and one or more processors, the program, when executed by the one or more processors, causes the computing device to perform a method for automatically generating occluders for visual three-dimensional structures according to any one of claims 1-20.

22. A non-transitory computer readable storage medium storing a program associated with a computing device including one or more processors, the program, when executed by the one or more processors, causing the computing device to perform a method for automatically generating occluders for visual three-dimensional structures according to any one of claims 1-20.

23. A computer program product causing a computer processor to carry out a method according to any one of claims 1 to 20.

Images