Method for instant rendering of voxels

Abstract

A method for immediate rendering of voxels. The present invention relates to a computer-implemented method for visualizing a 3D model of a structure from a plurality of different viewpoints in sequence on a display. The method includes providing model geometry data and attribute information in one or more data structures on a data storage device; performing a rendering process for providing a visualization of the model on the display for each of a large number of viewpoints. The rendering process includes rendering a large number of similar 3D arrays in a single draw call, each array including at least eight voxels; removing non-visualized voxels by folding vertices to the center of the array; calculating a distortion on a mapping from voxel space to attribute volume space based on the removed voxels; assigning attributes to the voxels of the large number of arrays using the distortion value for retrieving attribute values from the attribute volume space and assigning attributes to the remaining voxels according to the calculated distortion; visualizing the voxels of the large number of arrays on the display. The present invention also relates to a computer system and a computer program product for performing the method.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional (3D) computer graphics, and more particularly, to an improved computer-implemented method and computer system for visualizing 3D structures using voxels. Background Art

[0002] In essence, rendering 3D scenes using voxels is known in the art and is used in many applications. It is also known to use 3D computer graphics techniques to convert data describing a 3D scene into a two-dimensional (2D) image for viewing on an electronic display screen. To visualize 3D scenes including structures, there are various prior art solutions, including ray tracing, ray marching, cone tracing, visualization of high-detail real volume data, meshing, marching cubes, dual contouring, greedy meshing and caching, and instantiated cubes.

[0003] US 4,710,876 A discloses the so-called Marching Cubes algorithm, an algorithm for extracting a polygonal mesh of an isosurface from a three-dimensional discrete scalar field (also known as a voxel). The premise of the algorithm is to divide the input volume into a set of discrete cubes. By assuming linear reconstruction filtering, it is easy to identify the individual cubes that contain a piece of a given isosurface, because the sample values at the cube vertices must straddle the target isosurface value. For each cube in the section containing the isosurface, a triangular mesh is generated that approximates the trilinear interpolation behavior within the interior cube.

[0004] In real-time computer graphics, geometry instancing is the practice of rendering multiple copies of the same mesh in a scene at once. This technique is mainly used for objects that can be represented as repetitive geometric structures without excessive repetition. Although the vertex data is replicated across all instanced meshes, each instance can have other differentiating parameters, such as a color that changes to reduce repetition. Most current video hardware (including desktop computers and smartphones) supports this type of instanced rendering. This method explicitly allows multiple copies of the mesh to be rendered sequentially by specifying differentiating parameters for each copy of the mesh in a separate stream, thereby improving the potential runtime performance of rendering instanced geometry.

[0005] Geometry instancing typically involves mapping static or pre - animated objects or geometries to particles or arbitrary points in space, which can then be rendered by almost any off - line renderer. It allows for per - instance changes to texture or color or other properties, which helps ensure that instances do not look like exact copies of each other. Since the instanced geometry only references the original object, the file size remains very small, and changing the original object changes all instances. Instancing can be achieved by using a deferred - load renderer to load the geometry only when actually rendering the bucket that contains the instances. This means that the geometries of all instances do not have to be in memory at the same time.

[0006] Modern graphics processing units (GPUs) are very efficient at manipulating computer graphics and image processing. For algorithms that process large data chunks in parallel, the highly parallel structure of GPUs makes them more efficient than general - purpose central processing units (CPUs). Hardware instancing is a known technique that allows a GPU to render multiple identical meshes in a single draw call, changing a limited amount of state (such as position and orientation) for each mesh. This is typically used to reduce the cost of drawing a moderate number of identical medium - to - low - detail meshes.

[0007] It is desirable to provide a viable and efficient solution for rendering 3D models of structures that combines the advantages of voxel rendering and hardware instancing. Summary of the Invention

[0008] Accordingly, an object of the present invention is to provide an improved method for efficiently rendering 3D models of large structures.

[0009] Another object is to provide such a method that can be executed on a variety of hardware, including personal computers and mobile devices.

[0010] Another object is to provide such a method that reduces the memory required to store model data.

[0011] Another object is to provide such a method that reduces the rendering time and allows for a higher frame rate when visualizing the model sequentially from different viewpoints.

[0012] Another object is to provide such a method that reduces the computational power and power consumption required on the rendering device.

[0013] Another object of the present invention is to provide a computer system and a computer program product for performing the method.

[0014] The proposed invention and the proposed algorithm according to the invention utilize voxels to visualize 3D objects or structures. The term "voxel" as used in this document specifically refers to a voxel in the classical sense, i.e., a three-dimensional volume pixel. This means that according to a preferred embodiment of the invention, a voxel can be an axis-aligned cube on a regular grid, with each voxel having a single set of attributes (e.g., texture, color, or material).

[0015] In an exemplary embodiment of the invention, in a first step, geometric data is provided in the form of an octree or a similar structure, which can divide space into eight or more identical sub-spaces in a 3D grid. This can include, for example, obtaining data by generating one or more point cloud scans of the structure, or rasterized polygon data related to the structure; and then storing the geometric data on a data storage device accessible by a computer system. The octree is traversed to obtain a bitmask as instantiation data, which is used to select the geometric information to be visualized using voxels for a corresponding viewing angle. This derivation is based on the viewing angle within the octree representing the scene including the structure. Here, the term "bitmask" is used in a broad sense, i.e., meaning a list of n binary values (e.g., values that can be true or false), where both the position and the value in the list are meaningful. For example, instead of a narrow bitmask, the list can be used with up to n indices that indicate which entries have a specific value, and the absence of an index indicates the opposite value.

[0016] Subsequently, in an exemplary embodiment of the invention, the geometric information of multiple 2×2×2 voxel arrays (whose data is stored according to the octree) is rendered, where the attribute information (such as texture) is not yet included. According to the current viewpoint, multiple identical processes are used to calculate and render the complete structure or a part thereof, and these processes are optimized using known hardware instantiation processes. According to the invention, this is achieved by rendering multiple identical cube arrays using hardware instantiation or a similar hardware feature (for example, NVIDIA's mesh shader), with each voxel array (or cube array) including eight cubes arranged in a 2×2×2 layout. The optimization according to the invention is based on the handling of multiple identical processes, and for this purpose, a certain overhead can be used for the hardware. The attribute information is provided separately, for example, in a separate buffer, and is subsequently assigned to individual voxels. Therefore, geometric data processing and subsequent shading are separate steps.

[0017] Subsequently, these voxels that do not need to be visualized are removed (i.e., "folded away") by folding the vertices of those voxels in a 2×2×2 array to the center of the array. Finally, in order to avoid the coloring artifacts that inevitably occur due to this folding, it is necessary to change the attribute geometry (e.g., texture geometry). To this end, in an exemplary embodiment of the present invention, the texture geometry of the cube is distorted based on a calculation in which those voxels in the 2×2×2 array are considered to have been folded away. The assignment of coloring or other attributes is based on this distorted geometry in order to correct and thereby prevent the incorrect assignment of colors or other attributes to the cube faces adjacent to the cube faces of the voxels that are omitted (folded). Advantageously, the attribute data itself is not affected by the distortion of the texture geometry.

[0018] Thus, a first aspect of the present invention relates to a computer-implemented method for sequentially visualizing a three-dimensional model of a structure from a plurality of different viewpoints on a display. The method includes the steps of: providing geometric data and attribute information of the model in one or more data structures on a data storage device; and, for each of a large number of viewpoints, performing a rendering process for providing a visualization of the model on the display. The rendering process includes the steps of:

[0019] - rendering a large number of similar elements in a single draw call, where the large number of elements includes a large number of three-dimensional arrays, each array including at least eight voxels;

[0020] - removing one or more voxels that do not need to be visualized at least in a first array of the large number of arrays, where each voxel that does not need to be visualized is folded away by folding the vertices of the voxel to the center of the array;

[0021] - calculating a distortion on the mapping from voxel space to attribute volume space based on the one or more removed voxels;

[0022] - retrieving attribute values from the attribute volume space using the distorted values to assign attributes to the voxels of the large number of arrays, where the attributes are assigned to the remaining voxels according to the calculated distortion; and

[0023] - visualizing the voxels of the large number of arrays on the display.

[0024] In particular, the calculated distortion is a one-dimensional coordinate distortion, i.e., including the deformation of the attribute surface coordinates of the voxels in at least one dimension (especially only one dimension).

[0025] According to one embodiment of the method according to the present invention, the rendering of the large number of similar elements is performed in a single draw call using hardware instancing. In one embodiment, the step of rendering the large number of similar elements in a single draw call using hardware instancing is performed fully automatically by the GPU of the computer system executing the method.

[0026] According to another embodiment of the method, the geometric data is organized as a sparse voxel octree. In one embodiment, the method further comprises the steps of traversing the octree to obtain a bitmask (including also bitmasks in a broad sense) of at least a subset of the geometric data to be visualized based on the actual viewing point.

[0027] Preferably, these bitmasks can be 8-bit bitmasks, and each array in the large number of arrays can include eight cube voxels in a 2×2×2 configuration. Moreover, each bitmask can be or include a list that includes information about which child nodes of the nodes of the octree include valid data. The sparse voxel octree can optionally include a hierarchical set of bitmasks indicating the presence of child nodes, as well as any associated attribute information. Additionally or alternatively, the step of removing voxels can be performed based on the bitmask.

[0028] In another embodiment, a first array corresponds to a node of the octree, where the local coordinates stored in the node are used to assign attributes to the voxels in the first array, for example, using a single scaling or offset operation.

[0029] According to another embodiment of the method, each of the large number of three-dimensional arrays represents data at a specific level of detail. Which arrays to include in the large number of arrays to be rendered can be selected based on a level-of-detail calculation.

[0030] According to another embodiment of the method, the inner faces between the voxels of the same array are not rendered; and the original inner faces of the remaining voxels, which become visible due to the removal of the voxels adjacent to the remaining voxels, are assigned attributes. The step of calculating the distortion can be performed to avoid incorrect attribute assignment due to hardware sampling rules and attribute coordinate rounding operations.

[0031] According to another embodiment of the method, the step of rendering the first array includes accessing a single static index buffer containing 48×3 indices.

[0032] According to another embodiment of the method, each voxel is triangulated into a hexagon around the corners of the voxel.

[0033] According to another embodiment of the method, the step of folding vertices includes (only) folding those vertices that are not shared with one or more voxels to be visualized.

[0034] According to another embodiment of the method, the step of rendering the first array includes: accessing a single static vertex buffer having at least 27 vertices, each vertex being assigned an associated list of those voxels of the first array that use the vertex, for example, wherein the list is stored as a static bitmask array in shader code.

[0035] According to another embodiment of the method, the geometric data is quantized into an axis-aligned regular three-dimensional grid, for example, by sampling a three-dimensional function.

[0036] According to another embodiment of the method, the steps of rendering a large number of similar features, removing voxels, and calculating distortions are performed entirely automatically by the GPU of the computer system executing the method, at least in a single draw call.

[0037] According to another embodiment of the method, the geometric data is generated based on one or more point cloud scans of the structure or on rasterized polygon data related to the structure.

[0038] According to another embodiment of the method, the geometric data and the attribute information are provided in separate data structures, in particular, wherein the geometric data is organized as a sparse voxel octree or as a directed acyclic graph.

[0039] According to another embodiment of the method, the attribute information includes texture and / or color information.

[0040] According to yet another embodiment of the method, when receiving an input from a user to change the actual viewing point to a new viewing point, the method includes the following steps: for the new viewing point, performing a rendering process to provide a visualization of the model on a display. For example, the input can be received on an input device (such as a mouse, keyboard, or touch screen) of the computer system executing the method.

[0041] In one embodiment, the method further includes the following steps: traversing the data structure of the geometric data (such as a sparse voxel octree) to obtain a bitmask (including a bitmask in a broad sense) of at least a subset of the geometric data to be visualized based on the new viewing point.

[0042] A second aspect of the present invention relates to a computer system configured to execute the method according to the first aspect. The computer system includes at least a data storage device, a GPU, an input device (such as a mouse, keyboard, or touch screen), and a display device, wherein the GPU is configured to render a large number of features in a single draw call.

[0043] A third aspect of the present invention relates to a computer program product comprising program code stored on a machine-readable medium or implemented by an electromagnetic wave comprising a program code segment, and the computer program product has computer-executable instructions which, when run on a computer system according to the second aspect, in particular execute the method according to the first aspect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Hereinafter, the present invention will be described in detail by way of exemplary embodiments with reference to the accompanying drawings, wherein:

[0045] Figure 1 An exemplary computer system according to the present invention is shown;

[0046] Figure 2 A first exemplary voxel array generated by hardware instantiation is shown;

[0047] Figure 3 An exemplary large number of arrays including the first array generated by hardware instantiation is shown; Figure 2 ...

[0048] Figure 4 Vertices in the first array are illustrated;

[0049] Figures 5a to 5c The folding of unused voxels in the first array is illustrated;

[0050] Figure 6 The modification of mapping coordinates is illustrated;

[0051] Figures 7a to 7b The distortion of the texture surface of the voxels of the array is illustrated;

[0052] Figure 8 The first array is shown, in which unused voxels are removed and texture assignment is corrected; and

[0053] Figure 9 A flowchart illustrating an exemplary embodiment of the method according to the present invention is shown. DETAILED DESCRIPTION

[0054] Figure 1FIG. 0 shows an exemplary computer system 1 according to the present invention. It is configured to execute an embodiment of the method according to the present invention described further below. The computer system 1 may include a motherboard 10, which includes circuitry for powering and connecting to at least one on-board processor (not shown here), and the at least one on-board processor may include two or more discrete processors or a processor having multiple processing cores. The motherboard 10 may act as an interface between the microprocessor of the computer system 1 and the memory device 20. The memory device 20 may include one or more optical, magnetic, or solid-state drives, and may store instructions for the operating system and other software of the computer system 1. The motherboard 10 may also communicate with the random access memory (RAM) and read-only memory (ROM) of the computer system 10, and the ROM typically stores instructions for the basic input / output system (BIOS), which can be accessed by the microprocessor and cause the microprocessor to prepare for loading the operating system.

[0055] The motherboard 10 is also connected to a graphics processing unit (GPU) 15. In some systems, the GPU 15 may be integrated into the motherboard 10. The motherboard 10 may be connected to a communication adapter 13 (such as including a LAN adapter or a modem), and this communication adapter enables the computer system 1 to communicate data with a computer network such as the Internet.

[0056] The user of the computer system 1 interacts with the computer system by means of one or more input devices 11 (such as a keyboard or a mouse) and one or more display devices 12.

[0057] It should be understood that the computer system 1 includes an exemplary electronic processor-based system for executing the method. However, the method may also be executed using other electronic processor-based systems. Such systems may include: tablet computers, laptop computers and netbook computing devices, cellular smart phones, game consoles, and other imaging devices (such as medical imaging devices).

[0058] The user of the system 1 may operate the operating system to load a computer graphics-related software product, and this software product may be provided by downloading from the Internet or as tangible instructions carried on a computer-readable medium (such as an optical disc). The computer graphics-related software product includes a data structure that stores data defining various scenes, and this data includes at least geometric data 22 and texture data 24 (or other attribute data). The software product also includes instructions for the GPU 15 to manipulate the scene data in response to input received from the user via the input device 11.

[0059] In a preferred embodiment of the present invention, an octree data structure is used to store the geometric data 22. However, other types of data structures may also be used, such as a binary space partitioning tree. In particular, any sparse structure that can divide the space into eight or more identical subspaces according to a three-dimensional grid can be used, such as a Directed Acyclic Graph. The texture data 24 is provided in a different data structure. For the purposes of the present invention, this separation is necessary because there is actually no unique geometric structure stored, only a single general 2×2×2 cubic array, which is reused for each actual instance of the voxel combination in the scene. The only information that is actually unique is stored in the texture (such as color) and in the CPU-side octree (recursive bitmask).

[0060] According to the present invention, the GPU 15 is configured to render multiple identical meshes in a single draw call. For example, the GPU 15 can be configured to perform hardware instancing. Hardware instancing is a known technique that allows the GPU 15 to render multiple identical meshes in a single draw call, changing a limited amount of state for each mesh (such as position and orientation). As is well known, using this technique can reduce the cost of rendering a moderate number of identical medium- to low-detail meshes. Video cards that support hardware instancing are now widely used in a variety of devices, including desktop computers and smartphones. For example, these video cards include GeForce 6000 and later (NV40 GPU or later), ATI Radeon 9500 and later (R300 GPU or later), and PowerVR SGX535 and later (iPhone 3GS and later).

[0061] However, instead of (or in addition to) widely available hardware instancing, there are other more modern hardware features (such as compute shaders, geometry shaders, or mesh shaders) that can be used to implement and (possibly) improve the algorithm. Even though this document specifically mentions hardware instancing in some places, other hardware methods that can generate a large amount of geometry by reusing a small input dataset are also viable.

[0062] As further described below, according to some embodiments of the present invention, hardware instancing is used to render multiple identical three-dimensional cube arrays. Then each array is adjusted individually as needed to visualize a scene including the 3D model 2 with the correct geometric and texture information. Using the available hardware capabilities of the system 1, particularly those provided on the GPU 15 side, all of these steps can be performed completely automatically, efficiently, and in real time.

[0063] Figure 2An exemplary embodiment of such a 3D array 30 is illustrated. The shown cube array includes eight cubes or voxels 40 to 47 (voxel 42 is not visible) arranged in a 2×2×2 layout. The local XYZ coordinates of the cubes are in the range of -1 to 1.

[0064] Mapping from XYZ coordinates to cube indices:

[0065] index = X≥0? 1:0 |

[0066] Y≥0? 2:0 |

[0067] Z≥0? 4:0.

[0068] The mapping from local space to global space can be performed in a single scale / offset operation. Of course, different voxel layouts can be used, with a higher or lower number of voxels in the array. The 2×2×2 layout presented in this document relates to an exemplary embodiment that can preferably be used with geometric data stored in an octree.

[0069] Figure 3 An example of generating a large number of arrays 3 using hardware instantiation is shown, Figure 2 and the array 30 is highlighted in the upper left corner. The colors of each array in the large number of arrays 3 are tightly packed as 2×2×2 blocks into a 3D texture. Empty data blocks are shown in black.

[0070] In an alternative embodiment, a single 2×2×2 block in the 3D texture can hold the property (e.g., color) data of multiple arrays with disjoint bitmasks. In many cases, this can be used to significantly reduce the size of the 3D texture data.

[0071] Hardware instantiation can be seen as a two-step process. Whenever there is a significant change in the viewing point, the instantiation data generated on the CPU side (i.e., information specifying where to draw and how to color each individual instance) needs to be generated from the CPU-side octree. This can be done according to heuristics. The result of this step is stored in a temporary buffer and can be reused as long as it remains valid. However, applying the instantiation data during rendering is performed on the fly by the GPU and is a process that the graphics hardware is designed to perform repeatedly and efficiently.

[0072] The instantiation data is generated by traversing the CPU-side sparse octree. Only the blocks with at least one visible voxel are added to the instantiation data. If any traversed node ends with an empty bitmask (which would happen if all the children of the node were added to the instantiation data), then the node itself is not added to the instantiation data.

[0073] Each array corresponds to a node of the octree, which stores the geometric data of the objects to be visualized. The local coordinates of these nodes are used as mapping coordinates using a single scaling / offset operation.

[0074] Sampling of the 3D texture can be done using point filtering in the fragment shader.

[0075] Then, the information required for each array includes:

[0076] - Position (XYZ),

[0077] - Scaling or LOD level,

[0078] - Attributes, such as texture / color, (UVW), and

[0079] - Visible cube position mask.

[0080] The scaling or LOD level directly corresponds to the level in the octree. The texture / color is an offset of the 3D texture, which is combined with the local coordinates of the node to obtain the mapping coordinates.

[0081] Theoretically, this information can contain up to 168 bits, but in practice, it is possible to reduce it to at least 64 bits depending on the size of the input data.

[0082] Figure 4 Shows Figure 2 Array 30, exemplifying 27 vertices V0 to V26 of the array (not all vertices are visible).

[0083] A single static vertex buffer with 27 vertices sorted by increasing X, Y, Z values can be used as the input vertex buffer. The mapping from indices to local space coordinates can be stored either in the vertex buffer or directly in the vertex shader code.

[0084] Each vertex V0 to V26 has an associated list of the cubes that use that vertex (cube usage mask), which is stored as a static bitmask array in the shader code. The vertex V13, which forms the center of eight cubes, is not actually used for rendering and has a usage mask of 0x00.

[0085] Figures 5a to 5c Illustrates the process of removing unused cubes by "folding up". As Figure 5a Exemplified, a single static index buffer containing 48×3 indices is used as the input index buffer, where each cube of array 30 is triangulated into hexagons around the corners of the cube, and these triangles are formed by the vertices of the cube. Due to the index buffer layout, the inner faces between the cubes are never rendered.

[0086] In Figure 5b , a cube has been removed. In Figure 5c , three of the eight cubes of the array 30 have been removed. This is performed by folding unused triangles (i.e., triangles of unused cubes) to a zero region (e.g., folding to the center of the array) so that the hardware can quickly reject triangles. In summary, removing unused cubes by "folding up" involves folding those vertices of the unused cubes that are not shared with any other still visible cubes (specifically including the triangles formed by these vertices).

[0087] As such, in the "worst case", i.e., if seven of the eight cubes are folded up, rendering a single cube requires 48 triangles, which is not very efficient. However, in the best case, i.e., if no cubes of the array need to be removed, 8 cubes are produced from the same 48 triangles.

[0088] In the next step, attributes such as texture and color need to be assigned to the remaining voxels, i.e., those cubes that have not been folded up. On the modified triangles, the texture and color are determined based on the hardware sampling rules and the texture coordinate rounding operations performed by the GPU. Without applying the correction described further below, after folding, typically about every other triangle among the triangles will be assigned incorrect texture and color, which is unacceptable for many applications.

[0089] Figure 5b and Figure 5c illustrates this hardware - inherent problem, where previously invisible inner faces have been assigned the wrong color. That is, instead of assigning the color associated with the now - visible faces of those cubes, the hardware has incorrectly assigned the color of the unused, now - folded - up cubes.

[0090] In Figure 5c 's example, the remaining voxels 40, 43, 46, and 47 were previously neighbors of the removed voxels. Thus, the previously invisible inner faces of these cubes (i.e., the faces that would not have been rendered if no voxels were removed) are now visible and thus need to be assigned some texture. Due to the hardware - inherent problem mentioned above, this assignment is typically incorrect. As can be seen in the example of Figure 5c , the faces of the remaining voxels 40, 43, 46, and 47 that were previously inner faces have been incorrectly assigned the texture of the removed voxels instead of the texture of the remaining voxels.

[0091] According to the present invention, this problem is solved by using the correction described below.

[0092] Figure 6 Illustrates the modification of the mapping coordinates. The array is divided into positive (bright) and negative (dark) half - spaces along each axis. The mapping coordinates of the vertices are adjusted along each axis, where the number of visible cube influences is different in the positive / negative spaces. The amount of correction is a function of the cube usage count of the respective vertices V0 to V26 and can also depend on the specifications of the hardware used (e.g., GPU). Thus, the exact value of the correction amount can be determined using the exact knowledge of the hardware behavior.

[0093] Figures 7a to 7b Illustrates the distortion of a texture surface (or other property surface) on two exemplary arrays. Calculating the distortion (e.g., one - dimensional deformation) is necessary to induce the hardware to always assign the correct properties such as textures and / or colors. For simplicity, the arrays are shown in 2D. In Figure 7a the example, the upper - right cube has been removed, and in Figure 7b the example, the two cubes on the right have been removed. In both cases, the arrows indicate the distorted movement of the mapping coordinates.

[0094] These movements only include the distortion of the mapping coordinates and do not change any vertex positions used for rendering. The distortion is calculated on the mapping from the voxel space to the property volume space. The distortion only needs to be one - dimensional.

[0095] Preferably, the movement of the central vertex should be as small as possible, just enough to round the mapping coordinates in the correct direction. Thus, in the shown example, the arrows indicating the movement of the respective central vertices are quite short. However, in the shown example, the vertices on the outer edges of the array move completely towards the corners of the array. The vertical texture offset is 0, and now horizontal offsets are required along both the top and bottom edges. Simple bit operations in the shader can also be used to detect the required texture coordinate modifications.

[0096] Figure 8 Shows an array 30 with corrected color assignment according to the present invention. The previously unrendered inner faces now show the colors of the cubes associated with the now - visible faces. The shown array 30 is the same as that depicted in Figure 5c which had the colors incorrectly assigned.

[0097] Since the texture offset correction is symmetric along each axis, the texel boundaries match the cube boundaries correctly. Careful scaling of the offset at the center of the array is required to avoid color bleeding from the removed cubes.

[0098] Color and texture assignment corrections are performed separately for each array. More specifically, for each of the 26 vertices (excluding the central vertex) of each array, the steps of removing the cube and calculating the distortion are performed independently and in parallel. Each vertex "knows" how to perform these steps simply by looking at the bitmask of the visible cube, without the need for information from other vertices to do so. Whenever the viewpoint or anything else that would change the scene (i.e., the currently rendered image) changes, these steps are repeated. However, these steps themselves are not separate steps, but are performed instantaneously (automatically and in real time) by the GPU as part of the rendering process.

[0099] Each rendered array represents a portion of the source dataset at a specific level of detail (LOD). Also, different parts of the octree can be rendered at different LODs. Masking out the corresponding bits from the instance data of a node when traversing down the child nodes will automatically remove the corresponding cube from the current node. If all child nodes are traversed, the resulting mask is 0 and the node does not need to be rendered. Thus, the selection of which arrays to include in the set of rendered arrays is also based on the LOD of these arrays.

[0100] Figure 9 A flowchart illustrating an exemplary embodiment of method 100 according to the present invention is shown, which is used to visualize a 3D model of a structure from multiple different viewpoints in sequence on a display.

[0101] In a first step, for example, data of a scene (i.e., a view of the structure seen from a first viewpoint) is retrieved 110 from a data storage device of a computer system for performing method 100. This includes retrieving geometric data 22 stored in an octree and texture data 24 (or other attribute data) that may be stored in a separate data structure. Retrieving 110 the geometric data includes traversing the octree to obtain the data required to render the geometric structures visible in the scene.

[0102] Based on the retrieved geometric data 22, the available hardware capabilities of the system (especially those provided on the GPU side) are used to render multiple identical voxel arrays in a single draw call. In Figure 9 the exemplary embodiment shown, the GPU performs hardware instancing 120 to render multiple identical voxel arrays in a single draw call, each voxel array including eight cube voxels arranged in a 2×2×2 layout.

[0103] Since not all voxels of all 2×2×2 arrays need to be visualized, the unused voxels are folded up 130 (i.e., removed), as described above with reference to Figures 5a to 5c what has been described. This step is also performed by the GPU.

[0104] Some of the voxels are folded up. If certain faces of the voxels in the same array are former inner faces that are only visible because the unused voxels are folded up, the GPU cannot ensure that colors, textures, or other attributes will be correctly assigned to these faces. Thus, the mapping coordinates of the voxels that are neighbors of the folded-up voxels are distorted 140 to allow correct color or texture assignment to all faces. This step is also performed by the GPU and is illustrated above with reference to Figure 6 and Figures 7a to 7b illustrated.

[0105] Steps 120, 130, and 140 have been performed by the GPU. Textures, colors, or other attributes can be assigned 150 (or sampled) to the voxels, and a scene with a view of the 3D model can be visualized 160 on the screen of the computer system. If the scene changes 170 (e.g., due to a movement of the viewing point or zooming in or out by the user), the steps of method 100 are repeated, starting from retrieving 110 the data of the new scene.

[0106] Preferably, all steps of the illustrated method 100 are performed fully automatically and in real time by a computer system (e.g., Figure 1 system 1 depicted in

[0107] Although the present invention has been illustrated above with reference to some preferred embodiments, it must be understood that many modifications and combinations of different features of these embodiments can be made. All of these modifications fall within the scope of the appended claims.

Claims

1. A computer-implemented method for visualizing a three-dimensional model of a structure from a plurality of different viewpoints in sequence on a display, the method comprising the steps of: - Providing geometric data and attribute information of the model in one or more data structures on a data storage device; And - For each of a large number of viewpoints, performing a rendering process for providing visualization of the model on the display, Characterized in that The rendering process includes: - Rendering a large number of similar elements in a single draw call, wherein the large number of similar elements includes a large number of three-dimensional arrays, each array including at least eight voxels; - Removing one or more voxels that are not to be visualized at least in a first array of the large number of three-dimensional arrays, wherein each voxel that is not to be visualized is folded by folding the vertices of the voxel to the zero point or the center of the first array; - Calculating a distortion on a mapping from voxel space to attribute volume space according to the one or more removed voxels; - Retrieving attribute values from the attribute volume space using the distorted values to assign attributes to the voxels of the large number of three-dimensional arrays, wherein the attributes are assigned to the remaining voxels according to the calculated distortion; and - Visualizing the voxels of the large number of three-dimensional arrays on the display.

2. The method according to claim 1, Among them, Rendering the large number of similar elements in a single draw call using hardware instantiation, wherein the step of rendering the large number of similar elements in a single draw call using hardware instantiation is fully automatically performed by a graphics processing unit of a computer system executing the method.

3. The method according to claim 1 or 2, Among them, The geometric data is organized as a sparse voxel octree, and the method further includes the steps of: traversing the octree to obtain a bitmask of at least a subset of the geometric data to be visualized based on an actual viewpoint, wherein, - The bitmask is an 8-bit bitmask, and each array in the large number of three-dimensional arrays includes eight cube voxels in a 2×2×2 configuration; and / or - Each bitmask is or includes a list that includes information about which child nodes of the nodes of the octree include valid data; and / or - The sparse voxel octree includes a hierarchical set of bitmasks indicating the presence of child nodes, and any associated attribute information; and / or - The step of removing the voxels is performed based on the bitmask.

4. The method according to claim 3, Among them, The first array corresponds to a node of the octree, and local coordinates stored in the node are used to assign attributes to the voxels in the first array.

5. The method according to claim 1 or 2, Among them, Each array in the large number of three-dimensional arrays represents data at a specific level of detail, wherein which arrays to include in the large number of rendered arrays is selected based on level of detail calculation.

6. The method according to claim 1 or 2, wherein, - Inner faces between voxels of the same array are not rendered; and - Attributes are assigned to the original inner faces of the remaining voxels, where the original inner faces are visible due to the removal of the voxels adjacent to the remaining voxels.

7. The method according to claim 6, wherein, Perform steps for calculating warping to avoid incorrect attribute assignment due to hardware sampling rules and attribute coordinate rounding operations.

8. The method according to claim 1 or 2, wherein, - The step of rendering the first array includes accessing a single static index buffer containing 48×3 indices; and / or - Each voxel is triangulated into a hexagon around the corners of the voxel; and / or - The vertices of the voxels form triangles on the surface of the voxel, and the step of folding vertices includes folding the triangles formed by these voxels; and / or - The step of folding vertices includes folding those vertices that are not shared with one or more voxels to be visualized.

9. The method according to claim 8, wherein The step of folding vertices includes folding only those vertices that are not shared with one or more voxels to be visualized.

10. The method according to claim 1 or 2, wherein, The step of rendering the first array includes: accessing a single static vertex buffer having at least 27 vertices, and each vertex is assigned an association list of those voxels of the first array that use the vertex.

11. The method according to claim 1 or 2, wherein, The geometric data is quantized into an axis-aligned regular three-dimensional grid.

12. The method according to claim 1 or 2, wherein, At least the following steps are fully automatically performed by a graphics processing unit of a computer system executing the method: rendering a large number of similar elements, removing voxels, and calculating warping in a single draw call.

13. The method according to claim 1 or 2, wherein, The geometric data is generated based on: - One or more point cloud scans of the structure, or - Rasterized polygon data related to the structure.

14. The method according to claim 1 or 2, wherein, - The geometric data and the attribute information are provided in separate data structures; and / or - The attribute information includes texture and / or color information.

15. The method according to claim 1 or 2, wherein Upon receiving an input from the user to change the actual viewing point to a new viewing point, the method includes the following steps: for the new viewing point, perform a rendering process for providing a visualization of the model on the display.

16. The method according to claim 15, wherein, The input from the user to change the actual viewing point to a new viewing point is received on an input device of a computer system executing the method.

17. The method according to claim 15, wherein, The method further includes traversing the data structure of the geometric data.

18. A computer system, the computer system includes a data storage device, a graphics processing unit, an input device, and a display, the graphics processing unit is configured to render a large number of elements in a single draw call, characterized in that the computer system is configured to execute the method according to any one of the preceding claims.

19. A computer program product, the computer program product comprising program code, the program code being stored on a machine-readable medium or being implemented by an electromagnetic wave comprising a program code segment, and the computer program product having computer-executable instructions for performing the method according to any one of claims 1 to 17.

Images