Texture transfer in mesh simplification
By converting high-resolution textured mesh into colored point clouds and projecting onto a simplified mesh surface, the complexity of textured data transfer and downsampling filtering in textured polygon mesh simplified by using high-quality resampling filters is solved. , realizes efficient texture data transmission and consistency filtering, suitable for rendering of multi-resolution content.
Patent Information
- Application Number
- CN202410857309.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-07-04
- Filing Date
- 2024-06-28
- Publication Date
- 2025-01-07
AI Technical Summary
The simplification and complexity of textured polygon mesh, especially while maintaining high-quality UV parameterization and avoiding artifacts, makes it difficult to achieve efficient texture data transmission and downsampling filtering.
Convert high-resolution textured mesh into colored point clouds and spatially sorted and projected to simplified mesh surfaces, filtered using high-quality resampling filters, working directly with source texels to avoid ray projection and parameterization Mapping.
A method of quickly transferring texture data from high-resolution mesh to independent parameterization simplified mesh is implemented, ensuring high-quality filtering and consistency, avoiding artifacts and excessive blur, suitable for distribution and rendering of multi-resolution content.
Smart Images
Figure CN119273827A_ABST
Abstract
Description
Technical Field
[0001] The present invention generally relates to the simplification of textured polygonal meshes (e.g., textured triangle meshes). The present invention proposes a simple and fast algorithm for transferring texture data from a high-resolution mesh to an independently parameterized simplified mesh. The method naturally allows high-quality filtering when downsampling the texture. Background Art
[0002] For polygonal meshes that do not include any attributes (such as textures), geometric simplification of meshes is a well-studied problem. For example, the Quadric Error edge-collapse simplification of Garland and Heckbert or the volume-preserving method of Lindstrom and Turk are widely used to decimate planar meshes to facilitate the distribution and rendering of multi-resolution 3D content.
[0003] However, simplification of textured meshes is more complex. It is still considered an important and active area of academic research. One of the difficulties is that even if a mesh represents a single manifold surface, its parameterization ("unwrapping") in texture space (using UV coordinates) may rarely be continuous, and often consists of multiple islands or "patches" mapped to the 3D surface. The surface then inevitably includes texture "seams" or UV discontinuities, which complicate the removal of polygons (e.g., triangles) in mesh simplification (which are vertex or edge collapsed).
[0004] In computer graphics, game development, and similar fields, it is common practice to assume high-quality UV parameterization, where the number of seams is minimized. Mesh simplification is then restricted to prevent collapsing geometry around UV seams, so that the parameterized topology is preserved and texture images can simply be scaled down after the mesh is simplified. Unfortunately, however, this restriction severely limits automatic mesh reduction.
[0005] In order to achieve the aggressive simplification required for distributing multi-resolution content (such as, large-scale urban photogrammetric reconstruction), the original UV parameterization cannot be kept. In other words, the simplified meshes need to be "unwrapped" independently and the texture atlas needs to be regenerated. In order to transfer color information, a high-quality 1:1 mapping between the parameterizations of the two meshes needs to be constructed, which is a very non-trivial task subject to state-of-the-art computer graphics research.
[0006] In photogrammetry, the mapping problem between different parameterizations can be avoided by computing the texture of each level of detail (LOD) mesh independently. Unfortunately, this approach suffers from artifacts when switching between the resulting textured LODs. With independent texturing, it is difficult to ensure that the projection sources and their blending weights are chosen exactly the same at every point on the surface.
[0007] Therefore, there is a need to transfer textures from the high-resolution textured mesh to the simplified mesh. In certain implementations utilizing more than two levels of detail, there is a need to transfer textures from the finest LOD to all coarser LODs while ensuring perfect consistency.
[0008] Furthermore, high quality filtering is also desirable when downsampling textures in order to avoid excessive blurring, moire, and other sampling artifacts.
[0009] In manual 3D modeling, 3D artists often use a method of "texture baking" to store pre-computed data such as high-resolution normals, ambient occlusion, etc. in dedicated textures. In theory, this technique can be used to transfer colors from a high-resolution mesh to an independently unwrapped simplified mesh. Ray casting from a target surface is usually used to sample the source surface, however this can be expensive and does not guarantee high quality filtering. The present invention proposes to work directly with source color samples and perform their projection to the target surface, rather than casting rays in the opposite direction. Summary of the invention
[0010] It is therefore an object of the present invention to provide an improved method for simplifying textured polygonal meshes.
[0011] In particular, it is an object to provide a method that provides a simple and fast solution for transferring texture data from a high-resolution mesh to one or more independently parameterized simplified meshes while applying a high-quality downsampling filter.
[0012] A first aspect of the invention relates to a computer-implemented method for simplifying a textured polygonal mesh of a three-dimensional model (e.g., a model of an environment), the method comprising:
[0013] - obtaining 3D mesh data and texture data associated with a first textured polygonal mesh, the first textured polygonal mesh having a first level of detail, the texture data comprising a plurality of texels;
[0014] - generating a simplified polygonal mesh for a second level of detail based on the 3D mesh data, the second level of detail being lower than the first level of detail;
[0015] - generating a simplified texture for simplifying the polygonal mesh based on the texture data; and
[0016] - Providing a simplified textured polygonal mesh based on the generated simplified polygonal mesh and the generated simplified texture.
[0017] According to this aspect of the invention, generating a simplified texture comprises:
[0018] - converting the first textured polygonal mesh into a colored point cloud, wherein each point of the point cloud corresponds to a texel;
[0019] - Spatial sorting of points in the point cloud;
[0020] -projecting the points of the point cloud onto the surface of the simplified polygonal mesh; and
[0021] - Filter the projected points using the resample filter.
[0022] According to some embodiments of the method, converting the first textured polygonal mesh into a colored point cloud comprises: for each texel corresponding to a surface point on a surface of the first textured polygonal mesh:
[0023] - Calculate the world coordinates of the surface point; and
[0024] - Add point-color pairs to the array.
[0025] According to some embodiments of the method, spatially sorting the points comprises:
[0026] - Use the kd sorting algorithm to achieve the sorting of the set of points implied by the k-dimensional tree; and
[0027] - Reorder points in-place.
[0028] In some implementations, a fast selection partial sort algorithm is used to find the median at each level of the k-dimensional tree.
[0029] According to some embodiments of the method, projecting the point includes: for each texel of the simplified mesh and the corresponding world position:
[0030] - find all nearby points in the colored point cloud that are within a given radius around the corresponding world position, and
[0031] - Projecting the found point along the normal vector of the plane of the surface of the simplified mesh containing the texels of the simplified mesh to said plane.
[0032] The resampling filter is preferably a high quality filter. According to some embodiments of the method, the resampling filter is a cubic filter.
[0033] According to some embodiments of the method, the texture information includes color information, and the texture includes color. In some embodiments, the color information is at least 8-bit (eg, 16-bit, 32-bit, etc.) RGBA color information.
[0034] According to some implementations, the method includes receiving user input defining a lower level of detail.
[0035] According to some embodiments of the method, at least the steps of generating a simplified polygonal mesh, generating a simplified texture and providing a simplified textured polygonal mesh are performed iteratively for a plurality of different levels of detail.
[0036] According to some embodiments of the method, the simplified texture comprises fewer texels than texture data associated with the first textured polygonal mesh.
[0037] According to some embodiments of the method, generating a simplified polygonal mesh includes using mesh reduction or mesh simplification (ie, a mesh reduction or mesh simplification algorithm). In one embodiment, quadratic error edge collapse simplification is used. In another embodiment, a volume preservation method is used.
[0038] According to some embodiments of the method, the first textured polygonal mesh is a first textured triangle mesh and the simplified polygonal mesh is a simplified triangle mesh.
[0039] A second aspect of the invention relates to a computer system comprising one or more processors, a data memory, a graphics processing unit (GPU), an input device and a display screen, wherein the computer system is configured to perform a method according to any one of the preceding claims.
[0040] A third aspect of the invention relates to a computer program product comprising a program code stored on a machine-readable medium or embodied by an electromagnetic wave comprising program code segments, and having computer executable instructions for executing the method according to the first aspect of the invention, in particular when running on a computer system according to the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The present invention is described in detail below by way of exemplary embodiments with reference to the accompanying drawings, in which:
[0042] Figure 1a The source model with the level of detail is shown;
[0043] Figure 1b A simplified model with the texture resolution reduced to half is shown;
[0044] Figure 2An exemplary embodiment of a computer system according to the present invention is shown;
[0045] Figure 3 shows a flowchart illustrating an exemplary embodiment of a computer-implemented method according to the present invention; and
[0046] Figure 4 A further example is shown Figure 3 Flowchart of the steps of the method for generating a simplified texture. DETAILED DESCRIPTION
[0047] Figure 1a and Figure 1b A 3D model 2, 2' of an urban environment (outline) is shown as an example of a textured triangular mesh. Figure 1a shows a model with an original high level of detail (LOD) ("source model" 2), and Figure 1b The same model with a lower LOD is shown ("simplified model" 2'). The simplified model 2' is generated using the method according to the invention by simplifying both the geometry and the texture of the source model 2. The simplified model 2' has a texture resolution that is reduced by 50% relative to the source model 2. In other words, the texel size is doubled.
[0048] The present invention proposes a simple and fast algorithm to transfer texture data from a high-resolution mesh to an independently parameterized simplified mesh. The method naturally allows high-quality filtering when downsampling the texture. The proposed algorithm is partially inspired by texture baking, but does not require ray casting and works directly with source texels, much like an image resampling filter. The method does not require explicit construction of a 1:1 mapping between parameterizations.
[0049] Figure 2An exemplary computer system 1 according to the present invention is shown. The system 1 is configured to perform an embodiment of the method according to the present invention described further below. The computer system 1 may include a motherboard 10, which includes circuits for powering and interfacing at least one onboard processor (e.g., a central processing unit (CPU), not shown here), which may include two or more discrete processors or a processor with multiple processing cores. The motherboard 10 may act as an interface between a microprocessor of the computer system 1 and a memory device 20. The memory device 20 may include one or more optical, magnetic, or solid-state drives, and may store instructions for an operating system and other software for the computer system 1. The motherboard 10 may also communicate with a random access memory (RAM) and a read-only memory (ROM) of the computer system 1, which typically stores instructions for a basic input output system (BIOS), which the microprocessor can access and which prepares the microprocessor for loading an operating system. The motherboard 10 also interfaces with a graphics processor unit (GPU) 15. In some systems, the GPU 15 may be integrated into the motherboard 10. The motherboard 10 may be connected to a communication adapter 13, such as a LAN adapter or a modem, which places the computer system 1 in data communication with a computer network such as the Internet. A user of the computer system 1 interfaces with it by means of one or more input devices 11, such as a keyboard or mouse, and one or more display devices 12.
[0050] It should be understood that the computer system 1 comprises an exemplary electronic processor-based system for performing the method. However, the method may also be performed using other electronic processor-based systems. Such systems may include tablet computers, laptops and netbook computing devices, cellular smart phones, game consoles, and other imaging devices, such as medical imaging devices.
[0051] A user of the system 1 may operate an operating system to load a computer graphics-related software product, which may be provided by downloading from the Internet or as tangible instructions carried on a computer-readable medium such as a compact disk. The computer graphics-related software product includes a data structure storing data including at least geometric data 22 and texture data 24 (or other attribute data). The texture data 24 may be provided in a different data structure than the geometric data 22, such as as a raster image. The display device 12 is configured to display, for example, a 3D model 2 of an environment, such as Figure 1a and Figure 1b1. The 3D model 2 includes a textured triangular mesh generated from the geometric data 22 and the texture data 24. The software product also includes instructions for the processor or CPU of the system 1 to manipulate the 3D model 2 in response to input received from the user via the input device 11. This manipulation of the 3D model 2 includes at least simplification of the textured triangular mesh to provide different levels of detail (LOD) for the 3D model 2. This preparation of data can be performed in an offline process (e.g., on a CPU). The GPU 15 can be used to display the mesh to the user at the desired LOD.
[0052] Figure 3 A flow chart illustrating an exemplary embodiment of a method 100 according to the present invention is shown. The method starts by obtaining 110 a textured triangle mesh (or any other textured polygonal mesh), i.e., a source mesh to be simplified. The mesh comprises geometric data 22 (i.e., a non-textured triangle (polygonal) mesh) and texture data 24 (i.e., a plurality of texels).
[0053] Based on the geometric data 22, a simplified mesh, i.e. a mesh having fewer triangles (or other polygons) than the source mesh, is generated 130. This step can be performed using volume preserving methods and other processes known per se to those skilled in the art, such as the above-mentioned quadratic error edge collapse simplification of Garland and Heckbert.
[0054] Based on the texture data 24 and also based on the geometry data 22, a simplified texture is generated 150 that fits the simplified mesh. Figure 4 This step is further described.
[0055] After both generating the simplified mesh and fitting the simplified texture, the simplified texture is applied to the simplified mesh to provide a simplified textured triangular mesh 170. The simplified textured triangular mesh may be provided as a separate data file or added to a data file including the source mesh. The method 100 may be repeated for a plurality of different LODs, thereby providing a plurality of simplified textured triangular meshes.
[0056] Figure 4 A further example generating 150 is shown Figure 3 100 is a flowchart of the step of simplifying the texture of the method 100. In an exemplary method according to the present invention, this step includes the step of simplifying the texture by, for example, running Figure 2 The algorithm on the computer system shown performs the following four sub-steps:
[0057] - Convert the source mesh to a colored point cloud (152);
[0058] - spatially sorting the points of the point cloud (154);
[0059] - projecting the points onto the surface of the simplified mesh (156); and
[0060] - Filter the points (158).
[0061] The step of converting 152 the source mesh into a colored point cloud includes, for each texture pixel (texel) of the source texture atlas corresponding to a point on the surface of the mesh, calculating the world coordinates of the surface point and adding the point-color pair to a temporary array in RAM, effectively converting the textured mesh into a colored point cloud in which each point corresponds to a texel.
[0062] In some cases, the source mesh may contain polygons that are not covered by a sufficient density of texture samples. For these areas, artificial texture samples are generated that are interpolated between the actual texels in order to avoid resampling artifacts in the target texture. The density of the artificial samples is determined by the classic Nyquist theorem.
[0063] If single precision floating point numbers are used to represent (normalized) world coordinates, and 8-bit RGBA colors are added, then a point-color pair requires 16 bytes of RAM. Optionally, a 32-bit source face ID may be included to allow identification of the original face normal, so that a point record will total 20 bytes.
[0064] In the step of spatial sorting 154 of points, the spatial sorting of the points of the colored point cloud is performed, which realizes fast nearest neighbor query and distance query. For example, the "kd-sorting" algorithm can be used to perform sorting, resulting in the order implied by the k-dimensional tree (kd tree) on the point set, but in fact the tree is not explicitly constructed. The points are reordered in situ without the need for additional allocation. In some embodiments, the "Quick Select" partial sorting algorithm can be used to find the median at each "tree" level. In this case, the whole process will have a complexity of O (n log n). Finding the nearest neighbor in the implicit tree or finding all points within the radius of the point will have an average complexity of O (log n). Efficient kd-sorting is essential for the success and practicality of the method. The applicant's experiment shows that it takes about two seconds to perform kd-sorting on 16-core processors in parallel to 100 million points.
[0065] The step of projecting 156 the points includes: for each texel of the simplified mesh ("simple texel") and the corresponding world position, finding all nearby points ("source points") of the colored point cloud within a given radius ("search radius"). The search radius depends on the expected maximum geometric error introduced by the independent meshing and simplification processes. It can be user-selected or automatically defined. The found source points are then projected along the plane normal to the plane of the polygons (e.g., triangles) of the simplified mesh ("simple face") containing the simple texel.
[0066] In order to avoid excessive blurring, moire and other sampling artifacts, high quality filtering is desired when downsampling the texture. Therefore, in the plane, the projected samples are filtered 158 using a cubic filter with a cutoff period corresponding to the sampling distance of the simple grid, that is, the radius of the cubic kernel is chosen to be twice the distance between adjacent target texels according to the Nyquist theorem. A similar high quality resampling filter can be used instead of the cubic filter. Then, a weighted average of the source samples is calculated, where the weights are based on the cubic kernel and the radial distance of the projected samples from the center of the kernel (the location of the target texel). Therefore, this step includes performing appropriate 2D filtering (downsampling) on the original texture.
[0067] In rare cases where, at some point on a simple surface, no (or not enough) source texels can be found in the given search radius, or if they mostly fall into the negative lobe of the cubic kernel, the nearest source texel outside the given search radius is found (however, it may be far away), and only its color is copied. In addition to weighting the projected samples by the reconstruction inner kernel, the samples are multiplied by max(dot(n s , n t , 0), where n s and n t are the source normal and the target face normal, respectively. Advantageously, this step prevents color bleeding across sharp edges and from backface geometry. To reduce color discontinuities on sharp edges, the method may optionally include projecting the source point along a normal that is smoothly interpolated on a simple mesh face.
[0068] At the top level, in a photogrammetric setting, the finest LOD mesh (i.e., the input mesh) is textured in the same way as before by projecting and blending the input image onto the input mesh. However, for all coarser levels of detail, the above methods and algorithms are used to transfer the texture from the nearest higher LOD and downsample it to the current resolution. In this way, all LODs can be obtained, for example, so that each coarse LOD has half the texture resolution of the previous higher LOD. This means that in embodiments with more than two levels of detail (n levels of detail), the methods and algorithms according to the present invention can be iteratively used to first transfer the texture from the finest LOD mesh (LOD-0) to the nearest coarser mesh (LOD-1), then from LOD-1 to LOD-2, etc., until the coarsest LOD (LOD-n-1) is textured based on the second coarsest mesh (LOD-n-2).
[0069] Although the present invention has been described above with reference to some preferred embodiments, it must be understood that many modifications and combinations of different features of the embodiments are possible. All these modifications are within the scope of the appended claims.
Claims
1. A computer-implemented method (100) for simplifying a textured polygonal mesh of a three-dimensional model, the method comprising: Obtaining (110) 3D mesh data (22) and texture data (24) associated with a first textured polygonal mesh (2), the first textured polygonal mesh (2) having a first level of detail, the texture data (24) comprising a plurality of texels; generating (130) a simplified polygonal mesh for a second level of detail based on the 3D mesh data (22), the second level of detail being lower than the first level of detail; generating (150) a simplified texture for the simplified polygonal mesh based on the texture data (24); as well as providing (170) a simplified textured polygonal mesh (2') based on the generated simplified polygonal mesh and the generated simplified texture, Features Generating (150) the simplified texture comprises: converting (152) the first textured polygonal mesh (2) into a colored point cloud, wherein each point of the point cloud corresponds to a texel; spatially ordering the points of the point cloud (154); projecting (156) the points of the point cloud onto a surface of the simplified polygonal mesh; and The projected points are filtered using a resampling filter (158).
2. The method (100) according to claim 1, wherein: Converting (152) the first textured polygonal mesh (2) into a colored point cloud comprises: for each texel corresponding to a surface point on a surface of the first textured polygonal mesh (2), Calculate the world coordinates of the surface point, and Add point-color pairs to the array.
3. The method (100) according to claim 1 or claim 2, wherein: Spatially ordering the points (154) includes Using the kd-sort algorithm to sort the set of points implied by the k-dimensional tree; and Reorder the points in-place.
4. The method (100) according to claim 3, wherein: The median value at each level of the k-dimensional tree is found using the fast selection partial sorting algorithm.
5. The method (100) according to any one of the preceding claims, wherein: Projecting the point (156) includes: for each texel of the simplified grid and the corresponding world position, find all nearby points in the colored point cloud that are within a given radius around the corresponding world position, and The found point is projected onto a plane along a normal vector of the surface of the simplified mesh containing texels of the simplified mesh.
6. The method (100) according to any one of the preceding claims, wherein: The resampling filter is a cubic filter.
7. The method (100) according to any one of the preceding claims, wherein: The texture information (24) includes color information, and the texture includes color.
8. The method (100) according to claim 6, wherein: The color information is at least 8-bit RGBA color information.
9. The method (100) according to any one of the preceding claims, comprising: Receives user input defining a lower level of detail.
10. The method (100) according to any one of the preceding claims, wherein: At least the steps of generating (130) the simplified polygonal mesh, generating (150) the simplified texture and providing (170) the simplified textured polygonal mesh (2') are performed iteratively for a plurality of different levels of detail.
11. The method (100) according to any one of the preceding claims, wherein: The simplified texture includes fewer texels than the texture data (24) associated with the first textured polygonal mesh (2).
12. The method (100) according to any one of the preceding claims, wherein: Generating (130) the simplified polygonal mesh includes using mesh reduction or mesh simplification, in particular quadratic error edge collapse simplification or volume preservation methods.
13. The method (100) according to any one of the preceding claims, wherein: The first textured polygonal mesh (2) is a first textured triangular mesh and the simplified polygonal mesh is a simplified triangular mesh.
14. A computer system (1), comprising one or more processors, a data storage device (20), a graphics processing unit (15), an input device (11) and a display screen (12), Features The computer system (1) is configured to perform a method (100) according to any one of the preceding claims.
15. A computer program product, comprising a program code, which is stored on a machine-readable medium or embodied by an electromagnetic wave comprising program code segments, and which has computer-executable instructions for executing the method according to any one of claims 1 to 13, in particular when running on a computer system according to claim 14.