Orthogonal atlas: Texture Mapping Generation Using Orthogonal Projection For Dynamic Grid

Abstract

Generation of texture mappings using orthogonal projections is performed in a fast and efficient manner. Described herein is a method of generating a texture map that spends significantly less time and also allows the mapping to exploit temporal dependencies between the content of different frames. Texture mapping may be used for automatic generation of volumetric content, or may be used for more efficient compression of dynamic grids. Texture map generation described herein includes a way to generate a texture atlas using orthogonal projections. A new stretch metric for orthogonal projection is described and a merge algorithm is designed to optimally cluster triangles into single image blocks. In addition, packaging techniques that attempt to optimize size and temporal stability may be used for grid image tiles.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Patent Application Serial No. 63 / 378,565, filed on October 6, 2022, entitled “ORTHOATLAS: TEXTURE MAP GENERATION FOR DYNAMIC MESHES USING ORTHOGRAPHIC PROJECTIONS,” which is incorporated herein by reference in its entirety for all purposes. Technical Field

[0003] The present invention relates to three-dimensional graphics. More particularly, the present invention relates to generating texture maps using orthographic projections. Background Art

[0004] Recently, new methods for compressing volumetric content such as point clouds based on projection from 3D to 2D are being standardized. This method, also called V3C (Visual Volumetric Video-based Compression), maps the 3D volumetric data into several 2D image patches, which are then further arranged into atlas images, which are subsequently encoded with a video encoder. The atlas images correspond to the geometry of the points, the corresponding textures, and an occupancy map that indicates which of the positions are to be considered for point cloud reconstruction.

[0005] In 2017, MPEG published a Call for Proposals (CfP) for the compression of point clouds. After evaluating several proposals, MPEG is currently considering two different techniques for point cloud compression: 3D native coding techniques (based on octrees and similar coding methods), or 3D to 2D projection followed by traditional video coding. In the case of dynamic 3D scenes, MPEG is using Test Model Software (TMC2) based on patch surface modeling, projection of patches from 3D images to 2D images and encoding of 2D images with a video encoder such as HEVC. This approach has been shown to be more efficient than native 3D coding and to achieve competitive bitrates with acceptable quality.

[0006] Due to the success of projection-based methods (also called video-based methods or V-PCC) for encoding 3D point clouds, it is expected that the standard will include more 3D data, such as 3D meshes, in future versions. However, the current version of the standard is only suitable for the transmission of a set of disconnected points, so there is no mechanism to send the connectivity of the points, which is required in 3D mesh compression.

[0007] Methods to extend the functionality of V-PCC to meshes have also been proposed. One possible approach is to encode the vertices using V-PCC and then encode the connectivity using a mesh compression method such as TFAN or Edgebreaker. A limitation of this approach is that the original mesh must be dense so that the point cloud generated from the vertices is not sparse and can be efficiently encoded after projection. In addition, the order of the vertices affects the encoding of the connectivity, so different methods of reorganizing the connectivity of the mesh have been proposed. An alternative way to encode sparse meshes is to encode the vertex positions in 3D using RAW image patch data. Since RAW image patches directly encode (x, y, z), in this approach all vertices are encoded as RAW data, while connectivity is encoded using a similar mesh compression method as described above. Note that in the RAW image patches, the vertices can be sent in any preferred order, so the order resulting from the connectivity encoding can be used. This method can encode sparse point clouds, however, RAW image patches are not efficient for encoding 3D data, and more data such as the attributes of triangular faces may be lost from this approach.

[0008] UVAtlas from Microsoft is the most advanced automatic texture map generation, but it takes a lot of time and is only optimized for local frames. V-PCC uses orthogonal projection to generate image patches, but only for point clouds, so it does not solve image patch generation on meshes. Summary of the invention

[0009] The generation of texture maps using orthogonal projections is performed in a fast and efficient manner. A method of generating texture maps is described herein that takes significantly less time and also allows the mapping to exploit temporal correlations between the content of different frames. Texture maps can be used for automatic generation of volumetric content, or can be used for more efficient compression of dynamic meshes. The texture map generation described herein includes a method for generating a texture atlas using orthogonal projections. A new stretch metric for orthogonal projections is described, and a merging algorithm is designed to optimally cluster triangles into a single image patch. In addition, packing techniques that attempt to optimize size and temporal stability can be used for mesh image patches.

[0010] In one aspect, a method programmed in a non-temporary memory of a device includes generating an image block from dynamic mesh information, and packing the image block on a texture atlas using orthogonal projection. Generating the image block from the dynamic mesh information also includes: generating a list of neighboring triangles, calculating triangle properties, adding adjacent triangles, checking vertex occlusion, and checking surface occlusion. Generating a list of neighboring triangles includes adding triangles that share vertices with the triangle to the list. Generating a list of neighboring triangles includes adding triangles that share edges with the triangle to the list. Calculating triangle properties includes calculating normals and surface areas for each triangle. The method also includes selecting a seed triangle whose normal is most consistent with the most frequent orientation that has not yet been added to the image block. The method also includes merging the image blocks based on a calculated cost, wherein the calculated cost is based on the perimeter of the image block and an orthogonal stretch value. Packing the image blocks on the texture atlas using orthogonal projection includes implementing frame scaling, image block orientation, and temporal stabilization.

[0011] In another aspect, a device includes a non-temporary memory for storing an application and a processor coupled to the memory, the application being used to: generate image blocks from dynamic mesh information, and pack the image blocks on a texture atlas using orthogonal projection, the processor being configured to process the application. Generating image blocks from dynamic mesh information also includes: generating a list of neighboring triangles, calculating triangle properties, adding adjacent triangles, checking vertex occlusion, and checking surface occlusion. Generating a list of neighboring triangles includes adding triangles that share vertices with the triangle to the list. Generating a list of neighboring triangles includes adding triangles that share edges with the triangle to the list. Calculating triangle properties includes calculating normals and surface areas for each triangle. The application is also configured to select a seed triangle whose normal is most consistent with the most frequent orientation that has not yet been added to the image block. The application is also configured to merge the image blocks based on a calculated cost, wherein the calculated cost is based on the perimeter of the image block and an orthogonal stretch value. Packing the image blocks on the texture atlas using orthogonal projection includes implementing frame scaling, image block orientation, and temporal stabilization.

[0012] In another aspect, a system includes one or more cameras and an encoder for acquiring three-dimensional content, the encoder being configured to: generate image blocks from dynamic mesh information, and pack the image blocks on a texture atlas using orthogonal projection. Generating image blocks from dynamic mesh information also includes: generating a list of neighboring triangles, calculating triangle properties, adding adjacent triangles, checking vertex occlusion, and checking surface occlusion. Generating a list of neighboring triangles includes adding triangles that share vertices with the triangle to the list. Generating a list of neighboring triangles includes adding triangles that share edges with the triangle to the list. Calculating triangle properties includes calculating normals and surface areas for each triangle. The encoder is also configured to select a seed triangle whose normal is most consistent with the most frequent orientation that has not yet been added to the image block. The encoder is also configured to merge the image blocks based on a calculated cost, wherein the calculated cost is based on the perimeter of the image block and an orthogonal stretch value. Packing the image blocks on the texture atlas using orthogonal projection includes implementing frame scaling, image block orientation, and temporal stabilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 Diagram illustrating texture parameterization in accordance with some embodiments.

[0014] Figure 2 A diagram illustrating neighboring triangle analysis in accordance with some embodiments.

[0015] Figure 3 A diagram illustrating calculations of triangle properties in accordance with some embodiments.

[0016] Figure 4 A diagram illustrating selecting a seed in accordance with some embodiments.

[0017] Figure 5 Diagrams showing different groups of connected components in accordance with some embodiments.

[0018] Figure 6 A diagram illustrating orthogonal stretching according to some embodiments.

[0019] Figure 7 Illustrate the results of orthogonal stretching according to some embodiments.

[0020] Figure 8 A diagram illustrating packaging in accordance with some embodiments.

[0021] Fig. 9 An image illustrating frame scaling in accordance with some embodiments.

[0022] Fig.10 An image illustrating an orientation of image patches in accordance with some embodiments.

[0023] Fig.11Illustrate a temporally stabilized image in accordance with some embodiments.

[0024] Fig.12 A block diagram illustrating an exemplary computing device configured to implement a texture map generation method using orthographic projection in accordance with some embodiments. DETAILED DESCRIPTION

[0025] A mesh consists of a set of polygons that usually describe the surface of a volume. An efficient way to describe the surface properties of a mesh (e.g., its color characteristics) is to generate a texture atlas that maps the properties of a 3D surface to a 2D surface. However, mapping a 3D surface to a 2D surface is a non-trivial problem, and state-of-the-art methods such as Microsoft's UV atlas resort to time-consuming optimization to find suitable segmentation in the mesh and map it using a face reconciliation function. In the latest international point cloud compression standard, orthogonal projection is used to generate texture mapping images for point clouds. Although texture mapping images are easy to generate, texture mapping images are only used for point clouds, so they do not consider the connected structure present in the mesh. The method described in this article is capable of generating texture images for a mesh, and the method uses orthogonal projections similar to those used in the V-PCC standard. Texture mapping can be used for the automatic generation of volume content, or it can be used for more efficient compression of dynamic meshes.

[0026] The texture map generation described in this paper includes a way to generate a texture atlas using orthographic projection. A new stretch metric for orthographic projection is described, and a merging algorithm is designed to optimally cluster triangles into a single image patch. Additionally, a packing technique that attempts to optimize size and temporal stability is applied to mesh image patches.

[0027] Texture map generation using orthogonal projection includes patch generation and patch packing. Patch generation includes seed selection using connected components (e.g., triangles sharing edges / vertices); orthogonal stretching: a projection distortion measure for orthogonal projection; and a rate-distortion based merging algorithm using orthogonal stretching and patch perimeter. Patch packing includes: frame / patch scaling, patch rotation, and temporal alignment.

[0028] Figure 1 A diagram illustrating texture parameterization according to some embodiments. In step 100, image blocks are generated. After the image blocks are generated, the image blocks are packed in step 102. For example, the image blocks are packed on a texture atlas.

[0029] Generating the image patch includes generating connected components in step 110 and performing projection in step 130 .

[0030] Generating connected components includes: in step 112, generating a list of neighboring triangles; in step 114, for each triangle, calculating triangle properties; in step 116, selecting seeds (triangles); in step 118, adding neighbors from the neighboring triangles according to a criterion; after looking at all neighboring triangles, if there are any triangles that cannot be added to the image patch, then in step 116, restarting the process with different seeds (triangles); and when there are no remaining triangles, in step 120, merging the image patches to generate a large surface of connected triangles. Performing a projection includes the following steps: in step 132, checking for vertex occlusion, and in step 134, checking for surface occlusion. If there is any vertex occlusion or surface occlusion, remove the triangle from the image patch and include it in a different image patch.

[0031] Figure 2 A diagram illustrating neighboring triangle analysis in accordance with some embodiments. Triangle 200 has many neighboring and adjacent triangles. Triangle neighborhoods can be defined by sharing a single vertex (params.useVertexCriteria=true) or sharing edges (params.useVertexCritieria=false). Neighboring triangles are triangles that share at least one vertex with a given triangle. For example, triangles 1-15 {T1,...,T 15} all share vertices with triangle 200. Adjacent triangles share edges with triangle 200. For example, triangles 4, 10, and 14 {T4, T 10 and T 14} shares an edge with triangle 200. A list of neighboring triangles and adjacent triangles may be maintained for each triangle. A flag may also be included to preserve boundaries.

[0032] Figure 3 A diagram illustrating the calculation of triangle properties according to some embodiments. For each triangle, some properties are calculated (e.g., normal and surface area in 3D). The surface area and normal of the triangle can be derived from the projected area.

[0033] Figure 4 Illustrate a diagram of selecting a seed in accordance with some embodiments. The most frequent orientation is obtained by examining the class histogram of the triangle. If params.bUseSeedHistogram = true, the seed is the triangle whose normal is most consistent with the most frequent orientation and has not yet been added to a connected component. If params.bUseSeedHostogram = false, the seed is the first available triangle (e.g., not already in a connected component).

[0034] To add a neighboring triangle to the list, the following criteria are checked: whether the category of the triangle is the same as the orientation of the connected component; whether the angle between the last inserted normal and the current normal is less than a certain threshold; whether the area of ​​the image patch is less than a fraction of the total area; and whether the number of triangles is less than a threshold. In some embodiments, the neighboring triangle is added to the list only if all of the above criteria are true.

[0035] Figure 5 Diagrams showing different groups of connected components in accordance with some embodiments.

[0036] The merge function merges adjacent connected components according to cost, based on the perimeter of the connected components and orthogonal stretch (e.g., stretching caused by orthogonal projection). The merge algorithm begins by generating an ordered list of connected components. The list is sorted by 1) the smallest number of triangles and 2) the most consistent average normal (weighted by triangle area) with the orientation. Each connected component starts with the following cost: COST(M i )=L 2 (M i )+λPERIMETER(M i ), where M i are connected components; L is the rate distortion; and λ is a parameter that provides a weight between rate distortion and girth. If λ is 0, girth is not considered in the cost, and if λ is a very high value, only girth is considered in the cost and rate distortion is insignificant. Further illustrating this example, if few bits are available, distortion is not that important and a very large value for λ is used. If quality is very important, λ is 0 or very small and distortion is minimized in order to minimize cost.

[0037] When the list is not empty, do the following:

[0038] Remove the top element of the list (M i ));

[0039] Get a list of candidate neighbors ( in It is with M i List of connected components that share edges / vertices);

[0040] For each adjacent candidate:

[0041] Calculate the merger cost: COST(M i +M j )=L 2 (M i +M j )+λPERIMETER(M i );

[0042] If a merge operation with one of its neighbors (connected components that share at least one edge) reduces the cost: (COST(M i +M j ) <COST(M i )+COST(M j ), then choose the neighbor that minimizes the joint cost:

[0043] If a neighbor is selected for merging, the connected component (M i →M j ) and rearrange the list.

[0044] Figure 6 The diagram illustrates orthogonal stretching according to some embodiments. Described here is a function (e.g., a projection function) that maps points in 3D space to 2D space. What connects points in 3D space to points in 2D space is barycentric coordinate interpolation. Barycentric coordinate interpolation involves using the area of ​​the projected triangle to generate weights, and then using the weights to derive any point on the surface. This can be done in any direction (e.g., x, y, z). For example, by connecting vertices to point p, triangles can be generated within triangle 600 (and other triangles), and then the value of point p in 3D space can be obtained by interpolating the areas of these triangles in the projected space. Using points q1, q2, q3 at the vertices, by weighting them using the area of ​​the projected triangle, the value q can be obtained. By solving the following equation, the ratio between the areas is obtained to regenerate points in 3D space based on the projection in 2D space.

[0045] S XY (p) = q

[0046]

[0047] in:

[0048] <p,p2,p3> =[(x2-x)(y3-y)-(x3-x)(y2-y)] / 2

[0049] <p,p3,p2> =[(x3-x)(y2-y)-(x2-x)(y3-y)] / 2

[0050] <p,p1,p2> =[(x1-x)(y2-y)-(x2-x)(y1-y)] / 2

[0051]

[0052] In view of the above, the Jacobian matrix of the function can be obtained. By decomposing the Jacobian matrix into eigenvalues ​​and eigenvectors, the stretch (e.g., the reduction in size of a triangle from 3D space to 2D space) can be determined. The stretch is the ratio between the area in 3D space and the area in 2D space. In some embodiments, the area is pre-calculated so that the stretch can be calculated preemptively or quickly.

[0053]

[0054] Largest (Γ) and smallest (Y) singular values:

[0055]

[0056] in

[0057]

[0058] Triangle stretch L 2 XY , L 2 XZ , L 2 ZY

[0059]

[0060] Grid L 2 Stretch:

[0061]

[0062] in or or

[0063] Figure 7 The diagram illustrates the results of orthogonal stretching according to some embodiments. In the original mesh 700, there are 2351 connected components (e.g., the original triangles), the perimeter is 186071, and the stretch is 1.0978. In mesh 702, clustering with neighbors is performed (no additional analysis is required), so there are now 251 connected components, and the perimeter is reduced to 50721.7; however, the stretch is slightly increased to 1.14037. In mesh 704, merging is also achieved, so there are 55 connected components, the perimeter is 28935.2, but the stretch is 1.6506. In the case of 55 connected components, much fewer bits are required. In mesh 706, projection analysis is performed (e.g., to avoid overlapping of connected components), which results in 68 connected components, the perimeter is 31455.1 and the stretch is 1.69315. Through projection, texture coordinates can be determined based on the projection information.

[0064] Figure 8 The diagram illustrates a schematic diagram of packaging according to some embodiments. After generating the image blocks (e.g., connected components), the image blocks are placed on the 2D images in the video, and then the video is encoded. The image blocks are scaled (e.g., frame scaled) to fit the surface. For each image block, image block scaling is implemented.

[0065] Fig. 9 An image illustrating frame scaling according to some embodiments. The initial scaling is used to generate a projection that occupies a large portion of the texture mapping surface. For example, for a 2K×2K surface, the image blocks are adjusted to occupy the surface in the most efficient way possible. The size of the image blocks is maximized to occupy the entire surface. If packing is unsuccessful, the scaling will be reduced by 10% (or other value) to provide more space for connected components of the projection.

[0066]

[0067] Where adjustment is the scaling value (e.g., 10%), BB is the size of the bounding box 900, height is the height of the texture map 902, and occRes is the occupied resolution. The occupied resolution is the resolution of 16×16 (or other sized) blocks, not the pixel-by-pixel resolution.

[0068] In the example image, BB Y is the maximum size, so the texture map height is set to be close to or equal to the BB Y However, sometimes the image block is too large to fit into the texture map. When the image block is too large, each image block is scaled down by 10%. If the scaled image block still does not fit into the texture map, the image block is scaled down by another 10%, and the process repeats until the image block fits into the texture map.

[0069] For each image patch, the patch scale, orientation and position can be adjusted.

[0070] In addition to frame scaling, image patch scaling can also be performed to adjust the texture size on a per-image patch basis. Image patch scaling enables specific areas with more details (e.g., faces) to use more texture space. In an exemplary implementation, if params.bPatchScaling = true, the initial size of the image patch is doubled (or increased by another amount). If the image patch cannot be packed, its size is reduced by 10%. If the total size reduction exceeds 50% of the image patch size, the packing is considered to have failed and a new frame scaling is calculated.

[0071] Fig.10The images of the tile orientation according to some embodiments are illustrated. Similar to V-PCC, the tile can be rotated in the texture atlas domain to obtain a better fit. However, in some embodiments, in order to maintain the wrapping of the triangle, only 0°, 90°, 180° and 270° rotations are allowed. For example, image 1000 shows an unrotated tile in the texture map, and image 1002 shows a rotated tile.

[0072] Fig.11 Illustrate temporally stabilized images in accordance with some embodiments. Matching image patches are determined by examining the projection direction and the ratio of the volume of the intersection of the two image patches in 3D to the volume of the union of the two image patches in 3D. The matching image patches will have the same position (U0, V0), the same orientation (O), and the same scale (S) as the matching image patches. Image 1100 shows a texture map without temporal stabilization, while image 1102 shows a texture map with temporal stabilization.

[0073] Texture mapping coordinates can be derived by applying the following transformation:

[0074]

[0075] in:

[0076]

[0077] in:

[0078]

[0079] Fig.12The block diagram of an exemplary computing device configured to implement a texture mapping generation method using orthogonal projection according to some embodiments is illustrated. The computing device 1200 can be used to obtain, store, calculate, process, transmit and / or display information such as images and videos including 3D content. The computing device 1200 can implement any aspect of encoding / decoding. Generally, a hardware structure suitable for implementing the computing device 1200 includes a network interface 1202, a memory 1204, a processor 1206, one or more I / O devices 1208, a bus 1210 and a storage device 1212. The choice of processor is not important, as long as a suitable processor with sufficient speed is selected. The memory 1204 can be any conventional computer memory known in the art. The storage device 1212 can include a hard drive, CDROM, CDRW, DVD, DVDRW, high-definition optical disk / drive, ultra-high-definition drive, flash memory card or any other storage device. The computing device 1200 may include one or more network interfaces 1202. Examples of network interfaces include network cards connected to Ethernet or other types of LANs. I / O devices 1208 may include one or more of the following devices: keyboard, mouse, monitor, screen, printer, modem, touch screen, button interface, and other devices. Texture map generation using orthogonal projection application 1230 for implementing texture map generation using orthogonal projection may be stored in storage device 1212 and memory 1204 and processed in the manner in which applications are typically processed. Computing device 1200 may include Fig.12 In some embodiments, the texture map generation hardware 1220 using orthogonal projection is included. Although Fig.12 The computing device 1200 in the embodiment includes an application 1230 and hardware 1220 for implementing the texture map generation using orthogonal projection, although the texture map generation method using orthogonal projection can be implemented on the computing device in hardware, firmware, software, or any combination thereof. For example, in some embodiments, the texture map generation application 1230 using orthogonal projection is programmed in a memory and executed using a processor. In another example, in some embodiments, the texture map generation hardware 1220 using orthogonal projection is programmed hardware logic including gates specifically designed to implement the texture map generation method using orthogonal projection.

[0080] In some embodiments, the texture map generation application 1230 using orthogonal projection includes several applications and / or modules. In some embodiments, the module also includes one or more sub-modules. In some embodiments, fewer or additional modules may be included.

[0081] Examples of suitable computing devices include personal computers, laptop computers, computer workstations, servers, mainframe computers, handheld computers, personal digital assistants, cellular / mobile phones, smart appliances, game consoles, digital cameras, digital video cameras, camera phones, smart phones, portable music players, tablet computers, mobile devices, video players, video disc recorders / players (e.g., DVD recorders / players, high-definition disc recorders / players, ultra-high-definition disc recorders / players), televisions, home entertainment systems, augmented reality devices, virtual reality devices, smart jewelry (e.g., smart watches), vehicles (e.g., self-driving vehicles), or any other suitable computing device.

[0082] To utilize the texture map generation method using orthogonal projection, the device acquires or receives 3D content (eg, point cloud content). The texture map generation method using orthogonal projection can be implemented with the help of a user, or can be implemented automatically without user involvement.

[0083] In operation, the generation of texture maps using orthogonal projection is performed in a fast and efficient manner. Today, texture map generation relies on complex optimizations to reduce mapping distortion and texture seams. However, this process is not suitable for real-time applications and is usually performed on a frame-by-frame basis, which means that it does not exploit the temporal correlation between texture maps. A method for generating texture maps is described herein that takes significantly less time and also allows the mapping to exploit the temporal correlation between the content of different frames. Texture maps can be used for automatic generation of volumetric content, or can be used for more efficient compression of dynamic meshes.

[0084] Some embodiments of ORTHOATLAS: Texture map generation using orthographic projection for dynamic meshes

[0085] 1. A method of programming in a non-transitory memory of a device, comprising:

[0086] generating image patches from dynamic mesh information; and

[0087] Pack image tiles on a texture atlas using orthographic projection.

[0088] 2. The method of clause 1, wherein generating the image block from the dynamic mesh information further comprises:

[0089] Generate a list of neighboring triangles;

[0090] Compute triangle properties;

[0091] Add adjacent triangles;

[0092] Checking for vertex occlusion; and

[0093] Check for surface obstructions.

[0094] 3. The method of clause 2, wherein generating the list of neighboring triangles comprises adding triangles that share vertices with the triangle to the list.

[0095] 4. The method of clause 2, wherein generating the list of neighboring triangles comprises adding triangles that share edges with the triangle to the list.

[0096] 5. A method according to clause 2, wherein calculating triangle properties includes calculating the normal and surface area of ​​each triangle.

[0097] 6. The method of clause 2, further comprising selecting a seed triangle that has not yet been added to the image patch and whose normal is most consistent with the most frequent orientation.

[0098] 7. The method of clause 2 further comprises merging image blocks based on a calculated cost, wherein the calculated cost is based on a perimeter of the image blocks and an orthogonal stretch value.

[0099] 8. The method of clause 1, wherein packing image tiles on a texture atlas using orthogonal projection includes implementing frame scaling, image tile orientation, and temporal stabilization.

[0100] 9. A device comprising:

[0101] Non-transitory memory for storing applications for:

[0102] generating image patches from dynamic mesh information; and

[0103] Packing image tiles on a texture atlas using orthographic projection; and

[0104] A processor is coupled to the memory, the processor being configured to process the application.

[0105] 10. The apparatus of clause 9, wherein generating the image block from the dynamic mesh information further comprises:

[0106] Generate a list of neighboring triangles;

[0107] Compute triangle properties;

[0108] Add adjacent triangles;

[0109] Checking for vertex occlusion; and

[0110] Check for surface obstructions.

[0111] 11. The apparatus of clause 10, wherein generating the list of neighboring triangles comprises adding triangles that share vertices with the triangle to the list.

[0112] 12. The apparatus of clause 10, wherein generating the list of neighboring triangles comprises adding triangles that share edges with the triangle to the list.

[0113] 13. An apparatus as described in clause 10, wherein calculating triangle properties includes calculating a normal and a surface area of ​​each triangle.

[0114] 14. The apparatus of clause 10, wherein the application is further configured to select a seed triangle not yet added to the image block whose normal is most consistent with the most frequent orientation.

[0115] 15. The apparatus of clause 10, wherein the application is further configured to merge the image blocks based on a calculated cost, wherein the calculated cost is based on a perimeter of the image blocks and an orthogonal stretch value.

[0116] 16. An apparatus as described in clause 9, wherein packing image tiles on a texture atlas using orthogonal projection includes implementing frame scaling, image tile orientation, and temporal stabilization.

[0117] 17. A system comprising:

[0118] One or more cameras for acquiring three-dimensional content; and

[0119] An encoder configured to:

[0120] generating image patches from dynamic mesh information; and

[0121] Pack image tiles on a texture atlas using orthographic projection.

[0122] 18. The system of clause 17, wherein generating the image block from the dynamic mesh information further comprises:

[0123] Generate a list of neighboring triangles;

[0124] Compute triangle properties;

[0125] Add adjacent triangles;

[0126] Checking for vertex occlusion; and

[0127] Check for surface obstructions.

[0128] 19. The system of clause 18, wherein generating the list of neighboring triangles comprises adding triangles that share vertices with the triangle to the list.

[0129] 20. The system of clause 18, wherein generating the list of neighboring triangles comprises adding triangles that share edges with the triangle to the list.

[0130] 21. A system according to clause 18, wherein calculating triangle properties includes calculating a normal and a surface area of ​​each triangle.

[0131] 22. The system of clause 18, wherein the encoder is further configured to select a seed triangle not yet added to the image block whose normal is most consistent with the most frequent orientation.

[0132] 23. The system of clause 18, wherein the encoder is further configured to merge the image blocks based on a calculated cost, wherein the calculated cost is based on a perimeter of the image blocks and an orthogonal stretch value.

[0133] 24. The system of clause 17, wherein packing image tiles on a texture atlas using orthogonal projection comprises implementing frame scaling, image tile orientation, and temporal stabilization.

[0134] The present invention has been described by way of specific embodiments, which contain details that facilitate understanding of the construction and operating principles of the present invention. Such references to specific embodiments and details thereof herein are not intended to limit the scope of the claims appended hereto. It will be apparent to those skilled in the art that various other modifications may be made in the embodiments selected for illustration without departing from the spirit and scope of the present invention as defined by the claims.

Claims

1. A method of programming in a non-transitory memory of a device, comprising: Generate image patches from dynamic mesh information; as well as Pack image tiles on a texture atlas using orthographic projection.

2. The method of claim 1, wherein generating the image block from the dynamic grid information further comprises: Generate a list of neighboring triangles; Compute triangle properties; Add adjacent triangles; Check vertex occlusion; as well as Check for surface obstructions.

3. The method of claim 2, wherein generating a list of neighboring triangles comprises adding triangles that share vertices with the triangle to the list. 4 . The method of claim 2 , wherein generating a list of neighboring triangles comprises adding triangles that share edges with the triangle to the list.

5. The method of claim 2, wherein computing triangle properties comprises computing a normal and a surface area of ​​each triangle.

6. The method of claim 2, further comprising selecting a seed triangle that has not yet been added to the image block and whose normal is most consistent with the most frequent orientation.

7. The method of claim 2, further comprising merging the image blocks based on a calculated cost, wherein the calculated cost is based on a perimeter of the image blocks and an orthogonal stretch value.

8. The method of claim 1, wherein packing image tiles on a texture atlas using orthogonal projection includes implementing frame scaling, image tile orientation, and temporal stabilization.

9. An apparatus comprising: Non-transitory memory for storing applications for: Generate image patches from dynamic mesh information; as well as Pack image tiles on a texture atlas using orthographic projection; and A processor is coupled to the memory, the processor being configured to process the application.

10. The apparatus of claim 9, wherein generating the image block from the dynamic grid information further comprises: Generate a list of neighboring triangles; Calculate triangle properties; Add adjacent triangles; Check vertex occlusion; as well as Check for surface obstructions.

11. The apparatus of claim 10, wherein generating a list of neighboring triangles comprises adding triangles that share vertices with the triangle to the list.

12. The apparatus of claim 10, wherein generating a list of neighboring triangles comprises adding triangles that share edges with the triangle to the list.

13. The apparatus of claim 10, wherein calculating triangle properties comprises calculating a normal and a surface area of ​​each triangle.

14. The apparatus of claim 10, wherein the application is further configured to select a seed triangle that has not yet been added to the image block and whose normal is most consistent with the most frequent orientation.

15. The apparatus of claim 10, wherein the application is further configured to merge the image blocks based on a calculated cost, wherein the calculated cost is based on a perimeter of the image blocks and an orthogonal stretch value.

16. The apparatus of claim 9, wherein packing image tiles on a texture atlas using orthogonal projection comprises implementing frame scaling, image tile orientation, and temporal stabilization.

17. A system comprising: one or more cameras for acquiring three-dimensional content; and An encoder configured to: Generate image patches from dynamic mesh information; as well as Pack image tiles on a texture atlas using orthographic projection.

18. The system of claim 17, wherein generating the image blocks from the dynamic mesh information further comprises: Generate a list of neighboring triangles; Calculate triangle properties; Add adjacent triangles; Check vertex occlusion; as well as Check for surface obstructions.

19. The system of claim 18, wherein generating a list of neighboring triangles comprises adding triangles that share vertices with the triangle to the list.

20. The system of claim 18, wherein generating a list of neighboring triangles comprises adding triangles that share edges with the triangle to the list.

21. The system of claim 18, wherein computing triangle properties comprises computing a normal and a surface area of ​​each triangle.

22. The system of claim 18, wherein the encoder is further configured to select a seed triangle that has not yet been added to the image block and whose normal is most consistent with the most frequent orientation.

23. The system of claim 18, wherein the encoder is further configured to merge image blocks based on a calculated cost, wherein the calculated cost is based on a perimeter of the image block and an orthogonal stretch value.

24. The system of claim 17, wherein packing image tiles on a texture atlas using orthogonal projection includes implementing frame scaling, image tile orientation, and temporal stabilization.

Images