Mesh compression based on atlas sampling using general topology charts

Abstract

A method, apparatus, and system are provided for atlas sampling based dynamic mesh compression. The process may include receiving a mesh frame having polygons representing a surface of an object, determining that the mesh frame is associated with an irregular UV chart in response to one or more singularity components in the mesh frame, performing boundary detection on the UV chart to identify one or more boundary loops in the UV chart, and compressing information of the identified one or more boundary loops into a bitstream.

Description

[Technical Field]

[0001] Cross-reference of related applications This application claims priority to U.S. Provisional Application No. 63 / 326,182, “Atlas Sampling Based Mesh Compression with Charts of General Topology,” filed on 31 March 2022, and is a continuation of and claims priority to U.S. Patent Application No. 18 / 121,334, also titled “Atlas Sampling Based Mesh Compression with Charts of General Topology,” filed on 14 March 2023, which is incorporated herein by reference in its entirety.

[0002] This disclosure relates to a set of advanced video coding techniques. More specifically, this disclosure relates to video-based dynamic mesh compression. [Background technology]

[0003] The world's advanced three-dimensional (3D) representations are enabling even more immersive interactions and communication. To achieve the realism of 3D representations, 3D models are becoming more sophisticated than ever before, and a considerable amount of data is linked to the creation and consumption of these 3D models. 3D meshes are widely used in immersive 3D model content.

[0004] A 3D mesh can consist of several polygons that describe the surface of a volumetric object. Dynamic mesh sequences can require large amounts of data because they can contain a significant amount of information that changes over time. Therefore, efficient compression techniques are needed to store and transmit such content.

[0005] Mesh compression standards IC, MESHGRID, and FAMC were previously developed to address dynamic meshes with permanent connectivity and time-varying geometry and vertex attributes. However, these standards do not take into account time-varying attribute maps and connectivity information.

[0006] Furthermore, especially under real-time constraints, it is difficult for volume acquisition techniques to generate a consistent, dynamic mesh. This type of dynamic mesh content is not supported by existing standards. [Overview of the project] [Means for solving the problem]

[0007] According to various embodiments, a sampling-based dynamic mesh compression method may be provided. The method may be performed by at least one processor and may include the steps of: determining one or more sample locations associated with an input mesh based on one or more sampling rates; determining an occupation status associated with each of the one or more sample locations, wherein the occupation status indicates whether each of the one or more sample locations lies within the boundary of one or more polygons defined by the input mesh; generating a sample-based occupation map based on the occupation status associated with each of the one or more sample locations; and compressing the sample-based occupation map into a plurality of single-channel or multi-channel images using a video codec.

[0008] According to various embodiments, a device for sampling-based dynamic mesh compression may be provided. The device may include at least one memory configured to store program code, and at least one processor configured to read the program code and operate as instructed by the program code. The program code may include a first determination code configured to cause at least one processor to determine one or more sample locations associated with an input mesh based on one or more sampling rates, a second determination code configured to cause at least one processor to determine the occupancy status associated with each of the one or more sample locations, wherein the occupancy status indicates whether each of the one or more sample locations lies within the boundary of one or more polygons defined by the input mesh, a first generation code configured to cause at least one processor to generate a sample-based occupancy map based on the occupancy status associated with each of the one or more sample locations, and a compression code configured to cause at least one processor to compress the sample-based occupancy map into a plurality of single-channel or multi-channel images using a video codec.

[0009] According to various embodiments, a non-transitory computer-readable medium storing computer instructions may be provided. When the instructions are executed by one or more processors for sampling-based dynamic mesh compression, the one or more processors are caused to determine one or more sample positions associated with an input mesh based on one or more sampling rates, determine an occupancy status respectively associated with each of the one or more sample positions, the occupancy status indicating whether each of the one or more sample positions is within the boundaries of one or more polygons defined by the input mesh, generate a sample-based occupancy map based on the occupancy status respectively associated with each of the one or more sample positions, and include one or more instructions for compressing the sample-based occupancy map into a plurality of single-channel or multi-channel images using a video codec.

[0010] Further features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings.

Brief Description of the Drawings

[0011] [Figure 1] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a schematic diagram of a simplified block diagram of a streaming system according to an embodiment of the present disclosure. [Figure 3] FIG. 3 is a schematic diagram of a simplified block diagram of a video encoder and decoder according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is an exemplary diagram of 2D atlas sampling of a mesh according to an embodiment of the present disclosure. [Figure 5] FIG. 5 is an exemplary diagram of connectivity in a mesh according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is an exemplary diagram of a reconfigured mesh according to an embodiment of the present disclosure. [Figure 7A]A diagram showing a 2D simply-connected manifold chart according to an embodiment of the present disclosure. [Figure 7B] A diagram showing a 2D simply-connected non-manifold chart according to an embodiment of the present disclosure. [Figure 7C] A diagram showing a 2D non-simply-connected non-manifold chart according to an embodiment of the present disclosure. [Figure 8] A flowchart showing compression using atlas sampling according to an embodiment of the present disclosure. [Figure 9] A flowchart showing mesh reconstruction according to an embodiment of the present disclosure. [Figure 10] A block diagram of a computer system suitable for implementing an embodiment of the present disclosure.

Mode for Carrying Out the Invention

[0012] A mesh can be composed of several polygons that describe the surface of a volumetric object. Information about its vertices in 3D space and how the vertices are connected can define each polygon, which is called connectivity information. In some embodiments, vertex attributes such as color and normal can be associated with mesh vertices. The attributes can also be associated with the surface of the mesh by utilizing mapping information that parameterizes the mesh with a 2D attribute map. Such a mapping is called UV coordinates or texture coordinates and can be defined using a set of parametric coordinates associated with the mesh vertices. A 2D attribute map can be used to store high-resolution attribute information such as texture, normal, and displacement. The high-resolution attribute information can be used for various purposes such as texture mapping and shading.

[0013] As mentioned above, 3D meshes or dynamic meshes can require large amounts of data because they can consist of a significant amount of information that changes over time. Existing standards do not take into account time-varying attribute maps and connectivity information. Existing standards also do not support volume acquisition techniques that generate always-on-a-connectivity dynamic meshes, especially under real-time conditions.

[0014] Therefore, a new mesh compression standard is needed for directly processing dynamic meshes that have time-varying connectivity information and, optionally, time-varying attribute maps. Embodiments of this disclosure enable efficient compression techniques for storing and transmitting such dynamic meshes in common topologies. Embodiments of this disclosure enable lossy and / or lossless compression for various applications such as real-time communications, storage, free-viewpoint video, AR, and VR.

[0015] According to one aspect of the present disclosure, a method, system, and non-temporary storage medium for dynamic mesh compression are provided. Embodiments of the present disclosure may also be applied to static meshes in which only one frame of the mesh or mesh content does not change over time.

[0016] Referring to Figures 1 and 2, one embodiment of the present disclosure for implementing the coding and decoding structures of the present disclosure will be described.

[0017] Figure 1 shows a simplified block diagram of a communication system 100 according to one embodiment of the present disclosure. The communication system 100 may include at least two terminals 110, 120 interconnected via a network 150. In the case of unidirectional data transmission, the first terminal 110 can encode video data, which may include mesh data at its local location, for transmission to the other terminal 120 via the network 150. The second terminal 120 can receive the encoded video data from the other terminal via the network 150, decode the encoded data, and display the restored video data. Unidirectional data transmission may be common in applications such as media delivery.

[0018] Figure 1 shows a second pair of terminals 130, 140 provided to support the bidirectional transmission of coded video, for example, during a video conference. In the case of bidirectional data transmission, each terminal 130, 140 can encode video data captured at a local location for transmission to other terminals via the network 150. Each terminal 130, 140 can also receive coded video data transmitted by other terminals, decode the coded data, and display the restored video data on a local display device.

[0019] In Figure 1, terminals 110-140 may be any other type of terminal, such as a server, personal computer, and smartphone, smart TV, and / or video codec and network transmission software, to support the associated applications. Network 150 represents any number of networks that transmit coded video data between terminals 110-140, including, for example, wired and / or wireless communication networks. Communication network 150 may exchange data over circuit-switched and / or packet-switched channels. Typical networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this description, the architecture and topology of network 150 may not be important to the operation of this disclosure unless described below herein.

[0020] Figure 2 shows an example of the deployment of a video encoder and decoder in a streaming environment as an application of the disclosed subject matter. The disclosed subject matter may be used in other video-enabled applications, such as video conferencing, digital TV, and storage of compressed video on digital media including CDs, DVDs, and memory sticks.

[0021] As shown in Figure 2, the streaming system 200 may include a capture subsystem 213 which includes a video source 201 and an encoder 203. The streaming system 200 may further include at least one streaming server 205 and / or at least one streaming client 206.

[0022] The video source 201 can, for example, create a stream 202 containing a 3D mesh and metadata associated with the 3D mesh. The video source 201 may include, for example, a 3D sensor (e.g., a depth sensor) or 3D imaging technology (e.g., a digital camera(s)) and a computing device configured to generate a 3D mesh using data received from the 3D sensor or 3D imaging technology. The sample stream 202, which may have a larger data volume compared to the encoded video bitstream, can be processed by an encoder 203 connected to the video source 201. As described in detail below, the encoder 203 may include hardware, software, or a combination thereof to enable or implement aspects of the subject matter of disclosure. The encoder 203 may also generate an encoded video bitstream 204. The encoded video bitstream 204 may have a smaller data volume compared to the uncompressed stream 202 and can be stored in a streaming server 205 for future use. One or more streaming clients 206 can access the streaming server 205 to obtain a video bitstream 209, which may be a copy of the encoded video bitstream 204.

[0023] The streaming client 206 may include a video decoder 210 and a display 212. The video decoder 210 can, for example, decode a video bitstream 209, which is an input copy of an encoded video bitstream 204, and generate an output video sample stream 211 that can be rendered on the display 212 or another rendering device (not shown). In some streaming systems, the video bitstreams 204, 209 may be encoded according to a specific video encoding / compression standard.

[0024] Figure 3 is an illustrative diagram of a framework 300 for dynamic mesh compression and mesh reconstruction using an encoder and decoder.

[0025] As shown in Figure 3, the framework 300 may include an encoder 301 and a decoder 351. The encoder 301 may include an input mesh 305 which is converted into a mesh 310 with a UV atlas, and the mesh 310 with the UV atlas is then converted into an occupancy map 315, a geometry map 320, an attribute map 325, and metadata 330. The decoder 351 may include a decoded occupancy map 335, a decoded geometry map 340, a decoded attribute map 345, a decoded metadata 350, and a reconstructed mesh 360.

[0026] According to one aspect of the present disclosure, the input mesh 305 may include one or more frames, each of which may be preprocessed by a series of operations to generate a mesh 310 with a UV atlas. For example, the preprocessing operations may include, but are not limited to, tracking, parameterization, remeshing, voxelization, etc. In some embodiments, the preprocessing operations may be performed only on the encoder side and not on the decoder side.

[0027] The mesh 310 with a UV atlas may be a 2D mesh. A 2D mesh with a UV atlas may be a mesh in which each vertex of the mesh can be associated with a UV coordinate on the 2D atlas. The mesh 310 with a UV atlas may be processed based on sampling and converted into multiple maps. For example, the UV atlas 310 may be processed based on sampling the 2D mesh with the UV atlas and converted into an occupancy map, a geometry map, and an attribute map. The generated occupancy map 335, geometry map 340, and attribute map 345 may be encoded using an appropriate codec (e.g., HVEC, VVC, AV1, etc.) and transmitted to a decoder over a network. In some embodiments, metadata (e.g., connectivity information, etc.) may also be transmitted to the decoder.

[0028] In one embodiment, the decoder 351 can receive an encoded occupation map, geometry map, and attribute map from the encoder. In addition to the embodiments described herein, the decoder 315 may use appropriate techniques and methods to decode the occupation map, geometry map, and attribute map. In one embodiment, the decoder 351 may generate a decoded occupation map 335, a decoded geometry map 340, a decoded attribute map 345, and decoded metadata 350. The input mesh 305 can be reconstructed into a reconstructed mesh 360 using one or more reconstruction filters and techniques based on the decoded occupation map 335, the decoded geometry map 340, the decoded attribute map 345, and the decoded metadata 350. In some embodiments, metadata 330 may be transmitted directly to the decoder 351, and the decoder 351 may use the metadata to generate a reconstructed mesh 360 based on the decoded occupation map 335, the decoded geometry map 340, and the decoded attribute map 345. Post-filtering techniques, including but not limited to remeshing, parameterization, tracking, and voxelization, may be applied to the reconstructed mesh 360.

[0029] An input mesh with a 2D UV atlas may have vertices, and each vertex may have a set of associated UV coordinates on the 2D atlas. Occupancy maps, geometry maps, and attribute maps can be generated by sampling one or more points / locations on the UV atlas. Each sample location may be occupied or unoccupied if its location lies inside a polygon defined by the mesh vertices. For each occupied sample location, its corresponding 3D geometry coordinates and attributes can be calculated by interpolation from the associated set of polygonal vertices.

[0030] According to one aspect of this disclosure, the sampling rate may be consistent throughout the entire 2D atlas. In some embodiments, the sampling rates for the u-axis and v-axis may be different, enabling anisotropic remeshing. In some embodiments, the entire 2D atlas may be divided into multiple regions, such as slices or tiles, each of which may have a different sampling rate.

[0031] According to one aspect of this disclosure, the sampling rate for each region (or the entire 2D atlas) may be signaled in high-level syntax, including, but not limited to, sequence headers, frame headers, slice headers, etc. In some embodiments, the sampling rate for each region (or the entire 2D atlas) may be selected from a set of pre-established rates assumed by both the encoder and the decoder. Because of the set of pre-established rates known by both the encoder and the decoder, signaling a particular sampling rate requires only sending an index within the pre-established rate set in the signal. Examples of such pre-established sets may be every 2 pixels, every 4 pixels, every 8 pixels, etc. In some embodiments, the sampling rate for each region (or the entire 2D atlas) of a mesh frame may be predicted from the set of pre-established rates, from previously used sampling rates in other already coded regions of the same frame, or from previously used sampling rates in other already coded mesh frames.

[0032] In some embodiments, the sampling rate for each region (or the entire 2D atlas) can be based on some characteristic of each region (or the entire 2D atlas). For example, the sample rate can be based on activity, with a higher sample rate for richly textured regions (or the entire 2D atlas) or high-activity regions (or the entire 2D atlas). As another example, a lower sample rate can be set for smooth regions (or the entire 2D atlas) or low-activity regions (or the entire 2D atlas).

[0033] In some embodiments, the sampling rate of each region of the mesh frame (or the entire 2D atlas) can be signaled in a way that enables the combination of prediction and direct signaling. The syntax can be configured to indicate whether the sampling rate is predicted or signaled directly by a signal. If predicted, which predictor sampling rate should be used can be further signaled by a signal. If sent by a direct signal, the syntax representing the value of the rate can be signaled by a signal.

[0034] FIG. 4 is an exemplary diagram of 2D atlas sampling of a triangular mesh 400 according to an embodiment of the present disclosure.

[0035] As shown in FIG. 4, the triangular mesh 400 has a plurality of positions and vertices, and each integer pixel is simply represented as a circle. Other representations may be used.

[0036] V1, V2,..., V D Assuming that the vertices of the polygons on the mesh are V, D defines the shape of each polygon. In the case of a triangular mesh, D is equal to 3, that is, each triangle has 3 vertices. Without loss of generality, in the remaining part of the present disclosure, it is assumed that the mesh is composed of triangular meshes, that is, D = 3.

[0037] For each vertex V i , the UV coordinates of that vertex are represented by (u i , v i ), the 3D position of that vertex is represented by (x i , y i , z i ), and other attributes of that vertex (such as color and normal) can be represented by a i . The UV coordinates of the vertex, that is, (u i , v i ) may be at integer positions or fractional positions. The 2D UV atlas of the input mesh may be of size W×H, and A is the coordinate (u A , v AA may be a sample point on a 2D UV atlas having the UV coordinates (u i ,v i If A lies inside a triangle in a 2D UV atlas defined by ), it can be marked as occupied; otherwise, it can be marked as unoccupied.

[0038] According to one aspect of this disclosure, an occupancy map may be generated based on sampling a 2D UV atlas (sometimes a mesh). Connectivity of points and / or positions on the occupancy map may be signaled (e.g., using metadata or in SPS or PPS) or inferred by a decoder. The orientation of connectivity (clockwise or counterclockwise) may be signaled by a high-level syntax such as a sequence header or slice header, or fixed (assumed) by an encoder and decoder.

[0039] In one embodiment, a one-channel binary occupancy map may be generated by scanning all sampling points on a 2D map. Each point on the one-channel binary occupancy map may indicate whether the sample point lies inside an arbitrary triangle (or other shape of the mesh polygon). In one embodiment, a one-channel non-binary occupancy map may be generated using other values ​​to indicate whether the sample point belongs to a vertex position or a boundary. A multi-channel occupancy map may be generated in which different channels have different indications. In some embodiments, an occupancy map may be generated that contains only vertex positions. The connectivity of these vertices may be either signaled to the decoder or inferred by the decoder.

[0040] Occupied maps can be compressed by any suitable image and video codecs. Occupied maps can be compressed as single-channel or multi-channel images, e.g., YUV420, YUV400, YUV444, RGB 444, etc. Occupied maps can also be compressed as binary images or images of any bit depth, e.g., 1-bit, 2-bit, 8-bit, 12-bit, 16-bit, etc. Occupied maps can be compressed by lossy or lossless codecs.

[0041] According to one aspect of this disclosure, a geometry map may be generated based on sampling a 2D UV atlas (sometimes a mesh). Unlike a normal image with RGB colors, a geometry map can store the 3D geometric coordinates of the sample points corresponding to those channels. For each occupied sample point A in the 2D UV atlas, its 3D geometric position may be interpolated by the three vertices (V1, V2, V3) of a triangle. Here, any suitable interpolation method can be employed. For example, a centroid-based method may be used to estimate the 3D geometric position of A.

[0042] As an example, if we assume that (λ1, λ2, λ3) are the centroid coordinates of A with respect to triangle (V1, V2, V3), then the following applies. (u A ,v A )=λ1·(u1,v1)+λ2·(u2,v2)+λ3·(u3,v3) (u A ,v A )=λ1 (u1,v1)+λ2 (u2,v2)+λ3 (u3,v3)...Equation (1) λ1 + λ2 + λ3 = 1 ....Equation (2)

[0043] Regarding the linear relationship between UV coordinates and 3D coordinates, A u,v The 3D position of (x A ,y A ,z A ) can be calculated as follows: (x A ,yA ,z A )=λ1·(x1,y1,z1)+λ2·(x2,y2,z2)+λ3·(x3,y3,z3)....Equation (3)

[0044] In some embodiments, the estimated 3D geometry position of A is, i.e., (x A ,y A ,z A ) can be converted to an unsigned integer in order to be encoded by image codecs and video codecs.

[0045] In the case of an unoccupied point A, its 3D geometry position (x A ,y A ,z A ) can be any number. In some embodiments, to facilitate the coding efficiency of the geometry map, the 3D geometry locations of unoccupied points may be assigned to be similar to their spatial and temporal neighbors. In some embodiments, the 3D geometry locations of unoccupied points may be assigned values ​​that are impossible for geometry locations, such as (-1,-1,-1), in which case the occupancy map may not be signaled.

[0046] To generate a geometry map associated with a sampled location, according to one aspect of this disclosure, a 3-channel geometry map can be generated by scanning all the sampling points on a 2D map, where each point on the 3-channel geometry map represents its 3D geometric location. For example, the first channel may represent the x-position, the second channel may represent the y-position, and the third channel may represent the z-position. In some embodiments, three single-channel geometry maps can be generated, each map representing the 3D geometric location along one axis. For example, the first map may represent the x-position, the second map may represent the y-position, and the third map may represent the z-position. In some embodiments, a single-channel geometry map representing the depth (distance) to the projection plane can be generated. The projection plane may be different for each different triangle, in which case the projection plane information can be signaled to the decoder as side information.

[0047] According to one aspect of this disclosure, an attribute map may be generated based on sampling a 2D UV atlas (sometimes a mesh).

[0048] Each occupied sample point A may have attributes (e.g., normal, color), which can be interpolated by the three vertices (V1, V2, V3) of a triangle. Any suitable interpolation method can be employed.

[0049] As an example, if we assume that (λ1, λ2, λ3) are the centroid coordinates of A with respect to triangle (V1, V2, V3), then the following applies. (u A ,v A )=λ1 (u1,v1)+λ2 (u2,v2)+λ3 (u3,v3)....Equation (4) λ1 + λ2 + λ3 = 1 ....Equation (5)

[0050] A u,v The attribute value of a A It can be calculated as follows: a A=λ1·a1+λ2·a2+λ3·a3....Formula (6)

[0051] In some embodiments, each unoccupied point A has attribute a A This can be any number. In some embodiments, to facilitate the coding efficiency of the attribute map, the attribute values ​​of unoccupied points may be assigned to be similar to their spatial and temporal neighbors. In some embodiments, the attribute values ​​of unoccupied points may be assigned to values ​​that are impossible for the attribute, such as (-1,-1,-1), in which case the occupancy map may not be signaled.

[0052] To generate an attribute map associated with a sampled location, according to one aspect of this disclosure, an N-channel attribute map can be generated by scanning all sampled points on a 2D map, each point on the map showing its attribute value. For example, for an attribute that is color, the first channel may show the R color, the second channel may show the G color, and the third channel may show the B color. In embodiments where the attribute is a normal, the three channels may be normal values ​​on the x, y, and z axes. In some embodiments, N single-channel attribute maps can be generated, each showing an attribute value. For example, for an attribute that is color, the first map may show the R color, the second map may show the G color, and the third map may show the B color. In some embodiments, a texture map associated with a mesh can be considered an additional attribute map and can be coded by any suitable image or video codec.

[0053] Figure 5 shows an example of connectivity in a mesh according to an embodiment of the present disclosure, in which an encoder may generate connectivity information and a decoder may use it to reconstruct the mesh.

[0054] The decoder can reconstruct the mesh after decoding the occupation map, geometry map, and attribute map. For each occupied point on the occupation map, the decoder can reconstruct one or more vertices on the mesh, and the corresponding geometry and attribute values ​​of the points can be obtained from the corresponding locations in the decoded geometry and attribute maps. Connectivity information between vertices can be reconstructed by the decoder either by implicitly inferring it from the occupied locations or by explicitly signaling it.

[0055] According to one aspect of this disclosure, connectivity information related to an input mesh and / or a 2D UV atlas can be implicitly inferred from the occupancy status of adjacent points on the decoded occupancy map by certain rules. The direction of connectivity (clockwise or counterclockwise) can be parsed from high-level syntax such as sequence headers and slice headers, or it can be fixed (assumed) by the encoder and decoder. According to some embodiments, the reconstructed mesh may have different connectivity from the input mesh. For example, the input mesh may be a triangular mesh, and the reconstructed mesh may be a quadrilateral mesh.

[0056] According to one aspect of this disclosure, for every four adjacent points on the decoded occupancy map, if the number of occupied points is three or more, the triangular connectivity between the four points can be inferred by certain rules. For example, as shown in Examples (2) to (5) of Figure 5, if three of the four points are occupied, these points can be directly connected to form a triangle. For example, as shown in Example (1) of Figure 5, if all four points are occupied, these points can form two triangles. In some embodiments, different rules may be applied to different numbers of adjacent points.

[0057] In some embodiments, a quadrilateral mesh can be reconstructed when all four adjacent points are occupied. In some other embodiments, the reconstructed mesh may be a hybrid mesh, for example, where some areas within the mesh frame generate a triangular mesh and other areas generate a quadrilateral mesh. The connectivity type can be signaled with a high-level syntax such as a sequence header or a slice header.

[0058] In some embodiments, connectivity information can also be reconstructed by explicitly signaling it. If the polygon cannot be reconstructed by implicit rules, the encoder can signal the connectivity information in the bitstream, for example, as metadata. Any suitable means can be used to reduce the overhead of explicit signaling, such as depending on the polygon boundaries.

[0059] Figure 6 is an illustrative diagram of the reconstructed mesh 600 generated using the combination of implicit rules and explicit signaling described in Figure 5.

[0060] When an atlas sampling-based method is used for mesh compression, the mesh's geometric information is encoded as a geometric image sampled from a 2D UV atlas. As shown in Figures 7A–7C, the UV atlas 700 may or may not be a manifold chart (also known as a "UV chart"), and may or may not be simply connected. A 2D UV chart is a manifold chart in which each vertex in the chart has a neighborhood that is isomorphic with respect to an open disk or half disk. A 2D UV chart is a 2D simply connected chart in which each vertex in the chart is path-connected, and each path between two points is isomorphic to any other path between the same two points.

[0061] For example, Figure 7A shows a 2D simply connected manifold chart according to an embodiment of the present disclosure. Figure 7B shows a 2D simply connected non-manifold chart according to an embodiment of the present disclosure. A 2D simply connected non-manifold chart looks like a bowtie, with two or more regions touching at a single point (e.g., vertex v3). Vertex v1 is an interior vertex and therefore a normal vertex. Vertices v2 and v4-v10 are boundary vertices, each satisfying equation (1) and being normal vertices. For example, vertex v2 is two Incident face and three Incident edge Vertex v4 has two Connection surface and three Connecting edge Vertex v5 has two Connection surface and three Connecting edge Vertex v6 ​​has two Connection surface and three Connecting edge It has, and vertex v7 is one Connection surface and two Connecting edge It has, and vertex v8 is one Connection surface and two Connecting edge It has, and vertex v9 is one Connection surface and two Connecting edge It has, and vertex v10 is one Connection surface and two Connecting edge It has . On the other hand, vertex v3 is a boundary vertex, and two Connection surface and four Connecting edge Therefore, vertex v3 does not satisfy equation (1) and is a singular vertex.

[0062] According to other aspects of this disclosure, Incident boundary edge The number of unique vertices can be used to identify them. For example, normal boundary vertices have two Connection boundary It can only have three or more singular boundary vertices. Connection boundary It has.

[0063] According to some aspects of this disclosure, in order to detect boundary loops in a non-manifold mesh, a non-manifold mesh can be converted to a manifold mesh without affecting the boundary loop configuration, and thus boundary loops in a manifold mesh can correspond to boundary loops in a non-manifold mesh. Boundary loops in a manifold mesh can be detected by a suitable algorithm. Based on the boundary loops in the manifold mesh, the corresponding boundary loops in a non-manifold mesh can be determined.

[0064] In some examples, the boundary loop detection procedure for a non-manifold mesh includes three steps, referred to as the first step, the second step, and the third step.

[0065] In the first step (also called the topological manipulation step), for each singular vertex of the mesh, Connecting edge The elements can be indexed (e.g., ordered) into a cyclic sequence in either a clockwise or counterclockwise direction. In the cyclic sequence, the first Connecting edge and last Connecting edge are consecutive Connecting edge This is the case for a unique vertex. Connecting edge Loop edge pairs can be detected based on the cyclic sequence. Loop edge pairs are two consecutive edges in the cyclic sequence that are not on the same face. Connecting edge It is defined as follows: Due to the periodic nature of indexing, the first and last Connecting edge Unless they are edges on the same face, the first and last Connecting edge Note that modular arithmetic rules can be used to define a loop edge pair.

[0066] For each loop edge pair, two Connection surface It is possible to identify the alias vertices and split the singular vertices into two identified ones. Connection surface It is possible to replace singular vertices in the array. Alias ​​vertices are two loop edge pairs. Connection boundaryIt can be configured to have only two identified Connection surface The original index of a singular vertex within the array can be replaced with a new index, which is recorded as an alias of the original index of the singular vertex.

[0067] After the first step (topological manipulation step), the non-manifold mesh becomes a manifold mesh.

[0068] In the second step (called the boundary loop detection step), boundary loops in the manifold mesh can be detected using an appropriate algorithm developed to detect boundary loops in the manifold mesh.

[0069] In the third step (called the boundary loop for non-manifold meshes), after the boundary loop for the manifold mesh has been determined, the new index can be replaced with the original index according to the alias relationship in order to determine the corresponding boundary loop for the non-manifold mesh.

[0070] In some examples, a verification operation can be performed in the third step. The verification operation can verify whether the number of internal boundary loops detected in the non-manifold mesh matches the number of holes detected based on Euler properties and Betti numbers.

[0071] According to one aspect of this disclosure, the relationship between the Euler property χ and the Betti number of a simply connected mesh can be expressed as equation (7).

number

[0072] Furthermore, according to one aspect of this disclosure, two vertices of a singular edge are singular vertices. The topological operation in the first step can be applied to the vertices of a singular edge in a single 2-composite mesh to transform the non-manifold mesh into a manifold mesh in order to detect boundary loops in the non-manifold mesh.

[0073] Figure 7C shows a 2D non-simply connected non-manifold chart according to an embodiment of the present disclosure. In this example, the 2D non-simply connected non-manifold chart includes vertices v0 through v10 and has two further regions in the chart and a single point (e.g., vertex v5) to which the “holes” are in contact, as shown in Figure 7C.

[0074] Vertex v10 is an interior vertex and therefore a normal vertex. Vertices v0 through v9 are boundary vertices. Vertices v0-v4 and v6-v9 are each two Connection boundary It is a normal boundary vertex because it has four Connection boundary It possesses and is a unique vertex.

[0075] According to one aspect of this disclosure, the three-step boundary loop detection procedure for non-manifold meshes described above can be applied to a 2D non-simple connection non-manifold chart to detect boundary loops.

[0076] In some examples, boundary loop detection results can be verified according to Euler properties and Betti numbers. In the example in Figure 7C, the Euler property χ = 11 - 22 + 11 = 0, and the Betti numbers b0 = 1, b2 = 0, and g = 0, so the expected number of holes is h = b1 - 2g = b0 + b2 - χ - 2g = 1. Therefore, according to the Euler properties and Betti numbers, the mesh has one hole (internal boundary loop) that matches the result from the boundary loop detection procedure for non-manifold meshes.

[0077] According to one aspect of this disclosure, the boundary loop detection procedure for a non-manifold mesh can be generalized to handle high-dimensional simply connected meshes. For example, in the case of a simply connected 3D mesh, the topological operation of the first step can similarly be performed to divide the 3D singular vertices / edges, such as vertices / edges where two or more tetrahedra meet, in order to convert the non-manifold mesh into a 3D manifold mesh. Then, "boundary faces" can be determined within the 3D manifold mesh.

[0078] Furthermore, it should be noted that the boundary loop detection procedure can be applied to meshes that are not formed from individual elements. In some examples, meshes that are not formed from individual elements can be remeshed into meshes with simply connected complexes. For example, a quadrilateral mesh can be divided into triangular meshes. Then, the boundary loop detection procedure can be applied to meshes with individual complexes.

[0079] According to one aspect of this disclosure, several methods are proposed to improve existing atlas sampling-based mesh compression methods for UV charts with common topologies. For example, a 2D UV chart is a regular chart if it is a manifold and simply connected (as shown in Figure 7A), and an irregular chart if it is not (as shown in Figures 7B and 7C).

[0080] In some embodiments, the problem of irregular charts in atlas sampling-based mesh compression can be addressed by reparameterizing the mesh before mesh compression using an algorithm that generates regular UV charts using an algorithm such as an isochart algorithm. After reparameterization, for each UV chart, the encoder 301 determines whether the UV chart is regular, and then converts irregular UV charts to regular UV charts. After converting irregular UV charts to regular UV charts, existing atlas sampling-based methods can be easily applied without modification, and there is no need to signal additional information in the bitstream. Another advantage of the reparameterization preprocessing is that, with a carefully tuned algorithm, it is possible to reduce the number of UV charts and / or generate spatially and / or temporally coherent UV charts to improve mesh compression efficiency.

[0081] In some other embodiments, it is determined whether the UV chart is regular. For example, as shown in Figure 7B, the vertices Connection boundary If the number is not 2, a non-manifold vertex is identified, and the number of “holes” in the UV chart is derived by calculating the Euler properties of the UV chart, as shown in Figure 7C. If the UV chart is regular, the chart can be processed using existing atlas sampling-based methods. Otherwise, boundary loop detection algorithms, such as those based on topological operations, are used to detect all boundary loops in the chart, including outer and inner boundary loops surrounding the “holes”. Note that whether the chart is regular or irregular, there may only be one outer boundary loop for the chart. All boundary loop information (including outer and inner boundaries) is signaled in a bitstream in order to properly reconstruct the mesh.

[0082] In one embodiment, the number of boundary loops and the length of each boundary loop are signaled in the bitstream. A protocol can be established between the encoder 301 and the decoder 351 regarding how to sort the boundary loops in a predetermined manner. For example, one predetermined manner is to first signal the total number of boundary loops in the bitstream before signaling the information of other boundary loops. Thus, if the total number of boundary loops is 1, it means that the UV chart has no holes inside. Another predetermined manner is to signal the information of the outer boundary loops before the other inner loops in the bitstream. Each boundary loop is signaled, and the boundary loop includes the UV coordinates and 3D coordinates of the boundary vertices, as well as the mapping between these two types of coordinates.

[0083] In another embodiment, the length of each boundary loop can be predicted or inferred from the decoder side. For example, a protocol can be established between encoder 301 and decoder 351 so that if the predicted residual of a boundary vertex (either XYZ coordinates or UV coordinates) is greater than a threshold, it indicates a new boundary loop from the current vertex. The threshold can be fixed for encoder 301 and decoder 351 or signaled via a bitstream.

[0084] In some embodiments, the decoder 351 decodes boundary loop information from the bitstream and uses the decoded boundary loops for each UV chart to triangulate (connect vertices to faces) its UV chart. For example, if only one boundary loop is signaled for a UV chart, the decoder 351 triangulates the UV chart using an existing method. Otherwise, the decoder 351 determines that there is one or more internal boundary loops in the UV chart. The external and internal boundary loops are decoded from the bitstream and specified by the triangulation algorithm. As a result, "holes" enclosed by internal boundaries are not triangulated. In this way, the mesh is appropriately reconstructed by an atlas sampling-based method, regardless of the topology of the UV chart.

[0085] Figure 8 is a flowchart illustrating a process 800 for encoding a 2D UV chart using atlas sampling-based dynamic mesh compression according to an embodiment of the present disclosure.

[0086] In operation S801, the encoder 301 starts an atlas sampling-based dynamic mesh compression process.

[0087] In operation S810, the encoder 301 receives a mesh frame having polygons representing the surface of an object. In some embodiments, one or more sample positions associated with the mesh frame may be determined based on one or more sampling rates. For example, the encoder 301 may determine one or more sample positions associated with the input mesh frame 305 based on one or more sampling rates. In some embodiments, the encoder 301 may determine one or more sample positions associated with the processed mesh 310 with a UV atlas based on one or more sampling rates.

[0088] In some embodiments, one or more sampling rates may be based on a first sampling rate for a first axis and a second sampling rate for a second axis. In some embodiments, one or more sampling rates may be signaled in high-level syntax. In some embodiments, the input mesh and / or processed mesh may be divided into one or more regions, and one or more sampling rates may be based on one or more respective sampling rates for each of the one or more regions. In some embodiments, one or more respective sampling rates for each of the one or more regions may be based on the respective characteristics of the one or more regions. The respective characteristics may include the texture of the region, the activity of the region, and the smoothness of the region. In some embodiments, one or more sampling rates may be based on the sampling rates of adjacent regions. In some embodiments, one or more sampling rates may be based on one or more previous sampling rates of one or more already coded mesh frames.

[0089] In operation S820, the encoder 301 determines whether the mesh frame is associated with an irregular UV chart in response to one or more singularity components in the mesh frame. For example, a 2D UV chart is a regular chart if it is a manifold and simply connected. As shown in Figure 7A, vertex v1 is a regular interior vertex. Vertices v2 and v4~v6 are regular boundary vertices and satisfy equation (1) respectively. Specifically, vertex v2 is two Connection surface and three Connecting edge Vertex v4 has two Connection surface and three Connecting edge Vertex v5 has two Connection surface and three Connecting edge It has two vertices v6 Connection surface and three Connecting edgeIt has. Therefore, the UV chart associated with the mesh shown in Figure 7A is a regular UV chart. In contrast, the vertex v3 of the mesh shown in Figure 7B is a boundary vertex, and has two Connection surface and four Connecting edge Therefore, vertex v3 does not satisfy equation (1) and is a singular vertex, while vertices v7 to v10 are regular boundary vertices and each satisfies equation (1). Thus, the 2D UV chart associated with the meshes shown in Figures 7B and 7C is an irregular UV chart (the mesh shown in Figure 7C has an internal boundary loop). In other words, a UV chart is irregular if it is non-manifold and non-simple connected at least one of the two. A mesh frame is determined to be associated with an irregular UV chart when it has at least one non-manifold vertex. Connection boundary When this is the case, the vertices within the mesh frame (e.g., v3 in Figure 7B or v5 in Figure 7C) are non-manifold.

[0090] In operation S830, encoder 301 performs boundary detection on the UV chart to identify one or more boundary loops within the UV chart. In some embodiments, after determining that the UV chart is an irregular UV chart, encoder 301 determines the Euler properties of the UV chart and then identifies one or more boundary loops within the UV chart based on the determined Euler properties of the UV chart. As described above, the number of internal boundary loops detected in a non-manifold mesh coincides with the number of holes detected in the UV chart based on the Euler properties and Betti numbers. As shown in Figure 7C, the Euler property χ = 11 - 22 + 11 = 0, and the Betti numbers b0 = 1, b2 = 0, and g = 0, so the expected number of holes is h = b1 - 2g = b0 + b2 - χ - 2g = 1. Thus, according to the Euler properties and Betti numbers, the mesh has one hole (internal boundary loop) that coincides with the result from the boundary loop detection procedure for a non-manifold mesh.

[0091] In operation S840, the encoder 301 compresses (e.g., encodes) the information of one or more identified boundary loops into metadata 330 in the bitstream associated with the mesh frame, along with the rest of the video codec, including the occupancy map 315, geometry map 320, attribute map 325, etc. In some embodiments, the encoder 301 signals the number of one or more identified boundary loops, as well as the UV coordinates and corresponding 3D coordinates of the vertices of each boundary map in the bitstream. In some embodiments, the encoder 301 encodes the information of one or more identified boundary loops according to a predetermined protocol so that the decoder 351 can decode the information of one or more identified boundary loops from the bitstream and reconstruct the mesh frame accordingly.

[0092] Figure 9 is a flowchart showing mesh reconstruction according to an embodiment of the present disclosure.

[0093] In operation S901, decoder 351 starts the mesh reconstruction processor. In operation S910, decoder 315 decodes from the bitstream information of boundary loops associated with mesh frames having polygons representing the surface of an object. In some embodiments, there is a predetermined protocol between encoder 301 and decoder 351. According to the protocol, encoder 301 encodes information of boundary loops associated with mesh frames into a bitstream, and decoder 351 decodes the information accordingly.

[0094] In operation S920, the decoder 351 determines, based on the decoding of information from the bitstream, that the mesh frame has a UV chart with one or more internal boundary loops. As described above, the UV chart is an irregular UV chart if the mesh frame has at least one internal boundary loop.

[0095] In operation S930, the decoder 351 triangulates the vertices of the mesh frame based on one or more internal boundary loops in the UV chart.

[0096] In operation S940, the decoder 351 reconstructs the mesh frame through atlas sampling of the UV chart.

[0097] In some embodiments, the decoder 351 can decode the received occupancy map, attribute map, and geometry map. For example, the decoder 351 can decode the encoded occupancy map 315, geometry map 320, and attribute map 325 to generate a decoded occupancy map 335, a decoded geometry map 340, and a decoded attribute map 345. The respective vertices associated with each occupancy location in the decoded occupancy map can be reconstructed. For example, the decoder 351 can reconstruct the respective vertices associated with each occupancy location and / or pixel based on the decoded occupancy map 335. The 3D geometric coordinates associated with each occupancy location can be obtained based on the decoded occupancy map and the decoded geometry map. For example, the decoder 351 can obtain the 3D geometric coordinates associated with each occupancy location based on the decoded occupancy map 335 and the decoded geometry map 340. The attribute values ​​associated with each occupancy location can be obtained based on the decoded occupancy map and the decoded attribute map. For example, the decoder 351 can obtain attribute values ​​associated with each occupied location based on the decoded occupation map 335 and the decoded attribute map 345.

[0098] Connectivity information can be inferred based on the decoded map or from the explicit signaling of metadata. For example, decoder 351 can infer connectivity information based on the decoded occupancy map 335, the decoded geometry map 340, the decoded attribute map 345, and the metadata 350.

[0099] In some embodiments, connectivity information associated with the input mesh and / or 2D UV atlas can be implicitly inferred from the occupancy status of adjacent points on the decoded occupancy map by certain rules (e.g., Figures 5(1)-(5)). The orientation of connectivity (clockwise or counterclockwise) can be parsed from high-level syntax such as sequence headers or slice headers, or it can be fixed (assumed) by the encoder and decoder. In some embodiments, the reconstructed mesh may be a hybrid mesh, for example, where some areas within the mesh frame generate a triangular mesh and other areas generate a quadrilateral mesh. The connectivity type can be signaled by high-level syntax such as sequence headers or slice headers.

[0100] In some embodiments, connectivity information may also be reconstructed by explicit signaling. If the polygon cannot be reconstructed by implicit rules, the encoder may signal connectivity information in the bitstream (e.g., as metadata). Any suitable means may be used to reduce the overhead of explicit signaling, such as depending on the polygon boundaries.

[0101] Finally, the mesh can be reconstructed based on the decoded map and connectivity information. For example, decoder 351 can generate a reconstructed mesh 360 based on the decoded occupancy map 335, the decoded geometry map 340, the decoded attribute map 345, and the metadata 350. Decoder 351 can generate the reconstructed mesh 360 using appropriate reconstruction filters and / or post-processing techniques. According to some embodiments, the reconstructed mesh may have different connectivity than the input mesh. For example, the input mesh may be a triangular mesh, and the reconstructed mesh may be a quadrilateral mesh.

[0102] The techniques described above can be implemented as computer software that uses computer-readable instructions and is physically stored on one or more computer-readable media. For example, Figure 10 shows a computer system 1000 suitable for implementing a particular embodiment of the present disclosure.

[0103] Computer software can be coded using any suitable machine code or computer language that can be assembled, compiled, linked, or subjected to similar mechanisms, in order to create code that contains instructions that can be executed directly or through interpretation, microcode execution, etc., by a computer's central processing unit (CPU), graphics processing unit (GPU), etc.

[0104] Instructions can be executed on various types of computers or computer components, including, for example, personal computers, tablet computers, servers, smartphones, game consoles, and Internet of Things devices.

[0105] The components shown in Figure 10 for computer system 1000 are illustrative and are not intended to imply any limitation on the scope of use or functionality of computer software implementing embodiments of this disclosure. The configuration of the components should not be construed as having any dependencies or requirements relating to any one or combination of components exemplified in non-limiting embodiments of computer system 1000.

[0106] The computer system 1000 may include certain human interface input devices. Such human interface input devices may respond to input from one or more users, for example, through tactile input (pressing keys, swiping, moving data gloves, etc.), voice input (voice, clapping hands, etc.), visual input (gestures, etc.), or olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., voices, music, ambient sounds), images (e.g., scanned images, photographic images taken from a still image camera), and video (e.g., 2D video, 3D video including stereoscopic video, etc.).

[0107] The input human interface device may include one or more of the following (only one of each is shown): a keyboard 1001, a mouse 1002, a trackpad 1003, a touchscreen 1010, a data glove, a joystick 1005, a microphone 1006, a scanner 1007, and a camera 1008.

[0108] The computer system 1000 may also include certain human interface output devices. Such human interface output devices may stimulate the senses of one or more users, such as tactile output, sound, light, and smell / taste. Such human interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen 1010, data glove, or joystick 1005, although there may also be tactile feedback devices that do not function as input devices). For example, such devices may include audio output devices (e.g., speaker 1009, headphones (not shown)), visual output devices (screens 1010 including CRT screens, LCD screens, plasma screens, OLED screens, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), each having or not having touchscreen input capability, each having or not having tactile feedback capability, some of which may be capable of outputting more than three dimensions by means such as two-dimensional visual output or stereoscopic output), and printers (not shown).

[0109] The computer system 1000 may further include human-accessible storage devices and related media such as CD / DVD ROM / RW 1020, USB memory (thumb-drive) 1022, removable hard drive or solid-state drive 1023, legacy magnetic media such as tape and floppy disks (not shown), and optical media such as specialized ROM / ASIC / PLD-based devices such as security dongles (not shown).

[0110] Those skilled in the art should also understand that the term “computer-readable medium” as used in connection with the subject matter of this disclosure does not include transmission media, carrier waves, or other transient signals.

[0111] The computer system 1000 may further have interfaces to one or more communication networks. These networks may be, for example, wireless, wired, or optical. Networks may also be local, wide-area, metropolitan, automotive, and industrial, real-time, latency-tolerant, etc. Examples of networks include local area networks such as Ethernet, mobile communication networks including Wi-Fi, GSM, 3G, 4G, 5G, LTE, etc., wide-area digital networks of wired or wireless television including cable television, satellite television, and terrestrial television, and industrial networks including automotive and CANBus. Some networks generally require external network interface adapters attached to some general-purpose data ports or peripheral buses 1049 (e.g., USB ports on the computer system 1000), while others are generally integrated into the core of the computer system 1000 by being attached to the system bus as described later (e.g., an Ethernet interface to a PC computer system, or a mobile communication network interface to a smartphone computer system). Using any such network, the computer system 1000 can communicate with other entities. Such communications may be one-way communications, receive-only communications (e.g., television broadcasting), one-way transmit-only communications (e.g., sending from a CANbus to a specific CANbus device), or two-way communications to other computer systems, for example, using a local or wide-area digital network. Such communications may include communications with a cloud computing environment 1055. Specific protocols and protocol stacks may be used in each of those networks and network interfaces, as described above.

[0112] The aforementioned human interface device, human-accessible storage device, and network interface 1054 can be mounted on the core 1040 of the computer system 1000.

[0113] The core 1040 may include specialized programmable processing units in the form of one or more central processing units (CPUs) 1041, graphics processing units (GPUs) 1042, field-programmable gate areas (FPGAs) 1043, hardware accelerators 1044 for several tasks, etc. Such devices may be connected via a system bus 1048 along with read-only memory (ROM) 1045, random access memory 1046, and internal mass storage devices such as hard drives, SSDs, etc. 1047 that are not accessible to the internal user. In some computer systems, the system bus 1048 may be accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus 1048 or via a peripheral bus 1049. Peripheral bus architectures include PCI, USB, etc. A graphics adapter 1050 may be included in the core 1040.

[0114] The CPU 1041, GPU 1042, FPGA 1043, and accelerator 1044 can execute a combination of several instructions that can create the aforementioned computer code. The computer code can be stored in ROM 1045 or RAM 1046. Transient data can also be stored in RAM 1046, while persistent data can be stored, for example, in the internal mass storage device 1047. By using cache memory, it becomes possible to quickly store and retrieve data from any memory device, and it can be closely associated with one or more CPUs 1041, GPUs 1042, mass storage devices 1047, ROM 1045, RAM 1046, etc.

[0115] Computer-readable media may contain computer code for performing various computer implementations. The media and computer code may be specifically designed and constructed for the purposes of this disclosure, or they may be of a type readily available and well-known to those skilled in computer software technology.

[0116] As an example, and not for the purpose of limiting, a computer system having architecture 1000, specifically a core 1040, can provide functionality resulting from (multiple) processors (CPU, GPU, FPGA, accelerator, etc.) that execute software embodied in one or more tangible computer-readable media. Such computer-readable media may be user-accessible mass storage devices as described above, as well as media associated with some non-temporary storage devices of the core 1040, such as the core internal mass storage device 1047 or ROM 1045. Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core 1040. The non-temporary computer-readable media may include one or more memory devices or chips, depending on the specific needs. The software can cause the core 1040 and specifically the processors within it (including CPU, GPU, FPGA, etc.) to execute specific processes, or specific parts of specific processes, as described herein, including defining data structures stored in RAM 1046 and modifying such data structures according to processes defined by the software. In addition to or instead of the above, a computer system may provide functionality as a result of logic wired to or embodied in a circuit (e.g., accelerator 1044) and may operate in place of or with software to perform a particular process or a particular part of a particular process described herein. References to software may, as necessary, include logic, and vice versa. References to computer-readable media may, as necessary, include circuits that store software for execution (such as integrated circuits (ICs)), circuits that embody logic for execution, or both. The disclosure encompasses any suitable combination of hardware and software.

[0117] While this disclosure describes several non-limiting embodiments, there are many variations, permutations, and alternative equivalents that fall within the scope of this disclosure. Those skilled in the art will therefore understand that numerous systems and methods, not expressly shown or described herein, can be devised to embody the principles of this disclosure and thus fall within the spirit and scope of this disclosure. [Explanation of symbols]

[0118] 100 Communication Systems 110 terminals 120 devices 130 devices 140 devices 150 Networks 200 Streaming Systems 201 Video Sources 202 3D meshes and streams containing metadata associated with the 3D meshes 203 Encoder 204 encoded video bitstream 205 Streaming Servers 206 Streaming Clients 209 The video bitstream is the copy that is input. 210 Video Decoders 211 Output video sample stream that can be rendered 212 displays 213 Capture Subsystem 300 Frameworks 301 Encoder 305 Input Mesh Mesh with 310 UV Atlas 315 Occupation Map 320 Geometry Maps 325 Attribute Maps 330 Metadata 335 Decrypted Occupation Map 340 decoded geometry maps 345 Decrypted attribute map 350 decrypted metadata 351 Decoder 360 Reconstructed Mesh 400 Triangular Mesh 600 Reconstructed mesh generated using the combination of implicit rules and explicit signaling described in Figure 5 700 UV Atlas Process for encoding 2D UV charts using 800 atlas sampling-based dynamic mesh compression 1000 Computer Systems 1001 Keyboard 1002 Mouse 1003 Trackpad 1005 Joystick 1006 Microphone 1007 Scanner 1008 Camera 1009 Speaker Screens including 1010 CRT screens, LCD screens, plasma screens, and OLED screens. 1020 CD / DVD ROM / RW 1021 CD / DVD and other media 1022 USB memory sticks (thumb drives) 1023 Removable hard drive or solid-state drive 1040 cores 1041 CPU 1042 GPU 1043 Field Programmable Gate Area 1044 Hardware Accelerators 1045 Read-only memory 1046 random access memory 1047 Mass storage 1048 System Bus 1049 Local buses 1050 Graphics Adapter 1054 Network Interface 1055 Cloud Computing Environment

Claims

1. A method for dynamic mesh compression, which is performed by at least one processor, The steps include receiving a mesh frame having polygons representing the surface of an object, The steps of determining that the mesh frame is associated with an irregular UV chart in response to a component representing one or more unique vertices within the mesh frame, The steps include: performing boundary detection on the UV chart to identify one or more boundary loops within the UV chart; The steps include compressing the information of one or more identified boundary loops into a bitstream. Methods that include...

2. The method according to claim 1, wherein the UV chart is irregular if the UV chart is non-manifold and non-simple connection.

3. The method according to claim 1, wherein the mesh frame is determined to be associated with an irregular UV chart when the mesh frame has at least one non-manifold vertex.

4. The method according to claim 3, wherein a vertex in the mesh frame is non-manifold when the vertex has three or more connecting boundary edges.

5. The step of performing boundary detection on the UV chart to identify one or more boundary loops in the UV chart, The steps include determining the Euler properties of the UV chart, A step of identifying one or more boundary loops in the UV chart based on the determined Euler characteristics of the UV chart. The method according to claim 1, further comprising:

6. A computing device, At least one memory configured to store program code, At least one processor configured to execute the program code described above and to perform the method described in any one of claims 1 to 5, A computing device equipped with [a certain feature].

7. A computer program, when executed by one or more processors of a computing device, includes instructions causing the computing device to perform the method according to any one of claims 1 to 5.

Images