Adaptive Geometry Filtering for Mesh Compression

Abstract

The method and apparatus include computer code configured to cause one or more processors to determine one or more vertices in an input mesh and group the one or more vertices into one or more vertex groups. The grouping of each vertex in each group may be based on a topological distance of the each vertex. In embodiments, the one or more processors may also determine sets of filtering coefficients for the one or more vertex groups and signal the one or more vertex groups and the sets of filtering coefficients.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 416,382, filed October 14, 2022, and U.S. Application No. 18 / 485,770, filed October 12, 2023, the disclosures of which are incorporated herein by reference in their entireties.

[0002] FIELD OF THE DISCLOSURE Embodiments of the present disclosure relate to video coding and decoding, and more particularly to encoding and decoding meshes, including adaptive geometry filtering in mesh motion vector coding. [Background technology]

[0003] Advanced three-dimensional (3D) representations of the world are enabling more immersive interaction and communication. To achieve the sense of realism of 3D representations, 3D models are becoming ever more sophisticated, and a significant amount of data is tied to the creation and consumption of these 3D models. 3D meshes are widely used in 3D model immersive content.

[0004] A 3D mesh may consist of several polygons that describe the surface of a volumetric object. Dynamic mesh sequences may require large amounts of data, as they may contain a significant amount of information that changes over time. Therefore, efficient compression techniques are needed to store and transmit such content.

[0005] Mesh compression standards IC, MESHGRID, and FAMC were previously developed to address dynamic meshes with constant connectivity and time-varying geometry and vertex attributes. However, these standards do not consider time-varying attribute maps and connectivity information.

[0006] Furthermore, it is also difficult for volumetric acquisition techniques to generate always-on dynamic meshes, especially under real-time constraints, and this type of dynamic mesh content is not supported by existing standards.

[0007] As another example, glTF (GL Transmission Format) is a standard under development by the Khronos Group for efficient transmission and loading of 3D scenes and models by applications. glTF aims to minimize both the size of 3D assets and the runtime processing required to decompress them. A geometry compression extension to glTF 2.0 using Google Draco technology is being developed to reduce the size of glTF models and scenes. Summary of the Invention [Means for solving the problem]

[0008] According to one embodiment, a method and apparatus are provided that include computer code configured to cause one or more processors to perform the steps of: determining one or more vertices in an input mesh, the input mesh representing volumetric data of at least one three-dimensional (3D) visual content; grouping the one or more vertices into one or more vertex groups, the grouping of each vertex in each group being based on a topological distance of the each vertex; determining sets of filtering coefficients for the one or more vertex groups; and signaling the one or more vertex groups and the sets of filtering coefficients.

[0009] According to one embodiment, a method and apparatus are provided that include computer code configured to cause one or more processors to perform the steps of: receiving a coded bitstream associated with a mesh, the coded bitstream including information about vertices in the mesh and filtering coefficients associated with the vertices; obtaining from the coded bitstream one or more vertex groups included in the mesh; obtaining from the coded bitstream sets of filtering coefficients for the one or more vertex groups; generating a reconstructed mesh based on information about vertices in the mesh; and generating a refined reconstructed mesh using the reconstructed mesh, the one or more vertex groups, and the sets of filtering coefficients for the one or more vertex groups. [Brief explanation of the drawings]

[0010] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings. [Figure 1] FIG. 1 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a schematic diagram of a simplified block diagram of a streaming system according to an embodiment of the present disclosure. [Figure 3] FIG. 2 is a schematic diagram of a simplified block diagram of a video encoder and a video decoder according to an embodiment of the present disclosure. [Figure 4A] FIG. 10 is an exemplary diagram of UV parameterized mapping from a 3D mesh segment to a 2D chart, according to an embodiment of the present disclosure. [Figure 4B] FIG. 10 is an exemplary diagram of UV parameterized mapping from a 3D mesh segment to a 2D chart, according to an embodiment of the present disclosure. [Figure 5]1 is a flowchart of a process for encoding adaptive filtering coefficients according to an embodiment of the present disclosure. [Figure 6] 10 is a flowchart of a process for generating a reconstructed mesh using signaled adaptive filtering coefficients according to an embodiment of the present disclosure. [Figure 7] FIG. 1 illustrates an exemplary computer system suitable for implementing embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0011] The proposed features described below may be used separately or combined in any order. Furthermore, the embodiments may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0012] 1 shows a simplified block diagram of a communication system 100 according to one embodiment of the present disclosure. The communication system 100 may include at least two terminals 102 and 103 interconnected via a network 105. In the case of unidirectional data transmission, a first terminal 103 may locally encode video data for transmission to the other terminal 102 via the network 105. The second terminal 102 may receive the other terminal's coded video data from the network 105, decode the coded data, and display the recovered video data. Unidirectional data transmission may be common in media serving applications, for example.

[0013] 1 shows a second pair of terminals 101 and 104 provided to support two-way transmission of coded video, such as may occur during a video conference. For two-way transmission of data, each terminal 101 and 104 can code video data captured at a local location for transmission to the other terminal over network 105. Each terminal 101 and 104 can also receive coded video data transmitted by the other terminal, can decode the coded data, and can display the recovered video data on a local display device.

[0014] In FIG. 1 , terminals 101, 102, 103, and 104 may be depicted as servers, personal computers, and smartphones, although the principles of the present disclosure are not limited thereto. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. Network 105 represents any number of networks that convey coded video data between terminals 101, 102, 103, and 104, including, for example, wired and / or wireless communication networks. Communications network 105 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of network 105 may not be important to the operation of the present disclosure, unless otherwise described herein.

[0015] 2 illustrates the placement of a video encoder and a video decoder in a streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, and storage of compressed video on digital media, including CDs, DVDs, memory sticks, etc.

[0016] The streaming system may include a capture subsystem 203, which may include a video source 201, such as a digital camera, that creates an uncompressed video sample stream 213. The sample stream 213 may be emphasized as a high amount of data compared to an encoded video bitstream and may be processed by an encoder 202 coupled to the video source 201, which may be the camera described above. The encoder 202 may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream 204, which may be emphasized as a lower amount of data compared to the sample stream, may be stored on a streaming server 205 for future use. One or more streaming clients 212 and 207 may access the streaming server 205 to retrieve copies 208 and 206 of the encoded video bitstream 204. The client 212 may include a video decoder 211 that decodes the incoming copy of the encoded video bitstream 208 and creates an outgoing video sample stream 210, which may be rendered on a display 209 or another rendering device (not shown). In some streaming systems, video bitstreams 204, 206, and 208 may be encoded according to particular video coding / compression standards, examples of which are mentioned above and further described herein.

[0017] According to exemplary embodiments described further below, the term "mesh" refers to a configuration of one or more polygons that describe the surface of a volumetric object. Each polygon is defined by its vertices in 3D space and information about how the vertices are connected, called connectivity information. Optionally, vertex attributes, such as color, normals, etc., can be associated with mesh vertices. Attributes can also be associated with the surface of a mesh by utilizing mapping information that parameterizes the mesh in a 2D attribute map. Such mapping may be described by a set of parametric coordinates, called UV coordinates or texture coordinates, associated with the mesh vertices. The 2D attribute map is used to store high-resolution attribute information, such as texture, normals, and displacement. Such information can be used for various purposes, such as texture mapping and shading, according to exemplary embodiments.

[0018] Nevertheless, dynamic mesh sequences can require large amounts of data because they can consist of a significant amount of information that changes over time. For example, in contrast to a “static mesh” or “static mesh sequence,” in which the mesh information remains constant from one frame to another, a “dynamic mesh” or “dynamic mesh sequence” exhibits the changing motion of the vertices represented by the mesh from one frame to another. Therefore, efficient compression techniques are required for storing and transmitting such content. The mesh compression standards IC, MESHGRID, and FAMC were previously developed by MPEG to address dynamic meshes with constant connectivity and time-varying geometry and vertex attributes. However, these standards do not consider time-varying attribute maps and connectivity information. Digital Content Creation (DCC) tools typically generate such dynamic meshes. Conversely, generating constant-connectivity dynamic meshes presents a challenge for volumetric acquisition techniques, especially under real-time constraints. This type of content is not supported by existing standards. According to exemplary embodiments herein, aspects of a new mesh compression standard are described to directly handle dynamic meshes with time-varying connectivity information and, optionally, time-varying attribute maps, targeting lossy and lossless compression for various applications such as real-time communication, storage, free-viewpoint video, AR and VR, etc. Functionalities such as random access and scalable / progressive coding are also considered.

[0019] Figure 3 illustrates an exemplary framework 300 for dynamic mesh compression, such as a 2D atlas sampling-based method. Each frame of an input mesh 301 can be preprocessed by a series of operations, such as tracking, remeshing, parameterization, and voxelization. Note that these operations can be encoder-only, meaning they may not be part of the decoding process; such a possibility can be signaled in the metadata by a flag indicating 0 for encoder-only and 1 for other operations. A mesh with a 2D UV atlas 302 can then be obtained, with each vertex of the mesh having one or more associated UV coordinates on the 2D atlas. The mesh can then be converted into multiple maps, including a geometry map and an attribute map, by sampling on the 2D atlas. These 2D maps can then be coded by a video / image codec, such as HEVC, VVC, AV1, or AVS3. At the decoder 303 side, a mesh can be reconstructed from the decoded 2D maps. Optional post-processing and filtering can also be applied to the reconstructed mesh 304. Note that other metadata may be signaled to the decoder side for the purpose of 3D mesh reconstruction. Note that chart boundary information, including the uv and xyz coordinates of boundary vertices, can be predicted, quantized, and entropy coded in the bitstream. The quantization step size can be set on the encoder side to trade off between quality and bitrate.

[0020] In some implementations, a 3D mesh can be partitioned into segments (or patches / charts), and according to an exemplary embodiment, one or more 3D mesh segments may be considered a "3D mesh." Each segment consists of a set of connected vertices associated with geometry, attributes, and connectivity information. As shown in the volumetric data example 400 in FIG. 4A, a UV parameterization process 402 that maps 3D mesh segments to 2D charts, such as the 2D UV atlas 302 block described above, maps one or more mesh segments 401 to 2D charts 403 in the 2D UV atlas 404. Each vertex (v) in a mesh segment is n ) may be assigned 2D UV coordinates in the 2D UV atlas. n ) form connected components as their 3D counterparts. The geometry, attributes, and connectivity information of each vertex can be inherited from their 3D counterparts as well. For example, vertex v4 may indicate that it is directly connected to vertices v0, v5, v1, and v3, as well as each of the other vertices. Furthermore, such a 2D textured mesh, according to an exemplary embodiment, would further indicate information such as color information on a per-patch basis with each triangular patch, e.g., v2, v5, and v3 as one "patch."

[0021] For example, see example 450 of FIG. 4B, where a 3D mesh segment 451 can also be mapped to multiple separate 2D charts 451 and 452, as features of example 400 of FIG. 4A. In this case, a 3D vertex may correspond to multiple vertices in the 2D UV atlas. As shown in FIG. 4B, the same 3D mesh segment is mapped to multiple 2D charts in the 2D UV atlas, rather than a single chart as in FIG. 4A. For example, 3D vertices v1 and v4 are mapped to v1, v4, v5, v6, v7, v8, v9, v10, v11, v12, v13, v14, v15, v16, v17, v18, v19, v19, v20, v21, v22, v23, v24, v25, v26, v27, v28, v29, v29, v30, v31, v32, v33, v34, v35, v36, v37, v38, v39, v39, v40, v41, v42, v43, v44, v45, v46, v47, v48, v49, v41, v410, v411, v412, v413, v414, v42, v43, v44, v45, v46, v47, v48, v49, v49, v415, v416, v417, v418, v419, v420, v421, v422, v43, v44, v45, v46, v47, v48, v49, v49, v419, v419, v421, v43, v44, v 1' and v4, v 4'Thus, a typical 2D UV atlas of a 3D mesh may consist of multiple charts, each of which may contain multiple (usually three or more) vertices associated with their 3D geometry, attributes, and connectivity information.

[0022] FIG. 4B shows an example 453 illustrating the derived triangulation for a chart with boundary vertices B0, B1, B2, B3, B4, B5, B6, and B7. Given such information, any triangulation method can be applied to create connectivity between vertices (including boundary vertices and sampled vertices). For example, for each vertex, find the two closest vertices. Alternatively, for every vertex, successively generate triangles until a minimum number of triangles is achieved after a set number of attempts. As shown in example 453, there are repeating triangles of various regular shapes and various irregularly shaped triangles, each with its own unique dimensions, which are generally closest to the boundary vertex and may or may not be shared with any of the other triangles. Connectivity information can also be reconstructed by explicit signaling. If a polygon cannot be reconstructed by implicit rules, the encoder can signal connectivity information in the bitstream, according to an exemplary embodiment.

[0023] Boundary vertices B0, B1, B2, B3, B4, B5, B6, B7 are defined in 2D UV space. A boundary edge can be determined by checking whether the edge appears in only one triangle. According to an exemplary embodiment, the following information of a boundary vertex is important and should be signaled in the bitstream: geometry information, e.g., 3D XYZ coordinates (albeit currently in 2D UV parametric form) and 2D UV coordinates.

[0024] When a 3D bounding vertex corresponds to multiple vertices in a 2D UV atlas, as shown in Figure 4B, the mapping from 3D XYZ to 2D UV can be one-to-many. Therefore, a UV-to-XYZ (or called UV2XYZ) index can be signaled to indicate the mapping function. UV2XYZ can be a 1D array of indices that map each 2D UV vertex to a 3D XYZ vertex.

[0025] As mentioned above, dynamic mesh sequences may consist of a significant amount of information that changes over time, and therefore may require a large amount of data. The sampling-based methods used in related art may introduce several artifacts into the reconstructed mesh geometry that may degrade visual quality. Therefore, there is a need to develop efficient algorithms that reduce such artifacts and improve the quality of the reconstructed mesh.

[0026] The embodiments of the present disclosure may be used separately or combined in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders may be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0027] Embodiments of the present disclosure relate to a number of methods for filtering reconstructed geometry in mesh compression. Those skilled in the art will recognize that the methods disclosed herein may be applied individually or in any combination. Note that the methods may be applied to static meshes, where the mesh has only one frame or the mesh content does not change over time.

[0028] An advantage of the adaptive geometry filtering disclosed herein is that it improves the quality of the reconstructed mesh by signaling sets of filtering coefficients. The filtering coefficients can be trained at the encoder side based on the reconstructed mesh and the corresponding reference mesh, and each set of filtering coefficients can be trained on a group of similar vertices. The filtering coefficients can then be applied to each vertex and its neighboring vertices.

[0029] TIFF2025531301000002.tif34137

[0030] As a first step, the vertices can be classified into groups.

[0031] TIFF2025531301000003.tif19134

[0032] TIFF2025531301000004.tif16135

[0033] TIFF2025531301000005.tif15135

[0034] TIFF2025531301000006.tif12136

[0035] Thus, according to one embodiment, adaptive geometry filtering may include determining one or more vertices in an input mesh and grouping the one or more vertices into one or more vertex groups. Sets of filtering coefficients may then be determined for the one or more vertex groups. The one or more vertex groups and the sets of filtering coefficients may then be encoded and signaled. In some embodiments, they may be encoded within the same bitstream. In some embodiments, the sets of filtering coefficients may be encoded and signaled as metadata.

[0036] According to an embodiment of the present disclosure, adaptive geometry filtering can be applied at the encoder side as follows.

[0037] The vertices can be classified into groups using exactly the same method as the encoder. In some embodiments, the grouping method is agreed upon by the encoder and decoder. In some embodiments, the agreed upon method may include grouping each vertex into a respective group based on the topological distance of each vertex.

[0038] Then, for the i-th vertex group, the filter coefficients can be decoded from the bitstream and dequantized in the same way as in the encoder.

[0039] TIFF2025531301000007.tif24135

[0040] Thus, according to one embodiment, adaptive geometry filtering applied at a decoder may include receiving a coded bitstream associated with a mesh, the coded bitstream including information about vertices in the mesh and filtering coefficients associated with the vertices, and obtaining from the coded bitstream one or more vertex groups included in the mesh and sets of filtering coefficients for the one or more vertex groups. A reconstructed mesh may then be constructed based on the information about the vertices in the mesh, and the reconstructed mesh may be refined using the one or more vertex groups and sets of filtering coefficients for the one or more vertex groups.

[0041] Vertex Grouping

[0042] Vertex grouping can classify the vertices of the reconstructed mesh into groups based on various characteristics, as long as the same process can be performed on the decoder side.

[0043] In one embodiment, vertices may be grouped into regions based on the topology of the reconstructed mesh, with each vertex group having approximately the same number of vertices connected via edges. In some embodiments, topological distance, which measures the structural distance between two graphs or points in a metric space, such as the number of edges between two points, or Euclidean distance, may be used for grouping.

[0044] In the same or another embodiment, vertices may be grouped based on various properties that can be derived from the reconstructed mesh, which may include, but are not limited to, vertex curvature, vertex normal direction, vertex level of detail layer, etc.

[0045] Vertex Adjacency

[0046] TIFF2025531301000008.tif15138

[0047] TIFF2025531301000009.tif28139

[0048] TIFF2025531301000010.tif16135

[0049] Derivation of filters and coefficients

[0050] TIFF2025531301000011.tif21141

[0051] Depending on the embodiment, several different methods can be used to define the filter coefficients and the error function to be minimized.

[0052] Adaptive Laplacian Filter with Point-to-Point Loss Function

[0053] TIFF2025531301000012.tif13134

[0054]

number

[0055] TIFF2025531301000014.tif14155

[0056] TIFF2025531301000015.tif13142

[0057]

number

[0058] The above equation can be solved by any suitable method, such as the least squares method.

[0059] Adaptive Laplacian Filter with Point-to-Plane Loss Function

[0060] In the same or another embodiment, the same adaptive Laplacian filter as in the last embodiment may be applied, but the filter coefficients are optimized with a minimum point-to-plane error function.

[0061]

number

[0062] TIFF2025531301000018.tif25170

[0063] Adaptive Wiener Filter with Point-to-Point Loss Function

[0064] In the same or another embodiment, an adaptive Wiener filter can be applied as follows.

[0065]

number

[0066] TIFF2025531301000020.tif17137

[0067]

number

[0068] The above equations can also be solved by any suitable method, such as the least squares method.

[0069] Adaptive Wiener Filter with Point-to-Plane Loss Function

[0070] In the same or another embodiment, the same adaptive Wiener filter as in the last embodiment may be applied, but the filter coefficients may be optimized with a minimum point-to-plane error function.

[0071]

number

[0072] TIFF2025531301000023.tif24170

[0073] Thus, as disclosed herein, the encoder may use any combination of filter functions and loss functions. In embodiments, the encoder and decoder may agree on which filter functions and which loss functions to use. In embodiments, the encoder may signal a flag indicating whether all groups use the same filter function, the same loss function, different filter functions, or different loss functions. If different filter functions or loss functions may be used by the groups, the encoder may signal additional flags or information indicating the filter function and / or loss function for each group. In some embodiments, this signaling completely precedes the signaling of the vertex groups, while in some embodiments, this signaling precedes each vertex group.

[0074] Optimizing the filter coefficients

[0075] The filter coefficients can be fine-tuned before encoding based on the estimated rate-distortion performance. The distortion term (D) may be estimated by either a point-to-point loss function or a point-to-plane loss function, as described herein. The rate term (R) can be estimated from the distribution of the coefficients.

[0076] In one embodiment, the filter may be disabled for the i-th vertex group if the distortion term is greater than a threshold, where the threshold may be a function of the quantization step size of the filter coefficients.

[0077] TIFF2025531301000024.tif18138

[0078] Coding of Filter Coefficients

[0079] The quantized and optimized filter coefficients can then be entropy coded in a variety of ways within the bitstream.

[0080] In one embodiment, first, a binary flag is coded to indicate whether the filter is applied to the i-th group. If the flag is true, then the coefficient is coded. Otherwise, the coefficient is not coded. This binary flag can be coded with or without context in arithmetic coding.

[0081] Each coefficient value can be coded by fixed length coding or Exp-Golomb coding, for example.

[0082] In one embodiment, the coefficient values ​​may be coded as follows: one binary flag is coded to indicate whether the value is equal to 0; if this flag is false, then another binary flag is coded to indicate whether the value is equal to 1; if this flag is also false, then that value minus 2 is coded using Exp-Golomb coding.

[0083] In some embodiments, some of the coefficients may be predicted from other coefficients before encoding, for example, when an adaptive Wiener filter is applied, the first coefficient may be predicted from other coefficients.

[0084] FIG. 5 is a flowchart of a process 500 illustrating mesh compression, according to an embodiment of the present disclosure.

[0085] More than one vertex in an input mesh may be determined in operation 505. The input mesh may represent volumetric data of at least one three-dimensional (3D) visual content.

[0086] More than one vertex may be grouped into more than one vertex group at operation 510. In an embodiment, the grouping of each vertex in each group may be based on the topological distance of each vertex.

[0087] In an embodiment, grouping each vertex based on topological distance may include: determining neighboring vertices for each vertex, where neighboring vertices are vertices connected to the respective vertex by at least one edge; and grouping each vertex into a first group such that vertices in the first group are connected to the same number of vertices via edges.

[0088] In an embodiment, determining neighboring vertices of each vertex may include one or more of: determining neighboring vertices that are directly connected to each edge in the input mesh; or determining neighboring vertices that are connected to each vertex by multiple edges, where the multiple edges are less than or equal to a first threshold.

[0089] In operation 515, a set of filtering coefficients may be determined for more than one vertex group.

[0090] In an embodiment, determining the set of filtering coefficients may include determining the set of filtering coefficients based on the first loss function and the first filter function, wherein the set of filtering coefficients includes one or more filter coefficients for each group of the more than one vertex group.

[0091] In an embodiment, the first loss function may include one of a point-to-point loss function or a point-to-plane loss function. In the same or a different embodiment, the first filter function may include one of a Laplacian filter or a Weiner filter.

[0092] In some embodiments, following determining the set of filter coefficients, process 500 may include disabling a first filter coefficient for a first group of the more than one vertex group based on the first filter coefficient having a distortion higher than a distortion threshold.

[0093] In operation 520, more than one vertex group and set of filtering coefficients can be signaled.

[0094] In some embodiments, operation 520 may include signaling, for each group of the more than one vertex groups, a binary flag indicating whether the filter coefficient for the respective group is enabled, and signaling one or more coefficients associated with the respective group based on the binary flag for the respective group indicating that the filter coefficient is enabled. Operation 520 may also include signaling a second flag indicating whether the same first filter function is used to determine the set of filter coefficients for each group in the more than one vertex groups. In some embodiments, operation 520 may also include signaling a third flag indicating whether the same first loss function is used to determine the set of filter coefficients for each group in the more than one vertex groups.

[0095] FIG. 6 is a flowchart of a process 600 illustrating mesh reconstruction according to an embodiment of the present disclosure.

[0096] At operation 605, a coded bitstream associated with the mesh may be received. The coded bitstream may include information about vertices in the mesh and filtering coefficients associated with the vertices. At operation 610, more than one vertex group included in the mesh may be obtained from the coded bitstream. At operation 615, sets of filtering coefficients for the more than one vertex group may be obtained from the coded bitstream. At operation 620, a reconstructed mesh may be generated based on the information about the vertices in the mesh. In embodiments, the reconstructed mesh may be further refined, and at operation 625, a refined reconstructed mesh may be constructed using the reconstructed mesh, the more than one vertex group, and the sets of filtering coefficients for the more than one vertex group.

[0097] The proposed methods may be used separately or combined in any order. Although only triangular meshes were used to demonstrate various embodiments, the proposed methods can be used for any polygonal mesh. As mentioned above, it is assumed that the input mesh may contain one or more instances, a sub-mesh is a portion of the input mesh with one instance or multiple instances, and multiple instances can be grouped to form a sub-mesh.

[0098] The techniques described above can be implemented as computer software using computer-readable instructions, physically stored on one or more computer-readable media, or implemented by one or more specially configured hardware processors. For example, Figure 7 illustrates a computer system 700 suitable for implementing certain embodiments of the present disclosure.

[0099] Computer software may be coded using any suitable machine code or computer language that can be assembled, compiled, linked, or similar mechanisms to create code containing instructions that can be executed by a computer central processing unit (CPU), graphics processing unit (GPU), etc., directly, or via interpretation, microcode execution, etc.

[0100] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0101] 7 are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of the computer software implementing embodiments of the present disclosure. The arrangement of components should not be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of computer system 700.

[0102] Computer system 700 may also include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, via tactile input (e.g., keystrokes, swipes, data glove movements, etc.), audio input (e.g., voice, clapping, etc.), visual input (e.g., gestures, etc.), or olfactory input (not shown). Human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (e.g., voice, music, ambient sounds, etc.), images (e.g., scanned images, photographic images obtained from a still camera, etc.), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic vision, etc.).

[0103] The input human interface devices may include one or more of a keyboard 701, a mouse 702, a trackpad 703, a touchscreen 710, a joystick 705, a microphone 706, a scanner 708, and a camera 707 (only one of each is shown).

[0104] The computer system 700 may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through haptic output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., a touchscreen 710, or a haptic feedback device that provides haptic feedback via a joystick 705 but does not act as an input device), audio output devices (e.g., speakers 709, headphones (not shown)), visual output devices (e.g., screens 710, including CRT screens, LCD screens, plasma screens, and OLED screens (each of which may or may not have touchscreen input capabilities, each of which may or may not have haptic feedback capabilities, some of which may output two-dimensional visual output or three-dimensional or higher-dimensional output by means such as stereographic output), virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0105] The computer system 700 may also include human-accessible storage devices and their associated media, such as optical media including, for example, a CD / DVD ROM / RW 720 with a CD / DVD 711 or similar media, a thumb drive 722, a removable hard drive or solid state drive 723, legacy magnetic media (not shown) such as tape and floppy disks, and dedicated ROM / ASIC / PLD-based devices (not shown) such as security dongles.

[0106] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not encompass transmission media, carrier waves, or other transitory signals.

[0107] The computer system 700 may also include an interface 799 to one or more communication networks 798. The network 798 may be, for example, wireless, wired, or optical. The network 798 may further include local networks, wide area networks, metropolitan area networks, vehicular and industrial networks, real-time networks, delay-tolerant networks, and the like. Examples of networks 798 include Ethernet, WLAN, GSM, cellular networks including 3G, 4G, 5G, LTE, and the like, television wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, vehicular and industrial networks including CANBus, and the like. Some networks 798 generally require an external network interface adapter connected to a specific general-purpose data port or peripheral bus (750 and 751) (e.g., a USB port on the computer system 700), while others are typically integrated into the core of the computer system 700 by connecting to a system bus (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system), as described below. Using any of these networks 798, computer system 700 can communicate with other entities. Such communication may be one-way receive only (e.g., broadcast TV), one-way transmit only (e.g., from the CAN bus to a particular CAN bus device), or two-way, e.g., to other computer systems using a local or wide area digital network. As noted above, specific protocols and protocol stacks may be used with each of these networks and network interfaces.

[0108] The aforementioned human interface devices, human-accessible storage devices, and network interfaces may be connected to core 740 of computer system 700 .

[0109] The core 740 may include one or more central processing units (CPUs) 741, graphics processing units (GPUs) 742, graphics adapters 717, dedicated programmable processing units in the form of field programmable gate arrays (FPGAs) 743, hardware accelerators 744 for specific tasks, etc. These devices may be connected via a system bus 748, along with read-only memory (ROM) 745, random access memory 746, and internal mass storage 747, e.g., an internal hard disk, SSD, or the like, that is not user accessible. In some computer systems, the system bus 748 is accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices can be connected directly to the core's system bus 748 or via a peripheral bus 749. Peripheral bus architectures include PCI, USB, etc.

[0110] The CPU 741, GPU 742, FPGA 743, and accelerator 744 may execute several instructions, which may combine to constitute the aforementioned computer code. The computer code may be stored in ROM 745 or RAM 746. Transient data may also be stored in RAM 746, while permanent data may be stored, for example, in internal mass storage 747. The use of cache memory, which may be closely associated with one or more of the CPU 741, GPU 742, mass storage 747, ROM 745, RAM 746, etc., may enable fast storage and retrieval from any memory device.

[0111] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0112] By way of example, and not limitation, a computer system having architecture 700, and in particular core 740, can provide functionality as a result of the processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage, as described above, as well as specific storage of core 740 that is non-transitory in nature, such as core internal mass storage 747 or ROM 745. Software implementing various embodiments of the present disclosure can be stored on such devices and executed by core 740. Computer-readable media can include one or more memory devices or chips, depending on particular needs. The software can cause core 740, and in particular the processor therein (including a CPU, GPU, FPGA, etc.), to perform particular processes or particular portions of particular processes described herein, including defining data structures stored in RAM 746 and modifying such data structures in accordance with the software-defined processes. Additionally, or alternatively, a computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator 744), which may operate in place of or together with software to perform particular processes or portions of particular processes described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry (e.g., integrated circuits (ICs)) that stores software for execution, circuitry that embodies logic for execution, or both. The present disclosure encompasses any appropriate combination of hardware and software.

[0113] While this disclosure has described several exemplary embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope.

Claims

1. 1. A method for mesh compression executed by at least one processor, comprising: determining one or more vertices in an input mesh, the input mesh representing volumetric data of at least one three-dimensional (3D) visual content; grouping the one or more vertices into one or more vertex groups, the grouping of each vertex in each group being based on the topological distance of the each vertex; determining a set of filter coefficients for the more than one vertex group; signaling the more than one vertex group and the set of filter coefficients.

2. The step of grouping the respective vertices based on the topological distance includes: determining neighboring vertices for each of the vertices, the neighboring vertices being vertices connected to the respective vertex by at least one edge; and grouping the respective vertices into a first group such that the vertices in the first group are connected to the same number of vertices via edges.

3. The step of determining the neighboring vertices of each vertex comprises: determining the neighboring vertices directly connected to each edge in the input mesh; or 3. The method of claim 2, further comprising one of the steps of: determining the neighboring vertices connected to the respective vertex by a plurality of edges, the plurality of edges being less than or equal to a first threshold.

4. The step of determining the set of filter coefficients comprises: determining the set of filter coefficients based on a first loss function and a first filter function; the set of filter coefficients includes one or more filter coefficients for each group of the more than one vertex groups; The method of claim 1 , wherein each coefficient in the set of filter coefficients is associated with one or more vertices in the more than one vertex group.

5. the first loss function comprises one of a point-to-point loss function or a point-to-plane loss function; The method of claim 4 , wherein the first filter function comprises one of a Laplacian filter or a Weiner filter.

6. Following the step of determining the set of filter coefficients, the method further comprises:

2. The method of claim 1 , further comprising: disabling a first filter coefficient for a first group of the more than one vertex groups based on the first filter coefficient having a distortion higher than a distortion threshold.

7. The step of signaling the more than one vertex group and the set of filter coefficients comprises: signaling, for each of the more than one vertex groups, a binary flag indicating whether the filter coefficients for the respective group are enabled; and signaling one or more coefficients associated with the respective group based on the binary flag for the respective group indicating that the filter coefficient is enabled.

8. The method comprises:

8. The method of claim 7, further comprising signaling a second flag indicating whether the same first filter function is used to determine the set of filter coefficients for each group in the more than one vertex group.

9. The method comprises:

8. The method of claim 7, further comprising signaling a third flag indicating whether the same first loss function is used to determine the set of filter coefficients for each group in the more than one vertex group.

10. 1. An apparatus for an adaptive geometry filter for mesh compression, comprising: at least one memory configured to store program code; at least one processor configured to read the program code and to act as directed by the program code; The program code first decision code configured to cause the at least one processor to determine more than one vertex in an input mesh, the input mesh representing volumetric data of at least one three-dimensional (3D) visual content; and a first grouping code configured to cause the at least one processor to group the one or more vertices into one or more vertex groups, wherein the grouping of each vertex in each group is based on a topological distance of the each vertex; and second decision code configured to cause the at least one processor to determine sets of filter coefficients for the more than one vertex group; and a first signaling code configured to cause the at least one processor to signal the more than one vertex group and the set of filter coefficients.

11. The first grouping code is third decision code configured to cause the at least one processor to determine neighboring vertices for each of the vertices, the neighboring vertices being vertices connected to the respective vertex by at least one edge; and and second grouping code configured to cause the at least one processor to group the respective vertices into first groups such that vertices in the first groups are connected to an equal number of vertices via edges.

12. The third decision code is a fourth decision code configured to cause the at least one processor to determine the neighboring vertices directly connected to each of the edges in the input mesh; or 12. The apparatus of claim 11, further comprising: a fifth decision code configured to cause the at least one processor to determine the neighboring vertices connected to the respective vertex by a plurality of edges, the plurality of edges being less than or equal to a first threshold.

13. The second decision code is and a sixth decision code configured to cause the at least one processor to determine the set of filter coefficients based on a first loss function and a first filter function. the set of filter coefficients includes one or more filter coefficients for each group of the more than one vertex groups; The apparatus of claim 10 , wherein each coefficient in the set of filter coefficients is associated with one or more vertices in the more than one vertex group.

14. The program code Subsequent to determining the set of filter coefficients, 11. The apparatus of claim 10, further comprising: disabling code configured to cause the at least one processor to disable a first filter coefficient for a first group of the more than one vertex groups based on the first filter coefficient having a distortion higher than a distortion threshold.

15. The first signaling code is second signaling code configured to cause the at least one processor to signal, for each group of the more than one vertex groups, a binary flag indicating whether filter coefficients for the respective group are enabled; and and a third signaling code configured to cause the at least one processor to signal one or more coefficients associated with the respective group based on the binary flag for the respective group indicating that the filter coefficient is enabled.

16. The program code 11. The apparatus of claim 10, further comprising: fourth signaling code configured to cause the at least one processor to signal a second flag indicating whether the same first filter function is used to determine the set of filter coefficients for each group in the more than one vertex group.

17. The program code 16. The apparatus of claim 15, further comprising: fifth signaling code configured to cause the at least one processor to signal a third flag indicating whether the same first loss function is used to determine the set of filter coefficients for each group in the more than one vertex group.

18. The program code 16. The apparatus of claim 15, further comprising sixth signaling code configured to cause the at least one processor to signal a third flag indicating whether the same first loss function is used to determine the set of filter coefficients for each group in the more than one vertex group.

19. A non-transitory computer-readable medium having instructions stored thereon, The instructions include one or more instructions, the one or more instructions: When executed by one or more processors of a device for a geometry filter for mesh compression, the device causes the one or more processors to: determining one or more vertices in an input mesh, the input mesh representing volumetric data of at least one three-dimensional (3D) visual content; grouping the one or more vertices into one or more vertex groups, the grouping of each vertex in each group being based on the topological distance of the each vertex; determining a set of filter coefficients for the more than one vertex group; signaling the more than one vertex group and the set of filter coefficients; A non-transitory computer-readable medium for executing the method.

20. The step of grouping the respective vertices based on the topological distance includes: determining neighboring vertices for each of the vertices, the neighboring vertices being vertices connected to the respective vertex by at least one edge; and grouping the respective vertices into first groups such that vertices in the first group are connected to an equal number of vertices via edges.

Images