Efficient end-to-end edge breaker implementation

Abstract

Exemplary embodiments provide methods and apparatus for encoding and decoding mesh data. In the exemplary embodiments, mesh connectivity data is encoded as an ordered sequence of codes using edge breaker techniques. Some embodiments provide pre-processing and post-processing for non-manifold meshes, enabling efficient coding of such meshes as one or more manifold meshes. Some embodiments provide techniques for efficiently coding per vertex and / or per triangle attributes of a mesh. Some embodiments provide arithmetic coding techniques optimized for efficient coding of mesh connectivity and / or attribute data.

Description

Technical Field

[0001] This disclosure relates to the implementation of an efficient end-to-end edgebreaker.

Background Art

[0002] Cross-reference This application claims the priority of European Patent Application No. 23305505.2 filed on April 6, 2023 and European Patent Application No. 23305941.9 filed on June 14, 2023, each of which is incorporated herein by reference in its entirety.

[0003] This disclosure relates to systems and methods for encoding and decoding meshes based on edgebreaker technology. Edgebreaker is a technology capable of efficiently coding the connectivity of a triangular mesh. In its simplest implementation, a mesh encoded using edgebreaker is represented by an ordered sequence composed of symbols C, L, E, R, and S called a "CLERS" sequence. Generally starting from the first triangle, these symbols describe different ways of attaching new triangles and give information about whether and how different edges of the new triangle are connected to one or more of the existing triangles. Edgebreaker technology is described in more detail, for example, in the following sources. · (See, for example, Non-Patent Document 1.) Forward decoding using the corner table (CT) representation of the mesh, which leads to a very compact algorithm, is described. · (See, for example, Non-Patent Document 2.) · (See, for example, Non-Patent Document 3.) Forward decoding using the half-edge representation is described. · (See, for example, Non-Patent Document 4.) · (See, for example, Non-Patent Document 5.) The "spiral reversi" technique that uses the half-edge representation but decodes in the reverse way, leading to faster decoding of O(n), is described.

[0004] Following the MPEG V-Mesh Request for Proposal (now renamed V-DMC), the solution proposed by Apple was selected to form the basis of the MPEG V-Mesh Test Model (trademark). The proposal is described in Non-Patent Document 6. A schematic diagram outlining the proposal is given in Figures 1 and 2 of this disclosure.

[0005] The test model uses a static mesh coder for encoding the base mesh in some frames, which is then subdivided to obtain an approximation of the original mesh. Currently, the V-Mesh test model encodes the base mesh using a Google Draco implementation of an edgebreaker-based mesh encoder. In particular, the implementation uses a "spiral reversi" version of the edgebreaker as described in Non-Patent Document 5.

[0006] While edge breaker coding provides an efficient way to code the connectivity of a base mesh, it has some limitations. For example, edge breaker coding is only effective on its own for manifold surfaces without "holes" or "handles." Additional processing is required to bring such non-manifold surfaces into a format that can be coded using edge breaker techniques. Furthermore, edge breaker coding, on its own, only codes about the connectivity of the mesh. The CLERS sequence itself does not code additional properties of the mesh such as vertex locations and any other attributes of vertices (such as color, normal direction, UV coordinates, etc.). [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] J.Rossignac, “3D compression made simple:Edgebreaker with ZipandWrap on a corner-table,” in Proceedings International Conference on Shape Modeling and Applications, Genova, Italy, 2001

[0008] [Non-Patent Document 2] H. Lopes, G. Tavares, J. Rossignac, A. Szymczak and A. Safonova, “Edgebreaker: a simple compression for surfaces with handles.,” in ACM Symposium on Solid Modeling and Applications, Saarbrucken, 2002.

[0009] [Non-Patent Document 3] J. Rossignac, “Edgebreaker:Connectivity compression for triangle meshes,” GVU center, Georgia Institute of Technology, 1999

[0010] [Non-Patent Document 4] J.Rossigniac,“Course on triangle meshes and corner table,”2006

[0011] [Non-Patent Document 5] M. Isenburg and J. Snoeyink, “Spirale Reversi: Reverse decoding of the Edgebreaker encoding,” Computational Geometry, vol. 20, pp. 39-52, 2001

[0012]

Non-licensed Document 6

[0013]

Non-licensed Document 7

[0014]

Non-licensed Document 8

[0015]

Non-licensed literature 9

[0016]

Non-licensed literature 10

Summary of the Invention

[0017] Some exemplary embodiments support a mesh having a plurality of connected components. In one exemplary embodiment, the method includes encoding a first connected component of the mesh using edgebreaker coding, identifying uncoded corners in the mesh after encoding the first connected component of the mesh, and encoding a second connected component of the mesh using an uncoded corner identified as an entry corner.

[0018] In some embodiments, such a method can be implemented to include identifying at least one non-manifold edge in the mesh, replicating vertices along the non-manifold edge to separate the non-manifold edge into at least two manifold edges, stitching two of the at least two manifold edges together, and encoding the mesh using edgebreaker coding after stitching two of the at least two manifold edges together.

[0019] An exemplary coding method includes identifying at least one non-manifold edge in the mesh, where the non-manifold edge is defined by a parent vertex, preprocessing the mesh by replicating the parent vertex along the non-manifold edge to separate the non-manifold edge into at least two manifold edges, where each manifold edge is defined by a child vertex and each child vertex is replicated from its respective parent vertex, encoding the preprocessed mesh into a bitstream using edgebreaker coding, and providing information identifying the parent vertex of the child vertex in the bitstream.

[0020] An exemplary decoding method includes: obtaining information from a bitstream defining multiple connected components of a non-manifold mesh coded in an edge breaker; obtaining information from a bitstream relating child vertices to parent vertices; decoding at least two of the connected components, each of which has child vertices associated with its respective parent vertex; and reconstructing the non-manifold mesh by duplicating and removing the child vertices.

[0021] An exemplary encoding method comprises preprocessing a mesh having a hole by creating dummy vertices, wherein the dummy vertices are connected to each vertex along the perimeter of the hole; encoding the preprocessed mesh into a bitstream using edge breaker coding; encoding information identifying the dummy vertices into the bitstream; and encoding at least one attribute for each of a plurality of vertices in the preprocessed mesh, wherein the attribute is not encoded for the dummy vertices.

[0022] An exemplary decoding method comprises obtaining information encoding a mesh from a bitstream, obtaining information identifying at least one dummy vertex in the mesh from the bitstream, and decoding attribute information for multiple vertices in the mesh from the bitstream, wherein the attribute information is decoded but not decoded for at least one dummy vertex.

[0023] In some such embodiments, decoding attribute information for multiple vertices involves predicting attribute information for multiple vertices, wherein the attribute information is not predicted for dummy points. The decoder may remove dummy points.

[0024] An exemplary mesh coding method comprises coding the connectivity of the mesh, entropy coding a first table of the starting point positions of vertices in the mesh, and separately entropy coding a second table of the point residuals of vertices in the mesh.

[0025] An exemplary mesh decoding method includes decoding information indicating the connectivity of the mesh, entropy decoding a first table of the starting point positions of vertices in the mesh, and separately entropy decoding a second table of point residuals of vertices in the mesh.

[0026] An exemplary mesh coding method includes obtaining information defining a mesh having multiple connected components (CCs), where each vertex in the CC has a unique global range index; shifting the index of the vertex in the CC from the global range index to the CC range index for each CC; and coding each CC using the CC range index.

[0027] An exemplary mesh decoding method includes obtaining information encoding multiple connected components (CCs), where each vertex in a CC has its own CC range index; decoding each CC; and for each vertex, shifting the vertex index from the CC range index to the global range index.

[0028] An example of a mesh coding method is to obtain information describing a mesh having multiple triangles, each triangle having an attribute, to encode the mesh as an ordered sequence of symbols during edge breaker coding, and to encode the triangle's attribute, each triangle corresponding to one of the relevant symbols, where the attribute of the current triangle is encoded using a prediction based only on the attribute of at least one triangle preceding the current triangle in the ordered sequence of symbols. For example, the attribute of each triangle may be encoded as a delta value for the previous triangle in the sequence (the value of the attribute of the previous triangle acting as a prediction).

[0029] A mesh decoding method according to several embodiments includes obtaining an ordered sequence of symbols representing edge breaker coding of a mesh having a plurality of triangles, wherein each triangle has an attribute, and decoding the mesh and decoding the triangle's attribute, wherein the attribute of the current triangle is decoded using a prediction based only on the attribute of at least one triangle preceding the current triangle in the ordered sequence of symbols. [Brief explanation of the drawing]

[0030] [Figure 1] This is a functional block diagram of an exemplary mesh coding system. [Figure 2] This is a functional block diagram of an exemplary mesh decoding system. [Figure 3] This figure shows an example of an edgebreaker mesh codec according to several embodiments. [Figure 4] This is a flowchart of the encoding process according to several embodiments. [Figure 5] This is a flowchart of the decoding process according to several embodiments. [Figure 6] This figure shows an example of a corner table data structure. [Figure 7]This figure shows an example of an extended corner table to support UV coordinate attributes and normal vector attributes. [Figure 8] This figure shows an extended corner table data structure used in memory for the algorithm. [Figure 9] This diagram shows an exemplary coded entry point procedure. [Figure 10] This figure shows an exemplary procedure, preProcess, for performing input model transformation, cleanup, filling, and quantization. [Figure 11] This diagram shows the encoding procedure, which includes edge breaker topology coding and attribute prediction. [Figure 12] This diagram illustrates the procedure `startCompression`. [Figure 13A] This is a diagram showing the procedure `compressRec`. [Figure 13B] This is a diagram showing the procedure `compressRec`. [Figure 14] This is a diagram showing the procedure checkHandle. [Figure 15] This diagram illustrates an exemplary decryption entry point procedure. [Figure 16A] This diagram shows the initDecompression procedure, which allocates memory to store the recovery output and initializes other marking tables. [Figure 16B] This diagram shows the initDecompression procedure, which allocates memory to store the recovery output and initializes other marking tables. [Figure 17A] This figure illustrates the procedure `startDecompression`. [Figure 17B] This figure illustrates the procedure `startDecompression`. [Figure 18] This diagram shows the statement connectivityCompression. [Figure 19A] This diagram illustrates the decompress connectivity procedure. [Figure 19B] This diagram illustrates the decompress connectivity procedure. [Figure 20] This figure shows an example of the CheckHandle procedure for supporting shifts. [Figure 21] This is a diagram showing the statement attribute Decompression. [Figure 22A] This figure shows an example of the procedure DecompressVertices. [Figure 22B] This figure shows an example of the procedure DecompressVertices. [Figure 23] This figure shows an example of the procedure `decompressUvWithSeparateIndex`. [Figure 24] This figure shows an example of the procedure decompressUvWithSeparateIndex, which includes statement A. [Figure 25] This figure shows an example of the procedure decompressUvWithSeparateIndex, which includes statement B. [Figure 26] This figure shows an example of the procedure decompressUvWithSeparateIndex, which includes statement C. [Figure 27] This figure shows an example of a post-process procedure that converts the reconstructed CT mesh, with dummy points removed, into an indexed face set, and optionally recreates the non-manifold if lossless coding is used. [Figure 28] This diagram shows the preprocessing and postprocessing steps side by side. [Figure 29A] This figure shows a mesh with non-manifold vertices. [Figure 29B] This figure shows a mesh with non-manifold vertices. [Figure 29C] This figure illustrates an exemplary technique for correcting a problem by splitting vertices. [Figure 30]This diagram shows how two of the three surfaces resulting from duplicating non-manifold vertices are fixed by ristitching. [Figure 31A] This figure illustrates filtering of non-manifold surfaces according to several embodiments, and the exemplary techniques are sometimes called cutting and listing. [Figure 31B] The figure illustrates filtering of non-manifold surfaces in several embodiments, and exemplary techniques are sometimes called cutting and listing. [Figure 31C] This figure illustrates filtering of non-manifold surfaces according to several embodiments, and the exemplary techniques are sometimes called cutting and listing. [Figure 31D] This figure illustrates filtering of non-manifold surfaces according to several embodiments, and the exemplary techniques are sometimes called cutting and listing. [Figure 32] This is a diagram illustrating an example of a vertex duplication stage. [Figure 33] This figure shows an exemplary pre-processing operation for list stitching using a table of matching opposites. [Figure 34A] This figure shows an example of the procedure `deduplicateVertices`. [Figure 34B] This figure shows an example of a duplicate procedure. [Figure 35] This figure shows an example of a process for regenerating a non-manifold surface by duplicating and removing some vertices. [Figure 36] This figure shows a process in an exemplary embodiment in which holes in a mesh surface are filled with dummy vertices and dummy triangles. [Figure 37] This figure shows an example of a C++ procedure for filling all the holes in a mesh described as a corner table (O,V,OT,OTC). [Figure 38A]This figure shows an example of a C++ procedure for filling a single hole in a mesh, which is described as a corner table (O,V,OT,OTC). [Figure 38B] This figure shows an example of a C++ procedure for filling a single hole in a mesh, which is described as a corner table (O,V,OT,OTC). [Figure 38C] This figure shows an example of a C++ procedure for filling a single hole in a mesh, which is described as a corner table (O,V,OT,OTC). [Figure 39A] This figure shows attribute prediction using a major index table with procedures such as encodeAttributes. [Figure 39B] This figure shows attribute prediction using a major index table with procedures such as encodeAttributes. [Figure 40] This figure shows an example of the predicate isCornerVertexDummy. [Figure 41] This diagram shows the decodeAttributes procedure for the decryption counterpart. [Figure 42A] This figure shows an example of the procedure posEncodeWithPrediction. [Figure 42B] This figure shows an example of the procedure posEncodeWithPrediction. [Figure 43A] This figure shows strategies for multiparallelogram prediction and dummy point filtering of the position of corner c, according to several embodiments. [Figure 43B] This figure shows strategies for minimum stretching prediction and dummy point filtering of the UV coordinates of corner c, according to several embodiments. [Figure 44A] This figure shows an example of the procedure posDecodeWithPrediction. [Figure 44B] This figure shows an example of the procedure posDecodeWithPrediction. [Figure 45A]This figure shows an example of the uvEncodeWithPrediction procedure. [Figure 45B] This figure shows an example of the uvEncodeWithPrediction procedure. [Figure 46A] This figure shows an example of the statement predictUV, where dvec2 and dvec3 are double-precision vectors. [Figure 46B] This figure shows an example of the statement predictUV, where dvec2 and dvec3 are double-precision vectors. [Figure 47A] This figure shows the processing of UV seams using only the main index tables. [Figure 47B] This figure shows UV processing using a separate index table and selection strategy for corner cp, which is used for predicting UV coordinates with a separate index for corner c. [Figure 48A] The figure shows an example of the `encodeSeparateIndexAttributes` procedure. [Figure 48B] The figure shows an example of the `encodeSeparateIndexAttributes` procedure. [Figure 49A] The diagram shows an example of the procedure `decodeSeparateIndexAttributes`. [Figure 49B] The diagram shows an example of the procedure `decodeSeparateIndexAttributes`. [Figure 50] This diagram shows an example of the procedure "save". [Figure 51] This figure shows an example of pseudocode for controlling a Dirac encoder for the CLERS table. [Figure 52] This figure shows an example of reproducing a portion of the coding principles used for motion vectors in several embodiments. [Figure 53] This figure shows an example of an offset / shift for ExpGolomb coding. [Figure 54]This figure shows an example of code using an arithmetic coding context in several embodiments. [Figure 55] This figure shows the compression performance of the encoding methods described herein. [Figure 56] This is a block diagram of an example of a system in which various forms and embodiments are implemented. [Figure 57] This figure shows an exemplary implementation of one embodiment of dummy index removal. [Figure 58] This figure shows an exemplary implementation of another, more efficient embodiment of dummy index removal. [Figure 59A] This diagram illustrates the example procedure `encodePerFaceAttributes`. [Figure 59B] This diagram illustrates the example procedure `encodePerFaceAttributes`. [Figure 60] This figure shows an example of Dirac AC entropy coding for coding material IDs for each face. [Figure 61A] An example of a search order for corner[0]=c is shown (the operation is repeated for corner[1]=cn, then for corner[2]=cp), and the figure shows the face to be coded as Id and the adjacent triangles L, R, and F (there are always three adjacent triangles as dummy points are used to fill the holes). [Figure 61B] This diagram shows an example of a search order for corner[0]=c (the operation is repeated for corner[1]=cn, then for corner[2]=cp), and illustrates a neighborhood lookup that loops in the first direction across R, R1, R2, etc. [Figure 61C] The diagram shows an example of the search order for corner[0]=c (the operation is repeated for corner[1]=cn, then for corner[2]=cp), and the subsequent lookups in the opposite direction. [Modes for carrying out the invention]

[0031] (Summary of exemplary embodiments) Edgebreaker (EB) is the core of the algorithm, a technique for encoding and decoding (collectively "coding") the topology (connectivity and handles) of a mesh, and proposes the use of vertex position predictors to generate data efficient for some form of entropy coding.

[0032] Some embodiments of this disclosure provide fully end-to-end extended versions of EB. Some embodiments include both low-level and higher-level syntax for bitstreams.

[0033] Exemplary embodiments provide support for topology coding (connectivity and handles) and geometry using delta coding.

[0034] Some embodiments provide mesh cleanup for the removal of non-manifolds (pre-treatment for CT).

[0035] Some embodiments provide support for holes and boundaries through "virtual" dummy points.

[0036] Some embodiments include a lossless mode that, if necessary, reconstructs meshes with poor "connectivity" during decoding.

[0037] Some embodiments provide support for multiple linked components (CCs).

[0038] Some embodiments include support for texture coordinate coding using a unique index table or separate index tables.

[0039] Some embodiments include multiparallelograms and stretch prediction. Some such embodiments further provide support for dummy points and separate index tables when needed.

[0040] Some embodiments include a Dirac and / or RANS arithmetic coder (AC). Some such embodiments use Dirac coding with a specific binarization scheme and arithmetic coding context described herein for use.

[0041] Some embodiments include support for pre-quantized input meshes in addition to input mesh quantization.

[0042] Some embodiments include support for face-by-face identifiers or colors.

[0043] Some embodiments provide optimizations for per-face attribute prediction and coding.

[0044] Figure 3 shows an example of an edgebreaker mesh codec according to several embodiments. The top row is the encoded line, and the bottom row is the decoded line. As shown in Figure 3, exemplary embodiments include some or all of the following: During encoding, preprocessing 302 may be used to clean up potential connectivity problems (non-manifold edges and vertices) that may exist on the input mesh. This cleanup is performed because the edgebreaker algorithm itself does not act on meshes with such connectivity problems. In some embodiments, cleaning up non-manifold edges and vertices involves duplicating multiple points. Some embodiments track these duplicated vertices and merge them when decoding. This makes it possible to reduce the number of points in the decoded mesh, but requires sending some additional information in the bitstream. In some embodiments, since the edgebreaker algorithm does not handle holes on its own, this preprocessing 302 further includes adding some dummy points to fill up potential holes on the surface. Exemplary embodiments operate to fill holes before encoding and recreate holes after decoding. An exemplary embodiment uses "virtual" dummy points, generates and encodes dummy triangles connected to these dummy points, but the 3D positions of those points are not encoded or decoded. In some embodiments, vertex attributes are quantized if necessary. Those attributes may be provided to a coder that has already been quantized.

[0045] An exemplary embodiment encodes the connectivity of mesh AT304 using a modified version of the edge breaker algorithm that generates a CLERS table (a table made up of "C", "L", "E", "R", and "S" symbols). This stage also generates several tables in memory that will be used for the attribute prediction stage. Vertex attributes are then predicted in 306, starting from position 1. Other attributes are then predicted, ultimately relying on the position predictions, an example being the case for texture UV coordinates. The configuration and metadata are also given in the bitstream, and the CLERS table, any other connectivity, and all attribute prediction residuals are entropy coded in 310 and added to the bitstream.

[0046] In the exemplary decoding method, all entropy-coded subbitstreams are entropy-decoded at 312. We reconstruct mesh connectivity at 314 using a CLERS table and an edge breaker algorithm. We also manage handles that describe the topology by adding some additional information. We predict the positions of all other vertex at 316 using the mesh connectivity and a minimum set of vertex positions expressed in 3D coordinates. We then apply attribute residuals to correct the predictions and obtain reconstructed vertex positions. With respect to UV coordinates, other attributes are also decoded at 316, potentially relying on the decoded positions. Attribute connectivity, using a separate index table, is reconstructed using entropy-coded per-edge binary seam information.

[0047] In the 320 post-processing stage, we remove dummy triangles, optionally recreate the non-manifold problem if the coder is configured to perform lossless coding, and optionally dequantize the vertex attributes if the model has been quantized by the encoder.

[0048] Figure 4 shows flowcharts of the encoding process according to several embodiments.

[0049] Figure 5 shows flowcharts of the decoding process according to several embodiments.

[0050] An extended corner table. The corner table data structure representing the mesh is described in Non-Patent Document 1, and further described in Non-Patent Document 4.

[0051] Figure 6 shows a portion of a triangular mesh, including what is stored in the corner table (also called the OV table). For each corner index c of a corner in the corner table, cv=3 identifies the index of the vertex where that corner is located. Index ct=0 is the index of the triangle containing corner c. Within that triangle, index cn=0 identifies the next corner (counterclockwise), and cp=1 identifies the previous corner. Index co identifies the corner opposite corner c. For example, in Figure 6, corner co=5 is the opposite corner c=2 (and vice versa). Index cl identifies the corner to the left of corner c, and index cr identifies the corner to the right of corner c. The connectivity shown in Figure 6 may be recorded in a corner table with entries including:

[0052] [Table 1]

[0053] The "Corner Index" column corresponds to the position of each row in the table and therefore does not need to be stored in the table itself. The index of each corner triangle can be calculated as follows:

[0054]

number

[0055] Therefore, corners 0, 1, and 2 are all in the triangle with index 0, corners 3, 4, and 5 are all in the triangle with index 1, and so on. Additional relationships between entries in the corner table can be expressed as follows:

[0056]

number

[0057] In some embodiments, an additional table may be used to store the position coordinates of each vertex.

[0058] Figure 7 shows an example of an extended corner table used in some embodiments to support UV coordinate attributes and normal vector attributes, both using separate index tables, TC / OTC and N / ON, respectively. Texture coordinates and normal vectors can be stored in two other separate tables.

[0059] Exemplary embodiments extend the corner table data structure, as shown in Figure 7, to support other attributes, with or without secondary attribute indices. If the mesh does not have a separate index for dereferencing the UV coordinate attribute, the exemplary embodiments include a table of UV coordinates together with the position table. The UV coordinate table is then dereferred using V with respect to position. Otherwise, some embodiments also add two tables, TC and OTC, to code UV coordinate connectivity (dereferencing). TC is used to store the UV coordinate index, and OTC is used to store the opposite corner for UV connectivity. The same principle can be applied to add support for per-vertex normals (N / ON / normal_vectors) or per-vertex colors (C / OC / colors).

[0060] Figure 8 schematically shows an extended corner table data structure that may be used in memory in an exemplary embodiment. For greater flexibility, Figure 17 shows the use of a separate table for each row of the table.

[0061] Figure 8 summarizes the data structure we use to store all the information in our extended corner table. This technique can be extended to use or not use a dedicated index table modeled using pairs (index table / opposite table) for any per-vertex attribute type. In this implementation, per-face IDs are included in the table.

[0062] An overview of the coding procedure. In this section, we will provide several high-level procedures to better understand the structure and flow of an exemplary encoding process.

[0063] An exemplary coding entry point procedure is shown in Figure 9. It involves three steps, each performing preprocessing of the input mesh, edge breaking of it, coding and final entropy coding, and storage into a bitstream.

[0064] Figure 10 shows an exemplary procedure, preProcess, for performing input model transformation, cleanup, filling, and quantization. The characteristics of this procedure will be given in further detail below. It is introduced here for overall understanding before edgebreaker coding is introduced below. The save procedure, shown in Figure 25, will also be explained in more detail below.

[0065] Figure 11 shows the procedure encode, which includes edge breaker topology coding and attribute prediction. Features that are described in particular in detail herein are shown in bold. This procedure first initializes several data structures, which primarily assign several tables to marking attributes or triangles and table G_est to store the (predicted) estimation of location attributes. It then runs the edge breaker algorithm on each connected component (CC) of the mesh. To do so, we add a while loop during the procedure to manage several starting indices for handles that we update for each new component. After coding one CC, we run the procedure findRestartCorner to check if there are any unmarked (i.e., not yet coded) corners in the MC table that we can use as entry corners for coding new CCs.

[0066] Several exemplary embodiments support meshes having multiple connected components. In one exemplary embodiment, the method includes encoding a first connected component of the mesh using edge breaker coding, identifying uncoded corners in the mesh after encoding the first connected component of the mesh, and encoding a second connected component of the mesh using the uncoded corners identified as entry corners.

[0067] Figure 12 shows an exemplary procedure, startCompression. Features that are described in particular in detail herein are shown in bold. This procedure is the starting point of the actual edgebreaker algorithm for encoding CC. It encodes the first triangle and then begins the recursive encoding of CC. The method involves recording the processed corners into a table named processedCorners. This table will be used later to handle attributes that have separate index tables. We also add a specific procedure, encodeSeparateIndexAttributes, at the end of the processing to encode / predict attributes that have separate index tables. This final procedure leverages the processedCorners table to perform its processing.

[0068] The encodeAttributes, encodeSeparateIndexAttributes, and encodePerFaceAttributes procedures are described in more detail below. Essentially, they perform attribute predictions and store the results in several tables that will later be entropy-encoded. For now, note that the encodeAttributes procedure computes predictions for attributes using a primary index table for a given corner.

[0069] Figure 13 shows the procedure compressRec. The table oClers is the output table of CLERS symbols. Features that are described in particular in detail herein are shown in bold. This procedure is a core edge breaker algorithm for encoding CC. Processed corners are added to a table named processedCorners.

[0070] Figure 14 shows the procedure checkHandle.

[0071] Given the above explanation of how the CLERS and Handles tables are constructed during the coding stage, the following description illustrates exemplary embodiments for reconstructing the mesh topology from those tables and recreating the corner tables during decoding.

[0072] (Overview of the decryption procedure) Figure 15 shows an exemplary decoding entry point procedure. Features that are described in particular detail herein are shown in bold. The procedure involves several steps to generate the final mesh in an indexed faceset data structure by performing syntactic analysis and arithmetic decoding of the input bitstream, edgebreaker decoding of it using a corner table, and finally some post-processing on the corner table, respectively. In this section, we will only devise the edgebreaker decoding procedure (initDecompression and startDecompression) and the subprocedures related to topology decoding. Analysis and entropy decoding will be described in more detail below.

[0073] Figure 16 shows the initDecompression procedure, which allocates memory to store the decompression output and initializes other marking tables. Features that are described in particular in detail herein are shown in bold. In the initDecompression procedure, the “start” is a point of no prediction, and therefore its attributes are stored in a separate table containing global values ​​(sVertices, sUVCoords). In the counterpart, the predicted attributes are stored in the other input tables iVertices and iUVCoords. Those tables will be described further below.

[0074] This section describes exemplary tables used for topology decoding (V, O, TC, OTC, D, M, U, MTC, and duplicateIndexRemapping). The values ​​used to initialize O, TC, OTC, and D are recorded as their conditions will be used later to enable the proper execution of the algorithm. The exemplary embodiment includes tables for handling texture coordinate attributes (extended as well as per-vertex normals and colors in some embodiments), attributes with separate index tables (see the hasSeparateUvIndex statement), and a final duplicate removal remapping table. We have also added handling of dummy points.

[0075] Figure 17 shows an exemplary procedure, startDecompression. Features described in detail herein are shown in bold. The procedure startDecompression, which initiates decompression, is called after the initDecompression initialization stage.

[0076] An exemplary embodiment provides CC processing using offsets. Then, for each CC, after the connectivityDecompression and attributesDecompression stages, the vertex index and UV coordinate index are shifted back to make them valid in global index space. This processing can also be extended to any other attribute types in some embodiments. The exemplary embodiment further includes duplication removal post-processing, which will be described in more detail below.

[0077] (Recursion of decryption connectivity) Figure 18 shows the statement connectivityCompression. Features described in particular in detail herein are shown in bold. The connectivityDecompression statement presented in Figure 18 can be inserted in place in the pseudocode of Figure 17, and thus it accesses the setup variables. The statement is executed for each CC. In this logical block, the algorithm initializes the opposite corner of the CC handle using techniques such as those described in Non-Patent Literature 2. An exemplary embodiment further adds a first corner (index 0) in the processedCorners table. Finally, we reconstruct the connectivity of the first triangle and begin the connectivity decompression recursion described with reference to Figure 19.

[0078] Figure 19 shows an exemplary decompressConnectivity procedure. Features that are described in particular in detail herein are shown in bold. In an exemplary embodiment, the decompressConnectivity procedure supports shifting and manages CC, and the procedure includes recording a history for the processed corners. The procedure further provides support for the opposite corner and supports the handle using, for example, the technique described in Non-Patent Literature 2.

[0079] Figure 20 shows an example of a CheckHandle procedure that supports shifting. The zip procedure utilizes the shifted tables O and G (the code is not shown here).

[0080] At the end of the connectivityDecompression statement (see Figure 17), connectivity can be used together with the predicted attribute residuals, which have been reconstructed and previously entropy-decoded to reconstruct / estimate the attributes. The next steps are described in the following sections.

[0081] (Recursion of decryption attributes) Figure 21 shows the statement attributesCompression. The attributesDecompression statement can be inserted in place in the pseudocode in Figure 17, and thus it accesses the setup variables. The statement is executed per CC. In this logical block, we first reconstruct the attributes of the first triangle using decodeAttributes, and then recursively reconstruct the attributes of the other triangles by calling decompressVertices. This first step operates only on attributes using the main index table.

[0082] In the second recursion, we optionally reconstruct attributes that are dereferenced by the secondary index table. Since they depend on their output data, they are implemented after the first recursion has finished before being processed. The presented code shows the second step for UV coordinates, calling the decompressUvWithSeparateIndex procedure. Extensions to other types of per-vertex attributes, such as per-vertex normal vectors or colors, are also implemented in some embodiments. The prediction scheme at a lower level is adjusted accordingly.

[0083] Some embodiments further include a third recursion in which the per-face attributes are reconstructed, as will be described in more detail below.

[0084] Figure 22 shows an example of the procedure DecompressVertices. The procedure works with CC using a shifted table. Recursion traverses the triangular tree and reconstructs attributes as needed by calling the decodeAttributes procedure.

[0085] Figure 23 shows an example of the procedure `decompressUvWithSeparateIndex`. It behaves differently from `decompressVertices` by handling texture seams using the secondary index tables TC and OTC.

[0086] The procedure is broken down into three consecutive statements: statement A shown in Figure 24, statement B shown in Figure 25, and statement C shown in Figure 26. Each of these performs a loop on processedCorners. In some embodiments, the order of the corners is implicitly discovered, as in the case of the decompressVertices procedure, in which case the processedCorners table may not be necessary. However, the table is shown in this example for ease of coding and explanation.

[0087] Referring to Figure 24, using the iSeams table (coming from the bitstream), the recursion of statement A first assigns a code in [0,-1,-2] to each corner of the OTC table to determine whether they are on a seam or whether the UV coordinates will be the same on each side of the edge. In the first case, different UV indices are used for each side of the edge. Note that the exemplary embodiment also skips the final dummy point in this loop. It is not necessary to reconstruct the entire opposite table OTC (in terms of connectivity with respect to table O), and the table can be used only to store the code for statement B.

[0088] Referring to Figure 25, in the second pass, the recursion of statement B leverages the code from pass A and assigns UV coordinate indices to the corners by iterating over them. This pass populates table TC with UV indices. During the loop, corners that have already been visited or are associated with dummy points are skipped.

[0089] In the final pass, referring to Figure 26, the recursion from statement C uses the TC and MTC tables to calculate the final UV coordinates and mark the UV coordinates that have already been processed. If the bitstream was coded without prediction (cfg.uvPred=NONE), the values ​​in table iUVcoords are copied to the UV table using CC shift. Otherwise, the UV coordinates are predicted using the uvSepDecodeWithPredictionExt procedure as described below.

[0090] (Post-decryption processing) At the end of the startDecompression procedure, connectivity and attributes are decoded, as illustrated in Figure 15. In some embodiments, additional post-processing steps are performed to finalize the mesh. The postProcess procedure is shown in Figure 27. Different parts of this procedure are described below in contrast to the preProcess procedure (see Figure 28).

[0091] Figure 27 shows an example of the postProcess procedure, which converts the reconstructed CT mesh into an indexed face set while removing dummy points, and optionally recreates the non-manifold if lossless coding is used.

[0092] Exemplary pre- and post-processing steps are described in more detail in the following sections.

[0093] (Pre-processing and post-processing) In some embodiments, preprocessing is performed to convert several standard indexed faceset meshes into cornertable representations for efficient use by edge breaker algorithms optimized to operate on cornertables. Indexed faceset meshes are also known as face-vertex meshes. They are stored in ".obj", ".ply", or other common file formats that conventionally store triangular meshes. Face-vertex representations are also captured by graphics hardware for rendering. Conversion to cornertables can be performed using known techniques. However, while converting to cornertable representations, exemplary embodiments also perform filtering of non-manifold vertices and edges, as described below. Exemplary embodiments also introduce virtual dummy points and dummy faces to fill the final holes in the input mesh, as described below. Symmetrically, upon decoding, the final dummy points are removed, and non-manifold connectivity is regenerated in lossless mode. Figure 28 presents an overview of exemplary preprocessing and postprocessing side by side. Quantization and dequantization processes can be performed using standard uniform quantization mechanisms. In some embodiments, quantization and dequantization are not performed, for example, in embodiments where a pre-quantized mesh (e.g., a mesh with all integer attributes) is coded.

[0094] Figure 28 shows the preProcess and postProcess procedures side by side. The comments and captions in Figures 10 and 27 provide further details regarding preProcessing, respectively. In some embodiments, an optional action for duplication removal is also performed during preProcessing within the fixNonManifold procedure.

[0095] (Non-manifold filtering) For example, some meshes coming from 3D reconstructions from scanners may have some "inaccurate" connectivity in that they do not have a manifold topology that can be processed using standard edge breaker coding. For instance, a surface described by triangles is non-manifold at some vertices or edges and is therefore subject to the processing shown in Figures 9 and 10. The purpose of filtering is to correct the problem and enable edge breaker coding. In some embodiments, this is done by duplicating selected vertices so that triangles sharing a non-manifold edge or vertex no longer share the same edge or vertex.

[0096] Figures 29A and 29B show meshes with non-manifold vertices. Non-manifold vertices are indicated by dashed ellipses.

[0097] Figure 29C shows an exemplary technique for correcting the problem caused by subdividing vertices.

[0098] In the example in Figure 29A, one vertex (for example, at index i1 in the vertex table) is shared by two triangles, but those triangles do not share any edges (therefore, the vertex is a non-manifold). An exemplary embodiment corrects this non-manifold problem by duplicating the common vertex and creating a new vertex with the same 3D position and the same attributes, which is added to the vertex table, at index i in this table. n This will result in the following. In this case, the description of one face refers to i1, and the second one refers to i n This refers to vertices i1 and i n Although shown in Figure 29C as having slightly different positions after splitting, vertices 1 and i are represented in the attribute table. nIt should be understood that the positions of the triangles can be the same after the partition. After the process, the faces are no longer connected, and therefore there is no non-manifold problem. In this example, it will result in two connected components (CCs), each made up of just one triangle (a "connected component" is a set of triangles for which there is always a path of connectivity to the other triangle in the component). By performing this duplication, we can see that we can generate multiple connected components.

[0099] When techniques for modifying non-manifold vertices are applied to non-manifold edges, several surface continuity problems can arise. Referring to Figure 30, a single non-manifold edge is shared by three triangles. If we duplicate the vertices of the non-manifold edge, the faces will no longer be connected. If faces T1 and T2 were parts of a larger connected component, this would introduce a "crack" in the surface of the mesh. Furthermore, duplicating all of those vertices introduces more vertex data to encode. As shown in Figure 30, in one exemplary embodiment, a list stitching method can be performed to recombine two of the three surfaces that would result from duplicating the non-manifold vertices. (For clarity, the splitting and stitching processes do not change the positional attributes of the vertices, but the duplicated vertices are shown as having slightly different positions.) As a result of the stitching process shown in Figure 30, faces T1 and T3 are stitched together into a single manifold surface, while T2 remains a separate manifold surface.

[0100] Figures 31A–31D illustrate filtering of a non-manifold surface according to several embodiments. Exemplary techniques are sometimes referred to as cutting and list stitching. As shown in Figure 31A, the connectivity of a triangle on a non-manifold edge 3102 is cut by resetting the opposite corner to -2 (or another predetermined value as appropriate). The vertices are duplicated where necessary, as shown by the dashed ellipse in Figure 31B, where only the vertices of the top triangle are duplicated (for illustrative purposes, the positional attributes of the duplicated vertices may not actually be changed by the duplication process, but their vertices are shown in slightly offset positions). As shown in Figure 31C, the process swings left and right from the visited corner c. As shown in Figure 31D, the opposite corner is list stitched when the left vertex index lv is equal to the right vertex index rv, where 01=02=-2.

[0101] Several exemplary embodiments perform a vertex duplication operation to remove one or more non-manifold edges. A pre-processing step is then performed to repair the newly introduced holes through a list stitching process, if possible. This reduces or eliminates the need to handle specific cases during the rest of the process, which simplifies the implementation of the edge breaker and reduces the number of dummy points and UV seams (since geometry seams inevitably introduce texture seams) and their impact on the edge breaker.

[0102] In some embodiments, such a method may be implemented to identify at least one non-manifold edge in the mesh, duplicate vertices along the non-manifold edge to separate the non-manifold edge into at least two manifold edges, stitch together two of the at least two manifold edges, and then encode the mesh using edge breaker coding after stitching together two of the at least two manifold edges.

[0103] In some embodiments, the non-manifold filtering stage (shown by Figures 31A–31D) is performed during preprocessing as follows: The mesh is filtered for non-manifold surfaces. The triangular connectivity of the non-manifold surface is cut by resetting the O-table for some corners and setting the O-value to -2 at the appropriate corners. Vertices are duplicated if necessary. Triangular connectivity is list-stitched by correcting the O-table where appropriate. Filtering of non-manifold surfaces can be performed using techniques known in the Google Draco system, etc. In exemplary embodiments, a “list-stitching” stage is performed to reconnect some triangles. This also allows for the regeneration of the non-manifold surface in the case of lossless coding. The exemplary embodiment further includes a stage for duplicating vertices. The “duplicate vertices” stage is shown in Figure 32. Features that are described in particular in detail herein are shown in bold. Features in bold include features for handling duplicate removal for lossless mode and for list-stitching. In summary, the exemplary embodiment tracks the index of added vertices (in the lossless case), stores a table of corner pairs as shown in Figures 31C and 33, and “listitches” them at the end of processing in the final pass. Figure 33 shows an exemplary preprocessing operation of listitching using a table of matched opposites.

[0104] (Non-manifold filtering, lossless example) In some embodiments, lossless coding of the input mesh is performed to accurately reconstruct connectivity, including non-manifold problems. As shown in the previous section, the exemplary embodiments track inserted (duplicated) vertices in a table and the original vertices (referred to here as the parent of the inserted vertices) during preprocessing. Some data representing their overlaps is given in a bitstream and then used during decoding (at some stages) to reconstruct strict connectivity if necessary.

[0105] The availability of lossless modes for connectivity as described herein offers at least two benefits. First, it enables a fully lossless coding mode for both connectivity and attributes. Second, lossless coding of meshes allows for higher coding gains, even when lossy coding is used for the mesh as a whole (e.g., for attributes). These gains may result from avoiding iterated vertices during the subdivision process for coding, and then for displacement coding. Lossless coding of meshes can also extend to motion coding.

[0106] An exemplary coding method includes identifying at least one non-manifold edge in a mesh, the non-manifold edge being defined by a parent vertex; preprocessing the mesh by duplicating the parent vertex along the non-manifold edge to separate the non-manifold edge into at least two manifold edges, each manifold edge being defined by a child vertex, each child vertex being duplicated from its parent vertex; coding the preprocessed mesh into a bitstream using edge breaker coding; and providing information in the bitstream that identifies the parent vertex of the child vertex.

[0107] An exemplary decoding method includes: obtaining information from a bitstream defining multiple connected components of a non-manifold mesh coded in an edge breaker; obtaining information from a bitstream relating child vertices to parent vertices; decoding at least two of the connected components, each of which has child vertices associated with its respective parent vertex; and reconstructing the non-manifold mesh by duplicating and removing the child vertices.

[0108] In the coding stage, when positions are duplicated (potentially recursively) by non-manifold filtering, an exemplary embodiment tracks vertices that have the same original parent. A unique identifier (for example, ranging from 1 to the number of vertices ultimately added) is assigned to each vertex index associated with the duplicated vertex. This information can be stored, for example, in a duplicatesMap, with each entry associated with such an identifier at the duplicated vertex index. To populate the duplicatesMap, when a vertex is added to the map, it is determined whether the parent already exists in the map. If so, the parent already exists in the map and is associated with a unique identifier. In an alternative case, both the parent and the added vertex are added to the map, and new identifiers (which may be the current number of partitioned vertices) are associated with them.

[0109] The number of partitioned vertices is the number of original position indices that were duplicated during processing. The total number of entries in the duplicatesMap map at the end of processing is (number of partitioned vertices + number of added vertices).

[0110] When the topology is traversed during attribute encoding using the primary index table (see Figure 37), when a split vertex or added vertex is encountered, registration of the processed corner index and the corresponding unique split vertex index is performed. Some embodiments use the oDuplicateSymbolIdx array, which stores the processed corner index and the unique split vertex index, along with the oDuplicateSplitVertexIdx array. Since the processedCorners array is updated per triangle (rather than for two first vertices of any connected component), some embodiments further store a boolean value indicating whether there are duplicated vertices on the first two vertices of a first triangle of any connected component. An exemplary embodiment therefore further uses a boolean array of size (2 * connected components) named oDuplicatesOnStart.

[0111] An exemplary embodiment then signals some or all of the following information in the bitstream: The size of the oDuplicateSymbolIdx array is equal to the number of divided vertices + the number of added vertices. • An integer array oDuplicateSymbolIdx of size oDuplicateSymbolIdx. (Of course, any other software implementation features of this or any other described herein may be given names other than those used herein. For example, the integer array oDuplicateSymbolIdx may have alternative names such as perDuplicateProcessedCornerIndices). Some embodiments allow up to three duplicates related to a first processed corner index of any connected component (one corner for each of the three vertices of that first triangle). In some embodiments, this may be delta-coded and inserted as a variable unsigned integer. • An integer array of size oDuplicateSymbolIdx, which gives a unique value for each group of vertices duplicated from a divided parent vertex (including the original parent), encoded as a variable such as an unsigned integer. • A Boolean array of size 2 * (number of connected components) can be coded using Dirac arithmetic coding as follows: ○ A variable unsigned integer for dup01_encoded_size. The size in bytes of dup01Buffer, which includes the AC encoding flags for signaling the copy at position 0 or 1 in each CC (2*nbCC bits before encoding). ○ dup01_encoded_size. The number of bytes containing the encoded buffer.

[0112] Alternatively or additionally, exemplary embodiments signal some or all of the following information in the bitstream: • "encodeAttributes": A boolean indicating whether vertices are duplicated for each positional procedure call. Next, a Boolean statement indicating that this is the first occurrence. If it is not the first occurrence, the index of the parent partition vertex in the table of base partition vertices.

[0113] In some embodiments, the information is encoded arithmetically. When there are many partitions (which must be checked), the amount of information encoded in the stream can be reduced using this technique. Furthermore, the Boolean array (a) may be encoded only for CCs that contain duplicated vertices, adding another level of indirect means, where a first string of bits must define which CCs contain duplicated vertices, but the size of array (a) will be reduced by potentially removing ranges that have certain false values.

[0114] In some embodiments, a determination is made as to which encoding method produces the smallest bitstream, the mesh is encoded using the selected method, and an indication of the method used is signaled in the bitstream.

[0115] In some embodiments, to prevent encoding positions that have been duplicated more than once, the process returns from encodeAttributes without calculating the associated residuals in response to the determination that the duplicated vertices have already been processed. The corresponding functionality is also performed in the decoder. Such embodiments are useful for improving performance, particularly for meshes with many subdivisions.

[0116] The position topology of the input mesh is reconstructed losslessly after encoding / decoding when this duplication removal option is used. In some embodiments, the same technique is used to enable attribute duplication removal.

[0117] On the decoder side, the decoder retrieves data such as iDuplicatesOnStart, iDuplicateSymbolIdx, iDuplicateSplitVertexIdx, and cornerIndex (these can be called by currentDuplicateIndex or any other arbitrary name).

[0118] During decoding (see Figure 17), updates are performed in the deduplicateVertices procedure (see Figure 34) and within the index of the processed corners (adding an offset for each CC, as processedCorners is emptied after each CC). The decoder may then check whether the current corner is associated with a duplicated vertex (including an additional check for two first vertices of each CC, or using an alternative technique). The decoder then assigns a single index to each of the vertices duplicated during the processDuplicate procedure. Figures 34A–34B show examples of the deduplicateVertices and processDuplicate procedures. When first called with a given deduplicate vertex index, the current vertex index is associated with the deduplicate vertex index. In subsequent calls with a given deduplicate vertex index, the registered index corresponding to the first occurrence of the duplicated vertex is used to replace the duplicated index. This data is used to regenerate the non-manifold by removing some vertices before converting the CT to an indexed faceset. An example of this process is shown in Figure 35.

[0119] (Virtual dummy points for meshes with boundaries) Figure 36 shows the processing in an exemplary embodiment in which holes in the mesh surface are filled with dummy vertices and dummy triangles. Dummy points are indexed i in the vertex table. n Given, the mesh on the left contains a hole in the center. As shown on the right, that hole is at vertex i n It can be filled by adding and expanding the triangle into a sector shape to fill the hole.

[0120] One known method for handling meshes with boundaries (also called meshes with holes) is to fill each hole with a dummy point and connect this new vertex to the boundary of the hole using a triangular sector. The mesh is then encoded / decoded as if no holes existed. Some data representing which points are dummies is encoded in the bitstream, and thus the points and their associated dummy triangles (from the sector) can be removed after decoding. In such techniques, dummy points are generally encoded using spurious attribute values ​​(usually from interpolation).

[0121] In exemplary embodiments of this disclosure, since the non-manifold filtering itself can introduce holes in the surface, the process of filling holes with dummy points is performed after the non-manifold filtering. Furthermore, exemplary embodiments do not track the index of the dummy points and do not encode the attributes of the dummy points (e.g., position, UV coordinates, color, etc.). Accordingly, exemplary embodiments do not use false attributes of dummy points when predicting the attributes of non-dummy points.

[0122] In exemplary embodiments, dummy triangles are encoded, but the attributes of dummy points (location, UV code, etc.) are not encoded. Interpolated attributes are not used to compute the prediction. Rather, some embodiments track dummy point indices and skip them during the prediction.

[0123] An exemplary encoding method comprises preprocessing a mesh having a hole by creating dummy vertices, wherein the dummy vertices are connected to each vertex along the perimeter of the hole; encoding the preprocessed mesh into a bitstream using edge breaker coding; encoding information identifying the dummy vertices into the bitstream; and encoding at least one attribute for each of a plurality of vertices in the preprocessed mesh, wherein the attribute is not encoded for the dummy vertices.

[0124] An exemplary decoding method comprises obtaining information encoding a mesh from a bitstream, obtaining information identifying at least one dummy vertex in the mesh from the bitstream, and decoding attribute information for multiple vertices in the mesh from the bitstream, wherein the attribute information is decoded but not decoded for at least one dummy vertex.

[0125] In some such embodiments, decoding attribute information for multiple vertices involves predicting attribute information for multiple vertices, wherein the attribute information is not predicted for dummy points. The decoder may remove dummy points.

[0126] Exemplary embodiments can handle meshes with and without separate UV coordinate index tables. Some encoders, such as the VDMC encoder, use a base mesh with a separate index table, and the geometry consists of a small number of closed meshes (which are mostly closed but eventually have holes) rather than mesh patches. Thus, the economy of not encoding the first boundary is not significant in some embodiments.

[0127] Regarding the handling of "virtual" dummy points, exemplary procedures for handling dummy points according to several embodiments are shown in Figures 35 and 36.

[0128] In one embodiment where attributes use a primary index table, the preprocessing steps include filling in gaps by adding new corners and triangles to the corner table and setting up connectivity. The index table of dummy vertices, as well as some additional locations, is stored in memory, and other optional attributes are stored in the attribute table (this attribute data is used only to fill gaps, and the exemplary embodiment uses zeros). During connectivity coding / decoding with edge breakers, connectivity is used as is, including dummy triangles. For better prediction of other vertices and for data economy in the bitstream, this table is used to skip dummy point attributes during attribute prediction during coding and decoding.

[0129] In one embodiment where attributes use separate index tables, a different process is performed on the corner table. Such an embodiment does not recreate connectivity, but simply initializes a new TC element with -1 and a new OTC with -(boundary index + 4), and thus each added corner has attribute tables TC and OTC marked as special elements, which can be treated like seams. (Figure 47 shows an example of a textured seam.)

[0130] Figure 37 shows an example of a C++ procedure to fill all the holes in a mesh described as a corner table (O,V,OT,OTC).

[0131] Figures 38A to 38C show an example of a C++ procedure for filling a single hole in a mesh, which is described as a corner table (O,V,OT,OTC).

[0132] In an exemplary embodiment, the dummy point index table is encoded in the bitstream. See, for example, the syntax elements DATA_DUMMY_POINT_COUNT in Table 4 and DATA_DUMMY_POINTS_TABLE in Table 5 below. However, attributes related to dummy points are not encoded.

[0133] During post-processing, after the mesh decoding is complete, the exemplary embodiment removes dummy points and associated dummy triangles using a dummy point index. Thus, the output of the processing matches the original mesh with respect to the boundaries. This processing can be performed during transcoding to the final indexed faceset representation.

[0134] The vertex index is updated, removing references to dummy vertices and shifting subsequent indices accordingly.

[0135] One exemplary implementation of such a technique is shown in Figure 57. The complexity of this implementation is dummy_count*numOfTriangles. When the model has a large number of triangles and dummy points, processing can be costly. In such cases, it may be preferable to first define a mapping function using a list of dummy position indices, and then apply this remapping while keeping track of which triangles should not be included in the decoded mesh. (For example, a false boolean value may be assigned to a triangle containing at least one dummy vertex, which is then discarded.)

[0136] Encoding and decoding of attributes for each vertex. To calculate the prediction, the exemplary embodiment uses multiparallelogram prediction for the geometry, as described in Non-Patent Documents 7 and 8. The exemplary embodiment uses minimum stretch prediction for UV coordinates, as described in Non-Patent Document 9.

[0137] The exemplary embodiment further provides processing for handling dummy points to populate a separate index table of choice for holes and UV coordinates. The techniques described may be further used to support prediction of other attributes such as per-vertex normals, per-vertex colors, or any other per-vertex attributes.

[0138] Some embodiments provide prediction methods for handling dummy points and separate index tables (and thus UV seams).

[0139] In some embodiments, attribute prediction using a primary index table is implemented by a procedure such as encodeAttributes, presented in Figures 39A-39B. This procedure is called several times during edgebreaker coding. Its decoding counterpart procedure, decodeAttributes, is shown in Figure 41. Similarly, this procedure is also called several times during edgebreaker decoding.

[0140] Figure 40 shows an example of the predicate isCornerVertexDummy. During encoding, all dummy points are at the end of the vertex table, but this is not the case during decoding.

[0141] (Position residual coding) Figure 42 shows an example of the procedure posEncodeWithPrediction.

[0142] An exemplary embodiment stores the global coordinates of the starting point position and the residuals of other points in a separate table to obtain better entropy coding.

[0143] An exemplary mesh coding method comprises coding the connectivity of the mesh, entropy coding a first table of the starting point positions of vertices in the mesh, and separately entropy coding a second table of the point residuals of vertices in the mesh.

[0144] An exemplary mesh decoding method includes decoding information indicating the connectivity of the mesh, entropy decoding a first table of the starting point positions of vertices in the mesh, and separately entropy decoding a second table of point residuals of vertices in the mesh.

[0145] Several exemplary embodiments extend the multiparallelogram method to handle dummy point cases in a way that prevents erroneous predictions of parallelograms involving dummy points.

[0146] Some exemplary embodiments extend predictive decoding to support multiple concatenated components by using table shifts.

[0147] An exemplary mesh coding method includes obtaining information defining a mesh having multiple connected components (CCs), where each vertex in the CC has a unique global range index; shifting the index of the vertex in the CC from the global range index to the CC range index for each CC; and coding each CC using the CC range index.

[0148] An exemplary mesh decoding method includes obtaining information encoding multiple connected components (CCs), where each vertex in a CC has its own CC range index; decoding each CC; and for each vertex, shifting the vertex index from the CC range index to the global range index.

[0149] In the coding stage, the exemplary method includes, for example, calculating the position predictions of corner c and its associated vertex index v during the coding chain using the method shown in Figure 40. During corner prediction, we are confident that its associated triangular sector is always complete and that each of its corners has a valid opposite side, since the exemplary embodiment fills the gap using dummy points (even a single triangle becomes a pyramid made of four triangles in terms of connectivity).

[0150] Figure 43A shows strategies for multiparallelogram prediction and dummy point filtering of the position of corner c according to several embodiments.

[0151] Figure 43B shows strategies for minimum stretch prediction and dummy point filtering of the UV coordinates of corner c according to several embodiments.

[0152] In the first step, the process checks whether the corner c to be coded is attached to a dummy vertex (if so, then skip coding the corner), or whether we predict the first vertex of chain v=0 (we then code the position of v using global coordinates and no prediction). In this latter case, the process stores the positions in a specific table dedicated to entropy coding the starting vertex (table sVertices). By storing these in a specific table, rather than prediction, it (table oVertices) allows for better entropy coding of these tables.

[0153] Otherwise, in the second step, we test whether the vertex is the second or third vertex (v=1 or v=2). In this case, we do not perform prediction, but rather use delta coding with the vertex of the previous corner and append the result to the table of predicted vertices.

[0154]

number

[0155] Otherwise, the processing of a multiparallelogram for a given corner c is as follows: In the first loop, perform all lookups around its vertices, compute each parallelogram prediction using the opposite corner, and add them up (see Figure 41). A parallelogram predicting corner c from its sibling corner altC is valid for prediction only if the vertices of altC.o, altC.n, and altC.p have already been processed by connectivity recursion (which invokes prediction). To test this, we use a vertex marking table M, which is set to true for a vertex if it has already been visited by the connectivity coding loop. The tests during prediction are then ((M[V[O[altC]]]>0) and (M[V[OV.p(altC)]]>0) and (M[V[OV.n(altC)]]>0))).

[0156] We extend this to test whether altC.o, altC.n, and altC.p corners are not connected to dummy points (using the isCornerVertexDummy predicate). In summary, we use corner altC for parallelogram prediction only if data is available (parallelogram vertices are not dummies, have already been visited, and are therefore available during decoding). At the end of the loop, we divide the prediction sum by the number of valid parallelograms found, we round the result, and finally use the result to compute the residual (location - prediction) appended to the end of the output vertex table oVertices.push_back(G[V[c]]-predPos).

[0157] If no valid parallelogram is found, and the corner before c is a dummy point, we fall back to delta coding using G[V[OV.n(c)]] as the prediction; otherwise, we fall back to G[V[OV.p(c)]].

[0158] At the end of the edgebreaker coding procedure, the two tables sVertices and oVertices are then entropy coded and stored in the bitstream.

[0159] An example of reconstructing the corner vertex positions during decoding is shown in Figures 42A and 42B. The process is similar to that of coding and follows the same order as for vertex and fan search. Therefore, the obtained predictions are equal to those of coding. Note that in decoding, we use Table D to quickly determine whether a vertex is a dummy point. Table D is initialized by the procedure initDecompression (see Figure 16). Also note that in the predictive decoding procedure, we use table shifts to handle CC (this is not the case in the predictive coding procedure).

[0160] Figures 44A and 44B show an example of the procedure posDecodeWithPrediction. Table D indicates whether each vertex is a dummy or not.

[0161] (Residual coding of UV coordinates using a major index table) Predictive coding of UV coordinates using a primary index table is similarly extended to position. The difference is, in some embodiments, the use of minimum stretching instead of multiparallelograms for prediction, for example, as shown in Figure 41, where we do not sum the predictions (it is not multiparallelogram), but stop at the first (in the sense of prediction) valid neighbor in a triangular sector and calculate the minimum stretching. An exemplary uvEncodeWithPrediction procedure is shown in Figure 43. Exemplary prediction statements are shown in Figures 44A-44B.

[0162] Figures 45A and 45B show an example of the uvEncodeWithPrediction procedure.

[0163] Figures 46A and 46B show an example of the statement predictUV. dvec2 and dvec3 are double-precision vectors.

[0164] In the predictUV procedure shown in Figures 44A and 44B, we can see that we have one branch that uses prediction and two other paths that use fallback. The method uses geometry to make a first choice and then performs a minimum stretch prediction or fallback to delta of UVs for dummies on the vertex chain, or otherwise performs an average of the previous and next corner UVs. Note that in the case of the minimum stretch choice, the exemplary embodiment stores the prediction residual (in the table oUVCoords) and the orientation bit to be used at decoding (in the table orientations). In the counterpart, the two other paths do not store orientation information.

[0165] With respect to position, decoding is symmetric. The exemplary embodiment uses a table shift for CC processing, and we use the same logical cases as for position. The input iUV is extracted as iUV=iUVCoords[startVertex+V[c]-uvSvalueIdx-uvSkipNum], which includes the CC offset startVertex, the starting value in a separate table uvSvalueIdx, and uvSkips in the case of dummy points. This extracted iUV can be the delta (case v=1 or v=2) or, otherwise, the residual. The exemplary embodiment then performs the same prediction or fallback as for UV coding, calculating the last UV with respect to position and returning it as UV[v]=predUV+iUV, where predUV is the predicted UV or fallback.

[0166] (Residual coding of UV coordinates using different index tables) When the model utilizes a separate index table for UV coordinates, the exemplary embodiment handles potential UV seams during prediction (in addition to a separate table for the starting UV, multiple CCs, and dummy points). When a separate index table is not used, the positions of vertices located on the boundary of a UV patch are duplicated, and the mesh contains at least the same number of CCs as the UV patch. This is why, in the previous section, we treated UVs as positions, and they have the same structure. In the counterpart, the exemplary embodiment that uses a separate index table for UV coordinates has a more "compact" geometry without UV-specific CCs, and it is possible to handle UV patch boundaries using a separate UV index table. Edge chains on a mesh where the UV coordinates differ on both sides are sometimes called UV seams.

[0167] Figure 47A shows the processing of UV seams using only the main index table. Duplicated locations (indicated by dashed ellipses) have different UV coordinates. Figure 47B shows the corner c used for predicting UV coordinates, with a separate index for corner c. p This section describes UV processing using a separate index table and selection strategy. Dashed lines represent UV seams, and vertices along the seams may have the same position but different UV coordinates on different sides of the seam.

[0168] Several exemplary embodiments extend the UV predictor search strategy to support UV seams. In some such embodiments, this is accomplished by storing specific codes in TC and OTC tables to guide the algorithm.

[0169] An exemplary embodiment implements a search strategy for effective predictive corners, as shown in Figures 45A-45B. In this method, dummy points are implicitly treated and simply check whether sibling corners already have reconstructed UV coordinates. Effective predictor c for the UV of corner c pThe strategy for finding is to search for a first triangle in the sector that lies on the same side seam as the triangle at corner c, where the largest UV coordinate is already predicted. An exemplary embodiment selects a first corner where two UVs are already predicted, if none are, selects the first one where one UV is already predicted, and if none are predicted, selects the first one. In the search, we can turn left first and then right or vice versa, which does not matter, but we must use the same order during decoding. A detailed procedure for carrying out this work, an example of encodeSeparateIndexAttributes, is shown in Figure 46.

[0170] In some embodiments, decoding involves performing the same process (using a shifted table) to reconstruct the predicted values, and then the last UV coordinates, as shown in Figures 47A-47B. Note that this stage does not specifically utilize the isSeam table, as the isSeam table was used during edgebreaker decoding to create several valid TC and OTC tables that handle separate indices. In this stage, the TC table is used to discover the UV index for each corner, and the OTC table is used to detect the UV seams. The exemplary embodiment uses -4 in the OTC during encoding and -1 in the OTC during decoding, because values ​​between 0 and -4 are used for other purposes.

[0171] Figures 48A and 48B show an example of the encodeSeparateIndexAttributes procedure.

[0172] Figures 49A and 49B show an example of the procedure decodeSeparateIndexAttributes.

[0173] Predicting attributes for each face. Several exemplary embodiments encode and decode per-face color or ID. In some such embodiments, per-face attributes are coded according to the order of faces in the CLERS table. Each symbol in the table corresponds to a triangle. Whenever we encode / decode a new triangle (do T=0 or T++), the exemplary embodiments operate to encode the per-face attributes directly during connectivity encoding (see Figures 12 and 13) and connectivity decoding (see Figures 18 and 19). The exemplary embodiments then include, in these steps, encoding the attributes of the triangles in the table. Some embodiments directly add values ​​(color, ID, ...) into a dedicated table by performing delta coding with the previous triangle in the table, for example. Some embodiments handle the starting triangle using a separate table, similar to the techniques described herein for the starting vertex when encoding per-vertex attributes. Such embodiments may have more efficient entropy coding.

[0174] An exemplary embodiment uses a sequence of triangles in the order of the CLERS table to encode a delta value for each triangle, either for color (e.g., 8-bit RGB) or an integer (e.g., group identifier, subdivision depth). Some more complex predictions can be assumed using the same strategy depending on the attribute type.

[0175] One example of a mesh coding method is to obtain information describing a mesh having multiple triangles, each triangle having an attribute, and to encode the mesh as an ordered sequence of symbols during edge breaker coding, each triangle corresponding to one of the relevant symbols, and to encode the attributes of the triangle, the attributes of the current triangle are encoded using a prediction based only on the attributes of at least one triangle preceding the current triangle in the ordered sequence of symbols. For example, the attributes of each triangle may be encoded as a delta value for the previous triangle in the sequence (the value of the attribute of the previous triangle acting as a prediction).

[0176] A mesh decoding method according to several embodiments includes obtaining an ordered sequence of symbols representing edge breaker coding of a mesh having a plurality of triangles, wherein each triangle has an attribute, and decoding the mesh and decoding the triangle's attribute, wherein the attribute of the current triangle is decoded using a prediction based only on the attribute of at least one triangle preceding the current triangle in the ordered sequence of symbols.

[0177] Predicting attributes per vertex can be particularly useful for mesh coding systems that deal with adaptive subdivisions specific to groups of triangles. For example, each triangle in a base mesh might receive an integer that specifies the number of subdivisions it must receive to reconstruct the complete mesh.

[0178] Some embodiments implement a technique (referred to here as variant A) for encoding an ID (e.g., a material ID) on a face-by-face basis. An example of code implementing variant A is given in Figures 59A and 59B. In such embodiments, for each corner processed during the cross-section defining a new triangle, the following actions may be performed: A decision is made as to whether the current corner defines an actual face or a dummy face (a face is a dummy if one of its corners is associated with a dummy vertex). A determination is made as to whether the current ID is equal to the previous encoded ID, and the result of the comparison is stored in a Boolean vector, faceIdEqualToPrevious, where the last size is the number of triangles in the input mesh (before adding dummy vertices). If the current ID is equal to the previous encoded ID, no further encoding of the current ID is required. However, if the current ID is not equal to the previous ID, the following steps are performed for coding the current ID. A determination is made as to whether any of the faces opposite each of the current face corners have already been coded. The current triangle is defined by three corners: currentCorner, ov.n(currentCorner), and ov.p(currentCorner). The "opposite triangle" is defined as containing ov.O(currentCorner), the "right triangle" is defined as containing ov.O[ov.n(currentCorner)], and the "left" triangle is defined as containing ov.O[ov.p(currentCorner)]. For each of those triangles (as shown in Figures 61A to 61C, the faces opposite each corner of the current triangle as defined by the corner index), if it has not already been encoded, the loop is performed first through the faces containing currentCorner=ov.O[ov.n(currentCorner)] until that face is no longer the current face on which we are encoding the ID value, and stops when a face that has already been encoded is found. If no previously encoded face is found, we restart and loop in the opposite direction, i.e., through the faces containing currentCorner=ov.O[ov.p(currentCorner)]. This generates three Boolean values ​​(decodedRight, decodedLeft, decodedFacing) for each corner corresponding to an actual face. For those values ​​that are true, we know the corresponding face ID values ​​(faceIdRight, faceIdLeft, faceIdFacing) which have already been encoded. When the Boolean values ​​are false, the corresponding faceIdRight / Left / Facing values ​​can be set to invalid values ​​(for example, -1 if we only consider ID values ​​≥ 0). Next, the following process is performed.

[0179] [Table 2]

[0180] The face can then be marked as encoded. This process produces four Boolean vectors (faceIdEqualToPrevious, faceIdIsRight, faceIdIsLeft, faceIdIsFacing) and a vector containing an unpredicted value (notPredictedFaceId).

[0181] Figures 61A to 61C show an example of a search order for corner[0]=c (the operation is repeated for corner[1]=cn, then for corner[2]=cp). Figure 61A shows the face where the ID should be coded and the adjacent triangles L, R, and F. (There are always three adjacents as dummy points are used to fill the holes.) Figure 61B shows a neighborhood lookup looping in the first direction across R, R1, R2, etc. Figure 61C shows a subsequent lookup in the opposite direction.

[0182] Some embodiments implement an alternative technique (referred to here as variant B) for encoding the ID on a per-face basis. Such a technique may be implemented as follows: • Stores information indicating whether the current ID is equal to the previous one in the first vector of Boolean values. If they are not equal, store either the absolute value of the current ID or the difference of the previous one in the second vector of integer values.

[0183] Some embodiments implement an alternative technique (referred to here as variant C) for encoding the ID on a face-by-face basis. Such a technique may be implemented as follows: • Stores information indicating whether the current ID is equal to the previous one in the first vector of Boolean values. If they are not equal, store information indicating whether the current ID is equal to the most recently coded ID in the second vector of Boolean values ​​that was different from the previous one. If they are not equal, store either the absolute value of the current ID or the difference between it and the previous one in a third vector of integer values.

[0184] Using a larger number of the most recently used values ​​and coding indices that point to them instead of absolute values ​​may be implemented in some embodiments, but this is unlikely to lead to increased efficiency.

[0185] For at least a sample set of meshes processed within a VDMC context, their alternative methods (B and C), while less efficient than A, are still simpler to implement and faster to run.

[0186] An example of code for performing encoding using Alternate A is shown in Figures 59A and 59B. Decoding using Alternate A (for example, using the procedure decodePerFaceAttributes) is performed in a manner similar to encoding, as shown in Figures 59A and 59B.

[0187] In some embodiments, a mesh coding method includes: obtaining information describing a mesh having a plurality of triangles, each triangle having an attribute (such as an ID); coding the mesh as an ordered sequence of symbols during edge breaker coding, each triangle corresponding to one of the relevant symbols; and coding the attributes of the triangles, the attributes of the current triangle are coded using predictions based on the attributes of at least one triangle adjacent to the current triangle in the mesh (the left, right, opposite, and / or previous triangles).

[0188] In some such embodiments, encoding the attributes of a triangle involves encoding a plurality of Boolean vectors (such as faceIdEqualToPrevious, faceIdIsRight, faceIdIsLeft, and / or faceIdIsFacing vectors), wherein the plurality of Boolean vectors are encoded such that, for at least a plurality of triangles, each attribute collectively represents the corresponding triangles for which it is predicted.

[0189] As will be explained in more detail below, arithmetic coding of multiple Boolean vectors can also be performed.

[0190] In some embodiments, a mesh decoding method includes obtaining an ordered sequence of symbols representing edge breaker coding of a mesh having a plurality of triangles, wherein each triangle has an attribute, and decoding the mesh and decoding the triangle's attribute, wherein the attribute of the current triangle is decoded using a prediction based on the attribute of at least one triangle adjacent to the current triangle in the mesh.

[0191] In some such embodiments, decoding the attributes of a triangle includes encoding a plurality of Boolean vectors (such as faceIdEqualToPrevious, faceIdIsRight, faceIdIsLeft, and / or faceIdIsFacing vectors), where the plurality of Boolean vectors collectively represent the corresponding triangles for which each attribute is expected for at least a plurality of triangles.

[0192] As will be explained in more detail below, arithmetic decoding of multiple Boolean vectors can also be performed.

[0193] (Entropy coding) Figure 50 shows an example of the `save` procedure, which involves entropy coding and syntax writing. During bitstream creation (see Figures 48A–48B), the exemplary embodiment first writes some metadata in the header, then saves the topology payload, and finally saves the attribute (location, UV coordinates) payload. Several tables for both topology and attributes are entropy coded before saving. Exemplary entropy coding is described in this section.

[0194] The exemplary embodiment supports two or more different arithmetic encoders, such as Dirac coding or RANS coding, as described in Non-Patent Document 10. Depending on the data, one or the other of these codecs may provide better compression. We also found that, depending on the other table, one or the other is better at compression, depending on the model being compressed. The exemplary embodiment provides two or more arithmetic codec implementation forms guided by several options. Table 6 below summarizes each type of data for which an arithmetic coder is available in some embodiments. The following sections further elaborate on the configuration / binarization used for some data tables depending on the type of arithmetic coding used.

[0195] (Arithmetic coding of the CLERS table) When encoding a non-binary sequence of CLERS symbols representing a topology code used to reconstruct the topology of a traversed mesh using an edge breaker algorithm, one or more of the following techniques may be used:

[0196] The selection of a suitable binarization process and the associated probabilistic model for the bins may depend on the specific nature of the sequence of symbols to be encoded, which is adjusted by the number of symbols (relative to the number of faces in the mesh), and the topological structure of this mesh (e.g., the structural regularity of the mesh in question, which potentially affects not only global symbol statistics but also the interrelationships between consecutive symbols).

[0197] Some embodiments assign variable-length codes to a set of symbols (for example, five symbols "C", "L", "E", "R", and "S" for the basic edge breaker) depending on these expected statistical distributions. One known choice is to assign {'C', 0, 1 bit}, {'S', 1, 3 bits}, {'L', 3, 3 bits}, {'R', 5, 3 bits}, {'E', 7, 3 bits}.

[0198] C=000

[0199] S=001

[0200] L=011

[0201] R=101

[0202] E=111

[0203] In some embodiments, when encoding using RANS, the CLERS table created from these codes is directly provided to the encoder.

[0204] In some embodiments, when encoding using Dirac coding, it is possible to encode using several contexts (ctx). In some such embodiments, b0 is encoded using ctx0, b1 is encoded using ctx1, and b2 is encoded using ctx2, so that when b0=0, the two subsequent bits do not need to be coded.

[0205] In some embodiments, the context is selected bin by bin, not only in response to the value of the previous bin for the current symbol, but also in response to the value of the previous symbol. This is useful for CLERS sequences because there is a strong correlation between consecutive symbols.

[0206] An exemplary embodiment uses a global 5-bit (32 possible values) state defined using two previous symbol values. Alternative embodiments use a simplified or extended state definition.

[0207] In some embodiments, one of the 32-bit_0 contexts is selected for encoding the sign of the first bit that defines whether the symbol is "C" or not. If the symbol is not "C", the second bit is encoded using one of the 32 contexts that model the value of bit_1. Then, depending on the value of bit_1, two sets of 32 contexts are used to encode bit_2. Finally, the global state is updated considering the latest encoded symbol. In our current implementation, the state has 5 bits of 32 states, namely bit0 and bit3 from the previous one in addition to 3 bits for the latest value. Not all of those states are necessarily actually used, and using a simple mapping, we will only instantiate 20 contexts as 5 values of the current symbol × 4 possible states held from the previous one. Then, in this example, the total number of contexts used will be 20×4 = 80 (4 for bit_0, bit_1, bit_2 IF bit_1 is 0, bit_2 IF bit_1 is 1).

[0208] When the number of symbols is not large, it can be seen that using many contexts is inefficient. Then, based on the number of symbols to be encoded, exemplary embodiments operate to adjust the number of global states to be used. For example, instead of using a 32-bit global state (where 5×4 = 20 are useful states), an exemplary embodiment can limit it to 3 bits or 5 global states when capturing only the value of the latest symbol.

[0209] In a variant form, an exemplary embodiment operates to test multiple combinations and encode using the selected combination and encode information indicating the selected combination into the bit stream.

[0210] Some embodiments operate to define a bootstrapping method that progressively uses more states, for example, by copying appropriate state values ​​to uninitialized states when transitioning the model based on specific triggers based on the number of symbols to be encoded or state values, or other factors such as statistics on input values ​​(e.g., variance of probabilistic encoded bits).

[0211] An example of pseudocode for controlling the Dirac encoder for the CLERS table is given in Figure 51.

[0212] In one exemplary embodiment of the mesh decoding method, a bitstream is obtained that includes an entropy-encoded CLERS table representing the connectivity of the connected components of the mesh. A sequence of bits is entropy-decoded from the CLERS table, where the sequence of bits represents a sequence of symbols including at least one most recently decoded symbol. The current bit is entropy-decoded from the CLERS table, where the current bit is decoded using a context determined at least partially (for example, based on two most recently decoded symbols) on at least one most recently decoded symbol. A mesh decoding device according to some embodiments comprises one or more processors configured to carry out such a method.

[0213] In some embodiments, the context is determined not only by the most recently decoded symbol but also by the most recently decoded bit in the bit sequence. In some embodiments, the context is further based on the two most recently decoded bits in the bit sequence.

[0214] In some embodiments of mesh coding methods, a CLERS table is obtained, where the CLERS table represents the connectivity of at least one connected component of the mesh. A sequence of bits representing at least a portion of the CLERS table is coded, where the sequence of bits represents a sequence of symbols from the CLERS table, including at least one most recently coded symbol. Entropy coding involves entropy coding the current bit in the CLERS table, where the current bit is coded using a context determined at least partially on at least one most recently coded symbol (for example, on two most recently coded symbols).

[0215] In some embodiments, the context is further based at least in part on the most recently encoded bit in the bit sequence. In some embodiments, the context is further based on the two most recently encoded bits in the bit sequence.

[0216] In some embodiments, the number of contexts used to decode the bits of the CLERS table is determined based on the number of symbols in the CLERS table. In some such embodiments, the number of contexts is determined based on whether the number of symbols in the CLERS table is greater than a threshold (for example, a threshold of 3000 symbols as one of the other possible options), a first number of contexts is used when the number of symbols is less than the threshold, and a second number of contexts is used when the number of symbols is greater than the threshold, the second number being greater than the first number. As an example of the other possibilities, the first number could be 5 and the second number could be 20.

[0217] (AC coding of attribute residuals) When coding vertex attributes such as position or texture coordinates, the exemplary embodiment operates to predict those values ​​and then encode only the residuals. Absolute reference values ​​can be coded separately to avoid reducing the performance of the arithmetic coder by compromising the statistics.

[0218] When encoding using RANS, the exemplary embodiment provides the encoder with a table created from these residuals.

[0219] In some embodiments, some control over the context may be exercised when encoding using Dirac. An exemplary embodiment coding scheme is used for arithmetic coding of motion vectors based on a Dirac arithmetic coder. The code shown in Figure 52 provides an example for reproducing parts of the coding principles used for motion vectors in some embodiments.

[0220] In this method, to encode triplets of signed integer values ​​(one for each coordinate of the 3D position), the exemplary embodiment uses 51 contexts. For each value, we first encode whether this value is 0 as one bit using the first context. If the value is not null, we encode the sign as one bit using the second context. Then, we encode whether the absolute value of the value to be encoded as one bit is 1 using the third context. If the absolute value of the value to be encoded is greater than 1, the absolute value of the value to be encoded minus 1 is expGolomb encoded using 2 × 7 contexts: 7 for the exponential (prefix) part and 7 for the suffix part. This is just one example, and different configurations may be used in other embodiments. In some embodiments, an offset or shift value is used in conjunction with expGolomb coding. Such coding schemes allow for better compression of the distribution with a large probability of the value -1 0 +1 and can be adapted to distributions that do not conform to the ideal distribution ideally handled by expGolomb coding. Further details regarding the example offset / shift are given in the example code for ExpGolomb coding shown in Figure 53.

[0221] This modeling may not be optimal for the distribution of location or other attribute residuals. Therefore, some embodiments use more elaborate models to achieve better results.

[0222] Some exemplary embodiments proceed as follows (given values ​​are examples of dimensional setting values). In some embodiments, the threshold before falling back to expGolomb coding is extended (this is set through a bias parameter). In one exemplary embodiment, 1. Encode 1.1 bits and signal whether the value is 0 or not. 2. Next, if it is not 0, encode the sign as 1 bit. We are left to encode newvalue = abs(initialvalue) - 1. 3. Next, a code for the maximum length bias (e.g., a simple code) is used to encode newvalue < bias, repeating 0s and 1s many times until the bit count equals the value to be encoded, and stopping after the bias value. 4. Then, if the last bit transmitted was 0, the value is ≥ bias, and we encode (newvalue - bias) using bias-applied exp - golomb coding.

[0223] The context of step 3 can be adjusted by the previously encoded value having an absolute value smaller than the bias. Those numbers can be limited to be any value from 1 to the bias. A schematic example of the code for some embodiments is shown in FIG. 54. In this embodiment, the performance is affected by the loop order, component - by - component / residual - by - residual. Depending on the predictor used and the type of data, an optimal order can be selected (for our parallelogram predictor positions, it is better to loop through the value assigned to one component before passing through another component), and for texture coordinates and stretch minimization prediction, it is better to pass through the components first. When the state is replicated component - by - component, there is no such ambiguity, but the performance / complexity can be affected.

[0224] In some embodiments, the mesh encoding method includes obtaining attribute information representing the prediction residuals of the attributes of the vertices of a triangular mesh. The attribute residuals are entropy - encoded in a bitstream, where for at least one attribute of at least one vertex, the entropy encoding comprises encoding a first bit indicating whether the residual is 0, encoding a second bit indicating the sign of the residual, and encoding a sequence of bits, where the absolute value of the residual is determined at least in part based on the length of the sequence. <着

[0225] In some embodiments of the mesh decoding method, the attribute information is obtained from the bitstream, where the attribute information represents the prediction residuals of the attributes of the vertices of the triangular mesh. Entropy decoding of the attribute residuals is performed. For at least one attribute of at least one vertex, the entropy decoding includes decoding a first bit indicating whether the residual is zero, decoding a second bit indicating the sign of the residual, and decoding a sequence of bits, where the absolute value of the residual is determined based at least in part on the length of the sequence, and reconstructing the attribute of the vertex based at least in part on the determined residual.

[0226] In some embodiments, the sequence of bits ends in a pre-determined stop bit. For example, the sequence of bits can be zero or more "0" bits and is terminated by a "1" bit that serves as a stop bit. Alternatively, the sequence of bits can be zero or more "1" bits and is terminated by one "0" bit that serves as a stop bit. In some cases, the absolute value of the residual is determined as the length of the sequence.

[0227] In some embodiments, the entropy decoding further comprises decoding the codeword using exponential Golomb coding, where the absolute value of the residual value is determined based at least in part on the codeword. In some such embodiments, the codeword has a pre-determined maximum length of the sequence of bits and is decoded in response to the determination that it does not end in a pre-determined stop bit. In some such embodiments, the absolute value of the residual is determined as the sum of the value represented by the codeword and a bias value.

[0228] The value of the vertex attribute can be reconstructed by obtaining the prediction of the vertex attribute and adding the determined residual value to the prediction of the attribute. The attribute can be, for example, UV coordinates, position coordinates, or another attribute type.

[0229] Entropy coding of material attributes or other attributes per face. In some embodiments, when encoding faceID values ​​where materialID attributes or long-span consecutive triangles are assigned to share a common value using the encoding scheme detailed above, the array of values ​​defined by the encoding process may be arithmetically coded using the following scheme. The encoding / prediction process results in the following: The four Boolean vectors are faceIdEqualToPrevious, faceIdIsRight, faceIdIsLeft, and faceIdIsFacing. • The vector notPredictedFaceId contains unpredicted values.

[0230] The four Boolean vectors are arithmetically encoded using four distinct contexts.

[0231] A vector containing unsigned face ID values ​​is encoded using exponential Golomb coding and a number of contexts for encoding the suffix and prefix portions. These numbers are adjusted to take into account the range of faceId values, thereby balancing complexity / performance trade-offs. As an example, both prefix and suffix coding may use the same number of contexts as the number of bits used to represent the complete set of ID values. For values ​​ranging from 0 to 255, we might decide to use eight contexts for both.

[0232] In some embodiments, exponential Golomb coding may use an optimization "shift" parameter, as in the previous example.

[0233] In some embodiments, if the input face ID or material ID value is not unsigned, an additional parameter may be added to convert the range of values ​​so that all converted values ​​are positive. Alternatively, some embodiments convert the values ​​to a positive range using the following:

[0234]

number

[0235] Or any similar bijective transformation that converts the ID range to positive values.

[0236] In some embodiments, the five vectors may be encoded separately, resulting in five coded arrays of bytes.

[0237] In some embodiments, the number of elements coded into each of these vectors (the original vector size) can be signaled. The coded byte array size enables parallel decoding of each of these.

[0238] In some alternative embodiments, all information is encoded into a single byte array using multiple contexts, and decoding is performed incrementally, appropriately selecting the context by performing context selection during the decoding process mirroring selection in the encoding stage.

[0239] An example of Dirac AC entropy coding for coding material IDs on a per-face basis is shown in Figure 60.

[0240] In some embodiments of the mesh decoding method, a CLERS table representing the edge breaker coding of the triangular mesh topology is obtained from the bitstream. The triangular mesh topology is reconstructed from the CLERS table. Ordered attribute information representing the attributes of each face in the triangular mesh is obtained from the bitstream. The ordered attribute information is decoded to determine the attribute values ​​of each face.

[0241] In some embodiments of the mesh coding method, information describing the topology of a triangular mesh is obtained, and for each face in the triangular mesh, information indicating the attribute values ​​of each face is obtained. A CLERS table representing the edge breaker coding of the topology is coded in the bitstream. For each face, attribute information indicating the attribute values ​​of each face is coded in the bitstream.

[0242] In some embodiments, the ordered attribute information comprises at least a first vector, the first vector comprising entries for each of a plurality of faces, each entry in the first vector being a Boolean value indicating whether the attribute value of the current face is the same as the attribute value of the most recently decoded face.

[0243] In some embodiments, the ordered attribute information further comprises at least a second vector, the second vector comprising entries for each of a plurality of faces, each entry in the second vector being a Boolean value indicating whether the attribute of each current face is the same as the attribute value of an adjacent face in a first direction from the current face. In some such embodiments, the second vector includes entries only for faces that do not have an attribute value determined by the first vector.

[0244] In some embodiments, the ordered attribute information further comprises at least a third vector, the third vector comprising entries for each of a plurality of faces, each entry in the third vector being a Boolean value indicating whether the attribute of each current face is the same as the attribute value of an adjacent face in a second direction from the current face. In some such embodiments, the third vector includes entries only for faces that do not have an attribute value determined by the first or second vector.

[0245] In some embodiments, the ordered attribute information further comprises at least a fourth vector, the fourth vector comprising entries for each of a plurality of faces, each entry in the fourth vector being a Boolean value indicating whether the attribute of each current face is the same as the attribute value of an adjacent face in a third direction from the current face. In some such embodiments, the fourth vector includes entries only for faces that do not have attribute values ​​determined by the first, second, or third vectors.

[0246] In some embodiments, the ordered attribute information further comprises at least a fifth vector, the fifth vector comprising entries for each of a plurality of faces, and each entry in the fourth vector containing the attribute value of the respective current face. In some such embodiments, the fifth vector comprises entries only for faces that do not have attribute values ​​determined by the first, second, third, or fourth vectors.

[0247] In some embodiments, for example, with respect to a directional scheme known for both the encoder and decoder, the first direction is to the right, the second direction is to the left, and the third direction is forward. However, in different embodiments, these directions may be arranged differently.

[0248] In some embodiments, different vectors may be entropy-coded using different contexts. In one example, a first vector is entropy-coded using a first context, and a second vector is entropy-coded using a second context different from the first. In another example, a first vector is entropy-coded using a first context, a second vector is entropy-coded using a second context different from the first, and a fifth vector is entropy-coded using a fifth context different from the first and second contexts.

[0249] In some embodiments, the attribute value of each face is an integer value. In some embodiments, the attribute value of each face is a face ID value. In some embodiments, the attribute value of each face identifies a color. In some embodiments, the attribute value of each face identifies a material.

[0250] (Example of low-level syntax) This section describes the syntax used in some exemplary embodiments.

[0251] The syntax of an exemplary bitstream structure is shown in Table 1.

[0252] [Table 3]

[0253] Exemplary attribute bitstreams (DATA_ATTRIBUTES) from several embodiments are shown in Table 2.

[0254] [Table 4]

[0255] Illustrative configuration information bitstreams (CFG_INFO) from several embodiments are shown in Table 3.

[0256] [Table 5]

[0257] Exemplary data header information bitstreams (DATA_HEADER) in several embodiments are shown in Table 4.

[0258] [Table 6]

[0259] Exemplary data topology bitstreams (DATA _TOPOLOGY) in several embodiments are given in Table 5.

[0260] [Table 7-1]

[0261] [Table 7-2]

[0262] Table 6 shows examples of data position attribute bitstreams according to several embodiments.

[0263] [Table 8]

[0264] Table 7 shows exemplary UV coordinate attribute bitstreams from several embodiments.

[0265] [Table 9]

[0266] (Example extended syntax) Exemplary extended syntax that may be used in some embodiments is given in Appendix A. The syntax elements and their associated semantics described in Appendix A are given only as examples. This disclosure intends to provide variations and extensions of the syntax elements described in Appendix A.

[0267] (performance) In the context of V-DMC, the performance of the exemplary embodiment of the overall compression described herein was compared to the equivalent compression using Google Draco. See the results in the figure showing the compression gain for base mesh coding. Figure 55 shows the compression performance of the coding method described herein compared to Google Draco for base mesh coding in V-DMC. The results for each CTC test sequence were averaged over 300 frames. PSNR is not presented as it does not affect them. The results show an overall average gain of 5.4% above the Draco rate.

[0268] (Example system hardware) Exemplary embodiments of encoders and / or decoders (collectively, coders) configured to implement the embodiments described herein may be implemented using a system such as the system in Figure 56. Figure 56 is a block diagram of an example of a system in which various aspects and embodiments are implemented. System 1000 may be implemented as a device comprising various components described below and configured to implement one or more of the aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital TV receivers, personal video recording systems, connected household electrical appliances, and servers. The elements of System 1000 may be implemented individually or in combination as a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of System 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, System 1000 is communicably coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, the system 1000 is configured to implement one or more of the embodiments described herein.

[0269] System 1000 includes, for example, at least one processor 1010 configured to execute instructions loaded therein to implement various embodiments described herein. The processor 1010 may include embedded memory, input / output interfaces, and various other circuits known in the art. System 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040 which may include, but is not limited to, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, magnetic disk drives, and / or optical disk drives, as well as non-volatile and / or volatile memory. Storage device 1040 may, in non-limiting examples, include internal storage devices, accessory storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0270] System 1000 includes, for example, an encoder / decoder module 1030 configured to process data to give encoded or decoded video, the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module that may be included in the device to perform encoding and / or decoding functions. As is known, the device may include one or both encoding and decoding modules. Furthermore, the encoder / decoder module 1030 may be implemented as a separate element of System 1000 or may be incorporated into the processor 1010 as a combination of hardware and software, as is known to those skilled in the art.

[0271] Program code to be loaded onto a processor 1010 or encoder / decoder 1030 implementing the various embodiments described herein may be stored in a storage device 1040 and subsequently loaded into memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more of various items during the execution of the processing described herein. Such stored items include, but are not limited to, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of expressions, formulas, operations, and operational logic.

[0272] In some embodiments, the internal memory of the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for the processing required during encoding or decoding. However, in other embodiments, external memory of the processing device (for example, the processing device may be either the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or storage device 1040, for example, dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used, for example, to store the television's operating system. In at least one embodiment, high-speed external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group, also known as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (General Purpose Video Coding, a new standard developed by JVET (Joint Video Experts Team)).

[0273] Inputs to the elements of system 1000 can be provided through various input devices, as shown in block 1130. Such input devices include, but are not limited to, (i) a radio frequency (RF) portion for receiving RF signals transmitted wirelessly by a broadcaster, for example, (ii) a component (COMP) input terminal (or set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Other examples not shown in Figure 1C include composite video.

[0274] In various embodiments, the input device of block 1130 has respective relevant input processing elements known in the art. For example, the RF portion may be associated with elements suitable for (i) selecting a desired frequency (also called selecting a signal or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) again band-limiting to a narrower band of frequencies to select a signal frequency band (for example, sometimes called a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) multiplexing to select a desired stream of data packets. The RF portion of various embodiments includes one or more elements for performing these functions, such as implementers, frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF portion may include a tuner that performs various functions of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a frequency near the baseband) or to the baseband. In one embodiment of a set-top box, the RF section and its associated input processing elements receive an RF signal transmitted over a wired (e.g., cable) medium, filter it, down-convert it, and filter it again to a desired frequency band to perform frequency selection. Various embodiments may rearrange the order of the elements described above (and others), remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements in between existing elements, such as inserting an amplifier and an analog-to-digital converter. In various embodiments, the RF section includes an antenna.

[0275] Furthermore, the USB and / or HDMI terminals may include their respective interface processors for connecting the system 1000 to other electronic devices over USB and / or HDMI connections. It should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented as needed, for example, in a separate input processing IC or within the processor 1010. Similarly, aspects of USB or HDMI interface processing may be implemented as needed, in a separate interface IC or within the processor 1010. The demodulated, error-corrected, and multiplexed streams are fed to various processing elements, such as the processor 1010 and encoder / decoder 1030, which work in conjunction with memory and storage elements to process the data stream as needed for presentation on an output device.

[0276] Various elements of system 1000 may be provided within an integrated housing. Within the integrated housing, the various elements are interconnected with each other using a suitable connection configuration 1140, for example, an internal bus known in the art, including an inter-IC (I2C) bus, wiring, and a printed circuit board, and data can be transmitted between them.

[0277] System 1000 includes a communication interface 1050 that enables communication with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 1060. The communication interface 1050 may also include, but is not limited to, a modem or network card and the communication channel 1060, which may be implemented in a wired and / or wireless medium, for example.

[0278] In various embodiments, data is streamed to or provided to system 1000 using a wireless network such as a Wi-Fi network, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). In these embodiments, the Wi-Fi signal is received via a communication channel 1060 and a communication interface 1050 adapted for Wi-Fi communication. In these embodiments, communication channel 1060 is generally connected to an access point or router that provides access to an external network, including the Internet, to enable streaming applications and other over-the-top communications. In other embodiments, data streamed to system 1000 is provided using a set-top box that distributes data via an HDMI connection in input block 1130. Yet another embodiment provides data streamed to system 1000 using an RF connection in input block 1130. As described above, various embodiments provide data in a non-streaming manner. Furthermore, various embodiments use wireless networks other than Wi-Fi, for example, a cellular network or a Bluetooth network.

[0279] System 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. In various embodiments, the display 1100 may include, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 may be for a television, tablet, laptop, cell phone (mobile phone), or other device. The display 1100 may also be integrated into other components (for example, as in a smartphone) or separate (for example, an external monitor for a laptop). In various examples of embodiments, the other peripheral devices 1120 may include one or more of a standalone digital video disc (or digital multi-purpose disc) (DVR for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of System 1000. For example, a disc player performs the function of playing back the output of System 1000.

[0280] In various embodiments, control signals are communicated between system 1000 and the display 1100, speaker 1110, or other peripheral devices 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable inter-device control with or without user intervention. Output devices may be coupled to system 1000 via dedicated connections through their respective interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to system 1000 using communication channel 1060 via communication interface 1050. The display 1100 and speaker 1110 may be integrated into a single unit with other components of system 1000 in an electronic device such as a television. In various embodiments, the display interface 1070 includes a display driver, such as a timing controller (TCon) chip.

[0281] The display 1100 and speaker 1110 may, alternatively, be separate from one or more of the other components, for example, if the RF portion of input 1130 is part of a separate set-top box. In various embodiments where the display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection, such as an HDMI port, a USB port, or a COMP output.

[0282] The embodiments may be implemented by the processor 1010, by computer software implemented by hardware, or by a combination of hardware and software. In non-limiting embodiments, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type appropriate for the technical environment and, in non-limiting examples, may be implemented using any suitable data storage technology such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type appropriate for the technical environment and, in non-limiting examples, may include one or more of microprocessors, general-purpose computers, dedicated computers, and processors based on multicore architectures.

[0283] (Additional embodiments) A mesh coding method according to some embodiments comprises coding a first connected component of a mesh using edge breaker coding, identifying uncoded corners in the mesh after coding the first connected component of the mesh, and coding a second connected component of the mesh using the uncoded corners identified as entry corners.

[0284] A mesh coding method according to several embodiments comprises: identifying at least one non-manifold edge in the mesh; duplicating vertices along the non-manifold edge to separate the non-manifold edge into at least two manifold edges; stitching together two of the at least two manifold edges; and coding the mesh using edge breaker coding after stitching together two of the at least two manifold edges.

[0285] A mesh coding method according to some embodiments comprises: identifying at least one non-manifold edge in a mesh, the non-manifold edge being defined by a parent vertex; preprocessing the mesh by duplicating the parent vertex along the non-manifold edge to separate the non-manifold edge into at least two manifold edges, each manifold edge being defined by a child vertex, each child vertex being duplicated from its parent vertex; coding the preprocessed mesh into a bitstream using edge breaker coding; and providing information in the bitstream that identifies the parent vertex of the child vertex.

[0286] A mesh decoding method according to several embodiments comprises: obtaining information from a bitstream that defines multiple connected components of a non-manifold mesh coded in an edge breaker; obtaining information from a bitstream that associates child vertices with parent vertices; decoding at least two of the connected components, each of which has child vertices associated with its respective parent vertex; and reconstructing the non-manifold mesh by duplicating and removing the child vertices.

[0287] A mesh coding method according to several embodiments comprises preprocessing a mesh having holes by creating dummy vertices, wherein the dummy vertices are connected to each vertex along the outer perimeter of the hole; coding the preprocessed mesh into a bitstream using edge breaker coding; coding information identifying the dummy vertices into the bitstream; and coding at least one attribute for each of a plurality of vertices in the preprocessed mesh, wherein the attribute is not coded for the dummy vertices.

[0288] A mesh decoding method according to some embodiments comprises obtaining information to encode a mesh from a bitstream, obtaining information to identify at least one dummy vertex in the mesh from the bitstream, and decoding attribute information for a plurality of vertices in the mesh from the bitstream, wherein the attribute information is decoded but not decoded for at least one dummy vertex.

[0289] In some embodiments, decoding attribute information for multiple vertices comprises predicting attribute information for multiple vertices, wherein the attribute information is not predicted for dummy points.

[0290] Some embodiments further include the removal of dummy points.

[0291] A mesh coding method according to some embodiments comprises coding the connectivity of a mesh, entropy coding a first table of the starting point positions of vertices in the mesh, and separately entropy coding a second table of the point residuals of vertices in the mesh.

[0292] A mesh decoding method according to some embodiments includes decoding information indicating the connectivity of a mesh, entropy decoding a first table of the starting point positions of vertices in the mesh, and separately entropy decoding a second table of point residuals of vertices in the mesh.

[0293] A mesh coding method according to several embodiments comprises obtaining information defining a mesh having multiple connected components (CCs), wherein each vertex in the CC has a unique global range index; shifting the index of the vertex in the CC from the global range index to the CC range index for each CC; and coding each of the CCs using the CC range index.

[0294] A mesh decoding method according to several embodiments comprises obtaining information encoding multiple connected components (CCs), wherein each vertex in the CC has its own CC range index, decoding each of the CCs, and shifting the vertex index from the CC range index to the global range index for each vertex.

[0295] A mesh coding method according to several embodiments comprises: obtaining information describing a mesh having a plurality of triangles, each triangle having attributes; coding the mesh as an ordered sequence of symbols during edge breaker coding, each triangle corresponding to one of the relevant symbols; and coding the attributes of the triangle, the attributes of the current triangle are coded using predictions based only on the attributes of at least one triangle preceding the current triangle in the ordered sequence of symbols.

[0296] A mesh decoding method comprises obtaining an ordered sequence of symbols representing the edge breaker coding of a mesh having multiple triangles, wherein each triangle has attributes, and decoding the mesh and decoding the attributes of the triangle, wherein the attributes of the current triangle are decoded using a prediction based only on the attributes of at least one triangle preceding the current triangle in the ordered sequence of symbols.

[0297] A mesh coding method according to some embodiments comprises: obtaining information describing a mesh having a plurality of triangles, each triangle having attributes; coding the mesh as an ordered sequence of symbols during edge breaker coding, each triangle corresponding to one of the relevant symbols; and coding the attributes of the triangle, the attributes of the current triangle are coded using predictions based on the attributes of at least one triangle adjacent to the current triangle in the mesh.

[0298] In some embodiments, encoding triangle attributes involves encoding a plurality of Boolean vectors such that, for at least a plurality of triangles, each attribute collectively represents the corresponding triangles for which it is predicted.

[0299] Some embodiments further include performing arithmetic coding of multiple Boolean vectors.

[0300] In some embodiments, the attribute is an identifier.

[0301] A mesh decoding method according to several embodiments comprises obtaining an ordered sequence of symbols representing the edge breaker coding of a mesh having a plurality of triangles, wherein each triangle has attributes, and decoding the mesh and decoding the attributes of the triangle, wherein the attributes of the current triangle are decoded using predictions based on the attributes of at least one triangle adjacent to the current triangle in the mesh.

[0302] In some embodiments, decoding the attributes of a triangle involves decoding a plurality of Boolean vectors such that, for at least a plurality of triangles, each attribute collectively represents the corresponding triangles for which it is predicted.

[0303] In some embodiments, decoding multiple Boolean vectors involves arithmetic decoding of multiple Boolean vectors.

[0304] In some embodiments, the attribute is an identifier.

[0305] An apparatus according to several embodiments comprising one or more processors, wherein the apparatus is configured to carry out any of the methods described herein.

[0306] Apparatus according to some embodiments comprises at least one processor and a computer-readable medium storing instructions for performing any of the methods described herein.

[0307] Some embodiments include a computer-readable medium (e.g., a non-temporary medium) that stores instructions for carrying out any of the methods described herein.

[0308] Some embodiments include a computer-readable medium for storing a mesh encoded according to one of the methods described herein.

[0309] Some embodiments include a signal that carries a mesh encoded according to one of the methods described herein.

[0310] Some exemplary embodiments provide a lossless mode that allows mesh reconstruction using "poor" connectivity, enabling lossless recreation of non-manifold surfaces if desired.

[0311] Some exemplary embodiments provide support for holes and boundaries through "virtual" dummy points that are coded without coding the attributes of the dummy points.

[0312] Some exemplary embodiments provide support for multiple linked components (CCs).

[0313] Some embodiments provide an extension of multiparallelogram prediction for geometry to support virtual dummy points.

[0314] Some embodiments add support for texture coordinate coding using unique index tables and separate index tables. Some such embodiments provide an extension of stretch prediction to support dummy points and separate index tables when needed.

[0315] Some embodiments provide specific binarization schemes and AC context processing for use with Dirac arithmetic coding.

[0316] Some embodiments include support for per-face attributes (identifiers or colors) coded using a triangle chain.

[0317] It should be noted that one or more different hardware elements in the embodiments described are referred to as “modules” that perform (i.e., implement, execute, etc.) the various functions described herein with respect to each module. Modules as used herein include hardware (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field-programmable gate arrays (FPGAs), one or more memory devices) that are considered suitable by those skilled in the art for a given implementation. It should be noted that each described module may also include executable instructions to perform one or more functions described as being performed by each module, and such instructions may take the form of or include hardware (i.e., wired) instructions, firmware instructions, software instructions, etc., and may be stored in any suitable non-temporary computer-readable medium such as RAM, ROM, etc.

[0318] While features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in computer programs, software, or firmware embedded in computer-readable media for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital multipurpose disks (DVDs). Processors associated with software can be used to implement radio frequency transceivers for use in WTRUs, UEs, terminals, base stations, RNCs, or any host computer.

[0319] [Table 10-1]

[0320] [Table 10-2]

[0321] [Table 10-3]

[0322] [Table 10-4]

[0323] [Table 10-5]

[0324] [Table 10-6]

[0325] Table 10-7

[0326] Table 10-8

[0327] Table 10-9

[0328] Table 10-10

[0329] Table 10-11

[0330] Table 10-12

[0331] Table 10-13

[0332] Table 10-14

[0333] Table 10-15

[0334] Table 10-16

[0335] Table 10-17

[0336] Table 10-18

Claims

1. A mesh composite method, From the bitstream, obtain a CLERS table representing the edge breaker coding of the triangular mesh topology, Reconstructing the topology of the triangular mesh from the CLERS table, From the bitstream, obtain ordered attribute information representing the attributes of each face in the triangular mesh, For each face, the ordered attribute information is decoded in order to determine the attribute value of each face, A method for providing this.

2. A mesh composite device, and at least, From the bitstream, obtain a CLERS table representing the edge breaker coding of the triangular mesh topology, Reconstructing the topology of the triangular mesh from the CLERS table, From the bitstream, obtain ordered attribute information representing the attributes of each face in the triangular mesh, For each face, the ordered attribute information is decoded in order to determine the attribute value of each face, A device comprising one or more processors configured to perform the following.

3. The method according to claim 1, or the apparatus according to claim 2, wherein the ordered attribute information comprises at least a first vector, the first vector comprising entries for each of the plurality of faces, and each entry in the first vector is a Boolean value indicating whether the attribute value of the current face is the same as the attribute value of the most recently decoded face.

4. The method according to claim 1 or claim 3 dependent on claim 1, or the apparatus according to claim 2 or claim 3 dependent on claim 2.

5. The method according to claim 4, dependent on claim 1, or the apparatus according to claim 4, dependent on claim 2, wherein the second vector includes entries only for faces that do not have attribute values ​​determined by the first vector.

6. The method according to claim 1 or any one of claims 3 to 5 dependent on claim 1, wherein the ordered attribute information further comprises at least a third vector, the third vector comprising entries for each of a plurality of faces, and each entry in the third vector being a Boolean value indicating whether the attribute of each current face is the same as the attribute value of an adjacent face in a second direction from the current face, or the apparatus according to claim 2 or any one of claims 3 to 5 dependent on claim 2.

7. The method according to claim 6, dependent on claim 1, or the apparatus according to claim 6, dependent on claim 2, wherein the third vector includes entries only for faces that do not have attribute values ​​determined by the first vector or the second vector.

8. The method according to claim 1 or any one of claims 3 to 7 dependent on claim 1, wherein the ordered attribute information further comprises at least a fourth vector, the fourth vector comprising entries for each of the plurality of faces, and each entry in the fourth vector being a Boolean value indicating whether the attribute of the respective current face is the same as the attribute value of an adjacent face in a third direction from the current face, or the apparatus according to claim 2 or any one of claims 3 to 7 dependent on claim 2.

9. The method according to claim 8, dependent on claim 1, or the apparatus according to claim 8, dependent on claim 2, wherein the fourth vector includes entries only for faces that do not have attribute values ​​determined by the first vector, the second vector, or the third vector.

10. The ordered attribute information further comprises at least a fifth vector, the fifth vector comprising entries for each of the plurality of faces, each entry in the fourth vector comprising attribute values ​​for the respective current faces, according to the method of claim 1 or any one of claims 3 to 9 dependent on claim 1, or the apparatus according to claim 2 or any one of claims 3 to 9 dependent on claim 2.

11. The method according to claim 10, dependent on claim 1, or the apparatus according to claim 10, dependent on claim 2, wherein the fifth vector includes entries only for faces that do not have attribute values ​​determined by the first vector, the second vector, the third vector, or the fourth vector.

12. The method according to any one of claims 4 to 11 dependent on claim 1, wherein the first direction is to the right, the second direction is to the left, and the third direction is to the front, or the apparatus according to any one of claims 4 to 11 dependent on claim 2.

13. Entropy decoding of the first vector from the bitstream using the first context, Entropy decoding of at least the second vector from the bitstream using a second context different from the first context, A method according to any one of claims 4 to 12 dependent on claim 1, further comprising the above, or the apparatus according to any one of claims 4 to 12 dependent on claim 2.

14. Entropy decoding of the first vector from the bitstream using the first context, Entropy decoding of at least the second vector from the bitstream using at least a second context different from the first context, Entropy decoding of the fifth vector from the bitstream using a fifth context different from the first and second contexts, A method according to any one of claims 10 to 13 dependent on claim 1, further comprising the above, or the apparatus according to any one of claims 10 to 13 dependent on claim 2.

15. The method according to claim 1 or any one of claims 3 to 14 dependent on claim 1, wherein the attribute value of each face is an integer value, or the apparatus according to claim 2 or any one of claims 3 to 14 dependent on claim 2.

16. The method according to claim 1 or any one of claims 3 to 15 dependent on claim 1, wherein the attribute value of each face is a face ID value, or the apparatus according to claim 2 or any one of claims 3 to 15 dependent on claim 2.

17. The method according to claim 1 or any one of claims 3 to 16 dependent on claim 1, wherein the attribute values ​​of each face identify a color, or the apparatus according to claim 2 or any one of claims 3 to 16 dependent on claim 2.

18. The attribute values ​​of each face identify a material, according to the method of claim 1 or any one of claims 3 to 17 dependent on claim 1, or the apparatus according to claim 2 or any one of claims 3 to 17 dependent on claim 2.

19. A mesh coding method, To obtain information describing the topology of a triangular mesh, For each face in the triangular mesh, information indicating the attribute value of each face is obtained, Encoding a CLERS table representing the edge breaker coding of the topology into the bitstream, For each of the aforementioned faces, attribute information indicating the attribute value of each of the aforementioned faces is encoded in the bitstream, A method that includes [a certain feature].

20. A mesh coding device, wherein at least, To obtain information describing the topology of a triangular mesh, For each face in the triangular mesh, information indicating the attribute value of each face is obtained, Encoding a CLERS table representing the edge breaker coding of the topology into the bitstream, For each of the aforementioned faces, attribute information indicating the attribute value of each of the aforementioned faces is encoded in the bitstream, A device comprising one or more processors configured to perform the following.

21. The method according to claim 19, or the apparatus according to claim 20, wherein the ordered attribute information comprises at least a first vector, the first vector comprising entries for each of the plurality of faces, and each entry in the first vector being a Boolean value indicating whether the attribute value of the current face is the same as the attribute value of the most recently encoded face.

22. The method according to claim 19 or claim 21 dependent on claim 19, or the apparatus according to claim 20 or claim 21 dependent on claim 20.

23. The method according to claim 22, dependent on claim 19, or the apparatus according to claim 22, dependent on claim 20, wherein the second vector includes entries only for faces that do not have attribute values ​​determined by the first vector.

24. The method according to claim 19 or any one of claims 21 to 23 dependent on claim 19, wherein the ordered attribute information further comprises at least a third vector, the third vector comprising entries for each of a plurality of faces, and each entry in the third vector being a Boolean value indicating whether the attribute of each current face is the same as the attribute value of an adjacent face in a second direction from the current face, or the apparatus according to claim 20 or any one of claims 21 to 23 dependent on claim 20.

25. The method according to claim 24, dependent on claim 19, or the apparatus according to claim 24, dependent on claim 20, wherein the third vector includes entries only for faces that do not have attribute values ​​determined by the first vector or the second vector.

26. The method according to claim 19 or any one of claims 21 to 25 dependent on claim 19, wherein the ordered attribute information further comprises at least a fourth vector, the fourth vector comprising entries for each of a plurality of faces, and each entry in the fourth vector being a Boolean value indicating whether the attribute of each current face is the same as the attribute value of an adjacent face in a third direction from the current face, or the apparatus according to claim 20 or any one of claims 21 to 25 dependent on claim 20.

27. The method according to claim 26, dependent on claim 19, or the apparatus according to claim 26, dependent on claim 20, wherein the fourth vector includes entries only for faces that do not have attribute values ​​determined by the first vector, the second vector, or the third vector.

28. The ordered attribute information further comprises at least a fifth vector, the fifth vector comprising entries for each of the plurality of faces, and each entry in the fourth vector comprising attribute values ​​for the respective current faces, according to the method of claim 19 or any one of claims 21 to 27 dependent on claim 19, or the apparatus according to claim 20 or any one of claims 21 to 27 dependent on claim 20.

29. The method according to claim 28, dependent on claim 19, or the apparatus according to claim 28, dependent on claim 20, wherein the fifth vector includes entries only for faces that do not have attribute values ​​determined by the first vector, the second vector, the third vector, or the fourth vector.

30. The method according to any one of claims 22 to 29 dependent on claim 19, wherein the first direction is to the right, the second direction is to the left, and the third direction is to the front, or the apparatus according to any one of claims 22 to 29 dependent on claim 20.

31. Entropy encoding the first vector from the bitstream using the first context, Entropy encoding at least the second vector from the bitstream using at least a second context different from the first context, A method according to any one of claims 22 to 30 dependent on claim 19, further comprising, the, or the apparatus according to any one of claims 22 to 30 dependent on claim 20.

32. Entropy encoding the first vector from the bitstream using the first context, Entropy encoding at least the second vector from the bitstream using at least a second context different from the first context, Entropy encoding the fifth vector from the bitstream using a fifth context different from the first and second contexts, A method according to any one of claims 28 to 30 dependent on claim 19, further comprising, the above, or the apparatus according to any one of claims 28 to 30 dependent on claim 20.

33. The method according to claim 19 or any one of claims 21 to 32 dependent on claim 19, wherein the attribute value of each face is an integer value, or the apparatus according to claim 20 or any one of claims 21 to 32 dependent on claim 20.

34. The method according to claim 19 or any one of claims 21 to 33 dependent on claim 19, wherein the attribute value of each face is a face ID value, or the apparatus according to claim 20 or any one of claims 21 to 33 dependent on claim 20.

35. The method according to claim 19 or any one of claims 21 to 34 dependent on claim 19, wherein the attribute value of each face identifies a color, or the apparatus according to claim 20 or any one of claims 21 to 34 dependent on claim 20.

36. The attribute values ​​of each face identify a material, according to the method of claim 19 or any one of claims 21 to 35 dependent on claim 19, or the apparatus according to claim 20 or any one of claims 21 to 35 dependent on claim 20.

37. A mesh decoding method, From the bitstream, obtain attribute information representing the predicted residuals of the vertices of the triangular mesh, The attribute residual is entropy-decoded, wherein for at least one attribute of at least one vertex, the entropy decoding is performed as follows: Decoding a first bit that indicates whether the residual is zero, Decoding the second bit that indicates the sign of the residual, Decoding a sequence of bits, wherein the absolute value of the residual is determined at least partially on the length of the sequence. Reconstructing the attributes of the vertex based at least partially on the determined residuals, Including, A method that includes [a certain feature].

38. A mesh decoding device, wherein at least, From the bitstream, obtain attribute information representing the predicted residuals of the vertices of the triangular mesh, The attribute residual is entropy-decoded, wherein for at least one attribute of at least one vertex, the entropy decoding is performed as follows: Decoding a first bit that indicates whether the residual is zero, Decoding the second bit that indicates the sign of the residual, Decoding a sequence of bits, wherein the absolute value of the residual is determined at least partially on the length of the sequence. Reconstructing the attributes of the vertex based at least partially on the determined residuals, Including, A device comprising one or more processors configured to perform the following.

39. The method according to claim 37, or the apparatus according to claim 38, wherein the sequence of bits ends in a predetermined stop bit, and the absolute value of the residual is determined as the length of the sequence.

40. The method according to claim 37 or claim 39 dependent on claim 37, wherein, with respect to at least one attribute, the entropy decoding is the decoding of a codeword using exponential Golomb coding, and the absolute value of the residual is determined at least in part on the codeword, or the apparatus according to claim 38 or claim 39 dependent on claim 38.

41. The method according to claim 40, dependent on claim 37, or the apparatus according to claim 40, dependent on claim 38, wherein the codeword is decoded in response to a determination that the sequence of bits has a predetermined maximum length and does not end in a predetermined stop bit.

42. The method according to claim 40 or 41 dependent on claim 37, wherein the absolute value of the residual is determined as the sum of the values ​​represented by the codeword and the bias value, or the apparatus according to claim 40 or 41 dependent on claim 38.

43. A method according to claim 37 or claims 39 to 42 dependent on claim 37, further comprising obtaining a prediction of the attribute of the vertex, wherein reconstructing the attribute includes adding the determined residual value to the prediction of the attribute, or an apparatus according to claim 38 or claims 39 to 42 dependent on claim 38.

44. The attribute comprises at least one position coordinate or at least one UV coordinate, according to the method of claim 37 or claims 39 to 43 dependent on claim 37, or the apparatus according to claim 38 or claims 39 to 43 dependent on claim 38.

45. A mesh coding method, Obtaining attribute information that represents the predicted residuals of the vertex attributes of a triangular mesh, Entropy coding the attribute residuals in the bitstream, wherein for at least one attribute of at least one vertex, the entropy coding is performed. Encoding a first bit that indicates whether the residual is zero, Encoding a second bit that indicates the sign of the residual, Encoding a sequence of bits, wherein the absolute value of the residual is determined at least partially on the length of the sequence. Including, A method that includes [a certain feature].

46. A mesh decoding device, wherein at least, Obtaining attribute information that represents the predicted residuals of the vertex attributes of a triangular mesh, Entropy coding the attribute residuals in the bitstream, wherein for at least one attribute of at least one vertex, the entropy coding is performed. Encoding a first bit that indicates whether the residual is zero, Encoding a second bit that indicates the sign of the residual, Encoding a sequence of bits, wherein the absolute value of the residual is determined at least partially on the length of the sequence. Including, A device comprising one or more processors configured to perform the following.

47. The method according to claim 45, or the apparatus according to claim 46, wherein the sequence of bits ends in a predetermined stop bit, and the absolute value of the residual is determined as the length of the sequence.

48. The method according to claim 45 or claim 47 dependent on claim 45, wherein, with respect to at least one attribute, the entropy coding is to encode a codeword using exponential Golomb coding, the codeword is determined at least in part on the codeword, or the apparatus according to claim 46 or claim 37 dependent on claim 46.

49. The method according to claim 48, dependent on claim 45, or the apparatus according to claim 48, dependent on claim 46, wherein the codeword is encoded in response to a decision that the sequence of bits has a predetermined maximum length and does not end in a predetermined stop bit.

50. The method according to claim 48 or 49 dependent on claim 45, or the apparatus according to claim 48 or 49 dependent on claim 46, wherein the value represented by the codeword is determined based on the difference between the absolute value of the residual and the bias value.

51. A method according to claim 45 or claims 47 to 50 dependent on claim 45, further comprising obtaining a prediction of the attribute of the vertex, wherein the residual value is determined as the difference between the attribute and the prediction of the attribute, or an apparatus according to claim 46 or claims 47 to 50 dependent on claim 46.

52. The method according to claim 45 or claims 47 to 51 dependent on claim 45, wherein the attribute comprises at least one position coordinate or at least one UV coordinate, or the apparatus according to claim 46 or claims 47 to 51 dependent on claim 46.

53. A mesh decoding method, Obtain a bitstream containing an entropy-encoded CLERS table representing the connectivity of the connected components of the mesh, Entropy decoding of a sequence of bits from the CLERS table, wherein the sequence of bits represents a sequence of symbols including at least one most recently decoded symbol. Entropy decoding of the current bit from the CLERS table, wherein the current bit is decoded using a context determined at least partially based on the at least one most recently decoded symbol. A method that includes [a certain feature].

54. A mesh decoding device, wherein at least, Obtain a bitstream containing an entropy-encoded CLERS table representing the connectivity of the connected components of the mesh, Entropy decoding of a sequence of bits from the CLERS table, wherein the sequence of bits represents a sequence of symbols including at least one most recently decoded symbol. Entropy decoding of the current bit from the CLERS table, wherein the current bit is decoded using a context determined at least partially based on the at least one most recently decoded symbol. A device comprising one or more processors configured to perform the following.

55. The method according to claim 53, or the apparatus according to claim 54, wherein the context is at least partially further based on the most recently decoded bit in the sequence of bits.

56. The method according to claim 53, or the apparatus according to claim 54, wherein the context is further based on the two most recently decoded bits in the sequence of bits.

57. The method according to claim 53, or the apparatus according to claim 54, wherein the number of contexts used to decode the bits of the CLERS table is determined based on the number of symbols in the CLERS table.

58. The method according to claim 57, dependent on claim 53, or the apparatus according to claim 57, dependent on claim 54, wherein the number of contexts is determined based on whether the number of symbols in the CLERS table is greater than a threshold, a first number of contexts is used when the number of symbols is less than the threshold, and a second number of contexts is used when the number of symbols is greater than the threshold, the second number being greater than the first number.

59. A mesh coding method, Obtaining the CLERS table, which represents the connectivity of the mesh's connected components, Entropy encoding a sequence of bits from the CLERS table, wherein the sequence of bits represents a sequence of symbols including at least one most recently encoded symbol. Entropy encoding the current bit in the CLERS table, wherein the current bit is encoded using a context determined at least partially based on the at least one most recently encoded symbol. A method that includes [a certain feature].

60. A mesh coding device, wherein at least, Obtaining the CLERS table, which represents the connectivity of the mesh's connected components, Entropy encoding a sequence of bits from the CLERS table, wherein the sequence of bits represents a sequence of symbols including at least one most recently encoded symbol. Entropy encoding the current bit in the CLERS table, wherein the current bit is encoded using a context determined at least partially based on the at least one most recently encoded symbol. A device comprising one or more processors configured to perform the following.

61. The method according to claim 59, or the apparatus according to claim 60, wherein the context is at least partially based on the most recently encoded bit in the sequence of bits.

62. The method according to claim 59, or the apparatus according to claim 60, wherein the context is further based on the two most recently encoded bits in the sequence of bits.

63. The method according to claim 59, or the apparatus according to claim 60, wherein the number of contexts used to decode the bits of the CLERS table is determined based on the number of symbols in the CLERS table.

64. The method according to claim 63, dependent on claim 59, or the apparatus according to claim 63, dependent on claim 60, wherein the number of contexts is determined based on whether the number of symbols in the CLERS table is greater than a threshold, a first number of contexts is used when the number of symbols is less than the threshold, and a second number of contexts is used when the number of symbols is greater than the threshold, the second number being greater than the first number.

Images