Chart-based mesh compression

Abstract

A method executed at a decoder includes receiving a coded video bitstream including one or more 2D meshes corresponding to a 3D mesh. The method includes reconstructing one or more boundary vertices of each of the one or more 2D meshes. The method includes determining whether a reconstruction mode for reconstructing one or more interior vertices of each of the one or more 2D meshes is one of (i) a sampling mode and (ii) a non-sampling mode. The method includes reconstructing the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode. The method further includes reconstructing the 3D mesh based on the reconstructed one or more boundary vertices of each of the one or more 2D meshes and the reconstructed one or more interior vertices of each of the one or more 2D meshes.

Description

[Technical Field]

[0001] This application claims priority to U.S. Provisional Application No. 63 / 331,711, filed April 15, 2022, and U.S. Patent Application No. 18 / 127,432, filed March 28, 2023, the disclosures of which are incorporated herein by reference in their entireties.

[0002] This disclosure relates to a set of advanced video coding techniques, and more particularly to video-based mesh compression, including coding methods for chart-based mesh compression. [Background technology]

[0003] Advanced three-dimensional (3D) representations of the world are enabling more immersive forms of interaction and communication. To achieve realism in 3D representations, 3D models are becoming increasingly sophisticated, and a significant amount of data is associated with the creation and consumption of these 3D models. 3D meshes are widely used in 3D modeling for immersive content.

[0004] A 3D mesh may be composed of multiple polygons that describe the surface of a volumetric object. Dynamic mesh sequences can require large amounts of data because the mesh sequence may contain a significant amount of information that changes over time. Therefore, efficient compression techniques are needed to store and transmit such content.

[0005] Mesh compression standards IC, MESHGRID, and FAMC were previously developed to deal with dynamic meshes with constant connectivity and time-varying geometry and vertex attributes, but these standards do not consider time-varying attribute maps and connectivity information.

[0006] Furthermore, generating dynamic meshes with consistent connectivity, especially under real-time constraints, is also a challenge for volumetric acquisition techniques, as this type of dynamic mesh content is not supported by existing standards. Summary of the Invention [Means for solving the problem]

[0007] According to one or more embodiments, a method executed by at least one processor of a decoder includes receiving a coded video bitstream including one or more two-dimensional (2D) meshes corresponding to a three-dimensional (3D) mesh. The method includes reconstructing one or more boundary vertices of each of the one or more 2D meshes. The method includes determining whether a reconstruction mode for reconstructing one or more interior vertices of each of the one or more 2D meshes is one of (i) a sampling mode and (ii) a non-sampling mode. The method includes reconstructing the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode. The method includes reconstructing the 3D mesh based on the reconstructed one or more boundary vertices of each of the one or more 2D meshes and the reconstructed one or more interior vertices of each of the one or more 2D meshes.

[0008] According to one or more embodiments, a decoder comprises at least one memory configured to store program code and at least one processor configured to read the program code and operate as directed by the program code. The program code includes receiving code configured to cause the at least one processor to receive a coded video bitstream including one or more two-dimensional (2D) meshes corresponding to a three-dimensional (3D) mesh. The program code includes first reconstruction code configured to cause the at least one processor to reconstruct one or more boundary vertices of each of the one or more 2D meshes. The program code includes first determination code configured to cause the at least one processor to determine whether a reconstruction mode for reconstructing one or more interior vertices of each of the one or more 2D meshes is one of (i) a sampling mode and (ii) a non-sampling mode. The program code includes second reconstruction code configured to cause the at least one processor to reconstruct the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode. The program code includes third reconstruction code configured to cause the at least one processor to reconstruct the 3D mesh based on the one or more reconstructed boundary vertices of each of the one or more 2D meshes and the one or more reconstructed interior vertices of each of the one or more 2D meshes.

[0009] According to one or more embodiments, The computer-implemented method in the computer program isreceiving a coded video bitstream including one or more two-dimensional (2D) meshes corresponding to a three-dimensional (3D) mesh; reconstructing one or more boundary vertices of each of the one or more 2D meshes; determining whether a reconstruction mode for reconstructing one or more interior vertices of each of the one or more 2D meshes is one of (i) a sampling mode and (ii) a non-sampling mode; reconstructing the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode; and reconstructing the 3D mesh based on the reconstructed one or more boundary vertices of each of the one or more 2D meshes and the reconstructed one or more interior vertices of each of the one or more 2D meshes. include . [Brief explanation of the drawings]

[0010] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings.

[0011] [Figure 1] 1 is a schematic block diagram of a communication system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a schematic block diagram of a streaming system according to an embodiment of the present disclosure. [Figure 3] 1 is a schematic block diagram of a video encoder and decoder according to an embodiment of the present disclosure. [Figure 4] 1 illustrates an example of UV parameterized mapping from a 3D mesh segment to a 2D chart, according to an embodiment of the present disclosure. [Figure 5] Shows an example of different UV parameterizations where a 3D mesh segment is mapped to multiple separate charts. [Figure 6] 1 illustrates an exemplary 2D UV atlas with multiple charts according to an embodiment of the present disclosure. [Figure 7]10 illustrates an example of boundary vertices in a 2D patch, according to an embodiment of the present disclosure. [Figure 8] 1 illustrates an exemplary geometry encoding pipeline according to an embodiment of the present disclosure. [Figure 9(A)-(B)] 10 illustrates an example of chart boundary detection according to an embodiment of the present disclosure. [Figure 10(A)-(B)] 1A and 1B illustrate example geometry maps before and after filling unoccupied pixels, respectively, according to an embodiment of the present disclosure. [Figure 11] 1 illustrates an exemplary geometry decoding pipeline according to an embodiment of the present disclosure. [Figure 12] Here is an example of finding interior sampling points inside a polygon defined by a boundary. [Figure 13] 1 shows an exemplary flowchart for encoding a video bitstream according to an embodiment of the present disclosure. [Figure 14] 1 shows an exemplary flowchart for decoding a coded video bitstream according to an embodiment of the present disclosure. [Figure 15] FIG. 1 illustrates a computer system suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] The following detailed description of exemplary embodiments refers to the accompanying drawings, in which the same reference numbers in different drawings can identify the same or similar elements.

[0013] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit implementations to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations. Furthermore, one or more features or components of one embodiment may be incorporated into or combined with another embodiment (or one or more features of another embodiment). Additionally, in the flowcharts and descriptions of operations provided below, it is understood that one or more operations may be omitted, one or more operations may be added, one or more operations may occur (at least partially) concurrently, or the order of one or more operations may be rearranged.

[0014] It will be apparent that the systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual specific control hardware or software code used to implement these systems and / or methods is not limiting of the implementation. Thus, the operations and behavior of the systems and / or methods are described herein without reference to specific software code. It will be understood that software and hardware may be designed to implement the systems and / or methods based on the description herein.

[0015] Although particular combinations of features are recited in the claims and / or disclosed herein, these combinations are not intended to limit the disclosure of possible implementations. Indeed, many of these features can be combined in ways not specifically recited in the claims and / or disclosed herein. Although each dependent claim listed below may depend directly on only one claim, the disclosure of possible implementations includes each dependent claim in combination with all other claims in the claim set.

[0016] No element, act, or instruction used herein is construed as critical or essential unless explicitly stated as such. Also, as used herein, the articles "a" and "an" are intended to include one or more items and may be used interchangeably with "one or more." Where only one item is intended, the term "one" or similar language is used. Also, as used herein, the terms "have," "having," "include," "including," and the like are intended to be open-ended terms. Furthermore, the phrase "based on" is intended to mean "based at least in part on" unless otherwise specified. Furthermore, phrases such as "at least one of [A] and [B]" or "at least one of [A] or [B]" shall be understood to include A only, B only, or both A and B.

[0017] Throughout this specification, references to "one embodiment," "an embodiment," or similar language mean that a particular feature, structure, or characteristic described in connection with the illustrated embodiment is included in at least one embodiment of the solution. Thus, throughout this specification, "in one embodiment," "in an embodiment," and similar language may, but do not necessarily, all refer to the same embodiment.

[0018] Furthermore, the described features, advantages, and characteristics of the present disclosure may be combined in any suitable manner in one or more embodiments. Those skilled in the art will recognize, in light of the description herein, that the present disclosure may be practiced without one or more specific features or advantages of a particular embodiment. In other instances, certain embodiments may exhibit additional features and advantages that may not be present in all embodiments of the present disclosure.

[0019] Embodiments of the present disclosure relate to compressing meshes. A mesh may be composed of several polygons that describe the surface of a volumetric object. Each polygon can be defined by its vertices in 3D space and information about how the vertices are connected, called connectivity information. Optionally, vertex attributes, such as color and normals, may be associated with mesh vertices. Attributes can also be associated with the surface of a mesh by utilizing mapping information that parameterizes the mesh in a 2D attribute map. Such mapping can be defined using a set of parametric coordinates, called UV coordinates or texture coordinates, associated with the mesh vertices. The 2D attribute map can be used to store high-resolution attribute information, such as texture, normals, and displacement. The high-resolution attribute information can be used for various purposes, such as texture mapping and shading.

[0020] As mentioned above, 3D meshes or dynamic meshes can require a large amount of data because they can consist of a significant amount of information that changes over time. Existing standards do not consider time-varying attribute maps and connectivity information. Existing standards also do not support volumetric acquisition techniques that generate dynamic meshes with constant connectivity, especially under real-time conditions.

[0021] Therefore, a new mesh compression standard is needed to directly handle dynamic meshes that have time-varying connectivity information and may optionally have time-varying attribute maps. Embodiments of the present disclosure enable efficient compression techniques for storing and transmitting such dynamic meshes. Embodiments of the present disclosure enable lossy and / or lossless compression for various applications, such as real-time communication, storage, free-viewpoint video, AR and VR, etc.

[0022] According to one or more embodiments of the present disclosure, a method, system, and non-transitory storage medium for dynamic mesh compression are provided. The embodiments of the present disclosure can also be applied to static meshes, where only a single frame of the mesh or mesh content does not change over time.

[0023] One or more embodiments of the present disclosure for realizing the encoding and decoding structures of the present disclosure will be described with reference to FIGS.

[0024] 1 shows a schematic block diagram of a communication system 100 according to one embodiment of the present disclosure. The system 100 may include at least two terminals 110, 120 interconnected via a network 150. In the case of unidirectional data transmission, the first terminal 110 may locally encode video data, which may include mesh data, for transmission to the other terminal 120 via the network 150. The second terminal 120 may receive the other terminal's coded video data from the network 150, decode the coded data, and display the decoded video data. Unidirectional data transmission may be common in media serving applications, etc.

[0025] 1 shows a second pair of terminals 130, 140 provided to support two-way transmission of coded video, such as may occur during a video conference. For two-way transmission of data, each terminal 130, 140 can code video data captured at a local location for transmission to the other terminal over network 150. Each terminal 130, 140 can also receive coded video data transmitted by the other terminal, decode the coded data, and display the recovered video data on a local display device.

[0026] In FIG. 1 , terminals 110-140 may be, for example, servers, personal computers, smartphones, and / or any other type of terminal. For example, terminals 110-140 may be laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 150 represents any number of networks that convey coded video data between terminals 110-140, including, for example, wired and / or wireless communication networks. Communication network 150 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this discussion, the architecture and topology of network 150 may not be important to the operation of the present disclosure, unless otherwise described herein.

[0027] 2 shows the arrangement of a video encoder and decoder in a streaming environment as an example of an application of the disclosed subject matter. The disclosed subject matter can be used with other video-enabled applications including, for example, video conferencing, digital TV, and storage of compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0028] 2, the streaming system 200 may include a capture subsystem 213 that includes a video source 201 and an encoder 203. The streaming system 200 may further include at least one streaming server 205 and / or at least one streaming client 206.

[0029] The video source 201 may, for example, create a stream 202 including a 3D mesh and metadata associated with the 3D mesh. The 3D mesh may be composed of several polygons that describe the surface of a volumetric object. For example, the 3D mesh may include multiple vertices in 3D space, with each vertex associated with a 3D coordinate (e.g., x, y, z). The video source 201 may, for example, include a 3D sensor (e.g., a depth sensor) or 3D imaging technology (e.g., a digital camera) and a computing device configured to generate the 3D mesh using data received from the 3D sensor or 3D imaging technology. The sample stream 202, which may have a high amount of data compared to an encoded video bitstream, can be processed by an encoder 203 coupled to the video source 201. The encoder 203 may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoder 203 may also generate an encoded video bitstream 204. The encoded video bitstream 204, which may have a lower amount of data compared to the uncompressed stream 202, may be stored on the streaming server 205 for future use. One or more streaming clients 206 may access the streaming server 205 to retrieve a video bitstream 209, which may be a copy of the encoded video bitstream 204.

[0030] The streaming client 206 may include a video decoder 210 and a display 212. The video decoder 210 may, for example, decode a video bitstream 209, which is an inbound copy of the encoded video bitstream 204, and create an outbound video sample stream 211 that may be rendered on the display 212 or another rendering device (not shown). In some streaming systems, the video bitstreams 204, 209 may be encoded according to a particular video coding / compression standard.

[0031] FIG. 3 is an example diagram of a framework 300 for dynamic mesh compression and mesh reconstruction using an encoder and decoder.

[0032] 3, framework 300 may include encoder 301 and decoder 351. Encoder 301 may include one or more input meshes 305, one or more meshes with UV atlas 310, occupancy map 315, geometry map 320, attribute map 325, and metadata 330. Decoder 351 may include decoded occupancy map 335, decoded geometry map 340, decoded attribute map 345, decoded metadata 350, and reconstructed mesh 360.

[0033] According to one or more embodiments of the present disclosure, the input mesh 305 may include one or more frames, each of which may be pre-processed by a series of operations and used to generate a mesh with a UV atlas 310. By way of example, the pre-processing operations may include, but are not limited to, tracking, parameterization, remeshing, voxelization, etc. In some embodiments, the pre-processing operations may occur only on the encoder side and not on the decoder side.

[0034] The mesh with UV atlas 310 may be a 2D mesh. The 2D mesh may be a chart of vertices, each associated with a coordinate in 2D space (e.g., a 2D coordinate). Each vertex in the 2D mesh may be associated with a corresponding vertex in a 3D mesh, where the vertices in the 3D mesh are associated with a coordinate in 3D space. The compressed 2D mesh may be a version of the 2D mesh with reduced information compared to an uncompressed 2D mesh. For example, the 2D mesh may be sampled at a sampling rate that includes the points at which the compressed 2D mesh was sampled. The 2D mesh with UV atlas may be a mesh where each vertex of the mesh may be associated with a UV coordinate on the 2D atlas. The mesh with UV atlas 310 may be processed and converted into multiple maps based on the sampling. As an example, the UV atlas 310 may be processed and converted into an occupancy map, a geometry map, and an attribute map based on the sampling of the 2D mesh with UV atlas. The generated occupancy map 335, geometry map 340, and attribute map 345 can be encoded using an appropriate codec (e.g., HVEC, VVC, AV1, AVS3, etc.) and transmitted to a decoder. In some embodiments, metadata (e.g., connectivity information, etc.) can also be transmitted to the decoder.

[0035] In some embodiments, the decoder side can reconstruct a mesh from the decoded 2D map. Post-processing and filtering can also be applied to the reconstructed mesh. In some examples, metadata may be signaled to the decoder side for the purpose of 3D mesh reconstruction. If the boundary vertices of each patch are signaled, the occupancy map can be inferred from the decoder side.

[0036] According to certain aspects, decoder 351 can receive the encoded occupancy map, geometry map, and attribute map from the encoder. Decoder 315 can decode the occupancy map, geometry map, and attribute map using appropriate techniques and methods in addition to the embodiments described herein. In some embodiments, decoder 351 can generate decoded occupancy map 335, decoded geometry map 340, decoded attribute map 345, and decoded metadata 350. One or more reconstruction filters and techniques can be used to reconstruct input mesh 305 into reconstructed mesh 360 based on decoded occupancy map 335, decoded geometry map 340, decoded attribute map 345, and decoded metadata 350. In some embodiments, metadata 330 can be sent directly to decoder 351, and decoder 351 can generate reconstructed mesh 360 using the metadata based on decoded occupancy map 335, decoded geometry map 340, and decoded attribute map 345. Post-filtering techniques may also be applied to the reconstructed mesh 360, including but not limited to remeshing, parameterization, tracking, voxelization, etc.

[0037] According to some embodiments, a 3D mesh can be partitioned into a number of segments (or patches / charts). Each segment can consist of a set of connected vertices associated with geometry, attributes, and connectivity information. As shown in FIG. 4, the UV parameterization process maps a mesh segment 400 to a 2D chart (402, 404) in a 2D UV atlas. Each vertex in the mesh segment can be assigned a 2D UV coordinate in the 2D UV atlas. Vertices in the 2D chart (e.g., a 2D mesh) can form connected components with their 3D counterparts. The geometry, attributes, and connectivity information of each vertex can be inherited from their 3D counterparts as well.

[0038] According to some embodiments, a 3D mesh segment can also be mapped to multiple separate 2D charts. When a 3D mesh segment is mapped to separate 2D charts, a vertex in the 3D mesh segment may correspond to multiple vertices in the 2D UV atlas. As shown in Figure 5, a 3D mesh segment 500 that may correspond to 3D mesh segment 400 may be mapped to two 2D charts (502A, 502B) in the 2D UV atlas instead of a single chart. As shown in Figure 5, 3D vertices v1 and v4 have two 2D corresponding vertices v1' and v4', respectively.

[0039] FIG. 6 shows an example of a general 2D UV atlas 600 for a 3D mesh that includes multiple charts, where each chart may include multiple (e.g., three or more) vertices associated with their 3D geometry, attributes, and connectivity information.

[0040] Boundary vertices may be defined in 2D UV space. As shown in Figure 7, a filled vertex is a boundary vertex because it is on the boundary edge of a connected component (patch / chart). A boundary edge may be determined by checking whether the edge appears in only one triangle. Geometry information (e.g., 3D xyz coordinates) and 2D UV coordinates may be signaled in the bitstream.

[0041] Dynamic mesh sequences can require large amounts of data because they can consist of a significant amount of information that changes over time, thus requiring efficient compression techniques to store and transmit such content.

[0042] The embodiments of the present disclosure can be used individually or combined in any order. Furthermore, each of the method (or embodiment), encoder, and decoder can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium.

[0043] 8 shows an exemplary geometry encoding pipeline 800 according to one or more embodiments. The encoding pipeline 800 may be performed by the encoder 301. The geometry encoding pipeline may include partitioning (802) a mesh into connected components (e.g., charts or 2D meshes). After partitioning, chart boundary extraction (804) and chart classification (806) may occur.

[0044] In one or more examples, each chart can be coded using either a raw chart mode or a sampling mode based on the chart's statistics. In the raw chart mode, interior vertices can be directly coded (808) using prediction. In the sampling mode, the chart is adaptively sampled (810) in 2D UV space, packed (812) into a 2D geometry image atlas, and subsequently coded (818). Adaptive sampling (810) may be applied to allow for different charts with different sampling rates. The geometry image is then coded (818) using a video codec. Boundary vertices are extracted and coded (816) using prediction, quantization, and entropy coding. In one or more examples, the texture map may be updated using a recoloring algorithm to transfer colors from the original texture map to a new texture map based on the reconstructed geometry.

[0045] According to one or more embodiments, boundary vertices can be detected for each chart. A boundary vertex may be a vertex that lies on a boundary edge, where a boundary edge is an edge associated with a single triangle. As shown in Figures 9(A) and 9(B), there are two types of boundaries (e.g., exterior boundaries and interior boundaries). Figures 9(A) and 9(B) show examples of chart boundary detection. Bold vertices and edges may lie on exterior boundaries, dashed vertices and edges lie on interior boundaries, and other vertices and edges do not belong to any boundary. In Figure 9(A), there are no holes with one exterior boundary, and in Figure 9(B), there is one hole with one exterior boundary and one interior boundary.

[0046] For each chart, there is one exterior boundary loop (e.g., the solid boundary in FIGS. 9(A) and 9(B)). The exterior boundary loop may be defined by vertices B0, B1, B2, B3, B4, B5, B6, and B7. These vertices may be referred to as boundary vertices. As shown in FIGS. 9(A) and 9(B), the edge defined by the boundary vertices appears in a single triangle. For example, the edge defined between vertices B0 and B1 appears in a single triangle. In one or more examples, the exterior boundary loop may define the exterior polygon of the chart, and all other vertices may be inside this polygon. In one or more examples, there may be an interior boundary loop inside the exterior boundary loop (e.g., the dashed line in FIG. 9(B)). An interior boundary loop may be a loop defined by vertices that do not define a triangle. For example, the configuration in FIG. 9(A) does not include any interior boundary loops, while the configuration in FIG. 9(B) includes at least one interior boundary loop (e.g., the dashed line). As shown in Figure 9(B), dashed lines do not define triangles. Interior boundary loops may correspond to holes in the chart. In one or more examples, a vertex that does not belong to either an exterior or interior boundary loop may be identified as an interior vertex.

[0047] In some embodiments, the encoder reparameterization algorithm can eliminate holes inside the chart. By eliminating the holes on the encoder side, internal boundary loops can be eliminated from the chart (e.g., 2D mesh).

[0048] According to one or more embodiments, each chart can be coded differently based on the characteristics of the chart and / or one or more application scenarios. In some embodiments, two coding methods can be used. The first method can be a sampling-based method, in which the chart is sampled at a predetermined sampling rate (e.g., step size), and the resulting geometry and attribute maps can be coded by a video codec. The second method can be a raw chart coding mode, in which interior vertices and connectivity between vertices can be coded directly without sampling.

[0049] According to one or more embodiments, a control flag can be signaled initially in the high-level syntax to indicate whether adaptive sampling is applied. If the flag is true, adaptive sampling can be enabled so that different sampling rates can be applied to each chart. If the flag is false, adaptive sampling can be disabled so that the same sampling rate can be applied to all charts.

[0050] In one or more examples, a base sampling rate (BS) may be signaled in the high-level syntax regardless of whether adaptive sampling is enabled. The base sampling rate may be used in one or more of the following aspects: If adaptive sampling is enabled, the base sampling rate may be used as a predictor. Each chart may signal its sampling rate by its difference from the base sampling rate (e.g., S-BS). If adaptive sampling is disabled, the base sampling rate may be used as a constant sampling rate for all charts. The base sampling rate may be used as a quantization step in coding of border UV coordinates in charts, including raw charts, allowing for more efficient coding of border information.

[0051] You may signal a UV coordinate offset from the original sampling position for each chart. (offset u ,offset v ) can be used to find the sampled vertex in the packed geometry image. This allows for different placement of the texture and geometry images.

[0052] According to one or more embodiments, the geometry map may be encoded by one or more video codecs. To improve coding efficiency of the geometry map, unoccupied pixels may be filled with neighboring occupied pixels in the geometry map, as shown in Figures 10(A) and 10(B). Figure 10(A) shows an example geometry map before filling the unoccupied pixels, and Figure 10(B) shows an example geometry map after filling the unoccupied pixels. In one or more examples, occupancy information is not encoded in the bitstream but can instead be inferred from bound vertices. At the decoder side, the occupancy map can be subtracted from the UV coordinates of the bound vertices, and the occupied pixel values ​​in the geometry map can be used for mesh reconstruction.

[0053] Boundary information, including the UV and XYZ coordinates of boundary vertices, can be predicted, quantized, and entropy coded in the bitstream. The quantization step size may be set by the encoder to trade off quality and bitrate. Boundary vertices with different UV coordinates may have the same XYZ coordinates because they originate from the same 3D point. To ensure that the mesh reconstructed by the decoder is accurate across the entire 3D boundary edge, a 3D-to-2D mapping function is maintained; otherwise, the 3D reconstruction may contain holes or cracks that can significantly degrade the visual quality of the reconstructed 3D mesh. Therefore, a UV-to-XYZ (e.g., called UV2XYZ) index array may be signaled to indicate the mapping function for boundary vertices. The UV2XYZ index array may be a 1D array of indices that map each 2D UV vertex to a 3D XYZ vertex.

[0054] In one or more examples, for each chart, the number of boundary loops in the chart may be signaled, and for each boundary loop, the number of vertices in the boundary loop may be signaled. Exterior boundary loops may be signaled before other interior boundary loops. The following information may be signaled to convey boundary information: (i) 2D UV coordinates for each chart, (ii) UV2XYZ index array for each chart, and (iii) unique 3D XYZ coordinates. Duplicate 3D coordinates may be removed, but the correspondence may be preserved in the UV2XYZ index array. The unique boundary XYZ coordinates may be coded once and shared by all charts. Because the encoder can guarantee that each chart can have only one exterior boundary loop, the syntax table may be further simplified by skipping signaling the number of boundary loops for each chart. UV2XYZ coding maintains the uniqueness of boundary xyz vertices when they have multiple correspondences in different charts, ensuring that the reconstructed geometry is accurate across the entire boundary without holes or cracks.

[0055] According to one or more embodiments, boundary XYZ vertex information can be predicted from either a reconstructed geometry image (e.g., in the case of a sampled chart) or from previously reconstructed boundary XYZ vertices (e.g., in the case of a raw chart). The boundary XYZ vertex prediction residual can be derived by subtracting the original geometry from the predicted value as follows:

number

[0056] JPEG0007771393000002.jpg19170

[0057] The prediction residual may be quantized and entropy coded. For example, the residual may be quantized with a scalar value as follows:

number

[0058] Here, QS xyz is the quantization step size that can be signaled in the high-level syntax.

[0059] The reconstructed boundary XYZ vertices can be derived as follows:

number

[0060] Predictions for boundary XYZ vertices may be determined as follows: For boundary XYZ vertices that have correspondence in at least one sampled chart, predictions may be derived from the reconstructed geometry image as follows.

number

[0061] Here, Set(i) is the set of all the indices of the boundary UV vertices that correspond to the same boundary XYZ vertex and index i.

number

[0062] JPEG0007771393000007.jpg17170

number

[0063] For bounding XYZ vertices whose corresponding UV coordinates all belong to the raw chart, the prediction can be derived from the previously reconstructed bounding XYZ vertices as follows:

number

[0064] According to some embodiments, the raw chart coding mode directly encodes interior vertices without sampling them. In one or more examples, the interior vertices of a raw chart may be treated as boundary vertices and concatenated with the boundary vertices, where the same coding method as described above may apply. In one or more examples, the interior vertices of a raw chart may be separated from the boundary vertices. For example, for each raw chart, the boundary vertices may be coded first without the interior vertices, followed by the interior vertices. The interior vertices may be subjected to the same prediction, quantization, and entropy coding schemes as the boundary vertices. In one or more examples, only UV and XYZ coordinates need to be coded; no UV2XYZ index array is required for interior vertices. Both the number of boundary vertices and the number of interior vertices may be signaled for each raw chart.

[0065] FIG. 11 shows an exemplary geometry decoding pipeline 1100 according to one or more embodiments. The decoding pipeline 1100 may be performed by the decoder 351. Boundary vertices may be decoded (1102) and reconstructed (1106) from the boundary information substream. A geometry image may be decoded (1104) from the video substream. For each chart, the reconstruction of interior vertices differs in terms of the coding mode (e.g., raw chart mode, sampling mode). If raw chart mode is used, interior vertices are decoded (1108) directly from the boundary information substream. If a sampling mode is used in which the interior vertices are sampled vertices in the polygon defined by the boundary vertices, the sampled vertices are reconstructed (1110). After reconstruction, triangulation (1112) may be applied to each chart to derive connectivity information between the reconstructed boundary vertices and the interior vertices, resulting in a reconstructed mesh (1114). In one or more examples, a mesh filtering process (1116) may be applied to further smooth the geometry, such as on boundaries, to improve reconstruction quality.

[0066] In one or more examples, to find the correct correspondence, the decoded UV2XYZ index array may be used to reconstruct the bounding XYZ vertices. For example, for each chart, the decoded bounding UV coordinates may be derived by the following formula:

number

[0067] JPEG0007771393000011.jpg13170

[0068] In one or more examples, i=0,1,...,N j -1, where N j is the number of boundary vertices of the jth chart. The decoded boundary XYZ coordinates in the chart can be derived by:

number

[0069] JPEG0007771393000013.jpg16170

[0070] According to some embodiments, for charts coded by a sampling-based method, interior vertices can be reconstructed using decoded boundary vertices and a geometry image. For example, the reconstructed boundary UV coordinates may first define the polygon of the chart, and then all sampling points inside the bounding box of the chart are checked to see if they are inside the polygon. If they are inside the polygon, the points are identified as interior sampling points. Figure 12 shows an example of a boundary polygon (e.g., a connected point) where the interior sampling points are inside the boundary polygon. In one or more examples, a "cross-counting" algorithm can be used to check whether a point is inside a polygon. Other suitable algorithms known to those skilled in the art may also be used to check whether a point is inside a polygon. For example, a polygon defined by the reconstructed boundary UV coordinates is shown as follows:

number

[0071] The internally sampled UV coordinates may be expressed as follows:

number

[0072] where i=0,1,…,K j -1, and K j is the number of interior vertices in the jth chart.

[0073] JPEG0007771393000016.jpg24170

number

[0074] JPEG0007771393000018.jpg21170

[0075] JPEG0007771393000019.jpg44170

[0076] JPEG0007771393000020.jpg69170

[0077] Figure 13 shows an example process 1300 performed by an encoder, such as encoder 301, for compressing a 3D mesh and generating a coded video bitstream. The process may begin with operation S1302, as shown in Figure 5, and further in accordance with operation 802 (Figure 8), converting the 3D mesh into one or more 2D meshes. The process proceeds to operation S1304, where boundary extraction may be performed to identify boundary vertices in the one or more 2D meshes. For example, the boundary extraction may be performed in accordance with operation 804 (Figure 8).

[0078] The process proceeds to operation S1306, which performs chart classification, e.g., according to operation 806. The process proceeds to operation S1308, which performs interior vertex coding. For example, the interior vertices of one or more 2D meshes may be encoded according to a raw chart mode (operation 808) or a sampling mode (operations 810, 812, 818). The process proceeds to operation S1310, which performs boundary vertex coding, e.g., according to operation 816. The process proceeds to operation S1312, which generates a coded video bitstream including the encoded mesh information and any other signaling information, such as the coding mode of the interior vertices.

[0079] 14 shows an example process 1400 performed by a decoder, such as decoder 351, to decode a video bitstream and reconstruct a 3D mesh. The process may begin with operation S1402, which receives a coded video bitstream. The coded video bitstream may correspond to the bitstream generated in operation S1312 (FIG. 13). The coded video bitstream may include one or more 2D meshes that correspond to the 3D mesh.

[0080] The process proceeds to operation S1404, where one or more boundary vertices in each 2D mesh are reconstructed. The reconstruction of the boundary vertices may be performed in accordance with operation 1106 (FIG. 11). The process proceeds to operation S1406, where a reconstruction mode for reconstructing interior vertices of the one or more 2D meshes is determined. For example, a flag included in the coded video bitstream may determine whether to use a raw chart mode or a sampled mode for decoding the interior vertices.

[0081] The process proceeds to operation S1408, which reconstructs interior vertices of the one or more 2D meshes based on the determined reconstruction mode. The process proceeds to operation S1410, which reconstructs a 3D mesh based on the reconstructed boundary vertices of the one or more 2D meshes and the interior vertices of the one or more 2D meshes. The reconstruction performed in operation S1410 may include triangulation operation 1112, mesh reconstruction operation 1114, and mesh filtering operation 1116 of FIG. 11 .

[0082] The techniques described above may be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 15 illustrates a computer system 1500 suitable for implementing certain embodiments of the present disclosure.

[0083] Computer software may be coded using any suitable machine code or computer language that can be assembled, compiled, linked, or similar mechanisms to create code containing instructions that can be executed by a computer central processing unit (CPU), graphics processing unit (GPU), etc., directly, or via interpretation, microcode execution, etc.

[0084] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0085] 15 for computer system 1500 are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of the computer software implementing embodiments of the present disclosure. The arrangement of components should not be interpreted as having a dependency or requirement regarding any one or combination of components illustrated in the non-limiting embodiment of computer system 1500.

[0086] Computer system 1500 may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, via tactile input (e.g., keystrokes, swipes, data glove movements, etc.), audio input (e.g., voice, clapping, etc.), visual input (e.g., gestures, etc.), or olfactory input (not shown). Human interface devices may also be used to capture certain media not necessarily directly associated with conscious human input, such as audio (e.g., voice, music, ambient sounds, etc.), images (e.g., scanned images, photographic images obtained from a still camera, etc.), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic vision, etc.).

[0087] The input human interface devices may include one or more of a keyboard 1501, a mouse 1502, a trackpad 1503, a touchscreen 1510, a data glove, a joystick 1505, a microphone 1506, a scanner 1507, and a camera 1508 (only one of each is shown).

[0088] The computer system 1500 may also include certain human interface output devices. Such human interface output devices may stimulate one or more of a human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen 1510, data gloves, or joystick 1505, although haptic feedback devices that do not function as input devices may also be present). For example, such devices may include audio output devices (e.g., speakers 1509, headphones (not shown), etc.), visual output devices (e.g., screens 1510, including CRT screens, LCD screens, plasma screens, and OLED screens, each with or without touchscreen input capabilities, each with or without haptic feedback capabilities, some of which may output two-dimensional visual output or output in more than three dimensions through means such as stereographic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown)), and printers (not shown).

[0089] The computer system 1500 may also include human-accessible storage devices and their associated media, such as optical media including CD / DVD ROM / RW 1520 with media 1521 such as CD / DVD, thumb drive 1522, removable hard drive or solid state drive 1523, legacy magnetic media (not shown) such as tape or floppy disk, specialized ROM / ASIC / PLD-based devices (not shown) such as security dongles, etc.

[0090] Those skilled in the art will also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transitory signals.

[0091] The computer system 1500 may also include interfaces to one or more communication networks. The networks may be, for example, wireless, wired, optical, and may be local, wide area, metropolitan, vehicular and industrial, real-time, delay-tolerant, etc. Examples of networks include local area networks such as Ethernet, cellular networks including WLAN, GSM, 3G, 4G, 5G, LTE, etc., TV wired or wireless wide area digital networks including cable TV, satellite TV, terrestrial broadcast TV, and vehicular and industrial networks including CANBus, etc. Particular networks generally require an external network interface adapter connected to a particular general-purpose data port or peripheral bus 1549 (e.g., a USB port on computer system 1500); others are commonly integrated into the core of computer system 1500 by connection to a system bus as described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using any of these networks, computer system 1500 can communicate with other entities. Such communications can be unidirectional, receive only (e.g., broadcast TV), unidirectional transmit only (e.g., a CANBus to a particular CANBus device), or bidirectional, e.g., to other computer systems using local or wide-area digital networks. Such communications can include communications to a cloud computing environment 1555. As discussed above, particular protocols and protocol stacks can be used with each of these networks and network interfaces.

[0092] The aforementioned human interface devices, human-accessible storage devices, and network interface 1554 can be connected to core 1540 of computer system 1500 .

[0093] The core 1540 may include one or more central processing units (CPUs) 1541, graphics processing units (GPUs) 1542, specialized programmable processing units in the form of field programmable gate arrays (FPGAs) 1543, hardware accelerators for specific tasks 1544, etc. These devices may be connected via a system bus 1548, along with read-only memory (ROM) 1545, random access memory 1546, and internal mass storage 1547, e.g., an internal hard disk or SSD that is not user accessible. In some computer systems, the system bus 1548 is accessible in the form of one or more physical plugs, allowing expansion with additional CPUs, GPUs, etc. Peripheral devices may be connected directly to the core's system bus 1548 or via a peripheral bus 1549. Peripheral bus architectures include PCI, USB, etc. A graphics adapter 1550 may also be included in the core 1540.

[0094] The CPU 1541, GPU 1542, FPGA 1543, and accelerator 1544 may combine to execute specific instructions that may constitute the aforementioned computer code. That computer code may be stored in ROM 1545 or RAM 1546. Transient data may also be stored in RAM 1546, while permanent data may be stored in, for example, internal mass storage 1547. The use of cache memory, which may be closely associated with one or more of the CPU 1541, GPU 1542, mass storage 1547, ROM 1545, RAM 1546, etc., may enable fast storage and retrieval from any memory device.

[0095] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the art of computer software.

[0096] By way of example, and not limitation, architecture 1500, and in particular a computer system having core 1540, can provide functionality as a result of processors (including CPUs, GPUs, FPGAs, accelerators, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be user-accessible mass storage, as described above, as well as media associated with specific storage of core 1540 that is non-transitory in nature, such as core internal mass storage 1547 or ROM 1545. Software implementing various embodiments of the present disclosure can be stored on such devices and executed by core 1540. Computer-readable media can include one or more memory devices or chips, depending on particular needs. The software enables core 1540, and in particular the processors therein (including CPUs, GPUs, FPGAs, etc.), to perform particular processes, or particular portions of particular processes, described herein, including defining data structures stored in RAM 1546 and modifying such data structures through software-defined operations. Additionally, or alternatively, a computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerator 1544) that may work in place of, or in conjunction with, software to perform particular processes, or portions of particular processes, described herein. References to software may include logic, and vice versa, where appropriate. Where appropriate, references to computer-readable media may encompass circuitry (e.g., integrated circuits (ICs)) that store software for execution, circuitry that embodies logic for execution, or both. The present disclosure encompasses any suitable combination of hardware and software.

[0097] While this disclosure has described several non-limiting embodiments, there are alterations, permutations, and various substitute equivalents that fall within the scope of this disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the present disclosure and are therefore within its spirit and scope.

[0098] The above disclosure also encompasses the embodiments listed below.

[0099] (1) A method executed by at least one processor of a decoder, comprising: receiving a coded video bitstream including one or more two-dimensional (2D) meshes corresponding to a three-dimensional (3D) mesh; reconstructing one or more boundary vertices of each of the one or more 2D meshes; determining whether a reconstruction mode for reconstructing one or more interior vertices of each of the one or more 2D meshes is one of (i) a sampling mode and (ii) a non-sampling mode; reconstructing the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode; and reconstructing the 3D mesh based on the reconstructed one or more boundary vertices of each of the one or more 2D meshes and the reconstructed one or more interior vertices of each of the one or more 2D meshes.

[0100] (2) The method of feature (1), wherein the determination of the reconstruction mode is based on a flag included in the coded video bitstream.

[0101] (3) The method of any one of (1) to (2), further comprising decoding one or more geometry maps included in the coded video bitstream.

[0102] (4) The method according to feature (3), wherein the step of reconstructing the one or more interior vertices of each of the one or more 2D meshes based on determining that the reconstruction mode is the sampling mode includes the steps of: determining a boundary polygon of each of the one or more 2D meshes based on the corresponding reconstructed boundary vertices of each of the one or more 2D meshes; and identifying each vertex included in the boundary polygon of each of the one or more 2D meshes as an interior vertex.

[0103] (5) The method according to feature (4), wherein the step of reconstructing the 3D mesh includes a step of deriving interior vertices in the 3D mesh based on corresponding positions in the one or more decoded geometry maps, a sampling rate, and offsets in the one or more decoded geometry maps.

[0104] (6) The method of any one of features (1) to (5), wherein, based on determining that the reconstruction mode is the non-sampling mode, the one or more interior vertices of each of the one or more 2D meshes are reconstructed based on the reconstructed one or more boundary vertices of each of the one or more 2D meshes.

[0105] (7) The method according to feature (6), wherein the step of reconstructing the 3D mesh includes a step of reconstructing interior vertices in the 3D mesh by concatenating reconstructed interior vertices of the one or more 2D meshes with corresponding reconstructed boundary vertices in the one or more 2D meshes.

[0106] (8) The method of any one of features (1) to (7), wherein the step of reconstructing the 3D mesh includes a step of reconstructing boundary vertices in the 3D mesh based on a 2D-to-3D index array that maps one or more corresponding reconstructed boundary vertices of each of the one or more 2D meshes to the boundary vertices in the 3D mesh.

[0107] (9) A method for generating a coded video bitstream, comprising: at least one memory configured to store program code; and at least one processor configured to read the program code and to operate as directed by the program code, the program code comprising: receiving code configured to cause the at least one processor to receive a coded video bitstream including one or more two-dimensional (2D) meshes corresponding to a three-dimensional (3D) mesh; first reconstruction code configured to cause the at least one processor to reconstruct one or more boundary vertices of each of the one or more 2D meshes; and second reconstruction code configured to cause the at least one processor to reconstruct one or more boundary vertices of each of the one or more 2D meshes. a first determination code configured to cause the at least one processor to determine whether a reconstruction mode for reconstructing interior vertices of each of the one or more 2D meshes is one of (i) a sampling mode and (ii) a non-sampling mode; a second reconstruction code configured to cause the at least one processor to reconstruct the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode; and a third reconstruction code configured to cause the at least one processor to reconstruct the 3D mesh based on the reconstructed one or more boundary vertices of each of the one or more 2D meshes and the reconstructed one or more interior vertices of each of the one or more 2D meshes.

[0108] (10) The decoder of feature (9), wherein the determination of the reconstruction mode is based on a flag included in the coded video bitstream.

[0109] (11) The decoder of feature (9) or (10), wherein the program code further includes decoding code configured to cause the at least one processor to decode one or more geometry maps included in the coded video bitstream.

[0110] (12) The decoder of feature (11), wherein, based on determining that the reconstruction mode is the sampling mode, the second reconstruction code further includes: second determination code configured to cause the at least one processor to determine a boundary polygon of each of the one or more 2D meshes based on corresponding reconstructed boundary vertices of each of the one or more 2D meshes; and identification code configured to cause the at least one processor to identify each vertex included in the boundary polygon of each of the one or more 2D meshes as an interior vertex.

[0111] (13) The decoder of feature (12), wherein the third reconstruction code further includes derivation code configured to cause the at least one processor to derive interior vertices in the 3D mesh based on corresponding positions in the one or more decoded geometry maps, a sampling rate, and offsets in the one or more decoded geometry maps.

[0112] (14) The decoder of any one of features (9) to (13), wherein, based on determining that the reconstruction mode is the non-sampling mode, the one or more interior vertices of each of the one or more 2D meshes are reconstructed based on the reconstructed one or more boundary vertices of each of the one or more 2D meshes.

[0113] (15) The decoder of feature (14), wherein the third reconstruction code further includes fourth reconstruction code configured to cause the at least one processor to reconstruct interior vertices in the 3D mesh by concatenating reconstructed interior vertices of the one or more 2D meshes with corresponding reconstructed boundary vertices in the one or more 2D meshes.

[0114] (16) The decoder of any one of features (9) to (15), wherein the third reconstruction code further includes fourth reconstruction code configured to cause the at least one processor to reconstruct boundary vertices in the 3D mesh based on a 2D-to-3D index array that maps one or more corresponding reconstructed boundary vertices of each of the one or more 2D meshes to the boundary vertices in the 3D mesh.

[0115] (17) A non-transitory computer-readable medium having stored thereon instructions, when executed by at least one processor in a decoder, causing the at least one processor to perform the following steps: receive a coded video bitstream including one or more two-dimensional (2D) meshes corresponding to a three-dimensional (3D) mesh; reconstruct one or more boundary vertices of each of the one or more 2D meshes; determine whether a reconstruction mode for reconstructing one or more interior vertices of each of the one or more 2D meshes is one of (i) a sampling mode and (ii) a non-sampling mode; reconstructing the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode; and reconstructing the 3D mesh based on the reconstructed one or more boundary vertices of each of the one or more 2D meshes and the reconstructed one or more interior vertices of each of the one or more 2D meshes.

[0116] (18) The non-transitory computer-readable medium of feature (17), wherein the determination of the reconstruction mode is based on a flag included in the coded video bitstream.

[0117] (19) The non-transitory computer-readable medium of feature (17) or (18), wherein the instructions further cause the at least one processor to perform a step of decoding one or more geometry maps included in the coded video bitstream.

[0118] (20) The non-transitory computer-readable medium of feature (19), wherein the step of reconstructing the one or more interior vertices of each of the one or more 2D meshes based on determining that the reconstruction mode is the sampling mode further includes the steps of: determining a boundary polygon of each of the one or more 2D meshes based on the corresponding reconstructed boundary vertices of each of the one or more 2D meshes; and identifying each vertex included in the boundary polygon of each of the one or more 2D meshes as an interior vertex.

Claims

1. A method executed by at least one processor of a decoder, comprising: receiving a coded video bitstream including one or more two-dimensional (2D) meshes corresponding to a three-dimensional (3D) mesh; decoding one or more geometry maps included in the coded video bitstream; reconstructing one or more boundary vertices of each of said one or more 2D meshes; determining whether a reconstruction mode for reconstructing one or more interior vertices of each of the one or more 2D meshes is one of: (i) a sampling mode; and (ii) a non-sampling mode; reconstructing the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode; reconstructing the 3D mesh based on the reconstructed boundary vertices of each of the one or more 2D meshes and the reconstructed interior vertices of each of the one or more 2D meshes; Including, based on determining that the reconstruction mode is the sampling mode, Reconstructing the one or more interior vertices of each of the one or more 2D meshes comprises: determining a boundary polygon for each of the one or more 2D meshes based on the corresponding reconstructed boundary vertices for each of the one or more 2D meshes; and identifying each vertex included in the boundary polygon of each of the one or more 2D meshes as an interior vertex.

2. A method executed by at least one processor of a decoder, comprising: receiving a coded video bitstream including one or more two-dimensional (2D) meshes corresponding to a three-dimensional (3D) mesh; decoding one or more geometry maps included in the coded video bitstream; reconstructing one or more boundary vertices of each of said one or more 2D meshes; determining whether a reconstruction mode for reconstructing one or more interior vertices of each of the one or more 2D meshes is one of: (i) a sampling mode; and (ii) a non-sampling mode; reconstructing the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode; reconstructing the 3D mesh based on the reconstructed boundary vertices of each of the one or more 2D meshes and the reconstructed interior vertices of each of the one or more 2D meshes; Including, The step of reconstructing the 3D mesh comprises: deriving interior vertices in the 3D mesh based on corresponding positions in the one or more decoded geometry maps, a sampling rate, and offsets in the one or more decoded geometry maps.

3. A method executed by at least one processor of a decoder, comprising: receiving a coded video bitstream including one or more two-dimensional (2D) meshes corresponding to a three-dimensional (3D) mesh; reconstructing one or more boundary vertices of each of said one or more 2D meshes; determining whether a reconstruction mode for reconstructing one or more interior vertices of each of the one or more 2D meshes is one of: (i) a sampling mode; and (ii) a non-sampling mode; reconstructing the one or more interior vertices of each of the one or more 2D meshes based on the determined reconstruction mode; reconstructing the 3D mesh based on the reconstructed boundary vertices of each of the one or more 2D meshes and the reconstructed interior vertices of each of the one or more 2D meshes; Including, based on determining that the reconstruction mode is the non-sampling mode, The step of reconstructing the 3D mesh comprises: reconstructing interior vertices in the 3D mesh by concatenating reconstructed interior vertices of the one or more 2D meshes with corresponding reconstructed boundary vertices in the one or more 2D meshes.

4. The method of claim 1 , wherein the determination of the reconstruction mode is based on a flag included in the coded video bitstream.

5. The step of reconstructing the 3D mesh comprises:

4. The method of claim 1, further comprising: reconstructing boundary vertices in the 3D mesh based on a 2D to 3D index array that maps one or more corresponding reconstructed boundary vertices of each of the one or more 2D meshes to the boundary vertices in the 3D mesh.

6. at least one memory configured to store program code; at least one processor configured to read the program code and to act as directed by the program code; The program code A decoder configured to cause said at least one processor to perform the method of any one of claims 1 to 3.

7. A computer program arranged to cause a computer to carry out the method according to any one of claims 1 to 3.

Images