V-dmc and v-dmc normal line encoding integration in base mesh encoder

Abstract

An apparatus for processing mesh data receives a syntax element for a mesh that indicates whether to use a multi-parallelogram prediction or a cross product prediction to predict vertices of the mesh, selects one of the multi-parallelogram prediction or the cross product prediction as a selected prediction process for the mesh based on the syntax element, predicts a normal for a first vertex using the selected prediction process in response to determining that a first set of decoded normals is available for the first vertex, predicts a normal for a second vertex using a differential prediction process in response to determining that a second set of decoded normals is not available for the second vertex, and outputs a decoded version of the mesh based on the normal for the first vertex and the normal for the second vertex.

Description

[0001] This application claims priority to U.S. Patent Application No. 19 / 018,955, filed January 13, 2025; U.S. Provisional Patent Application No. 63 / 575,039, filed April 5, 2024; U.S. Provisional Patent Application No. 63 / 623,706, filed January 22, 2024; and U.S. Provisional Patent Application No. 63 / 622,028, filed January 17, 2024, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 19 / 018,955 claims the benefit of U.S. Provisional Patent Application Nos. 63 / 575,039, 63 / 623,706, and 63 / 622,028. Technical Field

[0002] This disclosure relates to video-based dynamic mesh decoding. Background Technology

[0003] Meshes can be used to represent the physical content of three-dimensional space. Meshes are useful in a variety of situations. For example, they can be used in the context of representing the physical content of an environment to locate virtual objects in extended reality applications such as augmented reality (AR), virtual reality (VR), or mixed reality (MR). Mesh compression is the process of encoding and decoding meshes. Encoding meshes reduces the amount of data required to store and transmit them. Summary of the Invention

[0004] This disclosure describes techniques related to integrating normal encoding into video-based dynamic mesh decoding (V-DMC). According to the techniques of this disclosure, the V-DMC decoder receives a syntax element of the mesh, which indicates either multi-parallelogram prediction or cross-product prediction. The V-DMC decoder then attempts to decode the normals of the vertices in the mesh using a signaled prediction process. However, for some vertices in the mesh, the other vertices required for the prediction process may not exist or have not yet been decoded. For such vertices, the V-DMC decoder can use differential prediction to predict the normals. If none of the other vertices required for differential prediction are available, the V-DMC decoder can receive the complete normal values ​​in the bitstream instead of the differences.

[0005] According to one example of this disclosure, an apparatus for processing mesh data includes: a memory; and processing circuitry coupled to the memory and configured to: receive a syntax element of the mesh, wherein a first value of the syntax element indicates that a vertex of the mesh should be predicted using multiparallelogram prediction and a second value of the syntax element indicates that a vertex of the mesh should be predicted using cross product prediction; a prediction process that selects either the multiparallelogram prediction or the cross product prediction as the mesh selection based on the syntax element; predicting the normal of the first vertex using the selected prediction process in response to determining that a first vertex has a first set of available decoded normals; predicting the normal of the second vertex using a differential prediction process in response to determining that a second vertex does not have a second set of available decoded normals; and outputting a decoded version of the mesh based on the normal of the first vertex and the normal of the second vertex.

[0006] According to another example of this disclosure, a method of processing mesh data includes: receiving a syntax element of the mesh, wherein a first value of the syntax element indicates that a vertex of the mesh is to be predicted using a multiparallelogram prediction and a second value of the syntax element indicates that a vertex of the mesh is to be predicted using a cross product prediction; a prediction process that selects either the multiparallelogram prediction or the cross product prediction as the mesh based on the syntax element; predicting a normal of the first vertex using the selected prediction process in response to determining that a first vertex has a first set of available decoded normals; predicting a normal of the second vertex using a differential prediction process in response to determining that a second vertex does not have a second set of available decoded normals; and outputting a decoded version of the mesh based on the normal of the first vertex and the normal of the second vertex.

[0007] According to another example of this disclosure, a computer-readable storage medium stores instructions that, when executed, cause one or more processors to: receive a syntax element of a mesh, wherein a first value of the syntax element indicates that a vertex of the mesh should be predicted using a multiparallelogram prediction and a second value of the syntax element indicates that a vertex of the mesh should be predicted using a cross product prediction; a prediction process that selects either the multiparallelogram prediction or the cross product prediction as the mesh selection based on the syntax element; predicting the normal of the first vertex using the selected prediction process in response to determining that a first vertex has a first set of available decoded normals; predicting the normal of the second vertex using a differential prediction process in response to determining that a second vertex does not have a second set of available decoded normals; and outputting a decoded version of the mesh based on the normal of the first vertex and the normal of the second vertex.

[0008] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating an example encoding and decoding system that can perform the techniques of this disclosure.

[0010] Figure 2 An example implementation of the V-DMC encoder is shown.

[0011] Figure 3 An example implementation of the V-DMC decoder is shown.

[0012] Figure 4 An example implementation of the intra-mode encoder for V-DMC is shown.

[0013] Figure 5 An example implementation of the intra-mode decoder for V-DMC is shown.

[0014] Figure 6 An example implementation of the V-DMC decoder is shown.

[0015] Figure 7 An example implementation of the decoding process for decoding the underlying grid connectivity is shown.

[0016] Figure 8A An example implementation of a basic mesh encoder is shown.

[0017] Figure 8B An example implementation of the basic mesh decoder is shown.

[0018] Figure 9 An example implementation of a V-DMC decoder performing normal decoding in lossless mode is shown.

[0019] Figure 10 An example implementation of a V-DMC decoder performing normal decoding in lossy intra-frame mode is shown.

[0020] Figure 11 An example implementation of a V-DMC decoder performing normal decoding in lossy inter-frame mode is shown.

[0021] Figure 12 An example implementation of a V-DMC decoder performing normal graph decoding in lossless mode is shown.

[0022] Figure 13 An example implementation of a V-DMC decoder performing normal graph decoding in lossy intra-frame mode is shown.

[0023] Figure 14 An example implementation of a V-DMC decoder performing normal graph decoding in lossy inter-frame mode is shown.

[0024] Figure 15 An example angle table representation is shown.

[0025] Figures 16A to 16C An example of normal prediction is shown.

[0026] Figure 17 This is a flowchart illustrating an example process for encoding a mesh.

[0027] Figure 18 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data.

[0028] Figure 19 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data. Detailed Implementation

[0029] A mesh is typically a collection of vertices in three-dimensional (3D) space that collectively represent one or more objects in that space. Vertices are connected by edges, and these edges form polygons that make up the faces of the mesh. Each vertex may also have one or more associated properties, such as texture or color. In most scenarios, having more vertices produces a higher quality mesh (e.g., more detailed and more realistic). However, having more vertices also requires more data to represent the mesh.

[0030] To reduce the amount of data required to represent a grid, lossy or lossless encoding can be used to encode the grid. In lossless encoding, the decoded version of the encoded grid perfectly matches the original grid. In contrast, in lossy encoding, the process of encoding and decoding the grid introduces losses such as distortion in the decoded version of the encoded grid.

[0031] In one example of a lossy encoding technique used for meshes, a mesh encoder decimates the original mesh to determine the base mesh. To decimate the original mesh, the mesh encoder subsamples or otherwise reduces the number of vertices in the original mesh, making the base mesh a coarse approximation of the original mesh with fewer vertices. The mesh encoder then subdivides the decimated mesh. That is, the mesh encoder estimates the positions of additional vertices between the vertices of the base mesh. The mesh encoder then deforms the subdivided mesh by moving the vertices, making the deformed mesh more closely match the original mesh.

[0032] After determining the desired deformation of the base mesh and sub-mesh, the mesh encoder generates a bitstream comprising data for constructing the base mesh and data for performing the deformation. The deformation-defining data is signaled as a series of displacement vectors indicating the movement or displacement of additional vertices determined by the sub-meshing process. To decode the mesh from the bitstream, the mesh decoder reconstructs the base mesh based on the signaled information, applies the same sub-meshing process as the mesh encoder, and then displaces the additional vertices based on the signaled displacement vectors.

[0033] This disclosure describes techniques related to integrating normal encoding into video-based dynamic mesh decoding (V-DMC), a technique being standardized in MPEG WG7 (3DGH). This disclosure focuses particularly on a specific implementation of normal encoding in the underlying mesh encoder within the V-DMC test model v6.0 (TMMv6.0) of ISO / IEC JTC 1 / SC 29 / WG 7, N733, October 2023, which is incorporated herein by reference. In V-DMC, the EdgeBreaker implementation is used to encode underlying mesh connectivity, and residual coding with attribute prediction can be used to encode underlying mesh attributes. However, neither the V-DMC tool nor the underlying mesh encoder supports encoding normals as attributes. The techniques of this disclosure integrate normal encoding into the V-DMC tool and propose implementing normal encoding in the underlying mesh encoder. The techniques of this disclosure allow for lossless, efficient encoding of normals in the underlying mesh encoder using a smaller bitstream.

[0034] A normal is a vector (x, y, z) perpendicular to the surface of a mesh. Normals are used to determine the orientation of a surface and to apply shading. In 3D modeling, normals play a crucial role in creating realistic objects, such as helping to define the shape of an object and how it interacts with light. Normals are also used in computer graphics to create smooth surfaces and calculate light reflection. Furthermore, normals are used in video games to create realistic environments and improve game performance. In short, normals are an essential component of 3D modeling and computer graphics and are used in a wide range of applications, from video games to scientific simulations.

[0035] The V-DMC decoder can be configured to determine the vertex normal by determining the predicted normal of the vertex, the difference in the received encoded bitstream, and determining that the final normal value of the vertex equals the predicted value plus the difference. The V-DMC decoder can be configured to perform different prediction processes based on the already decoded nearby vertices.

[0036] When performing multi-parallelogram prediction, the V-DMC decoder predicts that the normal value of the current vertex (c in the attached diagram below) is equal to the previous normal value (cp) plus the next normal value (cn) minus the relative normal value (co). The V-DMC decoder can make similar predictions for multiple triangles around the current vertex and set the final prediction as the average of the predictions for multiple triangles.

[0037] When performing cross-product prediction, the V-DMC decoder predicts the normal value of the current vertex (c) by determining the vector between the previous vertex (cp) and the current vertex (c), determining another vector between the next vertex (cn) and the current vertex (c), and obtaining the cross product of these two vectors. In some examples, the V-DMC decoder may perform this prediction for all or some triangles surrounding the current vertex and determine the predicted normal value as the average.

[0038] When performing differential prediction, the V-DMC decoder predicts the normal value of the current vertex (c) based on the decoded normal values ​​of a single vertex (cp or cn). The V-DMC decoder determines the actual normal value by receiving the difference in the bitstream and adding the difference to the predicted normal value. The V-DMC decoder can also perform other types of prediction.

[0039] According to the technique disclosed herein, the V-DMC decoder can receive a syntax element of a mesh indicating either multi-parallelogram prediction or cross-product prediction. The V-DMC decoder can then attempt to decode the normals of the vertices in the mesh using a signaled prediction process. However, for some vertices in the mesh, the other vertices required for the prediction process may not exist or have not yet been decoded. For such vertices, the V-DMC decoder can use differential prediction to predict the normals. If none of the other vertices required for differential prediction are available, the V-DMC decoder can receive the complete normal values ​​in the bitstream instead of the differences.

[0040] Figure 1 This is a block diagram illustrating an example encoding and decoding system 100 that can perform the techniques of this disclosure. The techniques of this disclosure generally involve decoding (encoding and / or decoding) grids. This decoding can be efficient in compressing and / or decompressing grid data.

[0041] like Figure 1 As shown, system 100 includes source device 102 and destination device 116. Source device 102 provides encoded data for decoding by destination device 116. Specifically, in Figure 1In this example, source device 102 provides data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 can include any of a wide range of devices, including desktop computers, laptop computers, tablet computers, set-top boxes, mobile phones (such as smartphones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, land or sea vehicles, spacecraft, aircraft, robots, LiDAR devices, satellites, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication.

[0042] exist Figure 1 In the example, source device 102 includes a data source 104, a memory 106, a V-DMC encoder 200, and an output interface 108. Destination device 116 includes an input interface 122, a V-DMC decoder 300, a memory 120, and a data consumer 118. According to this disclosure, the V-DMC encoder 200 of source device 102 and the V-DMC decoder 300 of destination device 116 can be configured to apply the techniques related to shift vector quantization disclosed herein. Therefore, source device 102 represents an example of an encoding device, while destination device 116 represents an example of a decoding device. In other examples, source device 102 and destination device 116 may include other components or arrangements. For example, source device 102 may receive data from an internal or external source. Similarly, destination device 116 may interface with an external data consumer, rather than including the data consumer in the same device.

[0043] like Figure 1 The system 100 shown is merely an example. In general, other digital encoding and / or decoding devices can perform the techniques related to shift vector quantization disclosed herein. Source device 102 and destination device 116 are merely examples of such devices, where source device 102 generates decoded data for transmission to destination device 116. This disclosure refers to a “decoding” device as a device that performs the decoding (e.g., encoding and / or decoding) of data. Thus, V-DMC encoder 200 and V-DMC decoder 300 represent examples of decoding devices, specifically, encoder and decoder, respectively. In some examples, source device 102 and destination device 116 can operate in a substantially symmetrical manner, such that each of source device 102 and destination device 116 includes both encoding and decoding components. Therefore, system 100 can support one-way or two-way transmission between source device 102 and destination device 116, for example, for streaming, playback, broadcasting, telephone, navigation, and other applications.

[0044] Generally, data source 104 represents the source of data (i.e., raw, unencoded data) and provides a series of sequential “frames” of data to V-DMC encoder 200, which encodes the data in the frames. Data source 104 of source device 102 may include a grid capture device such as any of a variety of cameras or sensors (e.g., a 3D scanner or light detection and ranging (LIDAR) device, one or more cameras), an archive containing previously captured data, and / or a data feed interface receiving data from a data content provider. Alternatively or additionally, the grid data may be computer-generated from a scanner, camera, sensor, or other data source. For example, data source 104 may generate computer-graphics-based data as source data, or a combination of real-time data, archived data, and computer-generated data. In each case, V-DMC encoder 200 encodes the captured data, pre-captured data, or computer-generated data. V-DMC encoder 200 may rearrange the frames from the received order (sometimes referred to as the “display order”) to a decoding order for decoding. The V-DMC encoder 200 can generate one or more bit streams including encoded data. The source device 102 can then output the encoded data to a computer-readable medium 110 via the output interface 108 for reception and / or retrieval by, for example, the input interface 122 of the destination device 116.

[0045] The memory 106 of source device 102 and the memory 120 of destination device 116 may represent general-purpose memory. In some examples, memory 106 and memory 120 may store raw data, such as raw data from data source 104 and raw decoded data from V-DMC decoder 300. Additionally or alternatively, memory 106 and memory 120 may store software instructions, respectively executable by, for example, V-DMC encoder 200 and V-DMC decoder 300. Although memory 106 and memory 120 are shown separately from V-DMC encoder 200 and V-DMC decoder 300 in this example, it should be understood that V-DMC encoder 200 and V-DMC decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memory 106 and memory 120 may store encoded data, such as output from V-DMC encoder 200 and input to V-DMC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, for example, to store raw decoded and / or encoded data. For instance, memory 106 and memory 120 may store data representing a grid.

[0046] Computer-readable medium 110 may represent any type of medium or device capable of transmitting encoded data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium enabling source device 102 to transmit encoded data directly to destination device 116 in real time, for example, via a radio frequency network or a computer-based network. Output interface 108 may modulate the transmitted signal including the encoded data, and input interface 122 may demodulate the received transmitted signal according to a communication standard, such as a wireless communication protocol. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network such as a local area network, a wide area network, or a global network (such as the Internet). The communication medium may include a router, a switch, a base station, or any other equipment that may be useful for facilitating communication from source device 102 to destination device 116.

[0047] In some examples, source device 102 can output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0048] In some examples, source device 102 may output encoded data to file server 114 or another intermediate storage device that may store the encoded data generated by source device 102. Destination device 116 may access the stored data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing encoded data and sending it to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded data from file server 114 via any standard data connection, including an internet connection. This standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a downloading protocol, or a combination thereof.

[0049] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (such as encoded data) according to cellular communication standards (such as 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to comply with specifications such as IEEE 802.11, IEEE 802.15 (e.g., ZigBee). ™ ),Bluetooth ™ Data (such as encoded data) is transmitted using other wireless standards, such as standard V-DMC encoder 200. In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include an SoC device for performing functions belonging to V-DMC encoder 200 and / or output interface 108, and destination device 116 may include an SoC device for performing functions belonging to V-DMC decoder 300 and / or input interface 122.

[0050] The technology disclosed herein can be applied to support encoding and decoding of any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices such as local or remote servers, geographic mapping or other applications.

[0051] The input interface 122 of the destination device 116 receives an encoded bitstream from a computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, etc.). The encoded bitstream may include signaling information defined by the V-DMC encoder 200 and also used by the V-DMC decoder 300, such as syntax elements having characteristics and / or processed values ​​describing the units being decoded (e.g., slices, pictures, picture groups, sequences, etc.). The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display that presents an image based on a grid.

[0052] The V-DMC encoder 200 and V-DMC decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the V-DMC encoder 200 and V-DMC decoder 300 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (codec) in the respective device. Devices including the V-DMC encoder 200 and / or V-DMC decoder 300 may include one or more integrated circuits, microprocessors, and / or other types of devices.

[0053] The V-DMC encoder 200 and V-DMC decoder 300 operate according to a decoding standard. This disclosure generally relates to the decoding (e.g., encoding and decoding) of images, thus including the process of encoding or decoding data. The encoded bitstream typically includes a series of values ​​for syntax elements representing decoding decisions (e.g., decoding modes).

[0054] This disclosure may generally relate to "signaling" certain information (such as syntax elements). The term "signaling notification" generally refers to communication of the value of a syntax element and / or other data used to decode encoded data. That is, the V-DMC encoder 200 may signal the value of a syntax element in the bit stream. Generally speaking, signaling notification means generating a value in the bit stream. As noted above, source device 102 may transmit the bit stream to destination device 116 substantially in real time or not in real time (such as when syntax elements are stored in storage device 112 for later retrieval by destination device 116).

[0055] Working Group 7 (WG7) (commonly known as the 3D Graphics and Haptic Decoding Group (3DGH)) is currently working on standardizing video-based dynamic mesh decoding (V-DMC) for XR applications. Current test models involve preprocessing the input mesh into a potentially simplified version called a “base mesh.” These base meshes may contain fewer vertices than the original mesh and can be encoded using a base mesh decoder, also known as a static mesh decoder. Preprocessing also generates displacement vectors and texture attribute maps, both of which are encoded using a V-DMC encoder. If the mesh is encoded in a lossless manner, the base mesh is no longer a simplified version and is used to encode the original mesh. For the lossless approach, the V-DMC TMM v6.0 tool operates in intra-frame mode, where the base mesh encoder becomes the primary encoding procedure.

[0056] The base mesh encoder encodes the mesh's connectivity and the properties associated with each vertex, which typically relate to, but are not limited to, the texture's localization and coordinates (e.g., UV coordinates). This localization includes the vertex's 3D coordinates (x, y, z), while the texture is stored as 2D UV coordinates (x, y) pointing to the pixel location in the texture map image. The base mesh in V-DMC is encoded using the EdgeBreaker algorithm, while connectivity is encoded using the CLERS opcode. Residuals of the properties are encoded using predictions from previously encoded / decoded vertices.

[0057] The EdgeBreaker algorithm is described in Jean-Eudes Marvie and Olivier Mocquard’s [V-DMC][EE4.4] An efficient EdgeBreaker implementation, ISO / IEC JTC1 / SC29 / WG7, m63344, April 2023 (hereinafter referred to as “m63344”), which is incorporated herein by reference. The CLERS opcode is described in J. Rossignac, “3D compression made simple: EdgeBreaker with ZipandWrapon a corner-table,” published in the International Conference on Shape Modeling and Applications, Genova, Italy, 2001 (hereinafter referred to as “Rossignac”) and in the ACM Symposium on Solid Modeling and Applications, Saarbrucken, 2002, by H. Lopes, G. Tavares, J. Rossignac, A. Szymczak, and A. Safonova, “EdgeBreaker: a simple compression for surfaces with handles.” (hereinafter referred to as “Lopes”), both of which are incorporated herein by reference.

[0058] The technology disclosed herein integrates normal encoding into the V-DMC tool. This technology integrates per-vertex normal encoding into the basic mesh encoder (also known as a static mesh encoder), and adds support for normal maps.

[0059] Background on the V-DMC test model will now be provided. A detailed description of the proposal chosen as the starting point for V-DMC standardization can be found in the following documents, all of which are incorporated herein by reference.

[0060] ● U.S. Provisional Patent Application No. 63 / 614,139, filed on December 22, 2023.

[0061] ● U.S. Provisional Patent Application No. 63 / 621,478, filed on January 16, 2024.

[0062] ● [V-CG] Apple's Dynamic Mesh Coding CfP Response, by Khaled Mammou, Jungsun Kim, Alexandros Tourapis, Dimitri Podborski, and Krasimir Kolarov, ISO / IEC JTC1 / SC29 / WG7, m59281, Apr 2022 (hereinafter referred to as "m59281").

[0063] ● V-DMC codec description, ISO / IEC JTC1 / SC29 / WG7, N00644, July 2023 (hereinafter referred to as "V-DMC codec description").

[0064] ● WD 4.0 of V-DMC, ISO / IEC JTC1 / SC29 / WG7, N00680, Jan. 2023 (hereinafter referred to as "N00680").

[0065] U.S. Provisional Patent Applications Nos. 63 / 614,139 and 63 / 621,478 explain V-DMC and basic lattice decoding. Figure 2 and Figure 3 The overall system model of the current V-DMC test model (TM) including the encoder and decoder architecture is shown. Figure 6 A detailed view of the V-DMC decoder is shown.

[0066] A mesh typically refers to a 3D data storage format where 3D data is represented by triangles. The data includes triangle connectivity and corresponding attributes. Mesh attributes typically refer to properties that can include many items such as per-vertex geometry (x, y, z), texture, normals, and per-vertex color.

[0067] Texture vs. Color: Texture and color attributes are different. Color attributes include the color of each vertex, while texture is stored as a texture map (image) and texture coordinates (UV coordinates). Each individual vertex is assigned UV coordinates corresponding to its (x,y) position on the texture map.

[0068] Texture encoding involves encoding both the per-vertex texture coordinates (UV coordinates) and the corresponding texture map. The UV coordinates are encoded in the underlying mesh encoder, while the texture map is encoded using a video encoder.

[0069] Preprocessing: The input mesh sequence is first preprocessed to generate atlases, base meshes, displacement vectors, and property maps.

[0070] Atlas encoding: Atlas parameterization involves packaging a 3D mesh into a 2D atlas, i.e., texture mapping. The atlas encoder encodes the information needed to parameterize the 3D mesh into a 2D texture map.

[0071] Base mesh: For lossy coding, the base mesh is typically a simplified mesh with a potentially smaller number of vertices. For lossless coding, the base mesh is a slightly simplified original mesh.

[0072] Basic mesh encoder: The basic mesh is a basic mesh encoder that uses a basic mesh encoder ( Figure 4 The basic mesh encoder uses a static mesh encoder to encode the mesh connectivity and properties (geometric, texture coordinates (UV coordinates), etc.) in a lossless manner.

[0073] Displacement encoder: Displacement is an indication of how the base mesh is transformed / shifted to create a per-vertex vector of the mesh. The displacement vector can be encoded as a V3C video component or decoded using arithmetic displacement.

[0074] Texture map encoder: Employs a video encoder to encode texture maps.

[0075] Lossless Mode: In lossless mode, there are no displacement vectors and the underlying mesh is not simplified. The underlying mesh encoder is a lossless encoder, therefore sufficient for V-DMC's lossless mode. The texture map is encoded using a lossless video encoder.

[0076] Lossy Mode: In lossy mode, the base mesh can be a simplified version of the original mesh. A displacement vector is used to subdivide the base mesh and shift it to obtain the reconstructed mesh. The texture map is encoded using a lossy video encoder.

[0077] Normals: V-DMC TMM v6.0 currently does not support normals. Similar to textures and colors, normals can be per-vertex normals, or they can be normal maps with corresponding normal coordinates.

[0078] Figure 2 and Figure 3 The encoder for the current V-DMC test model (TM) is shown. Figure 2 V-DM encoder 200 and decoder (in the middle) Figure 3 The overall system model of the V-DMC decoder 300 architecture is shown below. The V-DMC encoder 200 performs volumetric media conversion, and the V-DMC decoder 300 performs the corresponding reconstruction. The 3D media is converted into a series of sub-bitstreams: base mesh, displacement, and texture attributes. As described in N00680, the bitstreams also include additional atlas information to enable inverse reconstruction.

[0079] Figure 2An example implementation of the V-DMC encoder 200 is shown. Figure 2 In the example, the V-DMC encoder 200 includes a preprocessing unit 204, an atlas encoder 208, a base mesh encoder 212, a displacement encoder 216, and a video encoder 220. The preprocessing unit 204 receives an input mesh sequence and generates a base mesh, displacement vectors, and a texture attribute map. The base mesh encoder 212 encodes the base mesh. The displacement encoder 216 encodes the displacement vectors, for example, as V3C video components, or using arithmetic displacement decoding. The video encoder 220 encodes the texture attribute components (e.g., texture or material information) using any video codec, such as the High Efficiency Video Decoding (HEVC) standard or the Multi-Functional Video Decoding (VVC) standard.

[0080] The V-DMC encoder 200 will now be described in more detail. The preprocessing unit 204 represents the 3D volumetric data as a set of base meshes and corresponding refinement components. This is achieved by converting the input dynamic mesh representation into several V3C components: the base mesh, a set of displacements, a 2D representation of the texture map, and an atlas. The base mesh components are a simplified, low-resolution approximation of the original mesh in lossy compression, and the original mesh itself in lossless compression. The base mesh components can be encoded by the base mesh encoder 212 using any mesh codec.

[0081] Basic mesh encoder 212 in Figure 4 The representation is a static mesh encoder, and the underlying mesh is encoded using a specific implementation of the EdgeBreaker algorithm (e.g., m63344), where connectivity is encoded using CLERS opcodes (e.g., CLERS opcodes proposed by Rossignac and Lopes), and attribute residuals are encoded using predictions of the attributes of previously encoded / decoded vertices.

[0082] The various aspects of the base mesh encoder 212 will now be described in more detail. One or more sub-meshes are input to the base mesh encoder 212. The sub-meshes are generated by the preprocessing unit 204. The sub-meshes are generated from the original mesh through patented semantic segmentation. Each base mesh may include one or more sub-meshes.

[0083] The base mesh encoder 212 processes connected components. Connected components consist of clusters of triangles connected by adjacent triangles. A submesh may have one or more connected components. The base mesh encoder 212 can encode the connectivity and properties of one "connected component" at a time, and then perform entropy encoding on all "connected components".

[0084] The base mesh encoder 212 defines and categorizes the input base mesh into connectivity and attributes. Geometry and texture coordinates (UV coordinates) are categorized as attributes.

[0085] Figure 3 An example implementation of the V-DMC decoder 300 is shown. Figure 3 In the example, the V-DMC decoder 300 includes a demultiplexer 304, an atlas decoder 308, a basic mesh decoder 314, a displacement decoder 316, a video decoder 320, a basic mesh processing unit 324, a displacement processing unit 328, a mesh generation unit 332, and a reconstruction unit 336.

[0086] Demultiplexer 304 separates the encoded bitstream into atlas sub-bitstream, base mesh sub-bitstream, displacement sub-bitstream, and texture attribute sub-bitstream. Atlas decoder 308 decodes the atlas sub-bitstream to determine atlas information, thereby achieving inverse reconstruction. Base mesh decoder 314 decodes the base mesh sub-bitstream, and base mesh processing unit 324 reconstructs the base mesh. Displacement decoder 316 decodes the displacement sub-bitstream, and displacement processing unit 328 reconstructs the displacement vector. Mesh generation unit 332 modifies the base mesh based on the displacement vector to form a displacement mesh.

[0087] The video decoder 320 decodes the texture attribute sub-stream to determine the texture attribute map, and the reconstruction unit 336 associates the texture attributes with the displacement mesh to form a reconstructed dynamic mesh.

[0088] Figure 4 The intra-frame mode V-DMC encoder 400 is shown, and Figure 5 The intra-frame mode V-DMC decoder 500 is shown. The V-DMC encoder 400 generally represents a more detailed example implementation of the V-DMC encoder 200, particularly regarding intra-frame mode functionality, and the V-DMC decoder 500 represents a more detailed example implementation of the V-DMC decoder 300, particularly regarding intra-frame mode functionality. Figure 6 The V-DMC decoder 600 is shown, and a more detailed example implementation of the V-DMC decoder 300 is shown, particularly regarding the functionality of intra-frame and inter-frame modes.

[0089] Figure 4 Including the following abbreviations: m(i) - Basic Mesh d(i) - displacement m''(i) - Reconstruct the base mesh d''(i) - Reconstructed displacement A(i)-attribute graph A'(i) - Updated property graph M(i) - Static / Dynamic Mesh DM(i) - Reconstructing Deformed Mesh m'(i) - Reconstructing the quantized base grid d'(i) - Updated displacement e(i) - wavelet coefficients e'(i) - Quantization wavelet coefficients pe'(i) - Packed quantized wavelet coefficients rpe'(i) - Reconstructing Packed Quantized Wavelet Coefficients AB-Compressed Attribute Bitstream DB - Compressed Displacement Stream BMB - Compressed Basic Mesh Bit Stream The V-DMC encoder 400 receives, for example, the base mesh m(i) and displacement d(i) from the preprocessing system. The V-DMC encoder 400 also retrieves the mesh M(i) and the property map A(i).

[0090] The quantization unit 402 quantizes the base mesh, and the static mesh encoder 404 encodes the quantized base mesh to generate a compressed base mesh bitstream.

[0091] Displacement update unit 408 uses reconstructed quantized base mesh To update the displacement field To generate an updated displacement field The process considers reconstructing the base mesh. With the original base mesh The differences between them. By utilizing the subdivided surface mesh structure, the wavelet transform unit 410 applies the wavelet transform to... A set of wavelet coefficients is generated. The scheme is unaware of the applied transform and can utilize any other transform, including the identity transform. The quantization unit 412 quantizes the wavelet coefficients, and the image packing unit 414 packs the quantized wavelet coefficients into a 2D image / video, which can be compressed using a conventional image / video encoder in the same manner as V-PCC to generate a shift bitstream.

[0092] The attribute transfer unit 430 converts the original attribute map A(i) into an updated attribute map corresponding to the reconstructed deformable mesh DM(i). The padding unit 432 pads the updated attribute map by, for example, using interpolated samples to pad frames with empty samples, which improves decoding efficiency and reduces artifacts. The color space conversion unit 434 converts the attribute map into a different color space, and the video encoding unit 436 encodes the updated attribute map in the new color space using, for example, a video codec to generate an attribute bitstream.

[0093] Multiplexer 438 combines compressed attribute bitstream, compressed displacement bitstream, and compressed base mesh bitstream into a single compressed bitstream.

[0094] Image unpacking unit 418 and inverse quantization unit 420 apply image unpacking and inverse quantization to the reconstructed packed quantized wavelet coefficients generated by video coding unit 416 to obtain reconstructed versions of the wavelet coefficients. Inverse wavelet transform unit 422 applies inverse wavelet transform to the reconstructed wavelet coefficients to determine the reconstructed displacement. .

[0095] Inverse quantization cell 424 applies inverse quantization to the reconstructed quantization base mesh m'(i) to obtain the reconstructed base mesh. Deformed mesh reconstruction element 428 sub-division And reconstruct displacement Apply to its vertices to obtain a reconstructed deformable mesh .

[0096] Image unpacking unit 418, inverse quantization unit 420, inverse wavelet transform unit 422, and deformed mesh reconstruction unit 428 represent the displacement decoding loop. Inverse quantization unit 424 and deformed mesh reconstruction unit 428 represent the base mesh decoding loop. The V-DMC encoder 400 includes a displacement decoding loop and a base mesh decoding loop, enabling the V-DMC encoder 400 to make encoding decisions based on the same decoded mesh generated by the mesh decoder, such as determining an acceptable rate-distortion tradeoff, which may include distortion due to quantization and transform. The V-DMC encoder 400 can also use decoded versions of the base mesh, reconstructed mesh, and displacement to encode subsequent base meshes and displacements.

[0097] Control unit 450 typically represents the decision-making function of V-DMC encoder 400. During the encoding process, control unit 450 may make determinations, for example, regarding mode selection, rate allocation, quality control, and other such decisions.

[0098] Figure 5 A block diagram of an intra-decoder, which may be a part of, for example, the V-DMC decoder 300, is shown. The demultiplexer (DMUX) 502 separates the compressed bitstream (bi) into a mesh substream, a displacement substream for position and potentially for each vertex attribute, a zero or more attribute graph substream, and an atlas substream containing patch information, in the same manner as in V3C / V-PCC.

[0099] Demultiplexer 502 feeds the mesh substream to static mesh decoder 506 to generate the reconstructed quantized base mesh. The inverse quantization unit 514 inverse-quantizes the base mesh to determine the decoded base mesh. The video / image decoding unit 516 decodes the displacement substream, and the image unpacking unit 518 unpacks the image / video to determine quantization transform coefficients, such as wavelet coefficients. The inverse quantization unit 520 inverse-quantizes the quantization transform coefficients to determine the dequantized transform coefficients. The inverse transform unit 522 generates the decoded displacement field by applying an inverse transform to the unquantized coefficients. The deformable mesh reconstruction element 524 applies the reconstruction process to the decoded base mesh. And by adding a decoded displacement field This is used to generate the final decoded mesh (M''(i)). The attribute substream is directly decoded by the video / image decoding unit 526 to generate the attribute map. The color format / space conversion unit 528 can convert attribute maps into different formats or color spaces.

[0100] Figure 6 A V-DMC decoder 600 is shown, which is an example implementation of V-DMC decoder 300. V-DMC decoder 600 includes a demultiplexer (DMUX) 602 that receives a compressed bitstream b(i) and separates it into a base grid bitstream (BMB), a shift bitstream (DB), and an attribute bitstream (AB). A mode selection unit 604 determines whether the base grid data is encoded in intra-frame or inter-frame mode. If the base grid is encoded in intra-frame mode, a static grid decoder 606 decodes the grid data without relying on any previously decoded grids. If the base grid is encoded in inter-frame mode, a motion decoder 608 decodes the motion, and a base grid reconstruction unit 610 applies the motion to the decoded grid (m''(j)) stored in a grid buffer 612 to determine the reconstructed quantized base grid (m'(i)). An inverse quantization unit 614 applies inverse quantization to the reconstructed quantized base grid to determine the reconstructed base grid (m''(i)).

[0101] Video decoder 616 decodes the bitstream to determine a set of quantized transform coefficients or frames of quantized transform coefficients. Image unpacking unit 618 unpacks the quantized transform coefficients. For example, video decoder 616 may decode the quantized transform coefficients into frames, where the quantized transform coefficients are organized into blocks with a specific scan order. Image unpacking unit 618 converts the quantized transform coefficients from being organized in frames into an ordered series. In some implementations, for example, a context-based arithmetic decoder may be used to directly decode the quantized transform coefficients, and unpacking may not be necessary.

[0102] Regardless of whether the quantized transform coefficients are decoded directly or in-frame, the inverse quantization unit 620 inversely quantizes (e.g., inverse scales) the quantized transform coefficients to determine the dequantized transform coefficients. The inverse wavelet transform unit 622 applies an inverse transform to the dequantized transform coefficients to determine a set of displacement vectors. The deformable mesh reconstruction unit 624 uses the decoded displacement vectors to deform the reconstructed base mesh to determine the decoded mesh (M''(i)).

[0103] The video decoder 626 decodes the attribute bitstream to determine the decoded attribute value (A'(i)), and the color space conversion unit 628 converts the decoded attribute value into the desired color space to determine the final attribute value (A''(i)). The final attribute value corresponds to the attribute of the vertex of the decoded mesh, such as color or texture.

[0104] Figure 7 A complete overview of the EdgeBreaker mesh codec, with the top row representing the encoding lines and the bottom row representing the decoding lines, as described in m63344. Figure 7 An example of an EdgeBreaker-based end-to-end mesh codec is illustrated, which includes the following main steps: coding : - Preprocessing (702): Initially, preprocessing is performed to correct potential connectivity problems in the input mesh, such as non-manifold edges and vertices. This step is important because the EdgeBreaker algorithm employed cannot operate in cases of such connectivity problems. Solving non-manifold problems may involve duplicating some vertices, which are tracked for later merging during decoding. This optimization reduces the number of points in the decoded mesh but requires additional information in the bitstream. Dummy points are also added in this preprocessing stage to fill potential surface holes that the EdgeBreaker will not handle. The holes are then encoded by generating “virtual” dummy points by encoding the dummy triangles attached to these holes, thus encoding the holes without requiring 3D localization encoding. Vertex attributes are quantized in the preprocessing if necessary.

[0105] - Connectivity Encoding (704): Next, the connectivity of the grid is encoded using a modified EdgeBreaker algorithm to generate a CLERS table along with other memory tables for attribute prediction.

[0106] - Attribute prediction (706): Predict vertex attributes starting from geometric location attributes and extend to other attributes, some of which may depend on location prediction, such as for texture UV coordinates.

[0107] - Bitstream Configuration (708): Finally, the configuration and metadata are included in the bitstream. This includes the entropy decoding of the CLERS table and attribute residuals.

[0108] decoding : - Entropy Decoding (710): The decoding process begins with the decoding of all entropy-decoded sub-bit streams.

[0109] - Connectivity Decoding (714): Reconstructs mesh connectivity using CLERS tables and the EdgeBreaker algorithm, with additional information used to manage handles describing the topology.

[0110] - Attribute Prediction and Correction (716): Vertex localization is predicted using mesh connectivity and a minimal set of 3D coordinates. Attribute residuals are then applied to correct the predictions and obtain the final vertex localization. Other attributes are then decoded, potentially relying on previously decoded localizations, as in the case of UV coordinates. Connectivity of attributes using a separate index table is reconstructed using binary seam information entropy-encoded per edge.

[0111] - Post-processing (718): In the post-processing stage, dummy triangles are removed. Optionally, if the codec is configured for lossless decoding, the nonmanifold problem is recreated. If quantization is performed during encoding, vertex attributes are also optionally dequantized.

[0112] The data representation for basic grid decoding will now be described. The EdgeBreaker algorithm utilizes a corner table data representation, a concept originally introduced by Rossignac to enhance the efficiency of the EdgeBreaker algorithm. A comprehensive overview of the properties of the corner table (CT) can be found on Rossignac.

[0113] The aspects of attribute decoding in basic mesh decoding will now be described. The basic mesh encoder encodes both the attributes and connectivity of triangles and vertices. These attributes are typically encoded using a prediction scheme to predict vertex attributes using previously visited / encoded / decoded vertices. The predictions are then subtracted from the actual attribute values ​​to obtain residuals. Finally, the residual attribute values ​​are encoded using an entropy encoder to obtain the encoded basic mesh attribute bitstream. The attribute bitstream containing vertex attributes typically has geometric / positional attributes and UV coordinates (texture attributes), but can contain any number of attributes, such as per-vertex RGB values, etc.

[0114] Attribute decoding uses a prediction scheme to find the residuals between predicted and actual attributes. Finally, the residuals are entropy-encoded into a bitstream of underlying mesh attributes. Each vertex attribute is encoded in a different manner. Both the geometry and texture UV coordinates for 3D localization are encoded using the prediction process. To compute these predictions, a multi-parallelogram technique is used for geometry encoding (as described by Cohen and Isenburg), while a minimum stretching process is employed for UV coordinate encoding, as described in “Compressing Texture Coordinates with Selective Linear Predictions” by IM and SJ in Computer Graphics International, Tokyo, Japan, 2003, which is incorporated herein by reference.

[0115] It is worth noting that during angle prediction, it is guaranteed that the associated triangular sector is always complete and that each of its angles has a valid opposite angle. This is achieved by filling any holes with dummy points. In fact, even a single triangle will transform into a pyramid of four triangles in terms of connectivity.

[0116] The code for attribute compression is shown in Table 1. Geometric prediction uses a multi-parallelogram scheme, as shown in Table 2. The minimum stretching prediction scheme for UV coordinates is described in Tables 3 and 4.

[0117] Processing multiple parallelograms with a given angle involves performing a lookup around its vertices to compute and summarize the predictions for each parallelogram using the diagonals, such as... Figure 10 A and Figure 10 As shown in B. A parallelogram used to predict an angle from a sibling angle is considered valid for prediction only if the angle itself, its sibling corner, and the vertices sharing a common vertex have previously been processed by the connectivity recursion that triggered the prediction. To verify this condition, a vertex label table (specified as M) is used. This table contains elements set to true for vertices that have been cyclically visited by connectivity encoding. In parallelogram prediction, the parallelogram moves counterclockwise by oscillating around a “triangle fan.” If the next vertex, the previous vertex, and the opposite vertex are available in the parallelogram, then the parallelogram (and three other vertices) are used to predict the location of the current vertex.

[0118] At the end of the loop, the sum of predictions is divided by the number of valid parallelograms identified. The result is rounded and then used to calculate the residual (locality-predicted locality), which is appended to the end of the output vertex table. If no valid parallelogram is found, a fallback to differential decoding is employed.

[0119] For encoding and predicting UV coordinates using the master index table, the procedure follows a similar extension as that used for localization. The key difference is the use of a minimum stretching method instead of multiple parallelograms for prediction. Additionally, the predictions are not summed; instead, the process stops at the first valid (for prediction purposes) nearest neighbor within the triangular sector and calculates the minimum stretching, as follows: Figure 10 A and Figure 10 As described in B. Further details of the uvEncodeWithPrediction procedure can be found in Table 3, while the actual prediction statements are illustrated in Table 4.

[0120] The V-DMC tool has also added support for multiple attributes, where the mesh can have more than one texture map. Similarly, the basic mesh encoder now also has added support for separate indexing of UV coordinates. In this case, the UV coordinates do not have to be in the same order as the positioning (primary attribute).

[0121] The codes shown in Tables 1 through 4 are used only on the encoding side (encoder). The decoder follows similar steps, but in the reverse order.

[0122] Table 1: Encoder Attributes void EBBasicEncoder::encodeMainIndexAttributes(const int c, const intv) { const auto&V = _ovTable.V; const auto&G = _ovTable.positions; const auto&UV = _ovTable.uvcoords; const auto&OTC = _ovTable.OTC; const bool predictUVs = (UV.size()&&!OTC.size()); / / Predict UVs in the first pass if there is no separate index. bool bypasspos = false; if (cfg.deduplicate) { / / Check for duplicate locations const auto dupIt = _ovTable.duplicatesMap.find(_ovTable.V[c]); if (dupIt != _ovTable.duplicatesMap.end()) { isVertexDup.push_back(true); oDuplicateSplitVertexIdx.push_back(dupIt->second); } else isVertexDup.push_back(false); Vcur++; / / Return early if duplicates have been decoded. if (dupIt != _ovTable.duplicatesMap.end()) { if (processedDupIdx.find(dupIt->second) != processedDupIdx.end()) { bypasspos = true; / / No encoding needed, as it has already been processed. posSkipDup++; } else processedDupIdx.insert(dupIt->second); } / / Then, during decoding, the duplicate location information is used to copy the value. } / / Reindex the dummy vertices if (isCornerVertexDummy(c)) { / / Reindexing depends on how certain values ​​are encoded separately. if (cfg.posPred != EBConfig::PosPred::MPARA) oDummies.push_back(oVertices.size()); else oDummies.push_back(oVertices.size() + sVertices.size() +oDummies.size() + posSkipDup); }} if (!bypasspos) { / / Positioning switch (cfg.posPred) { case EBConfig::PosPred::NONE: oVertices.push_back(G[V[c]]); break; case EBConfig::PosPred::MPARA: posEncodeWithPrediction(c, v); break; } / / UV coordinates if (predictUVs) { switch (cfg.uvPred) { case EBConfig::UvPred::NONE: oUVCoords.push_back(UV[V[c]]); break; case EBConfig::UvPred::STRETCH: uvEncodeWithPrediction(c, v); break; } } } } Table 2: posEncodeWithPrediction using multiple parallelograms void EBBasicEncoder::posEncodeWithPrediction(const int c, const int v) { const auto MAX_PARALLELOGRAMS = 4; const auto& OV = _ovTable; const auto& V = _ovTable.V; / / No CC offset const auto& O = _ovTable.O; / / No CC offset const auto&G = _ovTable.positions; / / No CC offset / / Use separate tables for starting vertices and dummy vertices => Fewer unique symbols used for entropy decoding if ((v == 0) || isCornerVertexDummy(c)) { if (v == 0) sVertices.push_back(G[V[c]]); / / Starting point, stored as global coordinates return; } switch (v) { / / case 0 already includes dummy vertex processing case 1: / / Store the difference, case 2: / / Store the difference, oVertices.push_back(G[V[c]] - G[V[OV.p(c)]]); break; default: / / Store parallelogram estimates bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; glm::vec3 predPos(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(O[altC])))&& (!isCornerVertexDummy(OV.p(altC)))&& (!isCornerVertexDummy(OV.n(altC))))&& ((M[V[O[altC]]]]>0)&&(M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Parallelogram prediction estG = prevG + nextG - oppoGd glm::vec3 estG = G[V[OV.p(altC)]] + G[V[OV.n(altC)]]- G[V[O[altC]]]; predPos += estG; / / Cumulative parallelogram prediction ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) / / Use parallelogram prediction if feasible predPos = glm::round(predPos / glm::vec3(count)); / / Divide each component of the vector predPos and round it. else / / or fall back to the difference with an available value / / If prevIsDummy and must be marked, then G[V[OV.n(c)]] cannot be dummy. predPos = prevIsDummy ? G[V[OV.n(c)]] : G[V[OV.p(c)]]; oVertices.push_back(G[V[c]] - predPos); / / Storage residual = Location - Predicted location / / Switch ends } Table 3: uvEncodeWithPrediction void EBBasicEncoder::uvEncodeWithPrediction(const int c, const int v){ auto&OV = _ovTable; auto&V = _ovTable.V; auto&O = _ovTable.O; auto&G = _ovTable.positions; auto&UV = _ovTable.uvcoords; / / A separate table is used for the initial UV coordinates and dummy UV coordinates => Fewer unique symbols are used for entropy decoding if ((v == 0) || isCornerVertexDummy(c)) { if (v == 0) sUVCoords.push_back(UV[V[c]]); / / Starting point, stored as global coordinates This introduces an offset that will be processed during decoding, since there is no associated oUVCoords.push_back. return; } / / Switch based on vertex index. Case 0 has been handled along with the dummy vertex. switch (v) { case 1: / / Difference case 2: / / Difference oUVCoords.push_back(UV[V[c]] - UV[V[OV.p(c)]]); break; default: / / Parallelogram bool prevIsDummy = isCornerVertexDummy(OV.p(c)); bool nextIsDummy = isCornerVertexDummy(OV.n(c)); bool predWithDummies = nextIsDummy || prevIsDummy; if (predWithDummies) { int count = 0; bool last = false; int altC = c; glm::vec3 predPos(0.0, 0.0, 0.0); do / / Loop through the angles attached to the current vertex { if (count>= 1) break; / / If there are no multiple predictions, stop at the first corner found. if ((c != altC)&&isCornerVertexDummy(altC)) / / Stop after looping in both directions or after completing a full lap if (last) break; altC = c; last = true) } / / Ensure that pn from altC with the same V[altC] is decoded and is not a dummy vertex. else if (((!isCornerVertexDummy(OV.p(altC)))&&(!isCornerVertexDummy(OV.n(altC)))) &&((M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { glm::vec2 estUV(0, 0); glm::dvec2 firstestUV(0, 0); predictUV(altC, estUV, firstestUV, V, true, true, prevIsDummy); oUVCoords.push_back(UV[V[c]] - glm::round(estUV)); ++count; } altC = (!last) ? OV.p(O[OV.p(altC)]) : OV.n(O[OV.n(altC)]); / / Swing to the right or left while (altC != c); if (count == 0) / / No corner found { glm::vec2 predUV = prevIsDummy ? UV[V[OV.n(c)]] : UV[V[OV.p(c)]]; const auto resUV = UV[V[c]] - predUV; oUVCoords.push_back(resUV); } } else / / No dummy vertices int count = 0; int altC = c; glm::vec2 predUV(0, 0); bool first = true; glm::dvec2 firstestUV(0, 0); do { glm::vec2 estUV(0, 0); if (((!isCornerVertexDummy(OV.p(altC)))&& (!isCornerVertexDummy(OV.n(altC))))&& ((M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { predictUV(altC, estUV, firstestUV, V, false, first); predUV += estUV; ++count; } altC = OV.p(O[OV.p(altC)]); first = false; while (altC != c); if (count>0) { predUV = glm::round(predUV / glm::vec2(count)); oUVCoords.push_back(UV[V[c]] - predUV); } } break / / switch ends } Table 4: PredictUV / / Angle C, used for prediction / / predWithDummies boolean, used to predict whether to use a dummy branch. / / If the previous corner is a dummy point, then prevIsDummy is set to true (the default is false). void EBBasicEncoder::predictUV(const int c, glm::vec2&predUV, glm::dvec2&firstpredUV, const std::vector <int>&indices, bool predWithDummies, bool first, bool prevIsDummy) { const auto& OV = _ovTable; const auto& V = _ovTable.V; const auto& G = _ovTable.positions; const auto& UV = _ovTable.uvcoords; const auto& IDX = indices; const auto& O = _ovTable.O; const auto& ov = _ovTable; / / Do not accumulate UV predictions, stop after the first prediction glm::dvec2 uvPrev = UV[IDX[OV.p(c)]]; glm::dvec2 uvNext = UV[IDX[OV.n(c)]]; glm::dvec2 uvCurr = UV[IDX[c]]; glm::dvec3 gPrev = G[V[OV.p(c)]]; glm::dvec3 gNext = G[V[OV.n(c)]]; glm::dvec3 gCurr = G[V[c]]; glm::dvec3 gNgP = gPrev - gNext; glm::dvec3 gNgC = gCurr - gNext; glm::dvec2 uvNuvP = uvPrev - uvNext; double gNgP_dot_gNgC = glm::dot(gNgP, gNgC); double d2_gNgP = glm::dot(gNgP, gNgP); if (d2_gNgP > 0) { glm::dvec2 uvProj = uvNext + uvNuvP * (gNgP_dot_gNgC / d2_gNgP); glm::dvec3 gProj = gNext + gNgP * (gNgP_dot_gNgC / d2_gNgP); double d2_gProj_gCurr = glm::dot(gCurr - gProj, gCurr - gProj); const glm::dvec2 uvProjuvCurr = glm::dvec2(uvNuvP.y, -uvNuvP.x) *std::sqrt(d2_gProj_gCurr / d2_gNgP); glm::dvec2 predUV0(uvProj + uvProjuvCurr); glm::dvec2 predUV1(uvProj - uvProjuvCurr); if (first) { / / First triangle bool useOpp = false; bool flag = false; if (ov.OTC.size()) { / / If there are UVs and separate tables if (IDX[O[c]]>= 0) { if (MC[IDX[O[c]]]>0) { flag = true) } } } else { if (IDX[O[c]]>= 0) { if ((M[IDX[O[c]]]>0)&&(!isCornerVertexDummy(O[c]))) { flag = true) } } } if (flag) { glm::dvec2 uvOpp = UV[IDX[O[c]]]; float triangleArea_o = abs(0.5 * (uvNext[0] * uvPrev[1]+ uvPrev[0] *uvOpp[1]+ uvOpp[0] * uvNext[1] - uvNext[0]* uvOpp[1] - uvPrev[0]* uvNext[1] - uvOpp[0]*uvPrev[1])); if (triangleArea_o <DBL_EPSILON) { / / If the texture triangle is a degenerate triangle, do not use the diagonal. useOpp = false; } else { useOpp = true; } } if (useOpp) { glm::dvec2 uvOpp = UV[IDX[O[c]]]; if (length(uvOpp - predUV0) <length(uvOpp - predUV1)) { predUV = predUV1; } else { predUV = predUV0; } } else { bool orientation = length(uvCurr - predUV0) <length(uvCurr - predUV1); predUV = round(orientation ? predUV0 : predUV1); glm::vec2 resUV = UV[IDX[c]] - predUV; orientations.push_back(orientation); } firstpredUV = predUV; } else { if (length(firstpredUV - predUV0) <length(firstpredUV - predUV1)) { predUV = predUV0; } else { predUV = predUV1; } } } else { if (predWithDummies) / / If the next or previous corner is a dummy point { predUV = prevIsDummy ? UV[IDX[OV.n(c)]] : UV[IDX[OV.p(c)]]; } else / / then take the average of the two predictions { predUV = round((UV[IDX[OV.n(c)]] + UV[IDX[OV.p(c)]]) / 2.0f); } glm::vec2 resUV = UV[IDX[c]] - predUV; } } Currently, V-DMC software involves a base mesh encoder. The base mesh encoder encodes both the attributes and connectivity of triangles and vertices. These attributes are typically encoded using a prediction scheme to predict vertex attributes using previously visited / encoded / decoded vertices. The predictions are then subtracted from the actual attribute values ​​to obtain residuals. Finally, the residual attribute values ​​are encoded using an entropy encoder to obtain the encoded base mesh attribute bitstream. The attribute bitstream containing vertex attributes typically has geometric / positional attributes and UV coordinates (texture attributes), but can contain any number of attributes such as normals, per-vertex RGB values, etc.

[0123] The attribute encoder in the basic mesh encoder, such as Figure 8A As shown and including: - Topology / Connectivity: The topology in the base mesh is encoded using the EdgeBreaker with CLERS opcodes. This includes not only connectivity information but also the mesh's data structure (currently implemented using a corner table). Topology / connectivity information is used to find neighboring vertices.

[0124] - Attributes: These include geometry (3D coordinates), UV coordinates (texture), normals, RGB values, etc.

[0125] - Neighborhood attributes: These are the attributes of the neighboring vertices that are used to predict the attributes of the current vertex.

[0126] - Current Attributes: These are the attributes of the current vertex. Subtract the predicted attributes from the current vertex attributes to obtain the residual.

[0127] - Prediction. These predictions can be obtained from connectivity and / or from previously visited / encoded / decoded vertices. For example, geometric multiparallelogram processes, minimum stretching schemes for UV coordinates, etc.

[0128] - Residuals. These are obtained by subtracting the prediction from the original attribute. (For example, residual = current_vertex_attribute – predicted_attribute).

[0129] - Entropy coding. Finally, entropy coding is performed on the residual to obtain the bitstream.

[0130] Figure 8A and Figure 8B The encoder and decoder architecture for basic mesh encoding / decoding (also known as static mesh encoding / decoding) is shown. Figure 8A The basic mesh encoder 812 is shown, which represents Figure 2 The example implementation of the basic mesh encoder 212 in the example is as follows, while Figure 8B The basic mesh decoder 814 is shown, which represents Figure 3 The example implementation of the basic mesh decoder 314 in the example.

[0131] exist Figure 8A In the example, the base mesh encoder 812 determines the reconstructed neighbor attributes 830 and topological / connectivity information 832 to determine the prediction 834. The base mesh encoder 812 subtracts (842) the prediction 834 from the current attribute 836 to determine the residual 838. The reconstructed neighbor attributes 830 represent the decoded values ​​of the encoded vertex attributes, and the current attribute 836 represents the actual values ​​of the unencoded vertex attributes. Therefore, the residual 838 represents the difference between the actual values ​​of the unencoded vertex attributes and the predicted values ​​of these vertex attributes. The base mesh encoder 812 may entropy encode the residual 838 (840).

[0132] exist Figure 8B In the example, the base grid decoder 814 determines the reconstructed neighbor attribute 860 and topological / connectivity information 862 to determine the prediction 864 in the same manner as the base grid encoder 812 determines the prediction 834. The base grid decoder 814 performs entropy decoding (870) on the entropy-encoded residual value to determine the residual 868. The base grid decoder 814 adds the prediction 864 to the residual 868 (1172) to determine the reconstructed current attribute 866. The reconstructed current attribute 866 represents the decoded version of the current attribute 836.

[0133] The base mesh encoder is an integral part of the V-DMC architecture. Lossless coding in V-DMC heavily relies on the base mesh encoder, and in lossless coding, the quality of the base mesh encoding significantly impacts the final reconstructed mesh. The base mesh encoder also forms a significant portion of the overall V-DMC bitstream. Therefore, it may be necessary to further reduce the size of the base mesh encoder bitstream. The attribute bitstream encapsulates a large portion of the base mesh encoded bitstream. The attribute bitstream contains residual attribute values. It can be observed that the entropy of these residuals remains large, and correlations may still exist between residuals of neighboring vertices. The goal could be to reduce the size of the attribute bitstream and further reduce the entropy of the attributes and / or residuals to be encoded against the base mesh.

[0134] A normal is a vector perpendicular to a mesh surface. Normals are used to determine the orientation of a surface and to apply shading. In 3D modeling, normals play a crucial role in creating realistic objects. Normals help define the shape of an object and how it interacts with light. Normals are also used in computer graphics to create smooth surfaces and calculate light reflection. Furthermore, normals are used in video games to create realistic environments and improve game performance. In short, normals are an essential component of 3D modeling and computer graphics and are used in a wide range of applications, from video games to scientific simulations.

[0135] The current version of the V-DMC test model v6.0 (TMM v6.0) supports encoding of mesh localization and texture data, but lacks integration of normal encoding. The V-DMC base mesh decoder in m63344 encodes mesh connectivity, while localization and texture are encoded as vertex attributes. Normal encoding needs to be incorporated into the dynamic mesh decoder and supported as a mesh attribute. This disclosure describes techniques for introducing specific implementations of normal encoding into V-DMC TMM v6.0, including the base mesh EdgeBreaker implementation in m63344. This disclosure describes techniques for encoding normals and normal maps in V-DMC TMM.

[0136] This disclosure describes the following technologies: ● Integrate normal encoding into V-DMC.

[0137] ● Propose normal map encoding in V-DMC.

[0138] ● Propose per-vertex normal encoding in V-DMC.

[0139] ● Integrate normal encoding into the base mesh encoder within the V-DMC.

[0140] ● A prediction scheme for normal encoding in a basic mesh encoder is proposed.

[0141] The basic mesh decoder in the current version of V-DMC is shown as Figure 6 The basic mesh decoder 650 is described below. The various aspects of the system architecture will now be described. This disclosure categorizes normal encoding into two types / categories. Normal encoding typically refers to per-vertex normal values, meaning that each vertex can have a corresponding normal vector (x, y, z). Normal map encoding typically refers to using normal maps that include 2D normal images containing normal values ​​and corresponding per-vertex normal coordinates (u, v). The normal coordinates give the position of the normal value on the normal map.

[0142] V-DMC decoding conditions can be categorized as follows: (1) Lossless mode: intra-frame compression, (2) Lossy mode: intra-frame compression, (3) Lossy mode: inter-frame compression. Figures 9 to 14 An update to the V-DMC architecture is shown, where all three conditions are addressed for both normal encoding and normal map encoding. The structure of these graphs is largely the same as... Figure 5 , Figure 6 , Figure 7 The same applies, except that per-vertex normal encoding is added to the architecture.

[0143] Video decoder of normal map .

[0144] The technique disclosed herein can encode normal maps using the same video decoder used in texture map encoding.

[0145] Displacement decoder with normal support .

[0146] It can be a displacement vector of the normal line, just like how the displacement vector of the positioning code is used in V-DMC.

[0147] The displacement vector of a normal changes its direction and can be one-dimensional or three-dimensional. The displacement vector of a normal can be... Figures 4 to 6 The displacement vectors shown are encoded in the same way.

[0148] Motion decoder with normal support .

[0149] Motion encoding / decoding of normals can be implemented in the same way as motion encoding / decoding of localization. Motion vectors change normals from one frame to the next and are only used in inter-frame prediction decoding mode.

[0150] Static mesh decoder with normal support .

[0151] This static mesh encoder is suitable for normal map encoding. When using normal maps, the 2D normal map coordinates (u, v) need to be encoded. This can be achieved in two ways. The normal map can use the same coordinates as the texture map, or it can have separate coordinates.

[0152] If the texture and normal map share the same coordinates, they can share the same UV coordinates without any editing. However, if new normal coordinates are to be encoded, the static mesh encoder will use the same process as encoding the UV coordinates of the texture to encode the individual normal coordinates.

[0153] Figure 9 This illustrates normal encoding in lossless mode. Figure 9 In the example, the V-DMC decoder 900 receives an input mesh frame and extracts the Basic Mesh Sub-stream (BMSB). The Static Mesh and Normal Attribute Decoder 902 losslessly decodes the BMSB to determine the mesh's connectivity, geometry, UV coordinates, and normals. The V-DMC decoder 900 also extracts the Texture Map Sub-stream (TMSB) from the input mesh frame, and the video decoder 904 decodes the Texture Map Sub-stream to determine a 2D texture image. The 2D texture image may include multiple texture values, and the mesh's UV coordinates may correspond to texture values ​​in the image.

[0154] Figure 10 This illustrates normal coding in lossy intra-frame mode. Figure 10 In the example, the V-DMC decoder 1000 receives an input mesh frame and extracts the Base Mesh Sub-Stream (BMSB). The Static Mesh and Normal Property Decoder 1002 decodes the Base Mesh Sub-Stream to determine the connectivity, geometry, UV coordinates, and normals of the base mesh. The V-DMC decoder 1000 also extracts the Displacement Sub-Stream (DSB) from the input mesh frame, and the Displacement Decoder 1006 determines the displacement vector. The V-DMC decoder 1000 then modifies the base mesh based on the displacement to determine the reconstructed deformable mesh. The V-DMC decoder 1000 may also determine new normal values ​​or modify already determined normal values ​​based on the displacement vector. The V-DMC decoder 1000 also extracts the Texture Map Sub-Stream (TMSB) from the input mesh frame, and the Video Decoder 1004 decodes the Texture Map Sub-Stream to determine a 2D texture image. The 2D texture image may include multiple texture values, and the UV coordinates of the mesh may correspond to the texture values ​​in the image.

[0155] Figure 11 This illustrates normal coding in lossy inter-frame mode. Figure 11 In the example, V-DMC decoder 1100 receives an input mesh frame and extracts a base mesh sub-bitstream. Static mesh and normal attribute decoder 1002 decodes the base mesh sub-bitstream to determine motion vectors and, based on the motion vectors stored in base mesh buffer 1108 and the decoded base mesh, determines the connectivity, geometry, UV coordinates, and normals of the base mesh. V-DMC decoder 1100 also extracts a displacement sub-bitstream (DSB) from the input mesh frame, and displacement decoder 1106 determines displacement vectors. V-DMC decoder 1100 then modifies the base mesh based on the displacement vectors to determine a reconstructed deformable mesh. V-DMC decoder 1100 may also determine new normal values ​​or modify already determined normal values ​​based on the displacement vectors. V-DMC decoder 1100 also extracts a texture map sub-bitstream from the input mesh frame, and video decoder 1104 decodes the texture map sub-bitstream to determine a 2D texture image. The 2D texture image may include multiple texture values, and the UV coordinates of the mesh may correspond to texture values ​​in the image.

[0156] Figure 12 This illustrates normal map encoding in lossless mode. Figure 12 In the example, V-DMC decoder 1200 receives an input mesh frame and extracts a base mesh sub-bitstream. Static mesh and normal support decoder 1202 losslessly decodes the base mesh sub-bitstream to determine the mesh's connectivity, geometry, UV coordinates, and normal coordinates. V-DMC decoder 900 also extracts a texture map sub-bitstream from the input mesh frame, and video decoder 904 decodes the texture map sub-bitstream to determine a 2D texture image. The 2D texture image may include multiple texture values, and the mesh's UV coordinates may correspond to texture values ​​in the image. V-DMC decoder 1200 also extracts a normal map sub-bitstream (NMSB) from the input mesh frame, and video decoder 1210 decodes the normal map sub-bitstream to determine a 2D normal image. The 2D normal image may include multiple normal values, and the mesh's normal coordinates may correspond to normal values ​​in the image.

[0157] Figure 13 This illustrates normal map coding in lossy intra-frame mode. Figure 13 In the example, the V-DMC decoder 1300 receives an input mesh frame and extracts a base mesh sub-bitstream. The static mesh and normal attribute decoder 1302 decodes the base mesh sub-bitstream to determine the connectivity, geometry, UV coordinates, and normal coordinates of the base mesh. The V-DMC decoder 1300 also extracts a displacement sub-bitstream (DSB) from the input mesh frame, and the displacement decoder 1306 determines the displacement vector. Then, the V-DMC decoder 1300 modifies the base mesh based on the displacement to determine the reconstructed deformable mesh. The V-DMC decoder 1300 can also determine new normal coordinates or modify already determined normal coordinates based on the displacement vector. The V-DMC decoder 1300 also extracts a texture map sub-bitstream from the input mesh frame, and the video decoder 1304 decodes the texture map sub-bitstream to determine a 2D texture image. The 2D texture image may include multiple texture values, and the UV coordinates of the mesh may correspond to the texture values ​​in the image. The V-DMC decoder 1300 also extracts a normal map sub-bit stream from the input mesh frame, and the video decoder 1310 decodes the normal map sub-bit stream to determine a 2D normal image. The 2D normal image may include multiple normal values, and the normal coordinates of the mesh may correspond to the normal values ​​in the image.

[0158] Figure 14 This illustrates normal map coding in lossy inter-frame mode. Figure 14 In the example, the V-DMC decoder 1400 receives an input mesh frame and extracts a base mesh sub-bitstream. The static mesh and normal attribute decoder 1402 decodes the base mesh sub-bitstream to determine motion vectors and, based on the motion vectors stored in the base mesh buffer 1408 and the decoded base mesh, determines the connectivity, geometry, UV coordinates, and normal coordinates of the base mesh. The V-DMC decoder 1400 also extracts a displacement sub-bitstream from the input mesh frame, and the displacement decoder 1406 determines displacement vectors. The V-DMC decoder 1400 then modifies the base mesh based on the displacement vectors to determine a reconstructed deformable mesh. The V-DMC decoder 1400 may also determine new normal values ​​or modify already determined normal values ​​based on the displacement vectors. The V-DMC decoder 1400 also extracts a texture map sub-bitstream from the input mesh frame, and the video decoder 1404 decodes the texture map sub-bitstream to determine a 2D texture image. The 2D texture image may include multiple texture values, and the UV coordinates of the mesh may correspond to texture values ​​in the image. The V-DMC decoder 1400 also extracts a normal map sub-bit stream from the input mesh frame, and the video decoder 1410 decodes the normal map sub-bit stream to determine a 2D normal image. The 2D normal image may include multiple normal values, and the normal coordinates of the mesh may correspond to the normal values ​​in the image.

[0159] The aspects of a static mesh with normal support will now be described. The static mesh encoder employs an EdgeBreaker to encode the connectivity / topology of the underlying mesh and uses methods such as... Figure 8A and Figure 8B The prediction scheme shown encodes the underlying mesh attributes (position, UV coordinates). The attribute prediction scheme will be related to... Figure 15 A more detailed description is needed. Adding new properties to the base grid can include new prediction schemes and follow the principles outlined below. Figure 8A and Figure 8B The same process as shown.

[0160] This disclosure describes several prediction schemes for static grid coding.

[0161] Prediction schemes used by other attributes include minimum stretch prediction for UV coordinates (texture) and multi-parallelogram prediction for positioning. Both prediction schemes employ... Figure 15 The corner table representation is shown below. The published prediction scheme can also be represented using the default corner table data of the edge breaker.

[0162] Figure 15 An example angle table representation is shown. Figure 15 Use the following abbreviations: current vertex (c), next vertex (cn), previous vertex (cp), relative vertex (co). This order creates a sector around the current vertex c. Figure 15 The right corner (cr) and the left corner (cl) are also shown.

[0163] The prediction scheme according to the technique disclosed herein will now be described. Multi-parallelogram prediction will now be described. Estimated normals are calculated by looking around each vertex to utilize the diagonals for calculating and summing the predictions for each parallelogram. All parallelograms with all vertices visited are used as predictions, and finally, all predictions are averaged.

[0164] The expression for estNorm of a parallelogram is: estNorm = PrevNorm + NextNorm - OppositeNorm; Table 5: Normal Prediction: Multiple Parallelograms .

[0165] bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; glm::vec3 predNorm(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(O[altC])) &&(!isCornerVertexDummy(OV.p(altC))) &&(!isCornerVertexDummy(OV.n(altC)))) &&((M[V[O[altC]]]]>0)&&(M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Parallelogram prediction estNorm = prevNorm + nextNorm - oppoNorm glm::vec3 estNorm = Norm[V[OV.p(altC)]] + Norm[V[OV.n(altC)]]- Norm[V[O[altC]]]; predNorm += estNorm; / / Cumulative parallelogram prediction ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) / / Use parallelogram prediction if feasible predNorm = glm::round(predNorm / glm::vec3(count)); / / Divide and round each component of the vector predPos. else / / or fall back to the difference with an available value / / If prevIsDummy and must be marked, then G[V[OV.n(c)]] cannot be dummy. predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Additional features of multiple parallelograms : Version 7.0 of V-DMC's TMM introduces the following improvements to the multi-parallelogram method used for normal prediction: ● Perform predictions using parallelograms around the current vertex (the vertex whose normal is being predicted).

[0166] ● Walk around the fan around the current vertex ( Figure 15 ), and uses a parallelogram formed by c, cn, cp and co (which are the current vertex, the next vertex, the previous vertex and the opposite vertex).

[0167] ● Use the parallelogram to predict the normal to the current vertex.

[0168] ● Take the average of all predictions from different parallelograms around the current vertex.

[0169] ● Regularize the forecast (to improve the forecast).

[0170] ● Scale the predictions back to the original scale / quantization (because they will be lost after normalization).

[0171] ● Finally, subtract the prediction from the original normal and store the residual (e.g., Figure 11 A).

[0172] ● If no parallelograms are available around the fan, differential decoding is performed by predicting the previous or next vertex.

[0173] ● If the previous and next vertices are not available, the vertex may be located on the boundary and the boundary-adjacent vertices can be used for prediction.

[0174] ● If none of these conditions are met, then the vertex can be the first vertex of the grid, and therefore, the global value of that vertex is stored instead of being predicted.

[0175] Table 5.1 illustrates specific implementations of potential improvements to the multi-parallelograms used for normals in v7.0.

[0176] Table 5.1: Normal Prediction: Improved Multiparallelograms in v7.0 .

[0177] const auto MAX_PARALLELOGRAMS = 4; const auto&ov = _ovTable; const auto&V = ov.V; const auto&O = ov.O; const auto&Norm = ov.normals; const auto&v = ov.v(c); / / Has the vertex been predicted? if (MV[v]>0) return; / / Mark this vertex MV[v] = 1; / / Search for some parallelogram estimates around the corner vertices / / Since dummy points are not used, the triangular sector may be incomplete. / / However, since the vertices are not non-manifolds, each vertex has only one fan. / / Additionally, some vertical angles may not be definable due to boundary conditions. / / Use OV accessor and test to filter negative values glm::vec3 predNorm(0, 0, 0); / / Predict normals int count = 0; / / The number of valid parallelograms found int altC = c; / / Loop through the angles attached to the current vertex / / Swing to the right around the fan int nextC = ov.n(O[ov.n(altC)]); while (nextC>= 0&&nextC != c) { altC = nextC; nextC = ov.n(O[ov.n(altC)]); }; bool isBoundary = (nextC != c); / / Now positioned at the rightmost corner of the shared v, / / Turn left and assess possible predictions const int startC = altC; do { if (count>= MAX_PARALLELOGRAMS) break; const auto&oppoV = ov.v(O[altC]); const auto&prevV = ov.v(ov.p(altC)); const auto&nextV = ov.v(ov.n(altC)); if ((oppoV>-1&&prevV>-1&&nextV>-1)&& ((MV[oppoV]>0)&&(MV[prevV]>0)&&(MV[nextV]>0))) { / / Parallelogram prediction estNorm = prevNrm + nextNrm - oppoNrm glm::vec3 estNorm = Norm[prevV] + Norm[nextV]- Norm[oppoV]; predNorm += estNorm; / / Cumulative parallelogram prediction ++count; } altC = ov.p(O[ov.p(altC)]); / / Swing around the triangular fan } while (altC>= 0&&altC != startC); / / Incomplete fan or complete rotation / / 1. Use parallelogram prediction when feasible if (count>0) { predNorm = glm::round(predNorm / glm::vec3(count)); / / Center the prediction.

[0178] const int32_t center = ( 1u< <static_cast<uint32_t> ( qn-1 ) ); for (int c = 0; c<3; c++) { predNorm[c] = predNorm[c]- center; } / / Regularize the prediction predNorm = glm::normalize( predNorm ); if (!std::isnan( predNorm[0] ) ) { / / Quantize this normal const glm::vec3 minNrm = {-1.0, -1.0, -1.0}; const glm::vec3 maxNrm = {1.0, 1.0, 1.0}; const glm::vec3 diag = maxNrm - minNrm; const float range = std::max( std::max(diag.x, diag.y ), diag.z ); const int32_t maxNormalQuantizedValue = ( 1u<<static_cast<uint32_t>( qn ) ) - 1; for (int c = 0; c<3; c++) { predNorm[c] = static_cast <float>(std::floor( ( ( predNorm[c] - minNrm[c]) / range ) * maxNormalQuantizedValue + 0.5f ) ); } oNormals.push_back(Norm[v]- predNorm); oNrmFine.push_back(true); return; } } / / 2. Or roll back to the difference with an available value. const auto&c_p_v = ov.v(ov.p(c)); const auto&c_n_v = ov.v(ov.n(c)); if (c_p_v>-1&&MV[c_p_v]>-1) { oNormals.push_back(Norm[v] - Norm[c_p_v]); oNrmFine.push_back(false); return; } if (c_n_v>-1&&MV[c_n_v]>-1) { oNormals.push_back(Norm[v] - Norm[c_n_v]); oNrmFine.push_back(false); return; } / / 3. If located on the boundary / / Then use the difference from the previous vertex on the boundary. if (isBoundary) { const auto b = ov.p(startC); / / b is located on the boundary const auto b_v = ov.v(b); auto marked = MV[b_v]; if (marked>-1) { oNormals.push_back(Norm[v] - Norm[b_v]); oNrmFine.push_back(false); return; } } / / 4. No other options osNormals.push_back(Norm[v]); / / Global value (it is the starting point, and the push is performed in a separate table) Differential decoding (procedure #2) The process does not loop around the "fan" and does not perform multiple estimates. Instead, the process uses only a single corner (the previous or the next). The V-DMC encoder 200 examines the previous and / or next corner and estimates the difference between the normal of the previous and / or next vertex and the normal of the current vertex, and encodes that difference.

[0179] estNorm = PrevNorm; EncodedNorm = estNorm – actualNorm; Table 6: Normal Prediction: Differential Decoding oNormals.push_back(Norm[V[c]] - Norm[V[OV.p(c)]]); Additional features for multi-parallelogram prediction will now be described. V-DMC v7.0 TMM introduces the following improvements to the multi-parallelogram method for normal prediction: ● The prediction will be the normal of the previous vertex or the normal of the next vertex.

[0180] ● If the previous and next vertices are not available, the vertex may be located on the boundary and the boundary-adjacent vertices can be used for prediction.

[0181] ● If none of these conditions are met, the vertex can be the first vertex of the mesh, and therefore the V-DMC decoder 300 can store the global value of the vertex instead of predicting it.

[0182] Table 6.1 shows the specific implementation of the improved differential decoding of normals in v7.0 TMM.

[0183] Table 6.1 Normal Prediction: Improvements in Differential Decoding in v7.0 TMM const auto&ov = _ovTable; const auto&V = ov.V; const auto&O = ov.O; const auto&Norm = ov.normals; const auto&v = ov.v(c); / / Has the vertex been predicted? if (MV[v]>0) return; / / Mark this vertex MV[v] = 1; oNrmFine.push_back(false); / / For constant false difference glm::vec3 predNorm(0, 0, 0); / / Predictive localization int count = 0; / / The number of valid parallelograms found int altC = c; / / Loop through the angles attached to the current vertex / / Swing to the right around the fan int nextC = ov.n(O[ov.n(altC)]); while (nextC>= 0&&nextC != c) { altC = nextC; nextC = ov.n(O[ov.n(altC)]); }; bool isBoundary = (nextC != c); / / 1. Use the difference with available values const auto&c_p_v = ov.v(ov.p(c)); const auto&c_n_v = ov.v(ov.n(c)); if (c_p_v>-1&&MV[c_p_v]>-1) { oNormals.push_back(Norm[v] - Norm[c_p_v]); return; } if (c_n_v>-1&&MV[c_n_v]>-1) { oNormals.push_back(Norm[v] - Norm[c_n_v]); return; } / / 2. If located on the boundary / / Then the difference from the previous vertex on the boundary can be used. if (isBoundary) { const auto b = ov.p(altC); / / b is located on the boundary const auto b_v = ov.v(b); auto marked = MV[b_v]; if (marked>-1) { oNormals.push_back(Norm[v] - Norm[b_v]); return; } } / / 3. No other options osNormals.push_back(Norm[v]); / / Global value (it is the starting point, and the push is performed in a separate table) Process #3 The process does not loop around the "fan" and does not perform multiple estimates. Instead, the V-DMC decoder 300 averages the previous and next values ​​using the current triangle.

[0184] Table 7: Normal Prediction: Process #3 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); glm::vec3 predNorm(0, 0, 0); / / Predictive localization if ((!isCornerVertexDummy(OV.p(c)))&&(!isCornerVertexDummy(OV.n(c)))&&(M[V[OV.p(c)]]]>0)&&(M[V[OV.n(c)]]>0)){ / / The previous and next vertices have been visited and are not dummy vertices. predNorm = ((Norm[V[OV.p(c)]] + Norm[V[OV.n(c)]]) / 2.0f); }else { / / Or fall back to the difference with an available value / / If prevIsDummy and must be marked, then G[V[OV.n(c)]] cannot be dummy. predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; } oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Process #4 During this process, the V-DMC decoder 300 does not loop around the "fan" and does not make multiple estimates. Instead, the V-DMC decoder 300 uses the current triangle to average three values: the previous value, the next value, and the relative value.

[0185] Table 8: Normal Prediction: Process #4 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); glm::vec3 predNorm(0, 0, 0); / / Predictive localization if (((!isCornerVertexDummy(O[c]))&&(!isCornerVertexDummy(OV.p(c)))&&(!isCornerVertexDummy(OV.n(c))))&&((M[V[O[c]]]]>0)&&(M[V[OV.p(c)]]>0)&&(M[V[OV.n(c)]]>0))) { / / The previous vertex, next vertex, and relative vertex have all been visited and are not dummy vertices. predNorm = ((Norm[V[OV.p(c)]] + Norm[V[OV.n(c)]]+ Norm[V[O[c]]]) / 3.0f); }else { predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; } oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Process #5 During this process, the V-DMC decoder 300 loops around the "fan" and makes multiple estimates. The V-DMC decoder 300 uses the average of the previous and next normal values ​​to make an estimate. Finally, the V-DMC decoder 300 averages all estimates. Note that, as always, estimates are made only if these vertices have been previously visited.

[0186] Table 9: Normal Line Prediction: Process #5 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; glm::vec3 predNorm(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(OV.p(altC)))&&!isCornerVertexDummy(OV.n(altC))))&&((M[V[OV.p(altC)]]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Parallelogram prediction estNorm = (prevNorm + nextNorm) / 2 glm::vec3 estNorm = ((Norm[V[OV.p(altC)]] + Norm[V[OV.n(altC)]]) / 2.0f); predNorm += estNorm; / / Cumulative parallelogram prediction ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) / / Use parallelogram prediction if feasible predNorm = glm::round(predNorm / glm::vec3(count)); / / Divide and round each component of the vector predPos. else / / or fall back to the difference with an available value / / If prevIsDummy and must be marked, then G[V[OV.n(c)]] cannot be dummy. predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Process #6 During this process, the V-DMC decoder 300 loops around the "fan" and makes multiple estimates. The V-DMC decoder 300 uses the average of the previous corner, the next corner, and the relative normal values ​​to make an estimate. Finally, the V-DMC decoder 300 averages all estimates. Note that, as always, estimates are made only if these vertices have been previously visited.

[0187] Table 10: Normal Prediction: Process #6 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; glm::vec3 predNorm(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(O[altC]))&&(!isCornerVertexDummy(OV.p(altC)))&&(!isCornerVerte xDummy(OV.n(altC))))&&((M[V[O[altC]]]]>0)&&(M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Parallelogram prediction estNorm = (prevNorm + nextNorm) / 2 glm::vec3 estNorm = ((Norm[V[OV.p(altC)]] + Norm[V[OV.n(altC)]]+ Norm[V[O[altC]]]) / 3.0f); predNorm += estNorm; / / Cumulative parallelogram prediction ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) / / Use parallelogram prediction if feasible predNorm = glm::round(predNorm / glm::vec3(count)); / / Divide and round each component of the vector predPos. else / / or fall back to the difference with an available value / / If prevIsDummy and must be marked, then G[V[OV.n(c)]] cannot be dummy. predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Process #7 During this process, the V-DMC decoder 300 does not loop around the "fan" and does not perform multiple estimates. Instead, the V-DMC decoder 300 uses the current triangle during this process.

[0188] The V-DMC decoder 300 uses three values ​​to predict the normal of the current vertex: the previous value, the next value, and the relative value.

[0189] AvgTemp = (PrevNorm + NextNorm) / 2; estNorm = AvgTemp - (OppositeNorm - AvgTemp); Table 11: Normal Prediction: Process #7 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); glm::vec3 predNorm(0, 0, 0); / / Predictive localization if (((!isCornerVertexDummy(O[c]))&&(!isCornerVertexDummy(OV.p(c)))&&(!isCornerVertexDummy(OV.n(c))))&&((M[V[O[c]]]]>0)&&(M[V[OV.p(c)]]>0)&&(M[V[OV.n(c)]]>0))) { / / The previous vertex, next vertex, and relative vertex have all been visited and are not dummy vertices. glm::vec3 avgPN = ((Norm[V[OV.p(c)]] + Norm[V[OV.n(c)]]) / 2.0f); predNorm = avgPN - (Norm[V[O[c]]] - avgPN); }else { / / Or fall back to the difference with an available value / / If prevIsDummy and must be marked, then G[V[OV.n(c)]] cannot be dummy. predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; } oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Process #8 In this process, the V-DMC decoder 300 repeats process #7, but across the entire sector. During this process, the V-DMC decoder 300 loops around the "sector" and makes multiple estimates. Finally, the V-DMC decoder 300 averages all the estimates. Note that, as always, estimates are made only if these vertices have been previously visited.

[0190] The prediction process is the same as process #7, but it applies to all parallelograms around the current corner vertex.

[0191] AvgTemp = (PrevNorm + NextNorm) / 2; estNorm = AvgTemp - (OppositeNorm - AvgTemp); This process is similar to process #0 because the above equations can be simplified to the same equations in process #0.

[0192] Table 12: Normal Prediction: Process #8 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; glm::vec3 predNorm(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(O[altC]))&&(!isCornerVertexDummy(OV.p(altC)))&&(!isCornerVerte xDummy(OV.n(altC))))&&((M[V[O[altC]]]]>0)&&(M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Parallelogram prediction estNorm = (prevNorm + nextNorm) / 2 glm::vec3 avgPN = ((Norm[V[OV.p(altC)]] + Norm[V[OV.n(altC)]]) / 2.0f); glm::vec3 estNorm = avgPN - (Norm[V[O[altC]]] - avgPN); predNorm += estNorm; / / Cumulative parallelogram prediction ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) / / Use parallelogram prediction if feasible predNorm = glm::round(predNorm / glm::vec3(count)); / / Divide and round each component of the vector predPos. else / / or fall back to the difference with an available value / / If prevIsDummy and must be marked, then G[V[OV.n(c)]] cannot be dummy. predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Inverse distance weighted (L1), process #9 During this process, the V-DMC decoder 300 loops around the "fans" and makes multiple estimates. Finally, the V-DMC decoder 300 takes a weighted average of all estimates. Note that, as always, estimates are made only if these vertices have been previously visited.

[0193] Inverse distance weighting is used for estimation. This distance is calculated as L1 distance.

[0194] Table 13: Normal Prediction: Process #9 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; double totalweight = 0; glm::vec3 numerator(0, 0, 0); / / Predictive localization glm::vec3 predNorm(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(O[altC]))&&(!isCornerVertexDummy(OV.p(altC)))&&(!isCornerVerte xDummy(OV.n(altC))))&&((M[V[O[altC]]]]>0)&&(M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Parallelogram prediction glm::vec3 avgPN = ((Norm[V[OV.p(altC)]] + Norm[V[OV.n(altC)]]) / 2.0f); glm::vec3 estNorm = avgPN - (Norm[V[O[altC]]] - avgPN); / / Storage distance weight glm::vec3 gPrev = G[V[OV.p(altC)]]; glm::vec3 gNext = G[V[OV.n(altC)]]; glm::vec3 gCurr = G[V[altC]]; glm::vec3 gPgN_2 = ((gPrev + gNext) / 2.0f); glm::vec3 gPgN_2_gCurr = gCurr - gPgN_2; double d2_gPgN_2_gCurr = glm::dot(gPgN_2_gCurr, gPgN_2_gCurr); for (auto k = 0; k<3; ++k) { numerator[k] += estNorm[k] / d2_gPgN_2_gCurr;} totalweight += 1 / d2_gPgN_2_gCurr; ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) / / Use parallelogram prediction if feasible for (auto k = 0; k<3; ++k) { predNorm[k] = round(numerator[k] / totalweight); } else / / or fall back to the difference with an available value predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Inverse distance weighted (L2), process #10 Same as procedure #9, but uses L2 distance (mean square distance) instead of L1 distance to calculate the distance.

[0195] Table 14: Normal Prediction: Process #10 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; double totalweight = 0; glm::vec3 numerator(0, 0, 0); / / Predictive localization glm::vec3 predNorm(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(O[altC]))&&(!isCornerVertexDummy(OV.p(altC)))&&(!isCornerVerte xDummy(OV.n(altC))))&&((M[V[O[altC]]]]>0)&&(M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Parallelogram prediction glm::vec3 avgPN = ((Norm[V[OV.p(altC)]] + Norm[V[OV.n(altC)]]) / 2.0f); glm::vec3 estNorm = avgPN - (Norm[V[O[altC]]] - avgPN); / / Storage distance weight glm::vec3 gPrev = G[V[OV.p(altC)]]; glm::vec3 gNext = G[V[OV.n(altC)]]; glm::vec3 gCurr = G[V[altC]]; glm::vec3 gPgN_2 = ((gPrev + gNext) / 2.0f); glm::vec3 gPgN_2_gCurr = gCurr - gPgN_2; double d2_gPgN_2_gCurr = std::sqrt(glm::dot(gPgN_2_gCurr, gPgN_2_gCurr)); for (auto k = 0; k<3; ++k) { numerator[k] += estNorm[k] / d2_gPgN_2_gCurr;} totalweight += 1 / d2_gPgN_2_gCurr; ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) / / Use parallelogram prediction if feasible for (auto k = 0; k<3; ++k) { predNorm[k] = round(numerator[k] / totalweight); } else / / or fall back to the difference with an available value predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Process #11 During this process, the V-DMC decoder 300 loops around the "fan" and makes multiple estimates.

[0196] In this process, the estimate is the normal to the midpoint between the previous and next vertices. Then, the V-DMC decoder 300 selects the estimate that is closest to the current corner vertex.

[0197] Table 15: Normal Prediction: Process #11 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; double minDist = std::numeric_limits <double>::max(); glm::vec3 predNorm(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(O[altC]))&&(!isCornerVertexDummy(OV.p(altC)))&&(!isCornerVerte xDummy(OV.n(altC))))&&((M[V[O[altC]]]]>0)&&(M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Parallelogram prediction glm::vec3 avgPN = ((Norm[V[OV.p(altC)]] + Norm[V[OV.n(altC)]]) / 2.0f); glm::vec3 estNorm = avgPN - (Norm[V[O[altC]]] - avgPN); / / Storage distance weight glm::vec3 gPrev = G[V[OV.p(altC)]]; glm::vec3 gNext = G[V[OV.n(altC)]]; glm::vec3 gCurr = G[V[altC]]; glm::vec3 gPgN_2 = ((gPrev + gNext) / 2.0f); glm::vec3 gPgN_2_gCurr = gCurr - gPgN_2; double d2_gPgN_2_gCurr = std::sqrt(glm::dot(gPgN_2_gCurr, gPgN_2_gCurr)); if (d2_gPgN_2_gCurr <minDist) { minDist = d2_gPgN_2_gCurr; predNorm = estNorm; } ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) { / / use parallelogram prediction when possible ;} else / / or fall back to the difference with an available value predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Process #12 In this process, the V-DMC decoder 300 loops around the "fan" and uses the multi-parallelogram method used in process #1 to estimate the normal. However, instead of averaging all estimated normals, the V-DMC decoder 300 selects the estimated normal calculated from the triangle with the closest diagonal distance to the vertex of the current angle.

[0198] Table 16: Normal Prediction: Process #12 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; double minDist = std::numeric_limits <double>::max(); glm::vec3 predNorm(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(O[altC]))&&(!isCornerVertexDummy(OV.p(altC)))&&(!isCornerVerte xDummy(OV.n(altC))))&&((M[V[O[altC]]]]>0)&&(M[V[OV.p(altC)]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Parallelogram prediction glm::vec3 avgPN = ((Norm[V[OV.p(altC)]] + Norm[V[OV.n(altC)]]) / 2.0f); glm::vec3 estNorm = avgPN - (Norm[V[O[altC]]] - avgPN); / / Storage distance weight glm::vec3 gO = G[V[O[altC]]]; glm::vec3 gCurr = G[V[altC]]; glm::vec3 gO_2_gCurr = gCurr - gO; double d2_gO_2_gCurr = std::sqrt(glm::dot(gO_2_gCurr, gO_2_gCurr)); if (d2_gO_2_gCurr <minDist) { minDist = d2_gO_2_gCurr; predNorm = estNorm; } ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) { / / use parallelogram prediction when possible ;} else / / or fall back to the difference with an available value predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Process #13 During this process, the V-DMC decoder 300 loops around the "fan" and selects the normal of the vertex closest to the current corner vertex. The V-DMC decoder 300 may only consider the vertices of the previous and next corners in each parallelogram, as these corners will be closest to the current corner vertex.

[0199] Table 17: Normal Prediction: Process #13 bool prevIsDummy = isCornerVertexDummy(OV.p(c)); / / Search for some parallelogram estimates around the corner vertices int count = 0; int altC = c; double minDist = std::numeric_limits <double>::max(); glm::vec3 predNorm(0, 0, 0); / / Predictive localization do / / Loop through the angles attached to the current vertex { if (count>= MAX_PARALLELOGRAMS) break; if (((!isCornerVertexDummy(OV.p(altC)))&&(!isCornerVertexDummy(OV.n(altC))))&&((M[V[OV.p(altC)]]]>0)&&(M[V[OV.n(altC)]]>0))) { / / Calculate distance glm::vec3 gPrev = G[V[OV.p(altC)]]; glm::vec3 gNext = G[V[OV.n(altC)]]; glm::vec3 gCurr = G[V[altC]]; glm::vec3 gPrev_2_gCurr = gCurr - gPrev; double d2_gPrev_2_gCurr = std::sqrt(glm::dot(gPrev_2_gCurr, gPrev_2_gCurr)); glm::vec3 gNext_2_gCurr = gCurr - gNext; double d2_gNext_2_gCurr = std::sqrt(glm::dot(gNext_2_gCurr, gNext_2_gCurr)); if (d2_gPrev_2_gCurr <d2_gNext_2_gCurr) { if (d2_gPrev_2_gCurr <minDist) { minDist = d2_gPrev_2_gCurr; predNorm = Norm[V[OV.p(altC)]]; } } else { if (d2_gNext_2_gCurr <minDist) { minDist = d2_gNext_2_gCurr; predNorm = Norm[V[OV.n(altC)]]; } } ++count; } altC = OV.p(O[OV.p(altC)]); / / Swing around the triangular fan while (altC != c); if (count>0) { / / use parallelogram prediction when possible ;} else / / or fall back to the difference with an available value predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location Process #14 In this process, the V-DMC decoder 300 employs the minimal stretching process used in UV coordinate prediction during texture attribute decoding. This process is illustrated in Tables 3 and 4. The V-DMC decoder 300 can use the same technique, but instead of UV coordinates, it uses normals.

[0200] Process #15 In this process, the V-DMC decoder 300 uses the normal of the previous corner, the normal of the next corner, and the normal of the opposite corner to estimate the vertex of the current corner.

[0201] We found the line ( ) connect to the previous ( ) and the next ( ) We found the line ( ) connect to the previous ( ) and the next ( ) We found the line and Intersecting points ( ).

[0202] Then we estimate the point. normal on ( ) We use Use the parallelogram in procedure #1 to predict points. of .

[0203] We can perform this process for a single parallelogram or by moving around a "fan" and use a multi-parallelogram method for prediction, then take the average.

[0204] Process #16 In this process, the V-DMC decoder 300 can employ not only the upper and lower corners, but also the left and right corners. The V-DMC decoder 300 can perform this process for a single parallelogram or around a "fan" and use a multi-parallelogram method for prediction, then take the average.

[0205] Learning-based prediction, process #17 In this process, the V-DMC decoder 300 may employ a deep learning model and / or use neighboring and previously visited and / or decoded vertices to predict the normal of the current vertex.

[0206] This constitutes a learning-based prediction scheme.

[0207] Geometric-based cross product normal estimation, process #18 This process uses geometry to predict the normal to the current corner vertex.

[0208] The process uses geometry from the previous vertex, the next vertex, and the current vertex to predict the normal of the current vertex. The V-DMC decoder 300 finds the vector between the previous vertex and the current vertex, and finds another vertex between the next vertex and the current vertex. The V-DMC decoder 300 then performs a cross product between these two vertices to obtain the normal of the current vertex.

[0209] The prediction scheme for the normal of the current vertex involves using geometry to compute the vectors between the previous vertex and the current vertex, and between the next vertex and the current vertex. The V-DMC decoder 300 can then perform a cross product to derive the normal.

[0210] const glm::vec3 v12 = G[V[OV.p(c)]] - G[V[c]]; const glm::vec3 v13 = G[V[OV.n(c)]] - G[V[c]]; glm::vec3 normal = glm::cross(v13, v12); Since the normal is predicted based on vectors from geometry, the direction of the normal can lie in two directions based on the cross product of the two vectors (i.e., "cross(v12, v13)" or "cross(v13, v12)"). The V-DMC decoder 300 may additionally transmit the order of the cross product and / or the direction of the normal as additional information for the cross product solution.

[0211] Table 18: Predicting normals using the cross product. Procedure #18 .

[0212] bool prevIsDummy = isCornerVertexDummy(OV.p(c)); glm::vec3 predNorm(0, 0, 0); if (((!isCornerVertexDummy(OV.p(c)))&&(!isCornerVertexDummy(OV.n(c))))&&((M[V[OV.p(c)]]]>0)&&(M[V[OV.n(c)]]>0))) { const glm::vec3 v12 = G[V[OV.p(c)]] - G[V[c]]; const glm::vec3 v13 = G[V[OV.n(c)]] - G[V[c]]; glm::vec3 normal = glm::cross( v13, v12 ); normal = glm::normalize(normal); if ( std::isnan( normal[0] ) ) { break } / / Quantize this normal const glm::vec3 diag = cfg.maxNrm - cfg.minNrm; const float range = std::max( std::max( diag.x, diag.y ), diag.z ); const int32_t maxNormalQuantizedValue = (1u< <static_cast<uint32_t> (qn)) - 1; for (int c = 0; c<3; c++) { predNorm[c] = static_cast<uint32_t> (std::floor( ( ( normal[c] -cfg.minNrm[c]) / range ) * maxNormalQuantizedValue + 0.5f ) ); } } else { predNorm = prevIsDummy ? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; } oNormals.push_back(Norm[V[c]] - predNorm); / / Stored residual = Location - Predicted location The average of the cross products of the normal estimate, process #19 Procedure #19 indicates a potential modification to procedure #18.

[0213] In procedure #19, the V-DMC decoder 300 makes multiple predictions from all available triangles around the current vertex, instead of making a single prediction using the cross product as in procedure #18. The V-DMC decoder 300 then averages these predictions. Since the cross product is computed during geometric encoding in the EdgeBreaker, this procedure can be referred to as "CrossproductEB".

[0214] Since the normal is predicted based on vectors from geometry, the direction of the normal can lie in two directions based on the cross product of the two vectors (i.e., "cross(v12, v13)" or "cross(v13, v12)"). The V-DMC decoder 300 may additionally transmit the order of the cross product and / or the direction of the normal as additional information for the cross product solution.

[0215] Table 19: Normal prediction using multiple cross products. Procedure #19 .

[0216] bool prevIsDummy = isCornerVertexDummy(OV.p(c)); glm::vec3 predNorm(0, 0, 0); if (((!isCornerVertexDummy(OV.p(c)))&&(!isCornerVertexDummy(OV.n(c))))&&((M[V[OV.p(c)]]>0)&&(M[V[OV.n(c)]]>0))) { const glm::vec3 v12 = G[V[OV.p(c)]] - G[V[c]]; const glm::vec3 v13 = G[V[OV.n(c)]] - G[V[c]]; glm::vec3 normal = glm::cross( v13, v12 ); normal = glm::normalize(normal); if ( std::isnan( normal[0] ) ) { break; } / / Quantize the normal vector const glm::vec3 diag = cfg.maxNrm - cfg.minNrm; const float range = std::max( std::max( diag.x, diag.y ), diag.z ); const int32_t maxNormalQuantizedValue = (1u<<static_cast<uint32_t>(qn)) - 1; for (int c = 0; c<3; c++) { predNorm[c] = static_cast<uint32_t>(std::floor( ( ( normal[c] -cfg.minNrm[c]) / range ) * maxNormalQuantizedValue + 0.5f ) ); } } else { predNorm = prevIsDummy? Norm[V[OV.n(c)]] : Norm[V[OV.p(c)]]; } oNormals.push_back(Norm[V[c]] - predNorm); / / Store residual = location - predicted location The cross product after geometric compression. Procedure #20 In this process, the V-DMC encoder 200 can encode the geometry first, rather than predicting and encoding normals during EdgeBreaker encoding of the geometry. Once the geometry is encoded for all vertices, the V-DMC encoder 200 can use that geometry to compute the normal using the cross product of all triangles surrounding the vertex of the current angle. Since the geometry has been encoded, the V-DMC encoder 200 can use all surrounding triangles to predict the normal of the current vertex. This yields the best results because all geometry can be used to predict the normal, not just the decoded / encoded geometry. This process can be referred to as "Crossproduct Global".

[0217] Since the normal is predicted based on vectors from geometry, the direction of the normal can lie in two directions based on the cross product of the two vectors (i.e., "cross(v12, v13)" or "cross(v13, v12)"). The V-DMC encoder 200 may additionally transmit the order of the cross product and / or the direction of the normal as additional information for the cross product solution.

[0218] Table 20: Normal predictions using cross product after geometric compression. "Crossproduct Global", Procedure #20 / / Predicting normals from geometry eb::Model ebModelNorm; ModelConverter::convertCTMeshToModel(_ovTable, ebModelNorm); ebModelNorm.normals.clear(); ebModelNorm.computeVertexNormals(true, true); / / Quantize normals const glm::vec3 diag = cfg.maxNrm - cfg.minNrm; const float range = std::max( std::max( diag.x, diag.y ), diag.z ); const int32_t maxNormalQuantizedValue = (1u< <static_cast<uint32_t> (qn) ) - 1; for ( size_t i = 0; i<ebModelNorm.normals.size() / 3; i++ ) { for ( glm::vec3::length_type c = 0; c<3; ++c ) { uint32_t nrm = static_cast<uint32_t>( std::floor( ( ( ebModelNorm.normals[i * 3 + c] - cfg.minNrm[c]) / range ) * maxNormalQuantizedValue + 0.5f ) ); ebModelNorm.normals[i * 3 + c] = static_cast <float>( nrm ); } } CTMesh _ovTablePredNorm; ModelConverter::convertModelToCTMesh(ebModelNorm, _ovTablePredNorm,cfg.deduplicate); / / Store the predicted normal const auto&Norm = _ovTable.normals; const auto&V = _ovTable.V; for (auto i = 0; i <cTraverse.size(); ++i) { int c = cTraverse[i]; oNormals.push_back(Norm[V[c]] - _ovTablePredNorm.normals[V[c]]); Normal prediction using the projections of the relative vertex and the current vertex. Procedure #21 Figures 16A to 16C An example of normal prediction is shown. Figure 16A An example is shown that uses the previous vertex (P), the next vertex (N), and the relative vertex (O) to predict the current vertex C. Figure 16A The diagram shows the different vertices that can be used to predict the normal at the current vertex (C). Typically (as shown in the previous process), the V-DMC decoder 300 can use a combination of vertices P, N, and O to predict the normal at C. The V-DMC decoder 300 can also use the normals and / or geometry of neighboring vertices (P, N, O) to predict the normal at the current vertex.

[0219] However, in procedure #21, the V-DMC decoder 300 can use arbitrary (or dummy) X and Y, such as Figure 16B and Figure 16C As shown. X and Y are vertices generated by the projections of vertices C and O onto the PN plane.

[0220] The V-DMC decoder 300 can take a distance-weighted average of the normals at P and N to obtain the normals at X and Y.

[0221] Then, the V-DMC decoder 300 can use the combination of the normals at X, Y and O to obtain the normal at C.

[0222] A combination of such predictions could be: The equation is similar to the multi-parallelogram in process #1. However, the V-DMC decoder 300 can use X and Y vertices instead of P and N vertices.

[0223] Similarly, the V-DMC decoder 300 can perform this process for a triangle or around a "fan" at a vertex and make multiple predictions from different triangles / parallelograms, then take their average.

[0224] Other processes We can use different combinations of vertices surrounding the current vertex and their normals, geometry, and / or UV coordinates to predict the normal of the current vertex.

[0225] The vertices and / or corners surrounding the current vertex can be used. The properties of these neighbors can be used to predict the normal of the current vertex. These properties can include geometry, UV coordinates, normals, etc.

[0226] Updates to the cross-product-based process.

[0227] V-DMC's v7.0 TMM introduces the following potential improvements to cross-product-based normal prediction: ● In v7.0 TMM V-DMC, normals are predicted after the geometry is encoded / decoded. This means that the geometry of all vertices can be used to predict normals.

[0228] ● All available triangles around the current vertex are used to predict the normal of the current vertex.

[0229] ● Walk around the fan around the current vertex ( Figure 15 ), and uses the geometry of vertices c, cn, and cp (current vertex, next vertex, and previous vertex) to predict face normals.

[0230] ● Take the average of all face normals (within that fan) around the current vertex.

[0231] ● Note that, unlike other prediction schemes, cross-product-based prediction uses the geometry of the vertex to predict the normal. In contrast, previous methods used previously encoded / accessed normals to predict the normal of the current vertex.

[0232] ● If the geometry of all triangles around the current vertex is unavailable, fall back to differential decoding. In this case, the normal from the previous or next vertex will be used for prediction.

[0233] ● If the previous and next vertices are not available, the vertex may be located on the boundary and the boundary-adjacent vertices can be used for prediction.

[0234] ● If none of these conditions are met, the vertex can be the first vertex of the mesh, and therefore the V-DMC decoder 300 can store the global value of the vertex instead of predicting it.

[0235] The specific implementation code is shown in Table 21.

[0236] Table 21: Normal Prediction: Improvements to Cross Product-Based Normal Prediction in v7.0 const auto&ov = _ovTable; const auto&V = ov.V; const auto&O = ov.O; const auto&Norm = ov.normals; const auto&G = ov.positions; const auto&v = ov.v(c); / / Has the vertex been predicted? if (MV[v]>0) return; / / Mark this vertex MV[v] = 1; glm::vec3 predNorm(0, 0, 0); / / Predict normals int count = 0; / / The number of valid parallelograms found int altC = c; / / Loop through the angles attached to the current vertex / / Swing to the right around the fan int nextC = ov.n(O[ov.n(altC)]); while (nextC>= 0&&nextC != c) { altC = nextC; nextC = ov.n(O[ov.n(altC)]); }; bool isBoundary = (nextC != c); / / Now positioned at the rightmost corner of the shared v, / / Turn left and assess possible predictions const int startC = altC; do { const auto&prevV = ov.v(ov.p(altC)); const auto&nextV = ov.v(ov.n(altC)); / *if ((prevV>-1&&nextV>-1)&& ((MV[prevV]>0)&&(MV[nextV]>0)))* / if (prevV>-1&&nextV>-1) { const glm::vec3 v12 = G[prevV] - G[v]; const glm::vec3 v13 = G[nextV] - G[v]; predNorm += glm::cross( v13, v12 ); ++count; } altC = ov.p(O[ov.p(altC)]); / / Swing around the triangular fan } while (altC>= 0&&altC != startC); / / Incomplete fan or complete rotation / / 1. Using cross product if (count>0) { / / Regularize the prediction predNorm = glm::normalize( predNorm ); if (!std::isnan( predNorm[0] ) ) { / / Quantize this normal const glm::vec3 minNrm = {-1.0, -1.0, -1.0}; const glm::vec3 maxNrm = {1.0, 1.0, 1.0}; const glm::vec3 diag = maxNrm - minNrm; const float range = std::max( std::max( diag.x,diag.y ), diag.z ); const int32_t maxNormalQuantizedValue = ( 1u<<static_cast<uint32_t>( qn ) ) - 1; for (int c = 0; c<3; c++) { predNorm[c] = static_cast <float>(std::floor( ( ( predNorm[c] - minNrm[c]) / range ) * maxNormalQuantizedValue + 0.5f ) ); } oNormals.push_back(Norm[v] - predNorm); oNrmFine.push_back(true); return; } } / / 2. Or roll back to the difference with an available value. const auto&c_p_v = ov.v(ov.p(c)); const auto&c_n_v = ov.v(ov.n(c)); if (c_p_v>-1&&MV[c_p_v]>-1) { oNormals.push_back(Norm[v] - Norm[c_p_v]); oNrmFine.push_back(false); return; } if (c_n_v>-1&&MV[c_n_v]>-1) { oNormals.push_back(Norm[v] - Norm[c_n_v]); oNrmFine.push_back(false); return; } / / 3. If located on the boundary / / Then the difference from the previous vertex on the boundary can be used. if (isBoundary) { const auto b = ov.p(startC); / / b is located on the boundary const auto b_v = ov.v(b); auto marked = MV[b_v]; if (marked>-1) { oNormals.push_back(Norm[v] - Norm[b_v]); oNrmFine.push_back(false); return; } } / / 4. No other options osNormals.push_back(Norm[v]); / / Global value (it is the starting point, and the push is performed in a separate table) Supplement to V-DMC syntax These are the syntax elements in the V3C parameters (see U.S. Provisional Patent Application No. 63 / 621,478) and the underlying mesh syntax and semantics used in normal coding.

[0237] The syntax for implementing the techniques of this disclosure is in the delimiter <**> and< / **> Indicates between.

[0238] H.8.1.3.1.1 General Basic Mesh Sequence Parameter Set (RBSP) Syntax

[0239] Table H-2: (See U.S. Provisional Patent Application No. 63 / 621,478)

[0240] 8.3.4.5 Attribute Information Syntax

[0241] Basic Mesh Encoder Syntax : I.8.3.2 Grid Decoding Header Syntax

[0242] I.8.3.5 Mesh Attribute Encoding Parameter Syntax

[0243] I.8.3.5 Mesh Attribute Dequantization Parameter Syntax

[0244] old watch : Table I-10: Mesh attribute prediction process for MESH_ATTR_NORMAL type attributes

[0245] Updated submission form The next paragraph describes four prediction schemes, which are shown below: Procedure #2: MESH_NORMAL_DELTA Procedure #1: MESH_NORMAL_MPARA Procedure #19: MESH_NORMAL_CROSSEB Procedure #20: MESH_NORMAL_CROSSGL These forecasting schemes have been explained in previous paragraphs. The updated table is shown above: Table I-10: Mesh attribute prediction process for MESH_ATTR_NORMAL type attributes

[0246] Another updated proposal form Procedure #2: MESH_NORMAL_DELTA Procedure #1: MESH_NORMAL_MPARA Procedure #23: MESH_NORMAL_CROSS These forecasting schemes have been explained in previous paragraphs. The updated table is shown above: Table I-10: Mesh Attribute Prediction Methods for MESH_ATTR_NORMAL Type Attributes

[0247] Framework changes / flexibility The following changes / flexibility can be added to this framework: In the current implementation of the EdgeBreaker static mesh encoder (TMM v6.0), connectivity is decoded first, and then the attributes (geometric, UV coordinates) of each vertex are decoded for each vertex. However, by serializing or parallelizing the attribute encoding process, the system can have more flexibility and therefore better results. The V-DMC decoder 300 can (1) decode one attribute at a time for all vertices before moving to the next attribute, or (2) decode all attributes for one vertex before moving to the next vertex. The potential advantage of the first process (1) would be to give us better compression performance. The potential advantage of the second process (2) would be to give us computational parallelism and potentially better hardware implementations that could improve encoding / decoding time.

[0248] In TMM v7.0, attributes are decoded sequentially. First, the topology is decoded, then each attribute is decoded one by one. This was described above.

[0249] The V-DMC decoder 300 can employ multiple normal prediction processes instead of a single normal prediction process, and is designed to predict which normal prediction process is being used at the decoder.

[0250] We can also send additional bits to inform the decoder which normal prediction process is being used. This provides the flexibility in the software to have multiple normal prediction processes and to use the one best suited for the current sequence, current mesh, current submesh, current connected component, or some other vertex cluster.

[0251] The encoder-decoder architecture disclosed herein may include quantization and dequantization. The technique may also include quantization refinement for normal encoding. Aspects of quantization refinement are explained in "[V-DMC][EE4.4] Report on base mesh quantization refinement" by Jun Tian, ​​Xiaozhong Xu, and Shan Liu (Tencent) in ISO / IEC JTC1 / SC29 / WG7, m66074, published online in January 2024, which is incorporated herein by reference.

[0252] Note that cross-product-based normal estimation can be performed in either direction. The V-DMC decoder 300 may have signals / parameters received from the V-DMC encoder 200 to signal the direction of the normal. This signal can be at the sequence level, frame level, submesh level, connected component level, vertex / triangle cluster level, or even just the vertex level (one per vertex). The V-DMC decoder 300 can also predict the direction of the normal from previously encoded / decoded normals.

[0253] The aspects of signaling notification and frame functionality will now be described. For normal prediction itself, additional information from the decoder side is typically not required. However, flexibility can be added to the system, which may involve adding additional overhead and requiring the transmission of additional signaling bits.

[0254] Such signaling can occur at the sequence level, frame level, submesh level, connected component level, any other smaller vertex cluster, or even per-vertex level.

[0255] Additional signals that can be transmitted to the V-DMC decoder 300 may include: ● Indicate which normal was used to predict the signal.

[0256] ● Inform the level of signal at which the prediction is being applied.

[0257] ● If multiple types of predictions exist, the V-DMC encoder 200 can signal which prediction applies to which part of the data. This includes a signal indicating which direction the normal faces the surface. The perpendicular / normal to the surface may be in two directions. However, the normal will only face one direction. This signal can potentially help signal that direction in order to predict the normal.

[0258] Figure 17 This is a flowchart illustrating an example process for encoding a mesh. Although regarding the V-DMC encoder 200 ( Figure 1 and Figure 2 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 17 The process is similar to that of [the other process].

[0259] exist Figure 17 In the example, the V-DMC encoder 200 receives an input mesh (1702). The V-DMC encoder 200 determines a base mesh based on the input mesh (1704). The V-DMC encoder 200 determines a set of displacement vectors based on the input mesh and the base mesh (1706). The V-DMC encoder 200 outputs an encoded bitstream that includes an encoded representation of the base mesh and an encoded representation of the displacement vectors (1708). The V-DMC encoder 200 may additionally determine attribute values ​​from the input mesh and include the encoded representation of the attribute value vector in the encoded bitstream.

[0260] Figure 18 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data. Although regarding the V-DMC decoder 300 ( Figure 1 and Figure 3 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 18 The process is similar to that of [the other process].

[0261] exist Figure 18 In the example, the V-DMC decoder 300 determines a base mesh based on the encoded mesh data (1802). The V-DMC decoder 300 determines one or more displacement vectors based on the encoded mesh data (1804). The V-DMC decoder 300 uses one or more displacement vectors to deform the base mesh (1806). For example, the base mesh may have a first set of vertices, and the V-DMC decoder 300 may subdivide the base mesh to determine an additional set of vertices. To deform the base mesh, the V-DMC decoder 300 may modify the positions of this additional set of vertices based on one or more displacement vectors. The V-DMC decoder 300 outputs a decoded mesh based on the deformed mesh (1808). The V-DMC decoder 300 may, for example, output the decoded mesh for storage, transmission, or display.

[0262] Figure 19 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data. Although regarding the V-DMC decoder 300 ( Figure 1 and Figure 3 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 19 The process is similar to that of [the other process].

[0263] The V-DMC decoder 300 receives the syntax elements of the grid, wherein the first value of the syntax element indicates the vertices of the grid to be predicted using multi-parallelogram prediction and the second value of the syntax element indicates the vertices of the grid to be predicted using cross-product prediction (1902).

[0264] The V-DMC decoder 300 selects either multi-parallelogram prediction or cross-product prediction as the mesh selection prediction process based on syntax elements (1904). In response to determining that a first set of decoded normals exists for the first vertex, the V-DMC decoder 300 uses the selected prediction process to predict the normal of the first vertex (1906). If the selected prediction process is multi-parallelogram prediction, then in order to predict the normal of the first vertex using the selected prediction process, the V-DMC decoder 300 can be configured to determine the predicted normal value of the first vertex based on the previous normal value plus the next normal value minus the relative normal value. If the selected prediction process is cross-product prediction, then in order to predict the normal of the first vertex using the selected prediction process, the V-DMC decoder 300 can be configured to: determine a first vector between the previous vertex and the first vertex; determine a second vector between the next vertex and the first vertex; and determine the cross product of the first vector and the second vector.

[0265] In response to determining that no second set of decoded normals is available for the second vertex, the V-DMC decoder 300 uses a differential prediction process to predict the normal of the second vertex (1908). To predict the normal of the second vertex using the differential prediction process, the V-DMC decoder 300 can be configured to: identify a single vertex on the same triangle as the second vertex; set the predicted normal value of the second vertex to be equal to the vertex value of the normal of the single vertex; receive the difference; and add the difference to the predicted normal value of the second vertex to determine the normal of the second vertex.

[0266] The V-DMC decoder 300 outputs a decoded version of the mesh based on the normals of the first vertex and the second vertex (1910). For example, the normal of the first vertex may be a first vector perpendicular to the decoded version of the mesh at the first vertex and the normal of the second vertex may be a second vector perpendicular to the decoded version of the mesh at the second vertex.

[0267] The following numbered clauses illustrate one or more aspects of the devices and technologies described in this disclosure.

[0268] Clause 1. An apparatus for processing mesh data, the apparatus comprising: a memory; and processing circuitry coupled to the memory and configured to: receive a syntax element of a mesh, wherein a first value of the syntax element indicates that vertices of the mesh should be predicted using multiparallelogram prediction and a second value of the syntax element indicates that the vertices of the mesh should be predicted using cross product prediction; a prediction process that selects either the multiparallelogram prediction or the cross product prediction as the mesh selection based on the syntax element; predicting a normal of the first vertex using the selected prediction process in response to determining that a first vertex has a first set of available decoded normals; predicting a normal of the second vertex using a differential prediction process in response to determining that a second vertex does not have a second set of available decoded normals; and outputting a decoded version of the mesh based on the normals of the first vertex and the normals of the second vertex.

[0269] Clause 2. The apparatus according to Clause 1, wherein, in order to predict the normal of the second vertex using the differential prediction process, the processing circuitry is configured to: identify a single vertex on the same triangle as the second vertex; set the predicted normal value of the second vertex to be equal to the vertex value of the normal of the single vertex; receive a difference; and add the difference to the predicted normal value of the second vertex to determine the normal of the second vertex.

[0270] Clause 3. The device according to Clause 1, wherein the selected prediction process includes multi-parallelogram prediction and wherein, in order to predict the normal of the first vertex using the selected prediction process, the processing circuitry is configured to determine the predicted normal value of the first vertex based on the previous normal value plus the next normal value minus the relative normal value.

[0271] Clause 4. The device according to Clause 1, wherein the selected prediction process includes cross product prediction and wherein the normal of the first vertex is predicted using the selected prediction process, the processing circuitry being configured to: determine a first vector between the previous vertex and the first vertex; determine a second vector between the next vertex and the first vertex; and determine the cross product of the first vector and the second vector.

[0272] Clause 5. The device according to Clause 1, the device further comprising: receiving the complete normal value of the third vertex in the mesh data in response to determining that there is no available third set of decoded normals for the third vertex.

[0273] Clause 6. The device according to Clause 1, wherein the normal of the first vertex comprises a first vector perpendicular to the decoded version of the mesh at the first vertex and the normal of the second vertex comprises a second vector perpendicular to the decoded version of the mesh at the second vertex.

[0274] Clause 7. The device according to Clause 1, wherein the normal of the first vertex and the normal of the second vertex are losslessly encoded.

[0275] Clause 8. The device according to Clause 1, the device further comprising a display for presenting an image based on the decoded version of the grid.

[0276] Clause 9. A method for processing mesh data, the method comprising: receiving a syntax element of the mesh, wherein a first value of the syntax element indicates that vertices of the mesh are to be predicted using multiparallelogram prediction and a second value of the syntax element indicates that the vertices of the mesh are to be predicted using cross product prediction; a prediction process for selecting either the multiparallelogram prediction or the cross product prediction as the mesh based on the syntax element; predicting a normal of the first vertex using the selected prediction process in response to determining that a first vertex has a first set of available decoded normals; predicting a normal of the second vertex using a differential prediction process in response to determining that a second vertex does not have a second set of available decoded normals; and outputting a decoded version of the mesh based on the normals of the first vertex and the normals of the second vertex.

[0277] Clause 10. The method according to Clause 9, wherein predicting the normal of the second vertex using the differential prediction process comprises: identifying a single vertex on the same triangle as the second vertex; setting the predicted normal value of the second vertex to be equal to the vertex value of the normal of the single vertex; receiving a difference; and adding the difference to the predicted normal value of the second vertex to determine the normal of the second vertex.

[0278] Clause 11. The method according to Clause 9, wherein the selected prediction process includes multi-parallelogram prediction and wherein predicting the normal of the first vertex using the selected prediction process includes: determining the predicted normal value of the first vertex based on the previous normal value plus the next normal value minus the relative normal value.

[0279] Clause 12. The method according to Clause 9, wherein the selected prediction process includes cross product prediction and wherein predicting the normal of the first vertex using the selected prediction process includes: determining a first vector between the previous vertex and the first vertex; determining a second vector between the next vertex and the first vertex; and determining the cross product of the first vector and the second vector.

[0280] Clause 13. The method according to Clause 9, the method further comprising: receiving the complete normal value of the third vertex in the mesh data in response to determining that there is no available third set of decoded normals for the third vertex.

[0281] Clause 14. The method according to Clause 9, wherein the normal of the first vertex comprises a first vector perpendicular to the decoded version of the mesh at the first vertex and the normal of the second vertex comprises a second vector perpendicular to the decoded version of the mesh at the second vertex.

[0282] Clause 15. The method according to Clause 9, wherein the normal of the first vertex and the normal of the second vertex are losslessly encoded.

[0283] Clause 16. A computer-readable storage medium having instructions thereon, which, when executed, cause one or more processors to: receive a syntax element of a mesh, wherein a first value of the syntax element indicates that vertices of the mesh should be predicted using multiparallelogram prediction and a second value of the syntax element indicates that the vertices of the mesh should be predicted using cross product prediction; a prediction process that selects either the multiparallelogram prediction or the cross product prediction as the mesh based on the syntax element; predicting a normal of the first vertex using the selected prediction process in response to determining that a first vertex has a first set of available decoded normals; predicting a normal of the second vertex using a differential prediction process in response to determining that a second vertex does not have a second set of available decoded normals; and outputting a decoded version of the mesh based on the normals of the first vertex and the normals of the second vertex.

[0284] Clause 17. The computer-readable storage medium according to Clause 16, wherein, in order to predict the normal of the second vertex using the differential prediction process, the one or more processors are configured to: identify a single vertex on the same triangle as the second vertex; set the predicted normal value of the second vertex to be equal to the vertex value of the normal of the single vertex; receive a difference; and add the difference to the predicted normal value of the second vertex to determine the normal of the second vertex.

[0285] Clause 18. The computer-readable storage medium according to Clause 16, wherein the selected prediction process includes multi-parallelogram prediction and wherein, in order to predict the normal of the first vertex using the selected prediction process, the processing circuitry is configured to determine the predicted normal value of the first vertex based on the previous normal value plus the next normal value minus the relative normal value.

[0286] Clause 19. The computer-readable storage medium of Clause 16, wherein the selected prediction process includes cross product prediction and wherein the normal of the first vertex is predicted using the selected prediction process, wherein the one or more processors are configured to: determine a first vector between a previous vertex and the first vertex; determine a second vector between a next vertex and the first vertex; and determine the cross product of the first vector and the second vector.

[0287] Clause 20. The computer-readable storage medium according to Clause 16, the computer-readable storage medium further comprising: receiving the complete normal value of the third vertex in response to determining that there is no available third set of decoded normals for the third vertex.

[0288] Clause 21. A method for processing grid data, the method comprising: any technique or combination of techniques described in this disclosure.

[0289] Clause 22. The method according to any one of Clause 21, the method further comprising: generating the grid data.

[0290] Clause 23. An apparatus for processing grid data, the apparatus comprising: a memory configured to store the grid data; and one or more processors coupled to the memory, the one or more processors being implemented in a circuit and configured to perform any or a combination of techniques described in this disclosure.

[0291] Clause 24. The device as described in Clause 23, wherein the device includes a decoder.

[0292] Clause 25. The device as described in Clause 23, wherein said device includes an encoder.

[0293] Clause 26. The apparatus according to any one of Clauses 23 to 24, the apparatus further comprising means for generating the grid data.

[0294] Clause 27. The device according to any one of Clauses 23 to 26, the device further comprising a display for presenting an image based on the grid data.

[0295] Clause 28. A computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to perform any technology or combination of technologies described in this disclosure.

[0296] It should be recognized that, based on the examples, certain actions or events of any technique described herein may be performed in a different sequence, and may be added, combined, or omitted entirely (e.g., not all actions or events described are necessary for implementing the technique). Furthermore, in some examples, actions or events may be performed concurrently (e.g., through multithreading, interrupt handling, or multiple processors) rather than sequentially.

[0297] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0298] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead refer to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs utilize lasers to optically reproduce data. The combinations described above should also be included within the scope of computer-readable media.

[0299] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuit" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0300] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but implementation by different hardware units is not necessarily required. Rather, as described above, various units may be combined in a codec hardware unit, or various units may be provided by a collection of interoperable hardware units (including one or more processors as described above) combined with appropriate software and / or firmware.

[0301] Various examples have been described. These and other examples are within the scope of the following claims.< / float> < / float> < / double> < / double> < / double> < / float> < / int>

Claims

1. An apparatus for processing grid data, the apparatus comprising: Memory; and Processing circuitry, coupled to the memory and configured to: Receive a syntax element for a mesh, wherein a first value of the syntax element indicates that the vertices of the mesh should be predicted using multi-parallelogram prediction and a second value of the syntax element indicates that the vertices of the mesh should be predicted using cross-product prediction; The prediction process that selects either the multi-parallelogram prediction or the cross product prediction as the grid selection based on the syntax elements; In response to determining that a first set of decoded normals exists for a first vertex, the selected prediction process is used to predict the normals of the first vertex. In response to the determination that there is no available second set of decoded normals for the second vertex, the differential prediction process is used to predict the normals of the second vertex; as well as The decoded version of the mesh is output based on the normal of the first vertex and the normal of the second vertex.

2. The device of claim 1, wherein, in order to predict the normal of the second vertex using the differential prediction process, the processing circuit is configured to: Identify a single vertex on the same triangle as the second vertex; Set the predicted normal value of the second vertex to be equal to the vertex value of the normal of the single vertex; Receive the difference; as well as The difference is added to the predicted normal value of the second vertex to determine the normal of the second vertex.

3. The device of claim 1, wherein the selected prediction process includes multi-parallelogram prediction and wherein, in order to predict the normal of the first vertex using the selected prediction process, the processing circuitry is configured to: The predicted normal value of the first vertex is determined by adding the previous normal value to the next normal value and subtracting the relative normal value.

4. The device of claim 1, wherein the selected prediction process includes cross product prediction and wherein the normal of the first vertex is predicted using the selected prediction process, the processing circuitry being configured to: Determine the first vector between the previous vertex and the first vertex; Determine the second vector between the next vertex and the first vertex; and Determine the cross product of the first vector and the second vector.

5. The device according to claim 1, further comprising: In response to determining that there is no available third set of decoded normals for the third vertex, the complete normal value of the third vertex in the mesh data is received.

6. The device of claim 1, wherein the normal of the first vertex comprises a first vector perpendicular to the decoded version of the mesh at the first vertex and the normal of the second vertex comprises a second vector perpendicular to the decoded version of the mesh at the second vertex.

7. The device of claim 1, wherein the normal of the first vertex and the normal of the second vertex are losslessly encoded.

8. The device of claim 1, further comprising a display for presenting an image based on the decoded version of the grid.

9. A method for processing grid data, the method comprising: Receive a syntax element for a mesh, wherein a first value of the syntax element indicates that the vertices of the mesh should be predicted using multi-parallelogram prediction and a second value of the syntax element indicates that the vertices of the mesh should be predicted using cross-product prediction; The prediction process that selects either the multi-parallelogram prediction or the cross product prediction as the grid selection based on the syntax elements; In response to determining that a first set of decoded normals exists for a first vertex, the selected prediction process is used to predict the normals of the first vertex. In response to the determination that there is no available second set of decoded normals for the second vertex, the differential prediction process is used to predict the normals of the second vertex; as well as The decoded version of the mesh is output based on the normal of the first vertex and the normal of the second vertex.

10. The method of claim 9, wherein predicting the normal of the second vertex using the differential prediction process comprises: Identify a single vertex on the same triangle as the second vertex; Set the predicted normal value of the second vertex to be equal to the vertex value of the normal of the single vertex; Receive the difference; as well as The difference is added to the predicted normal value of the second vertex to determine the normal of the second vertex.

11. The method of claim 9, wherein the selected prediction process includes multi-parallelogram prediction and wherein predicting the normal of the first vertex using the selected prediction process includes: The predicted normal value of the first vertex is determined by adding the previous normal value to the next normal value and subtracting the relative normal value.

12. The method of claim 9, wherein the selected prediction process includes cross product prediction and wherein predicting the normal of the first vertex using the selected prediction process includes: Determine the first vector between the previous vertex and the first vertex; Determine the second vector between the next vertex and the first vertex; as well as Determine the cross product of the first vector and the second vector.

13. The method according to claim 9, further comprising: In response to determining that there is no available third set of decoded normals for the third vertex, the complete normal value of the third vertex in the mesh data is received.

14. The method of claim 9, wherein the normal of the first vertex comprises a first vector perpendicular to the decoded version of the mesh at the first vertex and the normal of the second vertex comprises a second vector perpendicular to the decoded version of the mesh at the second vertex.

15. The method of claim 9, wherein the normal of the first vertex and the normal of the second vertex are losslessly encoded.

16. A computer-readable storage medium having instructions stored thereon, the instructions causing one or more processors to: Receive a syntax element for a mesh, wherein a first value of the syntax element indicates that the vertices of the mesh should be predicted using multi-parallelogram prediction and a second value of the syntax element indicates that the vertices of the mesh should be predicted using cross-product prediction; The prediction process that selects either the multi-parallelogram prediction or the cross product prediction as the grid selection based on the syntax elements; In response to determining that a first set of decoded normals exists for a first vertex, the selected prediction process is used to predict the normals of the first vertex. In response to the determination that there is no available second set of decoded normals for the second vertex, the differential prediction process is used to predict the normals of the second vertex; as well as The decoded version of the mesh is output based on the normal of the first vertex and the normal of the second vertex.

17. The computer-readable storage medium of claim 16, wherein, in order to predict the normal of the second vertex using the differential prediction process, the one or more processors are configured to: Identify a single vertex on the same triangle as the second vertex; Set the predicted normal value of the second vertex to be equal to the vertex value of the normal of the single vertex; Receive the difference; as well as The difference is added to the predicted normal value of the second vertex to determine the normal of the second vertex.

18. The computer-readable storage medium of claim 16, wherein the selected prediction process includes multi-parallelogram prediction and wherein, in order to predict the normal of the first vertex using the selected prediction process, the one or more processors are configured to: The predicted normal value of the first vertex is determined by adding the previous normal value to the next normal value and subtracting the relative normal value.

19. The computer-readable storage medium of claim 16, wherein the selected prediction process includes cross product prediction and wherein the normal of the first vertex is predicted using the selected prediction process, wherein the one or more processors are configured to: Determine the first vector between the previous vertex and the first vertex; Determine the second vector between the next vertex and the first vertex; and Determine the cross product of the first vector and the second vector.

20. The computer-readable storage medium of claim 16, further comprising: In response to determining that there is no available third set of decoded normals for the third vertex, the complete normal value of the third vertex is received.

Images