V-dmc signaling improvement in displacement sub-bitstream for wavelet coefficient inter-frame prediction with fixed-point dequantization

Abstract

An apparatus for decoding encoded dynamic mesh data: determining a set of quantized integer coefficient values for displacement vectors of the encoded dynamic mesh data; determining a quantization parameter based on one or more syntax elements included in a displacement sub-bitstream of the encoded dynamic mesh data; inverse quantizing the set of quantized integer coefficient values based on the quantization parameter to determine a set of fixed-point dequantized coefficient values; determining a set of fixed-point transform coefficient values based on the set of fixed-point dequantized coefficient values; converting the set of fixed-point transform coefficient values to a set of floating-point transform coefficient values; inverse transforming the set of floating-point transform coefficient values to determine a set of reconstructed displacement vectors; and determining a reconstructed morphed mesh based on the set of reconstructed displacement vectors.

Description

[0001] This application claims priority to U.S. Patent Application No. 19 / 085,875, filed March 20, 2025, and U.S. Provisional Patent Application No. 63 / 569,562, filed March 25, 2024, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 19 / 085,875, filed March 20, 2025, claims the benefit of U.S. Provisional Patent Application No. 63 / 569,562, filed March 25, 2024. Technical Field

[0002] This disclosure relates to video-based dynamic mesh decoding. Background Technology

[0003] Meshes can be used to represent physical content in three-dimensional space. Meshes are used in a wide variety of situations. For example, meshes can be used in the context of representing the physical content of an environment to locate virtual objects in extended reality applications such as augmented reality (AR), virtual reality (VR), or mixed reality (MR). Mesh compression is the process of encoding and decoding meshes. Encoding meshes reduces the amount of data required to store and transmit them. Summary of the Invention

[0004] This disclosure generally relates to video-based dynamic trellis decoding. To determine the displacement vector, the trellis decoder performs inter-frame prediction on the wavelet coefficients after inverse quantization, as described in more detail below. The trellis decoder adds the inverse quantized coefficients (residuals) to the inter-frame prediction values ​​stored in the frame buffer. The advantage of this technique is that the correlation between wavelet coefficients in adjacent frames is generally greater than the correlation between corresponding quantized coefficients, resulting in additional decoding efficiency gains.

[0005] To reconstruct displacement vector wavelet coefficients according to the techniques described herein, the inverse quantization process uses parameters from the bitstream, such as displacement vector dimension, quantization parameter (QP), number of levels of detail (LoD), number of vertices per LoD, etc. In V-DMC, some of these parameters are signaled as part of the atlas metadata sub-bitstream. However, the arithmetically decoded (AC) displacement sub-bitstream does not contain the parameters required by the inverse quantization process to reconstruct the displacement vector wavelet coefficients from this sub-bitstream without relying on the atlas metadata sub-bitstream. By determining the quantization parameters based on one or more syntax elements included in the displacement sub-bitstream of the encoded dynamic mesh data, the techniques disclosed herein enable self-contained reconstruction of displacement vectors without relying on other sub-bitstreams.

[0006] According to an example of this disclosure, an apparatus for decoding encoded dynamic mesh data includes: one or more memories; and one or more processors implemented in a circuit and communicating with the one or more memories, the one or more processors being configured to: determine a set of quantized integer coefficient values ​​for the displacement vectors of the encoded dynamic mesh data; determine quantization parameters based on one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data; inverse quantize the set of quantized integer coefficient values ​​based on the quantization parameters to determine a set of fixed-point dequantization coefficient values; determine a set of fixed-point transform coefficient values ​​based on the set of fixed-point dequantization coefficient values; convert the set of fixed-point transform coefficient values ​​to a set of floating-point transform coefficient values; inverse transform the set of floating-point transform coefficient values ​​to determine a set of reconstructed displacement vectors; and determine a reconstructed deformable mesh based on the set of reconstructed displacement vectors.

[0007] According to another example of this disclosure, a method for decoding encoded dynamic mesh data includes: determining a set of quantized integer coefficient values ​​for the displacement vectors of the encoded dynamic mesh data; determining quantization parameters based on one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data; inverse quantizing the set of quantized integer coefficient values ​​based on the quantization parameters to determine a set of fixed-point dequantization coefficient values; determining a set of fixed-point transform coefficient values ​​based on the set of fixed-point dequantization coefficient values; converting the set of fixed-point transform coefficient values ​​into a set of floating-point transform coefficient values; inverse transforming the set of floating-point transform coefficient values ​​to determine a set of reconstructed displacement vectors; and determining a reconstructed deformable mesh based on the set of reconstructed displacement vectors.

[0008] According to another example of this disclosure, an apparatus for encoding dynamic mesh data includes: one or more memories; and one or more processors implemented in a circuit and communicating with the one or more memories, the one or more processors being configured to: determine a set of integer coefficient values ​​for the displacement vector of the encoded dynamic mesh data; determine a quantization parameter; quantize the set of integer coefficient values ​​based on the quantization parameter to determine a set of quantization coefficient values; and include one or more syntax elements indicating the quantization parameter in the displacement sub-bit stream of the encoded dynamic mesh data.

[0009] According to another example of this disclosure, a method for encoding dynamic mesh data includes: determining a set of integer coefficient values ​​for the displacement vector of the encoded dynamic mesh data; determining a quantization parameter; quantizing the set of integer coefficient values ​​based on the quantization parameter to determine a set of quantization coefficient values; and including one or more syntax elements indicating the quantization parameter in the displacement sub-bit stream of the encoded dynamic mesh data.

[0010] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0011] Figure 1 This is a block diagram illustrating an example encoding and decoding system that can perform the techniques of this disclosure.

[0012] Figure 2 An example implementation of a video-based dynamic mesh decoding (V-DMC) encoder is shown.

[0013] Figure 3 and Figure 4 An example implementation of the V-DMC decoder is shown.

[0014] Figure 5 An example of efficient compression of 2D curves through resampling is shown.

[0015] Figure 6 The displacement curves with a subdivision structure are shown, which approximate the shape of the original mesh.

[0016] Figure 7 A block diagram of the preprocessing system is shown.

[0017] Figure 8 An example implementation of the intra-mode encoder for V-DMC is shown.

[0018] Figure 9 An example implementation of the intra-mode decoder for V-DMC is shown.

[0019] Figure 10 An example of a displacement decoder is shown.

[0020] Figure 11 The sub-partitioning process is illustrated.

[0021] Figure 12 An example of the displacement decoder process is shown.

[0022] Figure 13A An example of a floating-point displacement decoder is shown.

[0023] Figure 13B An example of a fixed-point displacement decoder is shown.

[0024] Figure 14A An example of a floating-point displacement decoder is shown.

[0025] Figure 14B An example of a fixed-point displacement decoder is shown.

[0026] Figure 15This is a flowchart illustrating an example process for encoding a mesh.

[0027] Figure 16 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data.

[0028] Figure 17 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data. Detailed Implementation

[0029] A mesh is typically a collection of vertices in three-dimensional (3D) space that collectively represent one or more objects in that space. Vertices are connected by edges, and these edges form polygons that make up the faces of the mesh. Each vertex may also have one or more associated properties, such as texture or color. In most scenarios, having more vertices produces a higher quality mesh (e.g., more detailed and more realistic). However, having more vertices also requires more data to represent the mesh.

[0030] To reduce the amount of data required to represent a grid, lossy or lossless encoding can be used to encode the grid. In lossless encoding, the decoded version of the encoded grid perfectly matches the original grid. In contrast, in lossy encoding, the process of encoding and decoding the grid introduces losses such as distortion in the decoded version of the encoded grid.

[0031] In one example of a lossy encoding technique used for meshes, a mesh encoder decimates the original mesh to determine the base mesh. To decimate the original mesh, the mesh encoder subsamples or otherwise reduces the number of vertices in the original mesh, making the base mesh a coarse approximation of the original mesh with fewer vertices. The mesh encoder then subdivides the decimated mesh. That is, the mesh encoder estimates the positions of additional vertices between the vertices of the base mesh. The mesh encoder then deforms the subdivided mesh by moving the vertices, making the deformed mesh more closely match the original mesh.

[0032] After determining the desired deformation of the base mesh and sub-mesh, the mesh encoder generates a bitstream comprising data for constructing the base mesh and data for performing the deformation. The deformation-defining data is signaled as a series of displacement vectors indicating the movement or displacement of additional vertices determined by the sub-meshing process. To decode the mesh from the bitstream, the mesh decoder reconstructs the base mesh based on the signaled information, applies the same sub-meshing process as the mesh encoder, and then displaces the additional vertices based on the signaled displacement vectors.

[0033] This disclosure relates to encoding and decoding underlying grid data. More specifically, this disclosure describes various improvements to the displacement vector inter-frame prediction process in V-DMC technology, which is being standardized in MPEG WG7 (3DGH). This disclosure describes techniques for implementing a fixed-point (integer) quantization process in the inter-frame prediction scheme for displacement vector decoding.

[0034] To determine the shift vector, the trellis decoder can perform inter-frame prediction on the wavelet coefficients after inverse quantization, as described in more detail below. The trellis decoder adds the inverse quantized coefficients (residuals) to the inter-frame prediction values ​​stored in the frame buffer. The advantage of this technique is that the correlation between wavelet coefficients in adjacent frames is typically greater than the correlation between corresponding quantized coefficients, resulting in additional decoding efficiency gains.

[0035] To reconstruct displacement vector wavelet coefficients according to the techniques described herein, the inverse quantization process uses parameters from the bitstream, such as displacement vector dimension, quantization parameter (QP), number of levels of detail (LoD), number of vertices per LoD, etc. In V-DMC, some of these parameters are signaled as part of the atlas metadata sub-bitstream. However, the arithmetically decoded (AC) displacement sub-bitstream does not contain the parameters required by the inverse quantization process to reconstruct the displacement vector wavelet coefficients from this sub-bitstream without relying on the atlas metadata sub-bitstream. By determining the quantization parameters based on one or more syntax elements included in the displacement sub-bitstream of the encoded dynamic mesh data, the techniques disclosed herein enable self-contained reconstruction of displacement vectors without relying on other sub-bitstreams.

[0036] Figure 1 This is a block diagram illustrating an example encoding and decoding system 100 that can perform the techniques of this disclosure. The techniques of this disclosure generally involve decoding (encoding and / or decoding) grids. This decoding can be efficient in compressing and / or decompressing grid data.

[0037] like Figure 1 As shown, system 100 includes source device 102 and destination device 116. Source device 102 provides encoded data for decoding by destination device 116. Specifically, in Figure 1In this example, source device 102 provides data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 can include any of a wide range of devices, including desktop computers, laptop computers, tablet computers, set-top boxes, mobile phones (such as smartphones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, land or sea vehicles, spacecraft, aircraft, robots, LiDAR devices, satellites, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication.

[0038] exist Figure 1 In the example, source device 102 includes a data source 104, a memory 106, a V-DMC encoder 200, and an output interface 108. Destination device 116 includes an input interface 122, a V-DMC decoder 300, a memory 120, and a data consumer 118. According to this disclosure, the V-DMC encoder 200 of source device 102 and the V-DMC decoder 300 of destination device 116 can be configured to apply the techniques related to shift vector quantization disclosed herein. Therefore, source device 102 represents an example of an encoding device, while destination device 116 represents an example of a decoding device. In other examples, source device 102 and destination device 116 may include other components or arrangements. For example, source device 102 may receive data from an internal or external source. Similarly, destination device 116 may interface with an external data consumer, rather than including the data consumer in the same device.

[0039] like Figure 1 The system 100 shown is merely an example. In general, other digital encoding and / or decoding devices can perform the techniques related to shift vector quantization disclosed herein. Source device 102 and destination device 116 are merely examples of such devices, where source device 102 generates decoded data for transmission to destination device 116. This disclosure refers to a “decoding” device as a device that performs the decoding (e.g., encoding and / or decoding) of data. Thus, V-DMC encoder 200 and V-DMC decoder 300 represent examples of decoding devices, specifically, encoder and decoder, respectively. In some examples, source device 102 and destination device 116 can operate in a substantially symmetrical manner, such that each of source device 102 and destination device 116 includes both encoding and decoding components. Therefore, system 100 can support one-way or two-way transmission between source device 102 and destination device 116, for example, for streaming, playback, broadcasting, telephone, navigation, and other applications.

[0040] Generally, data source 104 represents the source of data (e.g., raw, unencoded data) and provides a series of sequential “frames” of data to V-DMC encoder 200, which encodes the data in the frames. Data source 104 of source device 102 may include a grid capture device, such as any of a variety of cameras or sensors, such as a 3D scanner, or a light detection and ranging (LIDAR) device, one or more video cameras, an archive containing previously captured data, and / or a data feed interface for receiving data from a data content provider. Alternatively or additionally, the grid data may be computer-generated from a scanner, camera, sensor, or other data source. For example, data source 104 may generate computer graphics-based data as source data, or a combination of real-time data, archived data, and computer-generated data. In each case, V-DMC encoder 200 encodes the captured, pre-captured, or computer-generated data. V-DMC encoder 200 may rearrange the frames from the received order (sometimes referred to as the “display order”) to a decoding order for decoding. The V-DMC encoder 200 can generate one or more bit streams including encoded data. The source device 102 can then output the encoded data to a computer-readable medium 110 via the output interface 108 for reception and / or retrieval by, for example, the input interface 122 of the destination device 116.

[0041] The memory 106 of source device 102 and the memory 120 of destination device 116 may represent general-purpose memory. In some examples, memory 106 and memory 120 may store raw data, such as raw data from data source 104 and raw decoded data from V-DMC decoder 300. Additionally or alternatively, memory 106 and memory 120 may store software instructions, respectively executable by, for example, V-DMC encoder 200 and V-DMC decoder 300. Although memory 106 and memory 120 are shown separately from V-DMC encoder 200 and V-DMC decoder 300 in this example, it should be understood that V-DMC encoder 200 and V-DMC decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memory 106 and memory 120 may store encoded data, such as output from V-DMC encoder 200 and input to V-DMC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, for example, to store raw decoded and / or encoded data. For instance, memory 106 and memory 120 may store data representing a grid.

[0042] Computer-readable medium 110 may represent any type of medium or device capable of transmitting encoded data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium enabling source device 102 to transmit encoded data directly to destination device 116 in real time, for example, via a radio frequency network or a computer-based network. Output interface 108 may modulate the transmitted signal including the encoded data, and input interface 122 may demodulate the received transmitted signal according to a communication standard, such as a wireless communication protocol. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network such as a local area network, a wide area network, or a global network (such as the Internet). The communication medium may include a router, a switch, a base station, or any other equipment that may be useful for facilitating communication from source device 102 to destination device 116.

[0043] In some examples, source device 102 can output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0044] In some examples, source device 102 may output encoded data to file server 114 or another intermediate storage device that may store the encoded data generated by source device 102. Destination device 116 may access the stored data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing encoded data and sending it to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded data from file server 114 via any standard data connection, including an internet connection. This standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a downloading protocol, or a combination thereof.

[0045] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (such as encoded data) according to cellular communication standards (such as 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to comply with specifications such as IEEE 802.11, IEEE 802.15 (e.g., ZigBee). ™ ),Bluetooth ™ Data (such as encoded data) is transmitted using other wireless standards, such as standard V-DMC encoder 200. In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include an SoC device for performing functions belonging to V-DMC encoder 200 and / or output interface 108, and destination device 116 may include an SoC device for performing functions belonging to V-DMC decoder 300 and / or input interface 122.

[0046] The technology disclosed herein can be applied to support encoding and decoding of any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices such as local or remote servers, geographic mapping or other applications.

[0047] The input interface 122 of the destination device 116 receives an encoded bitstream from a computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, etc.). The encoded bitstream may include signaling information defined by the V-DMC encoder 200 and also used by the V-DMC decoder 300, such as syntax elements having characteristics and / or processed values ​​describing the units being decoded (e.g., slices, pictures, picture groups, sequences, etc.). The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display that presents an image based on a grid.

[0048] The V-DMC encoder 200 and V-DMC decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the V-DMC encoder 200 and V-DMC decoder 300 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (codec) in the respective device. Devices including the V-DMC encoder 200 and / or V-DMC decoder 300 may include one or more integrated circuits, microprocessors, and / or other types of devices.

[0049] The V-DMC encoder 200 and V-DMC decoder 300 operate according to a decoding standard. This disclosure generally relates to the decoding (e.g., encoding and decoding) of images, thus including the process of encoding or decoding data. The encoded bitstream typically includes a series of values ​​for syntax elements representing decoding decisions (e.g., decoding modes).

[0050] This disclosure may generally relate to "signaling" certain information (such as syntax elements). The term "signaling notification" generally refers to communication of the value of a syntax element and / or other data used to decode encoded data. That is, the V-DMC encoder 200 may signal the value of a syntax element in the bit stream. Generally speaking, signaling notification means generating a value in the bit stream. As noted above, the source device 102 may transmit the bit stream to the destination device 116 substantially in real time or not in real time (such as when the syntax element is stored in the storage device 112 for later retrieval by the destination device 116).

[0051] This disclosure describes various improved techniques for vertex attribute encoding of the underlying mesh in video-based dynamic mesh decoding (V-DMC), which is being standardized in MPEG WG7 (3DGH). In V-DMC, an edgebreaker implementation is used to encode the connectivity of the underlying mesh, and residual coding with attribute predictions can be used to encode the attributes of the underlying mesh. This disclosure describes techniques for transforming and / or quantizing attributes and / or predictions and / or residuals for use in underlying mesh coding, which improves the decoding performance of the underlying mesh coding.

[0052] Working Group 7 (WG7), commonly known as the 3D Graphics and Haptic Decoding Group (3DGH), is currently working on standardizing video-based dynamic mesh decoding (V-DMC) for XR applications. The current test model involves preprocessing the input mesh into a simplified version, called the "base mesh." These base meshes (which typically contain fewer vertices than the original mesh) are encoded using a base mesh decoder (also known as a static mesh decoder). The preprocessing also generates a displacement vector and a texture attribute map, both of which are encoded using a video encoder. If the mesh is encoded in a lossless manner, the base mesh is no longer a simplified version and is used to encode the original mesh.

[0053] The base mesh encoder encodes the mesh connectivity and the attributes associated with each vertex, which typically relate to, but are not limited to, the location and coordinates of the texture. Location includes the vertex's 3D coordinates (x, y, z), while texture is stored as 2D UV coordinates (u, v) pointing to the pixel location in the texture map image. The edgebreaker algorithm is used to encode the base mesh in V-DMC, while the CLERS opcode is used to encode connectivity. Residuals of attributes are encoded using predictions from previously encoded / decoded vertices. Other types of static mesh decoders, such as Google Draco, can also be used. Other types of decoding can also be used for connectivity decoding and residual decoding.

[0054] The edgebreaker algorithm is described in Jean-Eudes Marvie and Olivier Mocquard's [V-DMC][EE4.4], Efficient Edgebreaker Implementation, ISO / IEC JTC1 / SC29 / WG7, m63344, April 2023 (hereinafter referred to as "m63344"). The CLERS operand code is described in J. Rossignac's "3D Compression Made Simple: Edgebreaker Using ZipandWrap on Corner Tables," published in the Proceedings of the International Conference on Shape Modeling and Applications, Genoa, Italy, 2001 (hereinafter referred to as "Rossignac"), and in H. Lopes, G. Tavares, J. Rossignac, A. Szymczak, and A. Safonova's "Edgebreaker: A Simple Compression Algorithm for Handled Surfaces," published in the ACM Workshop on Solid Modeling and Applications, Saarbrücken, 2002 (hereinafter referred to as "Lopes").

[0055] Additionally, the V-DMC encoder 200 can estimate the motion of the base mesh vertices and decode the motion vectors into a bitstream. The reconstructed base mesh can be subdivided into a finer mesh with additional vertices, and thus subdivided into additional triangles. The V-DMC encoder 200 can refine the positions of the subdivided mesh vertices to approximate the original mesh. The refined or vertex displacement vectors can be decoded into the bitstream. In the current test model, the displacement vectors are wavelet transformed (lifting scheme), quantized, and the coefficients are packed into 2D frames after inter-frame prediction, or directly decoded using an arithmetic decoder. The video frame sequence is decoded into the bitstream using a typical video decoder (e.g., the High Efficiency Video Decoding (HEVC) standard or the Multi-Functional Video Decoding (VVC) standard). Additionally, the texture frame sequence is decoded using a video decoder. Figure 4 A simplified architecture of the V-DMC decoder is illustrated.

[0056] Figure 2 and Figure 3 The encoder for the current V-DMC test model (TM) is shown. Figure 2 V-DM encoder 200 and decoder (in the middle) Figure 3 The overall system model of the V-DMC decoder 300 architecture is shown below. The V-DMC encoder 200 performs volumetric media conversion, and the V-DMC decoder 300 performs the corresponding reconstruction. The 3D media is converted into a series of sub-bitstreams: base mesh, displacement, and texture attributes. As described in N00680, the bitstreams also include additional atlas information to enable inverse reconstruction.

[0057] Figure 2 An example implementation of the V-DMC encoder 200 is shown. Figure 2 In the example, the V-DMC encoder 200 includes a preprocessing unit 204, an atlas encoder 208, a base mesh encoder 212, a displacement encoder 216, and a video encoder 220. The preprocessing unit 204 receives an input mesh sequence and generates a base mesh, displacement vectors, and a texture attribute map. The base mesh encoder 212 encodes the base mesh. The displacement encoder 216 encodes the displacement vectors, for example, as V3C video components, or using arithmetic displacement decoding. The video encoder 220 uses any video codec (such as HEVC or VVC) to encode the texture attribute components (e.g., texture or material information).

[0058] The V-DMC encoder 200 will now be described in more detail. The preprocessing unit 204 represents the 3D volumetric data as a set of base meshes and corresponding refined components. This is achieved by converting the input dynamic mesh representation into multiple V3C components: the base mesh, a set of displacements, a 2D representation of the texture map, and an atlas. The base mesh components are a simplified, low-resolution approximation of the original mesh in lossy compression, and the original mesh itself in lossless compression. The base mesh components can be encoded by the base mesh encoder 212 using any mesh codec.

[0059] The base mesh encoder 212 may employ, for example, a specific implementation of the Edgebreaker algorithm (e.g., m63344) to encode the base mesh, wherein connectivity is encoded using, for example, CLERS operation codes from Rossignac and Lopes, and residuals of attributes are encoded using predictions of attributes from previously encoded / decoded vertices.

[0060] The various aspects of the base mesh encoder 212 will now be described in more detail. One or more sub-meshes are input into the base mesh encoder 212. The sub-meshes are generated by the preprocessing unit 204. Sub-meshes are generated from the original mesh through semantic segmentation. Each base mesh may include one or more sub-meshes.

[0061] The basic mesh encoder 212 can process connected components. Connected components consist of clusters of triangles connected by adjacent triangles. A submesh may have one or more connected components. The basic mesh encoder 212 can encode the connectivity and properties of one "connected component" at a time, and then perform entropy encoding on all "connected components".

[0062] Figure 3 An example implementation of the V-DMC decoder 300 is shown. Figure 3 In the example, the V-DMC decoder 300 includes a demultiplexer 304, an atlas decoder 308, a basic mesh decoder 314, a displacement decoder 316, a video decoder 320, a basic mesh processing unit 324, a displacement processing unit 328, a mesh generation unit 332, and a reconstruction unit 336.

[0063] Demultiplexer 304 separates the encoded bitstream into atlas sub-bitstream, base mesh sub-bitstream, displacement sub-bitstream, and texture attribute sub-bitstream. Atlas decoder 308 decodes the atlas sub-bitstream to determine atlas information, thereby achieving inverse reconstruction. Base mesh decoder 314 decodes the base mesh sub-bitstream, and base mesh processing unit 324 reconstructs the base mesh. Displacement decoder 316 decodes the displacement sub-bitstream, and displacement processing unit 328 reconstructs the displacement vector. Mesh generation unit 332 modifies the base mesh based on the displacement vector to form a displacement mesh.

[0064] The video decoder 320 decodes the texture attribute sub-stream to determine the texture attribute map, and the reconstruction unit 336 associates the texture attributes with the displacement mesh to form a reconstructed dynamic mesh.

[0065] A more detailed description of the proposal chosen as the starting point for V-DMC standardization will now be provided. The following description details the current V-DMC test model and the displacement vector decoding in the working draft V-DMC WD 5.0, ISO / IEC JTC1 / SC29 / WG7, N00744, October 2023.

[0066] Figure 4 An example implementation of a V-DMC decoder 300 is shown, which can be configured to perform the decoding process as described in V-DMC WD 2.0, ISO / IEC JTC1 / SC29 / WG7, N00546, January 2023. (About...) Figure 2 The described process can also be performed, in whole or in part, by the V-DMC encoder 200.

[0067] The V-DMC decoder 300 includes a demultiplexer (DMUX) 402 that receives the compressed bitstream b(i) and separates it into a base grid bitstream (BMB), a shift bitstream (DB), and an attribute bitstream (AB). A mode selection unit 404 determines whether the base grid data is encoded in intra-frame or inter-frame mode. If the base grid is encoded in intra-frame mode, a static grid decoder 406 decodes the grid data without relying on any previously decoded grids. If the base grid is encoded in inter-frame mode, a motion decoder 408 decodes the motion, and a base grid reconstruction unit 410 applies motion to the decoded grids stored in a grid buffer 412 to determine the reconstructed quantized base grid (m'(i)). An inverse quantization unit 414 applies inverse quantization to the reconstructed quantized base grid to determine the reconstructed base grid (m''(i)).

[0068] Video decoder 416 decodes the displacement bitstream to determine a set of quantized transform coefficients or frames. For encoding and decoding purposes, the quantized transform coefficients can be considered as a two-dimensional structure (e.g., frames). Image unpacking unit 418 unpacks (e.g., concatenates) the quantized transform coefficients from the frames. Inverse quantization unit 420 inverse quantizes (e.g., inverse scaling) the quantized transform coefficients to determine the dequantized transform coefficients. Inverse wavelet transform unit 422 applies an inverse transform to the dequantized transform coefficients to determine a set of displacement vectors. Deformed mesh reconstruction unit 424 uses the decoded displacement vectors to deform the reconstructed base mesh to determine the decoded mesh (M''(i)).

[0069] The video decoder 426 decodes the attribute bitstream to determine the decoded attribute value (A'(i)), and the color space conversion unit 428 converts the decoded attribute value into the desired color space to determine the final attribute value (A''(i)). The final attribute value corresponds to the attribute of the vertex of the decoded mesh, such as color or texture.

[0070] Figure 5 The basic idea behind the proposed preprocessing is illustrated using 2D curves. The same concept is applied to the input 3D mesh. To generate the base mesh and displacement field .

[0071] exist Figure 5 First, the input 2D curve (represented by a 2D polyline) called the "original" curve is downsampled to generate a base curve / polyline called the "extracted" curve. Then, a subdivision process (such as the subdivision process described in "Surface Reduction Using Quadratic Error Metrics" by Garland et al. (https: / / www.cs.cmu.edu / ~garland / Papers / quadrics.pdf)) is applied to the extracted polyline to generate a "subdivided" curve. For example, in Figure 5 The example uses a sub-partitioning process with iterative interpolation. The process involves inserting a new point at the midpoint of each edge of the polyline in each iteration. In the illustrated example, two sub-partitioning iterations are applied.

[0072] The proposed process is independent of the selected sub-partitioning process and can be combined with other sub-partitioning processes. Then, the polyline deformation of the sub-partitions is performed to obtain a better approximation of the original curve. A displacement vector is calculated for each vertex of the sub-partitioned mesh. Figure 5 Arrow 502 in the figure), so that the shape of the displacement curve is as close as possible to the shape of the original curve (see arrow 502 in the figure). Figure 6 As illustrated by portion 504 of the displacement curve and portion 506 of the original curve, for example, the displacement curve may not perfectly match the original curve.

[0073] The advantage of subdivided curves lies in their subdivision structure, which allows for efficient compression, while providing a faithful approximation of the original curve. Compression efficiency is achieved based on the following: - The decimation / base curve has a small number of vertices and requires a limited number of bits to encode / transmit.

[0074] - Once the base / decimation curve is decoded, the subpartition curve is automatically generated by the decoder (i.e., no information is needed except for the subpartition process type and subpartition iteration count).

[0075] - Displacement curves are generated by decoding the displacement vectors associated with the vertices of the subpartitioned curves. In addition to allowing spatial / quality scalability, the subpartition structure also enables efficient transforms such as wavelet decomposition that provide high compression performance.

[0076] Figure 7 A block diagram of a preprocessing system 700 that can be included in a V-DMC encoder 200 is shown. Figure 7 In the example, the preprocessing system 700 includes a mesh extraction unit 710, an atlas parameterization unit 720, and a sub-partition surface fitting unit 730.

[0077] Mesh extraction unit 710 uses a simplification technique to extract the input mesh. And generate extraction mesh Extract the mesh. The mesh is then reparameterized by atlas parameterization unit 720, which can be, for example, using the UVAtlas tool. The resulting mesh is represented as... The UVAtlas tool only considers the extracted mesh when parametricizing the computation atlas. The geometric information may not be optimal for compression purposes. Other parameterization processes or tools can also be used with the proposed framework.

[0078] Applying reparameterization to the input mesh allows for the generation of a smaller number of patches. This reduces parameterization discontinuities and improves RD (rate distortion) performance. The sub-partitioned surface fitting element 730 applies the reparameterized mesh... and input grid As input and with a set of displacements Together, they generate the basic mesh. .First, Sub-partitioning is performed by applying a sub-partitioning process. This involves determining the position of each vertex of the sub-partitioned mesh relative to the original mesh. The displacement field is calculated using the nearest point on the surface. .

[0079] For the conditions of random access (RA), it can be achieved by indexing as The base grid of the reference frame The time-consistent remeshing is computed as input to the sub-partitioned surface fitting unit 730. This allows for remeshing tailored to the current mesh. Generate and target reference mesh The calculated sub-mesh structure is the same as the sub-mesh structure. This type of re-meshization process allows skipping the base mesh. Encoding and reuse with reference frames Associated base grid This also allows for better temporal prediction of both attribute and geometric information. For example, it enables the calculation and encoding of descriptions of how movement occurs. Vertices to match The location of the sports field Such time-consistent remeshing may not always be feasible. The techniques disclosed herein may also include comparing the distortions obtained with and without time-consistency constraints and selecting the mode that provides the best RD tradeoff.

[0080] It should be noted that the preprocessing system is not standard and can be replaced by any other system that generates the displacement sub-mesh surface. A potentially efficient implementation would constrain the 3D reconstruction unit to directly generate the displacement sub-mesh surface and avoid the need for such preprocessing.

[0081] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to perform displacement decoding. Depending on the application and target bit rate / visual quality, the V-DMC encoder 200 can optionally perform displacement decoding on a set of displacement vectors (referred to herein as displacement fields) associated with the vertices of the subdivided mesh. Encode it. Figure 8 An intra-frame coding process that can be performed by the V-DMC encoder 200 is illustrated.

[0082] Figure 8 Including the following abbreviations: m(i) - Basic Mesh d(i) - displacement m(i) - Reconstructing the base mesh d''(i) - Reconstructed displacement A(i)-attribute graph A'(i) - Updated property graph M(i) - Static / Dynamic Mesh DM(i) - Reconstructing Deformed Mesh m'(i) - Reconstructing the quantized base grid d'(i) - Updated displacement e(i) - wavelet coefficients e'(i) - Quantization wavelet coefficients pe'(i) - Packed quantized wavelet coefficients rpe'(i) - Reconstructing Packed Quantized Wavelet Coefficients AB – Compressed Attribute Bitstream DB – Compressed Displacement Bit Stream BMB – Compressed Basic Mesh Bit Stream b(i) – Compressed bit stream V-DMC encoder 200, for example, from Figure 7 The preprocessing system 700 receives the base mesh m(i) and displacement d(i). The V-DMC encoder 200 also retrieves the mesh M(i) and the property map A(i).

[0083] The quantization unit 802 quantizes the base grid, and the static grid encoder 804 encodes the quantized base grid to generate a compressed base grid bit stream (BMB).

[0084] Displacement update unit 808 uses reconstructed quantized base mesh To update the displacement field To generate an updated displacement field The process considers reconstructing the base mesh. With the original base mesh The differences between them. By utilizing the subdivided surface mesh structure, the wavelet transform unit 810 pairs A wavelet transform is applied to generate a set of wavelet coefficients (e(i)). This process is independent of the applied transform and can utilize any other transform, including the identity transform. The quantization unit 812 quantizes the wavelet coefficients, and the image packing unit 814 packs the quantized wavelet coefficients into a 2D image / video, which can be compressed using a conventional image / video encoder (e.g., a technique such as VVC) to generate a shifted bitstream.

[0085] The attribute transfer unit 830 converts the original attribute map A(i) into an updated attribute map corresponding to the reconstructed deformable mesh DM(i). The padding unit 832 pads the updated attribute map by, for example, using interpolated samples to pad frames with empty samples, which improves decoding efficiency and reduces artifacts. The color space conversion unit 834 converts the attribute map into a different color space, and the video encoding unit 836 encodes the updated attribute map in the new color space using, for example, a video codec to generate an attribute bitstream.

[0086] The multiplexer 838 combines the compressed attribute bitstream, the compressed displacement bitstream, and the compressed base grid bitstream into a single compressed bitstream (b(i)).

[0087] Image unpacking unit 818 and inverse quantization unit 820 apply image unpacking and inverse quantization to the reconstructed packed quantized wavelet coefficients generated by video coding unit 816 to obtain reconstructed versions of the wavelet coefficients. Inverse wavelet transform unit 822 applies inverse wavelet transform to the reconstructed wavelet coefficients to determine the reconstructed displacement. .

[0088] Inverse quantization cell 824 applies inverse quantization to the reconstructed quantization base mesh m'(i) to obtain the reconstructed base mesh. Deformed mesh reconstruction element 828 sub-division And apply the reconstructed displacement to its vertices. To obtain the reconstructed deformable mesh .

[0089] Image unpacking unit 818, inverse quantization unit 820, inverse wavelet transform unit 822, and deformed mesh reconstruction unit 828 represent the displacement decoding loop. Inverse quantization unit 824 and deformed mesh reconstruction unit 828 represent the base mesh decoding loop. The mesh encoder 800 includes a displacement decoding loop and a base mesh decoding loop, enabling the mesh encoder 800 to make encoding decisions based on the same decoded mesh generated by the mesh decoder, such as determining an acceptable rate-distortion tradeoff, which may include distortion due to quantization and transform. The mesh encoder 800 can also use decoded versions of the base mesh, reconstructed mesh, and displacement to encode subsequent base meshes and displacements.

[0090] Control unit 850 typically represents the decision-making function of V-DMC encoder 200. During the encoding process, control unit 850 may make determinations, for example, regarding mode selection, rate allocation, quality control, and other such decisions.

[0091] Figure 9 A block diagram of an intra-decoder, which may be a part of, for example, the V-DMC decoder 300, is shown. The demultiplexer (DMUX) 902 separates the compressed bitstream (b(i)) into a mesh substream, a displacement substream for positioning and potentially for each vertex attribute, a zero or more attribute graph substream, and an atlas substream containing patch information, in the same manner as in V3C / V-PCC.

[0092] Demultiplexer 902 feeds the mesh substream to static mesh decoder 906 to generate the reconstructed quantized base mesh. The inverse quantization unit 914 inverses the base grid to determine the decoding base grid. The video / image decoding unit 916 decodes the displacement substream, and the image unpacking unit 918 unpacks the image / video to determine quantization transform coefficients, such as wavelet coefficients. The inverse quantization unit 920 inverse-quantizes the quantization transform coefficients to determine the dequantization transform coefficients. The inverse transform unit 922 generates the decoded displacement field by applying an inverse transform to the unquantized coefficients. The deformable mesh reconstruction unit 924 decodes the base mesh. Apply the reconstruction process and add a decoded displacement field This is used to generate the final decoded mesh (M''(i)). The attribute substream is directly decoded by the video / image decoding unit 926 to generate the attribute map. The color format / space conversion unit 928 can convert attribute maps into different formats or color spaces.

[0093] As an additional or alternative scheme for packaging quantized wavelet coefficients into frames and decoding them into images or videos, a process can also be used to directly decode the quantized wavelet coefficients using a block-based arithmetic decoder. This process... Figure 10 The example illustrates this. Decoding quantized wavelet coefficients is performed from a reference buffer that contains quantized wavelet coefficients from previous frames (e.g., the preceding frame). Figure 10 In the example, decoder 1000 performs context-based arithmetic decoding 1002 on the shifted bitstream based on context update 1004. Decoder 1000 performs debinarization 1006 on the context-decoded bitstream to determine the values ​​of syntax elements, and performs coefficient-level decoding 1008 on the syntax elements. For intra-frame decoded shifts, decoder 1000 performs inverse quantization 1012 on the coefficient levels to determine inverse quantized coefficient levels, and then performs inverse wavelet transform 1014 on the inverse quantized coefficient levels to determine the shift. For inter-frame decoded shifts, decoder 1000 performs inter-frame prediction 1016 using a reference frame stored in frame buffer 1018, and adds the predicted value to the coefficient levels 1020 to determine the final coefficient levels. Decoder 1000 then performs inverse quantization 1012 on the final coefficient levels to determine inverse quantized coefficient levels, and then performs inverse wavelet transform 1014 on the inverse quantized coefficient levels to determine the shift.

[0094] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to implement a sub-partitioning process. Various sub-partitioning processes can be considered. One possible solution is a midpoint sub-partitioning process, which divides each triangle into four sub-triangles in each sub-partitioning iteration, such as... Figure 11 As described. A new vertex is introduced in the middle of each edge. In Figure 11In the example, triangle 1102 is subdivided to obtain triangle 1104, and triangle 1104 is subdivided to obtain triangle 1106. The subdivision process is applied independently to the geometry and texture coordinates because the connectivity of the geometry and texture coordinates is typically different. The subdivision process is computed on the edges. The newly introduced vertex at the center Location ,as follows:

[0095] in and It is the vertex and The location.

[0096] The same process is used to calculate the texture coordinates of the newly created vertices. For the normal vectors, an additional normalization step is applied as follows:

[0097] here: , and They are respectively with the vertex , and The associated normal vector.

[0098] It is a vector norm2.

[0099] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to apply wavelet transforms. Various wavelet transforms can be applied. The results reported for CfP are based on linear wavelet transforms.

[0100] The prediction process is defined as follows:

[0101] in It is on the side The middle vertex introduced, and , and These are the vertices. , and The value of the geometry / vertex attribute signal at that location.

[0102] The update process is as follows:

[0103] in It is the vertex The set of adjacent vertices.

[0104] The process may allow skipping the update process. Wavelet coefficients can be quantized, for example, by using a uniform quantizer with a dead zone.

[0105] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to use a local coordinate system or a canonical coordinate system for displacement. It is defined in the same Cartesian coordinate system as the input grid. A possible optimization is to... Transform from the regular coordinate system to the local coordinate system, which is defined by the normals of the sub-mesh at each vertex.

[0106] The advantage of considering a local coordinate system for displacement is that the tangential components of the displacement can be quantized to a greater extent than the normal components. In fact, the normal components of the displacement have a more significant impact on the quality of the reconstructed mesh than the two tangential components.

[0107] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to pack (and unpack) wavelet coefficients. The following procedure can be used to pack wavelet coefficients into a 2D image: The coefficients are traversed from low frequency to high frequency.

[0108] For each coefficient, determine The pixel blocks that should be stored in the raster order of the blocks (e.g., () index.

[0109] The position within a pixel block is calculated using the Morton order to maximize locality.

[0110] Other packing processes (e.g., zigzag order, raster order) can also be used. The encoder can explicitly signal the packing process used (e.g., atlas sequence parameters) in the bit stream. This can be done at the patch, patch group, tile, or sequence level.

[0111] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to encode and decode displacement video.

[0112] The proposed process is independent of the video decoding technology used. When decoding shift wavelet coefficients, a lossless method can be used because quantization is applied in a separate module. Another approach relies on the video encoder to compress the coefficients in a lossy manner and apply quantization in the original or transform domain.

[0113] The following paragraphs reproduce the syntax and semantics of V-DMC Working Draft (WD) 6.0, ISO / IEC JTC1 / SC29 / WG7, N00822 (d25), January 2024 (hereafter referred to as “WD 6.0”), which relates to the signaling of quantization parameters associated with the decoding (lifting process) of the shifted wavelet coefficients.

[0114] The various aspects of the atlas metadata sub-stream will now be described.

[0115] Atlas sequence parameter set from WD 6.0

[0116] Syntax elements related to the quantization process are shown in the delimiters. <q> and< / q> between.

[0117]

[0118]

[0119] `asve_subdivision_iteration_count` indicates the number of iterations used for subdivision. When it does not exist, the value of `asve_subdivision_iteration_count` is inferred to be 0.

[0120] `asve_1d_displacement_flag` equal to 1 indicates that only the normal vector (or x-component) of the displacement exists in the compressed geometric video. The other two components are inferred as 0. `asve_1D_displacement_flag` equal to 0 indicates that all three components of the displacement exist in the compressed geometric video.

[0121] `asve_displacement_reference_qp` specifies the initial value of the QuantizationParameter for the current frame. When it does not exist, `asve_displacement_reference_qp` is set to 49.

[0122] vdmc_quantization_parameters structure:

[0123]

[0124] A value of 1 for `vqp_lod_quantization_flag[qpIndex]` indicates that incremental decoding is used to transmit quantization parameters at the level of detail. A value of 0 for `vqp_lod_quantization_flag[qpIndex]` indicates that the quantization parameters will be the same across all levels of detail. `qpIndex` is the index of the set of quantization parameters.

[0125] `vqp_bitdepth_offset[qpIndex]` indicates the bit depth offset value applied to the quantization process of the bit shift. `qpIndex` is the index of the quantization parameter set.

[0126] `vqp_quantization_parameters[ qpIndex ][ k ]` indicates the quantization parameters to be used for the inverse quantization of the k-th component of the displacement. The value of `vqp_quantization_parameters[ qpIndex ][ k ]` should be in the range of 0 to 100 (inclusive). `qpIndex` is the index of the quantization parameter set.

[0127] `vqp_log2_lod_inverse_scale[ qpIndex ][ k ]` indicates the scaling factor applied to the k-th component of the displacement at each level of detail. `qpIndex` is the index of the quantization parameter set.

[0128] `vqp_lod_delta_quantization_parameter_value[qpIndex][i][k]` specifies the absolute difference between the `asve_displacement_reference_qp` value and the quantization parameter values ​​for the i-th layer and the k-th component. If not present, `vqp_lod_delta_quantization_parameter_value[qpIndex][i][k]` is inferred to be 0. `qpIndex` is the index of the quantization parameter set. The value of the `QuantizationParameter` for each LoD layer should be in the range of 0 to 100.

[0129] `vqp_lod_delta_quantization_parameter_sign[qpIndex][i][k]` specifies the sign of the difference between the `asve_displacement_reference_qp` value and the quantization parameter values ​​of the i-th layer and the k-th component. `vqp_lod_delta_quantization_parameter_sign[qpIndex][i][k]` equal to 0 indicates a positive difference. `vqp_lod_delta_quantization_parameter_sign[qpIndex][i][k]` equal to 1 indicates a negative difference. When it does not exist, the value of `vqp_lod_delta_quantization_parameter_sign[qpIndex][i][k]` is inferred to be 0. `qpIndex` is the index of the quantization parameter set.

[0130] A value of 1 for `vqp_direct_quantization_enabled_flag[qpIndex]` indicates that the inverse scaling factor is directly derived from the shift quantization parameters notified by the signaling, and the following calculations are performed: InverseScale[ qpIndex ][ i ][ k ] = 1 ÷ QuantizationParameter[qpIndex ][ i ][ k ] A value of 0 for vqp_direct_quantization_enabled_flag[qpIndex] indicates that the inverse scaling factor should be calculated as follows: bitDepthPosition = asps_geometry_3d_bit_depth_minus1 + 1 InverseScale[ qpIndex ][ i ][ k ] = pow(0.5, 16 + vqp_bitdepth_offset[ qpIndex ] – bitDepthPosition + (4 – QuantizationParameters[ qpIndex ][ i ][ k ])÷ 6) qpIndex is the index of the quantization parameter set.

[0131] Atlas frame parameter set from WD 6.0

[0132] Syntax elements related to the quantization process are shown in the delimiters. <q> and< / q> between.

[0133]

[0134] The fact that afve_overriden_flag equals 1 indicates that the parameters afve_subdivision_enable_flag, afve_transform_method_enable_flag, afve_transform_parameters_enable_flag, and afve_attribute_parameter_overwrite_flag exist in the atlas frame parameter set extension.

[0135] An afve_quantization_enable_flag value of 1 indicates that the vdmc_quantization_parameters(qpIndex, subdivisionCount) syntax structure exists in the atlas frame parameter set extension. When afve_quantization_enable_flag does not exist, its value is inferred to be 0.

[0136] Grid Patch Data Unit from WD 6.0

[0137] Syntax elements related to the quantization process are shown in the delimiters. <q> and< / q> between.

[0138]

[0139] The value of mdu_parameters_override_flag[ tileID ][ patchIdx ] equal to 1 indicates that the parameters mdu_subdivision_override_flag, mdu_quantization_override_flag, mdu_transform_method_override_flag, and mdu_transform_parameters_override_flag exist in the mesh patch with index patchIdx in the current atlas tile whose tile ID is equal to tileID.

[0140] A value of 1 for `mdu_quantization_override_flag[ tileID ][ patchIdx ]` indicates that the `vdmc_quantization_parameters(qpIndex, subdivisionCount)` syntax structure exists in the mesh patch at index `patchIdx` within the current atlas tile whose tile ID equals `tileID`. When `mdu_quantization_override_flag[ tileID ][ patchIdx ]` does not exist, its value is inferred to be 0.

[0141] The variable QpIndex in the current patch can be exported as follows: QpIndex = mdu_quantization_override_flag[tileID][patchIdx]? 2afve_quantization_enable_flag? 1: 0 The aspects of the bitstream of the shifted sub-bits after arithmetic decoding will now be described. Syntax elements related to the quantization process are shown in the delimiters. <q> and< / q> between.

[0142] Displacement Sequence Parameter Set (DSPS) from WD 6.0

[0143] `dsps_single_dimension_flag` indicates the number of dimensions of the displacement associated with it. `dsps_single_dimension_flag` equal to 0 indicates that all three components of the displacement are used. `dsps_single_dimension_flag` equal to 1 indicates that only the normal vector of the displacement is used.

[0144] MaxDimension = dsps_single_dimension_flag ? 1 : 3

[0145] Displacement Frame Parameter Set (DFPS) from WD 6.0

[0146] Displacement data cell syntax from WD 6.0

[0147]

[0148]

[0149] ddu_lod_count[ displID ] indicates the number of sub-division levels used to signal the displacement in the data unit associated with displId displID.

[0150] ddu_vertex_count_lod[ displID ][ i ] indicates the number of shifts at the i-th level of the wavelet transform of the data unit associated with displId displID.

[0151] As described above in relation to shift decoding, inter-frame prediction (lifting process) can be performed on the quantized wavelet coefficients after arithmetic decoding of the shift. Recently, the proposal "[V-DMC][EE4.7 Test 5.1] Quantization after inter-frame prediction in shift decoding based on arithmetic decoding" by H. Nishimura, K. Kawamura, K. Kishimoto, and J. Xu, ISO / IEC JTC1 / SC29 / WG7, m64391, July 2023 (hereinafter referred to as "m64391"), introduced inter-frame prediction of the wavelet coefficients themselves after inverse quantization, as illustrated. The inverse quantization coefficients (residuals) are added to the inter-frame prediction values ​​stored in the frame buffer. The benefit is that the correlation between wavelet coefficients in adjacent frames is greater than the correlation between corresponding quantization coefficients, which results in additional decoding efficiency gains. However, the inverse quantization process is implemented as floating-point operations, which are known to depend on the order of operations and the specific implementation on the platform. Since the reconstructed wavelet coefficients are stored in the frame buffer for inter-frame prediction, floating-point operations can cause the reconstructed wavelet coefficients at the encoder and decoder to differ depending on the specific implementation.

[0152] Figure 12 The arithmetic decoding process of the displacement bit stream is illustrated. Figure 12 In the example, decoder 1200 performs context-based arithmetic decoding 1202 on the shifted bitstream based on context update 1204. Decoder 1200 performs debinarization 1206 on the context-decoded bitstream to determine the values ​​of syntax elements, and performs coefficient-level decoding 1208 on the syntax elements. Decoder 1200 then inverse-quantizes 1210 the coefficient levels to determine the inverse-quantization coefficients. For intra-frame decoded shifts, decoder 1200 performs inverse wavelet transform 1214 on the inverse-quantized coefficient levels to determine the shift. For inter-frame decoded shifts, decoder 1200 performs inter-frame prediction 1016 using a reference frame stored in frame buffer 1218, and adds the predicted value to the inverse-quantized coefficient levels 1220 to determine the final inverse-quantized coefficient levels. Decoder 1200 then performs inverse wavelet transform 1214 on the final inverse-quantized coefficient levels to determine the shift.

[0153] According to the technology disclosed herein, the V-DMC encoder 200 and V-DMC decoder 300 can be configured to perform inter-frame prediction of shifted wavelet coefficients using fixed-point inverse quantization. This disclosure describes techniques for integrating a fixed-point (or integer) inverse quantizer as described in U.S. Provisional Patent Application No. 63 / 586,120, filed September 28, 2023, to mitigate potential drift problems caused by floating-point operations during the inverse quantization process. Furthermore, integer arithmetic can be definitively specified in standard specifications.

[0154] As proposed in U.S. Provisional Patent Application No. 63 / 620,665 and Geert Van der Auwera, Adarsh ​​Krishnan Ramasubramonian, Reetu Hooda, Anique Akhtar, Marta Karczewicz, “[V-DMC][EE4.7 Test5-related] Inter-frame prediction of V-DMC shift wavelet coefficients using fixed-point quantization,” ISO / IEC JTC1 / SC29 / WG7, m66302, January 2024 (hereinafter referred to as “m66302”). A fixed-point inverse quantizer is integrated to mitigate the effects of... Figure 13A , Figure 13B , Figure 14A and Figure 14B The illustrated dequantizer process introduces a potential drift problem caused by floating-point operations.

[0155] Figure 13A The process described in m64391 is illustrated and is Figure 12 A simplified version. Figure 13A The arithmetic process for encoding and decoding the shift is illustrated. On the encoder side, after performing a wavelet transform (WT) on the shift (1D or 3D), the predicted value from the reference buffer (REF) is subtracted, followed by quantization (Q). The arithmetic encoder (AE) decodes the quantization residual into a bitstream. On the decoder side, the shift (Rec.Displ.) is reconstructed. The reconstructed wavelet coefficients are obtained after arithmetic decoding (AD), inverse quantization (IQ), and addition with the predicted value from the reference buffer. These coefficients, obtained before the inverse wavelet transform (IWT), can also be used on the encoder side to store in the reference buffer for predicting future frames of wavelet coefficients.

[0156] Figure 13BThe arithmetic process used for decoding is illustrated. After arithmetic decoding, a fixed-point (integer) inverse quantization process (IQ 1304) (U.S. Provisional Patent Application No. 63 / 586,120) is applied. The AD output can be converted (e.g., by left shifting bits) into an integer residual of the fixed-point value. Subsequently, the inverse quantization residual is added to a predicted value from a reference buffer (1302), which is also stored as a fixed-point value. In this encoder example, IWT is implemented with floating-point arithmetic and in the encoder's reference buffer, thus including a fixed-point to floating-point conversion (inverted triangle).

[0157] Figure 14A and Figure 14B A video-based process for encoding and decoding shifts is illustrated. Figure 14A In the process, after performing a wavelet transform (WT) on the displacement (1D or 3D), the predicted value from the reference buffer (REF) is subtracted, followed by quantization (Q). The image packer (IP) packs the quantization residuals into frames (i.e., images), and the video encoder (VE) encodes the frames. On the decoder side, the displacement (Rec.Displ.) is reconstructed. The reconstructed wavelet coefficients are obtained after video decoding (VD), image unpacking (IU), and inverse quantization (IQ), and are added to the predicted value from the reference buffer. These coefficients, obtained before the inverse wavelet transform (IWT), are also stored on the decoder side for predicting future frames of wavelet coefficients.

[0158] Figure 14B A video-based process for encoding and decoding shifts is illustrated. After video decoding (VD) and image unpacking (IU), a fixed-point (integer) inverse quantization process (IQ 1404) (U.S. Provisional Patent Application No. 63 / 586,120) is applied. The inverse quantization residual is added to a predicted value from a reference buffer (1402), which is also stored as a fixed-point value. In this example, IWT is implemented using floating-point arithmetic and in the encoder's reference buffer, thus including a fixed-point to floating-point conversion (inverted triangle).

[0159] According to the technology disclosed herein, the V-DMC encoder 200 and the V-DMC decoder 300 can perform signaling notification of quantization parameters in an arithmetically decoded shift sub-bit stream for inter-frame prediction of shifts using fixed-point inverse quantization.

[0160] The techniques proposed in this disclosure are applied to arithmetic decoding of shifts or alternative decoding methods, such as variable-length decoding, neural network-based decoding, etc. It should be understood that various decoding methods can be unified with the specific implementation of fixed-point inverse quantizers and reference buffers.

[0161] In V-DMC, quantized displacement vector wavelet coefficients can be packed into 2D video frames and decoded using a video encoder or directly using an arithmetic encoder. In either case, the resulting bitstream is encapsulated in the V3C syntax described in WD 1.0 for V3C, ISO / IEC JTC1 / SC29 / WG7, N00797, 4th edition, January 2024 (hereinafter referred to as "WD 1.0"), and is considered a sub-bitstream of that syntax. Examples of other sub-bitstreams include the base mesh sub-bitstream, atlas metadata sub-bitstream, texture sub-bitstream, and other attribute sub-bitstreams.

[0162] To reconstruct the displacement vector wavelet coefficients according to the process presented in the previous paragraphs, the inverse quantization process uses parameters from the bitstream, such as the displacement vector dimension, quantization parameter (QP), number of levels of detail (LoD), number of vertices per LoD, etc. In V-DMC, some of these parameters are signaled as part of the atlas metadata sub-bitstream from WD 6.0, according to the syntax table above.

[0163] On the other hand, the arithmetically decoded (AC) displacement substream WD 6.0 does not include the parameters required for the inverse quantization process to reconstruct the displacement vector wavelet coefficients from the substream without relying on the atlas metadata substream. What is needed for V3C or V-DMC is that the AC displacement substream is self-contained, so that the decoder does not have to rely on other substreams for reconstructing the displacement wavelet coefficients.

[0164] It has been observed that the AC displacement sub-current currently lacks quantization parameters. Therefore, this disclosure proposes the parameters required for self-contained reconstruction by adding displacement wavelet coefficients from the atlas metadata sub-current, such as: asve_displacement_reference_qp (ASPS): The proposed parameter is to add this parameter to DSPS, which is equivalent to ASPS, along with other parameters required for inverse quantization, such as the number of mesh subdivisions and the bit depth of the mesh vertex positions.

[0165] afve_quantization_enable_flag (AFPS): The proposed parameter is to be added to DFPS, which is equivalent to AFPS.

[0166] mdu_quantization_override_flag (meshpatch_data_unit): There are currently no displacement data units defined at the mesh patch level.

[0167] vdmc_quantization_parameters(): This data is not yet included, and most of the parameters may need to be added.

[0168] As shown below, the added content is displayed in the delimiter. <add> and< / add> between:

[0169]

[0170] `dsps_subdivision_method` indicates the identifier of the method used to subdivide the mesh associated with the current displacement sequence parameter set. The following table describes the list of supported subdivision methods and their relationship to `dsps_subdivision_method`.

[0171] Table 1 – List of Subpartitioning Methods

[0172] `dsps_subdivision_iteration_count` indicates the number of iterations used for subdivision. When it does not exist, the value of `dsps_subdivision_iteration_count` is inferred to be 0.

[0173] `dsps_displacement_reference_qp` specifies the initial value of the QuantizationParameter for the current frame. When it does not exist, `asve_displacement_reference_qp` is set to 49.

[0174]

[0175]

[0176] `dqp_lod_quantization_flag[qpIndex]` equal to 1 indicates that incremental decoding is used to transmit quantization parameters at the level of detail. `dqp_lod_quantization_flag[qpIndex]` equal to 0 indicates that the quantization parameters will be the same for all levels of detail. `qpIndex` is the index of the quantization parameter set.

[0177] dqp_bitdepth_offset[ qpIndex ] indicates the bit depth offset value applied to the quantization process of the bit shift. qpIndex is the index of the quantization parameter set.

[0178] `dqp_quantization_parameters[ qpIndex ][ k ]` indicates the quantization parameters to be used for the inverse quantization of the k-th component of the displacement. The value of `dqp_quantization_parameters[ qpIndex ][ k ]` should be in the range of 0 to 100 (inclusive). `qpIndex` is the index of the quantization parameter set.

[0179] `dqp_log2_lod_inverse_scale[ qpIndex ][ k ]` indicates the scaling factor applied to the k-th component of the displacement at each level of detail. `qpIndex` is the index of the quantization parameter set.

[0180] `dqp_lod_delta_quantization_parameter_value[ qpIndex ][ i ][ k ]` specifies the absolute difference between the `dsps_displacement_reference_qp` value and the quantization parameter values ​​for the i-th layer and the k-th component. If it does not exist, the value of `dqp_lod_delta_quantization_parameter_value[ qpIndex ][ i ][ k ]` is inferred to be 0. `qpIndex` is the index of the quantization parameter set. The value of the `QuantizationParameter` for each LoD layer should be in the range of 0 to 100.

[0181] `dqp_lod_delta_quantization_parameter_sign[ qpIndex ][ i ][ k ]` specifies the sign of the difference between the `dsps_displacement_reference_qp` value and the quantization parameter values ​​of the i-th layer and the k-th component. `dqp_lod_delta_quantization_parameter_sign[ qpIndex ][ i ][ k ]` equal to 0 indicates a positive difference. `dqp_lod_delta_quantization_parameter_sign[ qpIndex ][ i ][ k ]` equal to 1 indicates a negative difference. When it does not exist, the value of `dqp_lod_delta_quantization_parameter_sign[ qpIndex ][ i ][ k ]` is inferred to be 0. `qpIndex` is the index of the quantization parameter set.

[0182] The inverse scaling factor can be calculated as follows: bitDepthPosition =<add> dsps_geometry_3d_bit_depth_minus1< / add> +1 InverseScale[ qpIndex ][ i ][ k ] = pow(0.5, 16 + dqp_bitdepth_offset[ qpIndex ] – bitDepthPosition + (4 – QuantizationParameters[ qpIndex ][ i ][ k ])÷ 6) qpIndex is the index of the quantization parameter set.

[0183] Please note that asps_geometry_3d_bit_depth_minus1 is not currently defined in WD 6.0. Therefore, it is proposed to include it in both ASPS and DSPS.

[0184] The increment of asps_geometry_3d_bit_depth_minus1 by 1 indicates the bit depth of the vertex coordinates in the mesh. asps_geometry_3d_bit_depth_minus1[j] should be in the range of 0 to 31 (inclusive).

[0185] The increment of 1 in dsps_geometry_3d_bit_depth_minus1 indicates the bit depth of the vertex coordinates in the mesh. dsps_geometry_3d_bit_depth_minus1[j] should be in the range of 0 to 31 (inclusive).

[0186] In some examples, the minimum bit depth can be set to 4, for example: The addition of 4 to `asps_geometry_3d_bit_depth_minus4` indicates the bit depth of the vertex coordinates in the mesh. `asps_geometry_3d_bit_depth_minus4[j]` should be in the range of 0 to 31 (inclusive).

[0187] dsps_geometry_3d_bit_depth_minus4 plus 4 indicates the bit depth of the vertex coordinates in the mesh. dsps_geometry_3d_bit_depth_minus4[j] should be in the range of 0 to 31 (inclusive).

[0188] Currently, in WD 6.0, the mesh patching described in Danillo B Graziosi, Alexandre Zaghetto, Ali Tabatabai, “[V-DMC][WD] Mesh Patch Concept of V-DMC”, ISO / IEC JTC1 / SC29 / WG7, m65886, January 2024, is not defined in the AC displacement sub-bit stream syntax. Nevertheless, the AC displacement sub-bit stream can be organized and decoded based on the mesh patching concept of the atlas metadata sub-bit stream, and quantization parameters can be signaled within such displacement mesh patch data cells.

[0189] In some examples, when signaling syntax elements associated with shift quantization in the AC shift sub-bitstream and atlas sub-bitstream, bitstream consistency requires that the values ​​of corresponding syntax elements not contradict each other. For example, if dsps_subdivision_iteration_count is 2, then the value of asve_subdivision_iteration_count is also 2.

[0190] In some examples, one or more syntax elements can be signaled in the atlas sub-stream that specify whether the syntax element associated with the shift quantization is signaled in the atlas sub-stream (e.g., a value of 0 indicates that the syntax element is not signaled in the atlas sub-stream, and a value of 1 indicates that the syntax element is signaled in the atlas sub-stream).

[0191] In some examples, only a few syntax elements associated with shift quantization may be signaled in the atlas sub-bit stream without being based on the values ​​of the syntax elements.

[0192] According to the technology disclosed herein, the V-DMC encoder 200 and the V-DMC decoder 300 can perform signaling notification of quantization parameters in the video-decoded displacement sub-bit stream for inter-frame prediction of displacement using fixed-point inverse quantization.

[0193] The technology disclosed herein is applicable to video decoding of displacement. It should be understood that various video decoding methods can be used in conjunction with, for example... Figure 14A and Figure 14B The illustrated fixed-point inverse quantizer and reference buffer are uniformly implemented and are proposed in U.S. Provisional Patent Applications Nos. 63 / 620,665 and 66302.

[0194] In V-DMC, quantized displacement vector wavelet coefficients can be packed into 2D video frames and decoded using a video encoder or directly using an arithmetic encoder. In either case, the resulting bitstream is encapsulated within the V3C syntax described in WD 1.0 and is considered a sub-bitstream of that syntax. Examples of other sub-bitstreams include: the underlying mesh, atlas metadata, textures, and other attributes.

[0195] To reconstruct the wavelet coefficients of the displacement vector, the inverse quantization process uses parameters from the bitstream, such as the displacement vector dimension, quantization parameters (QP), number of levels of detail (LoD), number of vertices per LoD, etc. In V-DMC, some of these parameters are signaled as part of the atlas metadata sub-bitstream from WD 6.0, according to the syntax table above.

[0196] On the other hand, the video-decoded displacement substream, as an attribute substream decoded, does not contain the parameters required for the inverse quantization process to reconstruct the displacement vector wavelet coefficients from this substream without relying on the atlas metadata substream. What is needed for V3C or V-DMC is that the video-decoded displacement substream is self-contained, so that the decoder does not need to rely on other substreams for reconstructing the displacement wavelet coefficients.

[0197] Therefore, this disclosure describes techniques for integrating quantization parameters and other desired parameters in a video frame. These parameters can be losslessly decoded into binary values ​​by assigning them to pixel values ​​in the video frame. If the bit depth of a parameter is greater than the pixel bit depth of the video frame, multiple pixel values ​​can be used to represent the parameter. Generally, the parameter can be decoded in the luminance or chrominance components of the video frame, or in color components such as RGB. The parameter can be grouped into blocks within the video frame, or lines of the video frame, or other regions. Slices or other partitions of the video frame can be dedicated to signaling these parameters. In all cases, the exact format and order of this proposed packing are determined so that the decoder can parse the parameters from the video frame. Parameters can be signaled by sequence, by frame, by subgrid region, by slice, by tile, etc.

[0198] In some examples, quantization parameters and other required parameters can be signaled in an SEI message, which is signaled in the video bitstream.

[0199] It should be recognized that, based on the examples, certain actions or events of any technique described herein may be performed in a different sequence, and may be added, combined, or omitted entirely (e.g., not all actions or events described are necessary for implementing the technique). Furthermore, in some examples, actions or events may be performed concurrently (e.g., through multithreading, interrupt handling, or multiple processors) rather than sequentially.

[0200] Figure 16 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data. Although regarding the V-DMC decoder 300 ( Figure 1 and Figure 3 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 16The process is similar to that of [the other process].

[0201] exist Figure 16 In the example, the V-DMC decoder 300 determines a base mesh based on encoded mesh data (1602). The V-DMC decoder 300 determines one or more displacement vectors based on the encoded mesh data (1604). The V-DMC decoder 300 uses these one or more displacement vectors to deform the base mesh (1606). For example, the base mesh may have a first set of vertices, and the V-DMC decoder 300 may subdivide the base mesh to determine an additional set of vertices. To deform the base mesh, the V-DMC decoder 300 may modify the positions of the additional set of vertices based on one or more displacement vectors. The V-DMC decoder 300 outputs a decoded mesh based on the deformed mesh (1608). The V-DMC decoder 300 may, for example, output the decoded mesh for storage, transmission, or display.

[0202] Figure 17 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data. Although regarding the V-DMC decoder 300 ( Figure 1 and Figure 3 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 17 The process is similar to that of [the other process].

[0203] exist Figure 17 In the example, the V-DMC decoder 300 determines a set of quantization integer coefficient values ​​(1702). The V-DMC decoder 300 determines quantization parameters based on one or more syntax elements included in the shifted sub-bit stream of the encoded dynamic mesh data (1704). To determine the quantization parameters based on one or more syntax elements included in the shifted sub-bit stream of the encoded dynamic mesh data, the V-DMC decoder 300 can be configured to receive a first syntax element indicating the initial value of the quantization parameters for the current frame. To determine the quantization parameters based on one or more syntax elements included in the shifted sub-bit stream of the encoded dynamic mesh data, the V-DMC decoder 300 can be configured to receive a difference value indicating the difference between the initial value of the quantization parameters and the new quantization parameters. For example, the first syntax element can be included in a sequence parameter set of the shifted sub-bit stream.

[0204] The V-DMC decoder 300 inverse-quantizes a set of quantized integer coefficient values ​​to determine a set of fixed-point inverse-quantization coefficient values ​​(1706). The V-DMC decoder 300 can be configured to inverse-quantize a set of quantized integer coefficient values ​​based on quantization parameters to determine a set of fixed-point inverse-quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the shift sub-bit stream. To determine a set of fixed-point transform coefficient values ​​based on a set of fixed-point inverse-quantization coefficient values, the V-DMC decoder 300 can be configured to determine that a set of fixed-point transform coefficient values ​​equals a set of fixed-point inverse-quantization coefficient values. In other examples, to determine a set of fixed-point transform coefficient values ​​based on a set of fixed-point inverse-quantization coefficient values, the V-DMC decoder 300 can be configured to add a set of reference values ​​to a set of fixed-point inverse-quantization coefficient values.

[0205] The V-DMC decoder 300 determines a set of fixed-point transform coefficient values ​​based on a set of fixed-point inverse quantization coefficient values ​​(1708). The V-DMC decoder 300 converts the set of fixed-point transform coefficient values ​​into a set of floating-point transform coefficient values ​​(1710). The V-DMC decoder 300 inversely transforms the set of floating-point transform coefficient values ​​to determine a set of reconstructed displacement vectors (1712). To inversely transform the set of floating-point transform coefficient values ​​to determine a set of reconstructed displacement vectors, the V-DMC decoder 300 can be configured to apply an inverse wavelet transform to the set of floating-point transform coefficient values.

[0206] The V-DMC decoder 300 can store a set of fixed-point transform coefficient values ​​in a reference buffer. Additionally, the V-DMC decoder 300 can determine a second set of quantized integer coefficient values; inverse quantize the second set of quantized integer coefficient values ​​to determine a second set of fixed-point inverse quantization coefficient values; determine a second set of fixed-point transform coefficient values ​​based on a set of fixed-point inverse quantization coefficient values ​​and a set of fixed-point transform coefficient values ​​stored in the reference buffer; convert the second set of fixed-point transform coefficient values ​​to a second set of floating-point transform coefficient values; and inverse transform the second set of floating-point transform coefficient values ​​to determine a second set of reconstructed shift vectors.

[0207] The V-DMC decoder 300 can modify the base mesh based on the reconstructed displacement vector to determine the reconstructed deformed mesh, and apply decoding properties to the reconstructed deformed mesh to determine the reconstructed dynamic mesh sequence. The V-DMC decoder 300 can output the reconstructed dynamic mesh sequence for display, storage, transmission, or other purposes.

[0208] The techniques disclosed herein can be applied to both arithmetic decoding and video-based decoding architectures, or other decoding methods, such as variable-length decoding and neural network-based decoding. It should be understood that various decoding methods can be unified with the specific implementation of fixed-point inverse quantizers and reference buffers.

[0209] The following numbered clauses illustrate one or more aspects of the devices and technologies described in this disclosure.

[0210] The following numbered clauses illustrate one or more aspects of the devices and technologies described in this disclosure.

[0211] Clause 1. An apparatus for decoding encoded dynamic mesh data, the apparatus comprising: one or more memories; and one or more processors implemented in a circuit and communicating with the one or more memories, the one or more processors being configured to: determine a set of quantized integer coefficient values ​​for displacement vectors of the encoded dynamic mesh data; determine quantization parameters based on one or more syntax elements included in a displacement sub-bit stream of the encoded dynamic mesh data; inverse quantize the set of quantized integer coefficient values ​​based on the quantization parameters to determine a set of fixed-point dequantization coefficient values; determine a set of fixed-point transform coefficient values ​​based on the set of fixed-point dequantization coefficient values; convert the set of fixed-point transform coefficient values ​​to a set of floating-point transform coefficient values; inverse transform the set of floating-point transform coefficient values ​​to determine a set of reconstructed displacement vectors; and determine a reconstructed deformable mesh based on the set of reconstructed displacement vectors.

[0212] Clause 2. The apparatus according to Clause 1, wherein, in order to determine the quantization parameter based on the one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data, the one or more processors are configured to: receive a first syntax element indicating an initial value of the quantization parameter at a first level of detail; and receive a second syntax element indicating the difference between the initial value of the quantization parameter at a second level of detail and a new quantization parameter.

[0213] Clause 3. The apparatus according to Clause 2, wherein the first syntax element is included in the sequence parameter set of the shift sub-bit stream.

[0214] Clause 4. The device according to any one of Clauses 1 to 3, wherein, in order to determine the set of fixed-point transform coefficient values ​​based on the set of fixed-point inverse quantization coefficient values, the one or more processors are configured to add the set of fixed-point transform coefficient values ​​to a set of predicted transform coefficient values.

[0215] Clause 5. The device according to any one of Clauses 1 to 4, wherein the one or more processors are configured to: inverse quantize the set of quantized integer coefficient values ​​based on the quantization parameters to determine the set of fixed-point inverse quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the displacement sub-bit stream.

[0216] Clause 6. The device according to any one of Clauses 1 to 5, wherein the one or more processors are further configured to: store the set of fixed-point transform coefficient values ​​in a reference buffer.

[0217] Clause 7. The apparatus according to Clause 6, wherein the one or more processors are further configured to: determine a second set of quantized integer coefficient values; dequantize the second set of quantized integer coefficient values ​​to determine a second set of fixed-point dequantization coefficient values; determine a second set of fixed-point transform coefficient values ​​based on the set of fixed-point dequantization coefficient values ​​and the set of fixed-point transform coefficient values ​​stored in the reference buffer; convert the second set of fixed-point transform coefficient values ​​to a second set of floating-point transform coefficient values; and inverse transform the second set of floating-point transform coefficient values ​​to determine a second set of reconstructed shift vectors.

[0218] Clause 8. The device according to any one of Clauses 1 to 7, wherein, in order to inversely transform the set of floating-point transform coefficient values ​​to determine the set of reconstructed displacement vectors, the one or more processors are further configured to apply an inverse wavelet transform to the set of floating-point transform coefficient values.

[0219] Clause 9. The device according to any one of Clauses 1 to 8, wherein the one or more processors are further configured to: modify the base mesh based on the reconstructed displacement vector to determine the reconstructed deformable mesh.

[0220] Clause 10. The device according to Clause 9, wherein the one or more processors are further configured to: apply decoding properties to the reconstructed deformable mesh to determine a sequence of reconstructed dynamic meshes.

[0221] Clause 11. A method for decoding encoded dynamic mesh data, the method comprising: determining a set of quantized integer coefficient values ​​for displacement vectors of the encoded dynamic mesh data; determining quantization parameters based on one or more syntax elements included in a displacement sub-bit stream of the encoded dynamic mesh data; inverse quantizing the set of quantized integer coefficient values ​​based on the quantization parameters to determine a set of fixed-point dequantization coefficient values; determining a set of fixed-point transform coefficient values ​​based on the set of fixed-point dequantization coefficient values; converting the set of fixed-point transform coefficient values ​​into a set of floating-point transform coefficient values; inverse transforming the set of floating-point transform coefficient values ​​to determine a set of reconstructed displacement vectors; and determining a reconstructed deformable mesh based on the set of reconstructed displacement vectors.

[0222] Clause 12. The method according to Clause 11, wherein determining the quantization parameter based on one or more syntax elements included in the shift sub-bit stream of the encoded dynamic mesh data comprises: receiving a first syntax element indicating an initial value of the quantization parameter for the current frame.

[0223] Clause 13. The method according to Clause 12, wherein determining the quantization parameter based on the one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data comprises: receiving a difference value indicating the difference between the initial value of the quantization parameter and the new quantization parameter.

[0224] Clause 14. The method according to Clause 12, wherein the first syntax element is included in the sequence parameter set of the shifted sub-bit stream.

[0225] Clause 15. The method according to Clause 11, the method further comprising: inverse quantizing the set of quantized integer coefficient values ​​based on the quantization parameters to determine the set of fixed-point inverse quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the displacement sub-bit stream.

[0226] Clause 16. An apparatus for encoding dynamic mesh data, the apparatus comprising: one or more memories; and one or more processors implemented in a circuit and communicating with the one or more memories, the one or more processors being configured to: determine a set of integer coefficient values ​​for a displacement vector of the encoded dynamic mesh data; determine quantization parameters; quantize the set of integer coefficient values ​​based on the quantization parameters to determine a set of quantization coefficient values; and include one or more syntax elements indicative of the quantization parameters in a displacement sub-bit stream of the encoded dynamic mesh data.

[0227] Clause 17. The apparatus according to Clause 16, wherein, in order to include one or more syntax elements indicating the quantization parameter in the shifted sub-bit stream of the encoded dynamic mesh data, the one or more processors are configured to include a first syntax element in the shifted sub-bit stream of the encoded dynamic mesh data, the first syntax element indicating an initial value of the quantization parameter for the current frame.

[0228] Clause 18. The apparatus according to Clause 17, wherein, in order to include one or more syntax elements indicating the quantization parameter in the shift sub-bit stream of the encoded dynamic mesh data, the one or more processors are further configured to include a difference value in the encoded dynamic mesh data, the difference value indicating the difference between the initial value of the quantization parameter and the new quantization parameter.

[0229] Clause 19. The apparatus according to Clause 17, wherein the first syntax element is included in the sequence parameter set of the shifted sub-bit stream.

[0230] Clause 20. The apparatus of Clause 16, wherein the one or more processors are configured to include parameters in the shifted sub-bit stream such that the video decoder inverse-quantizes the set of quantized integer coefficient values ​​to determine a set of fixed-point inverse-quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the shifted sub-bit stream.

[0231] Clause 21. A method for decoding encoded grid data, the method comprising: determining a base grid based on the encoded grid data; determining a set of coefficients using an inter-frame prediction process; receiving quantization parameter values ​​in the encoded grid data; determining an inverse scaling factor based on the quantization parameter values; performing inverse scaling on the set of coefficients based on the inverse scaling factor to determine a dequantized set of coefficients; determining a displacement vector based on the dequantized set of coefficients; deforming the base grid based on the displacement vector to determine a decoded grid; and outputting the decoded grid.

[0232] Clause 22. An apparatus for decoding encoded grid data, the apparatus comprising: one or more memory units; and one or more processing units implemented in a circuit, coupled to the one or more memory units and configured to perform the method described in accordance with Clause 21.

[0233] Clause 23. The device according to Clause 22, the device further includes a display configured to display the decoded mesh.

[0234] Clause 24. An apparatus for decoding encoded dynamic mesh data, the apparatus comprising: one or more memories; and one or more processors implemented in a circuit and communicating with the one or more memories, the one or more processors being configured to: determine a set of quantized integer coefficient values ​​for displacement vectors of the encoded dynamic mesh data; determine quantization parameters based on one or more syntax elements included in a displacement sub-bit stream of the encoded dynamic mesh data; inverse quantize the set of quantized integer coefficient values ​​based on the quantization parameters to determine a set of fixed-point dequantization coefficient values; determine a set of fixed-point transform coefficient values ​​based on the set of fixed-point dequantization coefficient values; convert the set of fixed-point transform coefficient values ​​to a set of floating-point transform coefficient values; inverse transform the set of floating-point transform coefficient values ​​to determine a set of reconstructed displacement vectors; and determine a reconstructed deformable mesh based on the set of reconstructed displacement vectors.

[0235] Clause 25. The apparatus according to Clause 24, wherein, in order to determine the quantization parameter based on the one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data, the one or more processors are configured to: receive a first syntax element indicating an initial value of the quantization parameter at a first level of detail; and receive a second syntax element indicating the difference between the initial value of the quantization parameter at a second level of detail and a new quantization parameter.

[0236] Clause 26. The apparatus according to Clause 25, wherein the first syntax element is included in the sequence parameter set of the shifted sub-bit stream.

[0237] Clause 27. The device according to Clause 24, wherein, in order to determine the set of fixed-point transform coefficient values ​​based on the set of fixed-point inverse quantization coefficient values, the one or more processors are configured to add the set of fixed-point transform coefficient values ​​to a set of predicted transform coefficient values.

[0238] Clause 28. The apparatus of Clause 24, wherein the one or more processors are configured to: inverse quantize the set of quantized integer coefficient values ​​based on the quantization parameters to determine the set of fixed-point inverse quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the shift sub-bit stream.

[0239] Clause 29. The device according to Clause 24, wherein the one or more processors are further configured to store the set of fixed-point transform coefficient values ​​in a reference buffer.

[0240] Clause 30. The apparatus of Clause 29, wherein the one or more processors are further configured to: determine a second set of quantized integer coefficient values; dequantize the second set of quantized integer coefficient values ​​to determine a second set of fixed-point dequantization coefficient values; determine a second set of fixed-point transform coefficient values ​​based on the set of fixed-point dequantization coefficient values ​​and the set of fixed-point transform coefficient values ​​stored in the reference buffer; convert the second set of fixed-point transform coefficient values ​​to a second set of floating-point transform coefficient values; and inverse transform the second set of floating-point transform coefficient values ​​to determine a second set of reconstructed shift vectors.

[0241] Clause 31. The device according to Clause 24, wherein, in order to inversely transform the set of floating-point transform coefficient values ​​to determine the set of reconstructed displacement vectors, the one or more processors are further configured to apply an inverse wavelet transform to the set of floating-point transform coefficient values.

[0242] Clause 32. The device according to Clause 24, wherein the one or more processors are further configured to modify the base mesh based on the reconstructed displacement vector to determine the reconstructed deformable mesh.

[0243] Clause 33. The apparatus of Clause 32, wherein the one or more processors are further configured to: apply decoding attributes to the reconstructed deformable mesh to determine a reconstructed dynamic mesh sequence. 34. A method for decoding encoded dynamic mesh data, the method comprising: determining a set of quantized integer coefficient values ​​for displacement vectors of the encoded dynamic mesh data; determining quantization parameters based on one or more syntax elements included in a displacement sub-bit stream of the encoded dynamic mesh data; inverse quantizing the set of quantized integer coefficient values ​​based on the quantization parameters to determine a set of fixed-point dequantization coefficient values; determining a set of fixed-point transform coefficient values ​​based on the set of fixed-point dequantization coefficient values; converting the set of fixed-point transform coefficient values ​​to a set of floating-point transform coefficient values; inverse transforming the set of floating-point transform coefficient values ​​to determine a set of reconstructed displacement vectors; and determining a reconstructed deformable mesh based on the set of reconstructed displacement vectors.

[0244] Clause 35. The method according to Clause 34, wherein determining the quantization parameter based on one or more syntax elements included in the shift sub-bit stream of the encoded dynamic mesh data comprises: receiving a first syntax element indicating an initial value of the quantization parameter for the current frame.

[0245] Clause 36. The method according to Clause 35, wherein determining the quantization parameter based on one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data comprises: receiving a difference value indicating the difference between the initial value of the quantization parameter and the new quantization parameter.

[0246] Clause 37. The method according to Clause 35, wherein the first syntax element is included in the sequence parameter set of the shifted sub-bit stream.

[0247] Clause 38. The method according to Clause 34, further comprising: inverse quantizing the set of quantized integer coefficient values ​​based on the quantization parameters to determine the set of fixed-point inverse quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the displacement sub-bit stream. 39. An apparatus for encoding dynamic mesh data, the apparatus comprising: one or more memories; and one or more processors implemented in a circuit and communicating with the one or more memories, the one or more processors being configured to: determine a set of integer coefficient values ​​for the displacement vector of the encoded dynamic mesh data; determine quantization parameters; quantize the set of integer coefficient values ​​based on the quantization parameters to determine a set of quantization coefficient values; and include one or more syntax elements indicating the quantization parameters in the displacement sub-bit stream of the encoded dynamic mesh data.

[0248] Clause 40. The apparatus according to Clause 39, wherein, in order to include one or more syntax elements indicating the quantization parameter in the shifted sub-bit stream of the encoded dynamic mesh data, the one or more processors are configured to include a first syntax element in the shifted sub-bit stream of the encoded dynamic mesh data, the first syntax element indicating an initial value of the quantization parameter for the current frame.

[0249] Clause 41. The apparatus according to Clause 40, wherein, in order to include one or more syntax elements indicating the quantization parameter in the shift sub-bit stream of the encoded dynamic mesh data, the one or more processors are further configured to include a difference value in the encoded dynamic mesh data, the difference value indicating the difference between the initial value of the quantization parameter and the new quantization parameter.

[0250] Clause 42. The apparatus according to Clause 40, wherein the first syntax element is included in the sequence parameter set of the shifted subbit stream.

[0251] Clause 43. The apparatus of Clause 39, wherein the one or more processors are configured to include parameters in the shifted sub-bit stream such that the video decoder inverse-quantizes the set of quantized integer coefficient values ​​to determine a set of fixed-point inverse-quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the shifted sub-bit stream.

[0252] Clause 44. A method for encoding dynamic mesh data, the method comprising: determining a set of integer coefficient values ​​for a displacement vector of the encoded dynamic mesh data; determining quantization parameters; quantizing the set of integer coefficient values ​​based on the quantization parameters to determine a set of quantization coefficient values; and including one or more syntax elements indicating the quantization parameters in a displacement sub-bit stream of the encoded dynamic mesh data.

[0253] Clause 45. The method according to Clause 44, wherein including one or more syntax elements in the shift sub-stream of the encoded dynamic mesh data indicating the quantization parameter comprises: including a first syntax element in the shift sub-stream of the encoded dynamic mesh data, the first syntax element indicating an initial value of the quantization parameter for the current frame.

[0254] Clause 46. The method according to Clause 45, wherein including one or more syntax elements in the displacement sub-bit stream of the encoded dynamic mesh data indicating the quantization parameter includes: including a difference value in the encoded dynamic mesh data, the difference value indicating the difference between the initial value of the quantization parameter and the new quantization parameter.

[0255] Clause 47. The method according to Clause 45, wherein the first syntax element is included in the sequence parameter set of the shifted sub-bit stream.

[0256] Clause 48. The method according to Clause 44, further comprising: including parameters in the shifted sub-bit stream such that the video decoder inverse-quantizes the set of quantized integer coefficient values ​​to determine a set of fixed-point inverse-quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the shifted sub-bit stream.

[0257] It should be recognized that, based on the examples, certain actions or events of any technique described herein may be performed in a different sequence, and may be added, combined, or omitted entirely (e.g., not all actions or events described are necessary for implementing the technique). Furthermore, in some examples, actions or events may be performed concurrently (e.g., through multithreading, interrupt handling, or multiple processors) rather than sequentially.

[0258] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0259] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead refer to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs utilize lasers to optically reproduce data. The combinations described above should also be included within the scope of computer-readable media.

[0260] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuit" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0261] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or IC sets (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but implementation by different hardware units is not necessarily required. Rather, as described above, various units can be combined in a codec hardware unit, or various units can be provided by a collection of interoperable hardware units (including one or more processors as described above) combined with appropriate software and / or firmware.

[0262] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. An apparatus for decoding encoded dynamic grid data, the apparatus comprising: One or more memory units; and One or more processors, implemented in a circuit and communicating with one or more memories, are configured to: Determine a set of quantized integer coefficient values ​​for the displacement vector of the encoded dynamic mesh data; The quantization parameters are determined based on one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data; Based on the quantization parameters, the set of quantized integer coefficient values ​​is dequantized to determine a set of fixed-point dequantization coefficient values; A set of fixed-point transform coefficient values ​​is determined based on the aforementioned set of fixed-point dequantization coefficient values; Convert the set of fixed-point transformation coefficient values ​​into a set of floating-point transformation coefficient values; The inverse transformation of the set of floating-point transformation coefficients determines a set of reconstructed displacement vectors; and The reconstructed deformable mesh is determined based on the set of reconstructed displacement vectors.

2. The apparatus of claim 1, wherein, in order to determine the quantization parameters based on the one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data, the one or more processors are configured to: Receive a first syntax element, the first syntax element indicating the initial value of the quantization parameter at a first level of detail; and Receive a second syntax element, which indicates the difference between the initial value of the quantization parameter at the second level of detail and the new quantization parameter.

3. The device of claim 2, wherein the first syntax element is included in the sequence parameter set of the shifted sub-bit stream.

4. The device of claim 1, wherein, in order to determine the set of fixed-point transform coefficient values ​​based on the set of fixed-point inverse quantization coefficient values, the one or more processors are configured to add the set of fixed-point transform coefficient values ​​to a set of predicted transform coefficient values.

5. The device of claim 1, wherein the one or more processors are configured to: inverse quantize the set of quantized integer coefficient values ​​based on the quantization parameters to determine the set of fixed-point inverse quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the displacement sub-bit stream.

6. The device of claim 1, wherein the one or more processors are further configured to: The set of fixed-point transformation coefficient values ​​are stored in a reference buffer.

7. The device of claim 6, wherein the one or more processors are further configured to: Determine the second set of quantized integer coefficient values; Dequantize the second set of quantized integer coefficient values ​​to determine the second set of fixed-point dequantization coefficient values; The second set of fixed-point transform coefficient values ​​is determined based on the first set of fixed-point dequantization coefficient values ​​and the first set of fixed-point transform coefficient values ​​stored in the reference buffer; Convert the second set of fixed-point transformation coefficient values ​​into the second set of floating-point transformation coefficient values; as well as The second set of floating-point transformation coefficients are inversely transformed to determine the second set of reconstructed displacement vectors.

8. The device of claim 1, wherein, in order to inversely transform the set of floating-point transform coefficient values ​​to determine the set of reconstructed displacement vectors, the one or more processors are further configured to apply an inverse wavelet transform to the set of floating-point transform coefficient values.

9. The device of claim 1, wherein the one or more processors are further configured to: modify the base mesh based on the reconstructed displacement vector to determine the reconstructed deformable mesh.

10. The device of claim 9, wherein the one or more processors are further configured to: apply decoding attributes to the reconstructed deformable mesh to determine a sequence of reconstructed dynamic meshes.

11. A method for decoding encoded dynamic grid data, the method comprising: Determine a set of quantized integer coefficient values ​​for the displacement vector of the encoded dynamic mesh data; The quantization parameters are determined based on one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data; Based on the quantization parameters, the set of quantized integer coefficient values ​​is dequantized to determine a set of fixed-point dequantization coefficient values; A set of fixed-point transform coefficient values ​​is determined based on the aforementioned set of fixed-point dequantization coefficient values; Convert the set of fixed-point transformation coefficient values ​​into a set of floating-point transformation coefficient values; The inverse transformation of the set of floating-point transformation coefficients determines a set of reconstructed displacement vectors; and The reconstructed deformable mesh is determined based on the set of reconstructed displacement vectors.

12. The method of claim 11, wherein determining the quantization parameters based on the one or more syntax elements included in the displacement sub-bitstream of the encoded dynamic mesh data comprises: Receive a first syntax element, which indicates the initial value of the quantization parameter for the current frame.

13. The method of claim 12, wherein determining the quantization parameters based on the one or more syntax elements included in the displacement sub-bit stream of the encoded dynamic mesh data comprises: Receive a difference value, which indicates the difference between the initial value of the quantization parameter and the new quantization parameter.

14. The method of claim 12, wherein the first syntax element is included in the sequence parameter set of the shifted sub-bit stream.

15. The method according to claim 11, further comprising: The set of quantized integer coefficient values ​​is inversely quantized based on the quantization parameters to determine the set of fixed-point inverse quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the displacement sub-bit stream.

16. An apparatus for encoding dynamic grid data, the apparatus comprising: One or more memory units; and One or more processors, implemented in a circuit and communicating with one or more memories, are configured to: Determine a set of integer coefficient values ​​for the displacement vector of the encoded dynamic mesh data; Determine the quantization parameters; The set of integer coefficient values ​​is quantized based on the quantization parameters to determine a set of quantized coefficient values; as well as The encoded dynamic grid data includes one or more syntax elements that indicate the quantization parameters in the displacement sub-bit stream.

17. The apparatus of claim 16, wherein, in order to include one or more syntax elements indicating the quantization parameters in the shifted sub-bit stream of the encoded dynamic mesh data, the one or more processors are configured to include a first syntax element in the shifted sub-bit stream of the encoded dynamic mesh data, the first syntax element indicating an initial value of the quantization parameters for the current frame.

18. The apparatus of claim 17, wherein, in order to include one or more syntax elements indicating the quantization parameter in the shift sub-bit stream of the encoded dynamic mesh data, the one or more processors are further configured to include a difference value in the encoded dynamic mesh data, the difference value indicating the difference between the initial value of the quantization parameter and the new quantization parameter.

19. The apparatus of claim 17, wherein the first syntax element is included in the sequence parameter set of the shifted sub-bit stream.

20. The apparatus of claim 16, wherein the one or more processors are configured to include parameters in the shifted sub-bit stream such that the video decoder inverse-quantizes the set of quantized integer coefficient values ​​to determine a set of fixed-point inverse-quantization coefficient values ​​without referencing any parameters included in the sub-bit stream other than the shifted sub-bit stream.

Images