V-DMC shift boost shift

Abstract

An apparatus for decoding encoded grid data is configured to: receive one or more syntax elements in a bitstream of encoded grid data; determining an offset value based on the one or more syntax elements; determining a transformation coefficient set; applying the offset to the set of transform coefficients to determine an updated set of transform coefficients; performing inverse transformation on the updated transformation coefficient set to determine a displacement vector set; and determining a decoding grid based on the set of displacement vectors.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 882,516, filed September 11, 2024; U.S. Provisional Patent Application No. 63 / 589,192, filed October 10, 2023; U.S. Provisional Patent Application No. 63 / 590,679, filed October 16, 2023; and U.S. Provisional Patent Application No. 63 / 621,478, filed January 16, 2024, the entire contents of which are incorporated herein by reference. U.S. Patent Application No. 18 / 882,516 claims the benefit of U.S. Provisional Patent Application Nos. 63 / 589,192, 63 / 590,679, and 63 / 621,478. Technical Field

[0002] This disclosure relates to video-based dynamic mesh decoding. Background Technology

[0003] Meshes can be used to represent the physical content of three-dimensional space. Meshes are useful in a variety of situations. For example, they can be used in the context of representing the physical content of an environment to locate virtual objects in extended reality applications such as augmented reality (AR), virtual reality (VR), or mixed reality (MR). Mesh compression is the process of encoding and decoding meshes. Encoding meshes reduces the amount of data required to store and transmit them. Summary of the Invention

[0004] This disclosure relates to video-based dynamic mesh decoding (V-DMC), and more specifically to signaling notification of displacement vectors. As will be explained in more detail below, a V-DMC encoder can be configured to transform the values ​​of the displacement vectors, for example using a wavelet transform employing a lifting scheme, to generate a set of transform coefficients corresponding to a certain dimension (e.g., X, Y, or Z) of these transform coefficients. However, this lifting scheme may introduce a bias into the transform coefficients. In this context, a bias refers to the fact that the average value obtained by the lifting transform is not zero.

[0005] According to the technology disclosed herein, a V-DMC encoder can be configured to: determine an offset in the transform coefficients, determine an offset based on the offset, and subtract the offset from the transform coefficients to determine offset-adjusted transform coefficients. In most decoding scenarios, offset adjustment causes the offset-adjusted transform coefficients to have more values ​​equal to or close to zero, and reduces the average magnitude of these values. The video encoder can then quantize the offset-adjusted transform coefficients (rather than the original transform coefficients). By making more values ​​equal to or close to zero, these values ​​can be signaled with fewer bits, and quantization errors are reduced, thereby improving the overall rate-distortion tradeoff achieved in the encoding and decoding processes.

[0006] The V-DMC encoder can also signal the V-DMC decoder to indicate the offset values ​​used during the encoding process. Therefore, after the V-DMC decoder dequantizes the offset-adjusted transform coefficients, it can add the offset to the dequantized offset-adjusted transform coefficients, effectively adding the offset and obtaining a decoded version of the original transform coefficients. Since the quantization and dequantization processes can be lossy, the decoded version of the original transform coefficients may not perfectly match the original transform coefficients determined by the V-DMC encoder. The V-DMC decoder can then perform an inverse transform on the transform applied by the V-DMC encoder, such as an inverse transform on a wavelet transform using a lifting scheme, to determine a decoded version of the displacement vector. Although the techniques of this disclosure can be applied to any component of the displacement vector, it has been observed that the described techniques may be particularly advantageous when used for the normal component of the normal vector (typically identified as the x-component).

[0007] According to an example of this disclosure, an apparatus for decoding coded grid data includes: a memory; and processing circuitry coupled to the memory and configured to: receive one or more syntax elements in a bitstream of the coded grid data; determine an offset value based on the one or more syntax elements; determine a set of transform coefficients; apply the offset to the set of transform coefficients to determine an updated set of transform coefficients; perform an inverse transform on the updated set of transform coefficients to determine a set of displacement vectors; and determine a decoded grid based on the set of displacement vectors.

[0008] According to an example of this disclosure, a method for decoding coded grid data includes: receiving one or more syntax elements in a bit stream of the coded grid data; determining an offset value based on the one or more syntax elements; determining a set of transform coefficients; applying the offset to the set of transform coefficients to determine an updated set of transform coefficients; performing an inverse transform on the updated set of transform coefficients to determine a set of shift vectors; and determining a decoded grid based on the set of shift vectors. According to an example of this disclosure, an apparatus for encoding grid data includes: a memory; and processing circuitry coupled to the memory and configured to: determine a set of shift vectors of the grid data; transform the set of shift vectors to determine a set of transform coefficients; determine an offset value for the set of transform coefficients; determine an offset value based on the offset value for the set of transform coefficients; subtract the offset value from the set of transform coefficients to determine offset-adjusted transform coefficients; quantize the offset-adjusted transform coefficients to determine quantization coefficients; and signal the quantization coefficients and an indication of the offset in a bit stream of the coded grid data.

[0009] According to an example of this disclosure, a method for encoding grid data includes: determining a set of displacement vectors of the grid data; transforming the set of displacement vectors to determine a set of transform coefficients; determining an offset value of the set of transform coefficients; determining an offset value based on the offset value of the set of transform coefficients; subtracting the offset value from the set of transform coefficients to determine transform coefficients for offset adjustment; quantizing the transform coefficients for offset adjustment to determine quantization coefficients; and signaling the quantization coefficients and the offset indication in a bit stream of the encoded grid data.

[0010] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0011] Figure 1 This is a block diagram illustrating an example encoding and decoding system that can perform the techniques of this disclosure.

[0012] Figure 2 An example implementation of the V-DMC encoder is shown.

[0013] Figure 3 An example implementation of the V-DMC decoder is shown.

[0014] Figure 4 An example of efficient compression of 2D curves through resampling is shown.

[0015] Figure 5 The displacement curves with a subdivision structure are shown, which approximate the shape of the original mesh.

[0016] Figure 6 A block diagram of the preprocessing system is shown.

[0017] Figure 7 An example of a V-DMC intra-frame encoder is shown.

[0018] Figure 8 An example of a V-DMC decoder is shown.

[0019] Figure 9 An example of a V-DMC intra-frame decoder is shown.

[0020] Figure 10 An example of a midpoint partitioning scheme is shown.

[0021] Figure 11 An example implementation of the forward lifting transformation is shown.

[0022] Figure 12 This is a flowchart illustrating an example process for encoding a mesh.

[0023] Figure 13 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data.

[0024] Figure 14 This is a flowchart illustrating an example process for encoding a mesh.

[0025] Figure 15 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data. Detailed Implementation

[0026] A mesh is typically a collection of vertices in three-dimensional (3D) space that collectively represent one or more objects in that space. Vertices are connected by edges, and these edges form polygons that make up the faces of the mesh. Each vertex may also have one or more associated properties, such as texture or color. In most scenarios, having more vertices produces a higher quality mesh (e.g., more detailed and more realistic). However, having more vertices also requires more data to represent the mesh.

[0027] To reduce the amount of data required to represent a grid, lossy or lossless encoding can be used to encode the grid. In lossless encoding, the decoded version of the encoded grid perfectly matches the original grid. In contrast, in lossy encoding, the process of encoding and decoding the grid introduces losses such as distortion in the decoded version of the encoded grid.

[0028] In one example of a lossy encoding technique used for meshes, a mesh encoder decimates the original mesh to determine the base mesh. To decimate the original mesh, the mesh encoder subsamples or otherwise reduces the number of vertices in the original mesh, making the base mesh a coarse approximation of the original mesh with fewer vertices. The mesh encoder then subdivides the decimated mesh. That is, the mesh encoder estimates the positions of additional vertices between the vertices of the base mesh. The mesh encoder then deforms the subdivided mesh by moving the vertices, making the deformed mesh more closely match the original mesh.

[0029] After determining the desired base mesh and the deformation of the sub-mesh, the mesh encoder generates a bitstream comprising data for constructing the base mesh and data for performing the deformation. The deformation data is signaled as a series of displacement vectors indicating the movement or displacement of additional vertices determined by the sub-meshing process. To decode the mesh from the bitstream, the mesh decoder reconstructs the base mesh based on the signaled information, applies the same sub-meshing process as the mesh encoder, and then displaces the additional vertices based on the signaled displacement vectors.

[0030] This disclosure relates to signaling notification of displacement vectors for video-based dynamic mesh decoding (V-DMC). As explained in more detail below, a V-DMC encoder can be configured to transform the values ​​of the displacement vectors, for example using a wavelet transform employing a lifting scheme, to generate a set of transform coefficients corresponding to a certain dimension (e.g., X, Y, or Z) of these transform coefficients. However, this lifting scheme may introduce a bias into the transform coefficients. In this context, a bias refers to the fact that the average value obtained by the lifting transform is not zero.

[0031] According to the technology disclosed herein, a V-DMC encoder can be configured to: determine an offset in the transform coefficients, determine an offset based on the offset, and subtract the offset from the transform coefficients to determine offset-adjusted transform coefficients. In most decoding scenarios, offset adjustment causes the offset-adjusted transform coefficients to have more values ​​equal to or close to zero, and reduces the average magnitude of these values. The video encoder can then quantize the offset-adjusted transform coefficients (rather than the original transform coefficients). By making more values ​​equal to or close to zero, these values ​​can be signaled with fewer bits, and quantization errors are reduced, thereby improving the overall rate-distortion tradeoff achieved in the encoding and decoding processes.

[0032] The V-DMC encoder can also signal the V-DMC decoder to indicate the offset values ​​used during the encoding process. Therefore, after the V-DMC decoder dequantizes the offset-adjusted transform coefficients, it can add the offset to the dequantized offset-adjusted transform coefficients, effectively adding the offset and obtaining a decoded version of the original transform coefficients. Since the quantization and dequantization processes can be lossy, the decoded version of the original transform coefficients may not perfectly match the original transform coefficients determined by the V-DMC encoder. The V-DMC decoder can then perform an inverse transform on the transform applied by the V-DMC encoder, such as an inverse transform on a wavelet transform using a lifting scheme, to determine a decoded version of the displacement vector. Although the techniques of this disclosure can be applied to any component of the displacement vector, it has been observed that the described techniques may be particularly advantageous when used for the normal component of the normal vector (typically identified as the x-component).

[0033] Figure 1 This is a block diagram illustrating an example encoding and decoding system 100 that can perform the techniques of this disclosure. The techniques of this disclosure generally involve decoding (encoding and / or decoding) grids. This decoding can be efficient in compressing and / or decompressing grid data.

[0034] like Figure 1 As shown, system 100 includes source device 102 and destination device 116. Source device 102 provides encoded data for decoding by destination device 116. Specifically, in Figure 1In this example, source device 102 provides data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 can include any of a wide range of devices, including desktop computers, laptop computers, tablet computers, set-top boxes, mobile phones (such as smartphones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, land or sea vehicles, spacecraft, aircraft, robots, LiDAR devices, satellites, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication.

[0035] exist Figure 1 In the example, source device 102 includes a data source 104, a memory 106, a V-DMC encoder 200, and an output interface 108. Destination device 116 includes an input interface 122, a V-DMC decoder 300, a memory 120, and a data consumer 118. According to this disclosure, the V-DMC encoder 200 of source device 102 and the V-DMC decoder 300 of destination device 116 can be configured to apply the techniques related to shift vector quantization disclosed herein. Therefore, source device 102 represents an example of an encoding device, while destination device 116 represents an example of a decoding device. In other examples, source device 102 and destination device 116 may include other components or arrangements. For example, source device 102 may receive data from an internal or external source. Similarly, destination device 116 may interface with an external data consumer, rather than including the data consumer in the same device.

[0036] like Figure 1 The system 100 shown is merely an example. In general, other digital encoding and / or decoding devices can perform the techniques related to shift vector quantization disclosed herein. Source device 102 and destination device 116 are merely examples of such devices, where source device 102 generates decoded data for transmission to destination device 116. This disclosure refers to a “decoding” device as a device that performs the decoding (e.g., encoding and / or decoding) of data. Thus, V-DMC encoder 200 and V-DMC decoder 300 represent examples of decoding devices, specifically, encoder and decoder, respectively. In some examples, source device 102 and destination device 116 can operate in a substantially symmetrical manner, such that each of source device 102 and destination device 116 includes both encoding and decoding components. Therefore, system 100 can support one-way or two-way transmission between source device 102 and destination device 116, for example, for streaming, playback, broadcasting, telephone, navigation, and other applications.

[0037] Generally, data source 104 represents the source of data (e.g., raw, unencoded data) and provides a series of sequential "frames" of data to V-DMC encoder 200, which encodes the data in the frames. Data source 104 may, for example, execute a framework or platform for generating graphics for video games, augmented reality, simulations, or any other such use cases. Data source 104 of source device 102 may include a graphics engine that generates raw mesh data from any combination of one or more sensors configured to obtain real-world data. Examples of such sensors include cameras, 2D scanners, 3D scanners, light detection and ranging (LIDAR) devices, video cameras, ultrasonic sensors, infrared sensors, inertial measurement sensors, sonar sensors, pressure sensors, thermal imaging sensors, magnetic sensors, laser rangefinders, photodetectors, etc. In other examples, the graphics engine may use modeling, simulation, animation, generative adversarial networks, etc., to generate entirely computer-generated meshes (i.e., not representing real-world scenes). In yet another example, data source 104 may not include a graphics engine and may instead obtain mesh data from storage units or other devices.

[0038] Regardless of whether the grid data is based on real-world sensor data, entirely computer-generated, obtained from an external source, or a combination thereof, the V-DMC encoder 200 can encode the grid data. The V-DMC encoder 200 can rearrange frames from their received order (sometimes referred to as "display order") to a decoding order for decoding. The V-DMC encoder 200 can generate one or more bit streams including the encoded data. The source device 102 can then output the encoded data via output interface 108 to a computer-readable medium 110 for reception and / or retrieval, for example, by an input interface 122 of a destination device 116.

[0039] The memory 106 of source device 102 and the memory 120 of destination device 116 may represent general-purpose memory. In some examples, memory 106 and memory 120 may store raw data, such as raw data from data source 104 and raw decoded data from V-DMC decoder 300. Additionally or alternatively, memory 106 and memory 120 may store software instructions, respectively executable by, for example, V-DMC encoder 200 and V-DMC decoder 300. Although memory 106 and memory 120 are shown separately from V-DMC encoder 200 and V-DMC decoder 300 in this example, it should be understood that V-DMC encoder 200 and V-DMC decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memory 106 and memory 120 may store encoded data, such as output from V-DMC encoder 200 and input to V-DMC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, for example, to store raw decoded and / or encoded data. For instance, memory 106 and memory 120 may store data representing a grid.

[0040] Computer-readable medium 110 may represent any type of medium or device capable of transmitting encoded data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium enabling source device 102 to transmit encoded data directly to destination device 116 in real time, for example, via a radio frequency network or a computer-based network. Output interface 108 may modulate the transmitted signal including the encoded data, and input interface 122 may demodulate the received transmitted signal according to a communication standard, such as a wireless communication protocol. The communication medium may include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network such as a local area network, a wide area network, or a global network (such as the Internet). The communication medium may include a router, a switch, a base station, or any other equipment that facilitates communication from source device 102 to destination device 116.

[0041] In some examples, source device 102 can output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0042] In some examples, source device 102 may output encoded data to file server 114 or another intermediate storage device that may store the encoded data generated by source device 102. Destination device 116 may access the stored data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing encoded data and sending it to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded data from file server 114 via any standard data connection, including an internet connection. This standard data connection may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a downloading protocol, or a combination thereof.

[0043] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (such as encoded data) according to cellular communication standards (such as 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to comply with specifications such as IEEE 802.11, IEEE 802.15 (e.g., ZigBee). ™ ),Bluetooth ™ Data (such as encoded data) is transmitted using other wireless standards, such as standard V-DMC encoder 200. In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include an SoC device for performing functions belonging to V-DMC encoder 200 and / or output interface 108, and destination device 116 may include an SoC device for performing functions belonging to V-DMC decoder 300 and / or input interface 122.

[0044] The technology disclosed herein can be applied to support encoding and decoding of any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices such as local or remote servers, geographic mapping or other applications.

[0045] The input interface 122 of the destination device 116 receives an encoded bitstream from a computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, etc.). The encoded bitstream may include signaling information defined by the V-DMC encoder 200 and also used by the V-DMC decoder 300, such as syntax elements having characteristics and / or processed values ​​describing the units being decoded (e.g., slices, pictures, picture groups, sequences, etc.). The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display that presents an image based on a grid.

[0046] The V-DMC encoder 200 and V-DMC decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic components, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the V-DMC encoder 200 and V-DMC decoder 300 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (codec) in the respective device. Devices including the V-DMC encoder 200 and / or V-DMC decoder 300 may include one or more integrated circuits, microprocessors, and / or other types of devices.

[0047] The V-DMC encoder 200 and V-DMC decoder 300 operate according to a decoding standard. This disclosure generally relates to the decoding (e.g., encoding and decoding) of images, thus including the process of encoding or decoding data. The encoded bitstream typically includes a series of values ​​for syntax elements representing decoding decisions (e.g., decoding modes).

[0048] This disclosure may generally relate to "signaling notification" of certain information (such as syntax elements). The term "signaling notification" generally refers to communication of the value of a syntax element and / or other data used to decode encoded data. That is, the V-DMC encoder 200 may signal the value of a syntax element in the bit stream. Typically, signaling notification refers to the generation of a value in the bit stream. As noted above, the source device 102 may transmit the bit stream to the destination device 116 substantially in real time or not in real time (such as when the syntax element is stored in storage device 112 for later retrieval by the destination device 116).

[0049] This disclosure relates to various improvements to the displacement vector quantization process in video-based dynamic mesh decoding (V-DMC) technology, which is being standardized in MPEG WG7 (3DGH).

[0050] MPEG Working Group 6 (WG7) (also known as the 3D Graphics and Haptic Decoding Group (3DGH)) is currently standardizing a video-based dynamic mesh decoding representation (V-DMC) for XR use cases. The current test model is based on the results of a call for proposals by Khaled Mammou, Jungsun Kim, Alexandros Tourapis, Dimitri Podborski, Krasimir Kolarov, [V-CG] Apple's Dynamic Mesh Coding CfP Response, ISO / IEC JTC1 / SC29 / WG7, m59281, April 2022, and covers preprocessing the input mesh into an approximate mesh (called the base mesh) with typically fewer vertices, which is then decoded using a static mesh decoder (cfr. Draco et al.). Additionally, the encoder estimates the motion of the base mesh vertices and decodes the motion vectors into a bitstream. The reconstructed base mesh can be subdivided into a finer mesh with additional vertices, and thus subdivided into additional triangles. The encoder can refine the positions of the subdivided mesh vertices to approximate the original mesh. The thinned or vertex displacement vectors can be decoded into the bitstream. In the current test model, the displacement vectors are wavelet transformed, quantized, and the coefficients are packed into 2D frames. The frame sequence is decoded into the bitstream using a typical video decoder (e.g., HEVC or VVC). Additionally, the texture frame sequence is decoded using a video decoder.

[0051] Figure 2 and Figure 3 The encoder for the current V-DMC test model (TM) is shown. Figure 2 V-DM encoder 200 and decoder (in the middle) Figure 3 The overall system model of the V-DMC decoder 300 architecture is shown below. The V-DMC encoder 200 performs volumetric media conversion, and the V-DMC decoder 300 performs the corresponding reconstruction. The 3D media is converted into a series of sub-bitstreams: base mesh, displacement, and texture attributes. As described in N00680, the bitstreams also include additional atlas information to enable inverse reconstruction.

[0052] Figure 2 An example implementation of the V-DMC encoder 200 is shown. Figure 2In the example, the V-DMC encoder 200 includes a preprocessing unit 204, an atlas encoder 208, a base mesh encoder 212, a displacement encoder 216, and a video encoder 220. The preprocessing unit 204 receives an input mesh sequence and generates a base mesh, displacement vectors, and a texture attribute map. The base mesh encoder 212 encodes the base mesh. The displacement encoder 216 encodes the displacement vectors, for example, as V3C video components, or using arithmetic displacement decoding. The video encoder 220 encodes the texture attribute components (e.g., texture or material information) using any video codec, such as the High Efficiency Video Decoding (HEVC) standard or the Multi-Functional Video Decoding (VVC) standard.

[0053] The V-DMC encoder 200 will now be described in more detail. The preprocessing unit 204 represents the 3D volumetric data as a set of base meshes and corresponding refined components. This is achieved by converting the input dynamic mesh representation into multiple V3C components, including: the base mesh, the displacement set, a 2D representation of the texture map, and the atlas. The base mesh components are a simplified, low-resolution approximation of the original mesh in lossy compression, and the original mesh itself in lossless compression. The base mesh components can be encoded by the base mesh encoder 212 using any mesh codec.

[0054] Basic mesh encoder 212 in Figure 4 The representation is a static mesh encoder, and the underlying mesh is encoded using a specific implementation of the Edgebreaker algorithm (e.g., m63344), where connectivity is encoded using CLERS opcodes (e.g., CLERS opcodes proposed by Rossignac and Lopes), and attribute residuals are encoded using predictions of the attributes of previously encoded / decoded vertices.

[0055] The various aspects of the base mesh encoder 212 will now be described in more detail. One or more sub-meshes are input into the base mesh encoder 212. The sub-meshes are generated by the preprocessing unit 204. Sub-meshes are generated from the original mesh through semantic segmentation. Each base mesh may include one or more sub-meshes.

[0056] The basic mesh encoder 212 can process connected components. Connected components consist of clusters of triangles connected by adjacent triangles. A submesh may have one or more connected components. The basic mesh encoder 212 can encode the connectivity and properties of one "connected component" at a time, and then perform entropy encoding on all "connected components".

[0057] The base mesh encoder 212 defines and categorizes the input base mesh into connectivity and attributes. Geometry and texture coordinates (UV coordinates) are categorized as attributes.

[0058] Figure 3 An example implementation of the V-DMC decoder 300 is shown. Figure 3 In the example, the V-DMC decoder 300 includes a demultiplexer 304, an atlas decoder 308, a basic mesh decoder 314, a displacement decoder 316, a video decoder 320, a basic mesh processing unit 324, a displacement processing unit 328, a mesh generation unit 332, and a reconstruction unit 336.

[0059] Demultiplexer 304 separates the encoded bitstream into atlas sub-bitstream, base mesh sub-bitstream, displacement sub-bitstream, and texture attribute sub-bitstream. Atlas decoder 308 decodes the atlas sub-bitstream to determine atlas information, thereby achieving inverse reconstruction. Base mesh decoder 314 decodes the base mesh sub-bitstream, and base mesh processing unit 324 reconstructs the base mesh. Displacement decoder 316 decodes the displacement sub-bitstream, and displacement processing unit 328 reconstructs the displacement vector. Mesh generation unit 332 modifies the base mesh based on the displacement vector to form a displacement mesh.

[0060] The video decoder 320 decodes the texture attribute sub-stream to determine the texture attribute map, and the reconstruction unit 336 associates the texture attributes with the displacement mesh to form a reconstructed dynamic mesh.

[0061] A detailed description of the proposal chosen as the starting point for V-DMC standardization can be found in m59281. The following description details the current V-DMC test model and displacement vector decoding in WD 2.0.

[0062] Preprocessing systems (such as those related to...) Figure 6 The described preprocessing system 600 can be configured to perform preprocessing on the input grid M(i). Figure 4 The basic idea behind the proposed preprocessing scheme is illustrated using 2D curves. The same concept can be applied to the input 3D mesh. To generate the base mesh and displacement field .

[0063] exist Figure 4First, the input 2D curve (represented by a 2D polyline) referred to as the original curve 402 is downsampled to generate a base curve / polyline referred to as the decimation curve 404. Then, a sub-partitioning scheme (such as the sub-partitioning scheme described in Garland et al.'s SurfaceSimplification Using Quadric Error Metrics (https: / / www.cs.cmu.edu / ~garland / Papers / quadrics.pdf)) is applied to the decimation polyline to generate the sub-partitioning curve 406. For example, in Figure 4 In this example, a subpartitioning scheme employing an iterative interpolation method is used. This subpartitioning scheme inserts a new point in the middle of each edge of the polyline during each iteration. In the illustrated example, two subpartitioning iterations are applied.

[0064] The proposed scheme is independent of the chosen sub-partitioning scheme and can be combined with other sub-partitioning schemes. Then, the polygonal lines of the sub-partitions are deformed or shifted to obtain a better approximation of the original curve. This better approximation is... Figure 4 The displacement curve 408 in the image. The displacement vector is calculated for each vertex of the sub-mesh. Figure 4 Arrow 410 in the figure), so that the shape of the displacement curve is as close as possible to the shape of the original curve (see arrow 410 in the figure). Figure 5 As illustrated by portion 508 of displacement curve 408 and portion 502 of original curve 402, for example, the displacement curve may not perfectly match the original curve.

[0065] The advantage of sub-partition curves is that they can have a sub-partition structure that allows for efficient compression while providing a faithful approximation of the original curve. Compression efficiency is achieved due to the following properties:

[0066] - The decimation / base curve has a small number of vertices and requires a limited number of bits to encode / transmit.

[0067] - Once the base / decimation curve is decoded, the subpartition curve is automatically generated by the decoder (i.e., no information is needed except for the subpartition scheme type and subpartition iteration count).

[0068] - Displacement curves are generated by decoding the displacement vectors associated with the vertices of the sub-partition curves. In addition to allowing spatial / quality scalability, the sub-partition structure also enables efficient transforms such as wavelet decomposition that provide high compression performance.

[0069] Figure 6 A block diagram of a preprocessing system 600 is shown, which may be included in or separate from the V-DMC encoder 200. The preprocessing system 600 is shown as... Figure 2The example implementation of the preprocessing unit 204 described herein. Figure 6 In the example, the preprocessing system 600 includes a mesh extraction unit 610, an atlas parameterization unit 620, and a sub-partition surface fitting unit 630.

[0070] Mesh extraction unit 610 uses a simplified technique to extract the input mesh. And generate extraction mesh The extraction grid The mesh is then reparameterized by atlas parameterization unit 620, which can be used, for example, with the UVAtlas tool. (Error! Reference source not found.) The generated mesh is represented as... The UVAtlas tool only considers the extracted mesh when parametricizing the computation atlas. The proposed framework may not be optimal for compression due to the limited geometric information. Further parameterization schemes or tools could be considered.

[0071] Applying reparameterization to the input mesh allows for the generation of a smaller number of patches. This reduces parameterization discontinuities and leads to better RD performance. The sub-partitioned surface fitting element 630 reparameterizes the mesh. and input grid As input, and to generate the base mesh and displacement set First, the sub-partitioning scheme is applied to... Perform subdivision. This involves determining the position of each vertex of the subdivided mesh within the original mesh. The nearest point on the surface is used to calculate the displacement field. .

[0072] For the case of random access (RA), one can consider indexing as... The base grid of the reference frame As input to the sub-partitioned surface fitting unit 630, time-consistent remeshing is computed. This allows for the calculation of the current mesh. Generate and use as reference mesh The calculated sub-mesh structures are identical. This type of re-meshment process allows skipping the base mesh. Encode and reuse with reference frame Associated base grid This method can also improve the temporal prediction accuracy of attribute and geometric information. It provides more precise descriptions of how movement occurs. Vertices to match The location of the sports field The calculations are performed, and the motion field is encoded. It should be noted that such time-consistent re-meshment is not always possible. The proposed system compares the distortion obtained with and without time-consistency constraints and selects the mode that provides the optimal RD tradeoff.

[0073] It should be noted that the preprocessing system is not standard and can be replaced by any other system that generates the displacement sub-mesh surface. A potentially efficient implementation would constrain the 3D reconstruction unit to directly generate the displacement sub-mesh surface and avoid the need for such preprocessing.

[0074] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to perform displacement decoding. Depending on the application and target bit rate / visual quality, the encoder may optionally perform displacement decoding on the set of displacement vectors (called displacement fields) associated with the vertices of the sub-partitioned mesh. Encode it as described in this chapter.

[0075] Figure 7 A V-DMC encoder 700 configured to implement an intra-frame coding process is shown. The V-DMC encoder 700 represents an example implementation of the V-DMC encoder 200.

[0076] Figure 7 Including the following abbreviations:

[0077] m(i) - base grid

[0078] d(i) - displacement

[0079] m''(i) - Rebuild the base mesh

[0080] d''(i) - Reconstructed displacement

[0081] A(i) - Property Graph

[0082] A'(i) - Updated property graph

[0083] M(i) - Static / Dynamic Mesh

[0084] DM(i) - Reconstructing the deformable mesh

[0085] m'(i) - Reconstructing the quantized base grid

[0086] d'(i) - Updated displacement

[0087] e(i) - wavelet coefficients

[0088] e'(i) - Quantization wavelet coefficients

[0089] pe'(i) - Packed quantized wavelet coefficients

[0090] rpe'(i) - Reconstruct pack quantized wavelet coefficients

[0091] AB - Compressed Attribute Bitstream

[0092] DB - Compressed Displacement Bit Stream

[0093] BMB - Compressed Basic Mesh Bit Stream

[0094] V-DMC encoder 200, for example, from Figure 6 The preprocessing system 600 receives the base mesh m(i) and displacement d(i). The V-DMC encoder 200 also retrieves the mesh M(i) and the property map A(i).

[0095] The quantization unit 702 quantizes the base mesh, and the static mesh encoder 704 encodes the quantized base mesh to generate a compressed base mesh bitstream.

[0096] Displacement update unit 708 uses reconstructed quantized base mesh Update displacement field To generate updated displacement fields The process considers reconstructing the base mesh. With the original base mesh The differences between them. By utilizing the subdivided surface mesh structure, the wavelet transform unit 710 applies the wavelet transform to the wavelet transform. This generates a set of wavelet coefficients. This scheme is generally independent of the applied transform and can employ any other transform, including identity transforms. According to the technology of this disclosure, the transform unit 710 includes an offset / displacement determination unit 711. The offset / displacement determination unit 711 can be configured to: transform the set of displacement vectors to determine a set of transform coefficients; determine an offset value for the set of transform coefficients; determine an offset value based on the offset value for the set of transform coefficients; and subtract the offset value from the set of transform coefficients to determine the offset-adjusted transform coefficients.

[0097] The quantization unit 712 quantizes the wavelet coefficients (e.g., the bias-adjusted transform coefficients determined by the bias / offset determination unit 711), and the image packing unit 714 packs the quantized wavelet coefficients into a 2D image / video that can be compressed using a conventional image / video encoder in the same way as V-PCC to generate a shift bitstream.

[0098] The attribute transfer unit 730 converts the original attribute map A(i) into an updated attribute map corresponding to the reconstructed deformable mesh DM(i). The padding unit 732 pads the updated attribute map by, for example, using interpolated samples to pad frames with empty samples, which improves decoding efficiency and reduces artifacts. The color space conversion unit 734 converts the attribute map into a different color space, and the video encoding unit 736 encodes the updated attribute map in the new color space using, for example, a video codec to generate an attribute bitstream.

[0099] The multiplexer 738 combines the compressed attribute bitstream, the compressed displacement bitstream, and the compressed base mesh bitstream into a single compressed bitstream.

[0100] Image unpacking unit 718 and inverse quantization unit 720 apply image unpacking and inverse quantization to the reconstructed packed quantized wavelet coefficients generated by video coding unit 716 to obtain reconstructed versions of the wavelet coefficients. Inverse wavelet transform unit 722 applies inverse wavelet transform to the reconstructed wavelet coefficients to determine the reconstructed displacement. .

[0101] Inverse quantization unit 724 applies inverse quantization to the reconstructed quantization base mesh m'(i) to obtain the reconstructed base mesh. Deformed mesh reconstruction element 728 sub-division And will reconstruct displacement The vertices are applied to reconstruct the base mesh to obtain the reconstructed deformable mesh. .

[0102] Image unpacking unit 718, inverse quantization unit 720, inverse wavelet transform unit 722, and deformed mesh reconstruction unit 728 represent the displacement decoding loop. Inverse quantization unit 724 and deformed mesh reconstruction unit 728 represent the base mesh decoding loop. The V-DMC encoder 700 includes a displacement decoding loop and a base mesh decoding loop, enabling the V-DMC encoder 700 to make encoding decisions based on the same decoded mesh generated by the mesh decoder, such as determining an acceptable rate-distortion tradeoff, which may include distortion due to quantization and transform. The V-DMC encoder 700 can also use decoded versions of the base mesh, reconstructed mesh, and displacement to encode subsequent base meshes and displacements.

[0103] Control unit 750 typically represents the decision-making functionality of V-DMC encoder 700. During the encoding process, control unit 750 can make determinations, for example, regarding mode selection, rate allocation, quality control, and other such decisions.

[0104] Figure 8A V-DMC decoder 800 is shown that can be configured to perform intra-frame or inter-frame decoding. V-DMC decoder 800 represents an example implementation of V-DMC decoder 300. About Figure 8 The described process can also be performed, in whole or in part, by the V-DMC encoder 200.

[0105] The V-DMC decoder 800 includes a demultiplexer (DMUX) 802 that receives a compressed bitstream b(i) and separates it into a base grid bitstream (BMB), a shift bitstream (DB), and an attribute bitstream (AB). A mode selection unit 804 determines whether the base grid data is encoded in intra-frame or inter-frame mode. If the base grid is encoded in intra-frame mode, a static grid decoder 806 decodes the grid data without relying on any previously decoded grids. If the base grid is encoded in inter-frame mode, a motion decoder 808 decodes the motion, and a base grid reconstruction unit 810 applies the motion to the decoded grid (m''(j)) stored in a grid buffer 812 to determine the reconstructed quantized base grid (m'(i)). An inverse quantization unit 814 applies inverse quantization to the reconstructed quantized base grid to determine the reconstructed base grid (m''(i)).

[0106] Video decoder 816 decodes the shifted bitstream to determine a set or frame of quantized transform coefficients. Image unpacking unit 818 unpacks the quantized transform coefficients. For example, video decoder 816 may decode the quantized transform coefficients into frames, where the quantized transform coefficients are organized into blocks with a specific scan order. Image unpacking unit 818 converts the quantized transform coefficients from being organized in frames into an ordered series. In some specific implementations, for example, a context-based arithmetic decoder may be used to directly decode the quantized transform coefficients, and unpacking may not be necessary.

[0107] Regardless of whether the quantized transform coefficients are directly decoded or decoded within a frame, the inverse quantization unit 820 performs inverse quantization (e.g., inverse scaling) on ​​the quantized transform coefficients to determine the dequantized transform coefficients. The inverse wavelet transform unit 822 applies the inverse transform to the dequantized transform coefficients to determine a set of shift vectors. The inverse wavelet transform unit 822 includes an offset unit 823. The offset unit 823 is configured to determine an offset value based on one or more syntax elements and applies this offset to the set of transform coefficients to determine an updated set of transform coefficients before the inverse wavelet transform unit 822 applies the inverse transform.

[0108] The deformable mesh reconstruction unit 824 uses a decoded displacement vector to deform the reconstructed base mesh to determine the decoded mesh (M''(i)). The video decoder 826 decodes the attribute bitstream to determine the decoded attribute values ​​(A'(i)), and the color space conversion unit 828 converts the decoded attribute values ​​into the desired color space to determine the final attribute values ​​(A''(i)). The final attribute values ​​correspond to attributes such as color or texture of the vertices of the decoded mesh.

[0109] Figure 9 A block diagram of an intra-frame decoder, which may be a part of, for example, the V-DMC decoder 300, is shown. The demultiplexer (DMUX) 902 separates the compressed bitstream (bi) into a mesh substream, a displacement substream for position and potentially for each vertex attribute, a zero or more attribute graph substream, and an atlas substream containing patch information, in the same manner as in V3C / V-PCC.

[0110] Demultiplexer 902 feeds the mesh substream to static mesh decoder 906 to generate the reconstructed quantized base mesh. The inverse quantization unit 914 inverse-quantizes the base mesh to determine the decoded base mesh. The video / image decoding unit 916 decodes the displacement substream, and the image unpacking unit 918 unpacks the image / video to determine the quantization transform coefficients (e.g., wavelet coefficients). The inverse quantization unit 920 inverse-quantizes the quantization transform coefficients to determine the dequantization transform coefficients. The inverse transform unit 922 generates the decoded displacement field by applying the inverse transform to the unquantized coefficients. The deformable mesh reconstruction unit 924 decodes the base mesh. Apply reconstruction processing and add decoded displacement field This is used to generate the final decoded mesh (M''(i)). The attribute substream is directly decoded by the video / image decoding unit 928 to generate the attribute map. The color format / space conversion unit can convert attribute maps into different formats or color spaces.

[0111] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to implement a sub-partitioning scheme. Various sub-partitioning schemes can be considered. One possible solution is the midpoint sub-partitioning scheme, which divides each triangle into four sub-triangles in each sub-partitioning iteration, such as... Figure 10 As described in [the text]. A new vertex is introduced in the middle of each edge. Figure 10 In the example, triangle 1002 is subdivided to obtain triangle 1004, and triangle 1004 is subdivided to obtain triangle 1006. The subdivision process is applied independently to the geometry and texture coordinates because the connectivity of the geometry and texture coordinates is typically different. The subdivision scheme is computed on the edges. The newly introduced vertex at the center Location ,as follows:

[0112]

[0113] in and It is the vertex and The location.

[0114] The same process is used to calculate the texture coordinates of the newly created vertices. For the normal vectors, an additional normalization step is applied as follows:

[0115]

[0116] here:

[0117] • , and They are respectively with the vertex , and The associated normal vector.

[0118] • It is a vector norm2.

[0119] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to apply wavelet transforms. Various wavelet transforms can be applied. The results reported for CfP are based on linear wavelet transforms.

[0120] The prediction process is defined as follows:

[0121]

[0122] in

[0123] • It is at the edge The middle vertex introduced, and

[0124] • , and They are vertices. , and The value of the geometry / vertex attribute signal at that location.

[0125] The update process is as follows:

[0126]

[0127] in It is the vertex The set of adjacent vertices.

[0128] This scheme allows the update process to be skipped. Wavelet coefficients can be quantized, for example, by using a uniform quantizer with a dead zone.

[0129] We will now discuss local coordinate systems and canonical coordinate systems used for displacement. Displacement field It is defined in the same Cartesian coordinate system as the input grid. A possible optimization is to... Transform from the regular coordinate system to the local coordinate system, which is defined by the normals of the sub-mesh at each vertex.

[0130] A potential advantage of considering a local coordinate system for displacement is that the tangential components of the displacement can be quantized more heavily than the normal components. In fact, the normal components of the displacement have a more significant impact on the quality of the reconstructed mesh than the two tangential components.

[0131] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to pack wavelet coefficients. The wavelet coefficients are packed into a 2D image using the following scheme:

[0132] • Errata coefficients from low frequency to high frequency.

[0133] • For each coefficient, determine The pixel blocks that should be stored in the raster order of the blocks (e.g., () index.

[0134] • The position within a pixel block is calculated using the Morton order to maximize locality.

[0135] Other packing schemes (e.g., zigzag order, raster order) can be used. The encoder can explicitly signal the packing scheme used (e.g., atlas sequence parameters) in the bit stream. This can be done at the patch, patch group, tile, or sequence level.

[0136] The V-DMC encoder 200 can be configured for displacement video coding. The proposed scheme does not specify which video decoding technique to use. When decoding displacement wavelet coefficients, a lossless method can be used because quantization is applied in a separate module. Alternatively, a lossy method can be used, relying on the video encoder to compress the coefficients in a lossy manner and apply quantization in the original or transform domain.

[0137] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to process lifting transform parameter sets and associated semantics, examples of which are shown in Table 1 below.

[0138] Table 1

[0139]

[0140] The syntax_element[i][ltpIndex] with i equal to 0 can be applied to the shift. The syntax_element[i][ltpIndex] with i equal to non-zero can be applied to the (i-1)th attribute, where ltpIndex is the index of the list of lifting transformation parameters.

[0141] A `vmc_transform_lifting_skip_update_flag[i][ltpIndex]` equal to 1 indicates that the lifting transformation steps applied to the displacement are skipped in the `vmc_lifting_transform_parameters(index, lptIndex)` syntax structure, where `ltpIndex` is the index of the list of lifting transformation parameters. A `vmc_transform_lifting_skip_update_flag[i][ltpIndex]` where `i` equals 0 can be applied to the displacement. A `vmc_transform_lifting_skip_update_flag[i][ltpIndex]` where `i` is non-zero can be applied to the (i-1)th attribute.

[0142] vmc_transform_lifting_quantization_parameters_x[i][ltpIndex] indicates the quantization parameters to be used for the inverse quantization of the x-components of the displacement wavelet coefficients. The value of vmc_transform_lifting_quantization_parameters_x[index][ltpIndex] should be in the range of 0 to 51 (inclusive).

[0143] `vmc_transform_lifting_quantization_parameters_y[i][ltpIndex]` indicates the quantization parameters to be used for the inverse quantization of the y-components of the displacement wavelet coefficients. The value of `vmc_transform_lifting_quantization_parameters_x[index][ltpIndex]` should be in the range of 0 to 51 (inclusive).

[0144] vmc_transform_lifting_quantization_parameters_z[i][ltpIndex] indicates the quantization parameters to be used for the inverse quantization of the z-components of the displacement wavelet coefficients. The value of vmc_transform_lifting_quantization_parameters_x[index][ltpIndex] should be in the range of 0 to 51 (inclusive).

[0145] vmc_transform_log2_lifting_lod_inverse_scale_x[i][ltpIndex] indicates the scaling factor applied to the x-component of the displacement wavelet coefficients at each level of detail.

[0146] vmc_transform_log2_lifting_lod_inverse_scale_y[i][ltpIndex] indicates the scaling factor applied to the y-component of the displacement wavelet coefficients at each level of detail.

[0147] vmc_transform_log2_lifting_lod_inverse_scale_z[i][ltpIndex] indicates the scaling factor applied to the z-component of the displacement wavelet coefficients at each level of detail.

[0148] vmc_transform_log2_lifting_update_weight[i][ltpIndex] indicates the weighting coefficients of the update filter used for wavelet transform.

[0149] vmc_transform_log2_lifting_prediction_weight[i][ltpIndex] is the weighting coefficient of the prediction filter used for wavelet transform.

[0150] The V-DMC decoder 300 can be configured to perform inverse image packing of wavelet coefficients. The input to this process is:

[0151] • width, which is a variable indicating the width of the displaced video frame.

[0152] • height, which is a variable indicating the height of the displaced video frame.

[0153] • bitDepth, which is a variable indicating the bit depth of the shifted video frame.

[0154] • dispQuantCoeffFrame, which is a 3D array indicating the size of the packed quantized displacement wavelet coefficients, with width × height × 3.

[0155] • blockSize, which is a variable indicating the size of the displacement coefficient block.

[0156] • positionCount is a variable that indicates the number of positions in the subgrid of the subdivision.

[0157] The output of this process is dispQuantCoeffArray, which is a 2D array of size positionCount×3 indicating the quantization displacement wavelet coefficients.

[0158] Let the function extracOddBits(x) be defined as follows:

[0159] x = extracOddBits( x ) {

[0160] x = x & 0x55555555

[0161] x = (x | (x >> 1)) & 0x33333333

[0162] x = (x | (x >> 2)) & 0x0F0F0F0F

[0163] x = (x | (x >> 4)) & 0x00FF00FF

[0164] x = (x | (x >> 8)) & 0x0000FFFF

[0165] }

[0166] Let the function computeMorton2D(i) be defined as follows:

[0167] (x, y) = computeMorton2D( i ) {

[0168] x = extracOddBits( i >> 1 )

[0169] y = extracOddBits(i)

[0170] }

[0171] The wavelet coefficient inverse packing process is as follows:

[0172] pixelsPerBlock = blockSize blockSize

[0173] widthInBlocks = width / blockSize

[0174] shift = (1 << bitDepth) >> 1

[0175] for( v = 0; v < positionCount; v++ ) {

[0176] blockIndex = v / pixelsPerBlock

[0177] indexWithinBlock = v % pixelsPerBlock

[0178] x0 = (blockIndex % widthInBlocks) blockSize

[0179] y0 = (blockIndex / widthInBlocks) blockSize

[0180] ( x, y ) = computeMorton2D(indexWithinBlock)

[0181] x = x0 + x

[0182] y = y0 + y

[0183] for( d = 0; d < 3; d++ ) {

[0184] dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ x ][ y ][d ] – shift

[0185] }

[0186] }

[0187] The V-DMC decoder 300 can be configured to perform inverse quantization of wavelet coefficients. The input to this process is:

[0188] • positionCount is a variable that indicates the number of positions in the subgrid of the subdivision.

[0189] • dispQuantCoeffArray is a 2D array of size positionCount×3 indicating the quantization displacement wavelet coefficients.

[0190] • subdivisionIterationCount is a variable that indicates the number of subdivision iterations.

[0191] • liftingQP, which is a 1D array of size 3 indicating the quantization parameters associated with the three displacement dimensions.

[0192] • liftingLevelOfDetailInverseScale is a 1D array of size 3 that indicates the inverse scaling factor associated with the three displacement dimensions.

[0193] • levelOfDetailAttributeCounts, a 1D array of size (subdivisionIterationCount + 1) indicating the number of attributes associated with each subdivision iteration.

[0194] • bitDepthPosition is a variable that indicates the bit depth of the grid position.

[0195] The output of this process is dispCoeffArray, which is a 2D array of size positionCount×3 indicating the dequantized displacement wavelet coefficients.

[0196] The wavelet coefficient inverse quantization process is as follows:

[0197] for (d = 0; d < 3; ++d) {

[0198] qp = liftingQP[d]

[0199] iscale[ d ] = qp >= 0 ? pow( 0.5, 16 – bitDepthPosition + ( 4 – qp) / 6) : 0.0

[0200] ilodScale[ d ] = liftingLevelOfDetailInverseScale[ d ]

[0201] }

[0202] vcount0 = 0

[0203] for( i = 0; i < subdivisionIterationCount; i++ ) {

[0204] vcount1 = levelOfDetailAttributeCounts[ i ]

[0205] for( v = vcount0; v < vcount1; v++ ) {

[0206] for( d = 0; d < 3; d++ ) {

[0207] dispCoeffArray[ v ][ d ] = dispQuantCoeffArray[ v ][ d ] iscale[k]

[0208] }

[0209] }

[0210] vcount0 = vcount1

[0211] for( d = 0; d < 3; d++ ) {

[0212] iscale[d] = ilodScale[ d ]

[0213] }

[0214] }

[0215] The V-DMC decoder 300 can be configured to apply an inverse linear wavelet transform. The input to this process is:

[0216] • positionCount is a variable that indicates the number of positions in the subgrid of the subdivision.

[0217] • dispCoeffArray is a 2D array of size positionCount×3 that indicates the displacement wavelet coefficients.

[0218] • levelOfDetailAttributeCounts, a 1D array of size (subdivisionIterationCount + 1) indicating the number of attributes associated with each subdivision iteration.

[0219] • edges, which is a 2D array of size positionCount×2, which for each vertex v generated by the subdivision process described above, indicates the two indices (a, b) of the two vertices used to generate vertex v (i.e., v is generated as the midpoint of edge(a, b)).

[0220] • updateWeight is a variable that indicates the increase in update weight.

[0221] • predWeight is a variable that indicates the weight of the prediction.

[0222] • skipUpdate is a variable that indicates whether the update operation should be skipped (when 1) or not (when 0).

[0223] The output of this process is a dispArray, which is a 2D array of size positionCount×3 indicating the displacements to be applied to the grid positions.

[0224] The inverse wavelet transform process is as follows:

[0225] for( i = 0; i < subdivisionIterationCount; i++ ) {

[0226] vcount0 = levelOfDetailAttributeCounts[i]

[0227] vcount1 = levelOfDetailAttributeCounts[i + 1]

[0228] for ( v = vcount0; skipUpdate == 0 && v < vcount1; ++v ) {

[0229] a = edges[v][0]

[0230] b = edges[v][1]

[0231] for( d = 0; d < 3; d++ ) {

[0232] disp = updateWeight dispCoeffArray[v][d]

[0233] signal[a][d] -= disp

[0234] signal[b][d] -= disp

[0235] }

[0236] }

[0237] for ( v = vcount0; skipUpdate == 0 && v < vcount1; ++v ) {

[0238] a = edges[v][0]

[0239] b = edges[v][1]

[0240] for( d = 0; d < 3; d++ ) {

[0241] dispCoeffArray[v][d] += predWeight (dispCoeffArray[a][d] +dispCoeffArray[b][d])

[0242] }

[0243] }

[0244] }

[0245] for ( v = 0; v < positionCount; ++v ) {

[0246] for( d = 0; d < 3; d++ ) {

[0247] dispArray[v][d] = dispCoeffArray[v][d]

[0248] }

[0249] }

[0250] The V-DMC decoder 300 can be configured to perform position displacement. The input to this process is:

[0251] • positionCount is a variable that indicates the number of positions in the subgrid of the subdivision.

[0252] • positionsSubdiv is a 2D array of size 3 that indicates the positions of the subgrids.

[0253] • dispArray, which is a 2D array of size positionCount×3 indicating the displacement to be applied to the grid position.

[0254] • normals is a 2D array of size positionCount×3 that indicates the normals to be used when applying displacement to the submesh position.

[0255] • tangents, which is a 2D array of size positionCount×3 indicating the tangents to use when applying displacement to the subgrid position.

[0256] • bitangents, which is a 2D array of size positionCount×3 indicating the tangents to use when applying displacement to the subgrid position.

[0257] The output of this process is positionsDisplaced, which is a 2D array of size positionCount×3 indicating the positions of the subgrids of the subdivisions.

[0258] The position displacement process proceeds as follows:

[0259] for ( v = 0; v < positionCount; ++v ) {

[0260] for( d = 0; d < 3; d++ ) {

[0261] positionsDisplaced[ v ][ d ] = positionsSubdiv[ v ][ d ] +

[0262] dispArray[ v ][ 0 ] normals[ v ][ d ] +

[0263] dispArray[ v ][ 1 ] tangents[ v ][ d ] +

[0264] dispArray[ v ][ 2 ] bitangents[ v ][ d ]

[0265] }

[0266] }

[0267] As described above regarding wavelet transform, the displacement vector is transformed by a lifting transform, which includes a prediction step and a subsequent update process. Current implementations of the lifting transform have some drawbacks. For example, the prediction weights and update weights are 0.5 and 0.125, respectively, neither of which are optimal. This disclosure introduces a tuning offset to make the prediction and update processes more efficient. As another example, while the option to skip updates at all levels of detail is provided, the functionality to shut down the update step after a specific level is not provided. This functionality could be useful in the case of adaptive update weights based on the level of detail (i.e., when the update weights become so small during iterations that they have almost no effect on the update). The number of updates performed can be signaled using one of the following syntaxes:

[0268] When "vdmc_transform_lifting_skip_update_flag[index][ltpIndex]" is set to 1, "vdmc_transform_lifting_skip_update_level[index][ltpIndex]" or "vdmc_transform_lifting_update_levels[index][ltpIndex]" is used.

[0269] As another example, for LOD-based adaptive update weights (as originally proposed in Chao Cao, [V-DMC][NEW] "LOD-based adaptive update weight for Forward Linear Lifting WaveletTransform," ISO / IEC JTC 1 / SC 29 / WG 7 MPEG Input Document m64223, Geneva, Switzerland, July 2023 (hereinafter referred to as m64223)), these associated values ​​are signaled using the following syntax:

[0270] vdmc_transform_lifting_adaptive_update_weight_flag[ index ][ ltpIndex ]

[0271] vdmc_transform_lifting_adaptive_update_weight_scale[ index ][ltpIndex ]

[0272] vdmc_transform_log2_lifting_adaptive_update_weight [index][ltpIndex]

[0273] In the case of adaptive weights, a total of three syntax elements can be used, two of which are related to updating the weights via formulas, instead of using only one syntax element for updating the weights, leading to inefficient signaling notification. This disclosure describes solutions to these problems. More specifically, this disclosure describes an example of performing signal boosting updates of weights for each LOD in an adaptive scheme.

[0274] Figure 11 An example implementation of the forward lift transform is shown. The V-DMC encoder 200 can be configured to implement the offset lift transform as described herein, and the V-DMC decoder 300 can be configured to implement the inverse transform, which is essentially... Figure 11 The reverse process shown. The encoding process of the shift bit stream is in... Figure 11 The example illustrates the use of a lifting scheme to perform wavelet transform on the displacement vector.

[0275] LOD0 1100 represents the base mesh. LOD1 1102 represents the base mesh of the sub-mesh after the first sub-mesh, and LOD2 represents the base mesh of the sub-mesh after the second sub-mesh. The V-DMC encoder 200 can compare, for example, the sub-mesh of LOD 1102 with the original mesh to determine the displacement vectors of the vertices of the sub-mesh.

[0276] At branch point 1106, the V-DMC encoder 200 splits the input signal into two distinct signals, one corresponding to the displacement vectors of LOD0 1100 and LOD1 1102, and the other corresponding to the displacement vector of LOD2 1104. At prediction 1108, the V-DMC encoder 200 predicts the finest level value (i.e., the displacement vector of LOD2 1104) based on the lower level values ​​(i.e., the displacement vectors of LOD0 1100 and LOD1 1102). At subtractor 1110, the V-DMC encoder 200 determines the difference between the original displacement vector of LOD2 1104 and the predicted displacement vector of LOD2 1104 (i.e., the output of prediction 1108). This difference is referred to as the transformed displacement vector of LOD2 (DV' LOD2 1116). DV' LOD2 1116 typically has lower energy than the original displacement vector value of LOD2 1104, meaning that the value of DV' LOD2 1116 is generally closer to 0. Because the values ​​of DV' LOD2 1116 have less energy, these values ​​can be encoded with fewer bits, and the loss due to quantization is also smaller. At 1112, the V-DMC encoder 200 determines the update, and at the summer 1114, it adds this update to the displacement vectors of LOD0 1100 and LOD1 1102 to determine the updated DV' LOD0 1118 and DV' LOD1120.

[0277] Then, a similar process is performed on the updated displacement vectors DV' LOD0 1118 and DV' LOD1 1120. At branch point 1122, the V-DMC encoder 200 splits the input signal into two distinct signals, one corresponding to the updated DV' LOD0 1118 and the other to DV' LOD1 1120. At 1124, the V-DMC encoder 200 predicts the higher-level value (i.e., DV' LOD1 1120) based on the lower-level value (i.e., DV' LOD0 1118).

[0278] At subtractor 1126, V-DMC encoder 200 determines the difference between the original value of DV' LOD1120 and the predicted value of DV' LOD1120. This difference is called the transform shift vector of LOD1 (DV'' LOD1 1132). As explained above regarding DV' LOD21116, DV'' LOD1 1132 typically has less energy than DV' LOD1 1110, meaning that the value of DV'' LOD1 1132 is generally closer to 0. Because the values ​​of DV'' LOD1 1132 have less energy, these values ​​can be encoded with fewer bits, and the loss due to quantization is also smaller. At 1128, V-DMC encoder 200 determines an update and adds this update to DV' LOD0 1118 at summer 1130 to determine the updated DV'' LOD0 1134. As will be explained in more detail below, the updates performed at 1112 and 1128 further improved the compression performance.

[0279] According to the technology disclosed herein, when the forward lift transform is affected, only the encoder end changes. The forward transform starts from the finest level (denoted as LOD2), as... Figure 11 The lift transform is an iterative process in which the input signal is split into two signals. Then, the vertices (v1 and v2) of the lower LOD (LOD0 and LOD1) from the same side are used to predict the higher LOD (LOD2 in this example) samples (forward transform).

[0280] In the current implementation, the prediction is the average of the two vertices on the same side, i.e., predWeight = 0.5. The error signal is calculated by subtracting the prediction from the original signal. Finally, an update is performed to recalibrate the lower LOD sample. In TMMv4.0, updateWeight is 0.125.

[0281] The specific implementation of the forward lifting transform in the V-DMC reference software is as follows:

[0282] template<class T1, class T2>

[0283] void

[0284] computeForwardLinearLifting(

[0285] std::vector <t1>& signal,

[0286] const std::vector <vmesh::subdivisionlevelinfo>& infoLevelOfDetails,

[0287] const std::vector<int64_t>& edges,

[0288] const T2 predWeight,

[0289] const T2 updateWeight,

[0290] const bool skipUpdate) {

[0291] const auto lodCount = int32_t(infoLevelOfDetails.size());

[0292] assert(lodCount > 0);

[0293] const auto rfmtCount = lodCount - 1;

[0294] for (int32_t it = rfmtCount - 1; it >= 0; --it) {

[0295] const auto vcount0 = infoLevelOfDetails[it].pointCount;

[0296] const auto vcount1 = infoLevelOfDetails[it + 1].pointCount;

[0297] assert(vcount0 < vcount1 && vcount1 <= int32_t(signal.size()));

[0298] / / predict

[0299] for (int32_t v = vcount0; v < vcount1; ++v) {

[0300] const auto edge = edges[v];

[0301] const auto v1 = int32_t(edge & 0xFFFFFFFF);

[0302] const auto v2 = int32_t((edge >> 32) & 0xFFFFFFFF);

[0303] assert(v1 >= 0 && v1 <= vcount0);

[0304] assert(v2 >= 0 && v2 <= vcount0);

[0305] signal[v] -= predWeight (signal[v1] + signal[v2]);

[0306] }

[0307] / / update

[0308] for (int32_t v = vcount0; !skipUpdate && v < vcount1; ++v) {

[0309] const auto edge = edges[v];

[0310] const auto v1 = int32_t(edge & 0xFFFFFFFF);

[0311] const auto v2 = int32_t((edge >> 32) & 0xFFFFFFFF);

[0312] assert(v1 >= 0 && v1 <= vcount0);

[0313] assert(v2 >= 0 && v2 <= vcount0);

[0314] const auto d = updateWeight signal[v];

[0315] signal[v1] += d;

[0316] signal[v2] += d;

[0317] }

[0318] }

[0319] }

[0320] The following introduces the tuning offset in the prediction and update steps for the forward lifting transform, where the offset step is highlighted:

[0321] template<class T1, class T2>

[0322] void

[0323] computeForwardLinearLifting(

[0324] std::vector <t1>& signal,

[0325] const std::vector <vmesh::subdivisionlevelinfo>& infoLevelOfDetails,

[0326] const std::vector<int64_t>& edges,

[0327] const T2 predWeight,

[0328] const T2 updateWeight,

[0329] const bool skipUpdate) {

[0330] const auto lodCount = int32_t(infoLevelOfDetails.size());

[0331] assert(lodCount > 0);

[0332] const auto rfmtCount = lodCount - 1;

[0333] for (int32_t it = rfmtCount - 1; it >= 0; --it) {

[0334] const auto vcount0 = infoLevelOfDetails[it].pointCount;

[0335] const auto vcount1 = infoLevelOfDetails[it + 1].pointCount;

[0336] assert(vcount0 < vcount1 && vcount1 <= int32_t(signal.size()));

[0337] / / predict

[0338] for (int32_t v = vcount0; v < vcount1; ++v) {

[0339] const auto edge = edges[v];

[0340] const auto v1 = int32_t(edge & 0xFFFFFFFF);

[0341] const auto v2 = int32_t((edge >> 32) & 0xFFFFFFFF);

[0342] assert(v1 >= 0 && v1 <= vcount0);

[0343] assert(v2 >= 0 && v2 <= vcount0);

[0344] for (int32_t i = 0; i < 3; ++i) {

[0345] double val = signal[v1][i] + signal[v2][i];

[0346] signal[v][i] -= (val < 0 ? -((-val + predWeightOffset) predWeight) :

[0347] ((val + predWeightOffset) predWeight));

[0348] }

[0349] }

[0350] / / update

[0351] for (int32_t v = vcount0; !skipUpdate && v < vcount1; ++v) {

[0352] const auto edge = edges[v];

[0353] const auto v1 = int32_t(edge & 0xFFFFFFFF);

[0354] const auto v2 = int32_t((edge >> 32) & 0xFFFFFFFF);

[0355] assert(v1 >= 0 && v1 <= vcount0);

[0356] assert(v2 >= 0 && v2 <= vcount0);

[0357] std::vector <double>d(3);

[0358] for (int32_t i = 0; i < 3; ++i) {

[0359] double val = signal[v][i];

[0360] d[i] = (val < 0 ? -((-val + updateWeightOffset) updateWeight)

[0361] ((val + updateWeightOffset) updateWeight));

[0362] signal[v1][i] += d[i];

[0363] signal[v2][i] += d[i];

[0364] }

[0365] }

[0366] }

[0367] }

[0368] Here, the displacement vector transformation values ​​are rounded away from 0, which in some simulations resulted in point cloud-based BD rate gains of up to 1.9% and image-based BD rate gains of 4.4%, respectively.

[0369] In one example, predWeightOffset and updateWeightOffset are set to the value 0.0625. The tuning offset can also be subtracted in the inverse lift transform as part of the decoding process, as shown in the specific implementation below:

[0370] template<class T1, class T2>

[0371] void

[0372] computeInverseLinearLifting(

[0373] std::vector <t1>& signal,

[0374] const std::vector <vmesh::subdivisionlevelinfo>& infoLevelOfDetails,

[0375] const std::vector<int64_t>& edges,

[0376] const T2 predWeight,

[0377] const T2 updateWeight,

[0378] const bool skipUpdate) {

[0379] printf("Compute inverse linear lifting \n");

[0380] fflush(stdout);

[0381] const auto lodCount = int32_t(infoLevelOfDetails.size());

[0382] assert(lodCount > 0);

[0383] const auto rfmtCount = lodCount - 1;

[0384] for (int32_t it = 0; it < rfmtCount; ++it) {

[0385] const auto vcount0 = infoLevelOfDetails[it].pointCount;

[0386] const auto vcount1 = infoLevelOfDetails[it + 1].pointCount;

[0387] assert(vcount0 < vcount1 && vcount1 <= int32_t(signal.size()));

[0388] / / update

[0389] int32_t updateWeightOffset = (1 << updateWeight) >> 1;

[0390] for (int32_t v = vcount0; !skipUpdate && v < vcount1; ++v) {

[0391] const auto edge = edges[v];

[0392] const auto v1 = int32_t(edge & 0xFFFFFFFF);

[0393] const auto v2 = int32_t((edge >> 32) & 0xFFFFFFFF);

[0394] assert(v1 >= 0 && v1 <= vcount0);

[0395] assert(v2 >= 0 && v2 <= vcount0);

[0396] std::vector <double>d(3);

[0397] for(int32_t i = 0; i < 3; ++i) {

[0398] int32_t val = signal[v][i];

[0399] d[i] = (val < 0 ? -((-val - updateWeightOffset) updateWeight) : ((val - updateWeightOffset) updateWeight));

[0400] signal[v1][i] -= d[i];

[0401] signal[v2][i] -= d[i];

[0402] }

[0403] }

[0404] / / predict

[0405] int32_t predWeightOffset = (1 << predWeight) >> 1;

[0406] for (int32_t v = vcount0; v < vcount1; ++v) {

[0407] const auto edge = edges[v];

[0408] const auto v1 = int32_t(edge & 0xFFFFFFFF);

[0409] const auto v2 = int32_t((edge >> 32) & 0xFFFFFFFF);

[0410] assert(v1 >= 0 && v1 <= vcount0);

[0411] assert(v2 >= 0 && v2 <= vcount0);

[0412] for(int32_t i = 0; i < 3; ++i) {

[0413] int32_t val = signal[v1][i] + signal[v2][i];

[0414] signal[v][i] += (val < 0 ? -((-val - predWeightOffset) predWeight) :

[0415] ((val - predWeightOffset) predWeight));

[0416] }

[0417] }

[0418] }

[0419] }

[0420] In one example, an offset is added (instead of subtracted) in the inverse boosting scheme (shown in the thick lines above). The results show that using the offset on both the encoder and decoder simultaneously improves the image-based BD rate compared to changing it only at the encoder end.

[0421] Generally, update offset and prediction offset can be used together or separately in either or both of the encoder and decoder. The values ​​of update offset and prediction offset can be constants or signaled to the decoder in the bitstream, for example, in the boost parameter set, the sequence parameter set, or an equivalent. Update offset and prediction offset values ​​can also be determined mathematically.

[0422] In the current implementation of V-DMC TMM V6.0, a bias exists when plotting the predicted residual distribution of the x-component of the displacement vector. The error signal is calculated by subtracting the prediction from the original signal. Finally, an update is performed to recalibrate the lower LOD sample.

[0423] The bias in the prediction error signal is addressed by adding an offset after the prediction step, as follows:

[0424]

[0425] in, and = .

[0426] It should be noted that the offset is determined by calculating the mean or average of the residuals, but this is just an example, and the offset can also be determined by other methods.

[0427] The specific implementation of this solution is as follows, where, in the delimiter <add> and< / add> The specific implementation of the offset is shown in the diagram:

[0428] Encoder:

[0429] signal[v] -= predWeight (signal[v1] + signal[v2]);

[0430] <add>if(offsets[0]!=0 && offsets[1]!=0){

[0431] signal[v][0] = (signal[v][0] < 0 ? (signal[v][0] + 1 / offsets[0]) : (signal[v][0] - 1 / offsets[1]));

[0432] }< / add>

[0433] Decoder:

[0434] <add>if(offsets[0]!=0 && offsets[1]!=0){

[0435] signal[v][0] = (signal[v][0] < 0 ? (signal[v][0] - 1 / offsets[0]) : (signal[v][0] + 1 / offsets[1]));

[0436] }< / add>

[0437] signal[v] += predWeight (signal[v1] + signal[v2]);

[0438] Alternatively, a single approximate offset can be used instead of two offset values. The results show that using a single approximate offset yields an image-based BD rate geometric gain. A reduction in displacement bit flow is observed due to bias adjustment of the error signal.

[0439] The syntax proposed for this specific implementation is as follows, where the delimiter... <add> and< / add> The specific implementation of the offset is shown in the diagram:

[0440]

[0441]

[0442] In the example, `vltp_lifting_offsets[index][ltpIndex][i]` represents the predicted offset for signaling, and `num_offsets` is 2. Currently, this offset is signaled and used as its reciprocal. This offset can be used directly as a value instead of... To signal, in some examples, sequences without offset are signaled as 0. In some examples, a flag can be used to enable offset for predicting residuals.

[0443] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to skip updates at the LOD level. In TMMv5.0, the enhancement transform parameter set and semantics are as follows:

[0444]

[0445] To skip updates after a certain LOD, encoder parameters can be added and signaled in two ways. In the first example, if skipping updates is enabled, `vdmc_transform_lifting_skip_update_level[index][ltpIndex]` indicates the level at which updates are turned off in subsequent lift transformations, and can be delimited by the following table. <add> and< / add> The syntax shown in the text indicates that a signal is being sent.

[0446]

[0447] In another example, `vdmc_transform_lifting_update_levels[index][ltpIndex]` can be used to explicitly signal the total number of updates to be performed, as follows:

[0448]

[0449] The default values ​​for `vdmc_transform_lifting_skip_update_level[index][ltpIndex]` and `vdmc_transform_lifting_update_levels[index][ltpIndex]` are 3, and the range is from 0 to the maximum LOD. A value of 0 corresponds to no update being performed on any LOD level, while a value set to the maximum LOD corresponds to an update being performed on all LOD levels.

[0450] The V-DMC encoder 200 and V-DMC decoder 300 can be configured to update weight signaling notifications at the LOD level. In the adaptive weight update for each LOD (as proposed in m64223), the relevant syntax elements are delimited by the following delimiters. <add> and< / add> Signal notification in the manner shown above:

[0451]

[0452]

[0453] Where vdmc_transform_lifting_adaptive_update_weight_flag[index][ltpIndex] equals 1, it indicates that adaptive weights will be used for updates in the lifting transformation using the following formula:

[0454] if (!adaptiveUpdateWeight) {

[0455] const auto d = updateWeight signal[v];

[0456] signal[v1] += d;

[0457] signal[v2] += d;

[0458] } else {

[0459] const auto d =

[0460] updateWeight

[0461] pow(adaptiveUpdateWeightScaleValue, (lodCount - it - 2))

[0462] signal[v];

[0463] signal[v1] += d;

[0464] signal[v2] += d;

[0465] }

[0466] }

[0467] The value of adaptiveUpdateWeightScaleValue can range from 1.0 to 2.0, lodCount is the total number of LODs, and "it" is the LOD of the current iteration.

[0468] If adaptive weight updates are used, the scaling value is signaled using vdmc_transform_lifting_adaptive_update_weight_scale[index][ltpIndex].

[0469] According to the technique disclosed herein, the adaptive update weights in the boost transformation can be signaled to each LOD, as in the following example:

[0470] Example 1

[0471]

[0472] Example 2

[0473]

[0474] The potential advantage of this technology is that it retains the current signaling notification method for weights while extending to signaling notification for each LOD.

[0475] In some examples, the following examples are proposed to allow direct signaling notification of adaptive boosting update weight values ​​for each LoD:

[0476] Example 3

[0477]

[0478] Example 4

[0479]

[0480] According to the technology disclosed herein, the updated weights are signaled in the form of numerator and denominator values ​​to achieve high precision. The updated weights can be calculated as follows:

[0481] Lifting_update_weight[ index ][ ltpIndex ] =

[0482] vdmc_transform_lifting_update_weight_numerator [index][ltpIndex]

[0483] / vdmc_transform_lifting_update_weight_denominator [index][ltpIndex]

[0484] In some examples, the boosting update weights are not explicitly calculated; the numerator and denominator are used directly in the update equation. For example,

[0485] <add>

[0486] updateWeightOff = updateWeightDr / 2

[0487] d[i] = val < 0 ? -((-val updateWeightNr +

[0488] updateWeightOff) / updateWeightDr) : ((val updateWeightNr+updateWeightOff) / updateWeightDr);

[0489] < / add>

[0490] Here, updateWeightNr is vdmc_transform_lifting_update_weight_numerator[index][ltpIndex], and updateWeightDr is vdmc_transform_lifting_update_weight_denominator[index][ltpIndex]. Note that the above example can be combined with updated weight offsets.

[0491] To combine the two cases for when vdmc_transform_lod_quantization_flag[index][ltpIndex] equals 0 and 1, the following example is proposed.

[0492] Example 5

[0493]

[0494] Example 6

[0495]

[0496] Example 7

[0497] Encoding parameters can be added to enable adaptive weight updates in the lifting transform, allowing for flexible switching between fixed and adaptive weight update methods. This flexibility can be used in any of the examples mentioned in this section. An extension of this flexibility is shown below, based on Example 6:

[0498]

[0499] Figure 12 This is a flowchart illustrating an example process for encoding a mesh. Although regarding the V-DMC encoder 200 ( Figure 1 and Figure 2 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 12 The process is similar to that of [the other process].

[0500] exist Figure 12 In the example, the V-DMC encoder 200 receives an input mesh (1202). The V-DMC encoder 200 determines a base mesh based on the input mesh (1204). The V-DMC encoder 200 determines a set of displacement vectors based on the input mesh and the base mesh (1206). The V-DMC encoder 200 outputs an encoded bitstream that includes an encoded representation of the base mesh and encoded representations of the displacement vectors (1208). The V-DMC encoder 200 may additionally determine attribute values ​​from the input mesh and include encoded representations of the attribute value vectors in the encoded bitstream.

[0501] Figure 13 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data. Although regarding the V-DMC decoder 300 ( Figure 1 and Figure 3 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 13 The process is similar to that of [the other process].

[0502] exist Figure 13 In the example, the V-DMC decoder 300 determines a base mesh based on encoded mesh data (1302). The V-DMC decoder 300 determines one or more displacement vectors based on the encoded mesh data (1304). The V-DMC decoder 300 uses one or more displacement vectors to deform the base mesh (1306). For example, the base mesh may have a first set of vertices, and the V-DMC decoder 300 may subdivide the base mesh to determine an additional set of vertices for the base mesh. To deform the base mesh, the V-DMC decoder 300 may modify the positions of the additional set of vertices based on one or more displacement vectors. The V-DMC decoder 300 outputs a decoded mesh based on the deformed mesh (1308). The V-DMC decoder 300 may, for example, output the decoded mesh for storage, transmission, or display.

[0503] Figure 14 This is a flowchart illustrating an example process for encoding a mesh. Although regarding the V-DMC encoder 200 ( Figure 1 and Figure 2 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 14 The process is similar to that of [the other process].

[0504] exist Figure 14 In the example, the V-DMC encoder 200 determines a set of displacement vectors of the mesh data (1402). The V-DMC encoder 200 transforms the set of displacement vectors to determine a set of transform coefficients (1404). To transform the set of displacement vectors, the V-DMC encoder 200 is also configured to apply a wavelet transform with a lifting scheme, as described above. The V-DMC encoder 200 determines an offset value for the set of transform coefficients (1406). To determine this offset value for the set of transform coefficients, the V-DMC encoder 200 can be configured to determine an offset value for the normal component of the set of displacement vectors. The V-DMC encoder 200 determines an offset value based on the offset value of the set of transform coefficients (1408). The V-DMC encoder 200 subtracts the offset value from the set of transform coefficients to determine the offset-adjusted transform coefficients (1410). The V-DMC encoder 200 quantizes the offset-adjusted transform coefficients to determine quantization coefficients (1412). The V-DMC encoder 200 signals the quantization coefficients and the offset indication (1414) in the bit stream of the encoded grid data.

[0505] Figure 15 This is a flowchart illustrating an example process for decoding a compressed bitstream of grid data. Although regarding the V-DMC decoder 300 ( Figure 1 and Figure 3 The description is provided, but it should be understood that other devices can be configured to perform the same actions. Figure 15 The process is similar to that of [the other process].

[0506] exist Figure 15 In the example, the V-DMC decoder 300 determines the offset value (1404) based on one or more syntax elements. The V-DMC decoder 300 may, for example, extract a shift bit stream from a bit stream of encoded grid data and receive one or more syntax elements in the shift bit stream.

[0507] The V-DMC decoder 300 determines the set of transform coefficients (1406). To determine the set of transform coefficients, the V-DMC decoder 300 may, for example, receive a quantized set of transform coefficients and dequantize the quantized set of transform coefficients to determine the set of transform coefficients.

[0508] The V-DMC decoder 300 applies an offset to the transform coefficient set to determine the updated transform coefficient set (1408). To apply the offset to the transform coefficient set to determine the updated transform coefficient set, the V-DMC decoder 300 may add the offset to each coefficient in the transform coefficient set.

[0509] The V-DMC decoder 300 performs an inverse transform on the updated set of transform coefficients to determine the set of displacement vectors (1410). To perform the inverse transform on the updated set of transform coefficients, the V-DMC decoder 300 may apply an inverse lifting transform as described above. To perform the inverse transform on the updated set of transform coefficients to determine the set of displacement vectors, the V-DMC decoder 300 may perform an inverse transform on the updated set of transform coefficients to determine the values ​​of the normal components of the set of displacement vectors.

[0510] The V-DMC decoder 300 determines the decoding grid (1412) based on a set of displacement vectors. To determine the decoding grid, the V-DMC decoder 300 may be configured to: determine a base grid having a first set of vertices from the bitstream of the encoded grid data; subdivide the base grid to determine an additional set of vertices of the base grid; deform the base grid, wherein deforming the base grid includes: modifying the position of the additional set of vertices based on one or more displacement vectors; and determine the decoding grid based on the deformed base grid.

[0511] The first set of vertices may correspond to the highest level of detail (e.g., LOD0), and additional vertices may correspond to lower levels of detail (e.g., LOD1 to LODN, where N is greater than or equal to 2). The V-DMC decoder 300 may be configured to: determine a corresponding offset value for each level in the lower levels of detail; determine a corresponding set of transform coefficients for each level in the lower levels of detail; apply the corresponding offset of each level in the lower levels of detail to the corresponding set of transform coefficients for each level in the lower levels of detail to determine a corresponding updated set of transform coefficients for each level in the lower levels of detail; perform an inverse transform on the corresponding updated set of transform coefficients for each level in the lower levels of detail to determine a corresponding set of displacement vectors for each level in the lower levels of detail; and determine the decoded mesh based on the corresponding set of displacement vectors for each level in the lower levels of detail. To determine the corresponding offset value for each level in the lower levels of detail, the V-DMC decoder 300 may be configured to: receive the corresponding syntax for each level in the lower levels of detail.

[0512] The following numbered clauses illustrate one or more aspects of the devices and technologies described in this disclosure.

[0513] Clause 1A: A method for processing grid data, the method comprising: any technique or combination of techniques described in this disclosure.

[0514] Clause 2A: The method according to any one of Clauses 1, the method further comprising: generating the grid data.

[0515] Clause 3A: An apparatus for processing grid data, the apparatus comprising: a memory configured to store the grid data; and one or more processors coupled to the memory, the one or more processors being implemented in a circuit and configured to perform any or a combination of techniques described in this disclosure.

[0516] Clause 4A: The device as described in Clause 3A, wherein the device includes a decoder.

[0517] Clause 5A: The device as described in Clause 3A, wherein the device includes an encoder.

[0518] Clause 6A: The device according to any one of Clauses 3A to 4, the device further comprising: a device for generating the grid data.

[0519] Clause 7A: The device according to any one of Clauses 3A to 6 further includes a display for presenting an image based on data.

[0520] Clause 8A: A computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to perform any technology or combination of technologies described in this disclosure.

[0521] Clause 1B: An apparatus for decoding coded grid data, the apparatus comprising: a memory; processing circuitry coupled to the memory and configured to: receive one or more syntax elements in a bitstream of the coded grid data; determine offset values ​​based on the one or more syntax elements; determine a set of transform coefficients; apply the offsets to the set of transform coefficients to determine an updated set of transform coefficients; perform an inverse transform on the updated set of transform coefficients to determine a set of shift vectors; and determine a decoded grid based on the set of shift vectors.

[0522] Clause 2B: The apparatus according to Clause 1B, wherein, in order to perform an inverse transformation on the updated set of transform coefficients, the processing circuit is further configured to apply an inverse lift transformation.

[0523] Clause 3B: The apparatus according to Clause 1B or 2B, wherein, in order to perform an inverse transformation on the updated set of transformation coefficients to determine the set of displacement vectors, the processing circuit is configured to: perform an inverse transformation on the updated set of transformation coefficients to determine the values ​​of the normal components of the set of displacement vectors.

[0524] Clause 4B: The device according to any one of Clauses 1B to 3B, wherein, in order to apply the offset to the set of transform coefficients to determine the updated set of transform coefficients, the processing circuitry is configured to add the offset to each coefficient in the set of transform coefficients.

[0525] Clause 5B: An apparatus according to any one of Clauses 1B to 4B, wherein, in order to determine the decoded mesh, the processing circuitry is configured to: determine a base mesh having a first set of vertices from the bitstream of the encoded mesh data; subdivide the base mesh to determine an additional set of vertices of the base mesh; deform the base mesh, wherein deforming the base mesh includes: modifying the position of the additional set of vertices based on one or more displacement vectors; and determining the decoded mesh based on the deformed base mesh.

[0526] Clause 6B: The apparatus according to Clause 5B, wherein the first set of vertices corresponds to the highest level of detail, and the additional vertices correspond to lower levels of detail, and the processing circuitry is further configured to: determine a corresponding offset value for each level in the lower levels of detail; determine a corresponding set of transform coefficients for each level in the lower levels of detail; apply the corresponding offset of each level in the lower levels of detail to the corresponding set of transform coefficients for each level in the lower levels of detail to determine a corresponding updated set of transform coefficients for each level in the lower levels of detail; perform an inverse transform on the corresponding updated set of transform coefficients for each level in the lower levels of detail to determine a corresponding set of displacement vectors for each level in the lower levels of detail; and determine the decoded mesh based on the corresponding set of displacement vectors for each level in the lower levels of detail.

[0527] Clause 7B: The device according to Clause 6B, wherein, in order to determine the corresponding offset value for each of the lower detail levels, the processing circuitry is further configured to receive the corresponding syntax for each of the lower detail levels.

[0528] Clause 8B: The apparatus according to any one of Clauses 1B to 7B, wherein, in order to determine the set of transform coefficients, the processing circuitry is configured to: receive a quantized set of transform coefficients; and dequantize the quantized set of transform coefficients to determine the set of transform coefficients.

[0529] Clause 9B: An apparatus according to any one of Clauses 1B to 8B, wherein the processing circuitry is configured to: extract a shift bitstream from the bitstream of the encoded grid data; and receive the one or more syntax elements in the shift bitstream.

[0530] Clause 10B: A method for decoding coded grid data, the method comprising: receiving one or more syntax elements in a bitstream of the coded grid data; determining offset values ​​based on the one or more syntax elements; determining a set of transform coefficients; applying the offsets to the set of transform coefficients to determine an updated set of transform coefficients; performing an inverse transform on the updated set of transform coefficients to determine a set of shift vectors; and determining a decoded grid based on the set of shift vectors.

[0531] Clause 11B: The method according to Clause 10B, wherein performing an inverse transformation on the updated set of transform coefficients comprises: applying an inverse lifting transformation to the set of transform coefficients.

[0532] Clause 12B: The method according to Clause 10B or 11B, wherein performing an inverse transformation on the updated set of transformation coefficients to determine the set of displacement vectors comprises: performing an inverse transformation on the updated set of transformation coefficients to determine the values ​​of the normal components of the set of displacement vectors.

[0533] Clause 13B: The method according to any one of Clauses 10B to 12B, wherein applying the offset to the set of transform coefficients to determine the updated set of transform coefficients comprises: adding the offset to each coefficient in the set of transform coefficients.

[0534] Clause 14B: The method according to any one of Clauses 10B to 13B, wherein determining the decoded mesh comprises: determining a base mesh having a first set of vertices from the bitstream of the encoded mesh data; subdividing the base mesh to determine an additional set of vertices of the base mesh; deforming the base mesh, wherein deforming the base mesh comprises: modifying the position of the additional set of vertices based on one or more displacement vectors; and determining the decoded mesh based on the deformed base mesh.

[0535] Clause 15B: The method according to Clause 14B, wherein the first set of vertices corresponds to the highest level of detail and the additional vertices correspond to lower levels of detail, and the method further comprises: determining a corresponding offset value for each level in the lower levels of detail; determining a corresponding set of transform coefficients for each level in the lower levels of detail; applying the corresponding offset of each level in the lower levels of detail to the corresponding set of transform coefficients for each level in the lower levels of detail to determine a corresponding updated set of transform coefficients for each level in the lower levels of detail; performing an inverse transform on the corresponding updated set of transform coefficients for each level in the lower levels of detail to determine a corresponding set of displacement vectors for each level in the lower levels of detail; and determining the decoded mesh based on the corresponding set of displacement vectors for each level in the lower levels of detail.

[0536] Clause 16B: The method according to Clause 15B, wherein determining the corresponding offset value for each of the lower detail levels comprises: receiving the corresponding syntax for each of the lower detail levels.

[0537] Clause 17B: The method according to any one of Clauses 10B to 16B, wherein determining the set of transform coefficients comprises: receiving a quantized set of transform coefficients; and dequantizing the quantized set of transform coefficients to determine the set of transform coefficients.

[0538] Clause 18B: The method according to any one of Clauses 10B to 17B, the method further comprising: extracting a shift bitstream from the bitstream of the coded grid data; and receiving the one or more syntax elements in the shift bitstream.

[0539] Clause 19B: An apparatus for encoding grid data, the apparatus comprising: a memory; processing circuitry coupled to the memory and configured to: determine a set of displacement vectors of the grid data; transform the set of displacement vectors to determine a set of transform coefficients; determine an offset value for the set of transform coefficients; determine an offset value based on the offset value for the set of transform coefficients; subtract the offset value from the set of transform coefficients to determine offset-adjusted transform coefficients; quantize the offset-adjusted transform coefficients to determine quantization coefficients; and signal the quantization coefficients and the offset indication in a bitstream of the encoded grid data.

[0540] Clause 20B: The apparatus according to Clause 19B, wherein, in order to transform the set of displacement vectors, the processing circuit is further configured to apply a wavelet transform employing a lifting scheme.

[0541] Clause 21B: The apparatus according to Clause 19B or 20B, wherein, in order to determine the bias value of the set of transformation coefficients, the processing circuit is further configured to: determine the bias value of the value of the normal component of the set of displacement vectors.

[0542] Clause 22B: The apparatus according to any one of Clauses 19B to 21B, wherein, in order to determine the set of displacement vectors of the mesh data, the processing circuitry is further configured to: receive an input mesh; determine a base mesh based on the input mesh, wherein the base mesh includes a first set of vertices; determine a sub-mesh, wherein the sub-mesh includes an additional set of vertices; determine a first set of displacement vectors of the first set of vertices and a second set of displacement vectors of the additional set of vertices based on the input mesh and the base mesh; and output an encoded bitstream, the encoded bitstream including an encoded representation of the base mesh and an encoded representation of the displacement vectors.

[0543] Clause 23B: The device according to Clause 22B, wherein the first set of vertices corresponds to the highest level of detail and the additional vertices correspond to lower levels of detail, wherein, in order to determine the bias value of the set of transform coefficients, the processing circuitry is further configured to: determine the bias value of one level in the lower levels of detail; and in order to determine the offset value based on the bias value of the set of transform coefficients, the processing circuitry is further configured to: determine the offset value of the one level in the lower levels of detail.

[0544] Clause 24B: The apparatus according to Clause 23B, wherein the processing circuitry is further configured to: determine a corresponding bias value for a corresponding set of transform coefficients for each of the lower detail levels; determine a corresponding offset value based on the corresponding bias value for each corresponding set of transform coefficients; subtract the corresponding offset value from the corresponding set of transform coefficients to determine a corresponding bias-adjusted transform coefficient for each corresponding set of transform coefficients; quantize the corresponding bias-adjusted transform coefficient to determine a corresponding quantization coefficient for each corresponding set of transform coefficients; and signal an indication of the corresponding quantization coefficient and an indication of the corresponding offset in the bitstream of the encoded grid data.

[0545] Clause 25B: A method for encoding grid data, the method comprising: determining a set of displacement vectors of the grid data; transforming the set of displacement vectors to determine a set of transform coefficients; determining an offset value of the set of transform coefficients; determining an offset value based on the offset value of the set of transform coefficients; subtracting the offset value from the set of transform coefficients to determine transform coefficients for offset adjustment; quantizing the transform coefficients for offset adjustment to determine quantization coefficients; and signaling the quantization coefficients and an indication of the offset in a bit stream of the encoded grid data.

[0546] Clause 26B: The method according to Clause 25B, wherein transforming the set of displacement vectors comprises: applying a wavelet transform employing a lifting scheme.

[0547] Clause 27B: The method according to Clause 25B or 26B, wherein determining the bias value of the set of transformation coefficients comprises: determining the bias value of the values ​​of the normal components of the set of displacement vectors.

[0548] Clause 28B: The method according to any one of Clauses 25B to 27B, wherein determining the set of displacement vectors of the mesh data comprises: receiving an input mesh; determining a base mesh based on the input mesh, wherein the base mesh comprises a first set of vertices; determining a sub-mesh, wherein the sub-mesh comprises an additional set of vertices; determining a first set of displacement vectors of the first set of vertices and a second set of displacement vectors of the additional set of vertices based on the input mesh and the base mesh; and outputting an encoded bitstream, the encoded bitstream comprising an encoded representation of the base mesh and an encoded representation of the displacement vectors.

[0549] Clause 29B: The method according to Clause 28B, wherein the first set of vertices corresponds to the highest level of detail and the additional vertices correspond to lower levels of detail, wherein determining the bias value of the set of transform coefficients includes: determining the bias value of a level in the lower levels of detail, and determining the offset value based on the bias value of the set of transform coefficients includes: determining the offset value of the level in the lower levels of detail.

[0550] Clause 30B: The method according to Clause 29B further comprises: determining a corresponding bias value for a corresponding set of transform coefficients for each level in the lower detail level; determining a corresponding offset value based on the corresponding bias value for each corresponding set of transform coefficients; subtracting the corresponding offset value from the corresponding set of transform coefficients to determine a corresponding bias-adjusted transform coefficient for each corresponding set of transform coefficients; quantizing the corresponding bias-adjusted transform coefficient to determine a corresponding quantization coefficient for each corresponding set of transform coefficients; and signaling an indication of the corresponding quantization coefficient and an indication of the corresponding offset in the bitstream of the encoded grid data.

[0551] It should be recognized that, based on the examples, certain actions or events of any technique described herein may be performed in a different sequence, and may be added, combined, or omitted entirely (e.g., not all actions or events described are necessary for implementing the technique). Furthermore, in some examples, actions or events may be performed concurrently (e.g., through multithreading, interrupt handling, or multiple processors) rather than sequentially.

[0552] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. Thus, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0553] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead refer to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs utilize lasers to optically reproduce data. The combinations described above should also be included within the scope of computer-readable media.

[0554] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuit" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0555] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or IC sets (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but implementation by different hardware units is not necessarily required. Specifically, as described above, various units may be combined within a codec hardware unit, or may be provided by a collection of interoperable hardware units (including one or more processors as described above) combined with appropriate software and / or firmware.

[0556] Various examples have been described. These and other examples are within the scope of the following claims.< / double> < / vmesh::subdivisionlevelinfo> < / double> < / vmesh::subdivisionlevelinfo> < / vmesh::subdivisionlevelinfo>

Claims

1. An apparatus for decoding coded grid data, the apparatus comprising: Memory; and Processing circuitry, coupled to the memory and configured to: One or more syntax elements are received in the bit stream of the encoded grid data; The offset value is determined based on one or more of the syntax elements; Determine the set of transformation coefficients; The offset is applied to the set of transform coefficients to determine the updated set of transform coefficients; Perform an inverse transformation on the updated set of transformation coefficients to determine the set of displacement vectors; and The decoding grid is determined based on the set of displacement vectors.

2. The device according to claim 1, wherein, In order to perform an inverse transformation on the updated set of transform coefficients, the processing circuit is further configured to apply an inverse lift transform.

3. The device according to claim 1, wherein, In order to perform an inverse transformation on the updated set of transformation coefficients to determine the set of displacement vectors, the processing circuit is configured to perform an inverse transformation on the updated set of transformation coefficients to determine the values ​​of the normal components of the set of displacement vectors.

4. The device according to claim 1, wherein, In order to apply the offset to the set of transform coefficients to determine the updated set of transform coefficients, the processing circuit is configured to add the offset to each coefficient in the set of transform coefficients.

5. The device according to claim 1, wherein, To determine the decoding grid, the processing circuit is configured as follows: Determine a base grid with a first set of vertices from the bit stream of the encoded grid data; The base mesh is subdivided to determine the additional set of vertices of the base mesh; Deforming the base mesh, wherein deforming the base mesh includes: modifying the positions of the additional vertex set based on one or more translation vectors; and The decoding mesh is determined based on the deformed base mesh.

6. The device according to claim 5, wherein, The first set of vertices corresponds to the highest level of detail, and the additional vertices correspond to lower levels of detail, and the processing circuit is further configured to: Determine the corresponding offset value for each level in the lower level of detail; Determine the corresponding set of transformation coefficients for each level in the lower detail levels; The corresponding offset of each level in the lower detail level is applied to the corresponding set of transformation coefficients of each level in the lower detail level to determine the corresponding updated set of transformation coefficients for each level in the lower detail level. Perform an inverse transform on the corresponding updated set of transform coefficients for each of the lower detail levels to determine the corresponding set of displacement vectors for each of the lower detail levels; and The decoding mesh is determined based on the corresponding set of displacement vectors for each of the lower detail levels.

7. The device according to claim 6, wherein, In order to determine the corresponding offset value for each level in the lower detail levels, the processing circuitry is further configured to receive the corresponding syntax for each level in the lower detail levels.

8. The device according to claim 1, wherein, To determine the set of transformation coefficients, the processing circuit is configured as follows: Receive the set of quantization transform coefficients; and The quantized transform coefficient set is dequantized to determine the transform coefficient set.

9. The device according to claim 1, wherein, The processing circuit is configured as follows: Extract a shifted bitstream from the bitstream of the encoded grid data; and Receive one or more syntax elements from the bitstream.

10. The device according to claim 1, further comprising: A display configured to display the decoded grid.

11. A method for decoding coded grid data, the method comprising: One or more syntax elements are received in the bit stream of the encoded grid data; The offset value is determined based on one or more of the syntax elements; Determine the set of transformation coefficients; The offset is applied to the set of transform coefficients to determine the updated set of transform coefficients; Perform an inverse transformation on the updated set of transformation coefficients to determine the set of displacement vectors; and The decoding grid is determined based on the set of displacement vectors.

12. The method according to claim 11, wherein, Performing an inverse transformation on the updated set of transform coefficients includes applying an inverse lifting transformation to the set of transform coefficients.

13. The method according to claim 11, wherein, Performing an inverse transformation on the updated set of transformation coefficients to determine the set of displacement vectors includes: performing an inverse transformation on the updated set of transformation coefficients to determine the values ​​of the normal components of the set of displacement vectors.

14. The method according to claim 11, wherein, Applying the offset to the set of transform coefficients to determine the updated set of transform coefficients includes adding the offset to each coefficient in the set of transform coefficients.

15. The method according to claim 11, wherein, Determining the decoding grid includes: Determine a base grid with a first set of vertices from the bit stream of the encoded grid data; The base mesh is subdivided to determine the additional set of vertices of the base mesh; Deforming the base mesh, wherein deforming the base mesh includes: modifying the positions of the additional vertex set based on one or more translation vectors; and The decoding mesh is determined based on the deformed base mesh.

16. The method according to claim 15, wherein, The first set of vertices corresponds to the highest level of detail, and the additional vertices correspond to lower levels of detail. The method further includes: Determine the corresponding offset value for each level in the lower level of detail; Determine the corresponding set of transformation coefficients for each level in the lower detail levels; The corresponding offset of each level in the lower detail level is applied to the corresponding set of transformation coefficients of each level in the lower detail level to determine the corresponding updated set of transformation coefficients for each level in the lower detail level. Perform an inverse transform on the corresponding updated set of transform coefficients for each of the lower detail levels to determine the corresponding set of displacement vectors for each of the lower detail levels; and The decoding mesh is determined based on the corresponding set of displacement vectors for each of the lower detail levels.

17. The method according to claim 16, wherein, Determining the corresponding offset value for each level in the lower detail level includes receiving the corresponding syntax for each level in the lower detail level.

18. The method according to claim 11, wherein, Determining the set of transformation coefficients includes: Receive the set of quantization transform coefficients; and The quantized transform coefficient set is dequantized to determine the transform coefficient set.

19. The method according to claim 11, further comprising: Extract the shift bit stream from the bit stream of the encoded grid data; as well as Receive one or more syntax elements from the bitstream.

20. An apparatus for encoding grid data, the apparatus comprising: Memory; Processing circuitry, coupled to the memory and configured to: Determine the set of displacement vectors for the grid data; The set of displacement vectors is transformed to determine the set of transformation coefficients; Determine the bias value of the set of transformation coefficients; The offset value is determined based on the bias value of the set of transformation coefficients; Subtract the offset value from the set of transformation coefficients to determine the transformation coefficients for bias adjustment; The transformation coefficients of the bias adjustment are quantized to determine the quantization coefficients; as well as The quantization coefficients and the offset are signaled in the bit stream of the encoded grid data.

21. The device according to claim 20, wherein, In order to transform the set of displacement vectors, the processing circuit is further configured to apply a wavelet transform with a lifting scheme.

22. The device according to claim 20, wherein, In order to determine the bias value of the set of transformation coefficients, the processing circuit is further configured to determine the bias value of the normal component of the set of displacement vectors.

23. The device according to claim 20, wherein, To determine the set of displacement vectors of the grid data, the processing circuit is further configured to: Receive input grid; A base grid is determined based on the input grid, wherein the base grid includes a first set of vertices; Determine a sub-mesh, wherein the sub-mesh includes an additional set of vertices; Based on the input mesh and the base mesh, determine the first set of displacement vectors for the first vertex set and the second set of displacement vectors for the additional vertex set; and Output an encoded bitstream, which includes an encoded representation of the base grid and an encoded representation of the displacement vector.

24. The apparatus of claim 23, further comprising: A graphics engine configured to generate the input grid.

25. The device according to claim 23, wherein, The first set of vertices corresponds to the highest level of detail, and the additional vertices correspond to lower levels of detail, wherein To determine the bias value of the set of transform coefficients, the processing circuit is further configured to: determine the bias value of one of the lower detail levels, and In order to determine the offset value based on the bias value of the set of transformation coefficients, the processing circuit is further configured to determine the offset value of one of the lower detail levels.

26. The device according to claim 25, wherein, The processing circuit is further configured to: Determine the corresponding bias value for the set of transformation coefficients for each level in the lower detail level; The corresponding offset value is determined based on the corresponding bias value of each corresponding set of transform coefficients; Subtract the corresponding offset value from the corresponding set of transformation coefficients to determine the corresponding offset-adjusted transformation coefficients for each set of transformation coefficients; The corresponding bias adjustment transform coefficients are quantized to determine the corresponding quantization coefficients for each corresponding transform coefficient set; as well as In the bitstream of the encoded grid data, signals are sent to indicate the corresponding quantization coefficients and the corresponding offsets.

27. A method for encoding grid data, the method comprising: Determine the set of displacement vectors for the grid data; The set of displacement vectors is transformed to determine the set of transformation coefficients; Determine the bias value of the set of transformation coefficients; The offset value is determined based on the bias value of the set of transformation coefficients; Subtract the offset value from the set of transformation coefficients to determine the transformation coefficients for bias adjustment; The transformation coefficients of the bias adjustment are quantized to determine the quantization coefficients; as well as The quantization coefficients and the offset are signaled in the bit stream of the encoded grid data.

28. The method according to claim 27, wherein, Determining the set of displacement vectors for the grid data includes: Receive input grid; A base grid is determined based on the input grid, wherein the base grid includes a first set of vertices; Determine a sub-mesh, wherein the sub-mesh includes an additional set of vertices; Based on the input mesh and the base mesh, determine the first set of displacement vectors for the first vertex set and the second set of displacement vectors for the additional vertex set; and Output an encoded bitstream, which includes an encoded representation of the base grid and an encoded representation of the displacement vector.

29. The method according to claim 28, wherein, The first set of vertices corresponds to the highest level of detail, and the additional vertices correspond to lower levels of detail, wherein Determining the bias value of the set of transform coefficients includes: determining the bias value of one of the lower detail levels, and Determining the offset value based on the bias value of the set of transformation coefficients includes: determining the offset value of the one level in the lower detail level.

30. The method according to claim 29, further comprising: Determine the corresponding bias value for the set of transformation coefficients for each level in the lower detail level; The corresponding offset value is determined based on the corresponding bias value of each corresponding set of transform coefficients; Subtract the corresponding offset value from the corresponding set of transformation coefficients to determine the corresponding offset-adjusted transformation coefficients for each set of transformation coefficients; The corresponding bias adjustment transform coefficients are quantized to determine the corresponding quantization coefficients for each corresponding transform coefficient set; as well as In the bitstream of the encoded grid data, a signal is sent to indicate the corresponding quantization coefficient and the corresponding offset.

Images