Inverse quantization adjustment of 3D mesh displacement

Abstract

A decoder decodes quantized wavelet coefficients representing the displacement of vertices in a set of vertices of a three-dimensional (3D) mesh from a bitstream, and decodes quantization information from the bitstream indicating a quantization offset associated with a subset of vertices in the set of vertices. The decoder further performs inverse quantization on the quantized wavelet coefficients to determine wavelet coefficients, and adjusts the wavelet coefficients according to the quantization offset based on the vertices being in the subset to determine the displacement of the vertices.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 543,730, filed October 11, 2023, and U.S. Provisional Application No. 63 / 621,339, filed January 16, 2024, which are incorporated herein by reference in their entirety. Attached Figure Description

[0002] Examples of several embodiments of the various embodiments of this disclosure are described herein with reference to the accompanying drawings.

[0003] Figure 1 An exemplary mesh decoding / decoding system in which embodiments of this disclosure may be implemented is described.

[0004] Figure 2A A block diagram illustrating an example encoder for intra-frame encoding of 3D meshes according to some embodiments is provided.

[0005] Figure 2B A block diagram illustrating an example encoder for inter-frame coding of 3D meshes according to some embodiments is provided.

[0006] Figure 3 The illustration shows a diagram illustrating the example decoder.

[0007] Figure 4 This is a diagram illustrating an example process for generating the displacement of an input mesh (e.g., an input 3D mesh frame) to be encoded, according to some embodiments.

[0008] Figure 5 An example process for approximating and encoding the geometry of a 3D mesh, according to some embodiments, is described.

[0009] Figure 6 Examples of vertices of a mesh (e.g., a subdivided base mesh) corresponding to multiple levels of detail (LOD) are illustrated according to some embodiments.

[0010] Figure 7A Examples are illustrated using a packing method to fill images with displacements (e.g., displacement fields or vectors) according to some embodiments.

[0011] Figure 7B Examples of displacement images with labeled LODs are illustrated according to some embodiments.

[0012] Figure 8 Examples of quantizers for quantizing transformed wavelet coefficients representing displacements of a 3D mesh, according to some embodiments, are described.

[0013] Figure 9Examples of inverse quantizers, according to some embodiments, are described for inverse quantizing the quantized-transformed wavelet coefficients representing the displacement of a 3D mesh.

[0014] Figure 10 Examples of inverse quantizers with quantization offsets to inverse quantize the quantized-transformed wavelet coefficients representing the displacement of a 3D mesh are illustrated according to some embodiments.

[0015] Figure 11 A is a diagram illustrating an example process for iteratively inversely quantizing the quantized-transformed wavelet coefficients of vertices in each subset of the vertices of a 3D mesh, according to some embodiments.

[0016] Figure 11 B is a diagram illustrating an example process for iteratively inversely quantizing the quantized-transformed wavelet coefficients of vertices in each subset of the vertices of a 3D mesh, according to some embodiments.

[0017] Figure 12 A flowchart illustrating a method for applying a quantization scheme to transformed wavelet coefficients according to some embodiments is provided.

[0018] Figure 13 A flowchart illustrating a method for applying an inverse quantization scheme to quantized-transformed wavelet coefficients according to some embodiments is provided.

[0019] Figure 14 A block diagram illustrating an exemplary computer system in which embodiments of the present disclosure may be implemented. Detailed Implementation

[0020] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure, including its structures, systems, and methods, can be practiced without these specific details. The descriptions and representations herein are common means used by those skilled in the art to most effectively communicate the substance of their work to others skilled in the art. In other instances, well-known methods, procedures, components, and circuits have not been described in detail to avoid unnecessarily obscuring aspects of this disclosure.

[0021] References to "an embodiment," "embodiment," "example embodiment," etc., in this specification indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment must include that particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Additionally, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is believed that its influence on such feature, structure, or characteristic in conjunction with other embodiments is within the knowledge of those skilled in the art, whether explicitly described or not.

[0022] Furthermore, note that the various embodiments can be described as processes depicted as flowcharts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. While flowcharts can describe operations as sequential processes, many of these operations can be performed in parallel or simultaneously. Moreover, the order of operations can be rearranged. When an operation of a process is completed, the process terminates, but there may be additional steps not included in the diagram. A process can correspond to a method, function, procedure, subroutine, subroutine, etc. When a process corresponds to a function, the termination of the process can correspond to the function returning to the calling function or the main function.

[0023] The term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, including, or carrying instructions and / or data. Computer-readable media may include non-transitory media on which data can be stored but does not include carrier waves and / or transient electronic signals propagating via wireless or wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or magnetic tapes, optical storage media such as optical discs (CDs) or digital multifunction discs (DVDs), flash memory, memory, or memory devices. Computer-readable media may store code and / or machine-executable instructions thereon, which may represent procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. Code segments may be coupled to other code segments or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be passed, forwarded, or transmitted via any suitable means, including memory sharing, messaging, token passing, network transmission, etc.

[0024] Furthermore, embodiments can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., computer program products) that perform the necessary tasks can be stored in a computer-readable or machine-readable medium. The processor can perform the necessary tasks.

[0025] Traditional visual data uses a series of pixels to describe an object or scene, each pixel including its location in two dimensions (x and y) and one or more optional attributes, such as color. Volumetric visual data adds another dimension of location to this traditional visual data. Volumetric visual data uses a series of points to describe an object or scene, each point including its location in three dimensions (x, y, and z) and one or more optional attributes, such as color. Compared to traditional visual data, volumetric visual data can provide a more immersive way to experience visual data. For example, an object or scene described by volumetric visual data can be viewed from any (or multiple) angles, while traditional visual data is generally only viewable from the angle from which it was captured or rendered. Volumetric visual data can be used in many applications, including augmented reality (AR), virtual reality (VR), and mixed reality (MR). Volumetric visual data can be presented as volumetric frames describing an object or scene captured at a specific time instance, or as a sequence of volumetric frames (called a volumetric sequence or volumetric video) describing objects or scenes captured at multiple different time instances.

[0026] One format used to store volumetric visual data is a three-dimensional (3D) mesh (hereinafter referred to as a mesh or mesh frame). A mesh frame (or mesh) comprises a set of points (also called vertices) in three-dimensional (3D) space. Each vertex in the mesh includes geometric information indicating the vertex's location in 3D space. For example, the geometric information can use three Cartesian coordinates (x, y, and z) to indicate the vertex's location in 3D space. Furthermore, a mesh may include geometric information indicating multiple triangles. Each triangle comprises three vertices connected by three edges and one face. One or more types of attribute information can be stored for each face (of the triangle). Attribute information can indicate characteristics of the face's visual appearance. For example, attribute information can indicate the face's texture (e.g., color), the face's material type, the face's transparency information, the face's reflectivity information, the face's surface normal vector, the velocity at the face, the acceleration at the face, a timestamp indicating when the face (and / or vertex) was captured, or a modality indicating how the face (and / or vertex) was captured (e.g., running, walking, or flying). In another example, a face (or vertex) may include light field data in the form of multi-view-dependent texture information. The light field data may also be another type of optional attribute information.

[0027] Triangles in a mesh (e.g., represented by vertices and edges) can describe an object or scene. For example, triangles in a mesh can describe the external surfaces and / or internal structures of an object or scene. Objects or scenes can be generated computer-generated synthetically or based on captures of real-world objects or scenes. Geometric information of real-world objects or scenes can be obtained through 3D scanning and / or photogrammetry. 3D scanning can include laser scanning, structured light scanning, and / or modulated light scanning. 3D scanning obtains geometric information by moving one or more laser heads, structured light cameras, and / or modulated light cameras relative to the scanned object or scene. Photogrammetry obtains geometric information by triangulating the same features or points in 2D photographs at different spatial displacements. Mesh data can be in the form of mesh frames describing an object or scene captured at a specific time instance, or in the form of a sequence of mesh frames (referred to as a mesh sequence or mesh video) describing objects or scenes captured at multiple different time instances.

[0028] In addition to one or more types of attribute information, the data size of a mesh frame or sequence can be too large for storage and / or transmission in many applications. For example, a single mesh frame may include thousands, tens of thousands, or hundreds of thousands of triangles, each of which (e.g., vertices and / or edges) includes geometric information and one or more optional types of attribute information. The geometric information for each vertex may include three Cartesian coordinates (x, y, and z) represented, for example, using a total of 8 bits or 24 bits. The attribute information for each vertex may include a texture corresponding to three color components (e.g., R, G, and B color components) represented, for example, using a total of 8 bits or 24 bits. Thus, in this example, a single vertex includes 48 bits of information, with 24 bits of geometric information and 24 bits of texture. Encoding can be used to compress the size of the mesh frame or sequence to provide more efficient storage and / or transmission. Decoding can be used to decompress the compressed mesh frame or sequence for display and / or other forms of consumption (e.g., by machine learning-based devices, neural network-based devices, artificial intelligence-based devices, or other forms of consumption by other types of machine-based processing algorithms and / or devices).

[0029] Mesh compression can be lossy (e.g., introducing differences relative to the original data) for use in applications such as AR / VR glasses or any other 3D-capable devices, for distribution to and visualization by an end user. Lossy compression allows for very high compression ratios but incurs a trade-off between compression and the visual quality perceived by the end user. Other frames, such as those used in medical or geological applications, may require lossless compression to avoid altering the decompressed mesh.

[0030] Volumetric visual data can be stored after being encoded into a bitstream in a container (such as a file server on a network). End users can request specific bitstreams based on their needs. Users can also request adaptive streaming of the bitstream, where an algorithm considers the trade-off between network resource consumption and the visual quality perceived by the end user.

[0031] Figure 1 An exemplary grid decoding / decoding system 100 in which embodiments of the present disclosure may be implemented is described. The grid decoding / decoding system 100 includes a source device 102, a transmission medium 104, and a target device 106. The source device 102 encodes a grid sequence 108 into a bit stream 110 for more efficient storage and / or transmission. The source device 102 may store the bit stream 110 and / or may transmit the bit stream to the target device 106 via the transmission medium 104. The target device 106 decodes the bit stream 110 to display the grid sequence 108 or for other forms of consumption. The target device 106 may receive the bit stream 110 from the source device 102 via the storage medium or the transmission medium 104. The source device 102 and the target device 106 can be any of a variety of different devices, including an interconnected cluster of computer systems (also known as a computer cloud or cloud computing) acting as a seamless resource pool, servers, desktop computers, laptops, tablets, smartphones, wearable devices, televisions, cameras, video game consoles, set-top boxes, video streaming devices, autonomous vehicles, or head-mounted displays. Head-mounted displays can allow users to view VR, AR, or MR scenes and adjust the viewpoint of the scene based on the user's head movements. Head-mounted displays can be tethered to processing devices (such as servers, desktop computers, set-top boxes, or video game consoles) or can be completely independent.

[0032] To encode the mesh sequence 108 into a bitstream 110, the source device 102 may include a mesh source 112, an encoder 114, and an output interface 116. The mesh source 112 may provide or generate the mesh sequence 108 based on the capture of natural scenes and / or synthetically generated scenes. The synthetically generated scene may be a scene that includes computer-generated graphics. The mesh source 112 may include one or more mesh capture devices (e.g., one or more laser scanning devices, structured light scanning devices, modulated light scanning devices, and / or passive scanning devices), a mesh archive including previously captured natural scenes and / or synthetically generated scenes, a mesh feed interface for receiving the captured natural scenes and / or synthetically generated scenes from a mesh content provider, and / or a processor for generating synthetic mesh scenes.

[0033] like Figure 1As described, mesh sequence 108 may include a series of mesh frames 124. Mesh frames describe objects or scenes captured at a specific time instance. When mesh frames 124 of mesh sequence 108 are rendered continuously using constant or variable time, mesh sequence 108 can achieve the impression of motion. A (3D) mesh frame includes a set of vertices 126 in 3D space and geometric information about those vertices 126. A 3D mesh may include a set of vertices, edges, and faces that define the shape of a polyhedral object. Furthermore, a mesh frame includes multiple triangles (e.g., polygonal triangles). For example, a triangle may include vertices 134A to 134C and edges 136A to 136C, as well as face 132. Faces are typically composed of triangles (triangular meshes), quadrilaterals (quads), or other simple convex polygons (n-sided polygons) because this simplifies rendering, but these faces may also be more generally composed of concave polygons, or even polygons with holes. Each vertex in the vertices 126 may include geometric information indicating the point's location in 3D space. For example, geometric information can use three Cartesian coordinates (x, y, and z) to indicate the location of a point in 3D space. For example, geometric information can indicate multiple triangles, each comprising three vertices from vertices 126. One or more triangles can also include one or more types of attribute information. Attribute information can indicate characteristics of the point's visual appearance. For example, attribute information can indicate the texture (e.g., color) of a face, the material type of a face, the transparency information of a face, the reflectivity information of a face, the surface normal of a face, the velocity at a face, the acceleration at a face, a timestamp indicating when a face was captured, or the mode in which the face was captured (e.g., running, walking, or flying). In another example, one or more faces (or triangles) can include light field data in the form of multi-view-dependent texture information. Light field data can be another type of optional attribute information. Color attribute information for one or more faces can include a lightness value and two chromaticity values. The lightness value can represent the brightness of the point (or the lightness component Y). The chromaticity values ​​can represent the blue and red components (or chromaticity components Cb and Cr) of the point, separated from the brightness. Depending on the color scheme (e.g., RGB or monochrome), other color attribute values ​​are possible.

[0034] In some embodiments, the 3D mesh (e.g., one mesh frame in mesh frame 124) can be a static or dynamic mesh. In some examples, the 3D mesh can be represented (e.g., defined) by connectivity information, geometric information, and texture information (e.g., texture coordinates and texture connectivity). In some embodiments, the geometric information can represent the positions of the vertices of the 3D mesh in 3D space, and the connectivity information can indicate how the vertices will connect together to form polygons (e.g., triangles) constituting the 3D mesh. Additionally, texture coordinates indicate the positions of pixels in the 2D image corresponding to the vertices of the corresponding 3D mesh (or a submesh of the 3D mesh). In some examples, patch information can indicate how the texture coordinates defined relative to the 2D bounding box are mapped to the 3D space of the 3D bounding box associated with the patch, based on how points are projected onto a projection plane used for the patch. Additionally, texture connectivity information can indicate how the vertices represented by the texture coordinates will connect together to form polygons of the 3D mesh (or a submesh). For example, each texture or attribute patch of a texture image can correspond to a corresponding submesh defined using texture coordinates and texture connectivity.

[0035] In some embodiments, for each 3D mesh, one or more 2D images may represent the texture or properties associated with the mesh. For example, texture information may include geometric information listed as X, Y, and Z coordinates of vertices, and texture coordinates listed as 2D coordinates corresponding to the vertices. An example textured mesh may include texture connectivity information indicating the mapping between geometric coordinates and texture coordinates to form polygons, such as triangles. For example, a first triangle may be formed by three vertices, where the first vertex is defined as a first geometric coordinate (e.g., 64.062500, 1237.739990, 51.757801) corresponding to a first texture coordinate (e.g., 0.0897381, 0.740830). A second vertex of the triangle may be defined as a second geometric coordinate (e.g., 59.570301, 1236.819946, 54.899700) corresponding to a second texture coordinate (e.g., 0.899059, 0.741542). Finally, the third vertex of the triangle can correspond to the geometric coordinates listed in the third list that match the texture coordinates listed in the third list. However, it should be noted that in some cases, the vertices of a polygon, such as a triangle, can be mapped to a set of geometric and texture coordinates that may have different index locations in their respective lists. For example, a second triangle has a first vertex corresponding to a set of geometric coordinates listed in the fourth list and a set of texture coordinates listed in the seventh list. A second vertex corresponds to a set of geometric and texture coordinates listed in the first list, and a third vertex corresponds to a set of geometric and texture coordinates listed in the third and ninth lists.

[0036] Encoder 114 can encode the grid sequence 108 into a bitstream 110. To encode the grid sequence 108, encoder 114 can apply one or more prediction techniques to reduce redundant information in the grid sequence 108. Redundant information is information that can be predicted at the decoder and therefore may not need to be transmitted to the decoder for accurate decoding of the grid sequence 108. For example, encoder 114 can convert attribute information (e.g., texture information) of one or more grid frames in grid frame 124 from 3D to 2D, and then apply one or more 2D video encoders or encoding methods to the 2D image. For example, any of several different proprietary or standardized 2D video encoders / decoders can be used, including ITU-T H.1263, ITU-T H.1264 and Moving Picture Experts Group (MPEG)-4 Vision (also known as Advanced Video Decoding (AVC)), ITU-T H.1265 and MPEG-H Part 2 (also known as High Efficiency Video Decoding (HEVC), ITU-T H.1265 and MPEG-I Part 3 (also known as Universal Video Decoding (VVC)), WebM VP8 and VP9 decoders, and AOMedia Video 1 (AV1). Encoder 114 can encode the geometry of the mesh sequence 108 based on Video Dynamic Mesh Decoding (V-DMC). V-DMC specifies the bitstream syntax and semantics for transmitting or storing the mesh sequence, as well as the decoder operations for reconstructing the mesh sequence from the bitstream.

[0037] Output interface 116 may be configured to write and / or store bit stream 110 onto transmission medium 104 for transmission to target device 106. Alternatively, output interface 116 may be configured to transmit, upload, and / or stream bit stream 110 to target device 106 via transmission medium 104. Output interface 116 may include a wired and / or wireless transmitter configured to transmit, upload, and / or stream bit stream 110 according to one or more proprietary and / or standardized communication protocols such as Digital Video Broadcasting (DVB) standard, Advanced Television Systems Committee (ATSC) standard, Integrated Services Digital Broadcasting (ISDB) standard, Cable Data Service Interface Specification (DOCSIS) standard, 3rd Generation Partnership Project (3GPP) standard, Institute of Electrical and Electronics Engineers (IEEE) standard, Internet Protocol (IP) standard, and Wireless Application Protocol (WAP) standard.

[0038] The transmission medium 104 may include wireless, wired, and / or computer-readable media. For example, the transmission medium 104 may include one or more wires, cables, air interfaces, optical discs, flash memory, and / or magnetic storage. Additionally or alternatively, the transmission medium 104 may include one or more networks (e.g., the Internet) or file servers configured to store and / or transmit encoded video data.

[0039] To decode bitstream 110 into a grid sequence 108 for display or other forms of consumption, target device 106 may include input interface 118, decoder 120, and grid display 122. Input interface 118 may be configured to be read by source device 102 from bitstream 110 stored on transmission medium 104. Additionally or alternatively, input interface 118 may be configured to receive, download, and / or stream bitstream 110 from source device 102 via transmission medium 104. Input interface 118 may include a wired and / or wireless receiver configured to receive, download, and / or stream bitstream 110 according to one or more proprietary and / or standardized communication protocols (such as those mentioned above).

[0040] Decoder 120 can decode the mesh sequence 108 from the encoded bitstream 110. To decode the attribute information (e.g., texture) of the mesh sequence 108, decoder 120 can reconstruct a 2D image compressed using one or more 2D video encoders. Decoder 120 can then reconstruct the attribute information of the 3D mesh frame 124 based on the reconstructed 2D image. In some examples, decoder 120 can decode a mesh sequence that approximates the mesh sequence 108 due to, for example, lossy compression of the mesh sequence 108 by encoder 114 and / or errors introduced into the encoded bitstream 110 during transmission to the target device 106. Furthermore, decoder 120 can decode the geometry of the mesh sequence 108 from the encoded bitstream 110, as will be further described below. Subsequently, one or more of the decoded attribute information can be applied to the decoded mesh frame of the mesh sequence 108.

[0041] The grid display 122 can display the grid sequence 108 to a user. The grid display 122 may include a cathode ray tube (CRT) display, a liquid crystal display (LCD), a plasma display, a light-emitting diode (LED) display, a 3D display, a holographic display, a head-mounted display, or any other display device suitable for displaying the grid sequence 108.

[0042] It should be noted that the lattice decoding / decoding system 100 is presented as an example rather than a limitation. Figure 1In this example, the mesh decoding / decoding system 100 may have other components and / or arrangements. For example, the mesh source 112 may be external to the source device 102. Similarly, the mesh display 122 may be external to the target device 106, or may be omitted entirely if the mesh sequence is intended for consumption by a machine and / or storage device. In another example, the source device 102 may also include a mesh decoder, and the target device 106 may include a mesh encoder. In this example, the source device 102 may be configured to further receive encoded bitstreams from the target device 106 to support bidirectional mesh transmission between devices.

[0043] Figure 2A A block diagram illustrating an example encoder 200A for intra-frame encoding of a 3D mesh, according to some embodiments, is provided. For example, an encoder (e.g., encoder 114) may include encoder 200A.

[0044] In some examples, a grid sequence (e.g., grid sequence 108) may include a set of grid frames (e.g., grid frame 124) that can be individually encoded and decoded. The following will discuss... Figure 4 Further, a base mesh 252 can be determined (e.g., generated) based on a mesh frame (e.g., an input mesh) through an decimation process. During decimation, the mesh topology of the mesh frame can be reduced to determine the base mesh (e.g., the decimated mesh or the decimated base mesh). A mesh encoder 204 can encode the base mesh 252, whose geometric information (e.g., vertices) can be quantized by a quantizer 202 to generate the base mesh bitstream 254. In some examples, the base mesh encoder 204 can be an existing encoder, such as Draco or Edgebreaker.

[0045] Displacement generator 208 can generate displacements for vertices of a mesh frame based on the base mesh 252, as will be discussed below. Figure 4 and Figure 5 Further explanation. In some examples, displacement is determined based on the reconstructed base mesh 256. The reconstructed base mesh 256 may be determined (e.g., output or generated) by a mesh decoder 206 that decodes the encoded base mesh (e.g., in the base mesh bitstream 254) determined (e.g., output or generated) by a mesh encoder 204. The displacement generator 208 may use a subdivision scheme (e.g., a subdivision algorithm) to subdivide the reconstructed base mesh 256 to determine a subdivided mesh (e.g., a subdivided base mesh). Displacement 258 may be determined based on fitting the subdivided mesh to the original input mesh surface. For example, displacement 258 for vertices in a mesh frame may include displacement information (e.g., a displacement vector) indicating the displacement from the corresponding vertex's location in the subdivided mesh to the vertex's location in the mesh frame.

[0046] The displacement 258 can be transformed by wavelet transform 210 to generate wavelet coefficients (e.g., transform coefficients) representing the displacement information and which can be encoded (and subsequently decoded) more efficiently. The wavelet coefficients can be quantized by quantizer 212 and packed (e.g., arranged) by image packer 214 into a picture (e.g., one or more images or picture frames) to be encoded by video encoder 216. Multiplexer 218 can combine (e.g., multiplex) the displacement bitstream 260 output by video encoder 216 with the base grid bitstream 254 to form bitstream 266.

[0047] The attribute information 262 of the mesh frame (e.g., color, texture, etc.) can be encoded separately from the geometric information of the mesh frame described above. In some examples, the attribute information 262 of the mesh frame can be represented (e.g., stored) by an attribute graph (e.g., a texture graph) that associates each vertex of the mesh frame with the corresponding attribute information of that vertex. Attribute transfer 232 can reparameterize the attribute information 262 in the attribute graph based on the reconstructed mesh determined (e.g., generated or output) according to the mesh reconstruction component 225. The mesh reconstruction component 225 performs inverse or decoding functions and can be a decoder (e.g., ...). Figure 3 The same or similar components in the decoder 300. For example, the inverse quantizer 228 can inverse quantize the reconstructed base grid 256 to determine (e.g., generate or output) the reconstructed base grid 268. The video decoder 226, image unpacker 224, inverse quantizer 222, and inverse wavelet transform 220 can perform functions opposite to those of the video encoder 216, image packer 214, quantizer 212, and wavelet transform 210, respectively. Therefore, the reconstructed displacement 270 corresponding to displacement 258 can be generated by sequentially applying the video decoder 226, image unpacker 224, inverse quantizer 222, and inverse wavelet transform 220. The deformable grid reconstructor 230 can determine the reconstructed grid corresponding to the input grid frame based on the reconstructed base grid 268 and the reconstructed displacement 270. In some examples, the reconstructed grid can be the same decoded grid determined by the decoder based on decoding the base grid bitstream 254 and the displacement bitstream 260.

[0048] The attribute information of the reparameterized attribute map can be packed into the image (e.g., a 2D image or picture frame) by padding component 234. Padding component 234 can fill (e.g., pad) portions of the image that do not contain attribute information. In some examples, color space converter 236 can convert the color representation (e.g., an example of attribute information 262) from a first format to a second format (e.g., from RGB444 to YUV420) to achieve improved rate-distortion (RD) performance when encoding the attribute map. In examples, color space converter 236 can also perform chroma subsampling to further improve encoding performance. Finally, video encoder 240 encodes the image (e.g., a picture frame) representing the attribute information 262 of the grid frame to determine (e.g., generate or output) the attribute bitstream 264 multiplexed by multiplexer 218 into bitstream 266. In some examples, video encoder 240 can be an existing 2D video compression encoder, such as an HEVC encoder or a VVC encoder.

[0049] Figure 2B A block diagram illustrating an example encoder 200B for inter-frame coding of a 3D mesh, according to some embodiments, is provided. For example, an encoder (e.g., encoder 114) may include encoder 200B. Figure 2B As described, encoder 200B includes many of the same components as encoder 200A. Compared to encoder 200A, encoder 200B does not include the mesh encoder 204 and mesh decoder 206 corresponding to the decoder used for static 3D meshes. Instead, encoder 200B includes a motion encoder 242, a motion decoder 244, and a base mesh reconstructor 246. Motion encoder 242 determines a motion field (e.g., one or more motion vectors (MV)) that best approximates a base mesh 252 when applied to a reference base mesh 243 for reconstructing quantization.

[0050] The determined motion field can be encoded in bit stream 266 as motion bit stream 272. In some examples, the motion field (e.g., motion vectors in the x, y, and z directions) can be entropy-decoded into codewords (e.g., for each direction component) generated by a decoding scheme (such as unary codes, Golomb codes (e.g., Exp-Golomb codes), Rice codes, or combinations thereof). In some examples, the codewords can be arithmetically decoded, for example using CABAC. The prefix portion of the codeword can be context-decoded, and the decoded suffix portion can be bypassed. In some examples, the sign bit for each direction component of the motion vector can be decoded individually.

[0051] In some examples, the motion bitstream 272 may also include an indication of the reference base grid 243 for the selected reconstruction quantization.

[0052] In some examples, the motion bitstream 272 can be decoded by the motion decoder 244 and used by the base grid reconstructor 246 to generate the base grid 256 for reconstruction quantization. For example, the base grid reconstructor 246 can apply the decoded motion field to the reference base grid 243 for reconstruction quantization to determine (e.g., generate) the base grid 256 for reconstruction quantization.

[0053] In some examples, the reconstructed quantized reference base mesh m'(j) associated with a reference mesh frame having index j can be used to predict the base mesh m(i) associated with the current frame having index i. The base meshes m(i) and m(j) can include the same items: number of vertices, connectivity, texture coordinates, and texture connectivity. Vertex localization can differ between the base mesh m(i) and the base mesh m(j).

[0054] In some examples, the motion field f(i) can be computed by considering a quantized version of m(i) and a reconstructed quantized base mesh m'(j). The base mesh m'(j) may have a different number of vertices than m(j) (e.g., vertices may have been merged or removed). Therefore, the encoder can track the transformations applied to m(j) to determine (e.g., generate or obtain) m'(j) and apply it to m(i). This transformation can achieve the vertices of the base mesh m'(j) and the base mesh m(i) (represented as...). The transformation and quantization of the 1-to-1 correspondence between the versions can be achieved by subtracting from the position Pos(j,v) of vertex v of m'(j) as follows. The motion field f(i) is computed by quantizing and locating the vertex v to p(i,v): f(i,v) = Pos(i,v) - Pos(j,v). The motion field can be further predicted by using the connectivity information of the base grid m'(j), and the prediction residuals can be entropy encoded.

[0055] In some examples, since the motion field compression process may be lossy, the reconstructed motion field, denoted as f'(i), can be computed by applying a motion decoder component. The reconstructed quantized base grid m'(i) can then be computed by adding the motion field to the vertex localization within the base grid m'(j). To better utilize the temporal correlations in displacement and property map images (e.g., image sequences / videos), inter-frame prediction can be enabled in the video encoder.

[0056] In some embodiments, the encoder (e.g., encoder 114) may include encoder 200A and encoder 200B.

[0057] Figure 3 This illustrates a diagram of the example decoder 300. (It can be compared with...) Figure 2A and Figure 2BThe bit stream 330, which corresponds to bit stream 266 and can be received as a binary file, can be demultiplexed by demultiplexer 302 to separate bit stream 330 into base mesh bit stream 332, displacement bit stream 334, and attribute bit stream 336, which respectively carry base mesh geometry information, displacement geometry information, and attribute information. Attribute bit stream 336 may include one or more attribute graph substreams for each attribute type.

[0058] In some examples, for inter-frame decoding, the bitstream is demultiplexed into separate substreams, including: a motion substream, a displacement substream for localization and possibly for each vertex attribute, zero or more attribute graph substreams, and an atlas substream that contains patch information in the same manner as in V3C / V-PCC.

[0059] In some examples, the underlying grid bitstream 332 can be decoded in intra-frame or inter-frame mode. In intra-frame mode, the static grid decoder 320 can decode the underlying grid bitstream 332 (e.g., to generate a reconstructed underlying grid m'(i)), which is then dequantized by the inverse quantizer 318 to determine (e.g., generate or output) the decoded underlying grid 340 (e.g., reconstructed quantized underlying grid m"(i)). In some examples, the static grid decoder 320 can be coupled with... Figure 2A The corresponding mesh decoder 206.

[0060] In some examples, in inter-frame mode, the base grid bitstream 332 may include motion field information decoded by the motion decoder 324. In some examples, the motion decoder 324 may be coupled with... Figure 2B The motion decoder 324 corresponds to the motion decoder 244. For example, the motion decoder 324 can perform entropy decoding on the base grid bitstream 332 to determine motion field information. In inter-frame mode, the base grid bitstream 332 can indicate the previous base grid (e.g., reference base grid m'(j)) decoded by the static grid decoder 320 and stored (e.g., cached in) the grid buffer 322. The base grid reconstructor 326 can generate a quantized reconstructed base grid m'(i) by applying the decoded motion field (output by the motion decoder 324) to the previously decoded (e.g., reconstructed) base grid m'(j) stored in the grid buffer 322. In some examples, the base grid reconstructor 326 can be associated with... Figure 2B The base mesh reconstructor 246 corresponds to this. The quantized reconstructed base mesh can be dequantized by the inverse quantizer 318 to determine (e.g., generate or output) the decoded base mesh 340 (e.g., the reconstructed base mesh m"(i)). In some examples, the decoded base mesh 340 can be... Figure 2A and Figure 2B The base grid 268 of the reconstruction is the same.

[0061] In some examples, decoder 300 includes video decoder 308, image unpacker 310, inverse quantizer, and inverse wavelet transform 314, which determines (e.g., generates) the decoded shift 338 based on shift bitstream 334. Video decoder 308, image unpacker 310, inverse quantizer, and inverse wavelet transform 314 correspond to video decoder 226, image unpacker 224, inverse quantizer 222, and inverse wavelet transform 220, respectively, and perform the same or similar operations. For example, a picture frame (e.g., an image) received in shift bitstream 334 can be decoded by video decoder 308, the shift information can be unpacked from the decoded image by image unpacker 310, and inverse quantizer 312 can inverse quantize it into determined inverse-quantized wavelet coefficients representing the encoded shift information. Subsequently, the unquantized wavelet coefficients can be inversely transformed by the inverse wavelet transform 314 to determine the decoded displacement d"(i). In other words, the decoded displacement 338 (e.g., the decoded displacement field d"(i)) can be compared with... Figure 2A and Figure 2B The reconstructed displacement is the same as 270.

[0062] The deformable mesh reconstructor 316, corresponding to the deformable mesh reconstructor 230, can determine (e.g., generate or output) the decoded mesh 342 (M"(i)) based on the decoded displacements 338 and the decoded base mesh 340. For example, the deformable mesh reconstructor 316 can combine (e.g., add) the decoded displacements 338 to subdivide the decoded mesh 340 to determine the decoded mesh 342.

[0063] In some examples, decoder 300 includes video decoder 304 that decodes attribute bitstream 336, which includes encoded attribute information represented (e.g., stored) as 2D images (or picture frames), into determined attribute information 344 (e.g., decoded or reconstructed attribute information). In some examples, video decoder 304 may be an existing 2D video compression decoder, such as an HEVC decoder or a VVC decoder. Decoder 300 may include reconstructible attribute information... Figure 2A and Figure 2B The color space converter 236 in the color space converter 306 performs color format conversion.

[0064] Figure 4 Figure 400 illustrates an example process (e.g., a preprocessing operation) for generating a displacement 414 of an input mesh 430 (e.g., an input 3D mesh frame) to be encoded, according to some embodiments. In some examples, the displacement 414 may be... Figure 2A and Figure 2B The displacement 258 described in the text corresponds to this.

[0065] In Figure 400, the mesh extractor 402 determines (e.g., generates or outputs) an initial base mesh 432 based on (e.g., using) an input mesh 430. In some examples, the initial base mesh 432 can be determined (e.g., generated) based on the input mesh 432 through an extraction process. During the extraction process, the mesh topology of the mesh frame can be reduced to determine the initial base mesh (which may be referred to as the extracted mesh or the extracted base mesh). Figure 5 The lieutenant general explained that the extraction process may involve a downsampling process to remove vertices from the input grid 432, such that a small fraction (e.g., 6% or less) of the vertices in the input grid 430 can remain in the initial base grid 432.

[0066] The mesh subdivision unit 404 applies a subdivision scheme to generate the initial subdivided mesh 434. (The remaining text appears to be unrelated and possibly machine-generated gibberish.) Figure 5 In more detail, the subdivision scheme may involve upsampling the initial base mesh 432 to add more vertices to the 3D mesh based on the topology and shape of the original mesh to generate the initial subdivision mesh 434.

[0067] The fitting component 406 can fit the initially subdivided mesh to determine a deformed mesh 436 that more closely approximates the surface of the input mesh 430. (As for...) Figure 5 In more detail, fitting can be performed by moving the vertices of the initial subdivided mesh 434 toward the surface of the input mesh 430 so that the subdivided mesh 434 can be used to approximate the input mesh 430. In some specific implementations, fitting is performed by moving each vertex of the initial subdivided mesh 434 along the normal direction of the vertices until the vertices intersect the surface of the input mesh 430. The resulting mesh is a deformed mesh 436. The normal direction can be indicated by the vertex normal at the vertex, which can be obtained from the face normal of the triangle formed by the vertices.

[0068] The base mesh generator 408 can perform another fitting process to generate a base mesh 438 based on the initial base mesh 432. For example, the base mesh generator 408 can deform the initial base mesh 432 based on the deformed mesh 436, making the initial base mesh 432 approximate the deformed mesh 436. In some specific implementations, the fitting process can be performed in a manner similar to that of the fitting component 406. For example, the base mesh generator 408 can move each vertex along the normal direction of each vertex in the initial base mesh 432 (e.g., based on the vertex normal at each vertex) until the vertex reaches the surface of the deformed mesh 436. The output of this process is the base mesh 438.

[0069] The base mesh 438 can be output to the mesh reconstruction process 410 to generate the reconstructed base mesh 440. The reconstructed base mesh 440 can be subdivided by the mesh subdivision unit 418, and the subdivided mesh 442 can be input to the displacement generator 420 to generate (e.g., determine or output) the displacement 414, as described below. Figure 5 Further description. In some examples, mesh subdivision 418 may apply the same subdivision scheme as mesh subdivision 404. In these examples, the vertices in the subdivided mesh 442 have a one-to-one correspondence with the vertices in the deformed mesh 436. Thus, displacement generator 420 can generate displacement 414 by calculating the difference between each vertex of the subdivided mesh 442 and the corresponding vertex of the deformed mesh 436. In some specific implementations, this difference can be projected onto the normal direction of the associated vertex, and the resulting vector is displacement 414. In this way, only the sign and magnitude of displacement 414 need to be encoded in the bit stream, thereby improving decoding efficiency. In addition, because the base mesh 438 has been fitted toward the deformed mesh 436, the displacement 414 between the deformed mesh 436 and the subdivided mesh 442 (generated from the reconstructed base mesh 440) will have a small magnitude, which further reduces the payload and improves decoding efficiency.

[0070] In some examples, one advantage of applying the subdivision process is that it allows for more efficient compression while providing a reliable approximation of the original input mesh 430 (e.g., the surface or curve of the original input mesh 430). Compression efficiency is achieved because the base mesh (e.g., the extracted mesh) has fewer vertices compared to the number of vertices in the input mesh 430, thus requiring fewer bits to be encoded and transmitted. Furthermore, once the base mesh has been decoded, the subdivided mesh can be automatically generated by the decoder, requiring no information from the encoder other than the subdivision scheme (e.g., the subdivision algorithm) and the parameters used for subdivision (e.g., the subdivision iteration count). The reconstructed mesh can be determined by decoding the displacement information (e.g., displacement vectors) associated with the vertices of the subdivided mesh (e.g., the subdivided curves / surfaces of the base mesh). Not only does the subdivision process allow for spatial / quality scalability, but the efficient decoding of displacements using wavelet transforms (e.g., wavelet decomposition) further enhances compression performance.

[0071] In some embodiments, the mesh reconstruction process 410 includes components for encoding and subsequently decoding the base mesh 438. Figure 4 An example for intra-frame mode is illustrated, wherein the mesh reconstruction process 410 may include operations that can be performed separately with... Figure 2AThe quantizer 202, mesh encoder 204, mesh decoder 206, and inverse quantizer 228 operate in the same or similar manner as the quantizer 411, static mesh encoder 412, static mesh decoder 413, and inverse quantizer 416. For inter-frame mode, the mesh reconstruction process 410 may include the quantizer 202, motion encoder 242, motion decoder 244, basic mesh reconstructor 246, and inverse quantizer 228.

[0072] Figure 5 An example process for approximating and encoding the geometry of a 3D mesh, according to some embodiments, is illustrated. For illustrative purposes, the 3D mesh is illustrated as a 2D curve. The original surface 510 of the 3D mesh (e.g., a mesh frame) includes vertices (e.g., points) and edges connecting adjacent vertices. For example, points 512 and 513 are connected by an edge corresponding to surface 514.

[0073] In some examples, an decimation process (e.g., a downsampling process or decimation / downsampling scheme) can be applied to the original surface 510 of the original mesh to generate a downsampled surface 520 of the decimated (or downsampled) mesh. In the context of mesh compression, decimation refers to the process of reducing the number of vertices in a mesh while preserving the overall shape and topology of that mesh. For example, the original mesh surface 510 is decimated into a surface 520 with fewer samples (e.g., vertices and edges) but still retains the main features and shape of the original mesh surface 510. This downsampled surface 520 can correspond to the surface of the base mesh (e.g., the decimated mesh).

[0074] In some examples, after the extraction process, a subdivision process (e.g., a subdivision scheme or subdivision algorithm) can be applied to the downsampled surface 520 to generate an upsampled surface 530 with more samples (e.g., vertices and edges). The upsampled surface 530 can be part of a subdivided mesh (e.g., a subdivided base mesh) generated by subdividing the downsampled surface 520 corresponding to the base mesh.

[0075] Subdivision is a process commonly used after decimation in mesh compression to improve the visual quality of the compressed mesh. The subdivision process involves adding new vertices and faces to the mesh based on the topology and shape of the original mesh. In some examples, the subdivision process begins by taking the reduced mesh generated by the decimation process and iteratively adding new vertices and edges. For example, the subdivision process may include dividing each edge (or face) of the reduced / decimated mesh into shorter edges (or smaller faces) and creating new vertices at the division points. These new vertices are then connected to form new faces (e.g., triangles, quadrilaterals, or other polygons). By applying subdivision after the decimation process, a higher level of compression can be achieved without a significant loss of visual fidelity. Various subdivision schemes can be used, such as, for example, midpoint, Catmull-Clark subdivision, butterfly subdivision, ring subdivision, etc., or combinations thereof.

[0076] For example, Figure 5 An example of a midpoint subdivision scheme is illustrated. In this scheme, each subdivision iteration divides each triangle into four sub-triangles. A new vertex is introduced in the middle of each edge. The subdivision process can be applied independently to geometry and texture coordinates because the connectivity used for geometry and texture coordinates is typically different. The subdivision scheme computes the newly introduced vertices. In the first vertex ( ) and the second vertex ( The edge formed The location at the center ,as follows: , in and It is the vertex and The location. In some examples, the same process can be used to calculate the texture coordinates of the newly created vertex. For the normal vector, the normalization step can be applied as follows: , in , and Respectively with the vertex , and The associated normal vector. For vectors norm2.

[0077] Using a midpoint subdivision scheme, as illustrated in the upsampled surface 530, point 531 can be generated as the midpoint of edge 522, which connects points 532 and 533. Point 531 can be added as a new vertex. Edges 534 and 542 are also added to connect the newly added vertex corresponding to point 531. In some examples, the original edge 522 can be replaced by two new edges 534 and 542.

[0078] In some examples, the downsampled surface 520 can be iteratively subdivided to generate the upsampled surface 530. For example, the first subdivided mesh generated by the first iteration of subdivision applied to the downsampled surface 520 can be further subdivided according to the subdivision scheme to generate a second subdivided mesh, and so on. In some examples, the number of iterations corresponding to the subdivision level can be predetermined. In other examples, the encoder can indicate the number of iterations to the decoder, which can similarly generate subdivided meshes, as further described above.

[0079] In some embodiments, the subdivided mesh may deform toward (e.g., approximate) the original mesh to determine (e.g., acquire or obtain) a prediction of the original mesh having the original surface 510. Points on the subdivided mesh may move along computed normal vertices / orientations until they reach the original surface 510 of the original mesh. The distance between the intersection point on the original surface 510 and the subdivided point may be computed as a displacement (e.g., a displacement vector). For example, point 531 may move toward the original surface 510 along (e.g., represented by edge 542) the computed normal orientation of the surface. When point 531 intersects with surface 514 of the original surface 510 (of the original / input mesh), a displacement vector 548 may be computed. The displacement vector 548 applied to point 531 may produce a displacement surface 540 that better approximates the original surface 510. In some examples, displacement information (e.g., displacement vector 548) for vertices of the subdivided mesh (e.g., the upsampled surface 530 of the subdivided mesh) may be... Figure 2A and Figure 2B The shift bit stream 260, as illustrated in the example encoder, is encoded and transmitted. It should be noted that, as regarding... Figure 4 As explained, the subdivided mesh corresponding to the upsampled surface can be a subdivided mesh 442 that is compared with the deformed mesh 436 of the original surface 510 representing the input mesh.

[0080] In some embodiments, displacement d(i) (e.g., displacement field or displacement vector) can be calculated and / or stored based on local or global coordinates. For example, a global coordinate system is a reference system used to define the positioning and orientation of objects or points in 3D space. The global coordinate system provides a fixed reference frame independent of the object or point being described. The origin of the global coordinate system can be defined as the point where the three axes intersect. Any point in 3D space can be positioned relative to the origin by specifying its location along the three axes using Cartesian coordinates (x, y, z). For example, displacement can be defined in the same Cartesian coordinate system as the input or original mesh.

[0081] In a local coordinate system, mutually perpendicular normal vectors, tangent vectors, and / or binormal vectors can be defined for a local basis in 3D space to represent the orientation and positioning of an object relative to a reference frame in space. In some examples, the displacement field d(i) can be transformed from the normal coordinate system to a local coordinate system, such as one defined by the normals of the subdivided mesh at each vertex (often referred to as vertex normals). The normals at each vertex can be obtained by combining the face normals of the triangles formed by the vertices. In some examples, using a local coordinate system allows for further compression of the tangential component of the displacement compared to the normal component.

[0082] In some embodiments, the decoder (e.g.) Figure 3 The decoder 300 can receive and decode a base mesh corresponding to (e.g., having) the downsampled surface 520. Similar to the encoder, the decoder can apply a subdivision scheme to determine a subdivided mesh having an upsampled surface 530 generated from the downsampled surface 520. The decoder can receive and decode displacement information including a displacement vector 548, and determine a decoded mesh (e.g., a reconstructed mesh) based on the subdivided mesh (corresponding to the upsampled surface 530) and the decoded displacement information. For example, the decoder can add the displacement at each vertex to the corresponding vertex's location within the subdivided mesh. The decoder can obtain a reconstructed 3D mesh by combining the obtained / decoded displacements with the vertices of the subdivided mesh.

[0083] Figure 6 Examples of vertices corresponding to subdivisions of a mesh (e.g., a subdivided base mesh) at multiple levels of detail (LOD) are illustrated according to some embodiments. (See above regarding...) Figure 5As described, the subdivision process (e.g., a subdivision scheme) can be an iterative process in which the mesh can be subdivided multiple times, generating a hierarchical data structure containing multiple levels. Each level of the hierarchical data structure can include a different number of data samples (e.g., vertices and edges in the mesh) representing (e.g., forming) different densities / resolutions (e.g., also known as Level of Detail (LoD)). For example, after a first iteration of subdivision, a downsampled surface 520 (of the extracted mesh) can be subdivided into an upsampled surface 530. The upsampled surface 530 can be further subdivided into an upsampled surface 630, and so on. In this case, the vertices of the mesh with the downsampled surface 520 can be considered to be at or associated with LOD0. Vertices generated in the upsampled surface 530 after the first iteration of subdivision (e.g., vertex 632) can be at LOD1. Vertices generated in the upsampled surface 630 after another iteration of subdivision (e.g., vertex 634) can be at LOD2, and so on. In some examples, LOD0 can refer to a vertex generated by decimating an input (e.g., the original) mesh to produce a base mesh with (e.g., having) a downsampled surface 520. For example, a vertex at LOD0 could be... Figures 2A to 2B The basic grid for reconstruction quantization 256 Figure 3 The base mesh for reconstruction / decoding is 340. Figure 4 The vertices of the reconstructed base mesh are 440.

[0084] In some examples, the calculation of displacement in different LODs follows the same principles as described above. Figure 5 The mechanism described is the same. In some examples, the displacement vector 643 can be calculated from the location of vertex 641 in the original surface 510 (of the original mesh) to vertex 642 at LOD0 from the displacement surface 640 of the deformed mesh. Similarly, the displacement vectors 644 and 645 from the corresponding vertices 632 and 634 from LOD1 and LOD2, respectively, can be calculated. Therefore, in some examples, the number of subdivision iterations can correspond to the number of LODs, and one iteration in the iterations can correspond to one LOD in the LODs.

[0085] Figure 7A An example is illustrated where an image 720 (e.g., a picture or picture frame) with displacement 700 (e.g., a displacement field or vector) is filled using a packing method (e.g., a packing scheme or packing algorithm) according to some embodiments. Specifically, as described above regarding Figure 5 and Figure 6As described, displacements of 700 can be generated and these displacements can be packed into a 2D image. In some examples, the displacement can be a 3D vector containing values ​​for three components for distance. For example, the increment x value represents the offset along the x-axis from point A to point B in a Cartesian coordinate system. In some examples, the displacement vector can be represented by fewer than three components, such as one or two components. For example, when using a local coordinate system to store displacement values, the component with the highest importance can be stored as representing the displacement, and other components can be discarded.

[0086] In some examples, as will be further described below, the displacement values ​​can be transformed into other signal domains for better compression. For example, the displacement can be wavelet transformed and decomposed into and represented as wavelet coefficients (e.g., coefficient values ​​or transform coefficients). In these examples, the displacement 700 packed in image 720 can include the resulting wavelet coefficients (e.g., transform coefficients) that can be compressed more efficiently than the untransformed displacement values. At the decoder side, the decoder can decode the displacement 700 into wavelet coefficients and can apply an inverse wavelet transform process to reconstruct the original displacement values ​​obtained at the encoder.

[0087] In some examples, one or more displacements in displacement 700 can be quantized by the encoder before being packed into displacement image 720. In some examples, one or more displacements can be quantized before, after, or both before and after the wavelet transform. For example, Figure 7A This describes the wavelet transform values ​​8, 4, 1, and -1 for quantization in a displacement of 700°. At the decoder side, the decoder can perform inverse quantization to reverse or undo the quantization process performed by the encoder.

[0088] In general, quantization in signal processing can be seen as the process of mapping input values ​​from a larger set to output values ​​in a smaller set. This process is often used in data compression to reduce the amount, precision, or resolution of data into a more compact representation. However, this reduction can lead to information loss and introduce compression artifacts. The choice of quantization parameters (such as the number of quantization levels) is a trade-off between the desired level of precision and the resulting data size. Many different quantization techniques exist, such as selectable / enabled / applied uniform quantization, non-uniform quantization, and adaptive quantization. These techniques can be adopted depending on the specific requirements of the application.

[0089] In some examples, wavelet coefficients (e.g., displacement coefficients representing displacement signals) can be adaptively quantized based on the LOD. As explained above, the mesh can be iteratively subdivided to generate a hierarchical data structure comprising multiple LODs. In this example, each vertex and its associated displacement belong to the same level of the LOD structure, such as the LOD corresponding to the subdivision iteration in which that vertex was generated. In some examples, vertices at each LOD can be quantized based on quantization parameters corresponding to the LOD, which specify different intensity / precision levels of the signal to be quantized. For example, wavelet coefficients at LOD 3 could have a quantization parameter of, for example, 42, and wavelet coefficients at LOD 0 could have different, smaller quantization parameters of 28 to preserve more detail information in LOD 0.

[0090] In some examples, displacement 700 can be packed onto pixels in a displacement image 720 having width W and height H. In these examples, the size of displacement image 720 (e.g., W multiplied by H) can be greater than or equal to the number of components in displacement 700 to ensure that all displacement information can be packed. In some examples, displacement image 720 can be further divided into smaller regions (e.g., squares) called packing blocks 730. In these examples, the length of packing block 730 can be an integer multiple of 2.

[0091] Displacements 700 (e.g., displacement signals represented by quantized wavelet coefficients) can be packed into packing blocks 730 according to packing order 732. Each packing block 730 can be packed (e.g., arranged or stored) in the displacement image 720 according to packing order 722. Once all displacements 700 are packed, empty pixels in the image 720 can be filled with adjacent pixel values ​​to improve compression. Figure 7A In the example illustrated, the packing order 722 for packing blocks can be a raster order, and the packing order 732 for displacements within packing block 730 can be, for example, a Z-order. However, it should be understood that other packing schemes can be used for both blocks and displacements within blocks. In some embodiments, the packing scheme for blocks and / or displacements within blocks can be predetermined. In some embodiments, the packing scheme can be signaled by the encoder in the bitstream of each patch, patch group, tile, image, or image sequence. Accordingly, the signaled packing scheme can be obtained from the bitstream by the decoder.

[0092] In some examples, the packing order 732 can follow a space-filling curve that traverses space in a continuous, non-repeating manner. Some examples of space-filling curve algorithms (e.g., schemes) include Z-order curves, Hilbert curves, Peano curves, Moore curves, Sierpinski curves, Dragon curves, and others. Space-filling curves have been used in image packing techniques to efficiently store and retrieve images in a way that maximizes storage space and minimizes retrieval time. Space-filling curves are well-suited for this task because they provide a one-dimensional representation of a two-dimensional image. A common image packing technique using space-filling curves is called Z-order or Morton order. A Z-order curve is constructed by interleaving the binary representations of the x and y coordinates of each pixel in the image. This creates a one-dimensional representation of the image that can be stored in a linear array. To pack images using a Z-order curve, the image is first divided into small blocks, typically 8×8 or 16×16 pixels in size. Each block is then encoded using a Z-order curve, and each block is stored in a linear array. When it is necessary to retrieve an image, inverse Z-order curves are used to decode the blocks and reassemble them into the original image.

[0093] In some examples, once packaged, the displacement image 720 can be encoded and decoded using a regular 2D video codec.

[0094] Figure 7B An example of a displacement image 720 according to some embodiments is illustrated. As illustrated, these displacements can be ordered according to the Level of Displacement (LOD) of the displacements 700 filled in the displacement image 720. For example, displacement coefficients (e.g., quantized wavelet coefficients) can be ordered from lowest LOD to highest LOD. In other words, wavelet coefficients representing displacements for vertices at a first LOD can be packed (e.g., arranged and stored in the displacement image 720) according to the first LOD. For example, displacements 700 can be packed from lowest LOD to highest LOD. Higher LODs represent higher density vertices and correspond to more displacements compared to lower LODs. The portions of the displacement image 720 that are not at any LOD can be filled portions.

[0095] In some examples, displacements can be packed in reverse order from highest LOD to lowest LOD. In one example, the encoder can signal whether the displacements are packed from lowest LOD to highest LOD or vice versa.

[0096] In some examples, wavelet transform can be applied to displacement values ​​to generate wavelet coefficients (e.g., displacement coefficients) that can be more easily compressed. Wavelet transform is commonly used in signal processing to decompose a signal into a set of wavelets, which are smaller wave-like functions that allow them to capture local features in the signal. The result of wavelet transform is a set of coefficients representing the contribution of each wavelet at different scales and locations in the signal. This result can be used to detect and locate transient features in a signal and is generally used for signal analysis and data compression, such as image, video, and audio compression.

[0097] Using a 2D image as an example, wavelet transform is used to decompose an image (signal) into two discrete components called approximation / prediction and detail. The decomposed signal is further divided into high-frequency components (details) and low-frequency components (approximation / prediction) by passing it through two filters (a high-pass filter and a low-pass filter). In the 2D image example, two filtering stages (horizontal filtering and vertical filtering) are applied to the image signal. After each filtering stage of the decomposed components, a downsampling step is required to obtain wavelet coefficients, resulting in four sub-signals at each decomposition level. High-frequency components correspond to rapid changes or abrupt transitions in the signal (such as edges or lines in an image). On the other hand, low-frequency components refer to the global characteristics of the signal. Depending on the application, different filtering and compression methods can be implemented. Various types of wavelets exist, such as Haar, Daubechies, Symlets, etc., each with different characteristics such as frequency resolution, temporal positioning, etc.

[0098] In signal processing, lifting schemes are techniques used for both designing wavelets and performing Discrete Wavelet Transform (DWT). Lifting schemes offer several advantages in terms of computational efficiency and flexibility, serving as an alternative to the traditional filter bank implementation of DWT. A lifting scheme uses a series of lifting steps to decompose the signal, allowing the input signal (e.g., displacement for a 3D mesh) to be converted in situ into displacement coefficients. Within a lifting scheme, a series of lifting operations (e.g., lifting steps) can be performed. Each lifting operation involves a prediction step (e.g., prediction operation) and an update step (e.g., update operation). These lifting operations can be applied iteratively to obtain wavelet coefficients.

[0099] In various implementations of 3D mesh decoding, wavelet transforms with lifting (e.g., known as lifting schemes) can be used to transform the displacement for a 3D mesh frame. Specifically, the wavelet transform can "split" the input signal (e.g., the displacement signal) into two signals: an even-sample signal. and odd samples Signal. An even-numbered sample E may include two displacement signals associated with two vertices. and The two vertices are considered to be on the edges of the vertices associated with the input displacement signal. Odd-numbered samples This can represent the input signal corresponding to the vertex. As explained above, edge information can be determined (e.g., generated or received) based on the subdivision scheme applied to each mesh frame of the 3D mesh. It can be based on an even number of sample signals. The weighted sum is used to determine the odd number of samples. Signal prediction. Next, odd-numbered samples can be... Signal encoded as odd number of samples The difference between the signals. Furthermore, even-numbered sample signals can be adjusted based on the difference weighted by the updated weights. and For each of the odd-numbered samples, the updated weights can be compared with the odd-numbered samples. Signal correlation (e.g., with odd-numbered samples) (LOD correlation of the signal).

[0100] In some embodiments, wavelet coefficients (e.g., transformed wavelet coefficients) representing the displacement of vertices corresponding to a 3D mesh geometry can be quantized according to quantization parameters set for each of a plurality of LODs. Vertices may span multiple LODs (e.g., generated across multiple LODs). Furthermore, a dead-zone quantizer can be used to quantize the wavelet coefficients within each LOD. The dead-zone quantizer can be a type of quantizer that exhibits symmetric behavior around the quantized value of 0 and reaches its peak at zero. The region around the 0 output value of such a quantizer is called the dead zone. The dead zone can be configured to have a width different from the width of the other quantization steps of the quantizer. The dead-zone quantizer can be a uniform quantizer, such that the other quantization steps have a uniform width (e.g., quantization size). In other words, when the dead-zone quantizer is set to the same size as the quantization steps, each quantization step of the dead-zone quantizer is identical or uniform.

[0101] In some examples, wavelet coefficients at vertices of different Levels of Degrees (LODs) can be quantized (and inverse quantized) according to their corresponding LODs, which specify different intensity (e.g., precision) levels or signals to be scaled. For example, wavelet coefficients at vertices in LOD 3 may have a quantization scaling factor of, for example, 1 / 8, and wavelet coefficients at vertices in LOD 0 may have different, larger quantization scaling factor values, for example, 1, to preserve more detailed information in LOD 0 (e.g., such that wavelet coefficients at higher LODs are quantized with larger quantization steps). Relatedly, the inverse scaling factor can be, for example, 8 associated with LOD 3, and 1 associated with, for example, LOD 0.

[0102] Figure 8Example diagram 800 illustrates a quantizer, according to some embodiments, for quantizing transformed wavelet coefficients (as shown on the x-axis) representing displacements of a 3D mesh. The quantizer can be applied by an encoder (e.g., Figure 2A and Figure 2B The quantizer 212). According to some embodiments, the quantizer quantizes the transformed wavelet coefficients of the 3D mesh into quantized values ​​(e.g., integer values).

[0103] As shown in Figure 800, the quantizer can be a dead-zone quantizer, which has: a dead zone surrounding the quantized value 0, the width of which is related to the dead-zone parameter ( DZ Related to; and quantization step size Its size is equal to the scaling factor. s (For example, the quantization scaling factor). The width of this dead zone can be determined by the dead zone parameter ( DZ ) Determine or indicate, for example, the width is In some examples, this can be determined based on the scaling factor. s and dead zone offset / bias value The width of this dead zone is determined by... (or alternatively) ) indicates that the quantization step size It is an integer value, and other quantization step sizes have the following characteristics: The width represents uniformity.

[0104] In some examples, scaling value s and / or dead zone parameters DZ It can be associated with each LOD in the LODs associated with the vertices of the 3D mesh.

[0105] Quantizers can convert input signals And convert it (e.g., quantize) into an output signal. For 3D mesh encoding, the input signal It can be the displacement signal of the vertex (e.g., the transformed coefficients representing the displacement of the vertex), and the output signal This can be a quantized shift signal (e.g., quantized transform coefficients). For example, a quantizer can quantize the input signal... The sign is maintained as the output signal. The symbol. In some examples, the input signal It can be adjusted (e.g., if the dead zone is...) If indicated, then add; or alternatively, if the dead zone is... (If indicated, then subtract), and according to the scaling factor s Scaling (e.g., dividing by) s The quantizer can apply a floor function (or, alternatively, an up function) to the scaled, adjusted input signal. The value of the output signal. The smallest integer less than or equal to the stated value is determined. Alternatively, if an up-rounding function is applied, the quantizer can quantize the output signal. Determine the smallest integer greater than or equal to the stated value. For example, if =0 and If the value is 2, then a set of input signals 0, 2.5, 3.7, 6.8 can be quantized into corresponding output signals 0, 1, 1, 3.

[0106] Therefore, the quantizer can adjust the input signal according to the input signal. The range of values ​​that the input signal falls into will be within the range of values ​​that the input signal falls into. Quantization of signals , , , Wait. For and The choice of value determines the width of the dead zone, and how many signals with values ​​within this range / width are quantized into values ​​associated with the dead zone. (For example, ).right The choice of this value determines the width of the quantization step and how many signals with values ​​within this range (outside the dead zone) are quantized to non-zero values. For example, in... arrive Input values ​​within the range are quantized as (For example, 1). For instance, in the context of... arrive Input values ​​within the range are quantized as (For example, 2) etc.

[0107] Figure 9 Example diagram 900 illustrates an inverse quantizer, according to some embodiments, for inverse quantization of quantized-transformed wavelet coefficients (as shown on the y-axis) representing displacements of a 3D mesh. The inverse quantizer can be applied by a decoder (e.g., Figure 2A and Figure 2B The inverse quantizer 222, or Figure 3 The inverse quantizer 312). According to some embodiments, the quantizer quantizes the transformed wavelet coefficients of the 3D mesh into quantized values ​​(e.g., integer values). An example of the inverse quantizer may be a dead-zone inverse quantizer corresponding to a dead-zone quantizer, such as... Figure 8 As described in [the document / reference], a dead-zone inverse quantizer can be associated with LODs and / or quantization parameters in a plurality of LODs, which specify different precision (e.g., intensity) levels of the signal to be inverse quantized. The inverse quantizer can acquire the input signal... (corresponding to) Figure 8 Quantized displacement signal ), and use an inverse scaling factor (e.g., based on Figure 8 scaling factor in s Apply inverse scaling and use the inverse scaling value as the output signal. Output. For example, the input signal can be multiplied by a scaling factor. s The scaling factor corresponds to the reciprocal of the inverse scaling factor. In some examples, the input signal... The quantized integer value can be converted into a floating-point / double value after inverse scaling. Because, as... Figure 8 The error introduced by quantization at the encoder, as described in the paper, affects the output signal. Approximate to the original input signal quantized by the encoder .

[0108] In some examples, an inverse quantizer can convert the input signal... Inverse quantization to inverse quantization signal , , , One of them, which can be regarded as the reconstruction point representing the inverse quantization level.

[0109] In existing technologies, as mentioned above... Figure 9 As described, the inverse quantizer only applies an inverse scaling factor, regardless of whether the quantizer used to quantize the vertex displacement signal of the 3D mesh is a dead-zone quantizer or a uniform quantizer. This is achieved by applying a scaling factor to the quantized signal. Perform inverse quantization, and then (e.g., reconstruct) the signal (e.g., , , , This corresponds to a vertex (of the original 3D mesh), the position of which falls within a range associated with a scaling factor (e.g., the width of the quantization step). However, this inverse-quantized value could be an unquantized signal falling within a range of values ​​associated with the scaling factor. A poor approximation (or representation). For example, the input signal It can contain five values ​​between 1 and 2: 1.2, 1.3, 1.5, 0.9, and 1.6. If a scaling factor of 1.0 is applied, all five values ​​can be quantized to 1.0 and reconstructed as values ​​of 1.0. In this example, statistics for displacement signals within a quantization range (e.g., 1 to 2) can indicate five signals. The average value is 1.3. Therefore, among the five input signals... The mean value is 0.3, which introduces a mean error (and corresponding quality degradation) between the mean value and the mean value of the quantized (and dequantized) value of 1.0.

[0110] Embodiments of this disclosure relate to applying a quantization offset during the inverse quantization of quantized wavelet coefficients representing the displacement of vertices in a 3D mesh (e.g., a mesh frame). In some examples, the decoder may decode from the bitstream quantized wavelet coefficients representing the displacement of vertices from a set of vertices. Quantization information indicating a quantization offset associated with a subset of vertices in said set of vertices may be decoded from the bitstream. The wavelet coefficients (corresponding to the inverse-quantized wavelet coefficients) may then be determined as the displacement of the vertex based on the association of the quantization offset with the subset to which the vertex belongs. By applying a quantization offset (e.g., a reconstruction bias / offset / adjustment) to the subset of vertices, a more accurate displacement may be reconstructed or achieved. For example, in the above example where the average error of five signals is determined to be 0.3, the quantization offset may be indicated as 0.3 such that the average of the inverse-quantized signals is equal to the average of the original signals.

[0111] In some embodiments, a quantization offset can be applied to the inverse-quantized values. For example, the decoder can inverse-quantize the quantized wavelet coefficients to determine the wavelet coefficients. The decoder can then adjust the wavelet coefficients according to the quantization offset based on the location of the vertex within a subset to determine the vertex displacement. In some examples, the quantization offset can be determined for a subset within the current mesh frame (i.e., the current 3D mesh frame). For example, a corresponding quantization offset can be determined for each subset of vertices in the current mesh frame. A separate quantization offset can be determined for each subset of each mesh frame.

[0112] In some embodiments, a quantization offset can be applied to the quantized values. For example, the decoder can adjust the quantized wavelet coefficients according to the quantization offset based on the location of the vertex in a subset. Then, the decoder can determine the vertex displacement based on inverse quantization of the adjusted quantized wavelet coefficients.

[0113] In some examples, the set of vertices comprises a non-overlapping subset of vertices, and quantization information can be signaled (e.g., decoded) for each subset. For example, a subset may contain vertices of the same LOD within a LOD. In this example, the prediction / reconstruction error between the inverse-quantized signal and the original signal within each LOD can be reduced based on decoding the quantization offset for each LOD. In other examples, a subset may contain sub-mesh of a 3D mesh, a sub-mesh or a group of patches of a 3D mesh, or vertices in patches within a group of patches.

[0114] In some examples, indicators of whether quantization adjustment is enabled (e.g., mode indicators, flags, syntax elements) are decoded for a subset of vertices. The decoder can then further decode quantization information indicating quantization offsets based on the indicators of whether quantization adjustment is enabled.

[0115] These and other embodiments are described herein.

[0116] Figure 10This describes the quantization offset according to some embodiments ( ) and inverse quantization scaling factor ( s Figure 1000 illustrates an example of an inverse quantizer that inversely quantizes (e.g., inversely scales) the quantized-transformed wavelet coefficients representing the displacement of a 3D mesh. The 3D mesh can be processed as a sequence of mesh frames. The inverse quantizer of Figure 1000 reverses the operation of the inverse quantizer at the encoder, such as... Figure 8 As described in [the document / document], the inverse quantizer can be applied by the decoder (e.g., [other components]). Figure 2A and Figure 2B The inverse quantizer 222, or Figure 3 (Inverse quantizer 312).

[0117] In some examples, the quantization offset and one or more parameters for deriving the inverse quantization scaling factor can be determined (and decoded) by the decoder for each subset of vertices in a 3D mesh. In some examples, the quantization offset and one or more parameters can be determined for each subset of vertices in each mesh frame in a sequence of mesh frames. For example, a subset of vertices may correspond to vertices in the same LOD (or alternatively, the same submesh, patch group, patch, tile, etc.) of the current mesh frame. As explained above, selectively enabling quantization adjustment and signaling (e.g., decoding) the indication of the quantization offset associated with each subset of vertices can improve the statistics of the inverse-quantized wavelet to be closer to the statistics of the original unquantized wavelet coefficients, thus achieving less distortion (e.g., improved quality). These benefits can be achieved independently of whether the inverse quantizer is a uniform inverse quantizer or a dead-zone inverse quantizer, because the quantization error is more affected by the scaling factor (and inverse scaling factor) than the width of the dead zone in a dead-zone quantizer (and its corresponding dead-zone inverse quantizer).

[0118] As shown in the reconstruction and adjustment process of Figure 1000, which can be applied to a subset of vertices, the input quantized signal... (For example, Figure 8 Quantized displacement output signal It can be adjusted (e.g., shifted or biased) to be equal to the scaling factor. s A certain multiple plus quantization offset ( The value of ). In some examples, the input signal is quantized. You can use quantization offset Before making adjustments, first perform inverse scaling and quantization offset. It can be represented by the scaling factor s The scaling offset value. In other examples, the quantization offset. It can be based on the scaling factor ( s ) or the corresponding inverse scaling factor (e.g., 1 / s ) Quantized input signal The input quantized signal is applied before inverse quantization.

[0119] In Figure 1000, quantization offset It is shown as added to the quantized signal from the input. The derived inverse quantized signal The value of the quantization offset. In other examples, the quantization offset. It can be directly added to the inverse-quantized input signal. .

[0120] In some examples, the corresponding quantization offset Associated with each subset of vertices (such as vertices in the same LOD), and this same quantization offset value This can be applied to vertices in this subset. In some examples, the quantization offset... Setting it to zero disables quantization adjustments associated with wavelet coefficients corresponding to a subset of vertices (e.g., reconstructed point offset adjustments). For example, a value of 0 for an inversely quantized value (displayed as the output signal) would be... ), quantization offset It can be set to zero. This inversely quantized value of 0 corresponds to the value in the dead zone region, as calculated at the encoder, such as... Figure 8 As shown in the image.

[0121] As mentioned above, quantization offset This can be computed by the encoder and transmitted to the decoder to correct for the loss of detail caused by applying the quantizer to the vertex displacement. Typically, some subsets (when the subset is LOD, such as higher LOD) use larger quantizers (e.g., with larger scaling factors). s For larger quantizers, quantization offset Used to adjust for details lost due to quantization. However, a large quantizer may cause shifts over a large range of values ​​to be quantized to the same value, which can lead to unnecessary quantization offsets. Value. Therefore, in some embodiments, the quantization offset. The value can be adjusted by offset (a) δ Further adjustments will be made.

[0122] In some examples, the value can be adjusted by adding an offset (a) δ ) to quantization offset Further adjustments can be made. For example, a corresponding offset adjustment value (a) can be determined for each subset of vertices. δ For example, the offset adjustment value (a) can be determined based on subtracting the two offset values ​​between two subsets of the vertices. δFor example, between two offset values ​​determined for two consecutive LODs. For the current LOD, based on a specific subset, a consecutive LOD could be the next higher LOD, or in one example, the next lower LOD.

[0123] In some examples, quantization offset It can be quantized offset With offset adjustment value (a) δ The product of (a, b) is further adjusted. For example, the offset adjustment value (a) δ The scaling factor can be determined based on the quantization amount associated with a subset of vertices. For example, the scaling factor can be determined based on the LOD associated with a particular quantizer. In some examples, the scaling factor can be determined to be inversely proportional to the level of LOD. For example, the inverse relationship can be a negative power of two, where the power is based on the level of LOD. In this example, for LOD0, LOD1, and LOD2, the example corresponding scaling factors could be 2^(-0) = 1, 2^(-1) = 0.5, and 2^(-2) = 0.25. Offset adjustment value (a δ The function of ) is to provide a quantization offset for a subset of vertices utilizing a larger quantizer (such as a higher LOD). Applications have been significantly reduced.

[0124] In some embodiments, to prevent skewed distributions or outliers of displacement values ​​quantized to the same value within a value range from causing high quantization offsets. Quantization offset It can be limited so that it does not exceed a predetermined threshold. For example, the quantization offset can be... (with or without offset adjustment value (a) δ )) and threshold Comparison, making those above the threshold Quantization offset The value is set to the threshold. In some examples, quantization offset It can also be used with a threshold. Comparison, so that below the threshold Quantization offset The value is set to the threshold. For example, quantization offset. The value range can be set to In some examples, The value can be compared with the use The minimum value is the same.

[0125] In some examples, quantization offset In relation to threshold and / or Before comparison, the offset adjustment value (a) can be used first. δ Adjustments are made only when the threshold is exceeded. or Adjustments are only made when the set limits are met.

[0126] In some examples, quantization offset Can be used with threshold and / or Compare, but do not quantize the offset. Instead of being limited to these values, it can be based on (e.g., in response to) exceeding a threshold. and / or One of them is adjusted by the offset value (a) δ To adjust the quantization offset In other examples, the value used to determine whether to apply an offset adjustment (a) is... δ The threshold is different from that used to limit quantization offset. Threshold for possible values.

[0127] In some embodiments, applied to quantization offset The adjustments and / or thresholds can be performed in the same manner at both the encoder and decoder. For example, the encoder can signal one or more indications (e.g., flags) to enable adjustments, applied thresholds, or combinations thereof. In other embodiments, the encoder can apply adjustments and / or thresholds to the quantization offset. To determine the final quantization offset of a subset of vertices. The final quantization offset is transmitted to the decoder in the bitstream. In this type of embodiment, the final quantization offset is used to derive the final quantization offset. The processing is shifted to the encoder to reduce the computational complexity required at the decoder.

[0128] In some embodiments, for each subset of vertices, the quantization offset The encoder can transmit signals to the decoder.

[0129] In some embodiments, the quantization offset can be derived based on the quantization level associated with a subset of vertices. This eliminates the need for explicit signal transmission from the encoder to the decoder in the bitstream. For example, the encoder and decoder can independently and consistently compute the quantization offset of the vertices of the subset based on the LOD (e.g., the index of the LOD) associated with the subset. For example, the quantization offset can be determined based on the quantization scaling factor determined by the LOD-based index. .

[0130] Figure 11A is a diagram illustrating an example process of an inverse quantizer 1100A for iteratively inverse quantizing the quantized-transformed wavelet coefficients of vertices in each subset of vertices of a 3D mesh, according to some embodiments. The operation of the inverse quantizer 1100A can be performed by a decoder (e.g., Figure 2A and Figure 2B The inverse quantizer 222, or Figure 3 The inverse quantizer 312) is used in some embodiments. According to some embodiments, the inverse quantizer 1100A performs an inverse quantization scheme with a reconstruction adjustment process. The inverse quantizer 1100A can be a dead-zone inverse quantizer, as described above. Figure 10 The explanation is as follows.

[0131] In some examples, such as by Figures 2A to 2B The quantizer 212 performs a quantization process iteratively applied (e.g., performed on it) to the wavelet coefficient signals of each subset of vertices (e.g., from vertices at higher LOD to those at lower LOD). To reverse the operation of the quantization process, the inverse quantizer 1100A iteratively performs inverse quantization on each subset of vertices, as shown in the subset-based iterator 1102A (e.g., from vertices at lower LOD to those at higher LOD). For the quantized wavelet coefficients of the vertices in each subset, the inverse quantizer 1100A iteratively performs inverse quantization on each of the wavelet coefficients in the subset, as shown in the shift-based iterator 1104. Once all wavelet coefficients in a subset have been inverse quantized, the inverse quantizer 1100A inverse quantizes the wavelet coefficients in the next subset. Within each inverse quantization operation, the inverse quantizer 1100A may include a reconstruction adjuster 1106 and an inverse scaler 1108. The reconstruction adjuster 1106 may apply a quantization offset indicated by one or more offset indicators 1110, as described above. Figure 10 As explained in the text, and further discussed below. Figure 13 Further details. The inverse scaler 1108 can apply an inverse scaling factor indicated by one or more scaling indicators 1112, as described above. Figure 10 As explained in the text, and further discussed below. Figure 13 Further details.

[0132] In some examples, when a subset of a set of vertices in a 3D mesh corresponds to different LODs among multiple LODs of the vertices, the inverse quantizer 1100A can iteratively inverse quantize signal samples (e.g., displacement signals and their corresponding quantized-transformed wavelet coefficient representations) from a lower LOD to a higher LOD. For example, in each iteration, the next input signal associated with the LOD is... (For example, Figure 10 input signal The signal is converted (e.g., inverse quantization based on a quantization offset) to an output signal. The output signal is an inversely quantized representation of the vertex displacement coefficients. The subset-based iterator 1102A iterates for each higher LOD level until the highest LOD level is processed, at which point all quantized wavelet coefficient signals (e.g., the input signal) are processed. All of these will be dequantized (e.g., the input signal). For example, a base mesh of 900 vertices can be subdivided into an upsampled mesh of 57,600 vertices, for example, spanning four LOD levels (e.g., LOD0 including vertices with indices 1 to 900, LOD1 including vertices with indices 901 to 3600, LOD2 including vertices with indices 3601 to 14400, and LOD3 including vertices with indices 14401 to 57600). In this example, the associated displacements (e.g., displacement values ​​and quantized wavelet coefficients) have the same order as these vertices. In this example, the inverse quantizer 1100A can start from the lowest LOD, which could be LOD0.

[0133] In some embodiments, the reconstruction adjuster 1106 may determine whether to apply quantization adjustment (e.g., reconstruction adjustment) based on a first indication in one or more offset indications 1110 received from the bitstream (e.g., decoded). For example, the first indication may be received (e.g., decoded) and associated with each subset of vertices. In some examples, when the subset corresponds to a LOD, the one or more offset indications 1110 may further indicate the LOD index (indicating a particular LOD) and whether quantization adjustment operation (e.g., reconstruction adjustment operation) of the reconstruction adjuster 1106 is enabled for said LOD index. When the first indication indicates that the reconstruction adjuster 1106 is enabled, the one or more offset indications 1100 may also include a second indication of the quantization offset used in the quantization adjustment operation corresponding to the LOD.

[0134] In some embodiments, the reconstruction adjuster 1106 is enabled by default, in which case the first indication is not signaled and only the second indication is signaled. In these embodiments, the quantization offset of a subset of vertices (e.g., a particular LOD) can be signaled as equal to zero, which has the same effect as quantization offset being disabled or not used for vertices in said subset.

[0135] In some examples, a first indication may be signaled for a 3D mesh (e.g., a mesh frame) or a sequence of mesh frames. In some examples, based on the first indication enabling quantization adjustment operations, the reconstruction adjuster 1106 may determine (e.g., set) the same quantization offset for all subsets (e.g., LOD) based on a second indication. In other examples, based on the first indication enabling quantization adjustment operations, the reconstruction adjuster 1106 may determine (e.g., set) the quantization offset for each subset (e.g., LOD) based on a second indication decoded for said specific subset.

[0136] In some embodiments, one or more offset indicators 1100 may include a first indicator and / or a second indicator transmitted from the encoder to the decoder. For example, as described above... Figure 8 As explained, the encoder can generate and signal (e.g., encode) one or more offset indicators 1110 in the bitstream based on a comparison of compression results between disabled and enabled for one or more inverse quantization processes corresponding to one or more subsets. For example, the encoder can signal one or more offset indicators 1100 to the decoder to enable / disable the reconstruction adjuster 1106 associated with each subset of vertices to minimize inverse quantization error (e.g., maximize compression gain). Therefore, the inverse quantizer 1100A (e.g., the decoder) can apply (e.g., implement and / or execute) the operation of the reconstruction adjuster 1106 based on one or more offset indicators 1110 signaled by the encoder. Based on disabling quantization adjustment for wavelet coefficients associated with subsets of vertices, the inverse quantizer 1100A can skip (e.g., omit) the reconstruction adjuster 1106, and the inverse scaler 1108 can directly scale the subsets (e.g., those at LOD) to the corresponding subsets. N The quantized wavelet coefficients of the vertices of the signal are inversely quantized (e.g., inversely scaled).

[0137] In some examples, one or more offset indicators 1100 include a single indicator that indicates whether quantization adjustment is enabled (e.g., disabled or skipped) for all LODs or mesh frame sequences of the 3D mesh. In some examples, one or more offset indicators 1100 include a single indicator that indicates whether to adjust (or not adjust) one of the LODs corresponding to its quantized wavelet coefficients during quantization adjustment. For example, a single indicator could indicate that the lowest LOD level corresponding to the coarsest resolution (e.g., the last LOD or LOD0) will have its associated quantization adjustment operation disabled. This can be useful because inverse quantization in said LOD has a lower quantization level, and the reconstructed signal can be a sufficiently accurate representation of the original uncompressed signal.

[0138] In some examples, one or more offset indicators 1100 include an indicator for each corresponding LOD in the LODs associated with the vertices of the grid frame. For example, an indicator for a LOD may indicate whether quantization adjustment operations for that LOD are enabled or disabled. At the encoder, the encoder may compare the compression results between enabling and disabling quantization adjustment for the LOD to determine whether the indicator for quantization adjustment operations transmitted to the decoder in the bit stream is enabled or disabled. The decoder can then decode the indicator for the corresponding LOD from the bit stream and selectively perform quantization adjustment operations (of the reconstruction adjuster 1106) on the wavelet coefficients of the LOD according to the indicator.

[0139] In some examples, one or more offset indicators 1100 include an indicator for each corresponding LOD in the LODs associated with the vertices of the mesh frame. However, instead of comparing the compression results between quantization adjustment operations that enable and disable LODs to determine whether to signal the indication of the quantization adjustment operation, the encoder may compare the compression results between sets of reconstruction adjustment operations that enable / disable LODs to determine the combination of indications that increase (e.g., maximize) the compression gain. Similarly, although the above examples are described with respect to LODs, they can be similarly applied to subsets (which are not necessarily LODs).

[0140] In some examples, one or more offset indicators 1100 may indicate a LOD index that identifies the LOD among multiple LODs of the mesh frame that enables / disables quantization adjustment based on the indicator. For example, the indicator may include an LOD index and a binary indicator (e.g., a binary flag) whose value indicates enabling / disabling the quantization adjustment operation corresponding to the LOD index.

[0141] In some examples, one or more offset indicators 1100 can be signaled for each 3D mesh frame sequence, each mesh frame, each tile, each patch, each patch group, or each LOD. In some examples, one or more indicators include indicators that can be signaled for each LOD in a mesh frame.

[0142] In some embodiments, the indication is a first indication (e.g., a mode indication) to enable / disable quantization adjustment operation for each subset of vertices, which is not relayed between the encoder and decoder and is predetermined. For example, quantization adjustment operation of wavelet coefficient signals at all LODs can be enabled if relayed in one or more offset indications 1100.

[0143] In some embodiments, one or more offset indicators 1100 may include a second indicator (e.g., a flag or syntax element) signaled in the bitstream, which is used by the reconstruction adjuster 1106 to determine (e.g., derive or calculate) the adjusted quantized wavelet coefficients (e.g., the input signal). The quantization offset used. For example, the second indicator can indicate an index of a set of quantization offsets to specify one, or a specific value, or an exponent of a specific value (e.g., the value n represents...). (The index). In some examples, the second indication can be signaled for each 3D mesh sequence, each mesh frame, each tile, each facet, each facet group, and each LOD.

[0144] In some embodiments, one or more offset indicators 1110 may include a third indicator (e.g., a flag or syntax element) signaled in the bit stream, the indicator being used by the reconstruction adjuster 1106 to determine (e.g., derive or calculate) the adjusted quantized wavelet coefficients (e.g., the input signal). The quantization offset of the adjusted quantization offset value. For example, the third indicator can indicate the index of a set of offsets, or the exponent of a specific value (e.g., the value n represents...). The index is used to adjust one of the quantization offsets.

[0145] In some examples, the inverse scaler 1108 can determine the inverse scaling factor based on one or more scaling instructions 1112. For example, the one or more scaling instructions 1112 may include one or more scaling parameters, such as quantization parameters ( qp ) and / or bit depth parameters ( BDP ), as will be in the following text Figure 13 Further description is provided below. In some examples, one or more scaling parameters can be signaled (and associated with) for each subset of vertices (e.g., corresponding to the same LOD, or in the same 3D mesh sequence, in a submesh of a 3D mesh, in the same patch (or group of patches) of a submesh or 3D mesh, etc.). In some examples, scaling parameters can be derived across subsets of vertices (e.g., scaled by a fixed ratio value).

[0146] In some embodiments, one or more scaling indicators 1112 may include an indicator quantization parameter ( qp ) and / or bit depth parameters ( BDP One or more indicators are provided to determine whether signaling (e.g., presence) is performed in the bitstream. For example, one or more indicators can be signaled for each subset of vertices. Similar to how an encoder can determine whether one or more offset indicators should enable / disable quantization adjustment, an encoder can determine whether indicators are provided in the bitstream for quantization parameters (…). qp ) and / or bit depth parameters ( BDP One or more indications are used to signal (e.g., whether they exist). If one or more indications indicate that those parameters do not exist (i.e., no signaling is performed in the bit stream), then quantization parameters can be used. qp) and / or bit depth parameters ( BDP The default value.

[0147] Figure 11 B is a diagram illustrating an example process of an inverse quantizer 1100B, according to some embodiments, for iteratively inverse quantizing the quantized-transformed wavelet coefficients of vertices in each subset of vertices of a 3D mesh. The operation of the inverse quantizer 1100B can be performed by a decoder (e.g., Figure 2A and Figure 2B The inverse quantizer 222, or Figure 3 The inverse quantizer 312) is used in some embodiments. According to some embodiments, the inverse quantizer 1100B performs an inverse quantization scheme with a reconstruction adjustment process. The inverse quantizer 1100B can be a dead-zone inverse quantizer, as described above. Figure 10 As explained in the text. Very similar to subset-based iterator 1102A and shift-based iterator 1104A, inverse quantizer 1100B includes corresponding subset-based iterator 1102B and shift-based iterator 1104B, respectively. Unlike inverse quantizer 1100A, inverse quantizer 1100B may include inverse scaler 1116 and reconstruction scaler 1118 that reverse the order (e.g., sequence) of the operations performed by reconstruction adjuster 1106 and inverse scaler 1108, as described below. Figure 13 Further explanation follows. Similar to the inverse quantizer 1100A, the inverse scaler 1116 can operate according to one or more scaling instructions 1112, and similar to the reconstruction adjuster 1106, the reconstruction adjuster 1118 can operate according to one or more offset instructions 1110.

[0148] Figure 12 A flowchart 1200 illustrates a method for applying a quantization scheme to transformed wavelet coefficients according to some embodiments. In some examples, the method may be performed by an encoder (e.g., Figure 1 Encoder 114, Figure 2A encoder 200A or Figure 2B The encoder 200B is used for execution. The following descriptions of the various steps can be found above regarding... Figure 2A Quantizer 212 or Figure 2B The operation described by quantizer 212.

[0149] At box 1202, the encoder determines wavelet coefficients representing the displacements of a set of vertices of a three-dimensional (3D) mesh. For example, the wavelet coefficients can be generated by the encoder's wavelet transform (e.g., Figure 2A or Figure 2B The wavelet transform 210) is determined. As described above, the encoder can transform (e.g., transform) the determined displacements of the set of vertices into wavelet coefficients according to the wavelet transform (e.g., wavelet transform lifting scheme).

[0150] At box 1204, the encoder determines the subset based on the quantization scaling factor (e.g., with) associated with the subset. Figures 8 to 10 Quantization size in s A first wavelet coefficient corresponding to a subset of the set of vertices is quantized using a quantization scaling factor (associated with, and may be referred to as, a scaling factor). In some examples, a quantization scaling factor is determined for each subset of non-overlapping subsets of the set of vertices. The wavelet coefficients corresponding to vertices in each of the subsets can then be quantized according to the corresponding quantization scaling factor associated with the subset. For example, a subset of vertices may include vertices in the same LOD that are in multiple LODs. For example, a subset of vertices may include vertices in a submesh of a 3D mesh. For example, a subset of vertices may include vertices in a submesh or a patch of a 3D mesh.

[0151] In some examples, it can be based on the quantization scaling factor ( s_q ) and can be based on dead-zone bias parameters ( Possible dead zone offset () DZ ) to the wavelet coefficients ( The unquantized raw values ​​are quantized to determine the quantized wavelet coefficients. ),as follows:

[0152] sgn The function is to output wavelet coefficients ( The sign function of the sign of ) (e.g., the plus / minus function), and The magnitude is displayed as an absolute value function. If dead-zone offset is not used (or enabled), the dead-zone offset can be... DZ The value is set to 0. The quantizer can apply a lower bound rounding function (as shown above) to determine the smallest integer less than or equal to the wavelet coefficients scaled by the dead-zone offset (which may be optional). In other examples, the quantizer can apply an up bound function to determine the smallest integer greater than or equal to the wavelet coefficients scaled by the dead-zone offset.

[0153] In some examples, the dead zone offset can be based on a dead zone offset parameter that indicates (e.g., corresponds to) the dead zone offset. This is also shown above (for example, In some examples, the dead zone bias parameter ( The value can be negative. In some examples, the width of the dead zone offset can refer to... DZ Or the region surrounding 0, for example ,like Figure 8 The explanation is as follows.

[0154] Generally speaking, the quantization scaling factor ( s_q ) is a value between 0 (e.g., excluding endpoints) and 1 (e.g., including endpoints) such that the quantized wavelet coefficients will be smaller than the unquantized wavelet coefficients. Quantization scaling factor ( s_q ) can be the reciprocal of the quantization size ( s = 1 / s_q ),like Figures 8 to 10 As shown in the example. In some examples, the determined dead-zone offset can be smaller than the quantization size ( s Or inverse quantization scaling factor).

[0155] In some examples, it can be based on quantization parameters ( qp ) and bit depth parameters ( bitDepthPosition or BDP To determine the quantization scaling factor (e.g., the positive vectorization scaling factor) s_q For example, wavelet coefficients (e.g., representing displacement signals) d Quantization scaling factor () scale [ d The power of two can be determined based on the following relationship: scale[ d ] = qp >= 0? power( 2.0, 16 - BDP + ( 4 - qp ) / 6) : 0.0 In some examples, for each subset of the set of vertices, quantization parameters can be determined and / or signaled (e.g., encoded in a bitstream). qp ) and / or bit depth parameters ( BDP In some examples, quantization parameters and / or bit depth parameters can be signaled to a 3D mesh (e.g., a 3D mesh frame) or a sequence of 3D meshes (e.g., including 3D meshes).

[0156] In some examples, the quantization scaling factor can be determined as a fixed ratio between subsets (e.g., LODs), such that the quantization scaling factor is scaled (e.g., increased or decreased) according to the fixed ratio of each subset. In some examples, the quantization scaling factor, quantization parameter, and / or bit depth parameter can be derived from one subset to one or more other subsets.

[0157] In some examples, the displacement can be represented as three components, in which case the quantization scaling factor can be applied to each component. In a typical implementation, the quantization scaling factor can be determined to be the same for each component.

[0158] In some examples, the encoder iteratively quantizes the wavelet coefficients of each subset of the set of vertices. For instance, when each subset corresponds to vertices in the same LOD, the encoder can iteratively quantize the wavelet coefficients of each subset according to the order of LODs (e.g., from higher LOD to lower LOD).

[0159] At box 1206, the encoder encodes quantization information indicating the quantization offset associated with a subset in the bitstream used for quantizing the first wavelet coefficients. In some examples, the quantization information may further indicate one or more parameters used for quantization (such as quantization parameters) and / or bit depth parameters. As explained above, the quantization information can be signaled for each subset of vertices.

[0160] In some examples, the encoder determines the quantization offset associated with a subset based on the difference between the first wavelet coefficients and the quantized first wavelet coefficients. For example, the quantization offset can be determined as a measure of the center of the difference (e.g., mean, median, mode, etc.). In some examples, the quantization information includes the corresponding quantization offset for each subset of the non-overlapping subsets of the set of vertices, where the subset is one of the non-overlapping subsets.

[0161] In some examples, the quantization information includes an indicator of the quantization offset (e.g., one or more indicators, flags, or syntax elements). In some examples, the indicator may include the sign and magnitude of the quantization offset.

[0162] In some examples, the indicator may contain an index for selecting a quantization offset from multiple quantization offsets. For instance, the encoder may signal multiple quantization offsets in a list (e.g., an array or a table).

[0163] In some examples, the indicator (or indication) of the quantization offset can be entropy decoded, for example using unary codes, Rice codes, Golomb codes, Exp-Golomb codes, etc.

[0164] In some examples, quantization information includes an indication of whether quantization adjustment is enabled for each subset of (non-overlapping) subsets of vertices (e.g., selectively enabling or disabling quantization adjustment for each subset of vertices). For example, if the subset corresponds to a mesh frame in a mesh frame sequence (e.g., a 3D mesh), the indication can be signaled for each mesh frame. For example, if the subset corresponds to a LOD, the indication can be signaled for each LOD. For example, the indication (e.g., a mode indication for quantization adjustment) can be signaled for a 3D mesh frame sequence, each mesh frame, each submesh, each tile, each patch group, each patch, and / or each LOD.

[0165] In some examples, the encoder can compare disabling and enabling quantization adjustment operations corresponding to the subset (e.g., as described above). Figure 10 , Figure 11 A to Figure 11 B and the following text Figure 13 The compression result of inverse quantization (e.g., rate distortion optimization (RDO) cost) between the two is used to determine the indication. For example, if the compression gain increases (e.g., fewer bits need to be generated in the bitstream 260) when quantization adjustment operation (e.g., also known as inverse quantization adjustment or reconstruction adjustment) is enabled, the encoder can determine that the indication of quantization adjustment operation is enabled, and further indicate a second indication of quantization offset in the quantization information. For example, the indication of quantization adjustment can instruct the reconstruction adjuster (e.g., at the decoder). Figure 11 A's Reconstruction Adjuster 1106 or Figure 11 Is the reconstruction adjuster (1118) of B enabled? If the quantization adjustment is disabled (as indicated), the operation of the reconstruction adjuster can be disabled.

[0166] In some examples, the indication of the quantization adjustment operation can be entropy decoded, for example, using unary codes, Rice codes, Golomb codes, Exp-Golomb codes, etc.

[0167] In some examples, the encoder can offset the dead zone in the quantization information (e.g., DZ Value or dead zone bias parameter ( The dead zone offset is indicated by the signal. In some examples, the dead zone offset can be indicated by entropy decoding, for example, using unary codes, Rice codes, Golomb codes, Exp-Golomb codes, etc.

[0168] In some examples, the encoder also adjusts whether to adjust the offset value (a) δ The indication applied to the quantization offset is used for signal transmission, such as Figure 10 As explained in [the document]. In some examples, the encoder can signal whether to apply one or more thresholds to the quantization offset, such as... Figure 10 As explained in [the document]. In other examples, the quantization offset transmitted from the encoder to the decoder is adjusted based on the applied offset value (a). δ This is determined by (and / or one or more thresholds) such that no further indications are transmitted in the bit stream.

[0169] At box 1208, the encoder encodes the quantized first wavelet coefficients in the bit stream to represent the first displacement in the displacement. As described above, the encoder can quantize the wavelet coefficients corresponding to each of the subsets of vertices. Then, the encoder can encode the quantized wavelet coefficients for each of the subsets of vertices.

[0170] In some examples, an encoder can transmit (e.g., encode) quantized wavelet coefficients representing the displacements of a set of vertices in a 2D image. For instance, the wavelet coefficients of the vertices of a 3D mesh can be generated by an image packer (e.g., ...). Figure 2A and Figure 2B Image packer 214) arranged (e.g., packed) into 2D images (e.g., Figure 7A The displacement image 720 in the image is shown. In some examples, the wavelet coefficients can be arranged by a quantizer (e.g., before being arranged by the image packer). Figure 2A and Figure 2B Quantizer 212) quantization, such as Figure 2A and Figure 2B As described in [the document / article]. Furthermore, these 2D images can be encoded by 2D video codecs, such as [other codecs / programs]. Figure 2A and Figure 2B The video encoder 216 described herein.

[0171] Figure 13 A flowchart 1300 illustrates a method for applying an inverse quantization scheme to quantized-transformed wavelet coefficients according to some embodiments. In some examples, the method may be performed by a decoder (e.g., Figure 1 Decoder 120 or Figure 3 The decoder 300 is executed. The following descriptions of the various steps can be found in the above section regarding... Figure 2A and Figure 2B Inverse quantizer 222 or Figure 3 The operation described by the inverse quantizer 312.

[0172] At box 1302, the decoder decodes the quantized wavelet coefficients from the bitstream, representing the displacements of a set of vertices from a three-dimensional (3D) mesh. (See above regarding...) Figure 11 A and Figure 11 As explained in B, the decoder can similarly receive and decode quantized wavelet coefficients representing the displacements (including the displacements of the vertices) of the set of vertices.

[0173] In some examples, the set of vertices is ordered according to the level of detail (LOD) of the vertices, and the quantized wavelet coefficients can be decoded from the bitstream according to the LOD. For example, the decoder can sequentially apply an inverse wavelet transform (e.g., an inverse lifting scheme) to the transformed wavelet coefficients received in the bitstream to determine the (untransformed) quantized wavelet coefficients, including the quantized wavelet coefficients.

[0174] In some examples, the quantized wavelet coefficients are decoded based on decoding an image (e.g., a 2D image) containing the transformed wavelet coefficients from a bitstream. The decoder can determine the transformed wavelet coefficients from the decoded image based on the packing order of the wavelet coefficients in the image. For example, as described above... Figure 12 As explained in the text, the quantized wavelet coefficients are arranged (e.g., packed) into a 2D image by the image packer at the encoder (e.g., ...). Figure 7A The displacement image 720 in the image. Therefore, the decoder can include a video decoder (e.g., for decoding 2D images containing quantized wavelet coefficients). Figure 3 The video decoder 308). The decoder may contain an image unpacker (e.g., Figure 3 The image unpacker 310 reverses (e.g., unpacks) the operation of the image packer to determine the quantized wavelet coefficient sequence. In some examples, the decoder may include an inverse quantizer (e.g., inverse quantizer 312) to inverse quantize the first wavelet coefficients unpacked.

[0175] At box 1304, the decoder decodes from the bitstream quantization information indicating the quantization offset associated with a subset of vertices in the set of vertices. In some embodiments, the set of vertices comprises a non-overlapping subset of vertices, wherein the subset is one of the non-overlapping subsets. In some examples, the non-overlapping subset corresponds to the level of detail (LOD). In some examples, the non-overlapping subset corresponds to a submesh of a 3D mesh. In some examples, the non-overlapping subset corresponds to a facet within a submesh of a submesh.

[0176] In some examples, the quantization offset corresponds to a sequence of 3D meshes containing 3D meshes. For instance, the quantization offset may be associated with a subset of vertices across mesh frames that span a 3D mesh sequence (such as vertices in the same LOD across multiple LODs).

[0177] In some examples, the quantization information further indicates the corresponding quantization offset associated with each subset of non-overlapping subsets. For example, each subset may be associated with a different LOD among the LODs associated with the vertices of the 3D mesh, in which case the subset of vertices contains only the vertices in the first (same) LOD among the multiple LODs, and the vertices are in the subset based on whether the vertex is in the first LOD.

[0178] In some embodiments, the quantization information includes an indicator that indicates the quantization offset, as explained above. In some examples, the indicator may include the sign and magnitude of the quantization offset. In some examples, the indicator selects a quantization offset from a plurality of quantization offsets. For example, the plurality of quantization offsets may be stored in a list (e.g., an array or a table), which may be signaled from the encoder to the decoder. For example, the indicator may be an index in the list.

[0179] In some embodiments, the decoder receives (e.g., decodes) an indication of whether quantization adjustment is enabled from the bitstream (e.g., a mode indication). The adjustment of wavelet coefficients can be based on the indication of quantization adjustment being enabled.

[0180] In some examples, mode indications can be received for each subset of non-overlapping subsets of vertices. In some examples, mode indications can be received (and decoded) for a sequence of 3D meshes containing a 3D mesh or for each 3D mesh frame. Similar to quantization offset indications, one or more mode indications can be signaled for a sequence of 3D mesh frames, each mesh frame or for each subset of vertices corresponding to each submesh of a mesh frame / 3D mesh, each patch group of submeshes of a 3D mesh / mesh frame, each tile, each patch in a patch group, or each LOD. In some examples, based on mode indications indicating no quantization adjustment, reconstruction adjuster 1102 can be disabled, and the operation at box 1308 will be skipped (e.g., omitted).

[0181] In some examples, where the subset corresponds to different LODs, one or more mode indicators can be decoded from the bitstream to selectively enable quantization adjustment for a specific LOD among multiple LODs. For example, one or more mode indicators may include an index of the LOD (e.g., identifying the LOD) and a binary indication of whether to enable or disable (e.g., skip) the quantization adjustment operation corresponding to the LOD.

[0182] In some examples, the received quantization offset is determined by the encoder. For instance, the quantization offset value associated with a subset of vertices can be determined (derived / calculated) from the difference between the shift value after wavelet transform (lifting) and before quantization, and the shift value after inverse quantization and before inverse wavelet transform (inverse lifting). In some examples, the offset value associated with a subset of vertices can be further determined by an offset adjustment value (a). δ ) and / or one or more threshold adjustments, as described above Figure 10 As explained in [the document]. In some examples, the offset value can be scaled by a fixed value based on a subset of vertices (e.g., the LOD corresponding to the subset or the index of the LOD).

[0183] In other examples, the decoder itself can apply an offset adjustment value (a δ ) and / or one or more thresholds, as described above Figure 10 As explained in the text. For example, quantization information can indicate whether to adjust the offset value (a) δ One or more thresholds are applied to the quantization offset value. One or more thresholds can be predetermined or signaled within the quantization information. The offset adjustment value (a) δ It can be derived from the LOD associated with a subset of vertices, or it can be transmitted in the quantized information.

[0184] Related to the above Figure 12The encoding of the indication of whether quantization adjustment is enabled (e.g., mode indication) and / or quantization offset indication as described herein is related to the encoding of the indication. The decoder can perform entropy decoding to decode the indication of mode indication and / or quantization offset from the bit stream.

[0185] At box 1306, the decoder (e.g., Figure 2A and Figure 2B The inverse quantizer 222, or Figure 3 The inverse quantizer 312) inverse quantizes the quantized wavelet coefficients to determine the wavelet coefficients. In some embodiments, to perform inverse quantization, the decoder determines an inverse quantization scaling factor associated with a subset (e.g., corresponding to...). Figures 8 to 10 Quantization size in s In some examples, an inverse quantization scaling factor is determined for each subset of non-overlapping subsets of the set of vertices. Then, the quantized wavelet coefficients corresponding to the vertices in each subset can be inverse quantized according to the corresponding inverse quantization scaling factor associated with the subset. The decoder then scales the quantized wavelet coefficients based on the inverse quantization scaling factor to determine the wavelet coefficients.

[0186] In some examples, it can be based on the inverse quantization scaling factor ( s_i ) and quantization offset (e.g., or ) to quantized wavelet coefficients ( The quantized wavelet transform coefficients are inversely quantized to determine the inversely quantized wavelet coefficients. ),as follows:

[0187] In some examples, the offset adjustment value 'a' can be applied. δ To determine the inverse quantization scaling factor ( s_i ), to adaptively change the scaling factor according to, for example, the LOD level. s_i The value of ). For example, the offset adjustment value a. δ It can include a scaling factor ( s_lod ) or inverse scaling factor ( s_lod_i For example, in LOD0 s_i Can be multiplied s_lod (For example, 1 / 2), making LOD1... s_i The value is equal to half the value in LOD0.

[0188] sgn The function is to output wavelet coefficients ( The sign function of the sign of ) (e.g., the plus / minus function), and The magnitude is displayed as an absolute value function. .

[0189] In some examples, the quantization offset (e.g., bias or shift) can be added to the inversely scaled quantized wavelet coefficients. offset of ) ).

[0190] In some examples, the quantization offset (e.g., bias or shift) can be during inverse scaling ( The offset added before the quantized wavelet coefficients ( ).

[0191] In some examples, the quantization offset (e.g., bias or shift) can be an adjustment value a based on the offset. δ Adjusted offset ( This allows the offset value to be adaptively changed based on the LOD level.

[0192] In some embodiments, the inversely quantized wavelet coefficients are shifted (e.g., adjusted) to output the inversely quantized wavelet coefficients. It can get closer to the original wavelet coefficients at the encoder. .

[0193] In some examples, the inverse quantization scaling factor s_i Corresponding to by Figure 12 Quantization scaling factor applied to the encoder in s_q The reciprocal (for example, s_i = 1 / s_q In some examples, it can be based on quantization parameters ( qp ) and / or bit depth parameters ( bitDepthPosition or BDP ) to determine the inverse quantization scaling factor (e.g., s_i For example, quantized wavelet coefficients (e.g., representing displacement signals) d The inverse quantization scaling factor () iscale [ d The power of two can be determined based on the following relationship: iscale[ d ] = qp >= 0? power(0.5, 16 - BDP + ( 4 - qp ) / 6) : 0.0 In some examples, quantization parameters can be determined and / or decoded for each subset of the set of vertices (e.g., from a bitstream). qp ) and / or bit depth parameters ( BDP In some examples, quantization parameters and / or bit depth parameters can be decoded against a 3D mesh (e.g., a 3D mesh frame) or a sequence of 3D meshes (e.g., containing 3D meshes).

[0194] In some examples, the inverse quantization scaling factor can be determined as a fixed ratio between subsets (e.g., LODs), such that the inverse quantization scaling factor is scaled (e.g., increased or decreased) according to the fixed ratio of each subset. In these examples, the inverse quantization scaling factor, quantization parameters, and / or bit depth parameters are not transmitted in the bit stream.

[0195] In some examples, the displacement can be represented as three components, in which case an inverse quantization scaling factor can be applied to each component. In a typical implementation, the inverse quantization scaling factor can be determined to be the same for each component.

[0196] In some examples, the quantization information further indicates the quantization parameters and / or bit depth parameters associated with the subset. In some examples, the quantization information further indicates the quantization parameters or bit depth parameters that can be signaled for each corresponding subset of non-overlapping subsets of vertices.

[0197] At box 1308, the decoder adjusts the wavelet coefficients based on the quantization offset to determine the vertex displacement, according to the subset of vertices.

[0198] In some examples, wavelet coefficients are adjusted based on combining (e.g., adding) quantization offsets to the wavelet coefficients to reconstruct the displacement. For instance, a quantization offset can be added to the magnitude of the wavelet coefficients to reconstruct the displacement such that the sign of the wavelet coefficients is the same as the sign of the adjusted wavelet coefficients.

[0199] In some examples, the decoder can further perform an inverse transform on the wavelet coefficients (which have been inversely quantized) to reconstruct the vertex displacements. For example, the decoder (e.g., Figures 2A to 2B The inverse wavelet transform 220, or Figure 3 The inverse wavelet transform (314) can apply an inverse quantizer (e.g., an inverse lifting scheme) to the inverse-quantized wavelet coefficients to determine the reconstructed shift. In some examples, the decoder can iteratively apply the inverse transform to each subset of the inverse-quantized wavelet coefficients. For example, in the case where the subset corresponds to a LOD, the decoder can iteratively apply the inverse transform to the inverse-quantized wavelet coefficients in each LOD according to the order of the LODs (e.g., in ascending order).

[0200] In some examples, the decoder reconstructs the geometry of the 3D mesh based on determined displacements (and similarly determined displacements of vertices in the set of vertices of the 3D mesh). In other examples, the decoder reconstructs the geometry based on a first displacement and a base mesh. For example, the decoder can decode the base mesh associated with the 3D mesh from the bitstream. The decoder can then iteratively apply a subdivision scheme to the base mesh to generate vertex positions of the subdivided base mesh, where each subset is associated with a subdivision iteration. To reconstruct the geometry of the 3D mesh, the decoder can add the displacements of the vertices to the positions of the corresponding vertices in the subdivided base mesh. The reconstructed 3D mesh can be determined after the determined displacements are applied to the corresponding vertices of the subdivided base mesh.

[0201] Embodiments of this disclosure can be implemented in hardware using analog and / or digital circuitry, in software, by executing instructions by one or more general-purpose or special-purpose processors, or as a combination of hardware and software. Therefore, embodiments of this disclosure can be implemented in the environment of a computer system or other processing system. An example of such a computer system 1400 is... Figure 14 As shown in the attached figures above. The blocks depicted (such as...) Figure 1 The blocks in the flowcharts described herein can be executed on one or more computer systems 1400. Furthermore, each step in the flowcharts depicted herein can be implemented on one or more computer systems 1400. When more than one computer system 1400 is used to implement embodiments of this disclosure, the computer systems 1400 can be interconnected by one or more networks to form a cluster of computer systems that can act as a single, seamless resource pool. The interconnected computer systems 1400 can form a computer "cloud".

[0202] Computer system 1400 includes one or more processors, such as processor 1404. Processor 1404 may be, for example, a dedicated processor, a general-purpose processor, a microprocessor, or a digital signal processor. Processor 1404 may be connected to communication infrastructure 1402 (e.g., a bus or network). Computer system 1400 may also include main memory 1406, such as random access memory (RAM), and may also include secondary memory 1408.

[0203] Secondary storage 1408 may include, for example, a hard disk drive 1410 and / or a removable storage drive 1412, which represents a magnetic tape drive, an optical disc drive, etc. The removable storage drive 1412 can be read from and / or written to the removable storage unit 1416 in a well-known manner. The removable storage unit 1416 represents a magnetic tape, optical disc, etc., read from and written to by the removable storage drive 1412. Those skilled in the art will understand that the removable storage unit 1416 includes a computer-usable storage medium in which computer software and / or data are stored.

[0204] In an alternative embodiment, secondary storage 1408 may include other similar components for allowing computer programs or other instructions to be loaded into computer system 1400. Such components may include, for example, removable storage unit 1418 and interface 1414. Examples of such components may include program boxes and box interfaces (such as those found in video game devices), removable storage chips (such as EPROM or PROM) and associated sockets, thumb drives and USB ports, and other removable storage units 1418 and interfaces 1414 that allow software and data to be transferred from removable storage unit 1418 to computer system 1400.

[0205] Computer system 1400 may also include communication interface 1420. Communication interface 1420 allows software and data to be transferred between computer system 1400 and external devices. Examples of communication interface 1420 may include modems, network interfaces (such as Ethernet cards), communication ports, etc. Software and data transmitted via communication interface 1420 are in the form of signals, which may be electronic signals, electromagnetic signals, optical signals, or other signals that can be received by communication interface 1420. These signals are provided to communication interface 1420 via communication path 1422. Communication path 1422 carries signals and may be implemented using wires or cables, optical fibers, telephone lines, cellular telephone links, RF links, and other communication channels.

[0206] The computer system 1400 may also include one or more sensors 1424. The one or more sensors 1424 can measure or detect one or more physical quantities and convert the measured or detected physical quantities into electrical signals in digital and / or analog form. For example, sensor 1424 may include an eye-tracking sensor to track the user's eye movements. Based on the user's eye movements, the display of the 3D mesh can be updated. In another example, one or more sensors 1424 may include a head-tracking sensor to track the user's head movements. Based on the user's head movements, the display of the 3D mesh can be updated. In yet another example, one or more sensors 1424 may include a camera sensor for taking photographs and / or a 3D scanning device, such as a laser scanning device, a structured light scanning device, and / or a modulated light scanning device. The 3D scanning device can acquire geometric information by moving one or more laser heads, structured light, and / or modulated light cameras relative to the scanned object or scene. The geometric information can be used to construct a 3D mesh.

[0207] As used herein, the terms "computer program medium" and "computer-readable medium" are used to refer to tangible storage media, such as removable storage units 1416 and 1418 or a hard disk installed in hard disk drive 1410. These computer program products are components used to provide software to computer system 1400. The computer program (also referred to as computer control logic) may be stored in main memory 1406 and / or secondary storage 1408. The computer program may also be received via communication interface 1420. When executed, such a computer program enables computer system 1400 to implement the disclosures discussed herein. Specifically, when executed, the computer program enables processor 1404 to implement the processes of this disclosure, such as any of the methods described herein. Therefore, such a computer program represents the controller of computer system 1400.

[0208] In another embodiment, the features of this disclosure can be implemented in hardware using, for example, hardware components such as application-specific integrated circuits (ASICs) and gate arrays. Implementing a hardware state machine to perform the functions described herein will also be apparent to those skilled in the art.

Claims

1. A method comprising: Obtain quantized wavelet coefficients representing the displacement of a set of vertices of a 3D mesh from the bitstream; Quantization information indicating the quantization offset associated with each corresponding subset of the non-overlapping subsets of the set of vertices is obtained from the bit stream; The quantized wavelet coefficients are inversely quantized to determine the wavelet coefficients of a vertex in a subset of the non-overlapping subsets. as well as Based on the fact that the vertex is in the subset, the wavelet coefficients are adjusted according to the quantization offset associated with the subset to determine the displacement of the vertex.

2. A method comprising: Decode the quantized wavelet coefficients of the displacement of a vertex in a set of vertices of a 3D mesh from the bitstream; Decode quantization information from the bitstream that indicates the quantization offset associated with a subset of vertices in the set of vertices; The quantized wavelet coefficients are inversely quantized to determine the wavelet coefficients; as well as Based on the vertices in the subset, the wavelet coefficients are adjusted according to the quantization offset to determine the displacement of the vertices.

3. The method of claim 2, wherein the set of vertices comprises a non-overlapping subset of vertices, and wherein the subset is one of the non-overlapping subsets.

4. The method according to any one of claims 1 and 3, wherein the non-overlapping subset corresponds to the level of detail (LOD).

5. The method according to any one of claims 1 and 3, wherein the non-overlapping subset corresponds to a sub-mesh of the 3D mesh.

6. The method according to any one of claims 1 and 3, wherein the non-overlapping subset corresponds to a patch in a sub-mesh of the sub-mesh.

7. The method of claim 3, wherein the quantization information further indicates a corresponding quantization offset associated with each subset of the non-overlapping subsets.

8. The method of claim 7, wherein the vertices are in a plurality of levels of detail (LODs), wherein the subset of vertices includes only vertices in a first LOD of the plurality of LODs, and wherein the vertices are in the subset based on the vertices being in the first LOD.

9. The method according to any one of claims 1 to 8, wherein the quantization offset corresponds to a 3D mesh sequence including the 3D mesh.

10. The method of any one of claims 1 and 2, wherein the quantization information includes an indicator indicating the quantization offset.

11. The method of claim 10, wherein the indicator selects the quantization offset from a plurality of quantization offsets.

12. The method of claim 11, wherein the plurality of quantization offsets are stored in a table or list.

13. The method of any one of claims 10 and 11, wherein the quantization information further includes an additional indicator indicating an adjustment to the quantization offset, wherein the quantization offset is determined based on the indicator and the additional indicator.

14. The method according to any one of claims 1 to 13, further comprising: An indication is obtained from the bitstream as to whether quantization adjustment is enabled, wherein the wavelet coefficients are adjusted based on the indication of enabling quantization adjustment.

15. The method of claim 14, wherein the set of vertices comprises a non-overlapping subset of vertices, and wherein the indication is received for each subset of the non-overlapping subset.

16. The method according to any one of claims 14 to 15, wherein the instruction is received for a 3D mesh sequence including the 3D mesh.

17. The method according to any one of claims 1 to 16, wherein adjusting the wavelet coefficients comprises: The quantization offset is added to the wavelet coefficients to reconstruct the displacement.

18. The method of claim 17, wherein the quantization offset is added to the magnitude of the wavelet coefficients to reconstruct the displacement.

19. The method according to any one of claims 1 to 18, wherein inverse quantization of the quantized wavelet coefficients comprises: Determine the inverse quantization scaling factor based on the quantization parameters; as well as The quantized wavelet coefficients are scaled based on the inverse quantization scaling factor to determine the wavelet coefficients.

20. The method of claim 19, wherein the inverse quantization scaling factor is further based on a bit depth parameter.

21. The method of claim 20, wherein the quantization information further indicates the quantization parameter or the bit depth parameter associated with the subset.

22. The method of claim 21, wherein the set of vertices comprises a non-overlapping subset of vertices, the non-overlapping subset comprising the subset, and wherein the quantization information further indicates a quantization parameter or bit depth parameter for each corresponding subset in the non-overlapping subset.

23. The method according to any one of claims 1 to 22, wherein the set of vertices is ordered according to the level of detail (LOD) of the vertices, and wherein decoding the quantized wavelet coefficients of the vertices comprises: The inverse wavelet transform is sequentially applied to the transformed wavelet coefficients received in the bit stream to determine quantized wavelet coefficients that include the quantized wavelet coefficients.

24. The method according to any one of claims 1 to 23, further comprising reconstructing the geometry of the 3D mesh based on the determined displacement.

25. The method of claim 24, further comprising: Decode the underlying mesh associated with the 3D mesh; as well as The subdivision scheme is iteratively applied to the base mesh to generate the positions of the vertices of the subdivided base mesh, wherein each subset of the subsets is associated with a subdivision iteration.

26. The method of claim 25, wherein reconstructing the geometry comprises: The displacement of the vertex is added to the position of the corresponding vertex in the base mesh of the subdivision.

27. The method according to any one of claims 1 to 26, wherein decoding the quantized wavelet coefficients comprises: Decoding the image, including transformed wavelet coefficients, from the bitstream; as well as The transformed wavelet coefficients are determined from the decoded image based on the packing order of the wavelet coefficients in the image.

28. The method of claim 27, wherein decoding the quantized wavelet coefficients further comprises: The inverse wavelet transform is applied to the transformed wavelet coefficients to determine quantized wavelet coefficients that include the quantized wavelet coefficients.

29. A decoder comprising: One or more processors; as well as A memory that stores instructions, which, when executed by the one or more processors, cause the decoder to perform the method according to any one of claims 1 to 28.

30. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of the device, cause the device to perform the method according to any one of claims 1 to 28.

31. A method comprising: Determine the wavelet coefficients representing the displacement of a set of vertices in a 3D mesh; The first wavelet coefficients corresponding to the subset are quantized according to the quantization scaling factor associated with the subset of the set of vertices; Quantization information indicating the quantization offset associated with the subset is encoded in the bitstream used for the quantized first wavelet coefficients; as well as The quantized first wavelet coefficients are encoded in the bit stream to represent the first displacement in the displacement.

32. The method of claim 31, further comprising: The quantization offset associated with the subset is determined based on the difference between the first wavelet coefficients and the quantized first wavelet coefficients.

33. The method of claim 32, wherein the quantization information includes a corresponding quantization offset for each subset of the non-overlapping subsets of the set of vertices, and wherein the subset is one of the non-overlapping subsets.

34. The method according to any one of claims 32 to 33, wherein the quantization offset comprises the average of the differences.

35. The method according to any one of claims 31 to 34, wherein the set of vertices comprises a non-overlapping subset of vertices, and wherein the subset is one of the non-overlapping subsets.

36. The method of claim 35, wherein the non-overlapping subset corresponds to the level of detail (LOD).

37. The method of claim 35, wherein the non-overlapping subset corresponds to a sub-mesh of the 3D mesh.

38. The method of claim 35, wherein the non-overlapping subset corresponds to a patch in one of the sub-mesh sub-mesh.

39. The method of claim 35, wherein the quantization information further indicates a corresponding quantization offset associated with each subset of the non-overlapping subsets.

40. The method of claim 39, wherein the vertices are in a plurality of levels of detail (LODs), wherein the subset of vertices includes only vertices in a first LOD of the plurality of LODs, and wherein the vertices are in the subset based on the vertices being in the first LOD.

41. The method according to any one of claims 31 to 40, wherein the quantization offset corresponds to a 3D mesh sequence including the 3D mesh.

42. The method according to any one of claims 31 to 41, wherein the quantization information includes an indicator indicating the quantization offset.

43. The method of claim 42, wherein the indicator selects the quantization offset from a plurality of quantization offsets.

44. The method of claim 43, wherein the plurality of quantization offsets are stored in a table or list.

45. The method of any one of claims 42 to 44, wherein the quantization information further includes an additional indicator indicating an adjustment to the quantization offset, wherein the quantization offset is determined based on the indicator and the additional indicator.

46. ​​The method according to any one of claims 31 to 45, further comprising: An indication of whether to enable quantization adjustment is encoded in the bit stream, wherein the wavelet coefficients are adjusted based on the indication of enabling quantization adjustment.

47. The method of claim 46, wherein the set of vertices comprises a non-overlapping subset of vertices, and wherein the indication is received for each subset of the non-overlapping subset.

48. The method of claim 46, wherein the indication is encoded for a 3D mesh sequence including the 3D mesh.

49. The method of any one of claims 31 to 48, wherein the quantization offset is added to the quantized first wavelet coefficients to reconstruct the first displacement.

50. The method according to any one of claims 31 to 49, wherein the quantization offset is determined based on one or more thresholds.

51. An encoder comprising: One or more processors; as well as A memory that stores instructions, which, when executed by the one or more processors, cause the encoder to perform the method according to any one of claims 31 to 50.

52. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of the device, cause the device to perform the method according to any one of claims 31 to 50.

Images