Inverse quantization region adjustment for 3D mesh displacement

Abstract

The decoder decodes quantized wavelet coefficients representing the displacement of a vertex from a set of vertices of a three-dimensional (3D) mesh from a bitstream. The decoder further decodes quantization information representing multiple quantization offsets associated with a subset of vertices of the set of vertices from the bitstream, and determines wavelet coefficients by inversely quantizing the quantized wavelet coefficients. Based on the fact that a vertex is in a subset, the decoder determines the displacement of a vertex by adjusting the wavelet coefficients according to one of the multiple quantization offsets.

Description

Technology Field

[0001] Cross-reference regarding related applications

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 622,012 filed January 17, 2024, the entirety of which is incorporated herein by reference. Brief explanation of the drawing

[0003] Some examples of various embodiments of the present disclosure are described herein with reference to the drawings. FIG. 1 illustrates an exemplary mesh coding / decoding system in which an embodiment of the present disclosure can be implemented. FIG. 2a illustrates a block diagram of an exemplary encoder for intra-encoding a 3D mesh according to some embodiments. FIG. 2b illustrates a block diagram of an exemplary encoder for inter-encoding a 3D mesh according to some implementations. Figure 3 illustrates a drawing showing an exemplary decoder. FIG. 4 is a diagram showing an exemplary process for generating displacement of an input mesh (e.g., an input 3D mesh frame) to be encoded, according to some embodiments. FIG. 5 illustrates an exemplary process for approximating and encoding the geometry of a 3D mesh according to some embodiments. FIG. 6 illustrates an example of vertices of a fine mesh (e.g., fine base mesh) corresponding to a number of levels of detail (LOD) according to some embodiments. FIG. 7a illustrates an example of an image packed with displacement (e.g., a displacement field or vector) using a packing method according to some embodiments. FIG. 7b illustrates an example of a displacement image with LOD labeled according to some embodiments. FIG. 8 illustrates an example of a quantizer for quantizing transformed wavelet coefficients representing displacement of a 3D mesh, according to some embodiments. FIG. 9 illustrates an example of an inverse quantizer for inversely quantizing quantized transform wavelet coefficients representing displacement of a 3D mesh, according to some embodiments. FIG. 10a illustrates an example of an inverse quantizer having a quantization offset for inversely quantizing quantized transform wavelet coefficients representing displacement of a 3D mesh, according to some embodiments. FIG. 10b illustrates an exemplary drawing of an inverse quantizer in which a plurality of quantization offsets are applied to a vertex subset according to some embodiments. FIG. 10c illustrates a table showing exemplary quantization offsets for different intervals of possible values ​​of displacement for each LOD of a plurality of LODs, according to some embodiments. FIG. 11 illustrates an example of an inverse quantizer for iteratively inversely quantizing the quantized transform wavelet coefficients of a vertex in each vertex subset of a 3D mesh, according to some embodiments. FIG. 12 illustrates a flowchart of a method for applying a quantization scheme to transformed wavelet coefficients according to some embodiments. FIG. 13 illustrates a flowchart of a method for applying an inverse quantization scheme to quantized transform wavelet coefficients according to some embodiments. FIG. 14 illustrates a block diagram of an exemplary computer system in which an embodiment of the present disclosure can be implemented. Specific details for implementing the invention

[0004] In the following description, numerous specific details are presented to provide a complete understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure, including structures, systems, and methods, can be practiced without these specific details. The content described and expressed herein is the means generally used by those skilled in the art to most effectively convey their findings to others skilled in the art. In other cases, well-known methods, procedures, components, and circuits have not been described in detail to avoid unnecessarily obscuring aspects of the present disclosure.

[0005] References to “one embodiment,” “one embodiment,” “exemplary embodiment,” etc., in this specification indicate that while the described embodiment may include specific features, structures, or characteristics, not all embodiments are required to include specific features, structures, or characteristics. Furthermore, these phrases do not necessarily refer to the same embodiment. Additionally, when specific features, structures, or characteristics are described in relation to an embodiment, it is assumed that those skilled in the art know that such features, structures, or characteristics may also be applied in relation to other embodiments, regardless of whether they have been explicitly described.

[0006] Additionally, it should be noted that individual embodiments may be described as processes illustrated as flowcharts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. While flowcharts may describe operations as sequential processes, many operations may be performed in parallel or simultaneously. Furthermore, the order of operations may be rearranged. A process terminates when its operations are completed, but it may have additional steps not included in the drawings. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. If a process corresponds to a function, its termination may correspond to the function returning to the calling function or the main function.

[0007] The term “computer-readable medium” includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instruction(s) and / or data. A computer-readable medium may include non-transient media capable of storing data and not containing carrier waves and / or transient electronic signals propagated wirelessly or via a wired connection. Examples of non-transient media may include, but are not limited to, optical storage media such as magnetic disks or tapes, compact discs (CDs) or digital multi-purpose discs (DVDs), flash memory, memory, or memory devices. A computer-readable medium may store code and / or machine-executable instructions capable of representing any combination of procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Code segments may be coupled to other code segments or hardware circuits by transmitting and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be transmitted, conveyed, or sent through any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0008] Additionally, the embodiment may be implemented by hardware, software, firmware, middleware, microcode, a hardware description language, or any combination thereof. When implemented by software, firmware, middleware, or microcode, program code or code segments for performing the necessary task (e.g., a computer program product) may be stored on a computer-readable or machine-readable medium. Processor(s) may perform the necessary task.

[0009] Traditional visual data uses a series of pixels to describe an object or scene, and each pixel contains one or more optional attributes, such as position in two dimensions (x and y) and color. Volumetric visual data adds another positional dimension to this traditional visual data. Volumetric visual data uses a series of points to describe an object or scene, and each point contains one or more optional attributes, such as position in three dimensions (x, y, and z) and color. Compared to traditional visual data, volumetric visual data can provide a more realistic way to experience visual data. For example, an object or scene described by volumetric visual data can be viewed from any (or multiple) angle, whereas traditional visual data is generally only viewable from the angle from which it was captured or rendered. Volumetric visual data can be used in many applications, including Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR). Volumetric visual data may be in the form of volumetric frames describing an object or scene captured at a specific time instance, or in the form of a sequence of volumetric frames describing an object or scene captured at multiple different time instances (referred to as a volumetric sequence or volumetric video).

[0010] One format for storing volumetric visual data is a three-dimensional (3D) mesh (hereinafter referred to as mesh or mesh frame). A mesh frame (or mesh) contains a set of points in three-dimensional (3D) space, also referred to as vertices. Each vertex within the mesh contains geometric information representing the position of the vertex in 3D space. For example, the geometric information may represent the position of the vertex in 3D space using three Cartesian coordinates (x, y, and z). Additionally, the mesh may contain geometric information representing multiple triangles. Each triangle includes three vertices connected by three edges and faces. For each face (of a triangle), one or more types of attribute information may be stored. The attribute information may represent characteristics of the visual appearance of the face. For example, attribute information may represent the surface texture (e.g., color), the surface material type, the surface transparency information, the surface reflectance information, the surface normal vector, velocity on the surface, acceleration on the surface, a timestamp indicating when the surface (and / or vertex) was captured, or a modality indicating how the surface (and / or vertex) was captured (e.g., running, walking, or flying). In other examples, the surface (or vertex) may include light field data in the form of multiple viewpoint-dependent texture information. Light field data may be other types of optional attribute information.

[0011] Triangles within a mesh (e.g., represented by vertices and edges) can describe an object or scene. For example, triangles within a mesh can describe the external surfaces and / or internal structures of an object or scene. An object or scene can be synthetically generated by a computer or generated from a capture of a real-world object or scene. Geometric information of a real-world object or scene can be obtained by 3D scanning and / or photogrammetry. 3D scanning may include laser scanning, structured light scanning, and / or modulated light scanning. 3D scanning can obtain geometric information by moving one or more laser heads, structured light cameras, and / or modulated light cameras over the object or scene being scanned. Photogrammetry can obtain geometric information by triangulating the same features or points in different 2D photographs that have been moved spatially. Mesh data may be in the form of mesh frames describing an object or scene captured at a specific time instance, or in the form of a sequence of mesh frames describing an object or scene captured at multiple different time instances (referred to as a mesh sequence or mesh video).

[0012] The data size of a mesh frame or sequence, along with one or more types of attribute information, can be too large for storage and / or transmission in many applications. For example, a single mesh frame may contain thousands, tens of thousands, or hundreds of thousands of triangles, where each triangle (e.g., a vertex and / or edge) contains geometry information and one or more optional types of attribute information. The geometry information for each vertex may include three Cartesian coordinates (x, y, and z), each represented using, for example, a total of 8 bits or 24 bits. The attribute information for each point may include textures corresponding to three color components (e.g., R, G, and B color components), each represented using, for example, a total of 8 bits or 24 bits. Thus, in this example, a single vertex contains 48 bits of information, consisting of 24 bits of geometry information and 24 bits of texture. Encoding can be used to compress the size of the mesh frame or sequence to provide more efficient storage and / or transmission. Compressed mesh frames or sequences for display and / or other forms of consumption can be decompressed using decoding (e.g., by machine learning-based devices, neural network-based devices, artificial intelligence-based devices, or other types of machine-based processing algorithms and / or other forms of consumption by devices).

[0013] Mesh compression can be lossy (e.g., introducing differences from the original data) to enable distribution to end users on, for example, AR / VR glasses or any other 3D-enabled device, and visualization by the end user. Lossy compression allows for very high compression ratios but entails a trade-off between the compression perceived by the end user and the visual quality. Other frameworks, such as medical or geological applications, may require lossless compression to avoid altering the decompressed mesh.

[0014] Volumetric visual data can be stored in a container (e.g., a file server within a network) after being encoded into a bitstream. End users can request specific bitstreams based on their requirements. Users can also request adaptive streaming of the bitstream, where the trade-off between network resource consumption and the visual quality perceived by the end user is taken into account by an algorithm.

[0015] FIG. 1 illustrates an exemplary mesh coding / decoding system (100) in which an embodiment of the present disclosure may be implemented. The mesh coding / decoding system (100) includes a source device (102), a transmission medium (104), and a destination device (106). The source device (102) encodes a mesh sequence (108) into a bitstream (110) for more efficient storage and / or transmission. The source device (102) may store and / or transmit the bitstream (110) to the destination device (106) via the transmission medium (104). The destination device (106) decodes the bitstream (110) for displaying the mesh sequence (108) or for other forms of consumption. The destination device (106) may receive the bitstream (110) from the source device (102) via the storage medium or the transmission medium (104). The source device (102) and the destination device (106) may be any one of a number of different devices, including a cluster of interconnected computer systems operating as a pool of seamless resources (also referred to as a cloud of computers or a cloud computer), a server, a desktop computer, a laptop computer, a tablet computer, a smartphone, a wearable device, a television, a camera, a video gaming console, a set-top box, a video streaming device, an autonomous vehicle, or a head-mounted display. The head-mounted display enables a user to view a VR, AR, or MR scene and adjust the view of the scene based on the user's head movements. The head-mounted display may be tethered to a processing unit (e.g., a server, a desktop computer, a set-top box, or a video game counselor) or may be completely standalone.

[0016] To encode a mesh sequence (108) into a bitstream (110), the source device (102) may include a mesh source (112), an encoder (114), and an output interface (116). The mesh source (112) may provide or generate a mesh sequence (108) from a capture of a natural scene or a synthetically generated scene. A synthetically generated scene may be a scene containing computer-generated graphics. The mesh source (112) may include one or more mesh capture devices (e.g., one or more laser scanning devices, structured light scanning devices, modulated light scanning devices, and / or passive scanning devices), a mesh archive containing previously captured natural scenes and / or synthetically generated scenes, a mesh feed interface receiving captured natural scenes and / or synthetically generated scenes from a mesh content provider, and / or a processor generating synthetic mesh scenes.

[0017] As illustrated in FIG. 1, a mesh sequence (108) may include a series of mesh frames (124). A mesh frame describes an object or scene captured at a specific time instance. The mesh sequence (108) can achieve a moving impression when a constant or variable time is used to present the mesh frames (124) of the mesh sequence (108) in succession. A (3D) mesh frame includes a set of vertices (126) and geometry information of the vertices (126) in 3D space. The 3D mesh may include a set of vertices, edges, and faces that define the shape of a polyhedral object. Additionally, a mesh frame may include a plurality of triangles (e.g., polygonal triangles). For example, a triangle may include vertices (134A–C), edges (136A–C), and faces (132). Faces are typically composed of triangles (triangle meshes), quadrilaterals (quads), or other simple convex polygons (n-gons) to simplify rendering, but more generally, they may be composed of concave polygons or even polygons with holes. Each vertex (126) may contain geometry information representing the location of a point in 3D space. For example, geometry information may represent the location of a point in 3D space using three orthogonal coordinates (x, y, and z). For example, geometry information may represent multiple triangles, each containing three vertices (126). One or more of the triangles may additionally contain one or more types of attribute information. Attribute information may represent characteristics of the visual appearance of the point. For example, attribute information may represent the surface texture (e.g., color), the surface material type, the surface transparency information, the surface reflectance information, the surface normal vector, the velocity on the surface, the acceleration on the surface, a timestamp indicating when the surface was captured, and the modality indicating when the surface was captured (e.g., running, walking, or flying).In another example, one or more of the faces (or triangles) may contain light field data in the form of multiple viewpoint-dependent texture information. The light field data may be other types of optional attribute information. Color attribute information of one or more of the faces may include a luminance value and two chrominance values. The luminance value may represent the brightness of the point (or luminance component, Y). The chrominance values ​​may represent the blue and red components of the point (or chroma components, Cb and Cr), respectively, separate from the brightness. Different color attribute values ​​are possible based on different color schemes (e.g., RGB or monochromatic color schemes).

[0018] In some embodiments, the 3D mesh (e.g., one of the mesh frames (124)) may be a static or dynamic mesh. In some embodiments, the 3D mesh may be represented (e.g., defined) by connectivity information, geometry information, and texture information (e.g., texture coordinates and texture connectivity). In some embodiments, geometry information may represent the location of a vertex of the 3D mesh in 3D space, and connectivity information may represent how vertices are connected together to form a polygon (e.g., a triangle) that constitutes the 3D mesh. Additionally, texture coordinates represent the location of a pixel in a 2D image corresponding to a vertex of the 3D mesh (or a sub-mesh of the 3D mesh). In some embodiments, patch information may represent how texture coordinates defined for a 2D bounding box are mapped into the 3D space of a 3D bounding box associated with the patch, based on how the point is projected onto a projection plane for the patch. Additionally, texture connectivity information may indicate how vertices represented by texture coordinates are connected together to form polygons of a 3D mesh (or sub-mesh). For example, each texture or attribute patch of a texture image may correspond to a corresponding sub-mesh defined using texture coordinates and texture connectivity.

[0019] In some embodiments, for each 3D mesh, one or more 2D images may represent textures or attributes associated with the mesh. For example, texture information may include geometry information enumerated as the X, Y, and Z coordinates of a vertex and texture coordinates enumerated as 2D dimension coordinates corresponding to a vertex. An exemplary texture mesh may include texture connectivity information representing a mapping between geometry coordinates and texture coordinates to form a polygon such as a triangle. For example, a first triangle may be formed by three vertices, wherein the first vertex is defined as first geometry coordinates (e.g., 64.062500, 1237.739990, 51.757801) corresponding to first texture coordinates (e.g., 0.0897381, 0.740830). The second vertex of a triangle can be defined as the second geometry coordinates (e.g., 59.570301, 1236.819946, 54.899700) corresponding to the second texture coordinates (e.g., 0.899059, 0.741542). Finally, the third vertex of a triangle may correspond to the third enumerated geometry coordinates that match the third enumerated texture coordinates. However, note that in some cases, the vertices of a polygon such as a triangle may be mapped to sets of geometry coordinates and texture coordinates that may have different index positions in their respective lists of geometry coordinates and texture coordinates. For example, the second triangle has a first vertex corresponding to the fourth enumerated geometry coordinate set and the seventh enumerated texture coordinate set. The second vertex corresponds to the first enumerated geometry coordinate set and the first enumerated texture coordinate set, and the third vertex corresponds to the third enumerated geometry coordinate set and the ninth enumerated texture coordinate set.

[0020] The encoder (114) can encode the mesh sequence (108) into a bitstream (110). To encode the mesh sequence (108), the encoder (114) can apply one or more prediction techniques to reduce redundant information in the mesh sequence (108). Redundant information is information that can be predicted by the decoder and therefore does not need to be transmitted to the decoder for accurate decoding of the mesh sequence (108). For example, the encoder (114) can convert one or more attribute information (e.g., texture information) of the mesh frames (124) from 3D to 2D and then apply one or more 2D video encoders or encoding methods to the 2D image. For example, any one of a number of different proprietary or standardized 2D video encoders / decoders may be used, including ITU-T H.1263, ITU-T H.1264 and MPEG-4 Visual (also known as Advanced Video Coding (AVC)), ITU-T H.1265 and MPEG-H Part 2 (also known as High Efficiency Video Coding (HEVC)), ITU-T H.1265 and MPEG-I Part 3 (also known as Universal Video Coding (VVC)), WebM VP8 and VP9 codecs, and AOMedia Video 1 (AV1). The encoder (114) may encode the geometry of the mesh sequence (108) based on Video Dynamic Mesh Coding (V-DMC). V-DMC specifies encoded bitstream syntax and semantics for transmitting or storing a mesh sequence, and decoder operations for reconstructing a mesh sequence from a bitstream.

[0021] The output interface (116) may be configured to write and / or store the bitstream (110) to the transmission medium (104) for transmission to the destination device (106). Additionally or alternatively, the output interface (116) may be configured to transmit, upload, and / or stream the bitstream (110) to the destination device (106) via the transmission medium (104). The output interface (116) may include wired and / or wireless transmitters configured to transmit, upload, and / or stream the bitstream (110) according to one or more proprietary and / or standardized communication protocols (e.g., Digital Video Broadcasting (DVB) standards, Advanced Television Systems Council (ATSC) standards, Integrated Services Digital Broadcasting (ISDB) standards, Data over Cable Service Interface Specification (DOCSIS) standards, Third Generation Partnership Project (3GPP) standards, Institute of Electrical and Electronics Engineers (IEEE) standards, Internet Protocol (IP) standards, Wireless Application Protocol (WAP) standards, and / or any other communication protocol).

[0022] The transmission medium (104) may include wireless, wired, and / or computer-readable media. For example, the transmission medium (104) may include one or more wires, cables, wireless interfaces, optical discs, flash memory, and / or magnetic memory. Additionally or alternatively, the transmission medium (104) may include one or more networks (e.g., the Internet) or file servers configured to store and / or transmit encoded video data.

[0023] To decode the bitstream (110) into a mesh sequence (108) for display or other forms of consumption, the destination device (106) may include an input interface (118), a decoder (120), and a mesh display (122). The input interface (118) may be configured to read the bitstream (110) stored in the transmission medium (104) by the source device (102). Additionally or alternatively, the input interface (118) may be configured to receive, download, and / or stream the bitstream (110) from the source device (102) through the transmission medium (104). The input interface (118) may include wired and / or wireless receivers configured to receive, download, and / or stream the bitstream (110) according to one or more proprietary and / or standardized communication protocols, such as those referenced herein.

[0024] The decoder (120) can decode a mesh sequence (108) from an encoded bitstream (110). To decode attribute information (e.g., texture) of the mesh sequence (108), the decoder (120) can reconstruct a compressed 2D image using one or more 2D video encoders. Then, the decoder (120) can reconstruct attribute information of a 3D mesh frame (124) from the reconstructed 2D image. In some examples, the decoder (120) can decode a mesh sequence that approximates the mesh sequence (108), for example, due to lossy compression of the mesh sequence (108) by the encoder (114) and / or errors introduced into the encoded bitstream (110) during transmission to the destination device (106). Additionally, the decoder (120) can decode the geometry of the mesh sequence (108) from the encoded bitstream (110) as described further below. Then, one or more of the decoded attribute information can be applied to the decoded mesh frame of the mesh sequence (108).

[0025] The mesh display (122) can display the mesh sequence (108) to the user. The mesh display (122) may include a cathode ray tube (CRT) display, a liquid crystal display (LCD), a plasma display, a light-emitting diode (LED) display, a 3D display, a holographic display, a head-mounted display (HMD), or any other display device suitable for displaying the mesh sequence (108).

[0026] It should be noted that the mesh coding / decoding system (100) is presented as an example and is not limited to. In the example of FIG. 1, the mesh coding / decoding system (100) may include other components and / or arrays. For example, the mesh source (112) may be located outside the source device (102). Similarly, if the mesh sequence is intended to be consumed by a machine and / or storage device, the mesh display (122) may be located outside the destination device (106) or may be omitted entirely. In another example, the source device (102) may additionally include a mesh decoder, and the destination device (106) may include a mesh encoder. In this example, the source device (102) may be configured to further receive an encoded bitstream from the destination device (106) to support bidirectional mesh transmission between the devices.

[0027] FIG. 2a illustrates a block diagram of an exemplary encoder (200A) for intra-encoding a 3D mesh according to some embodiments. For example, an encoder (e.g., encoder (114)) may include the encoder (200A).

[0028] In some examples, a mesh sequence (e.g., mesh sequence (108)) may include a set of mesh frames (e.g., mesh frames (124)) that can be individually encoded and decoded. As described below in relation to FIG. 4, a base mesh (252) may be determined (e.g., generated) from a mesh frame (e.g., input mesh) through a decimation process. In the decimation process, the mesh topology of the mesh frame may be reduced to determine a base mesh (e.g., decimated mesh or decimated base mesh). A mesh encoder (204) may generate a base mesh bitstream (254) by encoding the base mesh (252), the geometry information (e.g., vertices) of which can be quantized by a quantizer (202). In some examples, the base mesh encoder (204) may be a conventional encoder such as Draco or Edgebreaker.

[0029] The displacement generator (208) can generate displacements for vertices of a mesh frame based on a base mesh (252), as described below in relation to FIGS. 4 and 5. In some embodiments, the displacement is determined based on a reconstructed base mesh (256). The reconstructed base mesh (256) may be determined (e.g., output or generated) by a mesh decoder (206) that decodes an encoded base mesh determined (e.g., output or generated) by a mesh encoder (204) (e.g., in a base mesh bitstream (254)). The displacement generator (208) may subdivide the reconstructed base mesh (256) using a subdivision method (e.g., a subdivision algorithm) to determine a subdivided mesh (e.g., a subdivided base mesh). The displacement (258) may be determined based on fitting the subdivided mesh to the surface of the original input mesh. For example, the displacement (258) for a vertex within a mesh frame may include displacement information (e.g., a displacement vector) representing the displacement from the position of the corresponding vertex within the subdivided mesh to the position of the vertex within the mesh frame.

[0030] The displacement (258) can be transformed by a wavelet converter (210) to generate wavelet coefficients (e.g., transform coefficients) that represent displacement information and can be encoded (and subsequently decoded) more efficiently. The wavelet coefficients can be quantized by a quantizer (212) and packed (e.g., array) by an image packer (214) into a picture (e.g., one or more images or picture frames) to be encoded by a video encoder (216). The mux (218) can combine (e.g., multiplex) the displacement bitstream (260) output by the video encoder (216) with the base mesh bitstream (254) to form a bitstream (266).

[0031] The attribute information (262) of the mesh frame (e.g., color, texture, etc.) may be encoded separately from the geometry information of the mesh frame described above. In some examples, the attribute information (262) of the mesh frame may be represented (e.g., stored) by an attribute map (e.g., texture map) that associates each vertex of the mesh frame with the attribute information corresponding to that vertex. The attribute transmission (232) may reparameterize the attribute information (262) within the attribute map based on the reconstructed mesh determined (e.g., generated or output) from the mesh reconstruction component (225). The mesh reconstruction component (225) performs an inverse or decoding function and may be the same or similar component within the decoder (e.g., the decoder (300) of FIG. 3). For example, the inverse quantizer (228) may determine (e.g., generated or output) the reconstructed base mesh (268) by inverse quantizing the reconstructed base mesh (256). The video decoder (226), image unpacker (224), inverse quantizer (222), and inverse wavelet converter (220) can each perform the inverse functions of the video encoder (216), image packer (214), quantizer (212), and wavelet converter (210). Thus, the reconstructed displacement (270) corresponding to the displacement (258) can be generated by applying the video decoder (226), image unpacker (224), inverse quantizer (222), and inverse wavelet converter (220) in that order. The modified mesh reconstructor (230) can determine the reconstructed mesh corresponding to the input mesh frame based on the reconstructed base mesh (268) and the reconstructed displacement (270). In some examples, the reconstructed mesh may be the same decoded mesh determined from the decoder based on the decoding base mesh bitstream (254) and the displacement bitstream (260).

[0032] The attribute information of the reparameterized attribute map can be packed into an image (e.g., a 2D image or a picture frame) by a padding component (234). The padding component (234) can fill (e.g., pad) parts of the image that do not contain attribute information. In some examples, the color space converter (236) can convert (e.g., convert) a color representation (e.g., an example of attribute information (262)) from a first format to a second format (e.g., from RGB444 to YUV420) to achieve improved rate-distortion (RD) performance when encoding the attribute map. In one example, the color space converter (236) can also perform chroma subsampling to further increase encoding performance. Finally, the video encoder (240) encodes an image (e.g., picture frame) representing the attribute information (262) of a mesh frame and determines (e.g., generates or outputs) an attribute bitstream (264) that is multiplexed into a bitstream (266) by the mux (218). In some examples, the video encoder (240) may be a conventional 2D video compression encoder such as an HEVC encoder or a VVC encoder.

[0033] FIG. 2b illustrates a block diagram of an exemplary encoder (200B) for inter-encoding a 3D mesh according to some embodiments. For example, an encoder (e.g., encoder (114)) may include encoder (200B). As illustrated in FIG. 2b, encoder (200B) includes many of the same components as encoder (200A). In contrast to encoder (200A), encoder (200B) does not include a mesh encoder (204) and a mesh decoder (206) corresponding to a coder for a static 3D mesh. Instead, encoder (200B) includes a motion encoder (242), a motion decoder (244), and a base mesh reconstructor (246). When the motion encoder (242) is applied to the reconstructed quantized reference base mesh (243), it can determine the motion field (e.g., one or more motion vectors (MV)) that is the closest base mesh (252).

[0034] The determined motion field can be encoded into the bitstream (266) as a motion bitstream (272). In some examples, the motion field (e.g., motion vectors in the x, y, and z directions) can be entropily coded (e.g., for each directional component) as a codeword derived from a coding method such as a unary code, a Colomb code (e.g., exponential Colomb code), a Rice code, or a combination thereof. In some examples, the codeword can be arithmetically coded using, for example, CABAC. The prefix portion of the codeword can be context-coded, and the coded suffix portion can be bypass-coded. In some examples, the sign bit for each directional component of the motion vector can be coded separately.

[0035] In some examples, the motion bitstream (272) may additionally include a representation of a selected reconstructed quantized reference base mesh (243).

[0036] In some examples, the motion bitstream (272) may be decoded by a motion decoder (244) and used by a base mesh reconstructor (246) to generate a reconstructed quantized base mesh (256). For example, the base mesh reconstructor (246) may determine (e.g. generate) the reconstructed quantized base mesh (256) by applying the decoded motion field to a reconstructed quantized reference base mesh (243).

[0037] In some examples, a reconstructed quantized reference base mesh m'(j) associated with a reference mesh frame having index j can be used to predict a base mesh m(i) associated with a current frame having index i. Base meshes m(i) and m(j) may have the same number of vertices, connectivity, texture coordinates, and texture connectivity. Vertex positions may differ between base meshes m(i) and m(j).

[0038] In some examples, the motion field f(i) can be computed by considering the quantized version of m(i) and the reconstructed quantized base mesh m'(j). The base mesh m'(j) may have a different number of vertices than m(j) (e.g., vertices may have been merged or removed). Therefore, the encoder can determine (e.g., generate or acquire) m'(j) by tracking the transformations applied to m(j) and apply it to m(i). These transformations can enable a one-to-one correspondence between the vertices of the base mesh m'(j) and the transformed and quantized version of the base mesh m(i), denoted as m^*(i). The motion field f(i) can be computed by subtracting the quantized position Pos(i,v)(i) of vertex v in m^* from the position Pos(j,v) of vertex v in m'(j) as follows: f(i,v) = Pos(j,v) - Pos(i,v). The motion field can be further predicted by using connectivity information of the base mesh m'(j), and the prediction residuals can be entropy encoded.

[0039] In some examples, since the motion field compression process may be lossy, the reconstructed motion field denoted by f'(i) can be computed by applying a motion decoder component. Then, the reconstructed quantized base mesh m'(i) can be computed by adding the motion field to the vertex locations within the base mesh m'(j). To better utilize temporal correlations in displacement and attribute map videos, inter-prediction can be enabled in the video encoder.

[0040] In some embodiments, the encoder (e.g., encoder (114)) may include encoder (200A) and encoder (200B).

[0041] FIG. 3 illustrates a diagram showing an exemplary decoder (300). A bitstream (330), which may correspond to the bitstream (266) of FIG. 2a and FIG. 2b and may be received as a binary file, may be demultiplexed by a demultiplexer (302) to separate the bitstream (330) into a base mesh bitstream (332), a displacement bitstream (334), and an attribute bitstream (336), each carrying base mesh geometry information, displacement geometry information, and attribute information, respectively. The attribute bitstream (336) may include one or more attribute map substreams for each attribute type.

[0042] In some examples, for inter-decoding, the bitstream is demultiplexed into distinct substreams including a motion substream, a position and potentially a displacement substream for each vertex attribute, zero or more attribute map substreams, and an atlas substream containing patch information in the same way as in V3C / V-PCC.

[0043] In some examples, the base mesh bitstream (332) may be decoded in intra mode or inter mode. In intra mode, the static mesh decoder (320) may decode the base mesh bitstream (332) (e.g., to generate a reconstructed base mesh m'(i)) and then dequantize the decoded base mesh (340) (e.g., a reconstructed quantized base mesh m(i)) by the dequantizer (318) to determine (e.g., generate or output). In some examples, the static mesh decoder 320 may correspond to the mesh decoder (206) of FIG. 2a.

[0044] In some examples, in inter mode, the base mesh bitstream (332) may contain motion field information that is decoded by the motion decoder (324). In some examples, the motion decoder (324) may correspond to the motion decoder (244) of FIG. 2B. For example, the motion decoder (324) may entropy decode the base mesh bitstream (332) to determine the motion field information. In inter mode, the base mesh bitstream (332) may represent a previous base mesh (e.g., reference base mesh m'(j)) that is decoded by the static mesh decoder (320) and stored (e.g., buffered) in the mesh buffer (322). A base mesh reconstructor (326) can generate a quantized reconstructed base mesh m'(i) by applying the decoded motion field (output by the motion decoder (324)) to a previously decoded (e.g., reconstructed) base mesh m'(j) stored in the mesh buffer (322). In some examples, the base mesh reconstructor (326) may correspond to the base mesh reconstructor (246) of FIG. 2B. The quantized reconstructed base mesh can be dequantized by the dequantizer (318) to determine (e.g., generate or output) the decoded base mesh (340) (e.g., reconstructed base mesh m(i)). In some examples, the decoded base mesh (340) may be identical to the reconstructed base mesh (268) of FIG. 2A and FIG. 2B.

[0045] In some examples, the decoder (300) includes a video decoder (308), an image unpacker (310), an inverse quantizer, and an inverse wavelet converter (314) that determine (e.g., generate) a displacement (338) decoded from a displacement bitstream (334). The video decoder (308), image unpacker (310), inverse quantizer, and inverse wavelet converter (314) correspond to the video decoder (226), image unpacker (224), inverse quantizer (222), and inverse wavelet converter (220), respectively, and perform the same or similar operations. For example, a picture frame (e.g., an image) received in a displacement bitstream (334) can be decoded by a video decoder (308), and the displacement information can be unpacked from the decoded image by an image unpacker (310) and dequantized by an inverse quantizer (312) to determine inverse quantized wavelet coefficients representing the encoded displacement information. Then, the dequantized wavelet coefficients can be inversely transformed by an inverse wavelet converter (314) to determine the decoded displacement d(i). In other words, the decoded displacement (338) (e.g., decoded displacement field d(i)) can be identical to the reconstructed displacement (270) of FIGS. 2a and FIGS. 2b.

[0046] The modified mesh reconstructor (316) corresponding to the modified mesh reconstructor (230) can determine (e.g., generate or output) the decoded mesh (342) (M(i)) based on the decoded displacement (338) and the decoded base mesh (340). For example, the modified mesh reconstructor (316) can determine the decoded mesh (342) by combining (e.g., adding) the decoded displacement (338) with the subdivided decoded mesh (340).

[0047] In some examples, the decoder (300) includes a video decoder (304) that determines determined attribute information (344) (e.g., decoded attribute information or reconstructed attribute information) by decoding an attribute bitstream (336) containing encoded attribute information represented (e.g., stored) in a 2D image (or picture frame). In some examples, the video decoder (304) may be a conventional 2D video compression decoder such as an HEVC decoder or a VVC decoder. The decoder (300) may include a color space converter (306) which can reverse the color format conversion performed by the color space converter (236) of FIGS. 2a and FIGS. 2b.

[0048] FIG. 4 is a drawing (400) illustrating an exemplary process (e.g., preprocessing operation) for generating a displacement (414) of an input mesh (430) (e.g., input 3D mesh frame) to be encoded, according to some embodiments. In some examples, the displacement (414) may correspond to the displacement (258) illustrated in FIG. 2a and FIG. 2b.

[0049] In drawing (400), the mesh decimator (402) determines (e.g., generates or outputs) an initial base mesh (432) based on (e.g., using) the input mesh (430). In some examples, the initial base mesh (432) may be determined (e.g., generated) from the input mesh (432) through a decimation process. In the decimation process, the mesh topology of the mesh frame may be reduced to determine the initial base mesh (which may be referred to as the decimated mesh or the decimated base mesh). As illustrated in FIG. 5, the decimation process may include a downsampling process to remove vertices from the input mesh (432) so that a small portion (e.g., 6% or less) of the vertices within the input mesh (430) may remain in the initial base mesh (432).

[0050] The mesh subdivider (404) generates an initial subdivided mesh (434) by applying a subdivision method. As discussed in more detail in relation to FIG. 5, the subdivision method may include upsampling the initial base mesh (432) to generate an initial subdivided mesh (434) by adding more vertices to the 3D mesh based on the topology and geometry of the original mesh.

[0051] The fitting component (406) can fit the initial subdivided mesh to determine a modified mesh (436) that can more closely approximate the surface of the input mesh (430). As discussed in more detail in relation to FIG. 5, fitting can be performed by moving the vertices of the initial subdivided mesh (434) toward the surface of the input mesh (430) so that the subdivided mesh (434) can be used to approximate the input mesh (430). In some embodiments, fitting is performed by moving each vertex of the initial subdivided mesh (434) along the vertical direction of the vertex until the vertex intersects the surface of the input mesh (430). The resulting mesh is the modified mesh (436). The normal direction can be indicated by the vertex normal at the vertex, which can be obtained from the face normal of the triangle formed by the vertex.

[0052] The base mesh generator (408) may perform other fitting processes to generate a base mesh (438) from an initial base mesh (432). For example, the base mesh generator (408) may deform the initial base mesh (432) according to the deformed mesh (436) so that the initial base mesh (432) becomes closer to the deformed mesh (436). In some embodiments, the fitting process may be performed in a manner similar to the fitting component (406). For example, the base mesh generator (408) may move each vertex within the initial base mesh (432) along its normal direction (e.g., based on the vertex normal at each vertex) until the vertex reaches the surface of the deformed mesh (436). The output of this process is the base mesh (438).

[0053] The base mesh (438) can be output to a mesh reconstruction process (410) to generate a reconstructed base mesh (440). The reconstructed base mesh (440) can be subdivided by a mesh subdivider (418), and the subdivided mesh (442) can be input to a displacement generator (420) to generate (e.g., determine or output) a displacement (414) as further described below in relation to FIG. 5. In some examples, the mesh subdivider (418) can apply the same subdivision method as applied by the mesh subdivider (404). In these embodiments, the vertices in the subdivided mesh (442) correspond one-to-one with the vertices in the deformed mesh (436). Thus, the displacement generator (420) can generate the displacement (414) by calculating the difference between each vertex of the subdivided mesh (442) and the corresponding vertex of the deformed mesh (436). In some embodiments, the difference may be projected in the normal direction of the associated vertex, and the generated vector is the displacement (414). In this way, only the sign and magnitude of the displacement (414) need to be encoded in the bitstream, thereby increasing coding efficiency. Additionally, since the base mesh (438) is fitted toward the deformed mesh (436), the displacement (414) between the deformed mesh (436) and the subdivided mesh (442) (generated from the reconstructed base mesh (440)) will have a small magnitude, which further reduces the payload and increases coding efficiency.

[0054] In some embodiments, one advantage of applying a subdivision process is that it allows for more efficient compression while providing a faithful approximation of the original input mesh (430) (e.g., the surface or curve of the original input mesh (430)). Compression efficiency can be achieved because the base mesh (e.g., the decimated mesh) requires encoding and transmitting fewer bits since it has fewer vertices compared to the number of vertices of the input mesh (430). Additionally, if the base mesh is decoded without any arbitrary information required from the encoder other than the subdivision method (e.g., subdivision algorithm) and parameters for subdivision (e.g., subdivision iteration count), the subdivided mesh can be automatically generated by the decoder. The reconstructed mesh can be determined by decoding displacement information (e.g., displacement vector) associated with the vertices of the subdivided mesh (e.g., the subdivided curve / surface of the base mesh). The segmentation process not only allows for spatial and quality scalability, but displacements can also be efficiently coded using wavelet transforms (e.g., wavelet decomposition), which further increases compression performance.

[0055] In some embodiments, the mesh reconstruction process (410) includes a component for encoding and then decoding the base mesh (438). FIG. 4 shows an example for an intra mode, where the mesh reconstruction process (410) may include a quantizer (411), a static mesh encoder (412), a static mesh decoder (413), and an inverse quantizer (416), which may each perform the same or similar operations as the quantizer (202), mesh encoder (204), mesh decoder (206), and inverse quantizer (228) of FIG. 2a. In an inter mode, the mesh reconstruction process (410) may include a quantizer (202), a motion encoder (242), a motion decoder (244), a base mesh reconstruction unit (246), and an inverse quantizer (228).

[0056] FIG. 5 illustrates an exemplary process for approximating and encoding the geometry of a 3D mesh according to some embodiments. For exemplary purposes, the 3D mesh is illustrated as a 2D curve. The original surface (510) of the 3D mesh (e.g., mesh frame) includes neighboring vertices (e.g., points) and edges connecting neighboring vertices. For example, points (512) and (513) are connected by edges corresponding to the surface (514).

[0057] In some examples, a decimation process (e.g., a downsampling process or a decimation / downsampling method) may be applied to the original surface (510) of the original mesh to create a downsampled surface (520) of the decimated (or downsampled) mesh. In the context of mesh compression, decimation refers to a process that reduces the number of vertices within a mesh while preserving its overall geometry and topology. For example, the original mesh surface (510) is decimated into a surface (520) having fewer samples (e.g., vertices and edges), but still retains the main features and geometry of the original mesh surface (510). This downsampled surface (520) may correspond to the surface of the base mesh (e.g., the decimated mesh).

[0058] In some embodiments, after the decimation process, a subdivision process (e.g., a subdivision method or a subdivision algorithm) may be applied to the downsampled surface (520) to generate an upsampled surface (530) having more samples (e.g., vertices and edges). The upsampled surface (530) may be part of a subdivided mesh (e.g., a subdivided base mesh) generated by subdividing the downsampled surface (520) corresponding to the base mesh.

[0059] Subdivision is a process commonly used after decimation in mesh compression to improve the visual quality of the compressed mesh. The subdivision process involves adding new vertices and faces to the mesh based on the topology and geometry of the original mesh. In some examples, the subdivision process begins by taking the reduced mesh generated by the decimation process and iteratively adding new vertices and edges. For instance, the subdivision process may include the step of splitting each edge (or face) of the reduced / decimated mesh into shorter edges (or smaller faces) and creating new vertices at the split points. These new vertices are then connected to form new faces (e.g., triangles, squares, or other polygons). By applying subdivision after the decimation process, a higher level of compression can be achieved without significant loss of visual fidelity. For instance, various subdivision methods may be used, such as midpoint, Catmull-Clark subdivision, Butterfly subdivision, Loop subdivision, or combinations thereof.

[0060] For example, Fig. 5 illustrates an example of a midpoint subdivision method. In this method, each subdivision iteration subdivides each triangle into four sub-triangles. A new vertex is introduced at the midpoint of each edge. The subdivision process can be applied independently of geometry and texture coordinates because the connections for geometry and texture coordinates are generally different. The subdivision method is as follows: the first vertex ( ) and the second vertex ( Edge formed by ) A newly introduced vertex at the center or middle of The location of Calculate:

[0061]

[0062] Here and is a vertex and It is the location. In some examples, the same process can be used to calculate the texture coordinates of newly generated vertices. For normal vectors, the normalization step can be applied as follows:

[0063]

[0064] , , and are each vertex , , and It is a normal vector associated with. is a vector It is norm2 of.

[0065] As shown in the up-sampled surface (530), using an intermediate point subdivision method, a point (531) can be generated as an intermediate point of the edge (522), which is the edge connection point (532) and point (533). The point (531) can be added as a new vertex. Edges (534) and (542) are also added to connect the added new vertex corresponding to the point (531). In some examples, the original edge (522) can be replaced with two new edges (534 and 542).

[0066] In some embodiments, a downsampled surface (520) may be repeatedly subdivided to produce an upsampled surface (530). For example, a first subdivided mesh produced from a first iteration of subdivision applied to the downsampled surface (520) may be further subdivided according to a subdivision method to produce a second subdivided mesh, etc. In some examples, the number of iterations corresponding to the level of subdivision may be predetermined. In other examples, the encoder may indicate the number of iterations for the decoder, which can similarly produce a subdivided mesh as described above.

[0067] In some embodiments, the subdivided mesh may be deformed (e.g., approximated) toward the original mesh to determine (e.g., obtained) the prediction of the original mesh having the original surface (510). A point on the subdivided mesh may be moved along a calculated vertical orientation until it reaches the original surface (510) of the original mesh. The distance between the intersection point on the original surface (510) and the point subdivided may be calculated as a displacement (e.g., a displacement vector). For example, a point (531) may be moved toward the original surface (510) along the calculated vertical orientation of the surface (e.g., indicated by an edge (542)). When the point (531) intersects the surface (514) of the original surface (510) (of the original / input mesh), a displacement vector (548) may be calculated. The displacement vector (548) applied to the point (531) may result in a displaced surface (540) that is closer to the original surface (510). In some examples, displacement information (e.g., displacement vector (548)) for a vertex of the subdivided mesh (e.g., upsampled surface (530) of the subdivided mesh) may be encoded and transmitted as a displacement bitstream (260) shown in the exemplary encoder of FIGS. 2a and 2b. For reference, as described in relation to FIG. 4, the subdivided mesh corresponding to the upsampled surface may be a subdivided mesh (442) compared with a deformed mesh (436) representing the original surface (510) of the input mesh.

[0068] In some implementations, the displacement d(i) (e.g., a displacement field or a displacement vector) may be computed and / or stored based on local or global coordinates. For example, a global coordinate system is a reference system used to define the position and orientation of an object or point in 3D space. It provides a fixed reference frame independent of the object or point being described. The origin of the global coordinate system may be defined as the point where the three axes intersect. Any point in 3D space may be positioned relative to the origin by specifying its location along the three axes using Cartesian coordinates (x, y, z). For example, the displacement may be defined in the same Cartesian coordinate system as the input or original mesh. Thus, the displacement may include three components (in the x, y, and z directions).

[0069] In a local coordinate system, (mutually perpendicular) normal, tangent, and / or binormal vectors can be determined to define a local basis for representing the orientation and position of an object in space relative to a reference frame. In some examples, the displacement field d(i) can be transformed from a canonical coordinate system to a local coordinate system, for example, defined by the normal to the mesh subdivided at each vertex (e.g., generally referred to as the vertex normal). The normal at each vertex can be obtained by combining the face normals of the triangle formed by the vertex. In some examples, using a local coordinate system can enable additional compression of the tangent component of the displacement relative to the normal component. For example, the displacement can be signaled as a scalar value (e.g., including sign and magnitude) that can be used to derive the displacement vector based on the normal at the vertex. For example, the displacement vector can be determined as the product of the scalar value at the vertex and the normalized normal vector (e.g., the unit normal vector). Therefore, using a local coordinate system, the displacement does not need to be signaled as three components corresponding to the direction of the canonical coordinate system.

[0070] In some embodiments, a decoder (e.g., the decoder (300) of FIG. 3) may receive and decode a base mesh corresponding to (e.g., having) a downsampled surface (520). Similar to an encoder, the decoder may determine a subdivided mesh having an upsampled surface (530) generated from the downsampled surface (520) by applying a subdivision method. The decoder may receive and decode displacement information including a displacement vector (548) and determine a decoded mesh (e.g., a reconstructed mesh) based on the subdivided mesh (corresponding to the upsampled surface (530)) and the decoded displacement information. For example, the decoder may add a displacement having the position of the corresponding vertex within the subdivided mesh at each vertex. The decoder may obtain a reconstructed 3D mesh by combining the acquired / decoded displacement with the position of the vertex of the subdivided mesh.

[0071] FIG. 6 illustrates an example of vertices of a subdivided mesh (e.g., subdivided base mesh) corresponding to a number of levels of detail (LOD) according to some embodiments. As described above in relation to FIG. 5, the subdivision process (e.g., subdivision method) may be an iterative process, wherein the mesh may be subdivided multiple times, and a hierarchical data structure comprising a number of levels is created. Each level of the hierarchical data structure may contain a different number of data samples (e.g., vertices and edges of the mesh) representing (e.g., forming) a different density / resolution (e.g., also referred to as a level of detail (LoD)). For example, a downsampled surface (520) (of the decimated mesh) may be subdivided into an upsampled surface (530) after a first iteration of subdivision. The upsampled surface (530) may be further subdivided into an upsampled surface (630), etc. In this case, the vertices of the mesh having the downsampled surface (520) may be considered to be within LOD0 or associated therewith. Vertices such as vertices (632) generated from the up-sampled surface (530) after the first iteration of subdivision may be in LOD1. Vertices such as vertices (634) generated from the sampled surface (630) after another iteration of subdivision may be in LOD2, and so on. In some examples, LOD0 may refer to vertices generated from the decimation of the input (e.g., original) mesh, which results in a base mesh containing (e.g., having) the down-sampled surface (520). For example, vertices in LOD0 may be vertices of the reconstructed quantized base mesh (256) of FIGS. 2a and 2b, the reconstructed / decoded base mesh (340) of FIG. 3, and the reconstructed base mesh (440) of FIG. 4.

[0072] In some examples, the calculation of displacement at different LODs follows the same mechanism described above in relation to FIG. 5. In some embodiments, the displacement vector (643) can be calculated from the location of the vertex (641) within the original surface (510) (of the original mesh), from the displacement surface (640) of the deformed mesh, to the vertex (642) at LOD0. The displacement vectors (644 and 645) of the corresponding vertices (632 and 634), respectively, can be calculated from LOD1 and LOD2. Thus, in some examples, multiple iterations of subdivision may correspond to the number of LODs, and one of the iterations may correspond to one LOD of LODs.

[0073] FIG. 7a illustrates an example of an image (720) (e.g., a picture or picture frame) packed with a displacement (700) (e.g., a displacement field or vector) using a packing method (e.g., a packing method or a packing algorithm) according to some embodiments. Specifically, the displacement (700) may be generated as described above in relation to FIG. 5 and FIG. 6 and may be packed into a 2D image. In some examples, the displacement may be a 3D vector containing values ​​for three components of distance. For example, the delta x value represents movement along the x-axis from point A to point B in an orthogonal coordinate system. In some examples, the displacement vector may be represented by fewer than three components, e.g., one or two components. For example, if a local coordinate system is used to store the displacement value, one component with the highest significance may be stored as representing the displacement, and the other components may be discarded.

[0074] In some examples, as further described below, the displacement value may be converted to a different signal domain to achieve better compression. For example, the displacement may be wavelet transformed and decomposed into wavelet coefficients (e.g., coefficient values ​​or transform coefficients). In these examples, the displacement (700) packed into the image (720) may include resulting wavelet coefficients (e.g., transform coefficients) that can be compressed more efficiently than the untransformed displacement value. On the decoder side, the decoder may decode the displacement (700) as wavelet coefficients and apply an inverse wavelet transform process to reconstruct the original displacement value obtained from the encoder.

[0075] In some examples, one or more of the displacements (700) may be quantized by an encoder before being packed into a displacement image (720). In some examples, one or more displacements may be quantized before the wavelet transform, after the wavelet transform, or before and after the wavelet transform. For example, FIG. 7a shows wavelet transform values ​​8, 4, 1, -1, etc. quantized from the displacements (700). On the decoder side, the decoder may perform inverse quantization to reverse or undo the quantization process performed by the encoder.

[0076] Generally, quantization in signal processing can be the process of mapping input values ​​from a larger set to output values ​​within a smaller set. This is often used for data compression to reduce the amount, precision, or resolution of data to a more compact representation. However, such reduction can lead to a loss of information and introduce compression artifacts. The selection of quantization parameters, such as the number of quantization levels, is a trade-off between the desired level of precision and the resulting data size. There are many different quantization techniques, such as uniform quantization, non-uniform quantization, and adaptive quantization, that can be selected, enabled, or applied. These can be used depending on the specific requirements of the application.

[0077] In some examples, wavelet coefficients (e.g., displacement coefficients representing a displacement signal) can be adaptively quantized according to the LOD. As previously mentioned, a mesh can be iteratively subdivided to create a hierarchical data structure containing multiple LODs. In this example, each vertex and its associated displacement belong to the same hierarchical level in the LOD structure, for example, the LOD corresponding to the subdivision iteration in which the vertex was generated. In some examples, a vertex in each LOD can be quantized according to a corresponding quantization parameter that specifies a different level of intensity / precision of the signal to be quantized. For example, wavelet coefficients in LOD 3 can have a quantization parameter of, say, 42, and wavelet coefficients in LOD 0 can have a different, smaller quantization parameter of 28 to preserve more detail in LOD 0.

[0078] In some examples, the displacement (700) may be packed onto pixels of a displacement image (720) having width W and height H. In one example, the size of the displacement image (720) (e.g., W multiplied by H) may be greater than or equal to the number of components of the displacement image (700) to ensure that all displacement information can be packed. In some examples, the displacement image (720) may be further subdivided into smaller areas (e.g., squares) referred to as packing blocks (730). In one example, the length of the packing block (730) may be an integer multiple of 2.

[0079] Displacements (700) (e.g., displacement signals represented by quantized wavelet coefficients) can be packed into packing blocks (730) according to a packing order (732). Each packing block (730) can be packed (e.g., arrayed or stored) into displacement images (720) according to a packing order (722). Once all displacements (700) are packed, empty pixels in the images (720) can be padded with neighboring pixel values ​​for improved compression. In the example illustrated in FIG. 7a, the packing order (722) for the packing blocks may be raster order, and the packing order (732) for the displacements within the packing blocks (730) may be, for example, Z-order. However, it should be understood that other packing methods may be used for both the blocks and the displacements within the blocks. In some embodiments, the packing method for the blocks and / or within the blocks may be predetermined. In some embodiments, the packing scheme may be signaled by an encoder as a bitstream per patch, patch group, tile, image, or image sequence. In this regard, the signaled packing scheme may be obtained by a decoder from the bitstream.

[0080] In some examples, the packing order (732) may follow a space-filling curve that specifies a spatial traversal in a continuous and non-repetitive manner. Some examples of space-filling curve algorithms (e.g., methods) include the Z-order curve, Hilbert curve, Peano curve, Moore curve, Sierpinski curve, and Dragon curve. Space-filling curves have been used in image packing techniques to efficiently store and retrieve images in a way that maximizes storage space and minimizes retrieval time. Space-filling curves are very suitable for this task because they can provide a one-dimensional representation of a two-dimensional image. One common image packing technique that uses space-filling curves is called Z-order or Morton order. A Z-order curve is constructed by interleaving the binary representations of the x and y coordinates of each pixel in an image. This generates a one-dimensional representation of the image that can be stored in a linear array. To use Z-order curves for image packing, the image is first divided into small blocks, typically 8x8 or 16x16 pixels in size. Then, each block is encoded using a Z-order curve and stored in a linear array. When the image needs to be retrieved, the blocks are decoded using an inverse Z-order curve and reassembled into the original image.

[0081] In some examples, once packed, the displacement image (720) can be encoded and decoded using a conventional 2D video codec.

[0082] FIG. 7b illustrates an example of a displacement image (720) according to some embodiments. As shown, the displacements (700) packed in the displacement image (720) can be ordered according to their LODs. For example, displacement coefficients (e.g., quantized wavelet coefficients) can be ordered from the lowest LOD (e.g., LOD 0) to the highest LOD (e.g., LOD 2). That is, wavelet coefficients representing the displacement for a vertex at the first LOD can be packed (e.g., arranged and stored in the displacement image (720)) according to the first LOD. For example, the displacements (700) can be packed from the lowest LOD to the highest LOD. Higher LODs represent higher vertex densities and correspond to more displacements compared to lower LODs. Parts of the displacement image (720) that are not at any LOD may be padded parts.

[0083] In some examples, displacements can be packed in reverse order from the highest LOD to the lowest LOD. In one example, the encoder can signal whether displacements are packed from the lowest LOD to the highest LOD or from the highest LOD to the lowest LOD.

[0084] In some examples, the wavelet transform is applied to displacement values ​​to generate wavelet coefficients (e.g., displacement coefficients) representing displacement signals that can be compressed more easily. The wavelet transform is commonly used in signal processing to decompose a signal into a set of wavelets, which are small wave-like functions that enable the capture of local features within the signal. The result of the wavelet transform is a set of coefficients representing the contribution of each wavelet at different scales and locations within the signal. It is useful for detecting and localizing transient features in a signal and is commonly used in signal analysis and data compression, such as image, video, and audio compression.

[0085] Taking a 2D image as an example, the wavelet transform is used to decompose an image (signal) into two discrete components known as prediction (also referred to as approximation) and detail. The decomposed signal is further divided into high-frequency components (detail) and low-frequency components (approximation / prediction) by passing through two filters: a high-pass and a low-pass filter. In the example of a 2D image, two filtering stages—horizontal and vertical filtering—are applied to the image signal. A downsampling step is also required after each filtering stage on the decomposed components to obtain wavelet coefficients, generating four sub-signals at each level of decomposition. High-frequency components correspond to abrupt changes or transitions in the signal, such as edges or lines in the image. Meanwhile, low-frequency components refer to the global characteristics of the signal. Depending on the application, different filtering and compression can be achieved. There are various types of wavelets, such as Haar, Daubechies, and Symlets, each possessing different characteristics such as frequency resolution and time localization.

[0086] In some examples, the displacement (700) (e.g., the transformation factor of the displacement) can be adaptively predicted / reconstructed between frames. Inter-prediction / reconstruction may be based on a LoD where the residual value of the displacement (e.g., the transformation factor residual) is calculated by subtracting a reference displacement (e.g., the transformation factor) from a target displacement (e.g., the target transformation factor). As a result, the displacement residual (e.g., the transformation factor residual of the displacement) is quantized instead of the displacement (e.g., the transformation factor of the displacement) and further compressed through the displacement image as discussed above in relation to FIGS. 7a and 7b. In other examples, an arithmetic codec may be used to compress the displacement or the displacement residual. The arithmetic codec may use different or adaptive (e.g., based on the LoD or block size) parameters to optimize the compression performance for coding the displacement or the displacement residual. Such parameters may include, but are not limited to, block size, context selection, block skipping, etc.

[0087] In additional examples, the displacement or displacement residual can be quantized (or dequantized) using a floating-point or fixed-point quantizer (dequantizer). In some examples, the reference displacement (e.g., reference transform displacement coefficient) can be reconstructed from the fixed-point dequantized displacement to perform inter-frame prediction for the displacement (e.g., transform coefficient of the displacement). In this example, a reference buffer can be used to store the reference displacement.

[0088] In signal processing, a lifting scheme is a technique for designing wavelets and performing a Discrete Wavelet Transform (DWT). It is an alternative approach to traditional filter bank implementations of DWT, offering several advantages in terms of computational efficiency and flexibility. It uses a series of lifting steps to decompose a signal, allowing an input signal—for example, an input signal representing displacement for a 3D mesh—to be transformed in-place into displacement coefficients. In a lifting scheme, a series of lifting operations (e.g., lifting steps) can be performed. Each lifting operation includes a prediction step (e.g., a prediction operation) and an update step (e.g., an update operation). These lifting operations can be applied repeatedly to obtain wavelet coefficients.

[0089] In some examples, displacement for a 3D mesh frame can be transformed using a wavelet transform with lifting, referred to, for example, as a lifting scheme. Specifically, the wavelet transform takes an input signal (e.g., a displacement signal) into two signals, namely, an even-sampled signal and odd sample signals It can be "divided". The even samples E are two displacement signals associated with two vertices considered to be on the edge of the vertex associated with the input displacement signal. and May include odd samples It can represent the original input signal. As mentioned above, edge information can be determined (e.g., generated or received) from the segmentation method applied to each mesh frame of the 3D mesh. Odd samples The prediction of the signal is an even-sample signal It can be determined based on the weighted sum of. Then, odd samples The signal is an odd sample It can be encoded as the difference between signals. Also, even sample signal and Each can be adjusted based on the difference weighted by the updated weight, said updated weight is odd samples Association with signal (e.g., odd samples) It can be associated with the signal's LOD.

[0090] In some embodiments, wavelet coefficients representing displacements corresponding to vertices of 3D mesh geometry (e.g., transformed wavelet coefficients) may be quantized according to quantization parameters set for each LOD of a plurality of LODs. The wavelet coefficients representing displacements may be wavelet coefficients of displacement or wavelet coefficient residuals of displacement (displacement residuals). Vertices may span across a plurality of LODs (e.g., may be generated). Additionally, wavelet coefficients within each LOD may be quantized by a dead zone quantizer. A dead zone quantizer may be a type of quantizer that exhibits symmetric behavior around a quantized value of 0 and reaches its peak value at 0. The region around the 0 output value of such a quantizer is referred to as a dead zone. The dead zone may be configured with a width different from that of other quantization steps of the quantizer. A dead zone quantizer may be a uniform quantizer such that other quantization steps other than the dead zone have a uniform width (e.g., quantization size). In other words, if the dead zone quantizer is set to the same size as the quantization steps, each quantization step of the dead zone quantizer is identical or uniform.

[0091] In some examples, wavelet coefficients of vertices at different LODs may be quantized (and de-quantized) according to their corresponding LODs, which specify different levels of intensity (e.g., precision) or the signal to be scaled. For example, wavelet coefficients of vertices at LOD 3 may have a quantization scaling factor of, for example, 1 / 8, and wavelet coefficients of vertices at LOD 0 may have a different and larger quantization scaling factor value of, for example, 1, to preserve more detailed information at LOD 0 (e.g., allowing wavelet coefficients at higher LODs to be quantized by a larger quantization step). In this regard, the de-quantization factor may be, for example, 8 associated with LOD 3, and, for example, 1 associated with LOD 0.

[0092] FIG. 8 illustrates an exemplary drawing (800) of a quantizer for quantizing transformed wavelet coefficients (as illustrated on the x-axis) representing displacement of a 3D mesh, according to some embodiments. The quantizer may be applied by an encoder (e.g., the quantizer (212) of FIG. 2a and FIG. 2b). According to some embodiments, the quantizer quantizes the transformed wavelet coefficients of the 3D mesh into quantized values ​​(e.g., integer values).

[0093] As shown in drawing (800), the quantizer comprises a dead zone surrounding a quantized value of zero having a width associated with a dead zone parameter (DZ), and a quantization step having the same size as a scaling factor s (e.g., quantization scaling factor). It may be a dead zone quantizer having. The width of this dead zone can be determined or indicated by a dead zone parameter (DZ), for example, the width is 2*DZ. In some examples, the width of this dead zone is the scaling factor s and the dead zone offset / bias value It can be determined according to, (or alternatively) It is expressed as ), where the quantization step is an integer value and the other quantization step is s* It has a uniform width expressed as.

[0094] In some examples, the scaling value s and / or the dead zone parameter DZ may be associated with each LOD of the LOD associated with the vertex of the 3D mesh.

[0095] The quantizer is the input signal Converts (e.g., quantization) and output signal It can be converted to. In the case of 3D mesh encoding, the input signal can be a vertex displacement signal (e.g., a transformation coefficient representing the vertex displacement), and the output signal can be a quantized displacement signal (e.g., quantized transform coefficients). For example, a quantizer is an input signal The sign of the output signal It can be maintained as the sign of. In some examples, the input signal It can be adjusted (e.g., the dead zone is If indicated as, it is added, or alternatively, the dead zone (If indicated as, it can be subtracted), and can be scaled according to a scaling factor s (e.g., divided by s). The quantizer takes the floor function (or alternatively the sealing function) as the scaled and adjusted input signal. Apply to the value of to output signal can be determined as the smallest integer less than or equal to the corresponding value. Alternatively, if a Shilling function is applied, the quantizer outputs the signal can be determined as the smallest integer greater than or equal to the corresponding value. For example, ga is 0 and When α is 2, the input signal set 0, 2.5, 3.7, 6.8 can be quantized into output signals 0, 1, 1, 3, respectively.

[0096] Therefore, the quantizer input signal Input signal depending on the range of values ​​to which it belongs quantized signal , , , It can be quantized into the like. and The selection of values ​​depends on the width of the dead zone, and how many signals with this range / width value are associated with the dead zone. Quantized into (e.g., ) can decide. The choice of can determine the width of the quantization step and how many signals with values ​​in this range (outside the dead zone) are quantized into non-zero values. For example, inside The input value of the range is 1 It is quantized into (e.g., 1). For example, inside The input value of the range is 2 It is quantized into (e.g., 2), etc.

[0097] FIG. 9 illustrates an exemplary diagram (900) of an inverse quantizer for inversely quantizing quantized transform wavelet coefficients (as illustrated on the y-axis) representing the displacement of a 3D mesh, according to some embodiments. The inverse quantizer may be applied by a decoder (e.g., the inverse quantizer (222) of FIG. 2a and 2b, or the inverse quantizer (312) of FIG. 3). According to some embodiments, the quantizer quantizes the transformed wavelet coefficients of the 3D mesh into quantized values ​​(e.g., integer values). The inverse quantizer may be an example of a dead zone inverse quantizer corresponding to the dead zone quantizer described in FIG. 8. The dead zone inverse quantizer may be associated with one of a plurality of LODs specifying different levels of precision (e.g., or intensity) of the signal to be inversely quantized, and / or a quantization parameter. The inverse quantizer (quantized displacement signal of FIG. 8) Input signal corresponding to It can take, and apply inverse scaling using an inverse scaling factor (e.g., based on the scaling factor s in Fig. 8), and the inversely scaled value as the output signal It can be output as. For example, the input signal can be multiplied by a scaling factor s corresponding to the reciprocal of the inverse scaling factor. In some examples, the input signal is a quantized integer value that can become a floating / double-precision value after inverse scaling. Due to errors introduced by quantization in the encoder as described in Fig. 8, the output signal is the original input signal quantized by the encoder Approximates .

[0098] In some examples, the inverse quantizer is the inverse quantization signal , , , Input signal as one of them It can be inversely quantized, and this can be regarded as a reconstruction point representing the inverse quantization level.

[0099] In a conventional embodiment, as described above with reference to FIG. 9, the inverse quantizer applies only an inverse scaling factor regardless of whether the quantizer for quantizing the displacement signal for the vertices of the 3D mesh is a dead zone quantizer or a uniform quantizer. Quantized signal By inversely quantizing, the inversely quantized (e.g., reconstructed) signal (e.g., , , , ) corresponds to a vertex (of the original 3D mesh) whose position is within a specific range associated with a scaling factor (e.g., the width of the quantization step). However, these inversely quantized values ​​are unquantized signals that fall within the range of values ​​associated with the scaling factor. It may be a poor approximation (or representation) of. For example, the input signal can include five values ​​between 1 and 2: 1.2, 1.3, 1.5, 0.9, and 1.6. If a scaling factor of 1.0 is applied, all five values ​​can be quantized to 1.0 and reconstructed as a value of 1.0. The statistics of a displacement signal within this exemplary quantization range (e.g., 1–2) are 1.3's The average value of the five signals can be represented. Therefore, the five input signals An average error of 0.3 (and a corresponding quality reduction) is introduced between the average of and the average of the quantized (and inversely quantized) values ​​of 1.0.

[0100] In some embodiments, to reduce the error introduced by quantization performed in the encoder, a quantization offset may be applied during the inverse quantization of quantized wavelet coefficients representing the displacement of vertices of a 3D mesh (e.g., mesh frame). For example, a decoder may decode quantized wavelet coefficients representing the displacement of vertices from a set of vertices from a bitstream. Quantization information representing the quantization offset associated with a subset of vertices in the set of vertices may be decoded from the bitstream. Then, the wavelet coefficients (corresponding to the inverse quantized wavelet coefficients) may be determined as the displacement (or displacement residual) of the vertices based on the quantization offset associated with the subset to which the vertices belong. By applying a quantization offset (e.g., reconstruction bias / offset / adjustment) to the subset of vertices, a more accurate displacement (or displacement residual) may be reconstructed or achieved. For example, in the above example where the average error for five signals is determined to be 0.3, the quantization offset can be indicated as 0.3 such that the average of the inverse quantized signal is equal to the average of the original signal.

[0101] In some implementations, a quantization offset may be applied to the inversely quantized value. For example, the decoder may determine the wavelet coefficients by inversely quantizing the quantized wavelet coefficients. Then, the decoder may adjust the wavelet coefficients according to the quantization offset to determine the vertex displacement (or displacement residual) based on whether the vertex is in a subset. In some examples, the quantization offset may be determined for a subset within the current mesh frame (i.e., the current 3D mesh frame). For example, each quantization offset may be determined for each vertex subset of the current mesh frame. A separate quantization offset may be determined for each subset per mesh frame.

[0102] In some embodiments, a quantization offset may be applied to the quantized values. For example, the decoder may adjust the quantized wavelet coefficients according to the quantization offset based on whether the vertex is in a subset. Then, the decoder may determine the vertex displacement (or displacement residual) based on inversely quantizing the adjusted quantized wavelet coefficients.

[0103] In some embodiments, a set of vertices includes non-overlapping subsets of vertices, and quantization information can be signaled (e.g., decoded) for each subset. For example, a subset may include vertices of the same LOD among LODs. In this example, the prediction / reconstruction error between the dequantized signal and the original signal within each LOD can be reduced based on decoding the quantization offset per LOD. In other examples, a subset may include vertices in a sub-mesh of a 3D mesh, a group of patches of said sub-mesh or 3D mesh, or one patch of said patch group.

[0104] In some examples, an indication of whether quantization adjustment is enabled (e.g., mode indication, flag, syntax element) is decoded for a vertex subset. Based on the indication that quantization adjustment is possible, the decoder may further decode quantization information indicating a quantization offset.

[0105] FIG. 10a is for inversely quantizing (e.g., inverse scaling) quantized transform wavelet coefficients representing displacement of a 3D mesh, according to some embodiments, An exemplary drawing (1000A) of an inverse quantizer having a quantization offset (1002) and an inverse quantization scaling factor(s) is illustrated. A 3D mesh can be processed as a sequence of mesh frames. The inverse quantizer of drawing (1000) can perform the reverse operation of a quantizer in an encoder, such as described in FIG. 8. The inverse quantizer can be applied by a decoder (e.g., the inverse quantizer (222) of FIG. 2a and 2b, or the inverse quantizer (312) of FIG. 3).

[0106] In some examples, one or more parameters used to derive the quantization offset (1002) and the inverse quantization scaling factor may be determined (and decoded) by a decoder for each subset of the vertex set of the 3D mesh. In some examples, the quantization offset (1002) and said one or more parameters may be determined for each subset of vertices for each mesh frame of the mesh frame sequence. For example, a vertex subset may correspond to vertices within the same LOD (or alternatively, the same sub-mesh, patch group, patch, tile, etc.) of the current mesh frame. As described above, optionally enabling quantization adjustment and signaling (e.g., decoding) the indication of the quantization offset associated with each vertex subset can improve the statistics of the inverse quantized wavelet to be closer to the statistics of the original non-quantized wavelet coefficients, thereby achieving less distortion (e.g., improved quality). These advantages can be achieved regardless of whether the inverse quantizer is a uniform inverse quantizer or a dead zone inverse quantizer, because the quantization error is more influenced by the scaling factor (and inverse scaling factor) than by the width of the dead zone within the dead zone quantizer (and the corresponding dead zone inverse quantizer).

[0107] As shown in the reconstruction adjustment process of drawing (1000A) applicable to a vertex subset, the quantized input signal (e.g., quantized displacement output signal of Fig. 8) ) is the scaling factor s + It can be adjusted (e.g., shifted or biased) to a value equal to a multiple of the quantization offset (1002). In some examples, the quantized input signal is the value of the quantization offset (1002) It can be inversely scaled first before being adjusted, and the quantization offset can represent an offset value scaled by a scaling factor s. In another example, the value of the quantization offset (1002) is the quantized input signal before inverse quantization according to the scaling factor(s) or the corresponding inverse scaling factor (e.g., 1 / s). It can be applied to. In drawing (1000A), the quantization offset (1002) is a quantization displacement input signal It is applied to the range (1004) of possible inverse quantized values. Each vertex subset can be associated with one quantization offset (1002).

[0108] In drawing (1000A), the quantization offset (1002) is a quantized input signal Inverse quantized signal derived from It is illustrated as being added to the size of. In another example, the quantization offset (1002) is the inversely quantized input signal It can be added directly to.

[0109] In some examples, each quantization offset (1002) is associated with each vertex subset, such as vertices within the same LOD, and these same quantization offset values This can be applied to vertices within this subset. In some examples, a quantization offset (1002) set to 0 indicates the disabling of quantization adjustments (e.g., reconstruction point offset adjustment processes) associated with wavelet coefficients corresponding to the vertex subset. For example, the quantization offset (1002) is 0 (output signal The inverse quantized value (indicated as) can be set to 0. This inverse quantized value 0 corresponds to a value within the dead zone region calculated by the encoder, as shown in FIG. 8.

[0110] As described above, the quantization offset (1002) is computed by the encoder and can be signaled to the decoder to compensate for the loss of detail due to the quantizer applied to the displacement (or displacement residual) for the vertex. Generally, for some subsets, such as higher LODs, a larger quantizer (e.g., having a larger scaling factor s) is used when the subset is an LOD. For the larger quantizer, the quantization offset It is used to adjust for details lost due to quantization. However, larger quantizers can result in displacement (or displacement residuals) of a larger range of values ​​being quantized to the same value, which leads to an unnecessarily large quantization offset. It can result in a value. Therefore, in some embodiments, the quantization offset (1002) is an offset adjustment value (a δ It can be further adjusted by ).

[0111] In some examples, the quantization offset (1002) is an offset adjustment value (a δ It can be further adjusted by adding ). For example, each offset adjustment value (a δ ) can be determined for each vertex subset. For example, the offset adjustment value (a δ ) can be determined based on subtracting two offset values ​​between two vertex subsets, such as between two offset values ​​determined for two consecutive LODs. For the current LOD, the consecutive LOD may be the next highest LOD or, for example, the next lowest LOD based on a specific subset.

[0112] In some examples, the quantization offset (1002) is an offset adjustment value (a δ Value of quantization offset (1002) having ) It can be further adjusted by multiplying. For example, the offset adjustment value (a δ) may be a scaling factor determined based on the amount of quantization associated with a vertex subset. For example, the scaling factor may be determined based on the LOD associated with a specific quantizer. In some examples, the scaling factor may be determined to be inversely proportional to the level of the LOD. As an example, the inverse relationship may be a negative power of 2, where the power is based on the level of the LOD. In this example, for LOD0, LOD1, and LOD2, the exemplary scaling factors, respectively, may be 2^(-0) = 1, 2^(-1) = 0.5, and 2^(-2) = 0.25. Offset adjustment value (a δ The effect of ) is the quantization offset for vertex subsets with larger quantizers, such as higher LOD. It is to apply a larger reduction to.

[0113] In some embodiments, a high value of the quantization offset (1002) To prevent a distorted distribution or outliers of displacement (or displacement residual) values ​​within a range of values ​​quantized to the same value that would cause [a specific error], the quantization offset (1002) may be limited so as not to exceed a predetermined threshold. For example, (offset adjustment value (a δ Quantization offset value (with or without ) is the threshold A higher quantization offset (1002) value is the threshold Threshold to be set to It can be compared with. In some examples, the quantization offset value is also the threshold Compared to the threshold lower quantization offset value This threshold It can be set to. For example, the range of values ​​for the quantization offset (1002) is [0, ] or [- ,0] or [- It can be set to ]. In some examples, The size of It can be the same as the size of the minimum value using.

[0114] In some examples, the quantization offset (1002) is a threshold value or Only when exceeding the limit set by, the threshold and / or Offset adjustment value (a) before being compared with δ It can be adjusted first by ).

[0115] In some examples, quantization offset value is the threshold and / or It can be compared to, but the quantization offset value Instead of limiting to these values, the quantization offset value is the threshold and / or Based on exceeding one of (e.g., in response) offset adjustment value (a δ It can be adjusted by ). In another example, the offset adjustment value (a δ The threshold value used to determine whether to apply ) is different from the threshold value used to limit the possible values ​​of the quantization offset (1002).

[0116] In some implementations, quantization offset The adjustments and / or thresholds applied to may be performed in the same manner in the encoder and decoder. For example, the encoder may signal one or more indications (e.g., flags) that enable adjustments, applied thresholds, or a combination thereof. In another embodiment, the encoder, the final quantization offset for a vertex subset, which is signaled from the bitstream to the decoder. To determine the quantization offset Adjustments and / or thresholds may be applied to it. In such an implementation, the final quantization offset The processing to derive is moved to the encoder, reducing the computational complexity required by the decoder.

[0117] In some embodiments, the quantization offset value of the quantization offset (1002) For each vertex subset, it can be signaled by the encoder to the decoder.

[0118] In some embodiments, the quantization offset value of the quantization offset (1002) It can be derived based on the quantization level associated with a vertex subset so that it does not need to be explicitly signaled in the bitstream from the encoder to the decoder. For example, the encoder and decoder derive the quantization offset for the vertices of the subset based on the LOD associated with that subset (e.g., the index of the LOD). It can be computed individually and identically. For example, quantization offset It can be determined based on a quantization scaling factor determined based on the index of LOD.

[0119] As described above, applying a quantization offset (1002) to the inversely quantized displacement (or displacement residual) values ​​of a vertex subset (e.g., vertices within the same LOD) can compensate for errors introduced during the quantization process in the encoder. However, not only are the quantization errors significantly different for each subset of vertices, but the quantization errors can also vary significantly for the displacement (or displacement residual) values ​​of vertices within the same vertex subset. Depending on the object or multiple objects represented by the 3D mesh frame, the inversely quantized values ​​in different ranges of values ​​may be associated with different quantization errors.

[0120] An embodiment of the present disclosure relates to applying a plurality of quantization offsets associated with each vertex subset during the in-quantization of quantized wavelet coefficients representing the displacement of a vertex in a vertex subset of a 3D mesh (e.g., a mesh frame). The wavelet coefficients representing the displacement may be wavelet coefficients of the displacement or wavelet coefficient residuals of the displacement. For example, a decoder may decode quantized wavelet coefficients representing the displacement of a vertex from a set of vertices from a bitstream. Quantization information representing a plurality of quantization offsets associated with a vertex subset of a set of vertices may be decoded from the bitstream. For example, a vertex subset may be associated with one of a plurality of LODs. The wavelet coefficients may be in-quantized to determine wavelet coefficients (e.g., non-quantized wavelet coefficients or in-quantized coefficients). Based on the vertex being in a vertex subset, the wavelet coefficients may be adjusted according to a quantization offset among a plurality of offsets to determine the displacement (or displacement residual) of the vertex.

[0121] In some embodiments, each of the plurality of quantization offsets may be associated with an interval of values ​​(e.g., inverse quantization wavelet coefficients). In some examples, the intervals of values ​​may not overlap such that any possible value of the wavelet coefficient is within only one interval. The quantization offset may be selected from the plurality of quantization offsets based on the value of the quantized wavelet coefficient associated with the interval of values ​​corresponding to the quantization offset (e.g., within). By applying different quantization offsets (e.g., reconstruction bias / offset / adjustment) to different ranges / intervals of the wavelet coefficient values, a more accurate displacement (or displacement residual) may be reconstructed or achieved.

[0122] In some embodiments, a plurality of quantization offsets includes at least a first quantization offset and a second quantization offset. In some examples, the first quantization offset may be associated with a dead zone of the quantizer for wavelet coefficients. For example, the first quantization offset may be associated with a value of 0 for wavelet coefficients. In some examples, the second quantization offset may be associated with an interval of values ​​that excludes the dead zone. For example, the second quantization offset may be associated with an interval of values ​​that excludes 0. In some examples, the interval of values ​​may be an interval of integers.

[0123] In some embodiments, a plurality of quantization offsets includes at least a first quantization offset, a second quantization offset, and a third quantization offset. The first quantization offset may be associated with a value of 0 and corresponds to a dead zone as described above in relation to FIG. 8. The second quantization offset may be associated with a first interval of values ​​that exclude 0. For example, the second quantization offset may be associated with a positive value. The third quantization offset may be associated with a second interval of values ​​that exclude 0 and a first interval of values. For example, the third quantization offset may be associated with a negative value.

[0124] In some embodiments, additional quantization offsets corresponding to non-overlapping intervals may be configured or included in the quantization information to provide a more localized quantization offset for each interval of possible values ​​of wavelet coefficients (i.e., inversely quantized wavelet coefficients). In some examples, the quantization information may include start and end values ​​for each interval and may associate each interval with a respective quantization offset of a plurality of quantization offsets.

[0125] FIG. 10b illustrates an exemplary drawing (1000B) of an inverse quantizer in which a plurality of quantization offsets (1012) are applied to a vertex subset according to some embodiments. For example, the quantization offsets (1012) are quantization offsets (1012A) ( ), quantization offset(1012B)( ), and quantization offset (1012C)( It may include at least two quantization offsets such as ). Similar to Figure (1000A), the inverse quantization scaling factor(s) may be used to inversely quantize the inversely quantized (e.g., inversely scaled) transform wavelet coefficients, which are the quantization displacement input signals shown on the y-axis. The inverse quantizer of Figure (1000B) may perform the inverse operation of the quantizer in the encoder, as described in Figure 8. The inverse quantizer may be applied by a decoder (e.g., the inverse quantizer (222) of Figures 2a and 2b, or the inverse quantizer (312) of Figure 3).

[0126] quantized input signal Inversely quantizing to obtain the inversely quantized output signal The process of determining the displacement signal of an in-vertex is the inverse quantization signal to be adjusted (e.g., expressing inverse quantized wavelet coefficients). It is similar to that described in relation to the figure (1000A) of FIG. 10a, except that one quantization offset from the quantization offset (1012) can be selected depending on the value of .

[0127] In some examples, each quantization offset (1012A–C) corresponds to a value within each interval (1014A–C) and can be applied thereto. For example, an inversely quantized signal Based on the fact that it is within the interval (1014B), the quantization offset (1012B) corresponding to the interval (1014B) is an inverse quantization signal It can be used to adjust and determine an adjusted reconstruction signal that represents the reconstructed displacement (or displacement residual) value for the vertex.

[0128] In some examples, intervals (1014A–C) include values ​​that do not overlap with at least one of the intervals containing a value of 0. For example, interval (1014A) is an inversely quantized output signal having a value of 0. It may include and correspond to. In one example, the interval (1014A) includes only a value of 0. For example, the interval (1014B) is an inversely quantized output signal It may include a range of positive values, and the interval (1014C) is an inversely quantized output signal It may include a range of negative values. In these embodiments, the decoder is an inversely quantized output signal By determining whether the value of is 0, positive, or negative, it is possible to determine whether to apply the quantization offset (1012A), quantization offset (1012B), or quantization offset (1012C), respectively.

[0129] Similar to how the quantization offset (1002) of FIG. 10a can be signaled to a decoder as a bitstream, each of the quantization offsets (1012A–C) can be signaled in a similar manner. For example, the value of the quantization offset can be signaled as a signed value. For example, the value of the quantization offset can be signaled by two indications: a first indication of the absolute value of the quantization offset, and a second indication of the sign of the quantization offset. In some examples, the value of the quantization offset can be signaled by an index to a table of offset values.

[0130] In some examples, the decoder may determine whether to apply a specific quantization offset based on comparing the quantization offset value with one or more thresholds, as well as the range of values ​​associated with the quantization offset. In one example, one or more thresholds may be associated with all quantization offsets. In another example, each quantization offset may be associated with one or more thresholds.

[0131] For example, the quantization offset is based on whether its value is within the interval associated with the corresponding quantization offset, in the inversely quantized output signal It can be selected from the quantization offset (1012) to be applied to. Then, the decoder, the selected quantization offset is applied to the inverse quantization signal Whether it applies to, the quantization offset is the first threshold Exceeding or second threshold It can be determined based on whether it is less than

[0132] FIG. 10c illustrates a table (1000C) showing exemplary quantization offsets for different intervals of possible values ​​of displacement (or displacement residual) for each of a plurality of LODs, according to some embodiments. An LOD may be an example of a subset of a set of vertices in a 3D mesh. As described in relation to FIGS. 5 through 6 and FIG. 9, FIG. 10a and FIG. 10b, the displacement (or displacement residual) of a vertex within a 3D mesh may be decoded into an inversely quantized (and inversely transformed) coefficient value. One of the quantization offset values ​​shown in the table (1000C) may be selected to apply to the inversely quantized value representing the displacement (or displacement residual) of a vertex, based on which of the three intervals (e.g., 0, a positive value, or a negative value) belongs within the vertex and which subset of the vertex (e.g., an LOD) belongs to.

[0133] In some implementations, the quantization offset value may be computed by the encoder as optimal for reducing the quantization error of the value within each interval of the value for each vertex subset (e.g., LOD). The computed quantization offset value may be signaled to the decoder as quantization information in the bitstream. It should be noted that the magnitude of the quantization offset value associated with 0 can generally be smaller than the values ​​in other intervals, e.g., positive or negative values. However, depending on the characteristics of the mesh frame, the quantization parameters, and the type of wavelet transform, this may vary, as in the case of LOD2, where the offset value associated with 0 is higher than the offset values ​​associated with other intervals.

[0134] Furthermore, the transformed coefficients representing displacement at higher LODs are more quantized than those at lower LODs, which results in higher quantization errors, typically indicated by larger offset values. For example, the quantization offset at LOD2 can be larger in magnitude than the quantization offset at LOD1. In other examples, the magnitude of the offset value is determined by the properties of the mesh sequence, quantization parameters, and wavelet transform.

[0135] Accordingly, by providing multiple quantization offsets for different intervals of possible inversely quantized wavelet coefficients representing the reconstructed displacement of a specific subset of vertices, quantization error can be reduced across subsets (e.g., LODs) and within each subset (e.g., within an LOD of an LOD).

[0136] FIG. 11 shows examples of inverse quantizers (1100A–B) for iteratively inversely quantizing the quantized transform wavelet coefficients of vertices in each vertex subset of a 3D mesh, according to some embodiments. The operation of the inverse quantizers (1100A–B) may be applied by a decoder (e.g., the inverse quantizer (222) of FIG. 2a and 2b, or the inverse quantizer (312) of FIG. 3). The inverse quantizers (1100A–B) may perform an inverse quantization scheme having a reconstruction adjustment process, according to some embodiments such as those described in relation to FIG. 10a through 10c. The inverse quantizers (1100A–B) may be dead zone inverse quantizers, as described above in FIG. 10a and 10b.

[0137] In some examples, as performed by the quantizer (212) of FIGS. 2a and 2b, the quantization process is repeatedly applied to the wavelet coefficient signal for each vertex subset (e.g., from a vertex of a higher LOD to a vertex of a lower LOD). To perform the operation of the quantization process in reverse, the inverse quantizer (1100A) repeatedly performs inverse quantization for each vertex subset (e.g., from a vertex of a lower LOD to a vertex of a higher LOD), as illustrated in the subset-based iterator (1102A). For the quantized wavelet coefficients of the vertices in each subset, the inverse quantizer (1100A) repeatedly performs inverse quantization for each wavelet coefficient in the subset, as illustrated in the displacement-based iterator (1104). Once all wavelet coefficients in a subset have been dequantized, the dequantizer (1100A) dequantizes the wavelet coefficients in the next subset. In each dequantization operation, the dequantizer (1100A) may include a reconstruction adjuster (1106) and an inverse scaler (1108). The reconstruction adjuster (1106) may apply a quantization offset indicated by offset indication(s) (1110), as described above in FIGS. 10a through 10c and further described below in relation to FIG. 13. The inverse scaler (1108) may apply an inverse scaling factor indicated by scaling indication(s) (1112), as described above in FIGS. 10a and 10b and further described below in relation to FIG. 13.

[0138] In some examples, when a subset of the vertex set of a 3D mesh corresponds to different LODs of multiple LODs of the vertices, the inverse quantizer (1100A) can iteratively inverse quantize signal samples (e.g., displacement signals and corresponding quantized transformed wavelet coefficient representations) from a lower LOD to a higher LOD. For example, in each iteration, the next input signal associated with the LOD (e.g., input signals in Figs. 10a and 10b) ) is the output signal, which is the inverse quantized coefficient representation of the displacement (or displacement residual) with respect to the vertex It is converted (e.g., inversely quantized according to the quantization offset). The subset-based iterator (1102A) is repeated for each higher LOD until the highest LOD level is processed, at which point all quantized wavelet coefficient signals (e.g., input signals) ) will be inversely quantized (e.g., input signal ). For example, a base mesh of 900 vertices can be subdivided into an upsampled mesh having 57,600 vertices across four LOD levels (e.g., LOD0 containing vertices with indices 1 through 900, LOD1 containing vertices with indices 901 through 3600, LOD2 containing vertices with indices 3601 through 14400, and LOD3 containing vertices with indices 14401 through 57600). In this example, the associated displacement (e.g., displacement value and quantized wavelet coefficient representation) (or displacement residual) has the same order as these vertices. In this example, the inverse quantizer (1100A) can start from the lowest LOD, which may be LOD0.

[0139] In some embodiments, the reconfiguration adjuster (1106) may determine whether to apply quantization adjustment (e.g., reconfiguration adjustment) based on a first indication of offset indication(s) (1110) received (e.g., decoded) from the bitstream. For example, the first indication may be received (e.g. decoded) and associated with each vertex subset. In some examples, if the subset corresponds to an LOD, the offset indication(s) (1110) may additionally indicate an LOD index (indicating a specific LOD) and indicate whether the quantization adjustment operation (e.g., reconfiguration adjustment operation) of the reconfiguration adjuster (1106) is enabled for that LOD index. If the first indication indicates that the reconfiguration adjuster (1106) is enabled, the offset indication(s) (1110) may additionally include a second indication of a quantization offset used for the quantization adjustment operation corresponding to the LOD.

[0140] In some embodiments, the reconfiguration adjuster (1106) is enabled by default, in which case the first indication is not signaled and only the second indication is signaled. In these embodiments, the quantization offset for a vertex subset (e.g., a specific LOD) may be signaled as equal to 0, which has the same effect as a quantization offset that is disabled or unused for the vertices of that subset.

[0141] In some examples, the first indication may be signaled for a 3D mesh (e.g., a mesh frame) or a series of mesh frames. In some examples, based on the first indication that quantization adjustment operation is possible, the reconfiguration adjuster (1106) may determine (e.g., set) the same quantization offset across all subsets (e.g., LODs) according to the second indication. In other examples, based on the first indication that quantization adjustment operation is possible, the reconfiguration adjuster (1106) may determine (e.g., set) the quantization offset for each subset (e.g., LOD) according to the second indication that is decoded for a specific subset. In some embodiments, the quantization offset value may be expressed in a fixed-point or floating-point form.

[0142] In some embodiments, the offset mark(s) (1110) may include a first mark and / or a second mark signaled to the decoder by the encoder. For example, as described above in FIG. 8, the encoder may generate and signal (e.g., encode) the offset mark(s) (1110) in the bitstream based on comparing the compression results between one or more de-quantization processes corresponding to one or more subsets that are disabled and enabled. For example, the encoder may signal the offset mark(s) (1110) to the decoder to enable / disable the reconstruction adjuster (1106) associated with each vertex subset to minimize the de-quantization error (e.g., maximize the compression gain). Accordingly, the inverse quantizer (1100A) (e.g., of a decoder) may apply (e.g., implement and / or perform) the operation of the reconstruction adjuster (1106) according to the offset indication(s) (1110) signaled by the encoder. Based on the fact that quantization adjustment is disabled for wavelet coefficients associated with a vertex subset, the inverse quantizer (1100A) may skip (e.g., omit) the reconstruction adjuster (1106), and the inverse scaler (1108) (e.g., LOD NIn this case, inverse quantization (e.g., inverse scaling) can be directly performed on the quantized wavelet coefficient signals of the vertices corresponding to the corresponding subset.

[0143] In some examples, the offset indicator(s) (1110) includes a single indicator that indicates whether to enable (e.g., disable or skip) quantization adjustment for all LODs or sequences of mesh frames of the 3D mesh. In some examples, the offset indicator(s) (1110) includes a single indicator that indicates one of the LODs for which the corresponding quantized wavelet coefficients should be adjusted (or not adjusted) in the quantization adjustment process. For example, the single indicator may indicate the lowest LOD level corresponding to the coarsest resolution (e.g., the last LOD or LOD0), and the associated quantization adjustment operation should be disabled. This is because inverse quantization may be useful in that the signal reconstructed with the LOD having a lower quantization level can be a sufficiently accurate representation of the original uncompressed signal.

[0144] In some examples, the offset indication(s) (1110) include indications for each LOD of the LOD associated with a vertex of the mesh frame. For example, one indication for one LOD may indicate whether the quantization adjustment operation for that LOD should be enabled or disabled. In the encoder, the encoder may determine whether the indication of the quantization adjustment operation signaled to the decoder in the bitstream is enabled or disabled by comparing the compression results between the quantization adjustment for the LOD being enabled and disabled. Then, the decoder may decode the indication for that LOD from the bitstream and optionally perform the quantization adjustment operation (of the reconfiguration adjuster (1106)) on the wavelet coefficients of the LOD according to the indication.

[0145] In some examples, the offset mark(s) (1110) include a mark for each LOD of the LOD associated with a vertex of the mesh frame. However, instead of an encoder comparing the compression results between quantization adjustment operations for the LODs, to determine whether a mark of a quantization adjustment operation is signaled, the encoder may compare the compression results between the activation / deactivation of a set of reconstruction adjustment operations corresponding to the LODs to determine a combination of marks that increases (e.g., maximizes) the compression gain. Similarly, although the above embodiments are described for LODs, they may similarly be applied to a subset (not necessarily LODs).

[0146] In some examples, the indication of the offset indication(s) (1110) may indicate an LOD index that identifies one of the LODs of a mesh frame for which quantization adjustment is enabled / disabled based on the indication. For example, the indication may include an LOD index and a binary indication (e.g., a binary flag) having a value indicating the enable / disable of the quantization adjustment operation corresponding to the LOD index.

[0147] In some examples, offset indication(s) (1110) may be signaled for every 3D mesh frame sequence, every mesh frame, every tile, every patch, every patch group, or every LOD. In some examples, one or more indications include indications that may be signaled for every LOD in a mesh frame.

[0148] In some embodiments, a first indication (e.g., mode indication) indicating whether quantization adjustment operation is enabled / disabled for each vertex subset is determined in advance without being signaled between the encoder and the decoder. For example, quantization adjustment operation for the wavelet coefficient signals of the vertices in all LODs may be enabled without being signaled in the offset indication(s) (1100).

[0149] In some embodiments, the offset indication(s) (1110) is an adjusted quantized wavelet coefficient (e.g., input signal It may include a second indication (e.g., a flag or syntax element) signaled in a bitstream representing a quantization offset used by the reconfiguration adjuster (1106) to determine (e.g., derive or compute) ). For example, the second indication may specify one of the quantization offsets, a specific value, or an index of a specific value by indicating an index to a set of quantization offsets (e.g., value n is (Expressing the index of ). In some examples, the second indication may be signaled for every sequence of 3D meshes, every mesh frame, every tile, every patch, every patch group, every LOD, etc.

[0150] In some embodiments, the offset indication(s) (1110) is an adjusted quantized wavelet coefficient (e.g., input signal It may include a third indication (e.g., a flag or syntax element) signaled in a bitstream representing a quantization offset used by a reconfiguration adjuster (1106) to determine (e.g., derive or compute) an adjusted quantization offset value for ). For example, the third indication may include an index for an offset set to adjust one of the quantization offsets, or an exponent of a specific value (e.g., value n is It can represent the exponent of.

[0151] In some embodiments, the offset display(s) (1110) may display a plurality of quantization offsets (1114A–B) corresponding to vertex subsets (1115A–B), respectively, as described above in relation to FIG. 10b and FIG. 10c. For example, a plurality of quantization offsets (1114A) may be signaled for vertex subsets (1115A), e.g., vertices of a first LOD. For example, a plurality of quantization offsets (1114B) may be signaled for vertex subsets (1115B), e.g., vertices of a second LOD. Each of the plurality of quantization offsets (1114A–B) may be signaled according to any one of the embodiments described above.

[0152] In some examples, the inverse scaler (1108) may determine an inverse scaling factor based on the scaling indicator(s) (1112). For example, the scaling indicator(s) (1112) may include one or more scaling parameters, such as a quantization parameter (qp) and / or a bit depth parameter (BDP), as further described below in FIG. 13. In some examples, one or more scaling parameters may be signaled (and associated with) for each subset of vertices (e.g., corresponding to the same LOD, within the same 3D mesh sequence, within a sub-mesh of a 3D mesh, within the same patch (or patch group) of a sub-mesh or 3D mesh, etc.). In some examples, the scaling parameters may be derived between vertex subsets (e.g., scaled by a fixed ratio value).

[0153] In some embodiments, the scaling indicator(s) (1112) may include one or more indicators indicating whether the quantization parameter (qp) and / or bit depth parameter (BDP) is signaled (e.g., present) in the bitstream. For example, one or more indicators may be signaled for each vertex subset. Similar to how the encoder can determine whether offset indicator(s) should enable or disable quantization adjustment, the encoder may determine one or more indicators indicating whether the quantization parameter (qp) and / or bit depth parameter (BDP) is signaled (e.g., present) in the bitstream. If one or more indicators indicate that these parameters are absent (i.e., not signaled in the bitstream), default values ​​for the quantization parameter (qp) and / or bit depth parameter (BDP) may be used.

[0154] Similar to the inverse quantizer (1100A), the inverse quantizer (1100B) may perform the inverse quantization method using a reconfiguration adjustment process according to some embodiments. The inverse quantizer (1100B) may be a dead zone inverse quantizer, as described above in FIGS. 10a and 10b. Like the subset-based iterator (1102A) and the displacement-based iterator (1104A), the inverse quantizer (1100B) includes the corresponding subset-based iterator (1102B) and displacement-based iterator (1104B), respectively. Unlike the inverse quantizer (1100A), the inverse quantizer (1100B) may include an inverse scaler (1116) and a reconfiguration adjuster (1118) that reverse the sequence (e.g., order) of operations performed by the reconfiguration adjuster (1106) and the inverse scaler (1108), as further described below in FIG. 13. Similar to the inverse quantizer (1100A), the inverse scaler (1116) may operate according to scaling indication(s) (1112), and similar to the reconfiguration adjuster (1106), the reconfiguration adjuster (1118) may operate according to offset indication(s) (1110).

[0155] FIG. 12 illustrates a flowchart (1200) of a method for applying a quantization scheme to transformed wavelet coefficients according to some embodiments. In some examples, the method may be performed by an encoder (e.g., the encoder (114) of FIG. 1, the encoder (200A) of FIG. 2a, or the encoder (200B) of FIG. 2b). The following description of various steps may refer to the operations described above in relation to the quantizer (212) of FIG. 2a or the quantizer (212) of FIG. 2b.

[0156] In block 1202, the encoder determines wavelet coefficients representing the displacement of a set of vertices of a three-dimensional (3D) mesh. For example, the wavelet coefficients may be determined by the encoder's wavelet transformer (e.g., the wavelet transformer (210) of FIG. 2a or FIG. 2b). As previously described, the encoder may convert (e.g., transform) the determined displacement of a set of vertices into wavelet coefficients according to a wavelet transform (e.g., a wavelet transform lifting scheme).

[0157] In block 1204, the encoder quantizes first wavelet coefficients corresponding to a subset of a vertex set according to a quantization scaling factor associated with the subset (e.g., associated with the quantization size s of FIGS. 8 through 10 and may be referred to as the scaling factor). In some examples, the quantization scaling factor is determined for each subset of non-overlapping subsets of the vertex set. Then, wavelet coefficients corresponding to vertices within each subset may be quantized according to each quantization scaling factor associated with the corresponding subset. For example, a vertex subset may include vertices in the same LOD among a plurality of LODs. For example, a vertex subset may include vertices in a sub-mesh of a 3D mesh. For example, a vertex subset may include vertices in a sub-mesh or a patch of a 3D mesh.

[0158] In some examples, wavelet coefficients (representing the original, unquantized values) Quantizing ) and quantized wavelet coefficients based on the quantization scaling factor (s_q) ) and possibly dead zone bias parameter( A dead zone offset (DZ) based on ) can be determined as follows:

[0159] =

[0160] The sgn function is the wavelet coefficient ( It is a sign function (e.g., a signum function) that outputs the sign of ), and The size of is an absolute function | It is indicated as |. If the dead zone offset is not used (or enabled), the dead zone offset (DZ) can be set to 0. The quantizer can determine a minimum integer less than or equal to the wavelet coefficient scaled according to the dead zone offset (which may be optional) by applying a floor function (as described above). In another example, the quantizer can determine a minimum integer greater than or equal to the wavelet coefficient scaled according to the dead zone offset by applying a ceiling function.

[0161] In some examples, the dead zone offset is the dead zone bias parameter that indicates (e.g., the corresponding) the dead zone offset ( It can be based on ), which is also illustrated above (e.g., 1 + DZ*s). In some examples, dead zone bias parameter( ) can be negative. In some examples, the width of the dead zone offset may refer to the area surrounding DZ or 0, e.g. 2*DZ, as described in FIG. 8.

[0162] Generally, since the quantization scaling factor (s_q) has a value of 0 (e.g., non-inclusive) to 1 (e.g., inclusive), the quantization wavelet coefficients will be smaller than the non-quantization wavelet coefficients. As illustrated in FIGS. 8 through 10, the quantization scaling factor (s_q) may be the reciprocal of the quantization size (s = 1 / s_q). In some examples, the determined dead zone offset may be smaller than the quantization size (s or the inverse quantization scaling factor).

[0163] In some examples, the quantization scaling factor (e.g., forward quantization scaling factor s_q) can be determined based on the quantization parameter (qp) and the bit depth parameter (bitDepthPosition or BDP). For example, the quantization scaling factor (scale[d]) for a wavelet coefficient (e.g., representing a displacement signal d) can be determined as a power of 2 according to the following relationship:

[0164] scale[d] = qp >= 0? power(2.0, 16 - BDP + (4 - qp) / 6) : 0.0

[0165] In some examples, quantization parameters (qp) and / or bit depth parameters (BDP) may be determined and / or signaled (e.g., encoded into a bitstream) for each subset of a set of vertices. In some examples, quantization parameters and / or bit depth parameters may be signaled for a 3D mesh (e.g., a 3D mesh frame) or a sequence of 3D meshes (e.g., containing a 3D mesh).

[0166] In some embodiments, the quantization scaling factor may be determined as a fixed ratio between subsets (e.g., LOD) such that the quantization scaling factor is scaled (e.g., increased or decreased) according to a fixed ratio per subset. In some examples, the quantization scaling factor, quantization parameter, and / or bit depth parameter may be derived from one subset to another subset(s).

[0167] In some examples, the displacement (or displacement residual) may be represented by three components, in which case a quantization scaling factor may be applied to each component. In a typical embodiment, the quantization scaling factor may be determined to be the same for each component.

[0168] In some examples, the encoder iteratively quantizes the wavelet coefficients of each subset of subsets of a set of vertices. For instance, if each subset corresponds to vertices of the same LOD, the encoder may iteratively quantize the wavelet coefficients per subset according to the order of LODs (e.g., from higher LODs to lower LODs).

[0169] In block 1206, the encoder encodes quantization information representing multiple quantization offsets associated with a subset in a bitstream for the quantized first wavelet coefficients. In some examples, the quantization information may additionally represent one or more parameters for quantization, such as quantization parameters and / or bit depth parameters. As previously mentioned, the quantization information may be signaled for each vertex subset.

[0170] In some examples, a plurality of quantization offsets may correspond to a plurality of non-overlapping intervals of values, as described above in relation to FIGS. 10b and 10c. For example, a plurality of quantization offsets may include a first quantization offset associated with a dead zone, which may correspond to a value of 0 for the inversely quantized wavelet coefficient. For example, a plurality of quantization offsets may include a second quantization offset associated with an interval of values ​​that excludes the dead zone, such as values ​​within a positive domain. Additionally, a plurality of quantization offsets may include a third quantization offset associated with an interval of values ​​that excludes the dead zone and the interval of values ​​associated with the second quantization offset. For example, the third quantization offset may be associated with values ​​within a negative domain.

[0171] In some examples, each quantization offset can be signaled for each vertex subset. In one example, one or more parameters may be associated with each quantization offset and included in the quantization information. For example, one or more parameters may include the sign of the quantization offset, the magnitude of the quantization offset, one or more thresholds, one or more quantization adjustments, etc.

[0172] In some examples, the encoder determines a quantization offset associated with a vertex subset to which the value belongs, based on the difference between the first wavelet coefficient and the first wavelet coefficient for the value of the first wavelet coefficient quantized in the interval of value. For example, the quantization offset may be determined as a measure of the center of the difference (e.g., mean, median, mode, etc.). In some examples, the quantization information includes a plurality of quantization offsets for each subset of non-overlapping subsets of the vertex set, where the subset is one of the non-overlapping subsets.

[0173] In some examples, the quantization information includes indicators of multiple quantization offsets (e.g., one or more indicators, flags, or syntax elements). In some examples, the indicators may include the sign and magnitude of each quantization offset.

[0174] In some examples, for each quantization offset, the indicator may include an index for selecting a quantization offset from a list of quantization offsets (e.g., an array or a table).

[0175] In some examples, the indicator (or mark) of the quantization offset can be entropy-coded using, for example, a unary code, a Rice code, a Colomb code, an exponential Colomb code, etc.

[0176] In some examples, quantization information includes an indication of whether quantization adjustment is enabled for each subset of (non-overlapping) vertex subsets (e.g., selectively enabling or disabling quantization adjustment per vertex subset). For example, if a subset corresponds to a single mesh frame (e.g., a 3D mesh) in a sequence of mesh frames, the indication may be signaled per mesh frame. For example, if a subset corresponds to an LOD, the indication may be signaled per LOD. For example, the indication (e.g., a mode indication for quantization adjustment) may be signaled per sequence of 3D mesh frames, per mesh frame, per sub-mesh, per tile, per patch group, per patch, and / or per LOD.

[0177] In some examples, the encoder may determine the indication by comparing the compression result of inverse quantization (e.g., rate-distortion optimization (RDO) cost) between the quantization adjustment operation corresponding to a subset (e.g., as described above in FIG. 10a through 10c, and below in FIG. 11) being disabled and the quantization adjustment operation being enabled. For example, if the compression gain increases when the quantization adjustment operation (e.g., also referred to as inverse quantization adjustment or reconfiguration adjustment) is enabled (e.g., fewer bits need to be generated in the displacement bitstream (260)), the encoder may determine the indication of the quantization adjustment operation to be enabled and may additionally indicate a second indication of the quantization offset in the quantization information. For example, the indication of the quantization adjustment may indicate whether the reconfiguration adjuster (e.g., the reconfiguration adjuster (1106) or the reconfiguration adjuster (1118) in FIG. 11 in the decoder) is enabled. If quantization adjustment is disabled (by indication), the operation of the reconfiguration adjuster may be disabled.

[0178] In some examples, the indication of the quantization adjustment operation can be entropy-coded using, for example, unary codes, Rice codes, Colomb codes, exponential Colomb codes, etc.

[0179] In some examples, the encoder indicates the dead zone offset in the quantization information (e.g., the DZ value, or the dead zone bias parameter ( It can signal )). In some examples, the indication of the dead zone offset can be entropy-coded using, for example, unary code, Rice code, Gollom code, exponential Gollom code, etc.

[0180] In some examples, the encoder has an offset adjustment value (a) as described in FIGS. 10a to 10c. δ ) additionally signals an indication of whether this is applied to the quantization offset. In some examples, the encoder may signal an indication of whether one or more thresholds are applied to the quantization offset, as described in FIGS. 10a through 10c. In other examples, the quantization offset signaled to the decoder by the encoder is determined such that no additional indication is signaled in the bitstream by applying an offset adjustment value (aδ) and / or one or more thresholds.

[0181] In block 1208, the encoder encodes quantized first wavelet coefficients to represent the first displacement among the displacements in the bitstream. As previously described, the encoder can quantize wavelet coefficients corresponding to each vertex subset. Then, the encoder can encode the quantized wavelet coefficients for each vertex subset.

[0182] In some examples, the encoder may signal (e.g., encode) quantized wavelet coefficients representing the displacement of a set of vertices in a 2D image. For example, wavelet coefficients for vertices of a 3D mesh may be arranged (e.g., packed) into a 2D image (e.g., displacement image (720) in Fig. 7a) by an image packer (e.g., image packer (214) in Figs. 2a and 2b). In some examples, the wavelet coefficients may be quantized by a quantizer (e.g., quantizer (212) in Figs. 2a and 2b) before being arranged by the image packer, as described in Figs. 2a and 2b. Additionally, the 2D image may be encoded by a 2D video codec, such as the video encoder (216) described in Figs. 2a and 2b. In some examples, quantized wavelet coefficients representing the displacement of a set of vertices can be compressed through arithmetic encoding.

[0183] FIG. 13 illustrates a flowchart (1300) of a method for applying an inverse quantization scheme to quantized transform wavelet coefficients according to some embodiments. In some examples, the method may be performed by a decoder (e.g., the decoder (120) of FIG. 1 or the decoder (300) of FIG. 3). For the following description of various steps, the operation described above may be referenced in relation to the inverse quantizer (222) of FIG. 2a and 2b or the inverse quantizer (312) of FIG. 3.

[0184] In block 1302, the decoder decodes quantized wavelet coefficients representing the displacement of vertices from a set of vertices of a three-dimensional (3D) mesh from a bitstream. As described above in relation to FIG. 11, the decoder can similarly receive and decode quantized wavelet coefficients representing the displacement (including the displacement of vertices) of a set of vertices.

[0185] In some examples, a set of vertices is aligned according to the level of detail (LOD) of the vertices, and quantized wavelet coefficients can be decoded from the bitstream according to the LOD. For example, a decoder can determine (unconverted) quantized wavelength coefficients containing quantized wavelength coefficients by applying an inverse wavelength transform (e.g., inverse wavelength transform) to the converted wavelength coefficients received in the bitstream in the above order.

[0186] In some examples, quantized wavelet coefficients are decoded based on decoding an image (e.g., a 2D image) containing wavelet coefficients transformed from a bitstream. The decoder can determine the transformed wavelet coefficients from the decoded image according to the packing order of the wavelet coefficients within the image. For example, as described above in FIG. 12, the quantized wavelet coefficients are arranged (e.g., packed) into a 2D image (e.g., the displacement image (720) in FIG. 7a) by an image packer in the encoder. Thus, the decoder may include a video decoder (e.g., the video decoder (308) in FIG. 3) that decodes a 2D image containing quantized wavelet coefficients. The decoder may include an image unpacker (e.g., the image unpacker (310) in FIG. 3) for reversing (e.g., unpacking) the operation of the image packer to determine the sequence of quantized wavelet coefficients. In some examples, the decoder may include an inverse quantizer (e.g., inverse quantizer (312)) to inversely quantize the unpacked first wavelet coefficients. In some examples, the quantized wavelet coefficients are decoded from the bitstream through arithmetic decoding.

[0187] In block 1304, the decoder decodes quantization information from the bitstream representing a plurality of quantization offsets associated with a vertex subset of a vertex set. For example, the quantization information may be the same as the quantization information encoded by the encoder in block (1206) of FIG. 12.

[0188] In some implementations, a set of vertices includes a non-overlapping vertex subset, and the subset is one of the non-overlapping subsets. In some implementations, the non-overlapping subset corresponds to a level of detail (LOD). In some examples, the non-overlapping subset corresponds to a submesh of a 3D mesh. In some examples, the non-overlapping subset corresponds to a patch within a submesh of one of the submeshes.

[0189] In some examples, multiple quantization offsets correspond to a sequence of 3D meshes containing 3D meshes. For example, a quantization offset may be associated with a subset of vertices across mesh frames of a sequence of 3D meshes, such as vertices across mesh frames in the same LOD of multiple LODs.

[0190] In some examples, the quantization information additionally represents a plurality of quantization offsets associated with each subset of non-overlapping subsets. For example, each subset may be associated with a different LOD among the LODs associated with the vertices of a 3D mesh, in which case the vertex subset contains only vertices from the first (same) LOD of the plurality of LODs, and the vertices are in the subset based on the vertices in the first LOD.

[0191] In some embodiments, the quantization information includes an indicator representing each of the plurality of quantization offsets as described above. In some examples, the indicator may include the sign and magnitude of the quantization offset. In some examples, the indicator selects a quantization offset from a list of quantization offsets (e.g., an array or a table) that can be signaled to a decoder by an encoder. For example, the indicator may be an index of the list.

[0192] In some embodiments, a plurality of quantization offsets may each be associated with a plurality of intervals of values, as described above in relation to FIG. 10b and FIG. 10c. For example, the plurality of intervals may include a range of non-overlapping values.

[0193] In some embodiments, the decoder receives (e.g., decoding) an indication from the bitstream of whether quantization adjustment is enabled (e.g., mode indication). Adjustment of wavelet coefficients may be based on an indication that quantization adjustment is possible.

[0194] In some examples, a mode indication may be received for each subset of non-overlapping vertex subsets. In some examples, a mode indication may be received (and decoded) for a sequence of 3D meshes containing 3D meshes or for each 3D mesh frame. Similar to the indication of a quantization offset, the one or more mode indications may be signaled for each sequence of 3D mesh frames, each mesh frame, or each sub-mesh of a mesh frame / 3D mesh, each patch group of a 3D mesh / mesh frame sub-mesh, each tile, each patch within a patch group, or for each vertex subset corresponding to each LOD. In some examples, based on a mode indication indicating no quantization adjustment, the reconfiguration adjuster (1102) may be disabled and the operation in block 1308 will be skipped (e.g., omitted).

[0195] In some examples, where a subset corresponds to different LODs, one or more mode indications may be decoded from the bitstream to selectively enable quantization adjustment for a specific LOD. For example, one or more mode indications may include an index of the LOD (e.g., identifying the LOD) and a binary indication of whether the quantization adjustment operation corresponding to the LOD is enabled or disabled (e.g., skipped).

[0196] In some examples, the received quantization offset is determined by the encoder. For example, the value of the quantization offset associated with a vertex subset can be determined (derived / calculated) from the difference between the displacement (or displacement residual) value before quantization after wavelet transform (lifting) and the displacement (or displacement residual) value before inverse wavelet transform (inverse lifting) after inverse quantization. In some examples, one or more quantization offset values ​​associated with a vertex subset are offset adjustment values ​​(a) as described above in FIGS. 10a through 10c. δ ) and / or may be further adjusted by one or more threshold values. In some examples, one or more of the quantization offset values ​​may be scaled to a fixed value based on a vertex subset, e.g., the LOD corresponding to the subset or the index of the LOD.

[0197] In another example, the decoder has an offset adjustment value (a) as described above in FIGS. 10a to 10c. δ ) and / or one or more thresholds can be applied independently. For example, quantization information is an offset adjustment value (a δ ) and / or it may indicate whether one or more thresholds are applied to one or more of the quantization offset values. One or more thresholds may be predetermined or signaled in the quantization information. Offset adjustment value (a δ ) can be derived based on the LOD associated with the vertex subset, or signaled from the quantization information.

[0198] In relation to the encoding of an indication of whether quantization adjustment is possible (e.g., mode indication) and / or the indication of the quantization offset described in FIG. 12, the decoder can perform entropy decoding to decode the mode indication and / or the indication of the quantization offset from the bitstream.

[0199] In block 1306, a decoder (e.g., the inverse quantizer (222) of FIG. 2a and 2b, or the inverse quantizer (312) of FIG. 3) determines wavelet coefficients by inversely quantizing the quantized wavelet coefficients. In some embodiments, to perform inverse quantization, the decoder determines an inverse quantization scaling factor associated with a subset (e.g., corresponding to the quantization size in FIG. 8 through 10). In some embodiments, the inverse quantization scaling factor is determined for each subset of non-overlapping subsets of the vertex set. Then, the quantized wavelet coefficients corresponding to the vertices within each subset can be inversely quantized according to each inverse quantization scaling factor associated with that subset. Then, the decoder determines wavelet coefficients by scaling the quantized wavelet coefficients based on the inverse quantization scaling factor.

[0200] In some examples, quantized wavelet coefficients (representing quantized, transformed wavelet coefficients) ) is the inverse quantized scaling factor (s_i) and quantization offset (e.g., or Inversely quantized wavelet coefficients based on ) It can be inversely quantized to determine ).

[0201] =

[0202] In some examples, the inverse quantization scaling factor (s_i) is the offset adjustment value a δ It can be determined by applying the scaling factor (s_i) to adaptively change its value, for example, according to the LOD level. For example, the offset adjustment value a δ It may include a scaling factor (s_lod) or an inverse scaling factor (s_lod_i). For example, s_i at LOD0 can be multiplied by s_lod (e.g., 1 / 2), so that the value of s_i at LOD1 becomes equal to half the value at LOD0.

[0203] The sgn function is the wavelet coefficient ( It is a sign function (e.g., a signum function) that outputs the sign of ), and The size of is an absolute function | It is displayed as |.

[0204] In some examples, the quantization offset (e.g., bias or shift) is the inversely scaled quantization wavelet coefficient ( Offset added to ) It can be.

[0205] In some examples, the quantization offset (e.g., bias or shift) is inverse scaling ( Offset added to quantized wavelet coefficients before becoming It can be.

[0206] In some examples, the quantization offset (e.g., bias or shift) is an offset adjustment value a δ Offset adjusted according to ( Since it can be, the offset value depending on the LOD level It changes adaptively.

[0207] In some embodiments, by shifting (e.g., adjusting) the inverse quantized wavelet coefficients, the output inverse quantized wavelet coefficients ( ) are the original wavelet coefficients in the encoder It can be even closer to.

[0208] In some examples, the inverse quantization scaling factor s_i corresponds to the reciprocal of the quantization scaling factor s_q applied by the encoder of FIG. 12 (e.g., s_i = 1 / s_q). In some examples, the inverse quantization scaling factor (e.g., s_i) may be determined based on the quantization parameter (qp) and / or the bit depth parameter (bitDepthPosition or BDP). For example, the inverse quantization scaling factor (iscale[d]) for a quantized wavelet coefficient (e.g., representing a displacement signal d) may be determined as a power of 2 according to the following relationship:

[0209] iscale[d] = qp >= 0? power(0.5, 16 - BDP + (4 - qp) / 6) : 0.0

[0210] In some examples, quantization parameters (qp) and / or bit depth parameters (BDP) may be determined and / or decoded for each subset of a set of vertices (e.g., from a bitstream). In some examples, quantization parameters and / or bit depth parameters may be decoded for a 3D mesh (e.g., a 3D mesh frame) or a sequence of 3D meshes (e.g., containing a 3D mesh).

[0211] In some examples, the inverse quantization scaling factor may be determined as a fixed ratio between subsets (e.g., LOD) so that the inverse quantization scaling factor scales (e.g., increase or decrease) according to a fixed ratio per subset. In these examples, the inverse quantization scaling factor, quantization parameters, and / or bit depth parameters are not signaled in the bitstream.

[0212] In some examples, the displacement (or displacement residual) may be represented by three components, in which case an inverse quantization scaling factor may be applied to each component. In a typical embodiment, the inverse quantization scaling factor may be determined to be the same for each component.

[0213] In some examples, the quantization information further indicates quantization parameters and / or bit depth parameters associated with the subsets. In some examples, the quantization information further indicates quantization parameters or bit depth parameters that can be signaled for each subset of non-overlapping vertex subsets.

[0214] In block 1308, the decoder determines the displacement (or displacement residual) of a vertex by adjusting the wavelet coefficients according to one of a plurality of quantization offsets based on whether the vertex is in a subset. The plurality of quantization offsets may be specific to the vertex subset.

[0215] In some examples, the decoder selects a quantization offset from a plurality of quantization offsets based on whether the value of the wavelet coefficient is within a range of values ​​associated with the selected quantization offset. For example, the plurality of quantization offsets may include a first quantization offset associated with a value of 0, a second quantization offset associated with a positive value, and a third quantization offset associated with a negative value. In this example, the decoder may select the first quantization offset based on the value of the wavelet coefficient being 0, the second quantization offset based on the value of the wavelet coefficient being greater than 0, or the third quantization offset based on the value of the wavelet coefficient being less than 0.

[0216] In some examples, wavelet coefficients are adjusted based on reconstructing the displacement (or displacement residual) by combining (e.g., adding) a selected quantization offset to the wavelet coefficients. For example, a quantization offset may be added to the magnitude of the wavelet coefficients to reconstruct the displacement (or displacement residual) such that the sign of the wavelet coefficients is the same as the sign of the adjusted wavelet coefficients.

[0217] In some examples, the decoder may further inverse transform the (inversely quantized) wavelet coefficients to reconstruct the displacement (or displacement residual) for a vertex. For example, the decoder (e.g., the inverse wavelet transformer (220) of FIGS. 2a and 2b, or the inverse wavelet transformer (314) of FIG. 3) may determine the reconstructed displacement (or displacement residual) by applying an inverse quantizer (e.g., an inverse lifting scheme) to the inversely quantized wavelet coefficients. In some examples, the decoder may repeatedly apply the inverse transform to each subset of the inversely quantized wavelet coefficients. For example, if the subset corresponds to LODs, the decoder may repeatedly apply the inverse transform to the inversely quantized wavelet coefficients in each LOD according to the order of the LODs (e.g., in increasing order). For the reconstructed displacement residual, the decoder may perform inter-frame prediction to determine the displacement.

[0218] In some embodiments, the decoder reconstructs the geometry of the 3D mesh based on determined displacements (and similarly determined displacements of vertices of the vertex set of the 3D mesh). In some examples, the decoder can reconstruct the geometry based on displacements and a base mesh. For example, the decoder can decode a base mesh associated with the 3D mesh from a bitstream. Then, the decoder can generate vertex locations of the subdivided base mesh by iteratively applying a subdivision method to the base mesh, where each subset of subdivisions is associated with an iteration of subdivision. To reconstruct the geometry of the 3D mesh, the decoder can add the vertex displacements to the corresponding vertex locations of the subdivided base mesh. The reconstructed 3D mesh can be determined after applying the determined displacements to the corresponding vertices of the subdivided base mesh.

[0219] Embodiments of the present disclosure may be implemented in hardware using analog and / or digital circuits, in software through the execution of instructions by one or more general-purpose or special-purpose processors, or as a combination of hardware and software. Consequently, embodiments of the present disclosure may be implemented in the environment of a computer system or other processing system. An example of such a computer system (1400) is shown in FIG. 14. As with the blocks in FIG. 1, the blocks shown in the figure may be executed in one or more computer systems (1400). Additionally, each step of the flowchart shown in the present disclosure may be implemented in one or more computer systems (1400). When more than one computer system (1400) is used to implement an embodiment of the present disclosure, the computer systems (1400) may be interconnected by one or more networks to form a cluster of computer systems that can act as a single pool of seamless resources. The interconnected computer systems (1400) may form a "cloud" of computers.

[0220] The computer system (1400) includes one or more processors, such as a processor (1404). The processor (1404) may be, for example, a special-purpose processor, a general-purpose processor, a microprocessor, or a digital signal processor. The processor (1404) may be connected to a communication infrastructure (1402) (e.g., a bus or a network). The computer system (1400) may also include a main memory (1406), such as random access memory (RAM), and an auxiliary memory (1408).

[0221] The auxiliary memory (1408) may include a hard disk drive (1410) and / or a removable storage drive (1412), such as a magnetic tape drive, an optical disk drive, etc. The removable storage drive (1412) may read from and / or write to the removable storage unit (1416) in a well-known manner. The removable storage unit (1416) represents a magnetic tape, an optical disk, etc., which is read by and written to by the removable storage drive (1412). As understood by those skilled in the art, the removable storage unit (1416) includes a computer-usable storage medium on which computer software and / or data is stored.

[0222] In an alternative embodiment, the auxiliary memory (1408) may include other similar means that enable a computer program or other instructions to be loaded into the computer system (1400). Such means may include, for example, a removable storage unit (1418) and an interface (1414). Examples of such means may include a program cartridge and a cartridge interface (e.g., found in a video game device), a removable memory chip (e.g., EPROM or PROM) and an associated socket, a thumb drive and a USB port, and other removable storage units (1418) and interfaces (1414) that enable software and data to be transferred from the removable storage unit (1418) to the computer system (1400).

[0223] The computer system (1400) may also include a communication interface (1420). The communication interface (1420) enables the transmission of software and data between the computer system (1400) and an external device. Examples of the communication interface (1420) may include a modem, a network interface (e.g., an Ethernet card), a communication port, etc. Software and data transmitted through the communication interface (1420) have electronic, electromagnetic, optical, or other signal forms that can be received by the communication interface (1420). These signals are provided to the communication interface (1420) through a communication path (1422). The communication path (1422) carries signals and may be implemented using a wire or cable, optical fiber, telephone line, mobile phone link, RF link, and other communication channels.

[0224] The computer system (1400) may also include one or more sensor(s) (1424). The sensor(s) (1424) may measure or detect one or more physical quantities and convert the measured or detected physical quantities into electrical signals in digital and / or analog form. For example, the sensor(s) (1424) may include an eye-tracking sensor for tracking the user's eye movements. Based on the user's eye movements, the display of the point cloud may be updated. In another example, the sensor(s) (1424) may include a head-tracking sensor for tracking the user's head movements. Based on the user's head movements, the display of the point cloud may be updated. In yet another example, the sensor(s) (1424) may include a camera sensor for capturing a picture and / or 3D scanning device, such as a laser scanning, structured light scanning, and / or modulated light scanning device. A 3D scanning device can obtain geometry by moving one or more laser heads, structured light, and / or modulated light cameras over an object or scene being scanned. The geometry information can be used to construct a point cloud.

[0225] As used herein, the terms “computer program medium” and “computer-readable medium” are used to refer to storage media of the same type, such as a hard disk installed in a hard disk drive (1410) or a removable storage unit (1416 and 1418). These computer program products are means for providing software to a computer system (1400). A computer program (also referred to as computer control logic) may be stored in main memory (1406) and / or auxiliary memory (1408). A computer program may also be received via a communication interface (1420). When executed, such a computer program enables the computer system (1400) to implement the present disclosure as discussed herein. In particular, when executed, the computer program enables the processor (1404) to implement the process of the present disclosure, such as any one of the methods described herein. Thus, such a computer program represents a controller of the computer system (1400).

[0226] In other embodiments, the features of the present disclosure may be implemented in hardware using hardware components, such as application-specific integrated circuits (ASICs) and gate arrays, for example. An implementation of a hardware state machine to perform the functions described herein will be apparent to those skilled in the art.

Claims

Claim 1 A method comprising: decoding quantized wavelet coefficients representing the displacement of a vertex from a set of vertices of a three-dimensional (3D) mesh from a bitstream; decoding quantization information from the bitstream representing a plurality of quantization offsets associated with a vertex subset of the set of vertices, wherein the plurality of offsets are associated with a plurality of intervals of values ​​of the vertex subset; determining wavelet coefficients by inversely quantizing the quantized wavelet coefficients; and determining the displacement of the vertex by adjusting the wavelet coefficients according to one of the plurality of quantization offsets based on the fact that the vertex is in the subset. Claim 2 A method comprising: decoding quantized wavelet coefficients representing the displacement of a vertex from a set of vertices of a three-dimensional (3D) mesh from a bitstream; decoding quantization information representing a plurality of quantization offsets associated with a vertex subset of the set of vertices from the bitstream; determining wavelet coefficients by inversely quantizing the quantized wavelet coefficients; and determining the displacement of the vertex by adjusting the wavelet coefficients according to one of the plurality of quantization offsets based on the fact that the vertex is in the subset. Claim 3 A method according to claim 1 or 2, wherein the vertex set comprises non-overlapping vertex subsets, and the subset is one of the non-overlapping subsets. Claim 4 In paragraph 3, the method, wherein the non-overlapping subsets correspond to each level of detail (LOD). Claim 5 In paragraph 3, the method wherein the non-overlapping subsets correspond to each submesh of the 3D mesh. Claim 6 In paragraph 3, the method wherein the non-overlapping subsets correspond to each patch within one of the submeshes. Claim 7 A method according to any one of claims 1 to 6, wherein the quantization information further represents a plurality of quantization offsets associated with each subset of the non-overlapping subsets. Claim 8 A method according to any one of claims 1 to 7, wherein the quantization information includes an indicator of each of the plurality of quantization offsets. Claim 9 A method according to any one of claims 2 through 8, wherein the plurality of offsets are associated with each of the plurality of intervals of the values ​​of the vertex subset. Claim 10 A method according to any one of claims 1 to 9, wherein the plurality of intervals include non-overlapping value ranges. Claim 11 A method according to any one of claims 1 to 10, wherein the plurality of offsets comprises: a first offset associated with a value of 0; and a second offset associated with a value of non-0. Claim 12 A method according to any one of claims 1 to 11, wherein the plurality of offsets comprises: a first offset associated with a value of 0; a second offset associated with a positive value; and a third offset associated with a negative value. Claim 13 A method according to any one of claims 1 to 12, wherein the adjusting comprises: selecting the quantization offset from the plurality of quantization offsets based on the fact that the value of the wavelet coefficient is in an interval of values ​​associated with the selected quantization offset. Claim 14 A method according to any one of claims 1 to 13, wherein the adjusting further comprises: determining the adjusted wavelet coefficients based on adding the quantization offset to the wavelet coefficients. Claim 15 A method according to any one of claims 1 to 14, further comprising the step of receiving from the bitstream an indication of whether quantization adjustment is enabled, wherein adjusting the wavelet coefficients is based on the indication that the quantization adjustment is enabled. Claim 16 In paragraph 15, the above set of vertices includes non-overlapping vertex subsets, and the above indication is received for each subset of the non-overlapping subsets, a method. Claim 17 In paragraph 15, the above indication is received for a 3D mesh sequence including the 3D mesh. Claim 18 A method according to any one of claims 1 to 17, wherein the inverse quantization comprises: determining an inverse quantization scaling factor based on a quantization parameter; and determining the wavelet coefficients by scaling the quantized wavelet coefficients based on the inverse quantization scaling factor. Claim 19 In paragraph 18, the above-mentioned inverse quantization scaling factor is additionally based on a bit depth parameter, a method. Claim 20 A method according to claim 19, wherein the quantization information further represents the quantization parameter or the bit depth parameter associated with the subset. Claim 21 A method according to any one of claims 1 to 20, wherein the vertex set comprises non-overlapping vertex subsets including the subset, and the quantization information further represents a quantization parameter or bit depth parameter for each subset of the non-overlapping subsets. Claim 22 A method according to any one of claims 1 to 21, wherein the set of vertices is aligned according to the level of detail (LOD) of the vertices, and the step of decoding the quantized wavelet coefficients for the vertices comprises: applying an inverse wavelet transform to the transformed wavelet coefficients received in the bitstream in sequence to determine the quantized wavelet coefficients including the quantized wavelet coefficients. Claim 23 A method according to any one of claims 1 to 22, further comprising the step of reconstructing the geometry of the 3D mesh based on the determined displacement. Claim 24 A method according to any one of claims 1 to 23, further comprising: a step of decoding a base mesh associated with the 3D mesh; and a step of repeatedly applying a subdivision method to the base mesh to generate the positions of the vertices of the subdivided base mesh, wherein each subset of the subsets is associated with the repetition of the subdivision. Claim 25 In claim 24, the step of reconstructing the geometry comprises: adding the displacement of the vertex to the position of the corresponding vertex of the subdivided base mesh. Claim 26 A method according to any one of claims 1 to 25, wherein the step of decoding the quantized wavelet coefficients comprises: decoding an image containing transformed wavelet coefficients from the bitstream; and determining the transformed wavelet coefficients from the decoded image according to the packing order of the wavelet coefficients in the image. Claim 27 In claim 26, the step of decoding the quantized wavelet coefficients further comprises: the step of applying an inverse wavelet transform to the transformed wavelet coefficients to determine quantized wavelet coefficients including the quantized wavelet coefficients. Claim 28 A method according to any one of claims 1 to 27, wherein the quantized wavelet coefficient is the quantized wavelet coefficient of the displacement of the vertex. Claim 29 A method according to any one of claims 1 to 27, wherein the quantized wavelet coefficient is a quantized wavelet coefficient for the displacement residual of the vertex. Claim 30 A method according to claim 29, wherein the quantized wavelet coefficients for the displacement residual of the vertex are determined by: determining the difference by subtracting a reference wavelet coefficient from the wavelet coefficient of the displacement of the vertex; and generating the quantized wavelet coefficients for the displacement residual by quantizing the difference. Claim 31 A method comprising: determining wavelet coefficients representing displacements of a set of vertices of a three-dimensional (3D) mesh; quantizing first wavelet coefficients corresponding to a subset of the set of vertices according to a quantization scaling factor associated with the subset; and encoding quantization information representing a plurality of quantization offsets associated with the subset in a bitstream for the quantized first wavelet coefficients. Claim 32 A method according to claim 31, wherein the plurality of quantization offsets are determined based on the differences between the wavelet coefficients corresponding to the subset and the quantized first wavelet coefficients. Claim 33 A method according to claim 31 or 32, wherein the quantization information is signaled for each subset of the vertex set. Claim 34 A method according to any one of claims 31 to 33, wherein the vertex set comprises non-overlapping vertex subsets, and the subset is one of the non-overlapping subsets. Claim 35 In paragraph 34, the above non-overlapping subsets correspond to each level of detail (LOD), a method. Claim 36 In paragraph 34, the method wherein the non-overlapping subsets correspond to each submesh of the 3D mesh. Claim 37 In paragraph 34, the above non-overlapping subsets correspond to each patch within one of the submeshes, a method. Claim 38 A method according to any one of claims 31 to 37, wherein the plurality of offsets are associated with each of the plurality of intervals of the values ​​of the vertex subset. Claim 39 In paragraph 38, the method wherein the plurality of intervals include non-overlapping value ranges. Claim 40 A method according to claim 38, wherein the plurality of offsets comprises: a first offset associated with a value of 0; and a second offset associated with a value of non-0. Claim 41 In paragraph 38, the plurality of offsets comprises: a first offset associated with a value of 0; a second offset associated with a positive value; and a third offset associated with a negative value. Claim 42 A method according to any one of claims 31 to 41, wherein the wavelet coefficients representing the displacements are the wavelet coefficients of the displacements of the set of vertices. Claim 43 A method according to any one of claims 31 to 41, wherein the wavelet coefficients representing the displacements are wavelet coefficients for the displacement residuals of the set of vertices. Claim 44 A method according to paragraph 43, wherein the wavelet coefficients for the displacement residuals of the set of vertices are determined by subtracting the reference wavelet coefficients of the displacements from the respective wavelet coefficients of the set of vertices. Claim 45 A non-transient computer-readable medium comprising instructions that cause the device to perform the method of any one of claims 1 through 44 when executed by one or more processors of the device. Claim 46 An encoder comprising one or more processors as an encoder; and a memory for storing instructions, wherein when the instructions are executed by the one or more processors, the encoder causes the encoder to perform the method of any one of claims 31 to 44. Claim 47 A non-transient computer-readable recording medium for storing a bitstream generated by a method according to any one of claims 31 to 44 for encoding a video. Claim 48 A decoder comprising: one or more processors as a decoder; and a memory for storing instructions, wherein when the instructions are executed by the one or more processors, the decoder causes the decoder to perform the method of any one of claims 1 to 30. Claim 49 A non-transient computer-readable medium storing a bitstream that, when decoded by a decoder, causes the decoder to perform a method according to any one of claims 1 to 30. Claim 50 A bitstream generated in accordance with any one of paragraphs 31 through 44.

Images