3D motion map for compressing time-varying grid texture

Abstract

A method and apparatus for encoding and decoding time-varying texture 3D meshes in a data stream is provided. By exploiting 3D motion information from a base mesh encoder, 3D motion maps are generated that can be derived into different types of maps, such as motion amplitude. Those maps have the property of a bijective mapping between object texture maps and motion texture maps. Since 3D movement cannot be directly used as a motion estimator for video encoding, the amplitude of the 3D motion is interpreted in order to construct an importance map that is converted into a QP map provided to video texture encoders and decoders.

Description

1. Technical Field This principle generally relates to the field of encoding, decoding, and rendering volumetric video represented as changing 3D meshes, changing texture images, and geometric motion information. Specifically, this principle relates to reducing the complexity of encoding and decoding dynamic texture images for transmission over a network. 2. Background Technology This section aims to introduce the reader to various aspects of the principles described and / or claimed below. This discussion is intended to provide background information to facilitate a better understanding of the various aspects of the principles. Therefore, it should be understood that these statements should be interpreted in this context, rather than as an admission of prior art.

[0003] Recent volumetric video encoders and decoders have proposed a motion detection stage in which two mesh variants are considered as candidates for each frame: the original base mesh and the base mesh. Alternatively, use the base mesh of the previous frame's topology. In the second case, vertex motion encoding is used, and the topology is skipped because a topology from the previous frame is used. However, time-varying texture images cannot benefit from motion encoding.

[0004] Currently, there is a lack of a time-varying texture mesh (TVTM) codec with better compression gain. Based on this principle, such a codec uses video coding guided by "3D motion" of animated texture maps (i.e., dynamic photometric properties) for animated meshes (i.e., dynamic geometry). 3. Summary of the Invention The following is a brief overview of the principle to provide a basic understanding of some aspects of it. This invention is not an exhaustive summary of the principle. It is not intended to identify key or essential elements of the principle. The following overview presents some aspects of the principle in a simplified form as an introduction to the more detailed description provided below.

[0006] This principle relates to a method for encoding 3D meshes in a sequence of 3D meshes within a data stream. The method involves deriving a motion map from the 3D meshes based on different 3D meshes in the sequence. These different 3D meshes can be a previous 3D mesh in the sequence, an I-reference 3D mesh, or any other encoded mesh. A mapping is then generated between the motion magnitude of the motion map and quantization parameters (QPs) to obtain a per-block QP table. A texture map is generated using the per-block QP table. The 3D meshes and texture maps are encoded in the data stream. In a variant, the motion map derivation is performed from triangular-by-triangle metadata indicating the surface regions to be encoded with the highest quality.

[0007] This principle also relates to an apparatus including a processor and memory associated with the processor, wherein the processor is configured to implement the above-described method.

[0008] This principle also relates to a method that includes decoding a 3D mesh from a data stream. A motion map is derived from the 3D mesh based on different 3D meshes in the sequence. These different 3D meshes can be a previous 3D mesh in the sequence, an I-reference 3D mesh, or any other encoded mesh. A mapping is generated between the motion magnitude of the motion map and quantization parameters (QPs) to obtain a block-wise QP table. Then, a texture map associated with the 3D mesh is decoded from the data stream using the block-wise QP table.

[0009] This principle also relates to an apparatus including a processor and memory associated with the processor, wherein the processor is configured to implement the above-described method. 4. Description of the attached drawings This disclosure will be better understood after reading the following description, which refers to the accompanying drawings, in which: Figure 1 An example is shown of the original mesh frame along with the corresponding patch UV atlas (i.e., the parameterization of the mesh) and the associated texture map with inter-patch fill (i.e., color gradients that fill the blank spaces between patches); Figure 2 shows the base mesh, which has a new UV atlas and texture map obtained after attribute transfer; Figure 3 An exemplary architecture of an apparatus configured to implement an encoding method and / or a decoding method according to embodiments of this principle is shown; Figure 4 An example of an embodiment of the syntax for encoding volumetric video into a data stream of time-varying textured meshes according to this principle is shown; Figure 5 Midpoint subdivision and motion interpolation are shown. 5. Detailed Implementation The principles of the invention will now be described more fully below with reference to the accompanying drawings, in which examples of the principles of the invention are shown. However, the principles of the invention can be embodied in many different forms and should not be construed as limited to the examples set forth herein. Therefore, while the principles are readily available in various modifications and alternatives, specific examples of the principles are shown by way of example in the drawings and will be described in detail herein. However, it should be understood that the principles are not intended to be limited to the specific forms disclosed; rather, this disclosure will cover all modifications, equivalents, and alternatives falling within the spirit and scope of the principles as defined by the appended claims.

[0012] The terminology used herein is for the purpose of describing particular examples only and is not intended to limit the principles herein. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that the terms “comprising” and / or “including” as used in this specification specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Furthermore, when an element is referred to as “in response to” or “connected to” another element, it may be directly “in response to” or “connected to” the other element, or there may be intermediate elements. Conversely, when an element is referred to as “directly in response to” or “directly connected to” another element, there are no intermediate elements. As used herein, the terms “and / or” include any and all combinations of one or more of the associated listed items and may be abbreviated to “ / ”.

[0013] It should be understood that although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the teachings of this principle.

[0014] Although some diagrams include arrows along the communication path to indicate the main communication direction, it should be understood that communication may occur in the opposite direction to the arrows depicted.

[0015] The block diagrams and operation flowcharts describe some examples, where each block represents a circuit element, module, or code section comprising one or more executable instructions for implementing a specified logical function. It should also be noted that in other implementations, the functions mentioned in a block may occur out of order. For example, depending on the functions involved, two blocks shown consecutively may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order.

[0016] The phrases “according to an example” or “in an example” used in this document mean that a particular feature, structure, or characteristic described in connection with that example may be included in at least one implementation of this principle. The phrases “according to an example” or “in an example” appearing in different places in the specification do not necessarily refer to the same example, nor are they necessarily separate or alternative examples that are mutually exclusive with other examples.

[0017] The reference numerals appearing in the claims are merely illustrative and should not be construed as limiting the scope of the claims. Although not explicitly described, examples and variations of the invention may be used in any combination or sub-combination.

[0018] Figure 1 Figure 2 shows an example of the original mesh 10 frame along with the corresponding patch UV atlas 11 (i.e., the parameterization of mesh 10) and the associated texture map 12 with inter-patch fill (i.e., color gradients filling the blank spaces between patches). UV parameterization provides a connection between the mesh vertices in 3D space and the color values ​​in the texture map's 2D space. Fill is typically added to aid in the spatial encoding of the texture map, especially at the edges of patches. Fill also results in better rendering of objects during content consumption. Like texture map 12, attributes of texture map 12 are passed. At the same step, other attributes such as normals, opacity, or reflectivity can be obtained and passed. Figure 2 shows a base mesh 20 with a new UV atlas 21 and texture map 22 obtained after attribute passing. Base mesh 20 is obtained by decimating the original mesh, or by decimating the base mesh of the previous frame (in the example...). Figure 1 The texture map 22 is obtained by applying motion to the base mesh 20. The texture map 23 also includes padding applied after attribute transfer. After refining the base mesh 20 and shifting its vertices using wavelet transform, a reconstructed deformable mesh 23 is obtained. The corresponding UV map 24 can then be obtained. The texture map 25 is the texture map 22 of the base mesh.

[0019] Based on this principle, a Time-Varying Texture Mesh (TVTM) codec is proposed to enhance compression gain in video coding by introducing “3D motion”-guided video coding of animated texture maps (dynamic photometric properties) from animated meshes (dynamic geometry). This is achieved by leveraging 3D motion information from the underlying mesh encoder to generate 3D motion maps that can be derived into different types of maps, such as motion amplitude. These maps possess the property of a bijective mapping between object texture maps and motion texture maps. 3D motion does not represent the full motion of pixels in a textured video. In fact, some movement on a texture frame can be due to mesh deformation, and some movement can be inherent to the texture and, for example, due to lighting variations (e.g., reflections or light flicker) during model capture. Ultimately, the object texture map may contain some motion due to geometric changes and some due to photometric variations. Therefore, 3D motion cannot be directly used as a motion estimator for video coding. This technique is used to reduce texture quality when object motion is important. In practice, when objects move very quickly, blurring is tolerable during rendering, thus allowing for a reduction in texture quality. Therefore, according to this principle, the amplitude of 3D motion is interpreted to construct an importance graph, which, in the case of a video texture encoder / decoder, is converted into a quantization parameter graph (QP graph). In the following embodiments, the video encoder and decoder are able to ingest a block-by-block QP table, with each block encoded or decoded using its associated specific QP, thus not uniformly across the entire frame.

[0020] In this embodiment, an encoding method using a motion graph as an importance graph to determine the QP graph is proposed. In a first step, the motion graph is derived from a 3D mesh. In a variant, this first step is performed from triangular metadata indicating the surface regions to be encoded with the highest quality (e.g., to preserve high-quality faces or hands). In a second step, the 3D mapping between motion amplitude and QP is managed to obtain a block-wise QP table. And in a third step, the texture map is encoded using the QP table.

[0021] In this embodiment, a canonical approach is proposed at the decoder side. The motion map is derived from a 3D mesh, and in a variant, from triangular metadata representing surface regions (such as faces or hands) to be encoded with the highest quality. A mapping between motion amplitude and QPs is established to obtain a block-wise QP table, similar to that used in the encoding phase. The texture map is then decoded using the QP table.

[0022] Based on this principle, the video encoder / decoder used to encode the texture map can ingest frame-by-frame information (i.e., a QP map), which associates each block of the video with a specific QP value, rather than using a global QP for the entire frame and the entire video. This information is ingested by the encoder to perform compression and by the decoder to perform decompression. This approach allows the frame-by-frame QP map (i.e., block-by-block QP metadata) to not be encoded in the bitstream, as it is regenerated during decoding before its texture map decoding phase. When encoding each new frame of the volumetric video (mesh + texture map), if the current frame is detected as static, it is treated as intra-frame, and a motion map with all zeros (no motion) is generated, but for filled regions, it is set to maximum motion (no need to preserve quality for filled regions). If the current frame is detected as dynamic, it is treated as inter-frame, and a motion map is generated using motion data computed by a 3D geometry codec.

[0023] For each new frame, the motion map is then converted into an importance map. This importance map is then downsampled to obtain a per-video-block importance value. The per-video-block importance value is converted into a 2D QP table (one QP per video block) and ingested by a video encoder for adaptively encoding the animated texture map. During decoding, similar steps are performed, using the decoded 3D geometry to reconstruct the 2D QP table and adaptively decode the texture map. In another embodiment, an extension is proposed that uses marked faces to enforce high texture quality on specific regions of the mesh surface.

[0024] The generated image is a 2D array, and its size can be the same as the texture image (as discussed later, it is not mandatory). Each pixel of this motion map... It contains unquantized 3D vectors that store 3D motion information. When processing between frames, the base mesh is encoded using a set of motion vectors, each modifying the vertices of the base mesh from the previous frame (e.g., the previous frame or a reference I-frame). The rest of the mesh is defined using topological and texture UV coordinates from the I-frame. Based on this principle, the UV coordinates of the base mesh are used to reconstruct the motion map, and the UV coordinates of the refined (fully reconstructed) model are used to interpolate missing 3D motion information for pixels in the projected triangles where no vertices are projected.

[0025] Figure 5 The diagram illustrates midpoint subdivision and motion interpolation. Triangles m1, m2, and m3 of the base mesh are retrieved. Midpoint refinements m'31, m'12, and m'23 with motion interpolation are introduced. On the right, Figure 8 illustrates a two-step subdivision. Motion is reconstructed for all vertices from the base mesh refinement (called mesh subdivision) by interpolating the motion vectors associated with the vertices of the base mesh. In some models, such as the VMesh test model, the base mesh is refined (subdivided) using a midpoint recursive subdivision scheme (other subdivision schemes can be used, thus adjusting the motion vectors). During subdivision, the system uses the UV coordinates of the vertices to interpolate the new UV coordinates at the newly generated vertices according to Formula Eq1.

[0026] Eq1:

[0027] in and It is a vector representing the 2D coordinates in UV space at the vertices of the base mesh triangle. For example, using a similar method according to formula Eq2, some interpolated motion is generated at the newly generated vertices.

[0028] Eq2:

[0029] in and This is a vector representing the 3D motion at the vertices of the base mesh triangle. In this example, two opposite vectors of similar magnitude will cancel each other out, and at the new vertex... Zero motion is generated at the point, as expected. Once the per-vertex motion values ​​for all vertices of the refined mesh are obtained, the mesh (i.e., its per-vertex UV coordinates and per-vertex motion values) is used to generate a motion map. This is done by rendering each triangle of the mesh into the texture image space, setting each motion pixel, which is part of the projection of the triangle, to the motion value obtained by interpolation based on the three motion values ​​from the vertices of the triangle.

[0030] The entire mesh is processed as follows: -with zero movement Initial size is The motion graph.

[0031] -The size is The Boolean occupancy graph is initialized to false.

[0032] -For texture coordinates , and and motion vectors , and Each triangle T of the subdivided grid.

[0033] o use , and To calculate the bounding boxes of triangles in image space.

[0034] o for each moving pixel in the box Calculate the coordinates of the center C of pixel P in UV space.

[0035] Use its UV coordinates to find the barycenter coordinates of C in the space of triangle T. The barycenter coordinates establish the connection between the different spaces (image and UV) in which T is located. It also allows us to know... coordinate Is it per vertex? coordinate , and It is part of a triangle.

[0036] If C is covered by the projection of T in image space (tested using barycentric coordinates), then Using barycentric coordinates to... , and Perform interpolation to calculate

[0037]

[0038]

[0039] Centroid coordinates allow for the location of points within the space of a triangle. Multiple related properties can be defined using this type of coordinate system. The dimension can be found using formula Eq3. The pixels of the image The UV coordinates of the center C.

[0040] Eq3:

[0041] And according to formula Eq4, find the value for the triangle ( , , 2D coordinates The centroid coordinates of point C .

[0042] Eq4:

[0043] Therefore, in order to test with a given centroid coordinate Whether point C (represented in the space of triangle T) lies inside the triangle depends on the following condition:

[0044] In the variant, since the VDMC subdivision is a simple midpoint subdivision, the same projection and interpolation methods can be used to derive the motion map before the subdivision stage. This will produce similar results. In the described embodiment, motion and UV coordinate interpolation are performed at the same VDMC block. For an efficient implementation, in the variant, the algorithm uses the base mesh instead of the refined mesh, and motion is not interpolated during subdivision because it has been used previously. Conversely, if the subdivision introduces new points in a non-linear manner, or if the UV coordinates are modified with some shift (or other fitting) after subdivision, for example, then the primary embodiment should be used, and the calculation of the motion vectors of the introduced vertices can also be adjusted according to the specific subdivision / fit.

[0045] According to this principle, the quality of the filler regions between patches is unimportant. Their role is to aid spatial encoding. Therefore, the regions between patches are set to high motion values, which causes the encoder to ignore the quality of these regions. The magnitude map is generated based on the following algorithm: set up

[0046] For each pixel

[0047] if

[0048]

[0049] otherwise

[0050] You can use values ​​greater than zero. Will This is set to a higher value, thus systematically assigning a "high pseudo-motion" to the gradient region, a value that is always higher than the motion of other pixels. The magnitude map is then further passed to create a normalized importance map according to formula Eq5.

[0051] Eq5:

[0052] Setting importance to 0 means that the pixel may experience a significant quality degradation due to its large movement. Conversely, setting importance to 1 means that the pixel's quality must be preserved as much as possible.

[0053] Using the proposed principles, it is helpful to generate motion maps and importance maps with resolutions lower than (or even higher than) texture maps. Simply set different parameters in the previous formula. and However, the proposed method involves sampling, which reduces the size of the motion map, collects motion at specific points, and may not reflect the average motion of the region. To overcome this, according to the second embodiment, a motion map and / or importance map are generated at the original texture map resolution, and then a low-pass filter is used to summarize the square regions of the map to obtain a lower-resolution map. For example, these lower-resolution maps may present a summary pixel for each video coding block (e.g., 8x8, 16x16, or 32x32) of the original map. Using, for example, a minimum filter for each block allows the coding quality on patch edges to be preserved, the quality on fully filled blocks to be discarded, and the quality on other blocks to be adapted to the lowest motion within the block. Other types of filters can be envisioned to maintain contour quality even when boundary blocks contain large motion. Even if the blocks have strong motion, boundary details can be preserved.

[0054] At this step, set The downsampled graph is generated, and a value between 0 and 1 is provided for each block of the video. To construct a QP graph (2D table) for the video encoder / decoder (one entry per video block), the target is determined for the video encoder. and the maximum around the target QP These values ​​can be set by the operator or determined automatically, for example according to formula Eq6 or Eq7, depending on whether the video encoder / decoder uses higher or lower quality for a lower QP.

[0055] Eq6:

[0056] Eq7:

[0057] In some cases, such as when encoding character performance, some surface areas may still require high quality even with fast movement (e.g., maintaining high quality on moving faces or hands). In such embodiments, an additional step is performed to force zero motion on some triangles of the mesh. For this purpose, the 3D model may be preprocessed, for example, with face or hand detection, and the underlying mesh triangles covered by this information may be marked so that the 3D motion generator always treats these triangles as static (zero motion), thus using the highest quality during encoding / decoding. For example, FaceId=0 may be used for faces that must be skipped by motion, and FaceId=1 may be used for other faces. The preprocessor (face detector, etc.) only provides this additional information to the encoder. These FaceIds can then be encoded / decoded using the FaceId mechanism. Thus, during motion map generation, if an active face skipping option is provided for a FaceId, then those faces with FaceId=0 are set to motion=0. During decoding, once the grid is decoded, if the face skip option is activated, the decoded FaceId will be used by the motion graph generator in the same way.

[0058] Figure 3 An exemplary architecture of a device 30, which can be configured to implement encoding and / or decoding methods according to embodiments of this principle, is shown. The device is connected to other devices via its bus 31 and / or via I / O interface 36.

[0059] Device 30 includes the following components connected together via data and address bus 31: Processor 32 (or CPU), for example, the processor is a DSP (or digital signal processor). ROM (or read-only memory) 33; RAM (or random access memory) 34; Storage interface 35; I / O interface 36 for receiving data to be sent from the application; and Power source (not shown in Figure 2), such as a battery.

[0060] According to the example, the power supply is external to the device. In each of the mentioned memories, the term "register" used in the specification can correspond to a small area (a few bits) or a very large area (e.g., the entire program or a large amount of received or decoded data). ROM 33 includes at least the program and parameters. ROM 33 can store algorithms and instructions to execute techniques according to these principles. When powered on, CPU 32 uploads the program to RAM and executes the corresponding instructions.

[0061] RAM 34 includes a program executed by CPU 32 and uploaded after device 30 is powered on, input data, intermediate data for different states of the method, and other variables used to execute the method.

[0062] The implementations described herein can be implemented in, for example, methods or processes, devices, computer program products, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., discussed only as a method or device), the implementations of the features in question can be implemented in other forms (e.g., programs). Devices can be implemented, for example, with appropriate hardware, software, and firmware. Methods can be implemented in, for example, devices, such as processors, which generally refer to processing devices, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0063] Device 30 is connected, for example, via bus 31 to a set of sensors 37 and a set of rendering devices 38. Sensors 37 may be, for example, a camera, microphone, temperature sensor, inertial measurement unit, GPS, humidity sensor, IR or UV light sensor, or wind sensor. Rendering devices 38 may be, for example, a display, speaker, vibrator, heater, fan, etc.

[0064] According to the example, device 30 is configured to implement methods for encoding, decoding, and rendering 3D scenes or volumetric videos in accordance with this principle, and belongs to a set that includes the following: Mobile devices; communication devices; Gaming devices; Tablet computer (or tablet PC); Laptop computers; Still image camera; Camera.

[0065] Figure 4 An example of an embodiment of a syntax for encoding volumetric video into a time-varying textured mesh data stream according to this principle is shown. The structure contains containers that organize the stream into individual syntax elements. The structure may include a header portion 41, which is a set of data common to each syntax element of the stream. For example, the header portion includes some metadata about the syntax elements, describing the properties and function of each of the syntax elements. The structure also includes a payload comprising syntax elements 42 and 43. Syntax element 42 includes data representing media content items, including the encoded geometry of the sequence and texture images. Syntax element 43 is part of the payload of the data stream and includes metadata according to this principle, such as the subsampling ratio.

[0066] The syntax presented in this document is an extension of the V3C syntax, which is a reference syntax for volumetric video represented by time-varying texture meshes. Additional information to be stored in the bitstream according to this principle includes indications (e.g., flags) of whether motion graphs are used to guide the encoding / decoding of texture graphs. Parameters. Local triangle constraints require additional information to activate / deactivate faceSkip. Of course, other syntaxes can be used.

[0067] In V3C, the `vdmc_ext_motion_maps_flag` syntax element exists in the bitstream to indicate whether motion maps are used for the texture video stream (i.e., attribute video). If enabled, other parameters can exist, which can be used to derive parameters such as importance, parameters required for QP calculations for the texture video stream, and local triangular-by-triangle constraints. This syntax element is associated with the sequence of dynamic mesh streams and requires information to be expressed to the V-DMC decoder so that the underlying mesh decoder explicitly generates motion maps. Therefore, this syntax element signals at the sequence parameter set level in the atlas bitstream of the dynamic mesh sequence (i.e., the Atlas Sequence Parameter Set (ASPS)). Since metadata needs to be defined for the V-DMC bitstream in the V3C syntax, there is an extension to the ASPS. This syntax can be included in the V-DMC extension. Additional parameters associated with the motion map can be signaled at the patch data cell level or the ASPS extension level. Including these parameters in the ASPS extension is efficient if the motion map parameters remain constant throughout the sequence and sub-mesh.

[0068] In the atlas frame sequence set attribute tile information structure, additional syntax elements for signaling QP changes (var) have been added based on this principle. Possible syntax is shown below.

[0069]

[0070]

[0071] A mesh can define multiple per-face / per-vertex attributes. To facilitate signaling which faces (i.e., a group of faces) are motion constraints, a syntax element, `bmsps_motion_constraints_facegroup_attribute_index`, exists in the base mesh sequence parameter set. This face group attribute index is common to the entire base mesh sequence.

[0072]

[0073] Specific additional sub-layers may exist, which can have different attribute indices for the face groups of motion constraint information. Each sub-layer can signal the face groups of motion constraint information using attribute indices different from those of other sub-layers. Therefore, each sub-layer can signal its own face group attribute index for identification. If none exists, the base sub-layer face group attribute index of the motion constraint information is used.

[0074] The implementations described herein can be implemented in, for example, methods or processes, devices, computer program products, data streams, or signals. Even if discussed only in the context of a single implementation (e.g., only as a method or device), the implementations of the features in question can be implemented in other forms (e.g., programs). Devices can be implemented, for example, with appropriate hardware, software, and firmware. Methods can be implemented in, for example, devices, such as processors, which generally refer to processing devices, including, for example, computers, microprocessors, integrated circuits, or programmable logic devices. Processors also include communication devices, such as smartphones, tablets, computers, mobile phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0075] The various processes and features described herein can be implemented in a wide variety of devices or applications, particularly those associated with data encoding, data decoding, view generation, texture processing, and other processing of image and associated texture and / or depth information. Examples of such devices include encoders, decoders, post-processors that process the output from decoders, pre-processors that provide input to encoders, video encoders, video decoders, video codecs, web servers, set-top boxes, laptops, personal computers, mobile phones, PDAs, and other communication devices. It should be understood that these devices can be mobile and even mounted in mobile vehicles.

[0076] Alternatively, the method can be implemented by a processor executing instructions, and such instructions (and / or data values ​​generated by the implementation) can be stored on a processor-readable medium (e.g., an integrated circuit, a software carrier) or other storage device (e.g., a hard disk, a compact disk (“CD”), an optical disk (e.g., a DVD, often referred to as a digital multifunction disk or digital video disk), random access memory (“RAM”), or read-only memory (“ROM”)). The instructions can form an application tangibly embodied on the processor-readable medium. The instructions can be, for example, in hardware, firmware, software, or a combination thereof. The instructions can be found, for example, in an operating system, a standalone application, or a combination of both. Thus, a processor can be characterized as, for example, a device configured to execute a process and a device comprising a processor-readable medium (such as a storage device) having instructions for executing the process. Furthermore, in addition to or instead of instructions, the processor-readable medium can store data values ​​generated by the implementation.

[0077] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information that can, for example, be stored or transmitted. The information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, a signal may be formatted to carry as data rules for writing or reading the syntax of the described embodiments, or as data actual syntax values ​​written by the described embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. It is well known that signals can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0078] Many implementations have been described. However, it should be understood that various modifications can be made. For example, elements of different implementations can be combined, supplemented, modified, or removed to produce other implementations. Furthermore, those skilled in the art will understand that other structures and processes can replace those disclosed, and the resulting implementations will perform at least substantially the same function in at least substantially the same manner to achieve at least substantially the same results as the disclosed implementations. Therefore, this application contemplates these and other implementations.

Claims

1. A method for encoding 3D meshes in a sequence of 3D meshes in a data stream, the method comprising: - Derive motion graphs from the 3D grids based on the different 3D grids in the sequence; - Generate a mapping between the motion amplitude of the motion graph and the quantization parameters (QP) to obtain a block-based QP table; - Generate texture maps using the block-based QP table; as well as - Encode the 3D mesh and the texture map in the data stream.

2. The method of claim 1, wherein deriving the motion map is performed from triangular metadata indicating the surface region to be encoded with the highest quality.

3. The method of claim 1 or 2, wherein the different 3D meshes in the sequence are the immediately preceding 3D meshes in the sequence, or wherein the different 3D meshes in the sequence are a reference 3D mesh.

4. An apparatus for encoding 3D meshes in a sequence of 3D meshes in a data stream, the apparatus comprising a memory associated with a processor configured to: - Derive motion graphs from the 3D grids based on the different 3D grids in the sequence; - Generate a mapping between the motion amplitude of the motion graph and the quantization parameters (QP) to obtain a block-based QP table; - Generate texture maps using the block-based QP table; as well as - Encode the 3D mesh and the texture map in the data stream.

5. The apparatus of claim 4, wherein deriving the motion map is performed from triangular metadata indicating the surface region to be encoded with the highest quality.

6. The apparatus of claim 4 or 5, wherein the different 3D meshes in the sequence are the immediately preceding 3D meshes in the sequence, or wherein the different 3D meshes in the sequence are a reference 3D mesh.

7. A method comprising: - Decode 3D meshes from data streams; - Derive motion graphs from the 3D meshes based on the different 3D meshes in the sequence of 3D meshes; - Generate a mapping between the motion amplitude of the motion graph and the quantization parameters (QP) to obtain a block-based QP table; as well as - Decode the texture map from the data stream using the block-based QP table.

8. The method of claim 7, wherein deriving the motion map is performed from triangular metadata of the surface region to be encoded with the highest quality data obtained from the data stream.

9. The method of claim 7 or 8, wherein the different 3D meshes in the sequence are the immediately preceding 3D meshes in the sequence, or wherein the different 3D meshes in the sequence are a reference 3D mesh.

10. The method of any one of claims 7 to 9, further comprising: The 3D mesh is rendered from a 3D perspective.

11. An apparatus comprising a memory associated with a processor, the processor being configured to: - Decode 3D meshes from data streams; - Derive motion graphs from the 3D meshes based on the different 3D meshes in the sequence of 3D meshes; - Generate a mapping between the motion amplitude and quantization parameters (QP) of the motion graph to obtain a block-by-block QP table; and - Decode the texture map from the data stream using the block-based QP table.

12. The apparatus of claim 11, wherein deriving the motion map is performed from triangular metadata of the surface region to be encoded with the highest quality data obtained from the data stream.

13. The apparatus of claim 11 or 12, wherein the different 3D meshes in the sequence are the immediately preceding 3D meshes in the sequence, or wherein the different 3D meshes in the sequence are a reference 3D mesh.

14. The apparatus of any one of claims 11 to 13, wherein the processor is further configured to render the 3D mesh for a 3D viewpoint.

Images