Method and device for encoding / decoding at least one attribute of an animated 3D object

Abstract

A method and apparatus for encoding at least one component of at least one attribute of an animated 3D object, and a method and apparatus for decoding at least one component of at least one attribute of an animated 3D object are disclosed. At least one component of at least one attribute of an animated 3D object is encoded by reframing the values ​​of the attributes into a first subset and a second subset—wherein the first subset of values ​​includes values ​​of at least one component within a range of values, and the second subset of values ​​includes values ​​of at least one component outside a range of values—and by encoding the first subset and the second subset of values.

Description

Technology Field

[0001] The present embodiments generally relate to a method and apparatus for encoding and decoding an animated mesh. Background Technology

[0002] Free viewpoint video can be implemented by capturing an animated model using a set of spatially dispatched physical capture devices (video, infrared, ...). Subsequently, the captured animated sequence can be encoded and transmitted to a terminal for playback from any virtual viewpoint having six degrees of freedom (6 dof). Different approaches exist for encoding the animated model. For example, the animated model Image / Video , Point cloud , or Textured mesh It can be expressed as

[0003] Image / Video In the base approach, a set of video streams plus additional metadata is stored, and warping or other reprojection is performed to generate an image from a virtual viewpoint during playback. This solution requires a lot of bandwidth and introduces many artifacts.

[0004] Point cloudIn this approach, an animated 3D point cloud is reconstructed from a set of input animated images, thereby making the 3D model representation more compact. Subsequently, the animated point cloud can be projected onto planes of a volume that enclose the animated point cloud and projected points (also known as patches) encoded into 2D coded video streams for transmission (e.g., using HEVC, AVC, VVC...). Such a solution is developed, for example, in the MPEG V-PCC standard ("ISO / IEC JTC1 / SC29 WG11, w19332, V-PCC codec description," Alpbach, Austria, April 2020). However, the characteristics of the model are very limited in terms of spatial expansion, and in the case of magnified views, some artifacts such as holes on the surface may appear.

[0005] Textured mesh In the approach, the animated textured mesh is, for example [1](A. Collet, M. Chuang, P. Sweeney, D. Gillett, D. Evseev, D. Calabrese, H. Hoppe, A. Kirk and S. Sullivan, "High-quality streamable free-viewpoint video," in ACM Transactions on Graphics (SIGGRAPH) , 2015, or in J. Carranza, C. Theobalt, M.A. Magnor and H.-P. Seidel, “Free-Viewpoint Video of Human Actors,” in ACM SIGGRAPHIt is reconstructed from a set of input animated images, such as in (2003). This type of reconstruction typically passes through an intermediate representation as voxels or point clouds. A characteristic of meshes is that photometric texture atlases can be encoded in standard video streams and geometric definition can be quite low. Point cloud solutions may require "complex" and "lossy" implicit and explicit projections (as in V-PCC) to obtain a planar representation compatible with video-based encoding approaches. In contrast, textured mesh encoding relies on texture coordinates (UVs) to perform mapping of texture images to the triangles of the mesh.

[0006] According to one embodiment, a method for encoding at least one component of at least one attribute of an animated 3D object is provided. The method comprises the steps of reframing the values ​​of at least one component of at least one attribute into a first subset and a second subset—wherein the first subset of values ​​includes values ​​of at least one component within a range of values, and the second subset of values ​​includes values ​​of at least one component outside a range of values—and encoding the first subset and the second subset of values.

[0007] According to another embodiment, an apparatus for encoding at least one component of at least one attribute of an animated 3D object is provided, which comprises one or more processors configured to reframe the values ​​of at least one attribute into a first subset and a second subset—wherein the first subset of values ​​includes values ​​of at least one component within a range of values, and the second subset of values ​​includes values ​​of at least one component outside a range of values—and to encode the first subset of values ​​and encode the second subset of values.

[0008] According to another embodiment, a method for decoding at least one component of at least one attribute of an animated 3D object is provided, wherein the decoding step comprises decoding a first subset of values ​​and a second subset of values, and reconstructing at least one component of at least one attribute from the first subset and the second subset, wherein the first subset of values ​​includes values ​​of at least one component within a range of values, and the second subset of values ​​includes values ​​of at least one component outside a range of values.

[0009] According to another embodiment, an apparatus for decoding at least one component of at least one attribute of an animated 3D object is provided, comprising one or more processors configured to decode a first subset of values ​​and a second subset of values ​​and to reconstruct at least one attribute from the first subset and the second subset, wherein the first subset of values ​​includes values ​​of at least one component within a range of values ​​and the second subset of values ​​includes values ​​of at least one component outside a range of values.

[0010] According to another embodiment, a method for encoding / decoding a 3D object is provided, which includes the step of encoding / decoding at least one component of at least one attribute of the 3D object.

[0011] According to another embodiment, a device for encoding / decoding a 3D object is provided, which includes one or more processors configured to at least encode / decode at least one component of at least one attribute of the 3D object.

[0012] According to another embodiment, a bitstream comprising the coding of at least one attribute of an animated 3D object is provided, which includes coded video data representing a first subset of values ​​of at least one component of at least one attribute of the animated 3D object, and coded metadata representing a second subset of values ​​of at least one component of at least one attribute of the animated 3D object, wherein the at least one attribute is reframed into a first subset and a second subset of values, the first subset of values ​​includes values ​​of at least one component within a range of values, and the second subset of values ​​includes values ​​of at least one component outside a range of values.

[0013] One or more embodiments also provide a computer program comprising instructions that, when executed by one or more processors, cause one or more processors to perform any one of encoding methods or decoding methods according to any of the embodiments described above. One or more of the embodiments also provide a computer-readable storage medium storing instructions for encoding or decoding at least one component of at least one attribute of a 3D object or an animated 3D object according to the methods described above. One or more embodiments also provide a computer-readable storage medium storing a bitstream generated according to the methods described above. One or more embodiments also provide a method and apparatus for transmitting or receiving a bitstream generated according to the methods described above. Brief explanation of the drawing

[0014] FIG. 1 illustrates a block diagram of a system in which aspects of the embodiments can be implemented. Figure 2 shows a block diagram of one embodiment of a video encoder. FIG. 3 shows a block diagram of one embodiment of a video decoder. Figure 4 illustrates an octahedral representation. FIG. 5 illustrates exemplary meshes having corresponding 3D objects and texture atlases. Figure 6 illustrates the evolution of the position attributes for a given frame of a basketball player model using Draco with CL=0 (no EdgeBreaker). Figure 7 illustrates the evolution of the position deltas for a given frame of a basketball player model using Draco with CL=4 (having EdgeBreaker). Figure 8 illustrates the evolution of the position deltas for a given frame of the ballerina model using Draco with CL=4 (having EdgeBreaker). Figure 9 illustrates the evolution of position deltas for a given frame of a basketball player model using Draco with CL=10 (having a valence-driven EdgeBreaker). Figure 10 illustrates the evolution of the position deltas for a given frame of the ballerina model using Draco with CL=10 (having a veilance-driven EdgeBreaker). Figure 11 illustrates the evolution of the deltas of the UVs for a given frame of a basketball player model using Draco with CL=10 (EdgeBreaker) and a 10-bit quantization step. Figure 12 illustrates the evolution of the deltas of the UVs for a given frame of the ballerina model using Draco with CL=10 (EdgeBreaker) and a 10-bit quantization step. Figure 13 illustrates the evolution of normals in octahedral space for a given frame of a ballerina model using Draco with CL=10 (EdgeBreaker) and a 7-bit quantization step. FIG. 14 illustrates an exemplary method for encoding a 3D object according to one embodiment. FIG. 15 illustrates an exemplary method for decoding a 3D object according to one embodiment. FIG. 16 illustrates an exemplary method for encoding at least one component of an attribute of an animated 3D object according to one embodiment. FIG. 17 illustrates an exemplary method for reframing the values ​​of the components of the attributes of an animated 3D object according to one embodiment. FIG. 18 illustrates a block diagram of an exemplary method for decoding at least one component of at least one attribute of an animated 3D object according to one embodiment. FIG. 19 illustrates an example of linear packing for ballerina model attributes in image planes for a single frame. Figure 20 illustrates the evolution of the rebased normal in octahedral space for a given frame of the ballerina model using Draco with CL=10 (EdgeBreaker) and a 7-bit quantization step. FIG. 21 illustrates an example of U, V signals for reframing and corresponding filtering according to one embodiment. FIG. 22 illustrates an example of a portion of a bitstream according to one embodiment. FIG. 23a illustrates an example of metadata encoded for a second subset according to one embodiment. FIG. 23b illustrates an example of metadata encoded for a second subset according to another embodiment. Specific details for implementing the invention

[0015] FIG. 1 illustrates a block diagram of an example of a system in which various embodiments and embodiments may be implemented. The system (100) may be implemented as a device comprising various components described below and configured to perform one or more of the embodiments described in this application. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected consumer electronics, and servers. The elements of the system (100) may be implemented in a single integrated circuit, multiple ICs, and / or separate components, either alone or in combination. For example, in at least one embodiment, the processing and encoder / decoder elements of the system (100) are distributed across multiple ICs and / or separate components. In various embodiments, the system (100) is communically coupled to other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, the system (100) is configured to implement one or more of the embodiments described in this application.

[0016] The system (100) includes at least one processor (110) configured to execute instructions loaded internally to implement, for example, the various embodiments described in the present application. The processor (110) may include embedded memory, an input / output interface, and various other circuit components as known in the art. The system (100) includes at least one memory (120) (e.g., a volatile memory device and / or a non-volatile memory device). The system (100) includes a storage device (140) which may include non-volatile memory and / or volatile memory, including but not limited to EEPROM, ROM, PROM, RAM, DRAM, SRAM, flash, magnetic disk drive, and / or optical disk drive. As a non-limiting example, the storage device (140) may include an internal storage device, an attached storage device and / or a network-accessible storage device.

[0017] For example, the system (100) includes an encoder / decoder module (130) configured to process data to provide an encoded video or a decoded video, and the encoder / decoder module (130) may include its own processor and memory. The encoder / decoder module (130) represents a module(s) that may be included in a device to perform encoding and / or decoding functions. As is known, the device may include one or both of the encoding and decoding modules. Additionally, the encoder / decoder module (130) may be implemented as a separate element of the system (100) or may be integrated within the processor (110) as a combination of hardware and software, as is known to those skilled in the art.

[0018] Program code to be loaded onto a processor (110) or an encoder / decoder (130) to perform the various embodiments described in the present application may be stored in a storage device (140) and subsequently loaded onto a memory (120) for execution by the processor (110). According to various embodiments, one or more of the processor (110), memory (120), storage device (140), and encoder / decoder module (130) may store one or more of various items during the execution of the processes described in the present application. Such items stored may include, but are not limited to, input video, decoded video or parts of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operation logic.

[0019] In several embodiments, memory within the processor (110) and / or encoder / decoder module (130) is used to store instructions and to provide working memory for processing required during encoding or decoding. However, in other embodiments, memory outside the processing device (e.g., the processing device may be either the processor (110) or the encoder / decoder module (130)) is used for one or more of these functions. The external memory may be memory (120) and / or storage device (140), e.g., dynamic volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store the operating system of the television. In at least one embodiment, high-speed, external dynamic volatile memory, such as RAM, is used as working memory for video coding and decoding operations, such as for MPEG-2, HEVC, or VVC.

[0020] Inputs to the elements of the system (100) may be provided through various input devices as shown in block (105). Such input devices include, but are not limited to, (i) an RF part that receives an RF signal transmitted wirelessly by, for example, a broadcaster, (ii) a composite input terminal, (iii) a USB input terminal, and / or (iv) an HDMI input terminal.

[0021] In various embodiments, the input devices of block (105) have their respective associated input processing elements as known in the art. For example, the RF portion may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting the signal to a band of frequencies), (ii) downconverting the selected signal, (iii) band-limiting it again to a narrower band of frequencies to select a signal frequency band that may be referred to as a channel in some embodiments, (iv) demodulating the downconverted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF portion of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band-limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include a tuner that performs various such functions, for example, including down-converting the received signal to a lower frequency (e.g., an intermediate frequency or near-baseband frequency) or to the baseband. In one set-top box embodiment, the RF section and its associated input processing element receive an RF signal transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering for a desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, for example, inserting amplifiers and analog-to-digital converters.In various embodiments, the RF portion includes an antenna.

[0022] Additionally, USB and / or HDMI terminals may include their own interface processors for connecting the system (100) to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing, for example, Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or within the processor (110) as needed. Similarly, aspects of USB or HDMI interface processing may be implemented, for example, within separate interface ICs or within the processor (110). Demodulation, error correction, and demultiplexed streams are provided to various processing elements, including a processor (110) and an encoder / decoder (130), which operate in combination with memory and storage elements to process data streams as needed, for example, for presentation on an output device.

[0023] Various elements of the system (100) may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and transmit data between them using a suitable connection arrangement (115), including an I2C bus, wiring, and printed circuit boards, for example, an internal bus known in the art.

[0024] The system (100) includes a communication interface (150) that enables communication with other devices through a communication channel (190). The communication interface (150) may include, but is not limited to, a transceiver configured to transmit and receive data through the communication channel (190). The communication interface (150) may include, but is not limited to, a modem or a network card, and, for example, the communication channel (190) may be implemented within a wired and / or wireless medium.

[0025] In various embodiments, data is streamed to the system (100) using a Wi-Fi network such as IEEE 802.11. The Wi-Fi signals in these embodiments are received through a communication channel (190) and a communication interface (150) adapted for Wi-Fi communication. The communication channel (190) in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communication. Other embodiments provide the streamed data to the system (100) using a set-top box that transmits data through the HDMI connection of the input block (105). Still other embodiments provide the streamed data to the system (100) using the RF connection of the input block (105).

[0026] The system (100) may provide output signals to various output devices, including a display (165), a speaker (175), and other peripheral devices (185). The other peripheral devices (185) include, in various examples of the embodiments, one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functions based on the output of the system (100). In various embodiments, control signals are communicated between the system (100) and the display (165), speakers (175), or other peripheral devices (185) using signaling such as AV.Link, CEC, or other communication protocols that enable device-to-device control with or without user intervention. The output devices may be communicably coupled to the system (100) through dedicated connections via their respective interfaces (160, 170, 180). Alternatively, output devices may be connected to the system (100) using a communication channel (190) through a communication interface (150). The display (165) and speaker (175) may be integrated into a single unit with other components of the system (100) in an electronic device, e.g., a television. In various embodiments, the display interface (160) includes a display driver, e.g., a timing controller (T Con) chip.

[0027] The display (165) and speaker (175) may alternatively be separate from one or more of the other components, for example, if the RF portion of the input (105) is part of a separate set-top box. In various embodiments where the display (165) and speakers (175) are external components, the output signal may be provided through dedicated output connections, such as HDMI ports, USB ports, or COMP outputs.

[0028] FIG. 2 illustrates an exemplary video encoder (200), such as a High Efficiency Video Coding (HEVC) encoder, which can be used to encode one or more attributes of an animated mesh according to one embodiment. FIG. 2 may also illustrate an encoder that is an improvement over the HEVC standard, or an encoder that employs technologies similar to HEVC, such as a Versatile Video Coding (VVC) encoder developed by the Joint Video Exploration Team (JVET).

[0029] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "encoded" or "coded" may be used interchangeably, the terms "pixel" or "sample" may be used interchangeably, and the terms "image," "picture," and "frame" may be used interchangeably. Generally, but not necessarily, the term "reconstructed" is used on the encoder side, whereas "decoded" is used on the decoder side.

[0030] Before encoding, the video sequence may undergo pre-encoding processing (201), for example, applying a color conversion to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0), or remapping the input picture components to obtain a signal distribution more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with pre-processing and may be attached to the bitstream.

[0031] In the encoder (200), the picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (202) and processed, for example, into units of coding units (CU). Each unit is encoded using, for example, an intra or inter mode. When a unit is encoded in intra mode, it performs intra prediction (260). In inter mode, motion estimation (275) and compensation (270) are performed. The encoder determines whether to use intra mode or inter mode to encode the unit (205), and indicates the intra / inter decision, for example, by a prediction mode flag. The encoder can also blend intra prediction results and inter prediction results, or blend results from different intra / inter prediction methods (263).

[0032] Predicted residuals are calculated, for example, by subtracting the predicted block from the original image block (210). The motion enhancement module (272) uses an already available reference picture to enhance the motion field of the block without referencing the original block. The motion field for a region can be considered as a set of motion vectors for all pixels having the region. If the motion vectors are sub-block based, the motion field can also be expressed as a set of all sub-block motion vectors within the region (all pixels within the sub-block have the same motion vector, and the motion vectors may vary depending on the sub-block). If a single motion vector is used for the region, the motion field for the region can also be expressed as a single motion vector (the same motion vectors for all pixels within the region).

[0033] Next, the predicted residuals are transformed (225) and quantized (230). The quantized transformation coefficients, as well as motion vectors and other syntax elements, are entropy-coded (245) to output a bitstream. The encoder can skip the transformation and apply quantization directly to the untransformed residual signal. The encoder can skip both the transformation and the quantization, that is, the residuals are coded directly without the application of transformation or quantization processes.

[0034] The encoder decodes the encoded block to provide a reference for additional predictions. Quantized transform coefficients are inversely quantized (240) and inversely transformed (250) to decode the prediction residuals. An image block is reconstructed by combining the decoded prediction residuals and the predicted block (255). In-loop filters (265) are applied to the reconstructed picture to perform deblocking / SAO (sample adaptive offset) filtering, for example, to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (280).

[0035] FIG. 3 illustrates a block diagram of an exemplary video decoder (300) that can be used to decode one or more attributes of an animated mesh according to one embodiment. In the decoder (300), the bitstream is decoded by decoder elements as described below. The video decoder (300) generally performs a decoding pass opposite to the encoding pass as described in FIG. 2. The encoder (200) also generally performs video decoding as part of encoding video data.

[0036] In particular, the input to the decoder includes a video bitstream that can be generated by a video encoder (200). The bitstream is first entropy decoded to obtain transform coefficients, motion vectors, and other coded information (330). Picture partition information indicates how the picture is partitioned. Thus, the decoder can partition the picture according to the decoded picture partitioning information (335). The transform coefficients are inversely quantized (340) and inversely transformed (350) to decode the prediction residuals. An image block is reconstructed by combining the decoded prediction residuals and the predicted block (355).

[0037] The predicted block can be obtained (370) from intra prediction (360) or motion compensation prediction (i.e., inter prediction) (375). The decoder can blend the intra prediction results and the inter prediction results, or blend results from multiple intra / inter prediction methods (373). Before motion compensation, the motion field can be enhanced by using reference pictures that are already available (372). In-loop filters (365) are applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380).

[0038] The decoded picture may further undergo post-decoding processing (385), for example, inverse remapping or inverse color conversion (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) which performs the inverse of the remapping process performed in pre-encoding processing (201). Post-decoding processing may use metadata derived from pre-encoding processing and signaled in the bitstream.

[0039] The present application provides various embodiments for encoding / decoding one or more attributes of an animated 3D object. According to one embodiment, the 3D object is represented as an animated point cloud or an animated 3D mesh. In the case of a 3D object represented as a 3D mesh, the following embodiments are described. The 3D mesh may be derived from the point cloud of the 3D object.

[0040] The mesh includes at least the following features: a list of vertices or a list of vertex positions, a topology defining connections between vertices, e.g., a list of faces, and optionally texture data, e.g., texture atlases associated with vertices, or color values.

[0041] According to one embodiment, video-based coding / decoding is used to encode / decode at least one component of at least one attribute of a vertex of an animated mesh. The attribute of a vertex refers to the attributes associated with each vertex of the mesh, such as the position of the vertex in 3D space (x,y,z), also referred to as geometric coordinates, texture coordinates (U,V) in a texture atlas associated with the vertex, the normal associated with the vertex, the color associated with the vertex, or general attributes. Some attributes may have only one component, while others may have multiple components.

[0042] An example of an end-to-end chain for encoding and transmitting animated textured meshes is presented in [1]. In this scheme, meshes are tracked over time to match the topology of the meshes. Texture atlases are encoded as video frames using an H.264-based encoder. The mesh is encoded by dividing the mesh sequence into a series of keyframes and predict frames. Keyframe meshes contain both geometric and connectivity information. Geometric information (vertex positions and UV coordinates) quantized to 16 bits is encoded. Connectivity information is delta-encoded using variable-byte triangle strips. Predict frames contain only delta geometric information. Linear motion predictors are used to compute the delta geometry, which is then quantized and compressed using Golomb coding. In [1], the mesh is encoded as metadata and without using video coding schemes.

[0043] Literature [J. Rossignac, "Edgebreaker: Connectivity compression for triangle meshes," GVU center, Georgia Institute of Technology, 1999 and J. Rossignac, "3D compression made simple: Edgebreaker with ZipandWrap on a corner-table," in Proceedings International Conference on Shape Modeling and ApplicationsIn [Genova, Italy, 2001], implementations of a scheme called EdgeBreaker are proposed for encoding static meshes. Edgebreaker provides an algorithm for encoding a static mesh topology as spiral triangle-strips over the mesh topology. The tree-strip chain topology is coded using very short codes, and the attributes (positions, UVs, normals, colors) of the vertices visited through the process are delta-encoded. Subsequently, the delta-encoded attribute tables are compressed using an arbitrary entropy coder. The input data structure of the algorithm is the corner table representation of the input mesh.

[0044] The EdgeBreaker algorithm uses the so-called CLERS table. Edgebreaker visits triangles in a spiral (depth-first) triangle-spanning-tree order and generates a string of descriptors for each triangle, which indicates how the mesh can be regenerated by connecting new triangles to previously reconstructed ones. The characteristic of Edgebreaker lies in the fact that all descriptors are symbols from the set {C, L, E, R, S}. No other parameters are required. Since half of the descriptors are C, the trivial code (C=0, L=110, E=111, R=101, S=100) guarantees an average of 2 bits per triangle.

[0045] The mesh vertices are delta-encoded. Edgebreaker compression is WRITE( delta , D) Using statements, delta A sequence of corrective vectors and points are stored in a string called , where D is a point or vector. The corrective vectors are encoded using a variable-length binary format in individual post-processing entropy compression steps. During decompression, the first call READ( delta) returns the decoded version of the first vertex. READ( delta Subsequent calls to ) return correction vectors, which are added to the vertex estimates.

[0046] When the tip vertex of a new triangle has not yet been visited, the triangle is triangle C, and the correction vector for the tip vertex of the current triangle is [C. Touma and C. Gotsman, "Triangle Mesh Compression," in Graphics Interface It is delta-encoded using the parallelogram rule as described in [ , 1998.].

[0047] In Google's literature ["Draco 3D graphics compression," 2017-2020 ([Online] Available: https: / / google.github.io / draco / ).], a library called Draco is presented for the compression of static point clouds and static meshes. For static meshes, the library relies on cutting edge implementations of edgebreaker algorithms. The system proposes three compression levels {CL0, CL4, CL10} that primarily guide the compression rate, which has a direct influence on the compression ratio. The compression level influence is the selection of the topology compression algorithm:

[0048] - CL0: No edgebreaker, simple topology encoding, very fast, low compression.

[0049] - CL4: Simple and fast implementation of Edgebreaker.

[0050] - CL10: An evolved and (slightly) slower implementation of a valence-driven Edgebreaker, with the most compact results.

[0051] Draco proposes quantization parameters for vertex positions (default qp=11), UVs (default qt=10), normals (default qn=7), colors (default qc=8), and generals (default qg=8). The default values ​​are represented in bits per component. The values ​​can be raised to 32 bits per component.

[0052] In Draco, positions, UVs, colors, and general properties are all delta-encoded using the parallelogram rule. Normals, due to their normalized and spherical nature, [Q. MEYER, J. It is quantized using an octahedral representation as in [G. SUSSNER, M. STAMMINGER and G. ANDGREINER, "On floating-point normal vectors." in Proceedings of the 21st Eurographics conference onRendering, Aire-la-Ville, Switzerland, 2010] (see Fig. 4). Fig. 4 illustrates an octahedral representation mapping the octahedrals of a sphere to the faces of an octahedron. Subsequently, the faces of the octahedron are projected onto a plane unfolded into a unit square.

[0053] During the compression process, the Draco implementation stores CLERS chains in specific tables and quantized attributes in their respective tables. For positions, UVs, colors, and general attributes, quantized delta values ​​are stored in memory as signed integers. For normals, quantized octahedral values ​​are stored in memory as unsigned I,J coordinates. The attribute tables are finally entropy-coded.

[0054] FIG. 5 illustrates exemplary meshes having corresponding 3D objects and texture atlases. The top of FIG. 5 illustrates a frame of an animated ballerina model provided by Microsoft®, for which mesh tracing was used, and corresponding stabilized texture atlases over time are provided. The bottom of FIG. 5 illustrates a frame of an animated basketball player provided by Owlii®, for which mesh tracing was not used.

[0055] Figure 6 illustrates the evolution of position attributes for a given frame of a basketball player model using Draco with CL=0 (no EdgeBreaker). The x-axis is the index of the sample within the attribute table. The y-axis is the x, y, and z coordinates, respectively. Since no EdgeBreaker is used, the attributes (x,y,z) are expressed in global coordinates.

[0056] Figure 7 illustrates the evolution of position deltas for a given frame of a basketball player model using Draco with CL=4 (EdgeBreaker). The x-axis is the index of the delta value in the attribute table. The y-axis is the x-delta, y-delta, and z-delta, respectively.

[0057] Figure 8 illustrates the evolution of position deltas for a given frame of a ballerina model using Draco with CL=4 (EdgeBreaker). The x-axis is the index of the delta value in the attribute table. The y-axis is the x-delta, y-delta, and z-delta, respectively.

[0058] Figure 9 illustrates the evolution of position deltas for a given frame of a basketball player model using Draco with CL=10 (having a veilance-driven EdgeBreaker). The x-axis is the index of the delta value in the attribute table. The y-axis is the x-delta, y-delta, and z-delta, respectively.

[0059] Figure 10 illustrates the evolution of position deltas for a given frame of a ballerina model using Draco with CL=10 (having a valence-driven EdgeBreaker). The x-axis is the index of the delta value in the attribute table. The y-axis is the x-delta, y-delta, and z-delta, respectively.

[0060] A default quantization of 11 bits per component was used for the basketball player model, and a default quantization of 10 bits per component was used for the ballerina model. Figure 6 illustrates the significant disparity (i.e., high variance) of the samples. Having high variance means that the data is not very suitable for mapping to spatial coherence-based encoders. Figure 6 also illustrates that the deviation is also very high, which means that the entire quantization space (10 or 11 bits) is essential to encode the signal.

[0061] Table 1 below illustrates a summary of the variances and deviations for basketball and ballerina position attribute signals obtained when using different configurations of Draco to encode animated mesh models.

[0062]

[0063] Figures 7 and 8 illustrate that when using delta-encoding for the mesh, the variance is much lower. Only a few sparse points can be seen. This means that even if the entire signal is noisy, the noise is very compact and flat, i.e., has small variations. Therefore, the signal is suitable for video encoding in terms of spatial coherence. In addition, the deviation is also much lower, which means that most of the signal can be encoded in a range of less than the original 10 or 11 bits.

[0064] In the case of Figures 9 and 10, only the first point (which is the start of the chain) is an absolute value, and all other samples are valence-driven deltas that consume very few bits. Furthermore, the deviation for these signals is about 6, which is extremely low and means that these values ​​(negative absolute values) can be re-encoded into a smaller bit range. A Draco configuration with CL=10 provides a signal that is much more suitable for video coding than CL=4 in terms of spatial coherence.

[0065] Here, UV features represent texture coordinates that map the texture atlas to the faces of the mesh. The texture associated with each vertex is identified by the couple of the U and V texture coordinates associated with the vertex. Table 2 provides a summary of the UV profiles presented in Figures 11 and 12, illustrating the evolution of the UV deltas for a given frame of the basketball player model and the ballerina model, respectively, using Draco with CL=10 (with EdgeBreaker) and a 10-bit quantization step. The x-axis is the index of the delta value within the attribute table. The y-axis is the U-delta and V-delta, respectively.

[0066] Figures 11 and 12 illustrate that CL=10 provides UV signals having the same characteristics as described for geometry. However, the total variance and deviations are slightly higher than for geometry. This is because the texture atlas jumps over the texture image, resulting in wider UV delta values ​​for patch variations.

[0067]

[0068] It can be seen that the ballerina variance and deviation values ​​are one order higher than the basketball values. Even though ballerina benefits from mesh tracking that stabilizes the atlas over time, the atlas generation algorithm from [1] appears to generate smaller patches than the atlas generation algorithm that does not use mesh tracking, as shown in Figure 5. This results in coarser UV deltas (higher deviations) and wider chain restarts corresponding to the coarser points on the ballerina UV plots.

[0069] As described above, in Draco, normal vectors are not encoded as deltas, but as absolute values ​​in the octahedral representation. Figure 13 illustrates the profile of normal evolution in octahedral space for the ballerina model. The x-axis is the index of the value in the attribute table, and the y-axis is the I and J values ​​in octahedral space, respectively.

[0070] Figure 13 illustrates that the signal does not appear to be suitable for spatial compression due to very high variance and deviations. The signal does not appear to be suitable for quantization space reduction because range extrema are used. Nevertheless, Figure 13 shows that the signal is split into two main parts (top and bottom).

[0071] FIG. 14 illustrates an exemplary method (1400) for encoding a 3D object according to one embodiment. According to this embodiment, the 3D object is an animated 3D textured mesh. According to other variations, the 3D object may be in any other format and may be converted into an animated 3D mesh. For example, the 3D object may be represented as a dynamic point cloud. The 3D points of the point cloud are further converted into a 3D mesh representation.

[0072] It should be understood that the 3D mesh, i.e., the sequence of 3D meshes, evolves over time, and that the vertices and optionally the topology evolve over time.

[0073] In 1401, a 3D animated textured mesh is input into an Edgebreaker-like algorithm [1,4,5,7], which provides as outputs the geometry, texture coordinates, and normal properties as delta tables as described above. Eventually, as in [1], the meshes can be stabilized over time before being input into the Edgebreaker-like algorithm. Thus, according to this variation, the meshes are tracked over time so that the topology of the meshes conforms.

[0074] In 1403, the topology generated by the Edgebreaker algorithm is encoded. According to one embodiment, the topology is encoded as metadata. In other embodiments, the topology is packed into video frames and encoded as video data. According to a variation, the topology is changed only in keyframes, and thus, only the topology for the keyframes is encoded. The topology is stored in synchronized metadata, such as SEI messages.

[0075] In 1402, each component of each attribute is packed into a component of the video image. Geometric (x, y, z) deltas quantized to N bits are stored in each component of a YUV 4:4:4 stream having N bits per component. UV deltas quantized to M bits are stored in the first and second components of a YUV 4:4:4 stream having M bits per component, leaving the last component as 0. According to one embodiment in which a 3D mesh includes a color associated with each vertex, the color data is also packed into at least one component of the video image.

[0076] More complex packing may be used. Octahedral normals quantized to 7 or 8 bits are encoded into the first and second components of a YUV 4:4:4 stream having 8 bits per component, leaving the last component 0. More complex packing may also be used. FIG. 19 illustrates an example of linear packing for ballerina model attributes in image planes for a single frame. From left to right, the illustrated attributes are geometric deltas of 10 bits per component (scaled to 3 for visibility), UV deltas of 10 bits per component, and normals of 8 bits per component. FIG. 19 illustrates sparse points of profile curves as "hot spots" within the images.

[0077] According to another variation, the geometric, UV, and normal components of the delta can also be spread into a single video luminance component (Y) of a YUV video signal (e.g., 4:0:0 monochrome or 4:2:0). Such a variation allows, for example, the use of a common 4:2:0 YUV video format used for distribution.

[0078] In 1404, packed attributes are encoded. For each type of attribute, an HEVC or VVC video stream of the nearest available bit depth (up to the upper value) is generated. All frames of each attribute are encoded to generate one stream per attribute (geometry, texture coordinates, normals).

[0079] The properties are encoded using a lossless video-based encoding, such as a HEVC encoder, a VVC encoder, or any other video-based encoder. Lossless encoding is used here for geometry, UV, and normal coordinates because quantization causes uncontrolled shifts and distortions in the reconstructed 3D model.

[0080] According to one embodiment, normal expansion is performed on the normal coordinates prior to encoding. As described above and illustrated in FIG. 13, the signal of the normal coordinates is divided into two main parts (top and bottom). After the expansion operation, the variance and deviation of the signal are significantly reduced, and accordingly, become more compatible with spatial coding. Such expansion operation can be performed in the following manner:

[0081] N normal is defined as the number of bits used to encode the normal before reframing. R normal = Power(2, N normal ) is the numerical range, and Rh = R normal / 2 sets the numerical range to half. I,J are the coordinates to be expanded, and (I',J') are the expanded coordinates: if (I > Rh), then I' = Rh + I - R normal And, in the other case, I'= Rh + I.

[0082] In the decoder, to reconstruct the normal signal, reconstruction is performed by the following: if (I' > Rh), I = I' - Rh, and otherwise, I = Rh + I'.

[0083] The same operations are performed for the J coordinates. Fig. 20 illustrates the evolution of the reconstructed normal in octahedral space for a given frame of the ballerina model using Draco with CL=10 (with EdgeBreaker) and a 7-bit quantization step. The x-axis is the index of the value in the attribute table. The y-axis is the I and J values ​​in octahedral space, respectively.

[0084] According to one embodiment, a lossless quantization reframing scheme is applied to at least one component of an attribute before encoding. Such a reframing scheme is described later with reference to FIGS. 16 and FIGS. 17. Such lossless quantization reframing further allows for reducing the payload or increasing the quality.

[0085] In 1405, texture atlases are encoded using a video-based encoder, such as the HEVC encoder, the VVC encoder, or any other video-based encoder.

[0086] In 1406, bitstreams generated from the encoding of topology, attributes, and textures are multiplexed into one or more bitstreams for storage or streaming.

[0087] According to one embodiment, bitstreams generated from the encoding of topology, attributes, and textures are embedded in V-PCC bitstreams using V-PCC syntax.

[0088] FIG. 15 illustrates an exemplary method (1500) for decoding a 3D object from a bitstream according to one embodiment. For example, the bitstream may be generated by an encoding method (1400) described with reference to FIG. 15.

[0089] In 1502, a bitstream containing syntax elements for the topology, textures, and attributes of an animated 3D mesh is received. The data is eventually demultiplexed. According to one embodiment, the received bitstream is adapted to V-PCC decoder syntax.

[0090] In 1503, the topology is decoded as metadata or video data according to the implemented variant. In 1504, attributes are decoded using a video-based decoder. According to one embodiment, an inverse lossless quantization reframing scheme is used as described below with reference to FIG. 18. In 1505, texture atlases are decoded using a video-based decoder. In 1506, a 3D animated textured mesh is reconstructed from the decoded topology, decoded attributes, and decoded textures.

[0091] As illustrated in FIGS. 7 through 12 and FIG. 20, most signals provided by the Edgedbreaker algorithm with CL=4 or CL=10 are highly concentrated at the origin, while a few values ​​are spread out. This characteristic is used to split the data to be encoded into two parts, maintaining the dense part during image compression but with a smaller bit range, and coding the sparse part in a separate table stored, for example, as metadata or SEI messages, which is ultimately entropy-coded. In a variation, the sparse part may also be encoded as video data using a video-based encoder. FIG. 21 illustrates an example of U,V signals having a dense part between lines L0 and L1 and a sparse part outside lines L0 and L1. Such splitting of the signal is also referred to as reframing. An exemplary embodiment is described below.

[0092] FIG. 16 illustrates an exemplary method (1600) for encoding at least one component of an attribute of an animated 3D mesh. The method (1600) is a lossless quantization reframing scheme that can be applied to any one of the components of the attribute of the 3D mesh, for example, to the x, y, or z components of the geometry, to the U or V of the texture coordinates, or to the I or J of the normal coordinates.

[0093] At least one attribute includes a set of values ​​to be encoded. According to one embodiment, the values ​​of the attribute are packed into components of an image. However, the principles presented herein may be applied to any representation of a set of values. For example, a set of values ​​may be represented in a 1D table.

[0094] In 1601, the set of values ​​is reframed into a first subset of values ​​and a second subset of values ​​as described with reference to FIG. 17.

[0095] FIG. 17 illustrates an exemplary method for reframing the values ​​of components of an attribute of an animated 3D mesh according to one embodiment. The reframing method aims to divide an input set of values ​​into the following two subsets: a first subset containing a dense portion of the signal concentrated at the origin and a second subset containing a sparse portion of the signal. According to one embodiment, the first subset of values ​​and the second subset of values ​​are determined as a function of a radius parameter defining a range of values, said first subset of values ​​includes a set of values ​​within said range of values, and said second subset of values ​​includes a set of values ​​outside the range of values.

[0096] In 1701, the radius is determined. According to a variation, the radius can be determined as half of the maximum deviation of the component. For example, in the example of the signal shown in FIG. 21, the radius R=11 can be taken as the maximum values ​​of the deviations U=22 and V=20 divided by 2. In another variation, the radius can be set manually.

[0097] The determined radius R above provides the radius containing the greatest density (between the two lines L0 and L1). Thus, the dense part of the signal is determined as the part of the value that lies within the range defined by the center of the signal + radius and the center of the signal - radius, as exemplified in the enlarged part of FIG. 21.

[0098] The number of bits required to quantize a first subset (dense part) of values ​​without loss is determined as a function of a determined radius R. Br = is defined as the number of bits required to encode the range. Br is the lossless requantization space required to encode all integer values ​​within the range [-R,+R]. In the previous example, with R=11, Br = floor(log2(2²)+1) = 5 bits, which is the number R' = 2 5 = 32 provides values.

[0099] According to one embodiment, the radius R is determined for each frame i. Then, each frame is quantized using its own determined number of bits.

[0100] According to one embodiment, a distinct radius R is determined for each component of the mesh's attributes. Accordingly, each component of the attribute is quantized using its own determined number of bits.

[0101] According to another embodiment, the same radius R is determined for all components of the mesh attribute. Accordingly, all components of the attribute are quantized using the same determined number of bits. According to this variation, the same radius determined for all components is determined to correspond to the maximum radius among the radii determined separately for each component of the attribute.

[0102] According to another embodiment, the maximum of allowed coarse points is set to automatically increase if a radius size is required. Such a maximum of allowed coarse points can be computed for average compression results and optimized to provide the best trade-off between the compressed reframed deltas and the raw coarse points.

[0103] In 1702, the values ​​of the input signal in the range [-R,+R] are quantized over a number of Br bits. The values ​​of the input signal that do not fall within the range [-R,+R] are stored in a separate table (a sparse table), and their positions within the input signal are also stored. This separate table forms a second subset of values.

[0104] According to one embodiment, a first subset of values ​​is set as an input set of values, values ​​within the range [-R, +R] are quantized on Br bits, and values ​​outside the range [-R, +R] are replaced with determined values. For example, in the case of a value outside the range [-R, +R], its original value within the first subset of values ​​is replaced with the nearest value within the radius to improve spatial compression.

[0105] According to another embodiment, values ​​outside the range [-R,+R] are removed from the first set of values. However, this embodiment results in a variable table size for each frame.

[0106] In 1703, the radius is encoded into the bitstream, for example, as a metadata syntax element.

[0107] In 1602, a first subset of values ​​is encoded using a video-based encoder, for example, the aforementioned encoder (200).

[0108] In 1603, a second subset of values ​​is encoded as metadata. The second subset of values ​​corresponding to the table of sparse points includes, for each sparse point, its value coded at the original bit size + its index within the original set of values. The index can be coded in the minimum number of bits required to address the original data before reframing. According to a variation, the sparse table is encoded in a Supplemental Enhancement Information (SEI) message and can be raw or entropy coded. For example, the second subset of values ​​is encoded in the SEI using an HEVC, VVC, or V-PCC encoder. According to another variation, the sparse table is encoded as video data using a video-based encoder.

[0109] According to another embodiment, as illustrated in FIG. 21, sparse points are located in opposite portions of dense points, and thus, sparse points can also be reframed losslessly across two subspaces, namely, a subspace below the dense space and a subspace above it: the size of these subspaces is equal to 1 + (original bit range - Br) / 2. According to this embodiment, the sparse table can be coded into two separate tables of the same size, for example, by reframing the signal and reframe the reframing signal in the same manner as for the dense portion. This embodiment is possible only when separate reframing is performed for each component of the attribute.

[0110] FIG. 18 discloses a block diagram of an exemplary method (1800) for decoding at least one component of at least one attribute of an animated 3D object according to one embodiment. At least one component of at least one attribute is decoded from a previously encoded bitstream using, for example, the method illustrated in FIG. 16.

[0111] In 1801, the radius parameter is decoded. The radius parameter defines the range of quantized values ​​of the first subset.

[0112] In 1802, a first subset of component values ​​is decoded using a video-based decoder, such as a decoder (300), for example. As described in FIG. 17, the first subset of values ​​includes a dense portion of the signal. The values ​​of the first subset are inversely quantized according to the decoded radius parameter.

[0113] In 1803, a second subset of values ​​is obtained by decoding an entropy-coded or raw-coded coarse table from metadata of a bitstream, such as an SEI message. In a variation, a second subset of values ​​is obtained by decoding the coarse table using a video-based decoder. The second decoded subset of values ​​includes the original values ​​of the coarse portion of the signal and the indices of the coarse data within the original signal for the components of the attributes.

[0114] In 1804, the values ​​of the attribute components are reconstructed from the first subset and the second subset. As the first subset is packed into the image components of the video, the first subset is unpacked from the image components.

[0115] According to one embodiment, the first subset includes the same number of values ​​as the components of the attribute. To reconstruct the components of the attribute, at each position of the value corresponding to the value from the second subset, the decoded value in the first subset is replaced with the corresponding value of the decoded second subset.

[0116] According to another embodiment, only the values ​​of the first subset are packed into the component, that is, the values ​​of the second subset are removed from the component of the attribute for packing. To reconstruct the component of the attribute, the corresponding values ​​of the decoded second subset are inserted at each position of the values ​​corresponding to the values ​​from the second subset.

[0117] According to another embodiment, when reconstructing the 3D animated object at 1506 in FIG. 15, the components of the attributes are reconstructed on the fly. The delta table is set to a first subset of decoded values. Whenever an element of the delta table is needed, the EdgeBreaker algorithm is modified to look up the current point in the sparsity table. If the stored index of the current point in the sparsity table is the same as the current index in the delta table, the value in the delta table is skipped, and the value from the sparsity table is used. The pointer to the current point in the sparsity table is incremented. Otherwise, the value from the delta table is read and used.

[0118] When the sparse table is not too large (in the alternative case), the gain of lossless requantization is important: for example, coding the model with 5 bits instead of 10 bits, or 8 bits instead of 16 bits, without loss. Coding gain for delta data video coding with minimal overhead is achieved with sparse data storage in SEI.

[0119] According to another embodiment, one radius per component may be used, and thus, different quantizations may be used for each component if necessary.

[0120] According to one embodiment, the values ​​of the component of the attribute to be encoded / decoded include residual values ​​obtained from predictions based on previously scanned values ​​according to the scan order of the vertices of the mesh, as described in the Edgebreaker algorithm.

[0121] According to another embodiment, the radius is determined by a multipass encoding scheme.

[0122] According to these embodiments, the maximum value R m is a radius R determined separately for all frames I of the sequence or for a subset of frames. iIt is determined over. Therefore, according to this embodiment, the number of bits for all frames or a subset is Br m = floor(log2(2*R m It is set to )+1).

[0123] According to another embodiment, when real-time encoding is required, for example for broadcasting, a forward approach may be devised to determine the radius. In this case, for each new frame, instead of using a deviation to determine the optimal bit depth, a fixed bit depth, for example, a user or application-defined bit depth, is used. All values ​​within this bit range are encoded using video coding, and the remaining values ​​are encoded into SEI messages using quantization reframing for the sparse points of the multipass solution.

[0124] According to one embodiment, the bitstream decoded in FIG. 15 is adapted to a V-PCC decoder. As illustrated in FIG. 22, the bitstream includes a syntax element (reframe_flag) indicating whether reframing of all or part of the attributes is enabled. According to a variation, a syntax element indicating whether reframing is enabled is set in the bitstream for each attribute.

[0125] The bitstream also includes information (radius) representing at least one radius. As previously described, the radius may be set individually for each component of each attribute, in which case the bitstream includes information representing the radius for each component of each attribute. Alternatively, the radius may be the same for all components of the attribute, in which case the bitstream includes information representing the radius for all components of the attribute. According to another embodiment, the radius may be the same for all attributes, in which case the bitstream includes information representing the radius for all attributes.

[0126] According to another embodiment, the radius may be changed in the sequence, for example, at each key frame. Accordingly, in this embodiment, the radius is signaled to the bitstream at each key frame.

[0127] FIG. 23a illustrates an example of metadata encoded for a second subset. For each sparse table encoded as metadata, the metadata includes the number of encoded sparse points of an attribute (Num_sparsePts), the raw or entropy-coded values ​​of the sparse points (SparsePtsValues), and their indices (SparsePtsIndex). FIG. 23b illustrates an example of metadata encoded for a second subset according to another embodiment, wherein reframing is permitted for the sparse data. In that embodiment, the metadata also includes a syntax element (RfS) indicating whether the sparse table is reframed. Thus, if the sparse table is reframed, the metadata includes the radius size (Radius') for the reframed sparse tables.

[0128] According to one example of the principles of the present invention illustrated in FIG. 24, in a transmission context between two remote devices A and B through a communication network NET, device A includes a processor associated with memory RAM and ROM configured to implement a method for encoding a 3D object as described in relation to FIG. 1 to FIG. 23b, and device B includes a processor associated with memory RAM and ROM configured to implement a method for decoding a 3D object as described in relation to FIG. 1 to FIG. 23b.

[0129] According to one example, the network is a broadcast network adapted to broadcast / transmit signals from device A to decoding devices including device B.

[0130] A signal intended to be transmitted by device A transmits at least one bitstream generated by a method for encoding a 3D object according to any one of the embodiments described above.

[0131] FIG. 25 illustrates an example of the syntax of such a signal transmitted via a packet-based transmission protocol. Each transmitted packet (P) includes a header (H) and a payload (PAYLOAD). According to embodiments, the payload (PAYLOAD) may include at least one of the following syntax elements: coded image data representing attribute values, coded metadata representing attribute values, a syntax element indicating whether reframing of all or part of the attributes is enabled, and a final radius size. The payload may also include any one of the syntax elements discussed above.

[0132] FIG. 26 illustrates an embodiment of a method (2600) for transmitting a signal according to any one of the embodiments described above. Such a method comprises the step (2601) of accessing data containing such a signal, and the step (2602) of transmitting the accessed data through a communication channel that may be implemented, for example, within a wired and / or wireless medium. According to one embodiment, the method may be performed by a device (100) illustrated in FIG. 1 or device A from FIG. 24.

[0133] Various methods are described herein, and each method comprises one or more steps or actions to achieve the described method. Unless a specific order of steps or actions is required for the proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Additionally, terms such as “first,” “second,” etc., may be used in various embodiments, for example, “first decoding” and “second decoding,” to modify elements, components, steps, actions, etc. The use of such terms does not imply a sequencing of modified actions unless specifically required. Accordingly, in such examples, the first decoding does not need to be performed before the second decoding and may occur, for example, before, during, or overlapping with the second decoding.

[0134] Furthermore, the embodiments described herein are not limited to VVC or HEVC and may be applied, for example, to other standards and recommendations, and any extensions of such standards and recommendations. Unless otherwise indicated or technically excluded, the embodiments described herein may be used individually or in combination.

[0135] Various numerical values ​​are used in this application. Specific values ​​are for exemplary purposes, and the described embodiments are not limited to these specific values.

[0136] Various embodiments involve decoding. "Decoding" as used in this application may include, for example, all or part of the processes performed on a received encoded sequence to produce a final output suitable for a display. In various embodiments, these processes include one or more of the processes typically performed by a decoder, for example, entropy decoding, inverse quantization, inverse transformation, and differential decoding. Whether the phrase "decoding process" is intended to specifically indicate a subset of operations or to generally indicate a broader decoding process will be clear based on the context of the specific descriptions and is believed to be well understood by those skilled in the art.

[0137] Various embodiments involve encoding. In a manner similar to the above discussion regarding “decoding,” “decoding” as used in this application may include all or part of the processes performed, for example, on an input video sequence to generate an encoded bitstream.

[0138] The embodiments and aspects described herein may be implemented, for example, in a method or process, a device, a software program, a data stream, or a signal. Even if discussed only in the context of a single form of embodiment (e.g., discussed only as a method), embodiments of the features discussed may also be implemented in other forms (e.g., a device or a program). A device may be implemented, for example, in suitable hardware, software, and firmware. Methods may be implemented in a device such as a processor, generally referred to as a processing device, which includes, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, a computer, a cellular phone, a portable / personal digital assistant ("PDA"), and other devices that facilitate the communication of information between end users.

[0139] References to "one embodiment," "one example," "one embodiment," or "one embodiment," as well as other variations thereof, imply that specific features, structures, properties, etc. described in relation to the embodiment are included in at least one embodiment. Accordingly, the phrases "in one embodiment," "in one example," "in one embodiment," or "in one embodiment," as well as the appearance of any other variations appearing throughout this application, do not necessarily all refer to the same embodiment.

[0140] Additionally, the present application may refer to "determining" various types of information. Determining information may include, for example, estimating information, calculating information, predicting information, or retrieving information from memory.

[0141] Additionally, the present application may refer to "accessing" various information. Accessing information may include, for example, receiving information, retrieving information (for example, from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0142] Additionally, the present application may refer to "receiving" various types of information. Receiving is intended to be a broad term, just like "accessing." Receiving information may include, for example, accessing information, or retrieving information (for example, from memory). Furthermore, "receiving" is typically involved in one way or another during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0143] For example, it should be understood that in the following cases of “A / B”, “A and / or B”, and “at least one of A and B”, the use of any of “ / ”, “and / or”, and “at least one” is intended to include the selection of the first enumerated option (A) alone, or the selection of the second enumerated option (B) alone, or the selection of both options (A and B). As another example, in the cases of “A, B and / or C” and “at least one of A, B and C”, these phrases are intended to include the selection of the first enumerated option (A) alone, or the selection of the second enumerated option (B) alone, or the selection of the third enumerated option (C) alone, or the selection of the first and second enumerated options (A and B) alone, or the selection of the first and third enumerated options (A and C) alone, or the selection of the second and third enumerated options (B and C) alone, or the selection of all three options (A, B and C). This can be extended to many items as described in this specification, as will be apparent to those skilled in the art and to this specification.

[0144] Additionally, as used herein, the term "signal" refers to indicating something specifically to a corresponding decoder. For example, in certain embodiments, the encoder signals a quantization matrix for dequantization. In this way, in one embodiment, the same parameter is used on both the encoder side and the decoder side. Thus, for example, the encoder may transmit a specific parameter to the decoder so that the decoder can use the same specific parameter (explicit signaling). Conversely, if the decoder already has other things in addition to the specific parameter, signaling may be used without transmission (implicit signaling) simply to allow the decoder to know and select the specific parameter. By avoiding the transmission of any actual functions, bit saving is realized in various embodiments. It should be understood that signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder in various embodiments. Although the aforementioned expression relates to the verb form of the word "signal," the word "signal" may also be used as a noun in this specification.

[0145] As is apparent to those skilled in the art, embodiments may generate various signals formatted to convey information that may be stored or transmitted, for example. The information may include, for example, instructions for performing a method, or data generated by one of the described embodiments. For example, a signal may be formatted to convey a bitstream of the described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information conveyed by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over various different wired or wireless links. The signal may be stored on a processor-readable medium.

Claims

Claim 1 A method comprising the step of encoding at least one component of at least one attribute of an animated 3D mesh, wherein the encoding step comprises: obtaining an ordered sequence of residual prediction values ​​of at least one component of at least one attribute of an animated 3D mesh; reframing the residual prediction values ​​of the ordered sequence into a first subset and a second subset, wherein the first subset includes residual prediction values ​​of at least one component within a range of values, and the second subset includes residual prediction values ​​of at least one component outside a range of values; encoding the first subset using a video-based encoder; and encoding the second subset as metadata. Claim 2 A device comprising one or more processors, wherein the one or more processors are configured to encode at least one component of at least one attribute of an animated 3D mesh, and the encoding comprises obtaining an ordered sequence of residual prediction values ​​of at least one component of at least one attribute of an animated 3D mesh, reframing the residual prediction values ​​of the ordered sequence into a first subset and a second subset — wherein the first subset includes residual prediction values ​​of at least one component within a range of values, and the second subset includes residual prediction values ​​of at least one component outside a range of values ​​— encoding the first subset using a video-based encoder, and encoding the second subset as metadata. Claim 3 In claim 1, the second subset is encoded in the SEI message, method. Claim 4 A method according to claim 1, wherein the ordered sequence of residual prediction values ​​is obtained from an Edgebreaker algorithm. Claim 5 A method according to claim 1, wherein the sequence of residual prediction values ​​is ordered according to the order in which the vertices of the animated 3D mesh are visited to encode the topology of the animated 3D mesh. Claim 6 The method of claim 1, wherein the step of reframing residual prediction values ​​of at least one component of at least one attribute into a first subset and a second subset comprises: determining the first subset and the second subset as a function of a radius parameter defining a range of values; and encoding the radius parameter. Claim 7 A method according to claim 6, wherein the radius parameter is determined as half of the maximum deviation of the residual prediction values ​​of at least one component of at least one attribute. Claim 8 A method according to claim 6, wherein a radius parameter is determined for each of at least two frames of an animated 3D mesh, and the range of values ​​is determined as a function of the maximum value among the determined radius parameters. Claim 9 A method according to paragraph 3, wherein the first subset comprises residual prediction values ​​of at least one component of at least one attribute, and at least one residual prediction value corresponding to a residual prediction value belonging to the second subset is replaced in the first subset with a value determined from at least one value of the first subset, or the first subset comprises residual prediction values ​​of at least one component of at least one attribute, and residual prediction values ​​belonging to the second subset are removed. Claim 10 A device according to paragraph 2, wherein encoding the second subset comprises encoding the position of the at least one residual prediction value in an ordered sequence of residual prediction values ​​of at least one component of at least one attribute for at least one residual prediction value of the second subset. Claim 11 A computer-readable storage medium storing instructions for causing one or more processors to perform the method of claim 1. Claim 12 A method comprising the step of decoding at least one component of at least one attribute of an animated 3D mesh, wherein the decoding step comprises the step of decoding a first subset of values ​​using a video-based decoder, the step of decoding metadata representing a second subset of values, and the step of reconstructing an ordered sequence of residual prediction values ​​of at least one component of at least one attribute from the values ​​of the first subset and the values ​​of the second subset, wherein the values ​​of the first subset are within a range of values ​​and the values ​​of the second subset are outside a range of values. Claim 13 A device comprising one or more processors, wherein the one or more processors are configured to decode at least one component of at least one attribute of an animated 3D object, and the decoding comprises decoding a first subset of values ​​using a video-based decoder, decoding metadata representing a second subset of values, and reconstructing an ordered sequence of residual prediction values ​​of at least one component of at least one attribute from the values ​​of the first subset and the values ​​of the second subset, wherein the values ​​of the first subset are within a range of values ​​and the values ​​of the second subset are outside a range of values. Claim 14 In claim 12, the step of decoding metadata representing a second subset of values ​​comprises the step of decoding the position of said at least one value in at least one component of said at least one attribute for at least one value of the second subset. Claim 15 An apparatus according to claim 13, wherein reconstructing at least one component of at least one attribute from a first subset and a second subset comprises replacing a value placed at a position of a value belonging to the second subset in the first subset of values ​​with a corresponding value from the second subset, or inserting values ​​from the second subset into a corresponding position in the first subset of values. Claim 16 A method according to claim 1 or 12, wherein at least one attribute comprises at least one of geometric coordinates of a 3D mesh, texture coordinates within a frame atlas, normal coordinates, color data, or general data. Claim 17 A computer-readable storage medium storing instructions for one or more processors to perform the method of claim 12. Claim 18 A device comprising: - a device according to claim 13; and - at least one of (i) an antenna configured to receive a signal including data representing an animated 3D mesh, (ii) a band limiter configured to limit the signal to a band of frequencies including data representing an animated 3D mesh, or (iii) a display configured to display an animated 3D mesh. Claim 19 In paragraph 18, a device including a TV, mobile phone, tablet, or set-top box. Claim 20 A computer-readable storage medium storing a bitstream comprising coded data representing an ordered sequence of residual prediction values ​​of at least one component of at least one attribute of an animated 3D mesh, wherein the coded data comprises: - coded video data representing a first subset of values ​​of the at least one component, and - coded metadata representing a second subset of values ​​of the at least one component, wherein at least one component of the at least one attribute is reframed into a first subset and a second subset, wherein the first subset comprises residual prediction values ​​of at least one component within a range of values, and the second subset comprises residual prediction values ​​of at least one component outside a range of values. Claim 21 A computer-readable storage medium according to claim 20, further comprising at least one syntax element indicating whether reframing of residual prediction values ​​of at least one component of at least one attribute is enabled, or a radius parameter used to determine a first subset and a second subset. Claim 22 delete Claim 23 delete Claim 24 delete Claim 25 delete Claim 26 delete Claim 27 delete Claim 28 delete Claim 29 delete Claim 30 delete Claim 31 delete Claim 32 delete Claim 33 delete Claim 34 delete Claim 35 delete Claim 36 delete Claim 37 delete Claim 38 delete Claim 39 delete Claim 40 delete Claim 41 delete Claim 42 delete Claim 43 delete Claim 44 delete Claim 45 delete Claim 46 delete

Images