Displacement packing using single LoD per block

Abstract

Each block single level of detail (LoD) shift packing method enables slice decoding, scalability, and other video processing implementations. By using per-block LoD packing, the sequence shift bit rate is merely changed to a minimum extent.

Description

[0001] Cross Reference to Related Applications

[0002] This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application Serial No. 63 / 496,968, entitled “DISPLACEMENT PACKING USING SINGLE LoD PER BLOCK” and filed on April 19, 2023, which is incorporated by reference herein in its entirety for all purposes. TECHNICAL FIELD

[0003] The present invention relates to three-dimensional graphics. More specifically, the present invention relates to dynamic mesh coding. BACKGROUND

[0004] Recently, a new approach to compress volumetric content such as point clouds based on projection from 3D to 2D is being standardized. This approach, also known as V3C (Compression of Visual Volumetric Video) maps 3D volumetric data into several 2D facets, which are then further arranged into atlas images, which are subsequently encoded with a video encoder. The atlas images correspond to the geometry of the points, the corresponding texture and an occupancy map, which indicates which positions in the point cloud reconstruction are to be considered.

[0005] In 2017, MPEG issued a Call for Proposals (CfP) on compression of point clouds. After evaluating several proposals, MPEG is currently considering two different technologies for point cloud compression: either a 3D native coding technology (based on octree and similar coding methods), or a 3D to 2D projection followed by traditional video coding. In the case of dynamic 3D scenes, MPEG is using a Test Model Software (TMC2) based on a facet surface modeling, projection of the facets from 3D images to 2D images and encoding of the 2D images with a video encoder such as HEVC. This approach has proven to be more efficient than native 3D coding and enables competitive bitrates with acceptable quality.

[0006] Due to the success of the projection-based approach (also known as video-based approach or V-PCC) to encode 3D point clouds, it is expected that the standard will include more 3D data such as 3D meshes in future versions. However, the current version of the standard is only suitable for the transmission of a set of unconnected points, so there is no mechanism to send the connectivity of the points, which is required in 3D mesh compression.

[0007] Methods to extend the functionality of V-PCC to meshes have also been proposed. One possible way is to encode the vertices using V-PCC and then encode the connectivity using a mesh compression method such as TFAN or Edgebreaker. The limitation of this approach is that the original mesh must be dense so that the point cloud generated from the vertices is not sparse and can be efficiently encoded after projection. In addition, the order of the vertices affects the encoding of the connectivity, thus different methods to reorganize the mesh connectivity have been proposed. An alternative way to encode sparse meshes is to encode the vertex positions in 3D using RAW patches metadata. Since RAW patches directly encode (x,y,z), in this approach all the vertices are encoded as RAW data and the connectivity is encoded by a similar mesh compression method as described before. Note that in RAW patches the vertices can be sent in any preferred order, thus the order resulting from the connectivity encoding can be used. This approach can encode sparse point clouds, however, RAW patches are not efficient to encode 3D data and more data such as attributes of the triangular faces can be lost from this approach.

[0008] UVAtlas from Microsoft is the state of the art automatic texture mapping generation, but it requires a lot of time and is optimized for local frames only. V-PCC uses equirectangular projection to generate patches, but only for point clouds, so it does not solve the patch generation problem for meshes. SUMMARY

[0009] Each block single level of detail (LoD) displacement packing method enables slice decoding, scalability, and other video processing implementations. By using each block LoD packing implementation, only the lowest level of sequence displacement bitrate is changed.

[0010] In another aspect, a method programmed in a non-transitory memory of a device includes packing displacement information for a single level of detail (LoD) into one or more blocks and packing padding bits until an end of a current block of the one or more blocks. The method also includes repeating, for a subsequent LoD, packing displacement information for a single LoD into one or more blocks and packing padding bits until an end of a current block of the one or more blocks until all displacement information is packed. The method also includes receiving position count information indicating where each LoD begins and ends. Each subsequent LoD begins at the beginning of a block. The method also includes triggering each block LoD displacement packing with a flag. The method also includes dividing mesh information into different LoDs. The method also includes performing slice decoding. The method also includes decoding displacement information.

[0011] In another aspect, an apparatus includes a non-transitory memory to store an application and a processor coupled to the memory, the application to: pack displacement information for a single level of detail (LoD) into one or more blocks and pack padding bits up to an end of a current block of the one or more blocks, the processor configured to process the application. The application is further configured to, for a subsequent LoD, repeat packing displacement information for a single LoD into one or more blocks and packing padding bits up to an end of a current block of the one or more blocks until all displacement information is packed. The application is further configured to receive position count information indicating where each LoD starts and ends. Each subsequent LoD starts at the beginning of a block. The application is further configured to trigger each block LoD displacement packing with a marker. The application is further configured to divide mesh information into different LoDs. The application is further configured to perform slice decoding. The application is further configured to decode displacement information.

[0012] In another aspect, a system includes an encoder configured to encode a 3D mesh and a decoder configured to: pack displacement information for a single level of detail (LoD) into one or more blocks and pack padding bits up to an end of a current block of the one or more blocks. The decoder is further configured to, for a subsequent LoD, repeat packing displacement information for a single LoD into one or more blocks and packing padding bits up to an end of a current block of the one or more blocks until all displacement information is packed. The decoder is further configured to receive position count information indicating where each LoD starts and ends. Each subsequent LoD starts at the beginning of a block. The decoder is further configured to trigger each block LoD displacement packing with a marker. The decoder is further configured to divide mesh information into different LoDs. The decoder is further configured to perform slice decoding. The decoder is further configured to decode displacement information. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 A diagram illustrating a V-DMC decoder in accordance with some embodiments.

[0014] Figure 2 A diagram illustrating a previous packing implementation and a per block LoD packing implementation in accordance with some embodiments.

[0015] Figure 3 A diagram illustrating a packed image of displacements in accordance with some embodiments.

[0016] Figure 4 A flow diagram illustrating a per block single LoD displacement packing method in accordance with some embodiments.

[0017] Figure 5 Figure illustrates a block diagram of an exemplary computing device configured to implement a per-block single LoD displacement packing method, in accordance with some embodiments. DETAILED DESCRIPTION

[0018] A mesh is composed of a set of polygons that typically describe surfaces of a volume. One effective way to encode a dynamic mesh is to simplify the mesh to a base mesh, encode the base mesh with any mesh encoder, and then encode the displacement information, i.e., the difference between the subdivided base mesh and the original input mesh. The displacement information is provided per vertex and can be efficiently encoded with video. The displacement information is packed in the video data one level of detail (LoD) after another in raster scan order. This prevents the use of video tools that enable independent encoding of blocks, such as slices, because in one single block, there can be two or more LoD data present at the same time.

[0019] A method of packing displacement information is described herein that first divides the mesh information into levels of detail and then packs each level of detail data in an independent block. This allows for slice decoding of independent LoD and also enables decoder side scalability. In addition, a signal is provided that employs a high level syntax that allows for packing of blocks of different LoD. For example, the base mesh is level 0, each edge is split in half, all vertices in one half are level 1, and as edges are split in half, new levels exist (e.g., level 2, 3, etc.). Other tricks can be done based on different LoD. If a LoD does not fill a block, the remaining bits of that block can be filled with zeros or other fillers so that the next LoD starts at the beginning of the next block.

[0020] Figure 1 Figure illustrates a diagram of a V-DMC decoder, in accordance with some embodiments. The V-DMC decoder 100 includes many standard components. The image unpack component 102 is used to implement displacement packing with a single LoD per block. After the subdivision component, position Countl and LoD are generated, which go to the image unpack component 102. positionCountl can be used to know where each LoD starts and ends.

[0021] Figure 2 Figure illustrates a diagram of a previous packing implementation and a per-block LoD packing implementation, in accordance with some embodiments. The displacement packing stream 200 shows each LoD packed next to each other (e.g., LoD-0 followed by LoD-1, and so on). The displacement packing stream 202 shows a padding gap between each LoD (e.g., LoD-0 followed by zeros, then LoD-1, and so on).

[0022] Image 202 illustrates a previous packing implementation in which each LoD is packed next to each other with no gap in between so that different LoDs can be located in the same tile. For example, tiles 204, 206, and 208 show two different LoDs in the same tile. Continuing the example, tile 204 includes LoD-0 and LoD-1 in the same tile, tile 206 includes LoD-1 and LoD-2 in the same tile, and tile 208 includes LoD-2 and LoD-3 in the same tile. Different LoDs have different hatching / shading / patterns to distinguish from each other. For example, in the figure, LoD-0 has a cross-hatching pattern, LoD-1 has lines from bottom left to top right, and so on.

[0023] Image 252 illustrates a per-tile LoD packing implementation in which multiple LoDs are not allowed to occupy the same tile. For example, padding bits 260 (e.g., bits equal to 0) are used to occupy space to prevent multiple LoDs from occupying the same tile. Continuing the example, LoD-0 ends in tile 254, so padding bit 260 is used to fill tile 254. Then, LoD-1 can be packed into the next tile. LoD-1 ends in tile 256, so padding bit 260 fills the remaining portion of tile 256. Then LoD-2 starts in the next tile and ends in tile 258, which is filled by padding bit 260. LoD-3 then starts in the next tile until it ends, and the remaining portion of the tile is filled by padding bit 260. Although there are more "edges" in the video image, by having only one LoD per tile, slice decoding and other video processing techniques can be implemented.

[0024] Figure 3 An image illustrating displacement packing according to some embodiments is illustrated. Image 300 shows a video image with a previous packing implementation. Image 350 shows a video image with a per-tile LoD packing implementation. Gap 352 is shown as described herein in which padding bits are used to ensure that two different LoDs are not in the same tile. By using a per-tile LoD packing implementation, only the lowest level of sequence displacement bit rate is changed; however, slice decoding can now be performed, which could not be performed with the previous packing implementation. For example, if it is desired to decode the first level, it is not necessary to decode all of the video, only the first two tiles, since the first level is contained only in the first two tiles.

[0025] An example syntax for LoD packing includes:

[0026] bool

[0027] VMCEncoder::computeDisplacementVideoFramePerLoD(

[0028] const VMCFrame& frame,

[0029] Frame<uint16_t>&

[0030] dispVideoFrame, / / ,ColourSpace::YUV400p,ColourSpace::YUV444p

[0031] const VMCEncoderParameters& params) {

[0032] const auto pixelsPerBlock =

[0033] params.geometryVideoBlockSize*params.geometryVideoBlockSize;

[0034] const auto shift = uint16_t((1 << params.geometryVideoBitDepth) >>1);

[0035] const auto& disp = frame.disp;

[0036] const auto planeCount = dispVideoFrame.planeCount();

[0037] for (int32_t p = 0; p < planeCount; ++p) {

[0038] dispVideoFrame.plane(p).fill(shift);

[0039] }

[0040] auto numLods = frame.subdivInfoLevelOfDetails.size();

[0041] std::vector <int>numBlocksInLod;

[0042] numBlocksInLod.resize(numLods, 0);

[0043] numBlocksInLod[0] = std::ceil((double)frame.subdivInfoLevelOfDetails[0].pointCount / (double)pixelsPerBlock);

[0044] for (int i = 1; i < numLods; i++) {

[0045] auto& prevlod = frame.subdivInfoLevelOfDetails[i - 1];

[0046] auto& lod = frame.subdivInfoLevelOfDetails[i];

[0047] int numPointsInLoD = lod.pointCount - prevlod.pointCount;

[0048] numBlocksInLod[i] = std::ceil((double)numPointsInLoD / (double)pixelsPerBlock);

[0049] }

[0050] const int32_t totalBlocksInVideoFrame = (dispVideoFrame.width() * dispVideoFrame.height() - 1) / pixelsPerBlock;

[0051] int startBlock = 0;

[0052] for (int32_t lodIdx = 0; lodIdx < numLods; lodIdx++) {

[0053] auto& lod = frame.subdivInfoLevelOfDetails[lodIdx];

[0054] int vStart = 0;

[0055] int vEnd = lod.pointCount;

[0056] if (lodIdx > 0) {

[0057] auto& prevlod = frame.subdivInfoLevelOfDetails[lodIdx - 1];

[0058] vStart = prevlod.pointCount;

[0059] startBlock += numBlocksInLod[lodIdx - 1];

[0060] }

[0061] for (int32_t v = vStart; v < vEnd; ++v) {

[0062] const auto& d = disp[v];

[0063] auto blockIndex = (v - vStart) / pixelsPerBlock + startBlock;

[0064] auto indexWithinBlock = (v - vStart) % pixelsPerBlock;

[0065] if (params.displacementReversePacking) {

[0066] blockIndex = totalBlocksInVideoFrame - blockIndex;

[0067] indexWithinBlock = pixelsPerBlock - 1 - indexWithinBlock;

[0068] }

[0069] const auto x0 = (blockIndex % params.geometryVideoWidthInBlocks)

[0070] * params.geometryVideoBlockSize;

[0071] const auto y0 = (blockIndex / params.geometryVideoWidthInBlocks)

[0072] * params.geometryVideoBlockSize;

[0073] int32_t x = 0;

[0074] int32_t y = 0;

[0075] computeMorton2D(indexWithinBlock, x, y);

[0076] assert(x < params.geometryVideoBlockSize);

[0077] assert(y < params.geometryVideoBlockSize);

[0078] const auto x1 = x0 + x;

[0079] const auto y1 = y0 + y;

[0080] for (int32_t p = 0; p < planeCount; ++p) {

[0081] const auto dshift = int32_t(shift + d[p]);

[0082] assert(dshift>=0&&dshift<(1<< params.geometryVideoBitDepth ));

[0083] dispVideoFrame.plane(p).set(y1, x1, uint16_t(dshift));

[0084] }

[0085] }

[0086] }

[0087] return true;

[0088] }

[0089] if (_sps.displacementLoDPacking) {

[0090] reconstructDisplacementFromVideoFramePerLod(dispVideo.frame(frameIndex),

[0091] frame,

[0092] rec,

[0093] _sps.geometryVideoBlockSize,

[0094] _sps.geometryVideoBitDepth,

[0095] _sps.displacementReversePacking);

[0096] }

[0097] else {

[0098] reconstructDisplacementFromVideoFrame(dispVideo.frame(frameIndex),

[0099] frame,

[0100] rec,

[0101] _sps.geometryVideoBlockSize,

[0102] _sps.geometryVideoBitDepth,

[0103] _sps.displacementReversePacking);

[0104] }

[0105] static int32_t

[0106] reconstructDisplacementFromVideoFramePerLod(

[0107] const Frame<uint16_t>& dispVideoFrame,

[0108] VMCFrame & frame,

[0109] const TriangleMesh <meshtype>& rec,

[0110] const int32_t geometryVideoBlockSize,

[0111] const int32_t geometryVideoBitDepth,

[0112] const int32_t displacementReversePacking) {

[0113] printf("Reconstruct displacements from video frame per LoD\n");

[0114] fflush(stdout);

[0115] const auto geometryVideoWidthInBlocks =

[0116] dispVideoFrame.width() / geometryVideoBlockSize;

[0117] const auto pixelsPerBlock = geometryVideoBlockSize *geometryVideoBlockSize;

[0118] const auto shift = uint16_t((1 << geometryVideoBitDepth) >> 1);

[0119] const auto planeCount = dispVideoFrame.planeCount();

[0120] const auto pointCount = rec.pointCount();

[0121] auto& disp = frame.disp;

[0122] const int32_t totalBlocksInVideoFrame = (dispVideoFrame.width() *dispVideoFrame.height() - 1) / pixelsPerBlock;

[0123] disp.assign(pointCount, Vec3 <double>(0));

[0124] auto numLods = frame.subdivInfoLevelOfDetails.size();

[0125] std::vector <int>numBlocksInLod;

[0126] numBlocksInLod.resize(numLods, 0);

[0127] numBlocksInLod[0] = std::ceil((double)frame.subdivInfoLevelOfDetails[0].pointCount / (double)pixelsPerBlock);

[0128] for (int i = 1; i < numLods; i++) {

[0129] auto& prevlod = frame.subdivInfoLevelOfDetails[i - 1];

[0130] auto& lod = frame.subdivInfoLevelOfDetails[i];

[0131] int numPointsInLoD = lod.pointCount - prevlod.pointCount;

[0132] numBlocksInLod[i] = std::ceil((double)numPointsInLoD / (double)pixelsPerBlock);

[0133] }

[0134] int startBlock = 0;

[0135] for (int32_t lodIdx = 0; lodIdx < numLods; lodIdx++) {

[0136] auto& lod = frame.subdivInfoLevelOfDetails[lodIdx];

[0137] int vStart = 0;

[0138] int vEnd = lod.pointCount;

[0139] if (lodIdx > 0) {

[0140] auto& prevlod = frame.subdivInfoLevelOfDetails[lodIdx - 1];

[0141] vStart = prevlod.pointCount;

[0142] startBlock += numBlocksInLod[lodIdx - 1];

[0143] }

[0144] for (int32_t v = vStart; v < vEnd; ++v) {

[0145] auto blockIndex = (v - vStart) / pixelsPerBlock + startBlock;

[0146] auto indexWithinBlock = (v - vStart) % pixelsPerBlock;

[0147] if (displacementReversePacking) {

[0148] blockIndex = totalBlocksInVideoFrame - blockIndex;

[0149] indexWithinBlock = pixelsPerBlock - 1 - indexWithinBlock;

[0150] }

[0151] const auto x0 = (blockIndex % geometryVideoWidthInBlocks)

[0152] * geometryVideoBlockSize;

[0153] const auto y0 = (blockIndex / geometryVideoWidthInBlocks)

[0154] * geometryVideoBlockSize;

[0155] int32_t x = 0;

[0156] int32_t y = 0;

[0157] computeMorton2D(indexWithinBlock, x, y);

[0158] assert(x < geometryVideoBlockSize);

[0159] assert(y < geometryVideoBlockSize);

[0160] const auto x1 = x0 + x;

[0161] const auto y1 = y0 + y;

[0162] auto& d = disp[v];

[0163] for (int32_t p = 0; p < planeCount; ++p) {

[0164] const auto& plane = dispVideoFrame.plane(p);

[0165] d[p] = double(plane.get(y1, x1)) - shift;

[0166] }

[0167] }

[0168] }

[0169] return 0;

[0170] }

[0171] Atlas Sequence Parameter Set Extension RBSP Syntax

[0172] asps_vmc_extension() { descriptor

[0173] asps_vmc_ext_prevent_geometry_video_conversion_flagu(1)

[0174] asps_vmc_ext_prevent_attribute_video_conversion_flagu(1)

[0175] asps_vmc_ext_subdivision_method u(3)

[0176] if (asps_vmc_ext_subdivision_method!= 0)

[0177] asps_vmc_ext_subdivision_iteration_count u(8)

[0178] asps_vmc_ext_displacement_coordinate_system u(1)

[0179] asps_vmc_ext_transform_method u(3)

[0180] if (asps_vmc_ext_transform_method == LINEAR_LIFTING) {

[0181] vmc_lifting_transform_parameters(0, 0)

[0182] }

[0183] asps_vmc_ext_num_attribute_video u(7)

[0184] for (i=0; i< asps_vmc_ext_num_attribute_video; i++) {

[0185] asps_vmc_ext_attribute_type_id [ i ] u(8)

[0186] asps_vmc_ext_attribute_frame_width[ i ] ue(v)

[0187] asps_vmc_ext_attribute_frame_height[ i ] ue(v)

[0188] asps_vmc_vmc_ext_attribute_transform_method[ i ] u(3)

[0189] if (asps_vmc_ext_attribute_transform_method == LINEAR_LIFTING) {

[0190] vmc_lifting_transform_parameters( i + 1, 0 )

[0191] asps_vmc_ext_direct_attribute_projection_enabled_flag[ i ]u(1)

[0192] }

[0193] asps_vmc_ext_packing_method u(1)

[0194] asps_vmc_ext_lod_block_packing u(1)

[0195] As shown, a flag (asps_vmc_ext_lod_block_packing) can be set for LoD block packing. asps_vmc.ext_block-packing equal to 1 indicates that block boundaries are respected and displacement components are packed per level of detail. asps_vmc.ext_duplicate_vertex_removal_flag equal to 0 indicates that displacements for all levels of detail are packed together.

[0196] Next, an inverse packing process is invoked with parameters width, height, bitDepth, dispQuantCoeffFrame, blockSize, positionCountl, inverseDisplacementPacking, lodBlockPacking, subdivisionIterationCount, and levelOfDetailPositionCounts as inputs and dispQuantCoefArray as output.

[0197] if ( lodBlockPacking ) {

[0198] blocksInVideo = ( width * height - 1 ) / pixelsPerBlock

[0199] numBlocksInLod[0]=

[0200] ( levelOfDetailAttributeCounts[0] + pixelsPerBlock / 2 ) / pixelsPerBlock

[0201] for (i = 1; i < subdivisionIterationCount + 1; i++) {

[0202] numBlocksInLod[i] =

[0203] (levelOfDetailAttributeCounts[i] - levelOfDetailAttributeCounts[i - 1]

[0204] + pixelsPerBlock / 2) / pixelsPerBlock

[0205] }

[0206] startBlock = 0

[0207] for (lodIdx = 0; lodIdx < subdivisionIterationCount + 1; lodIdx++) {

[0208] vStart = 0

[0209] vEnd = levelOfDetailAttributeCounts[lodIdx]

[0210] if (lodIdx > 0) {

[0211] vStart = levelOfDetailAttributeCounts[lodIdx - 1]

[0212] startBlock = startBlock + numBlocksInLod[lodIdx - 1]

[0213] }

[0214] for (v = vStart; v < vEnd; v++) {

[0215] blockIndex = v / pixelsPerBlock

[0216] indexWithinBlock = v % pixelsPerBlock

[0217] if (inverseDisplacementPacking) {

[0218] blockIndex = blocksInVideo - blockIndex

[0219] indexWithinBlock = pixelsPerBlock – 1 - indexWithinBlock

[0220] }

[0221] x0 = (blockIndex % widthInBlocks) * blockSize

[0222] y0 = (blockIndex / widthInBlocks) * blockSize

[0223] (x, y) = computeMorton2D(indexWithinBlock)

[0224] x = x0 + x

[0225] y = y0 + y

[0226] for( d = 0; d < 3; d++ ) {

[0227] dispQuantCoeffArray[ v ][ d ] = dispQuantCoeffFrame[ x ][ y ][ d ] –shift

[0228] }

[0229] }

[0230] }

[0231] } else {

[0232] start = width * height - 1

[0233] for( vIdx = 0; vIdx < positionCount; vIdx++ ) {

[0234] if( inverseDisplacementPacking )

[0235] v = start - vIdx

[0236] else

[0237] v = vIdx

[0238] blockIndex = v / pixelsPerBlock

[0239] indexWithinBlock = v % pixelsPerBlock

[0240] x0 = (blockIndex % widthInBlocks) * blockSize

[0241] y0 = (blockIndex / widthInBlocks) * blockSize

[0242] (x, y) = computeMorton2D(indexWithinBlock)

[0243] x = x0 + x

[0244] y = y0 + y

[0245] for (d = 0; d < 3; d++ ) {

[0246] dispQuantCoeffArray[v][d] = dispQuantCoeffFrame[x][y][d] - shift

[0247] }

[0248] }

[0249] }

[0250] Figure 4 A flowchart illustrating a per-block single LoD displacement packing method according to some embodiments is shown. At step 400, position count information and LoD displacement information are received. For example, the position count information indicates the start and end of each LoD. At step 402, the displacement information for a single LoD (e.g., LoD-0) is packed (e.g., packed in a video bitstream). If the displacement information for the single LoD includes more bits than a single block in the bitstream, the subsequent block is padded. At step 404, when the end of the displacement information for the single LoD is reached, padding bits (e.g., zeros) are packed until the current block is padded. If the displacement information for a LoD ends at the end of a block, no padding bits are used for that block. The process is repeated until the displacement information for each LoD is packed, with each LoD starting at the beginning of a block. For example, the displacement information for LoD-0 is packed, followed by padding bits, then the displacement information for LoD-1 is packed, followed by padding bits, and so on until the last LoD (e.g., LoD-n) is packed. In another example, the displacement information for LoD-0 fills the first block, and then fills a portion of the second block, so the remaining portion of the second block is padded with padding bits. Then, the displacement information for LoD-1 starts at the beginning of the third block, and continues as described above. Two LoDs are not in the same block; rather, a single block will have one LoD and potentially padding bits. In some embodiments, fewer or additional steps are implemented. For example, steps are triggered based on a marker occurrence indication and / or to initiate per-block LoD displacement packing. In yet another example, dividing mesh information into different LoDs can be implemented. In another example, per-block LoD displacement packing enables further video processing, such as slice decoding and scalability. In some embodiments, decoding of the information is implemented. In some embodiments, the order of the steps is modified.

[0251] Figure 5 The diagram illustrates a block diagram of an exemplary computing device configured according to some embodiments to implement a displacement packing method for each block of a single LoD. The computing device 500 is capable of acquiring, storing, computing, processing, transmitting, and / or displaying information such as images and videos including 3D content. The computing device 500 is capable of implementing any aspect of encoding / decoding. Typically, a suitable hardware architecture for implementing the computing device 500 includes a network interface 502, a memory 504, a processor 506, one or more I / O devices 508, a bus 510, and a storage device 512. The choice of processor is not critical, as long as a suitable processor with sufficient speed is selected. The memory 504 can be any conventional computer memory known in the art. The storage device 512 can include a hard disk drive, CD-ROM, CDRW, DVD, DVDRW, high-definition optical disc / drive, ultra-high-definition drive, flash memory card, or any other storage device. The computing device 500 can include one or more network interfaces 502. Examples of network interfaces include network interface cards (NICs) connected to Ethernet or other types of LANs. I / O device 508 may include one or more of the following: keyboard, mouse, monitor, screen, printer, modem, touchscreen, button interface, and other devices. Each block single LoD offset packing application 530, used to implement the per-block single LoD offset packing method, may be stored in storage device 512 and memory 504, and processed in a manner typically used when processing applications. Computing device 500 may include... Figure 5 The diagram shows more or fewer components. In some embodiments, it includes a single LoD displacement package hardware 520 for each block. Although Figure 5 The computing device 500 includes an application 530 and hardware 520 for a per-block single LoD displacement packing method; however, the per-block single LoD displacement packing method can be implemented on the computing device using hardware, firmware, software, or any combination thereof. For example, in some embodiments, the per-block single LoD displacement packing application 530 is programmed in memory and executed using a processor. In another example, in some embodiments, the per-block single LoD displacement packing hardware 520 is programmable hardware logic, including logic gates specifically designed to implement the per-block single LoD displacement packing method.

[0252] In some embodiments, each block of single LoD displacement package application 530 includes several applications and / or modules. In some embodiments, the modules further include one or more sub-modules. In some embodiments, fewer or additional modules may be included.

[0253] Examples of suitable computing devices include personal computers, laptop computers, computer workstations, servers, mainframes, handheld computers, personal digital assistants, cellular / mobile telephones, smart appliances, gaming consoles, digital cameras, digital camcorders, camera phones, smart phones, portable music players, tablet computers, mobile devices, video players, video disc recorders / players (e.g., DVD recorders / players, high-definition disc recorders / players, ultra-high-definition disc recorders / players), televisions, home entertainment systems, augmented reality devices, virtual reality devices, smart jewelry (e.g., smart watches), vehicles (e.g., self-driving vehicles), or any other suitable computing device.

[0254] To utilize the per-block single LoD displacement packing method, a device obtains or receives 3D content (e.g., point cloud content). The per-block single LoD displacement packing method can be implemented with user assistance or can be implemented automatically without user involvement.

[0255] In operation, the per-block single LoD displacement packing method can enable slice decoding and other video processing implementations. By using the per-block LoD packing implementation, only the lowest level of sequence displacement bit rate is changed.

[0256] Some embodiments using per-block single LoD displacement packing

[0257] 1. A method programmed in a non-transitory memory of a device, comprising:

[0258] packing displacement information for a single level of detail (LoD) into one or more blocks; and

[0259] packing padding bits until an end of a current one of the one or more blocks.

[0260] 2. The method of clause 1, further comprising repeating, for a subsequent LoD, packing displacement information for a single LoD into one or more blocks and packing padding bits until an end of a current one of the one or more blocks until all displacement information is packed.

[0261] 3. The method of clause 1, further comprising receiving position count information indicating where each LoD begins and ends.

[0262] 4. The method of clause 1, wherein each subsequent LoD begins at a beginning of a block.

[0263] 5. The method of clause 1, further comprising triggering each block LoD displacement packing with a flag.

[0264] 6. The method of clause 1, further comprising dividing mesh information into different LoDs.

[0265] 7. The method of clause 1, further comprising performing slice decoding.

[0266] 8. The method of clause 1, further comprising decoding displacement information.

[0267] 9. An apparatus comprising:

[0268] a non-transitory memory storing an application, the application to:

[0269] pack displacement information of a single level of detail (LoD) into one or more blocks; and

[0270] pack padding bits until an end of a current block of the one or more blocks; and

[0271] a processor coupled to the memory, the processor configured to process the application.

[0272] 10. The apparatus of clause 9, wherein the application is further configured to repeat, for a subsequent LoD, packing displacement information of a single LoD into one or more blocks and packing padding bits until an end of a current block of the one or more blocks until all displacement information is packed.

[0273] 11. The apparatus of clause 9, wherein the application is further configured to receive position count information indicating where each LoD starts and ends.

[0274] 12. The apparatus of clause 9, wherein each subsequent LoD starts at a beginning of a block.

[0275] 13. The apparatus of clause 9, wherein the application is further configured to trigger each block LoD displacement packing with a marker.

[0276] 14. The apparatus of clause 9, wherein the application is further configured to divide mesh information into different LoDs.

[0277] 15. The apparatus of clause 9, wherein the application is further configured to perform slice decoding.

[0278] 16. The apparatus of clause 9, wherein the application is further configured to decode displacement information.

[0279] 17. A system comprising:

[0280] an encoder configured to encode a 3D mesh; and

[0281] a decoder configured to:

[0282] pack displacement information for a single level of detail (LoD) into one or more blocks; and

[0283] pack padding bits until the end of a current block of the one or more blocks.

[0284] 18. The system of clause 17, wherein the decoder is further configured to repeat, for a subsequent LoD, packing displacement information for a single LoD into one or more blocks and packing padding bits until the end of a current block of the one or more blocks until all displacement information is packed.

[0285] 19. The system of clause 17, wherein the decoder is further configured to receive position count information indicating where each LoD begins and ends.

[0286] 20. The system of clause 17, wherein each subsequent LoD begins at the start of a block.

[0287] 21. The system of clause 17, wherein the decoder is further configured to trigger each block LoD displacement packing with a marker.

[0288] 22. The system of clause 17, wherein the decoder is further configured to divide mesh information into different LoDs.

[0289] 23. The system of clause 17, wherein the decoder is further configured to perform slice decoding.

[0290] 24. The system of clause 17, wherein the decoder is further configured to decode displacement information.

[0291] The application has been described in reference to specific embodiments thereof. It will be apparent to those skilled in the art that various modifications can be made to the referenced embodiments without departing from the spirit or scope of the application as defined by the appended claims. Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. Various embodiments and / or examples of the present application are described herein. Embodiments described reflect the application of the present application to one or more particular scenarios. Those of ordinary skill in the art will recognize that the present application is not limited to the embodiments described and / or examples presented.< / int> < / double> < / meshtype> < / int>

Claims

1. A method programmed in a non-transitory memory of a device, comprising: packing displacement information for a single level of detail (LoD) into one or more blocks; and packing padding bits until the end of a current block of the one or more blocks.

2. The method of claim 1, further comprising repeating, for a subsequent LoD, packing displacement information for a single LoD into one or more blocks and packing padding bits until the end of a current block of the one or more blocks until all displacement information is packed.

3. The method of claim 1, further comprising receiving position count information indicating where each LoD begins and ends.

4. The method of claim 1, wherein each subsequent LoD begins at the start of a block.

5. The method of claim 1, further comprising triggering each block LoD displacement packing with a marker.

6. The method of claim 1, further comprising dividing mesh information into different LoDs.

7. The method of claim 1, further comprising performing slice decoding.

8. The method of claim 1, further comprising decoding displacement information.

9. An apparatus, comprising: a non-transitory memory for storing an application, the application for: packing displacement information for a single level of detail (LoD) into one or more blocks; and packing padding bits until the end of a current block of the one or more blocks; and a processor coupled to the memory, the processor configured to process the application.

10. The apparatus of claim 9, wherein the application is further configured for repeating, for a subsequent LoD, packing displacement information for a single LoD into one or more blocks and packing padding bits until the end of a current block of the one or more blocks until all displacement information is packed.

11. The apparatus of claim 9, wherein the application is further configured for receiving position count information indicating where each LoD begins and ends.

12. The apparatus of claim 9, wherein each subsequent LoD begins at the start of a block.

13. The apparatus of claim 9, wherein the application is further configured for triggering each block LoD displacement packing with a marker.

14. The apparatus of claim 9, wherein the application is further configured for dividing mesh information into different LoDs.

15. The apparatus of claim 9, wherein the application is further configured for performing slice decoding.

16. The apparatus of claim 9, wherein the application is further configured for decoding displacement information.

17. A system, comprising: an encoder, the encoder configured for encoding a 3D mesh; and a decoder, the decoder configured for: packing displacement information for a single level of detail (LoD) into one or more blocks; and packing padding bits until the end of a current block of the one or more blocks.

18. The system of claim 17, wherein the decoder is further configured to repeat packing displacement information for a single LoD into one or more blocks and packing padding bits until the end of a current block in the one or more blocks until all displacement information is packed for subsequent LoDs.

19. The system of claim 17, wherein the decoder is further configured to receive position count information indicating where each LoD starts and ends.

20. The system of claim 17, wherein each subsequent LoD starts at the beginning of a block.

21. The system of claim 17, wherein the decoder is further configured to trigger each block LoD displacement packing with a marker.

22. The system of claim 17, wherein the decoder is further configured to divide mesh information into different LoDs.

23. The system of claim 17, wherein the decoder is further configured to perform slice decoding.

24. The system of claim 17, wherein the decoder is further configured to decode displacement information.

Images