Projection of 3D point clouds for image-based compression of attributes

Abstract

Methods, apparatus and data stream are provided for encoding and decoding a three-dimensional (3D) point cloud. The 3D point cloud is processed to obtain a coded geometry, a reconstructed geometry obtained by decoding the coded geometry and adapted attributes. The adapted attributes are obtained by tracking indices of points of the point cloud or by transferring 5 attributes to the reconstructed geometry. The 3D point cloud is divided in a regular grid of blocks, projecting adapted attributes of points of each non-empty block on inner faces of the blocks and selecting one of the inner faces to obtain patches. The patches are packed in an atlas image according to a scanning process and the atlas image is encoded in a data stream with metadata indicating selected inner faces.

Description

[0001] Docket No. 2024P00802WQ

[0002] PROJECTION OF 3D POINT CLOUDS FOR IMAGE-BASED COMPRESSION

[0003] OF ATTRIBUTES

[0004] 1. Technical Field

[0005] The present principles generally relate to the domain of encoding, transmitting and decoding 3D point clouds with attributes. In particular, the present principles also relate to methods and devices to encode and decode the attributes of a 3D point cloud as patch atlases.

[0006] 2. Background

[0007] The present section is intended to introduce the reader to various aspects of art, which may be related to various aspects of the present principles that are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present principles. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art.

[0008] Dynamic 3D point clouds are a useful way to represent volumetric video. However, the encoding of 3D point clouds with attributes requires a huge amount of data. In the present application, color is often used to illustrate how the attributes are encoded according to the present principles. Color is only an example of attributes, the present principles apply to any kind of per point attributes like normal vectors, transparency, reflectance, haptics, etc. A way to encode geometry and or attributes of a 3D point cloud consists in projecting the points onto 2D surfaces and then encoding the surfaces by using 2D codecs, leveraging high end image or video coding methods. This is, for example, the approach that is used in MPEG V-PCC, using video codec, to encode positions and attributes of PCs. However, with these techniques, some points may be missed (because occulted at the projection step), leading to visual artifacts at the rendering because information is missing. The present principles propose methods and devices to overcome this drawback.

[0009] 3. Summary

[0010] The following presents a simplified summary of the present principles to provide a basic understanding of some aspects of the present principles. This summary is not an extensive Docket No. 2024P00802WQ overview of the present principles. It is not intended to identify key or critical elements of the present principles. The following summary merely presents some aspects of the present principles in a simplified form as a prelude to the more detailed description provided below.

[0011] Methods, apparatus and data stream are provided for encoding and decoding a three- dimensional (3D) point cloud. The 3D point cloud is processed to obtain a coded geometry, a reconstructed geometry obtained by decoding the coded geometry and adapted attributes. The adapted attributes are obtained by tracking indices of points of the point cloud or by transferring attributes to the reconstructed geometry. The 3D point cloud is divided in a regular grid of blocks, projecting adapted attributes of points of each non-empty block on inner faces of the blocks and selecting one of the inner faces to obtain patches. The patches are packed in an atlas image according to a scanning process and the atlas image is encoded in a data stream with metadata indicating selected inner faces. In an embodiment, non-projected adapted attributes are detected and predicted and related residuals are encoded in the data stream.

[0012] 4. Brief Description of Drawings

[0013] The present disclosure will be better understood, and other specific features and advantages will emerge upon reading the following description, the description making reference to the annexed drawings wherein:

[0014] - Figure 1A illustrates a method for encoding a point cloud according to the present principles;

[0015] - Figure IB shows a first embodiment of geometry encoding module according to the present principles;

[0016] - Figure 1C, shows a second embodiment of geometry encoding module according to the present principles;

[0017] - Figure 2 details attribute encoding module of Figure 1A, according to the present principles;

[0018] - Figure 3 shows an example architecture of a device which may be configured to implement a method for encoding attributes of a point cloud according to embodiments of the present principles;

[0019] - Figure 4 shows an example of an embodiment of the syntax of a stream when the data are transmitted over a packet-based transmission protocol; Docket No. 2024P00802WQ

[0020] 5. Detailed description of embodiments

[0021] The present principles will be described more fully hereinafter with reference to the accompanying figures, in which examples of the present principles are shown. The present principles may, however, be embodied in many alternate forms and should not be construed as limited to the examples set forth herein. Accordingly, while the present principles are susceptible to various modifications and alternative forms, specific examples thereof are shown by way of examples in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit the present principles to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present principles as defined by the claims.

[0022] The terminology used herein is for the purpose of describing particular examples only and is not intended to be limiting of the present principles. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises", "comprising," "includes" and / or "including" when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Moreover, when an element is referred to as being "responsive" or "connected" to another element, it can be directly responsive or connected to the other element, or intervening elements may be present. In contrast, when an element is referred to as being "directly responsive" or "directly connected" to other element, there are no intervening elements present. As used herein the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as" / ".

[0023] It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element without departing from the teachings of the present principles.

[0024] Although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows. Docket No. 2024P00802WQ

[0025] Some examples are described with regard to block diagrams and operational flowcharts in which each block represents a circuit element, module, or portion of code which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in other implementations, the function(s) noted in the blocks may occur out of the order noted. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending on the functionality involved.

[0026] Reference herein to “in accordance with an example” or “in an example” means that a particular feature, structure, or characteristic described in connection with the example can be included in at least one implementation of the present principles. The appearances of the phrase in accordance with an example” or “in an example” in various places in the specification are not necessarily all referring to the same example, nor are separate or alternative examples necessarily mutually exclusive of other examples.

[0027] Reference numerals appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims. While not explicitly described, the present examples and variants may be employed in any combination or sub-combination.

[0028] Figure 1A illustrates a method 10 for encoding a point cloud according to the present principles. An input 3D point cloud is obtained from a source. The input 3D point cloud comprises an input geometry 11 and input attributes 12. The source may be a memory storage or a stream obtained from a network. At a step 11, the input geometry of the 3D point cloud (that is the coordinates of each point in a frame of reference of the 3D space) and the input attributes (that are, per point, color, normal vector, transparency, etc.) are obtained from the input point cloud and processed so the geometry is encoded to obtain a coded geometry, the coded geometry is decoded to obtain a decoded geometry and the attributes are adapted to obtain adapted attributes. Two embodiments are possible at this last step.

[0029] Figure IB shows a first embodiment of geometry encoding module 13 according to the present principles. In this embodiment, input geometry 11 is encoded by a lossless method, that is, point indices are tracked. The adapted attributes 16 are input attributes 12. Figure 1C, shows a second embodiment of geometry encoding module 13 according to the present principles. In this embodiment, input geometry 11 is encoded by a lossy method that shuffles the coordinates of the points. Input attributes 12 are then transferred to the decoded geometry and so, adapted Docket No. 2024P00802WQ atributes are linked to decoded points as a function of input atributes 12 and correspond to the transferred atributes.

[0030] Figure 2 details atribute encoding module 17 of Figure 1A, according to the present principles. The proposed method is twofold. First, per point atributes are projected on 2D patches and the 2D patches are arranged (also called placed) onto a 2D patch atlas. In an embodiment, the creation and placement of patches are recomputed at the decoding side from coded geometry 15. In this embodiment, metadata describing parameters used for patches are not necessary. The data stream is lighter but computational resources are needed at the decoder. In another embodiment, metadata to retrieve projection parameters for each patch are associated with the patch atlas. Attribute encoding module 17 receives decoded geometry 15 and adapted atributes 16. Second, the present principles comprise methods to handle the attributes of points that are missed during the projection, for example because of occlusion. Missing atributes can be predicted at decoding by using incremental interpolation and an advanced inpainting process can be performed to fill the holes. At the encoding, residuals are computed by using incremental interpolation. The residuals are encoded. Missing atributes are predicted at the decoding by using incremental interpolation. Residuals are decoded and applied to predicted values.

[0031] The 3D space of the point cloud is divided in a regular 3D grid of blocks. The empty blocks are discarded. At a step 21, decoded geometry 15 and adapted atributes 16 are projected onto the six inner faces of each non-empty block to obtain 6 images. Even with this six directions projection, some points may be occluded and, so, are projected on one of the six images. At a step 22, these points are detected and stored in memory. In order to perform the processing, the point cloud is read from file as two tables, one of positions quantized in range [0,gridSize[ and one of colors, color of index i in color table is the color of point i in position table. Both tables have the same size. 3D “occupancy” is stored in memory in a table posOcc of size gridSize x gridSize x gridSize. Each value of the table stores the index of the point in the position and in the color table or -1. The following code illustrates the process. Docket No. 2024P00802WQ

[0032] This data structure is memory consuming but allows for 0(1) lookups for a voxel i,j,k. Other data structures can be used to get more compact memory such as association maps indexed by i,j,k, in this case lookups would in the order of 3 * o(log( gridSize).

[0033] At step 21, patches are generated by projections. An orthogonal projection method to be very fast. The point cloud is constrained to be axis aligned, quantized and cubicle of size gridSize (e.g. 1024x1024x1024, if gridSize=1024) and is decomposed into regular axis aligned cubicle blocks of size blockSize (e.g. 8x8x8 if blockSize=8 or 32x32x32 if blockilze=32). The blocks that contain no point from the original PC are discarded. For all the other blocks, the points that it contains are orthogonally projected respectively onto the 6 sides of the block box. The projection with a maximum of nearest projected points and the smallest average distance to the side is selected and stored as a 2D image of size blockSize x blockSize. The axis / side this image is attached to is also stored as metadata. So, the selected image stores a subset of the colors of the point set contained by the block. Indeed, the patch does not store all the color information of a block due to occlusion issues during the orthogonal projection. The list of point indices that are the source for these projected colors is kept, so, the occluded points can be detected at step 22. The selected image and its associated information constitute a patch. An Docket No. 2024P00802WQ

[0034] “occupancy map” is also generated. That is an image of the same size of the color image and store for each pixel if a color exists in the color map or not (this is used later to fill the empty parts by using a padding algorithm). At the end of the process, a scan going from bottom to top (or top to bottom), of the point cloud, storing for each slice the set of patches is obtained. During step 21, the blocks are processed in a specific order starting from the bottom left comer (or top left) of the point cloud and scanning horizontally each slice of the volume made of blocks. This set of slices is used at step 23 for unrolling and placing the patches, to accelerate the algorithm. A patch is stored in memory in a data structure, presented, for example as a C++ class. Docket No. 2024P00802WQ

[0035] CubePatch(int blockSize, glm::ivec2 axis): blockSize( blockSize), axis(_axis) inline void reset(int bi, int bj, int bk) { blockPos = glm::ivec3(bi,bj,bk); iFlip = false; proj Count = 0; avgDist = O.Of; map2dPos!dx.assign(blockSize * blockSize, -1); map2dDis.assign(blockSize * blockSize, blockSize);

[0036] / / return true if patch is vertical when seen from top (i.e. left or right proj) inline bool isV(void) { return axis[0] == 1; }

[0037] / / return true if patch is horizontal when seen from top (i.e. front or back proj) inline bool isH(void) { return axis[l] == 1; }

[0038] The construction algorithm is presented hereafter. bool computeProjections( mm::ModelPtr inputModel, std::vector<int>& posOcc, Docket No. 2024P00802WQ int gridSize, / / width of the cubical PC grid, must be a multiple of blocSize int blockSize, / / size of cubical bloc bool frameChanged)

[0039] / / 0- unprojected, 1- projected, 2- col interpolated std::vector<char> Hags(inputModel->getPositionCount(), 0);

[0040] / / int totalPointCount=0; / / total number of points int totalProjCount=0; / / total number of projected points

[0041] / / the final set of patches std::vector<CubePatch> patchSet; int minBj = 10000000; int maxBj = 0; int maxBlockPerLine = 0;

[0042] / / block patches cache, one for each projection direction

[0043] / / order here is important since patches will be stored in

[0044] / / their order of enum if score is identical std: : vector<CubePatch> cubePatches; cubePatches. emplace_back(blocSize, glm::ivec2(0, +1)); / / right cubePatches. emplace_back(blocSize, glm::ivec2(0, -1)); / / left cubePatches. emplace_back(blocSize, glm::ivec2(2, -1)); / / back Docket No. 2024P00802WQ cubePatches. emplace_back(blocSize, glm::ivec2(2, +1)); / / front cubePatches. emplace_back(blocSize, glm::ivec2(l, +1)); / / top cubePatches. emplace_back(blocSize, glm::ivec2(l, -1)); / / bottom

[0045] / / for each block const auto numBlocs = gridSize / blocSize;

[0046] / / vertical scan, slice by slice (start by top just for easy debug" for (auto bJ = numBlocs - 1; bJ >= 0; — bJ) { int lineBlocks = 0; / / number of blocks found for the current slice

[0047] / / the set of patches for the slice, will be reordered later into patchSet std: : vector<CubePatch> slicePatchSet;

[0048] / / int sliceStart = patchSet. size(); for (auto bK = 0; bK < numBlocs; ++bK) {

[0049] / / we prefer to have i axis blocks side by side if possible, so last loop for (auto bl = 0; bl < numBlocs; ++bl) { const auto iStart = bl * blocSize; const auto j Start = bJ * blocSize; const auto kStart = bK * blocSize;

[0050] / / used to allocate a patch only if at least one voxel is not empty for this block bool allocated = false;

[0051] / / iterate inside the block int nbPoints!n3dBlock = 0; glm::ivec3 posInBlock; for (pos!nBlock[0] = 0; pos!nBlock[0] < blocSize; ++pos!nBlock[0]) { Docket No. 2024P00802WQ Docket No. 2024P00802WQ int pj = posInBlock[axis2]; if (axisl == 1) { std::swap(pi, pj); / / align j to up const int pk = patch, axis [1] == +1 ?

[0052] (blocSize - posInBlock[axisO] - 1) : posInBlock[axisO] ; if (pk < patch. map2dDis [pi * blocSize + pj]) { patch.map2dDis[pi * blocSize + pj] = pk;

[0053] / / first time we hit this pixel if (patch. map2dPos!dx[pi * blocSize + pj] == -1) { ++patch. proj Count; patch.map2dPosIdx[pi * blocSize + pj] = posOcc[vox!dx]; ct best patch within the 6 patches ocated ){ ompute average distance to block side for each patch (auto pldx = 0; pldx < cubePatches. size(); ++pldx) { Docket No. 2024P00802WQ auto& patch = cubePatches [pldx]; double distSum = 0; for (auto i = 0; i < patch.blockSize; ++i) { for (auto j = 0; j < patch.blockSize; ++j) { const auto stride = i * patch.blockSize + j; / / we skip the points with no projection / / their distance is still set to max if (patch.map2dPosIdx[stride] >= 0) { distSum += patch. map2dDis [stride]; patch.avgDist = (float)(distSum / patch.proj Count); / find the best patch nt bestPatchldx = 0; nt proj Count = 0; loat avgDist = blockSize.val + 1; / search for best projection or (auto i = 0; i < cubePatches. size(); ++i) { const auto& patch = cubePatches[i];

[0054] / / strict test so first in list is selected (*) if (patch. projCount > projCount) { bestPatchldx = i; Docket No. 2024P00802WQ projCount = patch.projCount;

[0055] / / then search for smallest dist among the two remaining patches if (cubePatches [bestPatchldx + l].avgDist cubePatches [bestPatchldx] . avgDist)

[0056] / / recall that pat-ches are ordered (see previous (*)) bestPatchldx = bestPatchldx + 1;

[0057] / / store selected patch to the slice totalProj Count += projCount; const auto& selectedPatch = cubePatches [bestPatchldx]; slicePatchSet.push back(selectedPatch); maxBlockPerLine = std::max(maxBlockPerLine, lineBlocks);

[0058] / / compute a new order as well as flip flags for the blocks of this slice

[0059] / / see next section of the document std: : vector<int> newOrder(slicePatchSet.size()); computeNewOrderCandidate( gridSize, slicePatchSet, Docket No. 2024P00802WQ

[0060] Once all the slices are generated, an atlas image is generated at step 24 by writing the pixels of each patch (and flipping the patch image when needed) in the atlas image (also called final image herein). Step 24 to compute the atlas image is presented hereafter: Docket No. 2024P00802WQ outputOcc->reset(imgWidth, imgHeight, 0); int oldJstart = -1; int iBStart = 0;

[0061] / / iterate over all the blocks / patches for (auto blockldx = 0; blockldx < patchSet. size(); ++blockldx) { auto& map2d = patchSet[blockIdx].map2dPosIdx; const auto jStart = (patchSet[block!dx].blockPos[l] - minBj) * blocSize; if (j Start != oldJstart) { iBStart = 0; else {

[0062] ++iB Start; oldJstart = jStart; const auto iStart = iBStart * blocSize;

[0063] / / we start a new slice for (int i = 0; i < blocSize; ++i) { / / columns int dir = 1 ; int start = 0; int end = blocSize- 1; for (int j = start; j != end + dir; j = j + dir) { / / rows const auto pointldx = map2d[i * blocSize + j]; if (pointldx >= 0) { flags [pointldx] = 1; / / mark point as selected Docket No. 2024P00802WQ const glm::vec3 pos( inputModel->vertices[pointIdx * 3 + 0], inputModel->vertices[pointIdx * 3 + 1], inputModel->vertices[point!dx * 3 + 2] ); const glm::vec3 color( inputModel->colors[pointIdx * 3 + 0], inputModel->colors[pointIdx * 3 + 1], inputModel->colors[point!dx * 3 + 2]

[0064] );

[0065] / / do we mirror the columns ? int post, posJ;

[0066] / / if flip we mirror the columns if (patchSet[blockIdx].iFlip ) { post = iStart + (blocSize - 1 - i); posJ = imgHeight - 1 - (j Start + j); else { post = iStart + i; posJ = imgHeight - 1 - (j Start + j); outputMap->storeRGB(posI, posJ, color); outputOcc->storeRGB(posI, posJ, glm::vec3(255, 255, 255)); Docket No. 2024P00802WQ

[0067] Once the image and occupancy map are generated it is possible to use standard padding methods to fill the black portions of the image. Such a process leads to better predictions by image coders. The goal of this step is to place the 2D images of the patches, in a coherent location / orientation regarding its neighbor patches, inside a larger image that is encoded. The process is performed slice by slice by selecting successions of blocks representing contours of the objects. Each of these blocks may be flipped, so, the sides of the patch images are in correspondence in the atlas image. Heuristics and scanning the blocks of the slice are used for this process. A complete greedy algorithm for unrolling the patches is presented hereafter. Docket No. 2024P00802WQ for (int newPatchldx = 1; newPatchldx < sliceSize; ++newPatch!dx) { bool found = false;

[0068] / / we search for the best contour,

[0069] / / turning around 0,0 from previous voxel direction for (int seqldx = 0; seqldx < 8 && ! found; ++seqldx) { int i,j; getNextShift(prevI, prevJ, i, j); const auto idx = isCandidateAvailable(patchSet, selected, prevP atch->blockPos[0] + i, prevPatch->blockPos[2] + j); if (idx >=0) { newOrderfnewPatchldx] = idx; const auto patch = &patchSet[idx];

[0070] / / do we need flip ? bool flip = prevPatch->iFlip; if (Iflip

[0071] && patch->blockPos[0] <= prevPatch->blockPos[0]

[0072] && patch->blockPos[2] <= prevPatch->blockPos[2]) { flip = true; else if (flip

[0073] && patch->blockPos[0] >= prevPatch->blockPos[0]

[0074] && patch->blockPos[2] >= prevPatch->blockPos[2]) { flip = false; Docket No. 2024P00802WQ

[0075] / / prevPatch = patch; prevPatch->iFlip = flip; prevl = -i; prevJ = -j ; found = true; else { prevl = i; prevJ = j; f (! found) / / we select the first available for (int selldx = 0; selldx < selected. size() && ! found; ++selldx) { if (!selected[selldx]) { selected[selldx] = 1; const auto selPatchldx = selldx; newOrderfnewPatchldx] = selPatchldx; prevPatch = &patchSet[selPatch!dx]; prevPatch->iFlip = false; prevl = -1; prevJ = -1; found = true; Docket No. 2024P00802WQ

[0076] Where the getNextShift procedure is used to search a neighbor block (newl, new J) in a given order knowing the relative position (i,j), with i andj in {-1,0,1}, ofthe previously selected block with regard to the current block ( of coordinates 0,0 ). The coordinates are in 2D in the plan of the slice. The getNextShift procedure is defined as:

[0077] And the procedure isCandidateAvailable is defined as: Docket No. 2024P00802WQ

[0078] In an embodiment, prediction at encoding, comes at a step 26, after that the attributes of projected points have been encoded at step 24 and decoded at step 25. To predict the attribute Docket No. 2024P00802WQ of a point that was not projected, its closest projected neighbors are average. The point attributes for which the highest number of neighbors were projected (i.e. with highest prediction degree) are predicted. Once this first round of prediction is done, attributes of remaining points are predicted, this time including already predicted point attributes in addition to the projected ones. The method is decomposed as follows:

[0079] 1. For each non-projected point, its number of projected neighbors (maximum of 3x3x3- 1 values) is computed and stored. At this step, there is no predicted point yet.

[0080] 2. The color of non-projected and non-predicted points with highest count of projected and predicted neighbor points are interpolated using the colors of those neighbor points. Non-projected / non-predicted neighbors points of these newly predicted points get their counter incremented.

[0081] 3. Step 2 is iterated until the attributes of all the points are predicted.

[0082] At a step 27, the residuals are computed for each predicted point by computing the difference between the original decoded attribute and the predicted attribute. The obtained residuals can be coded by many means, using arithmetic coding (AC) or image / video coding at a step 28. Since values might be negative, they may be shifted by the overall minimum value, which value might be stored in the bitstream for shifting back at decoding. Other shifting methods can be used. Spatial coherence and locality are of interest while coding this kind of 3D localized values (potential neighborhoods similarly may lead to similar residuals). For example, Z-Order and Morton Codes to generate ID (for AC) or 2D (for image / video) orders coding can be used. In the case of AC coding, a new order using a 3D Z-order providing a unique ID Morton code per voxel (from 3D to ID). Each voxel (a.k.a. point) of index (i,j,k) then gets a unique index (the Morton code). The residuals are packed in a ID array by sorting using Morton code associated with their respective 3D point. The ID array of packed residuals is arithmetically encoded (using one context per color channel). In the case of image / video, a new order using a 3D Z-order providing unique ID Morton code per voxel (from 3D to ID) is computed and regenerated a 2D order from this packing which is used to store in the frame to code. Each voxel (point) of index (i,j,k) then gets a unique index (the Morton code).

[0083] Then, the points of the residuals are packed in a ID array by sorting using the Morton code associated to their respective 3D point. The residuals of packed points are stored in a 2D array (the image / frame to code) by using the ID index of each packed residual as a Morton Docket No. 2024P00802WQ code to generate its 2D position from Z-ordering. The image of packed residuals is encoded using image or video coder, using residuals per attribute channel at step 28.

[0084] At decoding, after the buffer of projected attributes is decoded and used to partially reconstruct the colors of the point cloud, the same z-order conversions are performed to find back the position, in ID buffer or in 2D image, of each non-projected voxel (i.e. block) to reconstruct the attributes by adding the decoded residuals on top of the predictions. Other ordering methods could be considered for the packing, to obtain even more locality, such as using Hilbert curves, for instance.

[0085] Figure 3 shows an example architecture of a device 30 which may be configured to implement a method for encoding attributes of a point cloud according to embodiments of the present principles. The device is linked with other devices via their bus 31 and / or via I / O interface 36.

[0086] Device 30 comprises following elements that are linked together by a data and address bus 31:

[0087] - a processor 32 (or CPU), which is, for example, a DSP (or Digital Signal Processor);

[0088] - a ROM (or Read Only Memory) 33;

[0089] - a RAM (or Random Access Memory) 34;

[0090] - a storage interface 35;

[0091] - an I / O interface 36 for reception of data to transmit, from an application; and

[0092] - a power supply (not represented in Figure 3), e.g. a battery.

[0093] In accordance with an example, the power supply is external to the device. In each of mentioned memory, the word « register » used in the specification may correspond to area of small capacity (some bits) or to very large area (e.g. a whole program or large amount of received or decoded data). The ROM 33 comprises at least a program and parameters. The ROM 33 may store algorithms and instructions to perform techniques in accordance with present principles. When switched on, the CPU 32 uploads the program in the RAM and executes the corresponding instructions.

[0094] The RAM 34 comprises, in a register, the program executed by the CPU 32 and uploaded after switch-on of the device 30, input data in a register, intermediate data in different states of the method in a register, and other variables used for the execution of the method in a register. Docket No. 2024P00802WQ

[0095] The implementations described herein may be implemented in, for example, a method or a process, an apparatus, a computer program product, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method or a device), the implementation of features discussed may also be implemented in other forms (for example a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end-users.

[0096] Device 30 is linked, for example via bus 31 to a set of sensors 37 and to a set of rendering devices 38. Sensors 37 may be, for example, cameras, microphones, temperature sensors, Inertial Measurement Units, GPS, hygrometry sensors, IR or UV light sensors or wind sensors. Rendering devices 38 may be, for example, displays, speakers, vibrators, heat, fan, etc.

[0097] In accordance with examples, the device 30 is configured to implement a method for encoding attributes of a point cloud, and belongs to a set comprising:

[0098] - a mobile device;

[0099] - a communication device;

[0100] - a game device;

[0101] - a tablet (or tablet computer);

[0102] - a laptop;

[0103] - a still picture camera;

[0104] - a video camera.

[0105] Figure 4 shows an example of an embodiment of the syntax of a stream when the data are transmitted over a packet-based transmission protocol. Figure 4 shows an example structure 4 of a stream encoding a 3D point cloud having attributes according to the present principle. The structure consists in a container which organizes the stream in independent elements of syntax. The structure may comprise a header part 41 which is a set of data common to every syntax element of the stream. For example, the header part comprises some of metadata about syntax elements, describing the nature and the role of each of them. The structure comprises a Docket No. 2024P00802WQ payload comprising an element of syntax 42 and at least one element of syntax 43 (there may be an element of syntax 43 for each attribute of the point cloud for example). Syntax element 42 comprises data representative of the 3D point cloud. It comprises data necessary to render the objects of the 3D scene (e.g. meshes, textures, lighting, etc.). Element of syntax 43 is a part of the pay load of the data stream and comprises the scene description. According to the present principles, metadata comprising every information needed for rendering the 3D point cloud with its attributes are encoded, for example, in element of syntax 43.

[0106] The implementations described herein may be implemented in, for example, a method or a process, an apparatus, a computer program product, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method or a device), the implementation of features discussed may also be implemented in other forms (for example a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, Smartphones, tablets, computers, mobile phones, portable / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end-users.

[0107] Implementations of the various processes and features described herein may be embodied in a variety of different equipment or applications, particularly, for example, equipment or applications associated with data encoding, data decoding, view generation, texture processing, and other processing of images and related texture information and / or depth information. Examples of such equipment include an encoder, a decoder, a post-processor processing output from a decoder, a pre-processor providing input to an encoder, a video coder, a video decoder, a video codec, a web server, a set-top box, a laptop, a personal computer, a cell phone, a PDA, and other communication devices. As should be clear, the equipment may be mobile and even installed in a mobile vehicle.

[0108] Additionally, the methods may be implemented by instructions being performed by a processor, and such instructions (and / or data values produced by an implementation) may be stored on a processor-readable medium such as, for example, an integrated circuit, a software carrier or other storage device such as, for example, a hard disk, a compact diskette (“CD”), an optical disc (such as, for example, a DVD, often referred to as a digital versatile disc or a digital Docket No. 2024P00802WQ video disc), a random access memory (“RAM”), or a read-only memory (“ROM”). The instructions may form an application program tangibly embodied on a processor-readable medium. Instructions may be, for example, in hardware, firmware, software, or a combination. Instructions may be found in, for example, an operating system, a separate application, or a combination of the two. A processor may be characterized, therefore, as, for example, both a device configured to carry out a process and a device that includes a processor-readable medium (such as a storage device) having instructions for carrying out a process. Further, a processor-readable medium may store, in addition to or in lieu of instructions, data values produced by an implementation.

[0109] As will be evident to one of skill in the art, implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the syntax of a described embodiment, or to carry as data the actual syntax-values written by a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.

[0110] A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. Additionally, one of ordinary skill will understand that other structures and processes may be substituted for those disclosed and the resulting implementations will perform at least substantially the same function(s), in at least substantially the same way(s), to achieve at least substantially the same result(s) as the implementations disclosed. Accordingly, these and other implementations are contemplated by this application.

Claims

1. Docket No. 2024P00802WQCLAIMS1. A method for encoding a three-dimensional (3D) point cloud, the method comprising:- processing the 3D point cloud to obtain a coded geometry, a reconstructed geometry obtained by decoding the coded geometry and adapted attributes;- dividing the 3D point cloud in a regular grid of blocks, projecting adapted attributes of points of each non-empty block on inner faces of the blocks and selecting one of the inner faces to obtain patches;- packing the patches in an atlas image according to a scanning process; and- encoding the atlas image in a data stream with metadata indicating selected inner faces.

2. The method of claim 1 further comprising detecting non-proj ected adapted attributes, predicting a value for each non-proj ected adapted attribute and encoding related residuals in the data stream.

3. The method of claim 2, wherein the predicting of the non-proj ected adapted attributes is based on a decoding of the packed adapted attributes.

4. The method of any one of claims 1 to 3, wherein the adapted attributes are obtained by tracking indices of points of the point cloud.

5. The method of any one of claims 1 to 3, wherein the adapted attributes are obtained by transferring attributes to the reconstructed geometry.

6. A device for encoding a three-dimensional (3D) point cloud, the device comprising a processor configured for:- processing the 3D point cloud to obtain a coded geometry, a reconstructed geometry obtained by decoding the coded geometry and adapted attributes;- dividing the 3D point cloud in a regular grid of blocks, projecting adapted attributes of points of each non-empty block on inner faces of the blocks and selecting one of the inner faces to obtain patches;- packing the patches in an atlas image according to a scanning process; and- encoding the atlas image in a data stream with metadata indicating selected inner faces.28Docket No. 2024P00802WQ7. The device of claim 6, wherein the processor is further configured for detecting nonprojected adapted attributes, predicting a value for each non-projected adapted attributes and encoding related residuals in the data stream.

8. The device of claim 7, wherein the predicting of the non-projected adapted attributes is based on a decoding of the packed adapted attributes.

9. The device of any one of claims 6 to 8, wherein the adapted attributes are obtained by tracking indices of points of the point cloud.

10. The device of any one of claims 6 to 8, wherein the adapted attributes are obtained by transferring attributes to the reconstructed geometry.

Images