Systems and methods for coding node vertices

Abstract

Methods and devices for coding a current node having at least two-edge vertices on a positive face of the current node. A centroid-vertex location for the current node is determined, and one or more face-vertex locations for the current node are determined. The centroid-vertex location and face-vertex locations are determined after determining one or more of a centroid-vertex location or face-vertex location for a positive neighboring node of the current node.

Description

SYSTEMS AND METHODS FOR CODING NODE VERTICESTECHNICAL FIELD

[0001] Disclosed are embodiments related to coding node vertices with respect to three-dimensional (3D) point cloud data, and in particular, coding of centroid vertices and control of parsing order of trisoup nodes.BACKGROUND

[0002] 3D point clouds can include an unstructured set of coordinates in 3D space, and are typically used to capture scene geometry and scale, for example, to represent 3D structures from the physical world. In addition to geometry, point clouds can store other information about the 3D points, so-called attributes. Typical attributes are color information, reflectance, and normal vectors, among others. Although applicable to point clouds with attributes, for the simplicity of presentation, aspects of the disclosure are presented in the context of geometry compression without attributes. This can include, for instance, compressing a set of 3D points D = (Xfe, Yk, Zk)k=1, where K is the number of points and Xk, Yk, Zkare coordinates for point k. Typical point clouds range in size from a few kilobytes to several gigabytes, which can put stress on an application requiring storage and / or transmission of such point clouds. Therefore, an efficient point cloud compression solution is an important enabler for industrial applications that rely on such point clouds.

[0003] Geometry based Point Cloud Compression (G-PCC) is an Moving Picture Experts Group (MPEG) standard that targets the use case of static point clouds, for instance, as described in Graziosi et al., An Overview of Ongoing Point Cloud Compression Standardization Activities: Video-Based (V-PCC) and Geometry-Based (G-PCC), APSIPA Transactions on Signal and Information Processing, 9(1), 2020. Additionally, there are inter tools for G-PCC, allowing compression of dynamic content. G-PCC uses octree coding to compress the geometry. Before applying this method, it is assumed that the coordinate values of the points in the point cloud have been quantized into integer coordinates and contained within a volume D x D x D . The point cloud is then partitioned into 8 sub-cubes with dimensions D / 2 x D / 2 x D / 2. If a sub-cube does not contain any points, it is unoccupied, and the partitioning process for this branch of the tree is terminated. This generates a tree structure (an octree), where each node can be represented using 8 bits, where each bit indicates the occupancy status of one sub-cube. This is shown in FIG. 1, which is an example 100 ofhow the position of two points can be represented by an octree. For lossy compression, octree partitioning is stopped at a pre-determined level, generating a sparser reconstruction, and the corresponding sequence of 8-bit words is entropy coded.

[0004] G-PCC also contains a module called triangle soup (trisoup), developed to favor surface point clouds, i.e., point clouds that are dense enough to capture surface structures. Just like regular octree G-PCC, this method uses octree coding to partition the point cloud into nodes (e.g., blocks having a width larger than 1). However, when using this module, the octree partitioning typically stops at a higher level in the tree, making the nodes larger. This level is pre-determined and set by the user / encoder. Instead of setting a fixed depth, the user sets a trisoup node size (nodeSize = 2n,n = 2, ...), where each node size corresponds to a depth in the octree where partitioning will stop. To represent the surfaces within the trisoup nodes, vertices are determined based on the point content of a node. In the decoder, the vertices are connected to each other forming surfaces in the form of triangles. There are, in the current version of the standard, three types of vertices: edge-vertices, centroid-vertices, and facevertices. How they are determined and signaled, includes the following.

[0005] First, the edge-vertices are determined. For instance, the edge-vertices 202a-202d are determined to be the position where the surface to be represented intersects the edges of the node 200a, as illustrated in FIG. 2A. An example of edge-vertex coding is provided in N. Svensson, Node Unique DSE, ISO / IEC JTC1 / SC29 / WG7 m63729, Geneva, July 2023. Typically, in the decoder, the edge-vertices are then connected via a centroid node 204 that is the arithmetic mean position of the edge-vertices, forming the triangles that the trisoup name refers to, as illustrated in FIG. 2B. In FIG. 2A and FIG. 2B, the edge-vertices are the solid black vertices positioned on the edge of the node where the surface intersects the edge.

[0006] In certain aspects, only signaling the edge-vertices could limit one to a coarse geometrical representation of the surfaces. Especially for low bitrates, where the node size is large, the representation will be coarse. This is illustrated in FIG. 3A, where a curved surface 302 is shown within a node. The curved surface would be reconstructed as a straight plane instead of an arc-shape. To ameliorate this issue, the codec has the possibility to add an offset 304 to the centroid vertices, as shown in FIG. 3B. Before calculating the length of this offset, the direction of the vector along which the offset will be shifted must be calculated. This is illustrated in FIG. 3B. For each triangle surrounding the centroid vertex, a normal will be calculated using a cross-product:= CEtx CEl+1, ( 1 )where C is the position of the centroid and Etis one of the edge-vertices and Ei+1is the upcoming vertex in clockwise order around the centroid. Then the offset-vector can be calculated as the average of those normal vectors:( 2 )In FIGs. 3A-3C, the centroid-vertex is illustrated as the hollow vertex positioned within the nodes.

[0007] Then the offset of the centroid vertex will be calculated as the average projected position along the offset- vector for all original points pt(not vertices) in the node that is within a certain threshold distance from the line segment that goes from the centroid to the edge of the node along in the direction of the offset- vector, as shown in FIG. 4, where all points within the dotted cylinder are projected to the offset-vector. The offset along the offset-vector is calculated as:(3) 1 offset = - ^ projfitpi - Cfi = 0where k is the number of points that are within this threshold. The distance from a point ptto the line segment, which has direction N and which is originating in centroid C is:d^ WN x Of - C)||2.Hence, all points where d; < th is used in equation (3). This is illustrated in FIG. 4, where all points inside the dotted cylinder contribute towards the offset length.

[0008] When the offset is applied to the original centroid it causes the reconstructed node to have the geometrical shape that is illustrated in FIG. 3C. Since the direction of the centroid vertex can be calculated using only the information from the edge-vertices, the only thing that needs to be signaled to the decoder is the offset of the centroid vertex.

[0009] However, the centroid vertex does not necessarily solve all problems with reconstruction. Instead of the reconstruction looking like an arc, or like part of a cylinder, it looks like a pyramid. Thus, recently within MPEG, the “face-vertex” was introduced to the codec. The face vertex is positioned on the face of a node, but only on the intersection of a line segment between two centroid vertices in adjacent nodes and the face that is the boundary between two nodes. Hence, the task of the encoder is to determine whether there are sufficiently many points in the surroundings of the intersection point between the line and the face, as illustrated with the larger circles 502 in FIG. 5A. If that is the case, for each face between two nodes where there is a centroid vertex activated in each node, one bit is signaled, telling the decoder whether to put a face vertex on this intersection point or not. In FIG. 5B, two face vertices are determined. The intersection point can be determined using the positions of the edge- and centroid-vertices, and therefore only one bit is required per face. The final reconstruction of the content of the node will look like FIG. 5C, which is significantly closer to the original arc-shape than the reconstruction in FIG. 3C. In FIGs. 5A-5C, the face-vertices are illustrated as the hollow and dotted vertices positioned on the faces of the node.

[0010] Typically, the above-described process is only performed for nodes with 4 or more edge-vertices. And originally, if there were 3 edge-vertices, only one triangle was used to represent the content of the node, since 3 vertices uniquely defines a plane. Additionally, the above-described process is only performed for nodes with 3 or more edge-vertices. If there only are 2 edge-vertices in a node, the vector that the centroid is offset along is determined using the neighboring centroid-vertex and the mean position of the 2 shared edge-vertices, as illustrated with respect to FIGs. 6A and 6B. These figures illustrate how to determine the orientation of the non-closed surface. The neighboring node used for prediction is the neighbor that shares the same 2 edge-vertices with the current node. Examples of how to model these surfaces are provided in N. Svensson, Reconstructing Non-Closed Surfaces in Trisoup Nodes, ISO / IEC JTC1 / SC29 / WG7 m66498, Online, January 2024. In sum, for certain nodes, one must know the information from the neighboring node (e.g., the position of the centroid), and this requires that the nodes to be parsed in a certain way.

[0011] When decoding the point cloud, surfaces of each node are reconstructed by populating all positions for points (called voxels) that intersect the modelled triangles using raytracing. Since the reconstructed point cloud will be quantized, the position coordinates of the resulting occupied voxels will be integers. A purpose of the trisoup module is to encode the point cloud at a low bit rate without losing much accuracy. Compared to octree G-PCC,the reconstructed point cloud will be denser when using trisoup, and this extra density is typically favored by the distortion metrics used in MPEG.SUMMARY

[0012] Certain challenges presently exist.

[0013] For instance, originally, when a node only contained 3 edge-vertices, the centroid was automatically deactivated, since 3 points defines a plane in 3D space. However, that has now been changed since the introduction of the face-vertex. After its introduction, the centroid-vertex is activated by default in all nodes with 2 or more edge-vertices, enabling Bjontegaard Delta rate (BD-rate) gains. However, for the case where the trisoup node only contains 3 edge-vertices, it is common to encode a zero offset. This results in signaling bits without improving the quality, which reduces efficiency.

[0014] Accordingly, in one aspect there is provided a method for coding a current node having three edge-vertices. The method comprises determining a distance between an initial centroid point and a projected centroid point, determining if a condition is satisfied based on the distance, and performing selective coding of a centroid-vertex for the current node based at least in part on whether the condition is satisfied. In embodiments, determining if a condition is satisfied comprises comparing the distance to a threshold value, and / or performing selective coding of the centroid-vertex based at least in part on comparing the distance to a threshold. In embodiments, performing selective coding of the centroid-vertex comprises coding the centroid-vertex when the distance is less than the threshold value, and not coding the centroidvertex when the distance is greater than or equal to the threshold value. In certain aspects, embodiments use the geometrical properties of neighboring nodes to predict whether a centroid vertex is necessary or not.

[0015] An advantage of embodiments disclosed herein is that they can reduce the size of the bitstream while not distorting the reconstruction significantly. Instead, the trade-off between bit rate and reconstruction distortion improves. This can be seen by the fact that embodiments will show consistent BD-rate gains on all sequences in the MPEG Common Test Conditions or “CTC” (both distortion metrics, DI and D2). In embodiments, determined information can be used to change the default position of the centroid-vertices to provide additional BD-rate gains.

[0016] As another example of challenges, certain limitations regarding coding order are currently in place. One such limitation is that the current specification draft only allowsthe decoder to perform one pass over the occupied trisoup nodes. Furthermore, this pass must be done in raster scan order. There are negative implications of this limitation.

[0017] As soon as the positions of the occupied trisoup nodes have been identified and encoded or decoded, only one pass over them is permitted, and in raster scan order. One form of raster scan order is illustrated in FIG. 7, and essentially requires first parsing the node that has x-position 0 and y-position 0, and incrementing in z-order. Once those nodes have been passed, the y-position is incremented by 1, and the nodes that have x-position 0 and y-position 1 are passed by incrementing in z-order. Once all nodes with x-position 0 have been passed, one increments the x-position by 1 and passes the second layer of nodes in the same fashion as the first layer. A different priority of the axes could have been chosen, for instance it could have been decided to increment the counters in x-y-z order instead. However, in the current draft of the standard the order is “z first, then y, and x last”.

[0018] To allow parallelization of the decoding process, “pipelining” processes have been introduced to the codec. In the current state of the codec, there are three pipeline steps. These are illustrated in FIG. 8, and the process can be described using the numbered list below, where the numbering describes the iteration number:1. Edge-vertices of slice 1 are encoded / decoded.2. Edge-vertices of slice 2 are encoded / decoded, centroid- and face-vertices of slice 1 are encoded / decoded.3. Edge-vertices of slice 3 are encoded / decoded, centroid- and face-vertices of slice 2 are encoded / decoded, slice 1 is being ray-traced.4. Edge-vertices of slice 4 are encoded / decoded, centroid- and face-vertices of slice 3 are encoded / decoded, slice 2 is being ray-traced.5. (repeated on next slices)In the example of FIG. 8, the comer dots represent edge-vertices, the central dots represent centroid-vertices, and the side dots represent face-vertices. The shaded blocks represent ray tracing.

[0019] A problem with only allowing one pass over the nodes is that it becomes impossible to make intra predictions from a neighboring node in a positive x- y- or z-direction. This can limit applications, for instance, where the position of centroid-vertices in nodes with 2 edge-vertices are predicted based on the neighboring centroid-vertex. If the 2 edge-verticesare positioned on a positive face, as is illustrated in FIG. 9 for instance, the node that shares that face should be used to predict the position of the centroid in the current node. However, this is not possible due to the raster scan order requirement: that centroid has not been encoded / decoded yet, which makes predicting from it impossible. In the example of FIG. 9, the centroid-vertex that will be used to predict the offset vector for node 902 has not been decoded yet. Thus, it would be left without any reconstructed content unless the parsing order is changed.

[0020] Accordingly, in one aspect there is provided a method for coding a current node having at least two-edge vertices on a positive face of the current node. The method comprises determining a centroid-vertex location for the current node, and determining one or more facevertex locations for the current node. The centroid-vertex location and face-vertex locations are determined after determining one or more of a centroid-vertex location or face-vertex location for a positive neighboring node of the current node. In certain aspects, embodiments move the decoding of certain centroid-vertices from a second process (e.g., second pipeline step) to a third process (e.g., third pipeline step), ahead of doing the raytracing.

[0021] In some aspects, there are provided computer programs comprising instructions that when executed by processing circuitry of an apparatus, such as an encoding or decoding apparatus, cause the apparatus to perform any of the methods disclosed herein. In one embodiment, there is provided a carrier containing the computer program, where the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium.

[0022] In another aspect there are provided apparatuses, such as encoders or decoders, that are configured to perform the methods disclosed herein. The apparatuses may include memory and processing circuitry coupled to the memory.

[0023] An advantage of embodiments disclosed herein is that they significantly improve the subjective quality of the reconstructed point cloud by reconstructing surfaces that previously were not in the reconstruction. Consequently, embodiments will show significant BD-rate gains on all sequences in the MPEG CTC (e.g., for distortion metric DI).BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate various embodiments.

[0025] FIG. 1 illustrates an example of how the position of two points can be represented by an octree.

[0026] FIGs. 2A-2B illustrate edge-vertices, a centroid, and a trisoup.

[0027] FIGs. 3A-3C illustrate a centroid-vertex positioned within a node.

[0028] FIG. 4 illustrates aspects of an offset-vector.

[0029] FIGs. 5A-5C illustrate aspects of face-vertices.

[0030] FIGs. 6A-6B illustrate an example of how to determine the orientation of a non-closed surface.

[0031] FIG. 7, FIG. 8, and FIG. 9 each illustrate an example of raster scan order.

[0032] FIG. 10 illustrates a system according to some embodiments.

[0033] FIG. 11 is a block diagram of an apparatus according to embodiments.

[0034] FIGs. 12A-12D illustrate aspects of centroid-vertex prediction according to some embodiments.

[0035] FIGs. 13-15 illustrate nodes being coded according to embodiments.

[0036] FIG. 16 is a flowchart illustrating a process according to an embodiment.

[0037] FIG. 17 is a flowchart illustrating a process according to an embodiment.DETAILED DESCRIPTION

[0038] FIG. 10 illustrates a system 1000 according to an embodiment. System 1000 includes an encoder 1002 and a decoder 1004. In some embodiments, encoder 1002 is in communication with decoder 1004 via a network 1010 (e.g., the Internet or other network). Encoder 1002 encodes source data 1001, such as input images, a video sequence, data relating to a scene or object, other data, etc., into a bitstream comprising encoded data, and may transmit the bitstream to decoder 1004 via network 1010. In some embodiments, encoder 1002 is not in communication with decoder 1004, and, in such an embodiment, rather than transmitting the bitstream to decoder 1004, the bitstream is stored in a data storage unit 1090 and decoder 1004 retrieves from the data storage unit 1090 the bitstream containing the encoded information. Data storage unit 1090 may be co-located with encoder 1002 or may be remote from encoder 1002.

[0039] In certain aspects, decoder 1004 decodes the information included in the encoded bitstream to produce data, for example, for display and / or post processing (e.g. a machine vision task, such as generating a 3D image of an object-of-interest (OOI)). Accordingly, decoder 1004 may be part of a device 1003 having a post processor 1005 (e.g., a postfilter or other image processor) and / or a display 1006. The device 1003 may be a mobile device, a set-top device, ahead-mounted display, or any other device. The post processor 1005 may perform machine vision tasks on the decoded pictures. One such machine vision task may be identifying an object in the picture and creating a 3D model of the object. The post processor 1005 may also comprise or consist of a postfilter, such as a neural network postfilter (NNPF), that that may receive reconstructed pictures from decoder 1004 and that may process the re constructed pictures using a specified postfilter. In the embodiment shown, post processor 1005 is separate from decoder 1004, but in other embodiments, post processor 1005 may be a component of decoder 1004.

[0040] FIG. 11 is a block diagram of an apparatus 1100 for implementing encoder 1002 and / or decoder 1004, according to some embodiments. When apparatus 1100 implements encoder 1002, apparatus 1100 may be referred to as an encoder apparatus, and when apparatus 1100 implements decoder 1004, apparatus 1100 may be referred to as a decoder apparatus.

[0041] As shown in FIG. 11, apparatus 1100 may comprise: processing circuitry (PC) 1102, which may include one or more processors (P) 1155 (e.g., one or more general purpose microprocessors and / or one or more other processors, such as an application specific integrated circuit (ASIC), field-programmable gate arrays (FPGAs), and the like), which processors may be co-located in a single housing or in a single data center or may be geographically distributed (i.e., encoder apparatus 1100 may be a distributed computing apparatus); at least one network interface 1148 (e.g., a physical interface or air interface) comprising a transmitter (Tx) 1145 and a receiver (Rx) 1147 for enabling apparatus 1100 to transmit data to and receive data from other nodes connected to a network 1010 (e.g., an Internet Protocol (IP) network) to which network interface 1148 is connected (physically or wirelessly) (e.g., network interface 1148 may be coupled to an antenna arrangement comprising one or more antennas for enabling encoder apparatus 1100 to wirelessly transmit / receive data); and a storage unit (a.k.a., “data storage system”) 1108, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments where PC 1102 includes a programmable processor, a computer readable storage medium (CRSM) 1142 may be provided. CRSM 1142may store a computer program (CP) 1143 comprising computer readable instructions (CRI) 1144. CRSM 1142 may be a non-transitory computer readable medium, such as, magnetic media (e.g., a hard disk), optical media, memory devices (e.g., random access memory, flash memory), and the like. In some embodiments, the CRI 1144 of computer program 1143 is configured such that when executed by PC 1102, the CRI causes encoder apparatus 1100 to perform steps described herein (e.g., steps described herein with reference to the flow charts). In other embodiments, encoder apparatus 1100 may be configured to perform steps described herein without the need for code. That is, for example, PC 1102 may consist merely of one or more ASICs. Hence, the features of the embodiments described herein may be implemented in hardware and / or software.

[0042] Referring now to FIGs. 12A-12D, embodiments can be used to predict whether a centroid-vertex is needed or not. In certain aspects, offsets are only sent in the cases where a centroid- vertex is needed. This may be applied, for instance, to a trisoup node. To predict whether a centroid-vertex is needed or not, the content of the already decoded neighboring nodes is utilized. In embodiments, this can refer to one or more (or all) of the neighbors that are positioned in a negative direction in relation to the current node.

[0043] One can assess the orientation of the surface based on the decoded vertices in the neighboring nodes. FIG. 12A shows two nodes (1202 on the left, 1204 on the right). Using FIG. 12A as an example, the edge-vertices and the centroid-vertex of the left node 1202 have already been decoded, and the edge-vertices of the right node 1204 have been decoded. Embodiments predict whether there is a need for a centroid-vertex in the right node 1204.

[0044] To be able to predict from a neighboring node, three conditions must be fulfilled according to some embodiments. First, there must be exactly 3 edge-vertices in the node that is being decoded, no more and no less. Second, 2 of the edge-vertices must be positioned on the same negative face (but not same edge), bordering a node that contains 3 or more edgevertices. Third, the centroid offset of the neighboring node should be 0, because if it is not, there is a chance that one could interfere with the generation of face vertices, which is likely to cause a drop in BD-rate. This scenario is illustrated in the example of FIG. 12A. However, other scenarios - including with other conditions - may be applied in embodiments.

[0045] FIG. 12B depicts a zoomed-in view on the two rightmost triangles in FIG. 12A. The two edge-vertices that are positioned on the face in between the nodes form, together with the centroid in the left node 1202, the triangle ABC. The normal vector n can be calculatedusing the following cross product: AB x AC. The normalized vector, i.e., the vector with the same direction as n but with the length 1, is denoted as n. The point c represents the arithmetic mean of the points BCD, which would have been the original position of the centroid of node 1204 if activated. According to embodiments, one can project the point c to the plane that is defined by the points A, Band C. The projected point is denoted as p in this example. The closest distance between c and p is denoted \pc | , and can be calculated according to n ■ Ac. One reason for this calculation is that if the distance \pc |is short, the surface being captured is likely part of a large flat segment and a centroid-vertex is probably not necessary. If the distance is long, a curved surface is likely being captured, and a centroid-vertex is probably needed. Therefore, a threshold value th is introduced in embodiments. If |pc| < th, a centroid-vertex is not necessary, and encoding or decoding of a centroid-vertex can be skipped or otherwise omitted.

[0046] According to embodiments, alternatives may be implemented. One alternative, and as illustrated with respect to FIG. 12C, is to project the point p to the normal vector of the plane that is defined by the points B, Band D (denoted N). Instead of comparing the threshold th to the distance |pc|, one can compare it to the distance |cnewc|. As another example, and in certain cases, one is able to predict from two or more (e.g., three) directions. That is, if 2 edge-vertices are positioned on more than 2 negative faces (e.g., between node 1204 and 1202, as well as between node 1204 and 1206). This is illustrated in FIG. 12D. In this case, 2 \pc | calculations can be performed, and both can be used to predict the existence of a centroid node. In that case, the arithmetic mean of the up to three calculated distances is used and compared to the threshold. If there is more than one prediction, the threshold can be shrunk since the credibility of the prediction becomes higher.

[0047] According to embodiments, the threshold value is determined based on the quantization step of the centroid drift. This means that the centroid position can only be shifted along the offset vector with a certain number of discrete steps. For example, the threshold value is set to one half of the centroid quantization step if there is one prediction, and one fourth of the centroid quantization step if there is more than one prediction. However, other thresholds can be used according to embodiments.

[0048] In addition to just deactivating the centroid, the predicted position cnewcan be used as a new default position for the centroid. One reason for this is that the predicted position may be closer to the real position than the arithmetic mean of the edge-vertices, allowing morezeros to be coded, which is cheaper since it is the most common offset and therefore has the shortest codeword. Embodiments can provide for increased efficiency and reduced signaling overhead.

[0049] Referring now to FIGs. 13-15, according to embodiments, an additional task is added in the third pipeline step or in similar processes. So instead of just doing the raytracing in the third pipeline step, the decoding of the centroid- and face-vertices, which are associated with nodes that just contain 2 edge-vertices, are moved from the second pipeline step to the third pipeline step. Aspects of embodiments are described below.

[0050] In the first pipeline step, the edge-vertices are encoded / decoded first. If only 2 edge-vertices are present within the node, and these 2 edge-vertices are positioned on a negative face of node as shown in FIG. 13, nothing is changed in the second pipeline step. These nodes have already been encoded / decoded and hence, these nodes can be used to predict from during the first pass over the nodes. In FIG. 13, the node currently being decoded is depicted with dotted lines. The negative faces relative to this current node are highlighted and labeled 1352a, 1352b, and 1352c. In certain aspects, a negative face may include a face positioned in the direction of the node with smaller coordinates in the defined coordinate system.

[0051] However, if the 2 edge-vertices are detected on a positive face, as shown in FIG.14 (positive faces 1462a, 1462b, 1462c), these nodes will initially be skipped in the second pipeline step. Or rather, the edge-vertices will be encoded / decoded, but the centroid- and facevertices will be initially skipped. Instead, the encoding / decoding of the centroids associated with nodes that have 2 edge-vertices positioned on positive face will be added to the third pipeline step, e.g., ahead of doing the raytracing, as depicted in FIG. 15. The positive face may include, for instance, a face that is positioned in the direction of the node with larger coordinates.

[0052] In embodiments, the pipeline steps are modified as follows:1. Edge-vertices of slice 1 are encoded / decoded.2. Edge-vertices of slice 2 are encoded / decoded, centroid- and face-vertices of slice 1 are encoded / decoded for all nodes but those containing just 2 edge vertices on a positive face.3. Edge-vertices of slice 3 are encoded / decoded, centroid- and face-vertices of slice 2 are encoded / decoded for all nodes but those containing just 2 edge vertices on a positiveface, centroid- and face-vertices of slice 1 containing just 2 edge vertices on a positive are encoded / decoded before these nodes are ray-traced.4. Edge-vertices of slice 4 are encode / decoded, centroid- and face-vertices of slice 3 are encoded / decoded for all nodes but those containing just 2 edge vertices on a positive face, centroid- and face-vertices of slice 2 containing just 2 edge vertices on a positive are encoded / decoded before these nodes are ray-traced.5. (repeated on next slices)

[0053] Referring now to FIG. 16, a process 1600 is provided according to some embodiments. The process may be performed, for instance, by an encoder 1002 or decoder 1004 as described with respect to FIG. 10 and FIG. 11. In certain aspects, embodiments use the geometrical properties of neighboring nodes to predict whether a centroid-vertex is necessary or not. In addition, in some embodiments, this information is used to change the default position of the centroid-vertices to provide additional BD-rate gains

[0054] The process may begin with step si 602 in which a distance between an initial centroid point and a projected centroid point is determined. The locations of the centroid point and projected centroid point, as well as the distance, may be determined, for instance, as described with respect in FIGs. 12A-12D. In step S1604, it is determined whether a condition is satisfied based on the distance between the two points. In embodiments, this includes comparing the distance to a threshold value. In step si 606, selective coding of a centroidvertex for the current node is performed based at least in part on whether the condition is satisfied. In embodiments, this includes performing selective coding of the centroid-vertex based at least in part on comparing the distance to a threshold. For example, and according to some embodiments, the centroid-vertex is coded when the distance is less than (or less than or equal to) the threshold value, and not coded when the distance is greater than or equal to (or just greater than) the threshold value. However, other conditions may be used to control the selective coding based on this distance according to embodiments, including other conditions where the distance indicates that the surface of the current node may be curved. The threshold may be set, for example, according to a quantization step. In step si 608, which may be optional in some embodiments, the projected centroid point is set as a default position for a centroid of the current node.

[0055] According to embodiments, the process 1600 is applied to a current node (i) having exactly three edge-vertices; (ii) where the neighboring node (e.g., a negative neighbor)has three or more edge-vertices; (iii) where two of the edge-vertices of the current node are position on the same negative face, but not the same edge; and / or (iv) where the centroid offset of the neighboring node is zero (or sufficiently small, e.g., below a threshold). Thus, the process 1600 may further include determining one or more of features (i)-(iv), and that any of steps sl602-sl608 are based at least in part on features (i)-(iv). However, in embodiments, other features / conditions may be set for process 1600 in addition to, or in place of, features (i)-(iv).

[0056] Referring now to FIG. 17, a process 1700 is provided according to some embodiments. The process may be performed, for instance, by an encoder 1002 or decoder 1004 as described with respect to FIG. 10 and FIG. 11. In certain aspects, embodiments move the coding of certain centroid-vertices from the second pipeline step to the third pipeline step, ahead of doing the raytracing. This may include, for example, applying the modified pipeline steps discussed above. In embodiments, the process 1700 is applied where the current node has at least two-edge vertices on a positive face. As such, the process may include determining that the current node has at least two-edge vertices positioned on the positive face, and in some instance, an order of coding steps for the node, or decisions regarding initially skipping one or more steps for a given node, is based at least in part on the determination.

[0057] The process may begin with step si 702 in which a centroid-vertex location for the current node is determined. In step si 704, one or more face-vertex locations for the current node are determined. In embodiments, the centroid-vertex location and face-vertex locations are determined for the current node after determining one or more of a centroid- vertex location or face-vertex location for a positive neighboring node of the current node. In steps si 706, which may be optional in some embodiments, ray tracing for the current node is performed.

[0058] In certain aspects, the node may be part of a “slice.” For instance, the current node can be part of a first slice and the positive neighboring node part of a second slice. In this respect, the current node and neighboring mode may be in adjacent columns of nodes. According to embodiments, the process 1700 can apply the ordering of steps as set forth in FIG. 15, and can be applied in pipeline processes across one or more slices.

[0059] Summary of Various Embodiments

[0060] Al . A method for coding a current node having three edge- vertices, the method comprising: determining a distance between an initial centroid point and a projected centroid point; determining if a condition is satisfied based on the distance; and performing selectivecoding of a centroid-vertex for the current node based at least in part on whether the condition is satisfied.

[0061] A2. The method of Al, wherein determining if a condition is satisfied comprises comparing the distance to a threshold value, and / or wherein performing selective coding of the centroid- vertex is based at least in part on comparing the distance to a threshold.

[0062] A3. The method of A2, wherein performing selective coding of the centroidvertex comprises: coding the centroid-vertex when the distance is less than the threshold value.

[0063] A4. The method of A2 or A3, wherein performing selective coding of the centroid-vertex comprises: not coding the centroid- vertex when the distance is greater than or equal to the threshold value.

[0064] A5. The method of any of A1-A4, wherein:(i) coding comprises encoding or decoding image or video data;(ii) coding comprises one or more Geometry based Point Cloud Compression (G- PCC) operations;(iii) the selective coding is coding of data comprising 3D point cloud data;(iv) the current node is a trisoup node; and / or(v) one or more vertices of the current node represent at least one surface within the node.

[0065] A6. The method of any of A1-A5, wherein a location of initial centroid point represents the arithmetic mean of the locations of the three edge-vertices of the current node, or wherein the initial centroid point is a default centroid point for the current node.

[0066] A7. The method of any of A1-A6, wherein the projected centroid point is a projection of the initial centroid point.

[0067] A8. The method of A7, wherein the projection is a projection to a plane defined by two edge-vertices of the current node and a centroid-vertex of a neighboring node.

[0068] A9. The method of A8, wherein the neighboring node is a negative neighboring node, and the two edge-vertices are positioned on a face between the neighboring node and the current node.

[0069] A10. The method of A8 or A9, wherein the two edge-vertices are not positioned on the same edge.

[0070] Al 1. The method of any of A1-A10, wherein the distance is a closest distance between the two points and is determined based on a vector n or n, where:n = AB x AC,where B and C are edge-vertices of the current node, A is a centroid-vertex of a neighboring node, n is a normalized vector of n.

[0071] A12. The method of any of A1-A7, wherein the projection is a projection to a plane defined by the three edge-vertices of the current node.

[0072] Al 3. The method of Al 2, wherein the distance is a closest distance between the two points and is determined based on a vector N or N, where:N = ~DB x DC,where B, C, and D are the three edge-vectors of the current node, and N is a normalized vector of N.

[0073] A14. The method of any of A1-A7, wherein the distance is an arithmetic mean of a distance between the initial centroid point and a first projected centroid point and a distance between the initial centroid point and a second projected centroid point, wherein the first projected centroid point is determined based at least in part on a first neighboring node, and wherein the second projected centroid point is determined based at least in part on a second neighboring node.

[0074] Al 5. The method of any of A1-A7, wherein the distance is an arithmetic mean of a distance between: (i) the initial centroid point and a first projected centroid point, (ii) a distance between the initial centroid point and a second projected centroid point, and (iii) a distance between the initial centroid point and a third projected centroid point, wherein the first projected centroid point is determined based at least in part on a first neighboring node, wherein the second projected centroid point is determined based at least in part on a second neighboring node, and wherein the third projected centroid point is determined based at least in part on a third neighboring node.

[0075] A16. The method of any of A1-A15, wherein a negative neighboring node of the current node comprises three or more edge-vertices.

[0076] A17. The method of any of A1-A16, wherein a centroid offset of a negative neighboring node of the current node is zero.

[0077] A18. The method of any of A1-A16, wherein a centroid offset of a negative neighboring node of the current node is a non-zero value.

[0078] Al 9. The method of any of Al -Al 8, further comprising: setting the projected centroid point as a default position for a centroid of the current node.

[0079] A20. The method of any of A2-A19, wherein the threshold value is based at least in part on a quantization step of a centroid drift.

[0080] A21. The method of any of A2-A20, wherein the threshold value is 1 / 2 or 1 / 4 a quantization step.

[0081] Bl. An apparatus for coding a current node having three edge-vertices, the apparatus configured to: determine a distance between an initial centroid point and a projected centroid point; determine if a condition is satisfied based on the distance; and perform selective coding of a centroid-vertex for the current node based at least in part on whether the condition is satisfied.

[0082] B2. The apparatus of Bl, further configured to perform any of the steps of A2- A21.

[0083] B3. The apparatus of Bl or B2, wherein the apparatus is an encoder apparatus or decoder apparatus.

[0084] Cl. A method for coding a current node having at least two-edge vertices on a positive face of the current node, comprising: determining a centroid-vertex location for the current node; and determining one or more face-vertex locations for the current node, wherein the centroid-vertex location and face-vertex locations are determined after determining one or more of a centroid-vertex location or face-vertex location for a positive neighboring node of the current node.

[0085] C2. The method of Cl, further comprising: performing ray-tracing for the current node.

[0086] C3. The method of Cl or C2, wherein the current node is part of a first slice and the positive neighboring node is part of a second slice.

[0087] C4. The method of C3, wherein the first and second slice comprise adjacent columns of nodes.

[0088] C5. The method of any of C1-C4, wherein:(i) coding comprises encoding or decoding image or video data;(ii) coding comprises one or more Geometry based Point Cloud Compression (G- PCC) operations;(iii) the current node comprises 3D point cloud data;(iv) the current node is a trisoup node; and / or(v) one or more vertices of the current node represent at least one surface within the node.

[0089] C6. The method of any of C1-C5, wherein one or more edge-vertices of the current node are coded as part of a first process.

[0090] C7. The method of any of C1-C6, wherein the centroid-vertex location and face-vertex locations of the current node are coded as part of a third process.

[0091] C8. The method of any of C1-C7, wherein at least a portion of a second process is skipped or omitted for the current node.

[0092] C9. The method of C8, wherein the skipping or omission is based at least in part on a determination that the current node comprises at least two edge-vertices on a positive face of the current node.

[0093] CIO. The method of any of C7-C9, wherein one or more of the first, second, and third process is a pipeline process.

[0094] Cl 1. The method of any of Cl -CIO, wherein the centroid-vertex location of the current node is determined based at least in part on the centroid-vertex of the positive neighboring node.

[0095] Cl 2. The method of any of Cl -Cl 1, further comprising: determining that the current node has at least two-edge vertices positioned on the positive face, wherein an order of coding steps for the node is based at least in part on the determination.

[0096] C13. The method of any of C1-C12, further comprising: coding an offset vector for the current node.

[0097] DI. An apparatus for coding a current node having at least two-edge vertices on a positive face of the current node, configured to: determine a centroid- vertex location for the current node; and determine one or more face-vertex locations for the current node, wherein the centroid-vertex location and face-vertex locations are determined after determining one ormore of a centroid-vertex location or face-vertex location for a positive neighboring node of the current node.

[0098] D2. The apparatus of DI, further configured to perform any of the steps of C2- C13.

[0099] D3. The apparatus of DI or D2, wherein the apparatus is an encoder apparatus or decoder apparatus.

[0100] El. A computer program comprising instructions which when executed by processing circuitry of an apparatus causes the apparatus to perform the method of any one of the above embodiments.

[0101] E2. A carrier containing the computer program of El, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium.

[0102] While the terminology in this disclosure is described in terms of the current codec and standards, the embodiments of this disclosure also apply to any existing or future codec, which may use a different, but equivalent terminology.

[0103] While various embodiments are described herein, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of this disclosure should not be limited by any of the above-described exemplary embodiments. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the disclosure unless otherwise indicated herein or otherwise clearly contradicted by context.

[0104] Additionally, while the processes described above and illustrated in the drawings are shown as a sequence of steps, this was done solely for the sake of illustration. Accordingly, it is contemplated that some steps may be added, some steps may be omitted, the order of the steps may be re-arranged, and some steps may be performed in parallel.

Claims

CLAIMS1. A method (1700) for coding a current node having at least two-edge vertices on a positive face of the current node, comprising:determining (si 702) a centroid-vertex location for the current node; and determining (si 704) one or more face-vertex locations for the current node, wherein the centroid-vertex location and face-vertex locations are determined after determining one or more of a centroid-vertex location or face-vertex location for a positive neighboring node of the current node.

2. The method of claim 1, further comprising:performing ray-tracing (si 706) for the current node.

3. The method of claim 1 or claim 2, wherein:the current node is part of a first slice and the positive neighboring node is part of a second slice.

4. The method of claim 3, wherein the first and second slice comprise adjacent columns of nodes.

5. The method of any of claims 1-4, wherein:(i) coding comprises encoding or decoding image or video data;(ii) coding comprises one or more Geometry based Point Cloud Compression (G- PCC) operations;(iii) the current node comprises 3D point cloud data;(iv) the current node is a trisoup node; and / or(v) one or more vertices of the current node represent at least one surface within the node.

6. The method of any of claims 1-5, wherein one or more edge-vertices of the current node are coded as part of a first process.

7. The method of any of claims 1-6, wherein the centroid-vertex location and face-vertex locations of the current node are coded as part of a third process.

8. The method of any of claims 1-7, wherein at least a portion of a second process is skipped or omitted for the current node.

9. The method of claim 8, wherein the skipping or omission is based at least in part on a determination that the current node comprises at least two edge-vertices on a positive face of the current node.

10. The method of any of claims 6-9, wherein one or more of the first, second, and third process is a pipeline process.

11. The method of any of claims 1-10, wherein the centroid-vertex location of the current node is determined based at least in part on the centroid-vertex of the positive neighboring node.

12. The method of any of claims 1-11, further comprising:determining that the current node has at least two-edge vertices positioned on the positive face, wherein an order of coding steps for the node is based at least in part on the determination.

13. The method of any of claims 1-12, further comprising:coding an offset vector for the current node.

14. An apparatus (1100) for coding a current node having at least two-edge vertices on a positive face of the current node, configured to:determine a centroid- vertex location for the current node; anddetermine one or more face-vertex locations for the current node,wherein the centroid-vertex location and face-vertex locations are determined after determining one or more of a centroid-vertex location or face-vertex location for a positive neighboring node of the current node.

15. The apparatus of claim 14, further configured to perform any of the steps of claims 2-13.

16. The apparatus of claim 14 or 15, wherein the apparatus is an encoder apparatus or decoder apparatus.

17. A method (1600) for coding a current node having three edge-vertices, the method comprising:determining (si 602) a distance between an initial centroid point and a projected centroid point;determining (si 604) if a condition is satisfied based on the distance; and performing (si 606) selective coding of a centroid-vertex for the current node based at least in part on whether the condition is satisfied.

18. The method of claim 17, wherein determining if a condition is satisfied comprises comparing the distance to a threshold value, and / or wherein performing selective coding of the centroid-vertex is based at least in part on comparing the distance to a threshold.

19. The method of 18, wherein performing selective coding of the centroid-vertex comprises: coding the centroid-vertex when the distance is less than the threshold value.

20. The method of claim 18 or 19, wherein performing selective coding of the centroidvertex comprises:not coding the centroid-vertex when the distance is greater than or equal to the threshold value.

21. The method of any of claims 17-20, wherein:(i) coding comprises encoding or decoding image or video data;(ii) coding comprises one or more Geometry based Point Cloud Compression (G- PCC) operations;(iii) the selective coding is coding of data comprising 3D point cloud data;(iv) the current node is a trisoup node; and / or(v) one or more vertices of the current node represent at least one surface within the node.

22. The method of any of claims 17-21, wherein a location of initial centroid point represents the arithmetic mean of the locations of the three edge-vertices of the current node, or wherein the initial centroid point is a default centroid point for the current node.

23. The method of any of claims 17-22, wherein the projected centroid point is a projection of the initial centroid point.

24. The method of claim 23, wherein the projection is a projection to a plane defined by two edge-vertices of the current node and a centroid-vertex of a neighboring node.

25. The method of claim 24, wherein the neighboring node is a negative neighboring node, and the two edge-vertices are positioned on a face between the neighboring node and the current node.

26. The method of claim 24 or 25, wherein the two edge-vertices are not positioned on the same edge.

27. The method of any of claims 17-26, wherein the distance is a closest distance between the two points and is determined based on a vector n or n, where:n = AB x AC,where B and C are edge-vertices of the current node, A is a centroid-vertex of a neighboring node, n is a normalized vector of n.

28. The method of any of claims 17-23, wherein the projection is a projection to a plane defined by the three edge-vertices of the current node.

29. The method of claims 28, wherein the distance is a closest distance between the two points and is determined based on a vector N or N, where:N = ~DB x DC,where B, C, and D are the three edge-vectors of the current node, and N is a normalized vector of N.

30. The method of any of claims 17-23, wherein the distance is an arithmetic mean of a distance between the initial centroid point and a first projected centroid point and a distance between the initial centroid point and a second projected centroid point,wherein the first projected centroid point is determined based at least in part on a first neighboring node, andwherein the second projected centroid point is determined based at least in part on a second neighboring node.

31. The method of any of claims 17-23, wherein the distance is an arithmetic mean of a distance between: (i) the initial centroid point and a first projected centroid point, (ii) a distance between the initial centroid point and a second projected centroid point, and (iii) a distance between the initial centroid point and a third projected centroid point, wherein the first projected centroid point is determined based at least in part on a first neighboring node,wherein the second projected centroid point is determined based at least in part on a second neighboring node, andwherein the third projected centroid point is determined based at least in part on a third neighboring node.

32. The method of any of claims 17-31, wherein a negative neighboring node of the current node comprises three or more edge-vertices.

33. The method of any of claims 17-32, wherein a centroid offset of a negative neighboring node of the current node is zero.

34. The method of any of claims 17-32, wherein a centroid offset of a negative neighboring node of the current node is a non-zero value.

35. The method of any of claims 17-24, further comprising:setting (si 608) the projected centroid point as a default position for a centroid of the current node.

36. The method of any of 18-35, wherein the threshold value is based at least in part on a quantization step of a centroid drift.

37. The method of any of claims 18-36, wherein the threshold value is 1 / 2 or 1 / 4 a quantization step.

38. An apparatus (1100) for coding a current node having three edge-vertices, the apparatus configured to:determine a distance between an initial centroid point and a projected centroid point; determine if a condition is satisfied based on the distance; andperform selective coding of a centroid-vertex for the current node based at least in part on whether the condition is satisfied.

39. The apparatus of claim 38, further configured to perform any of the steps of claims 18-37.

40. The apparatus of claim 38 or 39, wherein the apparatus is an encoder apparatus or decoder apparatus.

41. A computer program comprising instructions which when executed by processing circuitry of an apparatus causes the apparatus to perform the method of any one of the above embodiments.

42. A carrier containing the computer program of claim 41, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium.

Images