Method, encoder and decoder for encoding and decoding 3d point clouds

Abstract

A method for decoding a geometry of a 3D point cloud from a bitstream, preferably implemented in a decoder, comprising: receiving the bitstream, where the bitstream comprises vertex information comprising information about positions of vertices on edges of a cube of leaf nodes of an octree structure of the point cloud; obtaining the vertex information to determine the position of a vertex, at least one vertex being adjusted by an average position towards the surface points; and determining a point of the point cloud based on at least one adjusted vertex, and adjusting the average position of the at least one vertex towards the surface point according to the vertex information of at least one adjacent leaf node.

Description

Technical Field

[0001] The present invention relates to a method, an encoder and a decoder for encoding and decoding a 3D point cloud. Background Art

[0002] Point clouds have recently gained attention as a format for 3D data representation due to their versatility in representing all types of 3D objects or scenes. However, for all compression schemes, the reconstruction quality of the points of the point cloud is crucial. Summary of the invention

[0003] Therefore, an object of the present invention is to provide a method for decoding the geometry of a 3D point cloud from a bitstream and encoding the 3D point cloud into a bitstream, which method improves efficiency.

[0004] This problem is solved by a decoding method according to claim 1 , an encoding method according to claim 2 , an encoder according to claim 19 , a decoder according to claim 20 , a bit stream according to claim 21 and software according to claim 22 .

[0005] In a first aspect, a method for decoding the geometry of a 3D point cloud from a bitstream is provided, the method preferably being implemented in a decoder. The method comprises: receiving a bitstream, wherein the bitstream comprises vertex information, the vertex information comprising information about the positions of vertices on the edges of a cube of a leaf node of an octree structure of a point cloud; obtaining the vertex information to determine the positions of the vertices, wherein at least one vertex is adjusted toward an average position of a surface point; determining a point of the point cloud based on at least one adjusted vertex, and adjusting at least one vertex toward the average position of a surface point according to the vertex information of at least one adjacent leaf node.

[0006] Preferably, vertex information is obtained to determine the position of the vertex, wherein at least one vertex is adjusted toward the average position of the surface point, including: determining a virtual position based on the position of the vertices of a cube associated with a leaf node of an octree structure; constructing a triangle based on the vertices and the virtual position of a cube; determining a normal vector of the cube based on the constructed triangle; determining a center of mass position based on the normal vector; and adjusting at least one vertex toward the center of mass position.

[0007] Therefore, in a first step, a bitstream is received, and the bitstream may contain information about an octree structure of a volume of a point cloud that can be decoded. Preferably, the geometric structure of the point cloud is GPCC encoded. Therefore, preferably, by decoding from the bitstream, octree information about the volume of the point cloud can be provided. In addition, the bitstream includes vertex information, which includes the positions of vertices on the edges of a cube associated with a leaf node in the octree structure. Preferably, information about the existence of vertices is also included. Therefore, the vertex information can be provided by receiving and / or decoding a bitstream. The obtained vertex information can be used to determine the position of the vertex before further adjustment. Wherein, the bitstream is preferably encoded by a TriSoup encoding scheme at the encoder.

[0008] After decoding the vertex information and preferably also decoding the octree information from the bitstream described in the previous step, in the next step for reconstructing the point cloud geometry, triangles can be determined for each cube. In particular, a virtual position can first be determined based on the vertex positions of a cube associated with the leaf nodes of the octree structure, preferably by averaging the positions of the vertices of a cube. Then, based on the vertices and virtual positions of a cube, a set of triangles is constructed, preferably by connecting two consecutive vertices on the edge of the cube and the determined virtual positions clockwise. Therefore, the surface of the first set of triangles is determined by the vertex positions included in the bitstream and the determined virtual positions (i.e., the average positions of the vertices).

[0009] Subsequently, the normal vector of the cube can be determined based on the constructed triangles. Among them, the normal vector is a vector perpendicular to a given object. Therefore, the normal vector of each triangle in the first group of triangles can be determined separately. Then, the normal vector of the cube can be determined based on the sum of the determined normal vectors of the first group of triangles. Then, the center of mass position can be determined based on the virtual position and the normal vector of the cube. Through this adjustment, the determined center of mass position may be closer to the original position of the point cloud in the leaf node. In addition, according to the present invention, at least one vertex can be adjusted toward the average position of the surface point. Surface points are points that constitute the outer surface of the 3D point cloud, specifically located on the edge, and the edge converges at the vertex in the original point cloud. The original vertex information is established using the surface points derived from the original point cloud. The surface points are a subset of the 3D point cloud. In particular, the vertex can be related to the target surface, that is, different vertices can correspond to different surfaces. Surface points, such as Fig.14 1405 may be a point corresponding to the target surface. In particular, Fig.14, vertex 1401 may correspond to surface point 1405. Wherein the corresponding surface point is an original point in the point cloud that is close to the edge to which at least one vertex belongs. Preferably, this refers to a subset of the original points in the point cloud that are located within a distance of less than 1 / 4 of the leaf node width from its edge. More preferably, less than 1 / 8 (of the leaf node width). Preferably, the at least one vertex can be adjusted towards the determined centroid position to obtain a better modeling of the original point cloud surface. In particular, when moving along the vector The vertices on the edge of the axis with the maximum value can be adjusted along the edge they belong to to make the reconstructed surface more natural. From C mean To C. More specifically, we can first determine an axis, the vector Along this axis, it has a maximum value. Then, the vertices on the edge of the leaf node parallel to the determined axis can be found. According to the present invention, the above adjustment can be further based on the vertex information of at least one adjacent leaf node, for example, the information can be a centroid residual vector (drift). Among them, adjacent leaf nodes in the octree structure refer to leaf nodes that share a common boundary, in particular, leaf nodes that share an edge with a given leaf node. The "adjacent leaf nodes" associated with a given "leaf node" are defined as leaf nodes that share at least one common boundary with the given leaf node. Therefore, in the three-dimensional space configuration defined by the octree structure, the "adjacent leaf nodes" are adjacent to the given leaf node. Finally, these vertices can be based on the current leaf node's and the adjacent leaf nodes Then, according to the adjusted vertices and centroids, triangles are constructed in a similar way as above.

[0010] In order to reconstruct the points of the point cloud from the reconstructed triangles, voxelization can be performed, preferably by a ray tracing process, wherein in the ray tracing process, rays are emitted along three directions parallel to any of the three axes. Their origin is an integer coordinate point corresponding to the sampling precision required for rendering. The intersection of the ray with one of the reconstructed triangles (if any) is then determined and added to the list of rendered points, i.e., to the points of the point cloud. In the voxelization process, the surface of the reconstructed triangle is sampled with the rays to determine the points of the point cloud.

[0011] Preferably, the at least one vertex is adjusted only when the difference between the virtual position and the centroid position exceeds a threshold, preferably the threshold is based on the leaf node size, preferably a ratio s of the leaf node size, wherein Preferably, S=1 / 128. Therefore, before executing the method according to the present invention, it is first determined whether the prominence of the modeled surface in the leaf node is large enough. For example, by comparing the centroid residual with a threshold. This constraint reduces unnecessary adjustments, thereby improving computational efficiency and reducing complexity.

[0012] Preferably, at least one vertex is adjusted according to a fixed value, preferably, the fixed value is determined based on the type of point cloud and is encoded into or from the bitstream. In particular, the fixed value can be determined based on statistics of an input point cloud sequence that indicate properties, characteristics or features of a sequence of 3D point clouds (e.g., ordinary or non-ordinary). The statistics of the input point cloud sequence can include various metrics and attributes, such as: Descriptive statistics: These include metrics such as mean, median, mode, variance and standard deviation of point coordinates (x, y, z), as well as other attributes, such as color, intensity or normal vectors. Geometric properties: Features such as point density, spatial distribution or point cloud range can be quantified and summarized. Temporal properties: For a point cloud sequence that changes over time, properties such as motion or deformation of the point cloud can be analyzed and summarized. Topological properties: The connectivity or relationships between points in the point cloud can be studied to provide insights into the underlying structure of the data. This approach provides simplicity and consistency between similar point cloud types.

[0013] Preferably, at least one vertex is adjusted based on the difference between the virtual position and the centroid position, preferably further based on the boundary of the edge of the leaf node. As mentioned above, the vertex can be adjusted along certain determined edges towards the centroid position. In addition, the adjustment can be further based on the boundary of the edge of the leaf node, so that it can be ensured that the adjusted vertex does not exceed the leaf node to which it belongs. Therefore, this technique can perform more accurate vertex adjustment, thereby more accurately presenting the point cloud geometry.

[0014] Preferably, the at least one vertex is adjusted toward the average position of the surface point based on at least one adjacent leaf node, including: determining the eligibility for adjusting the vertex based on the vertex information of the current leaf node and the vertex information of the adjacent leaf nodes; if eligible, adjusting the vertex; therefore, the method involves adjusting the vertex position according to the vertex information from the adjacent leaf nodes; this is in addition to the original position of the vertex, as described in the previously described embodiment. The key to this preferred feature is the introduction of a "qualification" condition that controls whether the vertex should be adjusted. The eligibility condition is determined by considering the vertex information of the current leaf node and the vertex information of the adjacent leaf nodes. Therefore, by considering the vertex information of the adjacent leaf nodes, the method can more accurately reflect the original spatial relationship and geometric structure of the objects in the 3D point cloud. This makes the encoded and subsequently decoded 3D point cloud more representative of the original spatial arrangement and geometric characteristics of the object or scene.

[0015] Preferably, based on the vertex information of the current leaf node and the vertex information of the adjacent leaf nodes, determining the eligibility of adjusting the vertex includes: determining a first vector pointing from the virtual position of the current leaf node to the center of mass position of the current leaf node; determining a second vector pointing from the virtual position of the adjacent leaf node to the center of mass position of the adjacent leaf node; if the inner product of the first vector and the second vector is a positive number, then determining that at least one vertex is eligible for adjustment. This preferred feature introduces a vertex adjustment eligibility standard based on vector mathematics. Two vectors are established, one from the virtual position of the current leaf node to the center of mass of the current leaf node, and the other from the virtual position of the adjacent leaf node to the center of mass of the adjacent leaf node. The eligibility of adjusting the vertex depends on whether the inner product of the two vectors is a positive number. The technical effect of this feature is that it can provide a quantitative, geometrically reasonable method for determining whether a vertex is eligible for adjustment. The use of the inner product effectively measures the alignment of the two vectors. If the inner product is positive, it means that the directions pointed by the two vectors are substantially the same, which means that the adjacent leaf nodes have similar geometric features, so it is meaningful and appropriate to adjust the vertex.

[0016] Preferably, if eligible, adjusting the vertex includes: adjusting the vertex based on a weighted sum of the modulus value of the first vector and the modulus value of the second vector, preferably, the weight of the first vector is greater than the weight of the second vector. According to the above preferred features, once the vertex is deemed eligible for adjustment, the present invention details the manner of adjustment. It points out that the adjustment should be based on the weighted sum of the modulus values ​​of the two vectors from the embodiment of claim 7. In addition, it is suggested that the weight of the first vector (from the current leaf node) should be higher than the weight of the second vector (from the adjacent leaf node). The main technical effect here is that it provides subtle control over the vertex adjustment process. Weighting provides a method for prioritizing the geometric features of the current leaf node relative to the geometric features of the adjacent nodes when adjusting the vertex. This makes the vertex adjustment more respectful of local geometric features, thereby more accurately representing 3D objects or scenes in the encoded and decoded point cloud data.

[0017] In another aspect of the present invention, a method for encoding a 3D point cloud into a bitstream is provided, which method is preferably implemented in an encoder. The method for encoding a 3D point cloud comprises: obtaining vertex information of each cube associated with a leaf node from a point cloud surface, wherein the vertex information comprises position information of vertices on the edges of the cube of the leaf node of an octree structure of the point cloud; encoding the vertex information into a bitstream, wherein at least one vertex is adjusted toward an average position of a surface point; determining a point of the point cloud based on the encoded vertex information;

[0018] At least one vertex is adjusted toward an original position of the vertex based on vertex information of at least one adjacent leaf node.

[0019] Thus, by means of the encoding method, vertex information and preferably octree information is generated. This information can be encoded into a bitstream. Subsequently, on the encoder side, a reconstruction step can be performed. In this reconstruction step, the point cloud geometry is reconstructed, wherein the reconstruction step is identical to the step in the above-described decoding method. The attributes of the points of the point cloud (color, reflectivity, ...) are then encoded on the encoder side using the reconstructed geometry of the point cloud, for example, by RAHT (Regional Adaptive Hierarchical Transform), a prediction transform or a lifting transform.

[0020] Preferably, the geometric structure of the point cloud is encoded into the bitstream via Geometry-based Point Cloud Compression (G-PCC).

[0021] Preferably, the bitstream is an MPEG G-PCC compliant bitstream.

[0022] Preferably, the encoding method is further constructed according to the features described above in conjunction with the decoding method.

[0023] In another aspect of the present invention, an encoder for encoding a 3D point cloud into a bit stream is provided. The encoder comprises a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned encoding method are performed.

[0024] In another aspect of the present invention, a decoder for decoding a 3D point cloud from a bitstream is provided. The decoder comprises a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the above-mentioned decoding method are performed.

[0025] In another aspect of the present invention, a bit stream is provided, wherein the bit stream is encoded by the steps of the aforementioned encoding method.

[0026] In another aspect of the present invention, a computer-readable storage medium is provided, which includes instructions for executing the steps of the above method for encoding a 3D point cloud into a bitstream.

[0027] In another aspect of the present invention, a computer-readable storage medium is provided, comprising instructions for executing the steps of the above method for decoding a 3D point cloud from a bitstream. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Hereinafter, the present invention is described in more detail with reference to the accompanying drawings.

[0029] The accompanying drawings show:

[0030] Figure 1 A flowchart showing a method for decoding 3D point cloud geometry according to the present invention is shown.

[0031] Figure 2 An example of generating an octree structure is shown,

[0032] Figure 3 Show according to Figure 2 The octree of

[0033] Figure 4 An example showing the determination of the vertices on the edge of a cube is shown,

[0034] Figure 5 An example of generating a triangle is shown,

[0035] Figure 6 An example showing vertices on the edge of a cube,

[0036] Figure 7 Shows the triangles generated by vertices,

[0037] Figure 8 Show according to Figure 7 Example of determining triangle order,

[0038] Fig. 9 Schematic diagram showing the voxelization step,

[0039] Fig.10 shows an example of reconstructing a triangle using the centroid point C as the pivot point,

[0040] Fig.11 An example showing a normal vector,

[0041] Fig.12 shows an example of a 1D residual along the normal vector,

[0042] Fig.13 shows an example of a smooth convex surface in a leaf node with modeling triangles built from vertices and centroid points,

[0043] Fig.14 shows an example of a refined surface,

[0044] Fig.15 An example showing a visible hole created on a common face of two adjacent leaf nodes,

[0045] Fig.16 An example of a vertex on the surface F that does not satisfy one of the conditions of the present invention is shown,

[0046] Fig.17 An example of a vertex on a surface F that satisfies one of the conditions of the present invention is shown,

[0047] Fig.18 shows an example of thinning vertices on a face F between two adjacent leaf nodes A and B,

[0048] Fig. 19 shows a decoder or an encoder. DETAILED DESCRIPTION

[0049] Point clouds have recently gained attention as a format for 3D data representation due to their versatility in representing all types of 3D objects or scenes.

[0050] Therefore, many use cases can be addressed with point clouds, including

[0051] Film post-production,

[0052] Real-time 3D immersive telepresence or virtual reality (VR) / augmented reality (AR) applications,

[0053] Free viewpoint video (e.g. for watching sports),

[0054] Geographic Information Systems (also known as Cartography),

[0055] Cultural heritage (scans of rare objects stored in digital form),

[0056] Autonomous driving, including 3D mapping of the environment and real-time Lidar data collection

[0057] A point cloud is a set of points located in 3D space, with the option to attach additional values ​​to each point. These additional values ​​are often called point attributes. Therefore, a point cloud is a combination of geometry (the 3D position of each point) and attributes.

[0058] The attributes may be, for example, a three-component color, a material property such as reflectivity, and / or a two-component normal vector of the surface associated with the point.

[0059] Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, Lidar, scanners, or can be generated by computers (for example, in movie post-production). Depending on the use case, for mapping applications, point clouds can have thousands to billions of points.

[0060] The raw representation of point clouds requires a very high number of bits per point, at least twelve bits per spatial component X, Y or Z, and optionally more bits for attributes, e.g., three times 10 bits for color. Practical deployment of point cloud based applications requires compression techniques that can store and distribute point clouds over reasonable storage and transmission infrastructure.

[0061] For distribution to and visualization by end users, for example on AR / VR glasses or any other 3D-enabled device, compression can be lossy (like in video compression). Other use cases do require lossless compression, for example, medical applications or autonomous driving, to avoid altering the decision results obtained from the analysis of the compressed and transmitted point cloud.

[0062] Until recently, point cloud compression (aka PCC) had not been addressed by the mass market and no standardized point cloud codec was available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Moving Picture Experts Group or MPEG, started a work item on point cloud compression. This resulted in two standards, namely

[0063] MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based Point Cloud Compression (V-PCC)

[0064] MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-based Point Cloud Compression (G-PCC)

[0065] Both the V-PCC and G-PCC standards completed their first versions at the end of 2020.

[0066] The V-PCC encoding method compresses point clouds by performing multiple projections of 3D objects to obtain 2D patches that are packed into images (or videos when processing mobile point clouds). The acquired images or videos are then compressed using existing image / video codecs, allowing the use of already deployed image and video solutions. By its nature, V-PCC is only effective on dense and continuous point clouds, because image / video codecs cannot compress non-smooth patches, such as those obtained from the projection of sparse geometric data collected by Lidar.

[0067] The G-PCC coding method has two schemes for geometry compression. The first scheme is based on an occupancy tree (octree / quadtree / binary tree) representation of the point cloud geometry. Occupied nodes are split until a certain size is reached, and the occupied leaf nodes provide the location of the points, usually at the center of these nodes. A high level of compression can be obtained for dense point clouds by using neighbor-based prediction techniques. Sparse point clouds are also solved by directly encoding the locations of points within nodes with non-minimum sizes, by stopping tree construction when only isolated points exist in the node; this technique is called Direct Coding Mode (DCM).

[0068] The second scheme is based on prediction trees, where each node represents the 3D position of a point and the relationship between nodes is the spatial prediction from parent to child. This approach can only handle sparse point clouds and has the advantages of lower latency and simpler decoding compared to occupancy trees. However, the compression performance is only slightly better than the first occupancy-based approach, and the encoding is complex, with intensive searching for the best predictor (in a long list of potential predictors) when building the prediction tree.

[0069] In both schemes, the attribute (de)coding is performed after the geometry (de)coding is completed, resulting in a two-pass encoding. Thus, low latency is obtained by using slices that decompose the 3D space into independently coded sub-volumes without prediction between sub-volumes. When many slices are used, the compression performance may be severely affected.

[0070] An important use case is the transmission of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves over time. Moreover, AR / VR point clouds are usually locally 2D, as they represent the surface of objects most of the time. Therefore, AR / VR point clouds are highly connected (or dense), as points are rarely isolated but have many neighbors.

[0071] A dense (or solid) point cloud represents a continuous surface at a resolution such that the volumes (small cubes called voxels) associated with the points touch each other without showing any visible holes in the surface.

[0072] Such point clouds are typically used in AR / VR environments and viewed by end users through devices such as TVs, smartphones or headphones. They are either transmitted to the device or stored locally. Many AR / VR applications use dynamic point clouds, which change over time, rather than static point clouds. Therefore, the amount of data is huge and must be compressed. Today, lossless compression based on an octree representation of the geometry of the point cloud can achieve slightly less than one bit per point (1bpp). This may not be enough for real-time transmission, which may involve several million points per frame and frame rates of up to 50 frames per second (fps), resulting in hundreds of megabits of data per second.

[0073] Therefore, lossy compression can be used, with the usual requirements of maintaining acceptable visual quality, while compressing sufficiently to fit within the bandwidth provided by the transmission channel, while maintaining real-time transmission of frames. In many applications, bit rates as low as 0.1bpp (10 times higher compression than lossless coding compression) have made real-time transmission possible.

[0074] Codecs based on MPEG-I Part 5 (ISO / IEC 23090-5) or Video-based Point Cloud Compression (V-PCC) VPCC can achieve such low bitrates by using lossy compression with a video codec that compresses 2D frames obtained from the projection of a point cloud onto a plane. The geometry is represented by a sequence of projected patches assembled into a frame, each patch being a small local depth map. However, VPCC is not general and is limited to narrow types of point clouds that do not exhibit locally complex geometry (e.g., trees, hair), since the obtained projected depth maps are not smooth enough to be effectively compressed by video codecs.

[0075] Pure 3D compression techniques can handle any type of point cloud. It is still an open question whether 3D compression techniques can compete with VPCC (or any projection + image coding scheme) on dense point clouds. Standardization is still moving towards providing an extension (revision) of GPCC that provides competitive lossy compression that can compress dense point clouds as well as VPCC intra-frame coding, while maintaining the versatility of GPCC to handle any type of point cloud (dense point cloud, Lidar, 3D map). This extension will probably use the so-called TriSoup scheme, which works with octrees and will be described in detail in subsequent sections. TriSoup is being explored by the ISO / IEC standardization working group JTC1 / SC29 / WG7.

[0076] refer to Figure 1 , which shows a schematic diagram of a method for decoding geometric information of a 3D point cloud from a bitstream.

[0077] A method for decoding the geometry of a 3D point cloud from a bitstream, preferably implemented in a decoder, comprises the following steps:

[0078] S11: receiving a bit stream, wherein the bit stream includes vertex information;

[0079] S12: Acquire vertex information to determine the position of the vertex, wherein at least one vertex is adjusted toward the average position of the surface points;

[0080] S13: Determine a point of the point cloud based on at least one adjusted vertex, and adjust the average position of at least one vertex toward the surface point according to the vertex information of at least one adjacent leaf node.

[0081] To determine the octree information, the first step of the geometry encoding process is to construct and encode the octree, such as Figure 2 and Figure 3 As shown. The bounding box is the body 100 that contains all the points and is associated with a root node 112 (i.e., a single node at the top of the tree 110). The body 100 is first divided into eight sub-volumes 102 called octants (or octants), each represented by a node 114 in the tree 110. Octants 104 are then recursively split out of the sub-volumes 106 until a target level is reached, where the octant 106 is occupied by at least one point. Figure 2 and Figure 3 The target level is the level at which the volume is no longer segmented, and it can be determined based on the compression ratio.

[0082] Each octant (or node) is represented by an occupation byte containing one bit for each sub-octant, where the corresponding bit is set to 1 if the sub-octant is occupied by at least one node, and is set to zero otherwise. The occupation bytes 118 of all octants, e.g., "100010000001000 11000011", are serialized (in breadth-first order) and entropy encoded using a binary arithmetic encoder.

[0083] Figure 4 A block representation of a 3D surface 210 is shown, along with an example of a block 220 in a trisoup. The surface 210 intersects the block 220, which is therefore an occupied block, and the block 220 exists between multiple blocks 200 in the 3D space. Within the block 220, the closed portion of the surface 210 intersects the edge of the block at the six illustrated vertices 232 of the polygon 230, and if the edge of the block 220 contains the vertex, then the edge is said to be selected.

[0084] Figure 5 Block 220 in Trisoup is shown, omitting surface 210 for clarity, and showing unselected edge 270, selected edge 260, and i-th edge 250. Assume that i-th edge 250 is selected. To specify vertex v on edge i i , specify a scalar value to indicate the corresponding fraction of the length of edge 250.

[0085] like Figure 4 and Figure 5 As shown, within each octant 220 in the target level of the octree, Trisoup represents the original surface 210 as a set of triangles 245. The surface is encoded and used to obtain the position of the reconstructed (or decoded) points. First, the intersection of the surface represented by the original point with the edges of the octant is estimated by averaging the positions of the points closest to those edges in the octant. Secondly, the twelve edges of all octants and their associated intersections (if any) are stored as line segments and vertices respectively. Each (unique) line segment is then encoded as follows. The first single bit is arithmetically encoded, and the bit is set to 1 if the line segment is occupied by a vertex, otherwise it is set to 0. If it is occupied, the relative position of the vertex on the line segment is also arithmetically encoded.

[0086] The vertices 310 of the triangles are encoded along the edges 320 of the volume associated with the leaf nodes 300 of the tree, as Figure 6 As shown. These vertices 310 on the edge 320 are shared between the leaf nodes 300 with the common edge 320. This means that each edge belonging to at least one leaf node encodes at most one vertex. In this way, the continuity of the model is ensured through the leaf nodes.

[0087] As described above, encoding of TriSoup vertices may require two pieces of information per edge: a vertex flag indicating whether a TriSoup vertex is present on the edge, and, if so, the vertex position along the edge.

[0088] Therefore, the encoded data includes Octree data as well as TriSoup data.

[0089] The vertex flags are encoded by an adaptive binary arithmetic encoder that uses a specific context to encode the vertex flags. The positions of vertices on an edge of length n=2^s can be encoded with unit precision by pushing s bits into (bypassing / not entropy encoding) the bitstream.

[0090] Within a leaf node, if there are at least three vertices 310 on an edge 320 of the leaf node 300, a triangle is constructed from the TriSoup vertices. Figure 7 Reconstructed triangles 330 and 340 are depicted in FIG.

[0091] Obviously, other combinations of triangles 330 and 340 are possible. The selection of triangles results from a three-step process

[0092] 1. Determine the dominant direction along one of the three axes

[0093] 2. Sort TriSoup vertices according to dominant direction

[0094] 3. Construct a triangle based on an ordered list of vertices

[0095] Knowledge of the exact position of the triangle in the current leaf node is not required and can be inferred from the vertices.

[0096] Figure 8 will be used to explain this process. Each of the three axes is tested, and the one that maximizes the total surface of the triangle is chosen as the principal axis. For simplicity of the diagram, Figure 8 Only tests on two axes are described.

[0097] The first test (top) along the vertical axis is performed by vertically projecting the cube and TriSoup vertices 310 on the 2D plane. The vertices 310 are then sorted in clockwise order relative to the center of the projection node (square). Then, based on the ordered vertices, triangles 330, 340 are constructed according to a fixed rule. Here, when 4 vertices are involved, triangle 123 as well as triangle 134 are systematically constructed. When there are 3 vertices, the only possible triangle is 123. When there are 5 vertices, the fixed rule can be to construct triangles 123, 134 and 451. And so on, up to a maximum of 12 vertices.

[0098] The second test (left side) along the horizontal vertical axis is performed by projecting the cube and trisoup vertices horizontally onto the 2D plane.

[0099] The vertical projection shows the maximum total 2D surface of the triangle, therefore, the principal axis is chosen as the vertical axis, and the constructed TriSoup triangles are obtained in the order of vertical projection, such as Figure 8 As shown, it is located inside the node. It should be noted that taking the horizontal axis as the main axis will result in another construction of the triangle.

[0100] The principal axes are fully selected by maximizing the projection surface, thus achieving continuous reconstruction of point clouds without holes.

[0101] Rendering of TriSoup triangles into points can be performed by ray tracing. The set of all points rendered by ray tracing will form the decoded point cloud.

[0102] for Fig. 9 For the ray tracing shown, rays can be emitted along three directions parallel to the axis. Their origin is a point with integer (voxelized) coordinates of a precision corresponding to the sampling precision required for rendering. The intersection with one of the trisoup triangles (dashed point if any) is then voxelized (= rounded to the nearest point of the required sampling precision) and added to the list of rendered points.

[0103] After applying Trisoup to all leaf nodes, i.e., building triangles and obtaining points preferably by ray tracing, discard all copies of the same point in the list of rendered points (i.e., keep only one voxel among all voxels sharing the same position and volume) to obtain a set of decoded (unique) points.

[0104] Based on the above concept, the Trisoup encoding can be improved. For example, by calculating the centroid point, the coordinates of which are the average coordinates of all (ordered) vertices Vi, see Fig.10 , where the centroid C is depicted using a checkerboard fill. The centroid is used as the pivot point. A triangle consists of ordered vertices (V1, V2, …, V M ) is constructed by pivoting around the centroid point C, constructing the following M triangles: V1V2C, V2V3C, ..., V M-1 V M C, V M V1C.

[0105] The vertices are sorted clockwise, and it does not matter which vertex is chosen as V1.

[0106] This structure preserves the natural symmetry of the model without arbitrarily privileging triangles. In addition, it provides an additional degree of freedom to improve the accuracy of the model, namely, the location of the centroid point C.

[0107] Therefore, the position of the centroid point C can be further improved by encoding the residual position in the bitstream so that the position of the centroid point C is closer to the original point of the point cloud.

[0108] For example, C = C mean +C res .

[0109] Among them, C mean is the average position obtained by averaging the coordinates of all (ordered) vertices, and C res is the coding residual.

[0110] The coded residual can be a 3D residual. However, it is observed that a 3D residual is rarely advantageous because it requires many bits to be coded and such many bits are not fully compensated by the better accuracy of the model. Therefore, it is preferred to encode the 1D residual C res to encode.

[0111] For example, the normal vector can be constructed, such as Fig.11 As shown, the residual can be determined by the following formula:

[0112]

[0113] where α is a 1D signed scalar value encoded in the bitstream, see Fig.12 . Normal vector You can export it in two steps:

[0114] 1.

[0115] 2. Then normalize

[0116] where × is the cross product between two vectors (also called the vector (cross) product), and the side yes:

[0117]

[0118] It is understandable that the vector Can be parallel to the axis to simplify its determination and the calculation of the value α. can be parallel to the principal axis as a good approximation to the vector calculated above.

[0119] The value α is determined by the encoder, encoded into the bitstream, and obtained by the decoder by decoding the bitstream. The value α may be binarized, and each bit may be encoded using a binary entropy encoder such as an arithmetic encoder or a context adaptive binary encoder such as CABAC.

[0120] The value α can be binarized as

[0121] A flag f0 indicating whether α is equal to 0

[0122] a sign indicating whether α>0 or α<0,

[0123] A flag f1 indicating whether |α| is equal to 1,

[0124] The remainder |α|-2 encoded by the expGolomb encoder.

[0125] The value α can be obtained by the encoder by considering all points P in the point cloud belonging to the current leaf node K To determine. For each point P K , which is consistent with the straight line The distance d K From the following formula we get

[0126]

[0127] And if this distance d K Below a predefined threshold th (e.g., th=2), point P is used K Calculate the value α. Point P K Relative to the average point C mean The 1D residual r K Obtained by the scalar product (also called inner product or dot product)

[0128]

[0129] Therefore, the value α is obtained by

[0130]

[0131] Where S is point P K A set of such that their distance d K is below a threshold th, and |S| is the number of points belonging to this set.

[0132] In the current trisoup encoding, as described in conjunction with the previous figures, the centroid point C and the vertices of the leaf nodes (V0, ..., V i ) can be used to construct triangles, and then ray tracing methods can be applied to the constructed triangles to obtain the reconstructed points.

[0133] Point cloud data captured in natural scenes often have smooth surfaces, even in concave or convex areas. However, in the current trisoup encoding, concave or convex areas are modeled with triangles in leaf nodes, and the generated centroid residual C resmay be large (for example, C res can be about half the width of a leaf node), then use the triangles constructed within the leaf node (by using the centroid point C and the vertices (V0, ..., V i The modeled surface will have large protrusions and the generated surface will be far from naturally smooth and will have artifact effects. Fig.13 As shown, point 1301 represents the vertex along the edge in the leaf node, and point 1303 represents the centroid point C in the leaf node, and small point 1305 represents the point on the original surface (for clarity, only one of these small points is marked with label 1305). It can be seen that the modeled surface with sharp protrusions cannot model the original surface well enough, and the reconstructed surface may not maintain the smooth characteristics of the original surface, which may make the reconstructed point cloud have poor visual quality, especially when the leaf node size is relatively large, at a lower compression rate. Therefore, the compression efficiency of trisoup encoding in the current MPEG G-PCC is not optimal.

[0134] Therefore, it is preferred to improve the compression efficiency of the point cloud by reducing the non-smoothness of the reconstructed surface at the decoder side.

[0135] The triangle modeling in the leaf node can be refined to make the surface reconstructed on the decoder side closer to the original surface. Specifically, for example, after obtaining the decoded vertex and the decoded centroid point C on the decoder side, the vector in the leaf node of the convex region or the concave region can be oriented. The position of the vertex is adjusted in the direction of , so that this method can reduce the non-smoothness of the reconstructed point cloud and will not increase the bitstream size.

[0136] In some embodiments, the method according to the present invention follows the following steps:

[0137] On the decoder side, after decoding the vertices of each leaf node from the bitstream, each leaf node may be iterated to construct triangles for obtaining the reconstructed point cloud, preferably by a ray tracing method.

[0138] In detail, for each leaf node, e.g.

[0139] o If the current (i.e., processed) leaf node has more than 3 vertices,

[0140] First, the centroid point C can be determined to construct a modeling surface, which is composed of vertices and triangles constructed by the centroid point C. Specifically, the vertices (V0, ..., V i ) of the average point C mean , and the unit vector of the centroid residual The modulus value C of the centroid residual can then be decoded from the bitstream res, and finally can be Get the center of mass point C, where From C mean The vector to C can also be named

[0141] Then, the prominence of the modeled surface in the leaf node can be determined. In a preferred embodiment, the centroid residual C res Whether it is large relative to the size of the leaf nodes.

[0142] If the centroid residual C res Larger than the leaf nodes (more likely, based on the current vertex (V0, ..., V i ) and the centroid point C will have sharp protrusion artifacts in the generated modeled surface), then for all vertices V (V1, ..., V i ) are refined along vectors in all three axes. The vertices on the edge of the axis with the maximum value, the refinement can be along the edge to which these vertices belong, within the boundary of the edge in the leaf node, in a direction that can make the reconstructed surface more natural. (Detailed vertex refinement will be described later)

[0143] Otherwise, all vertices in the leaf nodes (V0, ..., V i ) may not be refined.

[0144] Then, all vertices in the leaf node (including the refined vertex V' and other non-refined vertices V) are combined with the centroid point C to construct triangles, as described above with reference to the accompanying drawings, and a ray tracing method can be applied to each triangle to obtain the reconstructed points in the leaf node.

[0145] Otherwise, if the leaf node has 3 vertices, there is no average point C in this processed leaf node mean and centroid point C, and only 1 triangle V1V2V3 can be constructed, and a ray tracing method can be applied on this triangle to obtain the reconstructed point in the leaf node.

[0146] o Otherwise, if a leaf node has less than 3 vertices, there will be no triangles built for the leaf node, and thus no reconstruction points will be generated for the leaf node.

[0147] On the encoder side, after encoding the vertices of each leaf node into the bitstream, each leaf node can be iterated to encode the information of the centroid position C, and for subsequent attribute encoding, it may also be necessary to obtain the reconstructed geometric information of the point cloud by constructing triangles based on the vertices and the centroid position C. And in order to obtain the refined modeling surface in the leaf node, the leaf node can be iteratively processed according to the same steps as described above in the decoder side. Of course, in addition, information indicating whether the point cloud is a plane is also encoded at the encoder.

[0148] In some embodiments, to determine the centroid residual C res If the size of the leaf nodes is large, the centroid residual C res is compared with a threshold value Th, which is based on the size N of the leaf node. The threshold value Th may be a ratio s of the size of the leaf node, Th=N*s, and the ratio s may be 0<s≤1 / 8, for example, And in actual implementation, in order to improve the accuracy of division / multiplication operations, C res Can be multiplied by 64 (by res This is achieved by applying a 6-bit left shift operation, so the comparison threshold Th needs to be multiplied by 64, becoming N / 2). And for example, for a leaf node, if C res >Th, then the vertices in the leaf nodes can be refined according to the above method; otherwise, the proposed surface refinement method will not be applied to the leaf nodes.

[0149] As mentioned above, if the centroid residual Cx es Relative to the size of the leaf node, all vertices in the leaf node (V0, ..., V i ) in the vectors along the three axes Vertices on the edge of the axis with the maximum value are refined along the edges to which these vertices belong, within the boundaries of the edges in the leaf nodes, and in a direction that can make the reconstructed surface more natural. In a preferred embodiment, detailed vertex refinement can follow the following steps:

[0150] First, determine the vector The axis along which the maximum value is axis_max;

[0151] Then, we can find the vertices on the edges parallel to axis axis_max in the leaf nodes (V0, ..., V i ),

[0152] Then, based on the centroid residual vector of the leaf node Change the axis_max dimension of these vertices and obtain the vertex V i ' can be described as Where, the offset is relative to V i The offset distance, is a unit vector pointing in the offset direction. In a preferred embodiment, Can be based on and the boundaries of the edges, where represents the centroid residual vector The projection vector along the axis_max direction, the edge boundary is the constraint to keep the vertex from exceeding the leaf node to which it belongs. And in such an embodiment, Can be followed by same direction, and the offset can be based on The modulus value and the edge boundary of the leaf node are obtained. In a preferred embodiment, the offset can be 1 / 3 of the modulus value. Fig.14 FIG. 14 shows a refined surface generated according to the present invention, where point 1401 on the edge is a vertex V used to model the surface according to the trisoup method, the arrow line parallel to the X axis represents the projection vector of the centroid residual, and point 1401a (only one of these points is labeled for clarity) is a point 1401a along the The refined vertex V′ is obtained by moving the vertex V in the direction, and the small point 1405 represents the point on the original surface (for clarity, only one of these points is marked with a number). It can be seen that the refined surface is closer to the original surface in the leaf node and the sharp protrusion artifacts are reduced.

[0153] As described above, the vertices on the edges of the leaf nodes can be refined one by one in order, and the refined vertices in the leaf nodes can be used to reconstruct the triangles to obtain the reconstructed point cloud. However, this refinement method is only based on the centroid offset information of the current leaf node, so the vertices on the edges belonging to two adjacent leaf nodes can be along the direction in the following example Offset by different distances:

[0154] Example (a): The centroid offset direction in two adjacent leaf nodes It's the opposite.

[0155] Or (b): Even if the centroids of two adjacent leaf nodes are offset in direction Likewise, their offset values ​​vary greatly.

[0156] Therefore, the above example will result in some visible holes on the surface of the reconstructed point cloud, such as Fig.15 As shown, squares 1501 represent adjacent faces of two adjacent leaf nodes, and points 1503 with dashed outlines on the edges represent unrefined vertices. and represents the vertex offset vector (including direction and value), and the dotted area represents the hole caused by the vertex offset. Fig.15 In the example, the holes are caused by the vertex offset vector and The directions in two adjacent leaf nodes are opposite, which corresponds to example (a); Fig.15 Although the vertex offset vector and , but their values ​​are very different, which can also cause holes, corresponding to example (b) above.

[0157] These visible holes may cause a reduction in visual quality as well as compression efficiency. Therefore, the problem to be solved by the present invention is to further improve the coding efficiency, preferably by eliminating the holes generated by the vertex thinning method.

[0158] In some embodiments, the centroid offset information of adjacent leaf nodes (encoded / processed) can be used to guide vertex refinement of leaf nodes, which can solve the visible hole problem. Specifically, each leaf node can be traversed, and for each leaf node, if the centroid residual C res If the size of the leaf node is large, it will be located parallel to The vertex V on the edge of the direction j , which is the same as the above refinement method. j After that, you can perform the following steps,

[0159] First, determine the vertex V in the current leaf node j Whether it needs to be refined can be based on the centroid offset information of its neighboring leaf nodes (encoded / processed).

[0160] Then, if the vertex V in the current leaf node j Need to be refined, then through the direction Move each vertex by an offset distance, where the offset is based on the centroid offset of the adjacent leaf nodes. The modulus value is determined, and the original vertex V j Will be updated to the refined vertex V′ j ; Otherwise, the vertex V in the current leaf node j will not be updated.

[0161] When obtaining the vertex of the leaf node (including the refined vertex V′ j and non-refined vertices), triangles can be constructed using these vertices and the determined centroid points, and then a ray tracing method is preferably used on these triangles in the leaf nodes to construct a point cloud.

[0162] The advantages of the proposed method include: the "hole problem" can be solved by eliminating the difference in vertex offsets between two or more adjacent leaf nodes. Using the proposed method, the refined vertices in leaf node A (refined by s1) and the refined vertices in the adjacent leaf nodes of A (refined by s2) can be made Fig.15 The same point on the middle edge.

[0163] According to the present invention, the eligibility for vertex thinning refers to the vertex V in the current leaf node. j In a preferred embodiment,

[0164] First, according to the conditions determined by the centroid offsets of the adjacent leaf nodes and the current leaf node, a face F of the current leaf node is selected, and the face F may contain vertices that need to be refined, wherein the conditions may include:

[0165] 1) Face F is parallel to the current leaf node direction.

[0166] 2) Find an adjacent leaf node that has been encoded / processed and shares face F with the current leaf node, and whose centroid offset is not 0.

[0167] 3) Centroid offset vector in adjacent leaf nodes The offset vector from the center of mass in the current leaf node The inner product between is greater than 0. For example, Fig.16 The example shown does not satisfy this condition, but Fig.17 The example shown in satisfies this condition. (In another embodiment, the components of the centroid offset vectors in the adjacent leaf nodes and the current leaf node are It's the opposite.)

[0168] Then, determine whether the vertices on the edge of face F satisfy a condition, that is, face F along The vertex V on the edge of the direction j The number of is greater than 1.

[0169] Preferably, the refinement eligibility of the vertices in the current leaf node may be true only if the vertices in the current leaf node satisfy the above conditions.

[0170] In some embodiments, if the vertex refinement eligibility in the current leaf node is true, the vertex offset offset can be determined, and the offset value can be determined based on the centroid offset modulus value in the encoded / processed adjacent leaf node, because in the current trisoup encoding, the centroid offset information is calculated sequentially leaf node by leaf node, and when processing the current leaf node, only the centroid offset information of the encoded / processed leaf node can be known and retrieved.

[0171] In a preferred embodiment, the vertex V on the face F j The offset of can be determined by the centroid offset modulus values ​​of adjacent leaf nodes sharing the face F. For example, the offset can be the weighted average of the centroid offset modulus values ​​of two adjacent leaf nodes A and B, which can be described as

[0172]

[0173] Wherein, k and b can both be set to 1 / 2. Preferably, k>b.

[0174] At the same time, the vertex V on face F j The offset can also be determined based on the edge boundaries of the leaf nodes. Similar to the vertex thinning method above, the edge boundaries are constraints to ensure that the vertex does not exceed the leaf node to which it belongs.

[0175] In some embodiments, the vertex refinement method may need to follow a principle that vertex refinement is parallel to the direction All points on the edge of and belonging to the same face of two adjacent leaf nodes must be refined by moving them in the same direction by the same offset distance.

[0176] In some embodiments, after obtaining vertex refinement qualifications and vertex offsets of vertices on face F in a leaf node, the vertex offsets are calculated along the vector (Its unit vector is ) direction, move each vertex by the offset distance, and move each vertex V j Thinned to V′ j , then, refine the vertex V′ j It can be expressed as follows:

[0177]

[0178] like Fig.18 As shown, leaf node A (left) is a processed leaf node, leaf node B is the current leaf node, its vertices need to be refined, two vertices V3 and V4 meet the refinement qualifications, and the arrow line from V4 to V′4 represents the offset vector And V3 and V4 are thinned into V'3 and V'4 according to equation (2). Then, the thinned vertices V'3 and V'4 are used together with the centroid point C in the current leaf node to reconstruct the triangle.

[0179] In some embodiments, the method steps for triangle modeling refinement in the proposed point cloud data trisoup encoding method are as follows:

[0180] On the decoder side, after decoding the vertices of each leaf node from the bitstream, each leaf node is iterated to construct triangles, thereby obtaining the reconstructed point cloud through a ray tracing method.

[0181] In detail, for each leaf node,

[0182] o If the leaf node being processed has more than 3 vertices,

[0183] First, the centroid point C can be determined to construct a modeling surface consisting of triangles constructed from vertices and the centroid point C. Specifically, the vertices in the leaf nodes (V0, ..., V i ) of the average point C mean and the unit vector of the centroid residual Then, the modulus value C of the centroid residual is decoded from the bitstream res , and finally can be Get the center of mass point C, where From C mean The vector to C can also be named

[0184] Then, the prominence of the modeled surface in the leaf node is determined. In a preferred embodiment, the centroid residual C res Whether it is large relative to the leaf node size.

[0185] If the centroid residual C res Larger than leaf nodes (depending on the current vertex (V0, ..., V i ) and the surface constructed by the centroid point C is more likely to have sharp protrusion artifacts in the generated modeled surface), then all the vertices V (V1, ..., V i ) to find the The vertex V on the edge of the direction j Then, for each vertex V j , use the above method proposed according to the present invention to determine its refinement qualifications.

[0186] If the vertex refinement eligibility in the current leaf node is true, the vertex offset offset is determined by the above method, and then each vertex V is transformed using the above method. j Thinned to V′ j .

[0187] Otherwise, all vertices in the leaf nodes (V0, ..., V i ) will not be refined.

[0188] Then, triangles are constructed using all vertices in the leaf nodes (including the refined vertices V' and other non-refined vertices V using the proposed method) combined with the centroid point C, and a ray tracing method can be applied to each triangle to obtain the reconstructed points in the leaf nodes.

[0189] Otherwise, if the leaf node has 3 vertices, there is no average point C in this processed leaf node mean and centroid point C, and only 1 triangle V1V2V3 can be constructed, and a ray tracing method is applied on this triangle to obtain the reconstructed point in the leaf node.

[0190] o Otherwise, if a leaf node has less than 3 vertices, no triangles are built for the leaf node, and thus no reconstruction points are generated for the leaf node.

[0191] On the encoder side, after encoding the vertices of each leaf node into the bitstream, iterate each leaf node to encode the information of the centroid position C, and for subsequent attribute encoding, it may also be necessary to obtain the reconstructed geometric information of the point cloud by constructing triangles based on the vertices and the centroid position C. And in order to obtain the refined modeling surface in the leaf node, the leaf node can be iteratively processed according to the same steps as described above in the decoder side.

[0192] Reference is now made to FIG. 19 , which shows a simplified block diagram of an example embodiment of an encoder or decoder 300 . The encoder or decoder 300 includes a processor 301 and a memory storage device 303 . The memory storage device 303 may store a computer program or application containing instructions that, when executed, cause the processor 301 to perform operations such as those described herein. For example, the instructions may encode and output an encoded bitstream or decode a bitstream and output points of a point cloud according to the methods described herein. It should be understood that the instructions may be stored on a non-transitory computer-readable medium, such as an optical disk, a flash memory device, a random access memory, a hard drive, and the like. When the instructions are executed, the processor 301 performs the operations and functions specified in the instructions so as to operate as a dedicated processor that implements the process. In some examples, such a processor may be referred to as a “processor circuit” or “processor circuitry”.

[0193] It is understood that decoders and / or encoders according to the present application can be implemented in many computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. A decoder or encoder can be implemented by software containing instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transient computer-readable memory, including CD, RAM, ROM, flash memory, etc.

[0194] It should be understood that standard computer programming techniques and languages ​​can be used to implement the decoder and / or encoder described herein and to implement the module, routine, process, thread or other software components of the method / process for configuring the encoder or decoder. The application is not limited to specific processors, computer languages, computer programming conventions, data structures, other such implementation details. Those skilled in the art will recognize that the described process can be implemented as a part of a computer executable code stored in a volatile or non-volatile memory, a part of a special integrated chip (ASIC), etc.

[0195] Certain adjustments and modifications may be made to the described embodiments. Therefore, the embodiments discussed above are considered to be illustrative rather than restrictive. In particular, the embodiments may be freely combined with each other.

Claims

1. A method for decoding the geometry of a 3D point cloud from a bitstream, preferably implemented in a decoder, the method comprising: receiving a bitstream, wherein the bitstream includes vertex information; acquiring the vertex information to determine positions of vertices, wherein at least one vertex is adjusted toward an average position of surface points; determining a point of the point cloud based on at least one adjusted vertex, The at least one vertex is adjusted toward the average position of the surface points according to the vertex information of at least one adjacent leaf node.

2. A method for encoding a 3D point cloud into a bitstream, preferably implemented in an encoder, comprising: Get the vertex information of each cube associated with the leaf node from the point cloud surface, encoding the vertex information into the bitstream, wherein at least one vertex is adjusted toward an average position of surface points; Determining points of the point cloud based on the encoded vertex information; The at least one vertex is adjusted toward the average position of the surface points according to the vertex information of at least one adjacent leaf node.

3. The method according to any one of claims 1 or 2, wherein: The obtaining of the vertex information to determine the positions of the vertices, wherein at least one vertex is adjusted toward the average position of the surface points, comprises: determining a centroid position, wherein the centroid position is configured as an approximation of the average position of surface points; At least one vertex is adjusted toward the centroid position.

4. The method according to claim 3, wherein: Determining the centroid position includes: Determining a virtual position according to a position of a vertex of a cube associated with a leaf node of the octree structure; The center of mass position is determined based on the virtual position.

5. The method according to claim 4, wherein: Determining the center of mass position based on the virtual position comprises: constructing a triangle according to the vertices of a cube and the virtual position; The centroid position is determined based on the constructed triangle.

6. The method according to claim 5, wherein: Determining the centroid position according to the constructed triangle comprises: Determining a normal vector of the cube based on the constructed triangle; The center of mass position is determined based on the normal vector.

7. The method according to any one of claims 4 to 6, wherein: At least one of the vertices is adjusted only when the difference between the virtual position and the centroid position exceeds a threshold, preferably, the threshold is based on the size of the leaf node.

8. The method according to claim 7, wherein: The threshold is based on a ratio s of the sizes of the leaf nodes, where 9. The method according to claim 8, wherein: s＝1 / 128。 10. The method according to any one of claims 1 to 9, wherein: At least one of the vertices is adjusted according to a fixed value.

11. The method according to claim 10, wherein: The fixed value is determined based on the type of the point cloud and is encoded into or from the bitstream.

12. The method according to any one of claims 4 to 11, wherein: At least one of the vertices is adjusted based on a difference between the virtual position and the center of mass position.

13. The method according to claim 12, wherein: At least one of the vertices is further adjusted based on a boundary of an edge of the leaf node.

14. The method according to any one of claims 1 to 5, wherein: The adjusting the average position of the at least one vertex toward the surface point according to the vertex information of at least one adjacent leaf node comprises: The adjustment eligibility of the vertex is determined according to the vertex information of the current leaf node and the vertex information of the adjacent leaf nodes; if the vertex is eligible for adjustment, the vertex is adjusted.

15. The method according to any one of claims 4 to 14, wherein: The step of determining the adjustment qualification of the vertex according to the vertex information of the current leaf node and the vertex information of the adjacent leaf node includes: Determine the vectors of the current leaf node and each of the adjacent leaf nodes; The adjustment eligibility of the vertex is determined according to the determined vector.

16. The method according to claim 15, wherein: Determining the adjustment qualification of the vertex according to the determined vector includes: Determine a first vector pointing from the virtual position of the current leaf node to the center of mass position of the current leaf node; determine a second vector pointing from the virtual position of the adjacent leaf node to the center of mass position of the adjacent leaf node; if the inner product of the first vector and the second vector is a positive number, determine that at least one of the vertices is eligible for adjustment.

17. The method according to claim 16, wherein: If the vertex is eligible for adjustment, adjusting the vertex includes: adjusting the vertex based on a weighted sum of the modulus value of the first vector and the modulus value of the second vector.

18. The method according to claim 17, wherein: The weight of the first vector is greater than the weight of the second vector.

19. An encoder for encoding a 3D point cloud into a bitstream, the encoder comprising at least one processor and a memory, wherein: The processor stores instructions which, when executed by the processor, perform the steps of the method according to claim 2 and any one of claims 3 to 18 dependent on claim 2.

20. A decoder for decoding a 3D point cloud from a bitstream, the decoder comprising at least one processor and a memory, wherein: The memory stores instructions which, when executed by the processor, perform the steps of the method according to claim 1 and any one of claims 3 to 18 dependent on claim 1.

21. A bit stream encoded by a method according to claim 2 and any one of claims 3 to 18 when dependent on claim 2.

22. A computer-readable storage medium comprising instructions which, when executed by a processor, perform the steps of the method according to any one of claims 1 to 18.

Images