Methods, encoders, and decoders for encoding and decoding 3D point clouds

Abstract

A method for decoding the geometry of a 3D point cloud from a bitstream, preferably implemented in a decoder, includes: receiving a bitstream, wherein the bitstream includes vertex information, the vertex information including information about the position of vertices on the edges of a cube of leaf nodes of an octree structure of the point cloud; acquiring the vertex information to determine the position of a vertex, wherein at least one vertex is adjusted toward an average position toward surface points; determining points of the point cloud based on at least one adjusted vertex, and adjusting the average position of at least one vertex toward surface points according to the vertex information of at least one adjacent leaf node.

Description

Technical Field

[0001] This invention relates to a method, encoder, and decoder for encoding and decoding 3D point clouds. Background Technology

[0002] Point clouds have recently gained attention as a format for representing 3D data due to their versatility in representing all types of 3D objects or scenes. However, for all compression schemes, the quality of point cloud reconstruction is crucial. Summary of the Invention

[0003] Therefore, the object of the present invention is to provide a method for decoding the geometry of a 3D point cloud from a bitstream and encoding a 3D point cloud into a bitstream, the method of which improves efficiency.

[0004] The problem is solved by the decoding method of claim 1, the encoding method of claim 2, the encoder of claim 19, the decoder of claim 20, the bitstream of claim 21, and the software of claim 22.

[0005] In a first aspect, a method is provided for decoding the geometry of a 3D point cloud from a bitstream, preferably implemented in a decoder. The method includes: receiving a bitstream, wherein the bitstream includes vertex information, the vertex information including information about the positions of vertices on the edges of a cube of leaf nodes of an octree structure of the point cloud; acquiring the vertex information to determine the positions of vertices, wherein at least one vertex is adjusted towards an average position toward surface points; determining points of the point cloud based on at least one adjusted vertex, and adjusting the average position of at least one vertex toward surface points according to the vertex information of at least one adjacent leaf node.

[0006] Preferably, obtaining vertex information to determine the position of a vertex, wherein at least one vertex is adjusted toward the average position of surface points, includes: determining a virtual position based on the position of a cube's vertices associated with the leaf nodes of an octree structure; constructing a triangle based on the vertices of the cube and the virtual position; determining the normal vector of the cube based on the constructed triangle; determining the centroid position based on the normal vector; and adjusting at least one vertex toward the centroid position.

[0007] Therefore, in the first step, a bitstream is received, and this bitstream may contain information about the octree structure of the volume of the decodeable point cloud. Preferably, the geometry of the point cloud is GPCC encoded. Therefore, preferably, octree information about the volume of the point cloud can be provided by decoding from the bitstream. Furthermore, the bitstream includes vertex information, which includes the positions of vertices on the edges of the cubes associated with the leaf nodes in the octree structure. Preferably, it also includes information about the presence of vertices. Therefore, vertex information can be provided by receiving and / or decoding the bitstream. The obtained vertex information can be used to determine the positions of the vertices before further adjustments. Preferably, the bitstream is encoded at the encoder using the TriSoup encoding scheme.

[0008] After decoding the vertex information and preferably also decoding the octree information from the bitstream described in the previous step, triangles can be determined for each cube in the next step for reconstructing the point cloud geometry. Specifically, virtual positions can first be determined based on the vertex positions of a cube associated with the leaf nodes of the octree structure, preferably by averaging the vertex positions of a cube. Then, based on the vertices and virtual positions of a cube, a set of triangles is constructed, preferably by connecting two consecutive vertices on the cube's edges clockwise with the determined virtual positions. Thus, the surface of the first set of triangles is determined by the vertex positions included in the bitstream and the determined virtual positions (i.e., the average position of the vertices).

[0009] Subsequently, the normal vector of the cube can be determined based on the constructed triangles. Here, the normal vector is a vector perpendicular to the given object. Therefore, the normal vector of each triangle in the first set of triangles can be determined separately. Then, the normal vector of the cube can be determined based on the sum of the determined normal vectors of the first set of triangles. Then, the centroid position can be determined based on the virtual position of the cube and the normal vector. Through this adjustment, the determined centroid position may be closer to the original position of the point cloud in the leaf node. Furthermore, according to the invention, at least one vertex can be adjusted toward the average position of the surface points. Surface points are points that constitute the outer surface of the 3D point cloud, specifically located on edges that converge at vertices within the original point cloud. Original vertex information is established using surface points derived from the original point cloud. Surface points are a subset of the 3D point cloud. In particular, vertices can be associated with a target surface; that is, different vertices can correspond to different surfaces. Surface points, for example... Figure 14 1405 could be a point corresponding to the target surface. Specifically, in Figure 14In this context, vertex 1401 can correspond to surface point 1405. The corresponding surface point is an original point in the point cloud that is close to the edge to which at least one vertex belongs. Preferably, this refers to a subset of original points in the point cloud that are located within a distance of less than 1 / 4 of the width of a leaf node from their edge. More preferably, less than 1 / 8 (the width of a leaf node). Preferably, the at least one vertex can be adjusted toward the determined centroid position to obtain a better model of the original point cloud surface. In particular, along the vector... Vertices on the edges of axes with maximum values ​​can be adjusted along the edges they belong to to make the reconstructed surface more natural. Here, vectors... From The vector to C. More specifically, we can first determine an axis, the vector... The axis has a maximum value. Then, vertices on the edges of leaf nodes parallel to the determined axis can be found. According to the invention, the above adjustments can be further based on vertex information of at least one adjacent leaf node, for example, this information could be a centroid residual vector (offset (drift)). Here, adjacent leaf nodes in the octree structure refer to leaf nodes sharing a common boundary, specifically leaf nodes sharing an edge with a given leaf node. An "adjacent leaf node" associated with a given "leaf node" is defined as a leaf node sharing at least one common boundary with the given leaf node. Therefore, in the three-dimensional spatial configuration defined by the octree structure, an "adjacent leaf node" is adjacent to the given leaf node. Finally, these vertices can be adjusted according to the current leaf node's... and adjacent leaf nodes Make adjustments. Then, based on the adjusted vertices and centroid, construct a triangle in a similar manner to the one described above.

[0010] To reconstruct points in the point cloud from the reconstructed triangles, voxelization can be performed, preferably via a ray tracing process, where rays are emitted along three directions parallel to any one of the three axes. Their origins are integer coordinate points corresponding to the sampling precision required for rendering. The intersection point of the ray with one of the reconstructed triangles (if any) is then determined and added to the list of rendered points, i.e., to the points in the point cloud. During voxelization, the surface of the reconstructed triangles is sampled using rays to determine the points in the point cloud.

[0011] Preferably, the at least one vertex is adjusted only when the difference between the virtual position and the centroid position exceeds a threshold. Preferably, the threshold is based on the leaf node size, and more preferably, it is a proportion of the leaf node size. ,in, Preferably, S = 1 / 128. Therefore, before performing the method according to the invention, it is first determined whether the convexity of the modeling surface in the leaf node is sufficiently large. For example, by comparing the centroid residual with a threshold. This constraint reduces unnecessary adjustments, thereby improving computational efficiency and reducing complexity.

[0012] Preferably, at least one vertex is adjusted based on a fixed value, which is determined based on the type of point cloud and encoded into or from a bitstream. Specifically, the fixed value can be determined based on statistics of the input point cloud sequence indicating attributes, characteristics, or features of the 3D point cloud (e.g., ordinary or non-ordinary) sequence. Statistics of the input point cloud sequence can include various measures and attributes, such as: descriptive statistics: these include measures such as the mean, median, mode, variance, and standard deviation of point coordinates (x, y, z), as well as other attributes such as color, intensity, or normal vectors. Geometric attributes: features such as point density, spatial distribution, or point cloud extent can be quantified and summarized. Temporal attributes: for point cloud sequences that change over time, attributes such as motion or deformation of the point cloud can be analyzed and summarized. Topological attributes: the connectivity or relationships between points in the point cloud can be studied to provide a deeper understanding of the underlying structure of the data. This approach provides simplicity and consistency between similar point cloud types.

[0013] Preferably, at least one vertex is adjusted based on the difference between the virtual position and the centroid position, and more preferably, based on the boundaries of the edges of the leaf nodes. As previously mentioned, the vertex can be adjusted along certain defined edges toward the centroid position. Furthermore, the adjustment can be further based on the boundaries of the edges of the leaf nodes, thereby ensuring that the adjusted vertex does not extend beyond its respective leaf node. Therefore, this technique allows for more precise vertex adjustment, resulting in a more accurate representation of the point cloud geometry.

[0014] Preferably, the adjustment of the at least one vertex toward the average position of surface points is based on at least one neighboring leaf node, comprising: determining eligibility for vertex adjustment based on vertex information of the current leaf node and vertex information of neighboring leaf nodes; and adjusting the vertex if eligible. Therefore, the method involves adjusting vertex positions based on vertex information from neighboring leaf nodes; this complements the original position of the vertex as described in the previously described embodiments. The key to this preferred feature is the introduction of a “eligibility” condition that controls whether a vertex should be adjusted. This eligibility condition is determined by considering the vertex information of the current leaf node and the vertex information of neighboring leaf nodes. Therefore, by considering the vertex information of neighboring leaf nodes, the method can more accurately reflect the original spatial relationships and geometry of objects in a 3D point cloud. This makes the encoded and subsequently decoded 3D point cloud more representative of the original spatial arrangement and geometric features of the object or scene.

[0015] Preferably, determining the eligibility of a vertex to be adjusted, based on the vertex information of the current leaf node and the vertex information of adjacent leaf nodes, includes: determining a first vector pointing from the virtual position of the current leaf node to the centroid position of the current leaf node; determining a second vector pointing from the virtual position of an adjacent leaf node to the centroid position of an adjacent leaf node; and determining that at least one vertex is eligible for adjustment if the inner product of the first and second vectors is positive. This preferred feature introduces a vertex adjustment eligibility criterion based on vector mathematics. Two vectors are established, one from the virtual position of the current leaf node to its centroid, and the other from the virtual position of an adjacent leaf node to its centroid. The eligibility of a vertex to be adjusted depends on whether the inner product of these two vectors is positive. The technical advantage of this feature is that it provides a quantitative and geometrically reasonable method for determining whether a vertex is eligible for adjustment. The use of the inner product effectively measures the alignment of the two vectors. If the inner product is positive, it means that the two vectors point in roughly the same direction, implying that adjacent leaf nodes have similar geometric features, and therefore adjusting the vertex is meaningful and appropriate.

[0016] Preferably, if eligible, adjusting a vertex includes adjusting the vertex based on a weighted sum of the magnitudes of a first vector and a second vector, preferably with the first vector having a greater weight than the second vector. According to the above preferred features, once a vertex is deemed eligible for adjustment, the invention details the manner of this adjustment. It states that the adjustment should be based on a weighted sum of the magnitudes of the two vectors from the embodiment of claim 7. Furthermore, it is suggested that the weight of the first vector (from the current leaf node) should be higher than the weight of the second vector (from adjacent leaf nodes). The main technical effect here is that it provides fine-grained control over the vertex adjustment process. Weighting provides a method for prioritizing the geometry of the current leaf node relative to the geometry of adjacent nodes when adjusting a vertex. This makes vertex adjustment more respectful of local geometry, thereby representing 3D objects or scenes more accurately in the encoded and decoded point cloud data.

[0017] In another aspect of the invention, a method for encoding a 3D point cloud into a bitstream is provided, preferably implemented in an encoder. The method for encoding a 3D point cloud includes: obtaining vertex information of each cube associated with a leaf node from the surface of the point cloud, wherein the vertex information includes position information of vertices on the edges of the cubes of the leaf nodes with respect to the octree structure of the point cloud; encoding the vertex information into a bitstream, wherein at least one vertex is adjusted toward the average position of surface points; and determining points of the point cloud based on the encoded vertex information.

[0018] Based on the vertex information of at least one adjacent leaf node, adjust at least one vertex toward its original position.

[0019] Therefore, vertex information and preferably octree information are generated using the encoding method. This information can be encoded into a bitstream. Subsequently, a reconstruction step can be performed on the encoder side. In this reconstruction step, the point cloud geometry is reconstructed, wherein the reconstruction step is the same as the steps in the decoding method described above. Then, on the encoder side, the attributes (color, reflectivity, etc.) of the points in the point cloud are encoded using the reconstructed geometry of the point cloud, for example, by using RAHT (Region Adaptive Hierarchical Transform), predictive transform, or lifting transform to encode the attributes of the points in the point cloud.

[0020] Preferably, the geometric structure of the point cloud is encoded into a bit stream by geometry-based point cloud compression (G-PCC).

[0021] Preferably, the bitstream is a bitstream conforming to MPEG G-PCC.

[0022] Preferably, the encoding method is further constructed based on the features described above in conjunction with the decoding method.

[0023] In another aspect of the invention, an encoder for encoding 3D point clouds into a bitstream is provided. The encoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned encoding method are performed.

[0024] In another aspect of the invention, a decoder for decoding 3D point clouds from a bitstream is provided. The decoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned decoding method are performed.

[0025] In another aspect of the invention, a bitstream is provided, wherein the bitstream is encoded by the steps of the aforementioned encoding method.

[0026] In another aspect of the invention, a computer-readable storage medium is provided, comprising instructions for performing the steps of the method described above for encoding a 3D point cloud into a bit stream.

[0027] In another aspect of the invention, a computer-readable storage medium is provided, comprising instructions for performing the steps of the method described above for decoding a 3D point cloud from a bitstream. Attached Figure Description

[0028] The invention is described in more detail below with reference to the accompanying drawings.

[0029] The accompanying diagram shows:

[0030] Figure 1 A flowchart of a method for decoding 3D point cloud geometry according to the present invention is shown.

[0031] Figure 2 An example of generating an octree structure is shown.

[0032] Figure 3 Showing according to Figure 2 Octree

[0033] Figure 4 This shows an example of determining the vertices on the edges of a cube.

[0034] Figure 5 An example of generating a triangle is shown.

[0035] Figure 6 Examples of vertices on the edges of a cube are shown.

[0036] Figure 7 This shows how triangles are generated from vertices.

[0037] Figure 8 Showing according to Figure 7 An example of determining the order of triangles.

[0038] Figure 9 A schematic diagram of the voxelization process is shown.

[0039] Figure 10 This example illustrates how to reconstruct a triangle using the centroid C as the pivot point.

[0040] Figure 11 An example of a normal vector is shown.

[0041] Figure 12 An example of a 1D residual along the normal vector is shown.

[0042] Figure 13 An example is shown with a smooth convex surface in a leaf node of a modeled triangle constructed from vertices and centroids.

[0043] Figure 14 An example of a refined surface is shown.

[0044] Figure 15 An example of a visible hole produced on the common surface of two adjacent leaf nodes is shown.

[0045] Figure 16 Examples of vertices on face F that do not satisfy one of the conditions of this invention are shown.

[0046] Figure 17 An example of a vertex on face F that satisfies one of the conditions of this invention is shown.

[0047] Figure 18 An example of a refined vertex on face F between two adjacent leaf nodes A and B is shown. Detailed Implementation

[0048] Point clouds have recently gained attention as a format for representing 3D data because of their versatility in representing all types of 3D objects or scenes.

[0049] Therefore, many use cases can be handled using point clouds, including

[0050] • Film post-production

[0051] • Real-time 3D immersive experience or virtual reality (VR) / augmented reality (AR) applications,

[0052] • Free-viewpoint video (e.g., for watching sports).

[0053] • Geographic Information System (also known as mapping).

[0054] • Cultural heritage (storing scans of rare items in digital form).

[0055] • Autonomous driving, including 3D environmental mapping and real-time LiDAR data acquisition.

[0056] A point cloud is a set of points in 3D space, and additional values ​​can be optionally assigned to each point. These additional values ​​are often referred to as point attributes. Therefore, a point cloud is a combination of geometry (the 3D position of each point) and attributes.

[0057] The attributes can be, for example, three-component color, material properties such as reflectivity, and / or the two-component normal vector of the surface associated with that point.

[0058] Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, LiDAR, scanners, or can be generated by computers (e.g., in film post-production). Depending on the use case, a point cloud can have thousands to billions of points for mapping applications.

[0059] The raw representation of a point cloud requires a very high number of bits per point; each spatial component X, Y, or Z requires at least twelve bits, and optionally, attributes require even more bits, such as color requiring three times 10 bits. Practical deployment of point cloud-based applications requires compression techniques that enable the storage and distribution of point clouds through a suitable storage and transmission infrastructure.

[0060] For distribution to and visualization by end users, such as on AR / VR glasses or any other 3D-enabled device, compression can be lossy (e.g., in video compression). Other use cases do require lossless compression, such as medical applications or autonomous driving, to avoid altering decision-making outcomes derived from point cloud analysis during compression and transmission.

[0061] Until recently, point cloud compression (also known as PCC) had not been solved by the mass market, and there was no standardized point cloud codec available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Moving Picture Experts Group or MPEG, launched a working project on point cloud compression. This resulted in two standards, namely...

[0062] • MPEG-I Part 5 (ISO / IEC 23090-5) or Video-Based Point Cloud Compression (V-PCC)

[0063] • MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-Based Point Cloud Compression (G-PCC)

[0064] Both the V-PCC and G-PCC standards completed their first versions at the end of 2020.

[0065] The V-PCC encoding method compresses point clouds by performing multiple projections of the 3D object to obtain 2D patches that are packaged into an image (or, when processing moving point clouds, into video). The acquired image or video is then compressed using existing image / video codecs, allowing the utilization of already deployed image and video solutions. By its very nature, V-PCC is only effective on dense and continuous point clouds because image / video codecs cannot compress non-smooth patches, such as those obtained from projections of sparse geometry acquired from LiDAR.

[0066] The G-PCC coding method has two schemes for geometric compression. The first scheme is based on an occupancy tree (octree / quadtree / binary tree) representation of the point cloud geometry. Occupied nodes are segmented until a certain size is reached, and the occupied leaf nodes provide the location of points, typically at the center of these nodes. High levels of compression for dense point clouds can be achieved by using neighbor-based prediction techniques. Sparse point clouds are also compressed by directly encoding the locations of points within nodes with non-minimum sizes, addressing this by stopping tree construction when only isolated points exist in a node; this technique is called Direct Coding Mode (DCM).

[0067] The second approach is based on a prediction tree, where each node represents the 3D location of a point, and the relationship between nodes is a spatial prediction from parent to child. This method can only handle sparse point clouds and has the advantages of lower latency and simpler decoding compared to occupancy-based methods. However, compared to the first occupancy-based method, the compression performance is only slightly better, and the encoding is complex, involving densely searching for the best predictor (among a long list of potential predictors) when constructing the prediction tree.

[0068] In both schemes, attribute (solution) encoding is performed after geometry (solution) encoding is complete, resulting in two-pass encoding. Therefore, low latency is achieved by using slices that decompose the 3D space into independently coded sub-volumetric units, without requiring prediction between sub-volumetric units. However, using many slices can severely impact compression performance.

[0069] One important use case is the delivery of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves over time. Furthermore, AR / VR point clouds are typically local 2D because they represent the surface of objects most of the time. Therefore, AR / VR point clouds are highly connected (or dense) because points are rarely isolated but have many neighbors.

[0070] Dense (or solid) point clouds represent continuous surfaces with a resolution that allows the volumes associated with points (small cubes called voxels) to contact each other without revealing any visible holes on the surface.

[0071] These point clouds are typically used in AR / VR environments and viewed by end users through devices such as TVs, smartphones, or headphones. They are either transmitted to the device or stored locally. Many AR / VR applications use dynamic point clouds, rather than static ones, which change over time. Therefore, the data volume is enormous and must be compressed. Currently, lossless compression based on octree representations of the point cloud's geometry can achieve slightly less than one bit per point (1 bpp). This may not be sufficient for real-time transmission, which can involve millions of points per frame at frame rates up to 50 frames per second (fps), resulting in hundreds of megabits of data per second.

[0072] Therefore, lossy compression can be used to maintain acceptable visual quality as is often required, while being sufficiently compressed to fit the bandwidth provided by the transmission channel, and to ensure real-time frame transmission. In many applications, bit rates as low as 0.1 bpp (compression up to 10 times higher than lossless coding compression) have made real-time transmission possible.

[0073] VPCC, a codec based on MPEG-I Part 5 (ISO / IEC 23090-5) or Video-Based Point Cloud Compression (V-PCC), can achieve such low bit rates by using lossy compression from a video codec that compresses 2D frames obtained from the projection of a point cloud onto a plane. The geometry is represented by a series of projected patches assembled into a frame, each patch being a small local depth map. However, VPCC is not universal and is limited to narrow types of point clouds that do not exhibit complex local geometries (such as trees or hair) because the resulting projected depth maps are not smooth enough to be effectively compressed by the video codec.

[0074] Pure 3D compression technology can handle any type of point cloud. Whether 3D compression can compete with VPCC (or any projection + image coding scheme) on dense point clouds remains an open question. Standardization is still moving towards providing an extension (revision) to GPCC that offers competitive lossy compression, compressing dense point clouds as well as VPCC intra-frame coding, while maintaining GPCC's versatility in handling any type of point cloud (dense point cloud, LiDAR, 3D map). This extension may use the so-called TriSoup scheme, which is applicable to octrees, and will be discussed in detail later. The ISO / IEC standardization working group JTC1 / SC29 / WG7 is exploring TriSoup.

[0075] refer to Figure 1 It shows a schematic diagram of a method for decoding geometric information of a 3D point cloud from a bitstream.

[0076] A preferred method for decoding the geometry of a 3D point cloud from a bitstream, implemented in a decoder, includes the following steps:

[0077] S11: Receive a bit stream, wherein the bit stream contains vertex information;

[0078] S12: Obtain vertex information to determine the position of the vertex, wherein at least one vertex is adjusted toward the average position of the surface points;

[0079] S13: Determine the points of the point cloud based on at least one adjusted vertex, and adjust the average position of at least one vertex toward the surface points according to the vertex information of at least one adjacent leaf node.

[0080] To determine octree information, the first step in the geometric coding process is to construct and encode the octree, such as... Figure 2 and Figure 3 As shown. The bounding box is the body 100 containing all points and is associated with the root node 112 (i.e., the single node at the top of the tree 110). This body 100 is first divided into eight sub-bodies 102 called octants, each represented by a node 114 in the tree 110. Then, octants 104 are recursively subdivided from the sub-bodies 106 until a target level is reached, where each octant 106 is occupied by at least one point. Figure 2 and Figure 3 The text is shaded. The target level is the level at which the volume is no longer segmented, and it can be determined based on the compression ratio.

[0081] Each octet (or node) is represented by an occupation byte containing one bit for each sub-octet. If the sub-octet is occupied by at least one node, the corresponding bit is set to 1; otherwise, it is set to zero. All octet occupation bytes, 118 bytes in total, such as "1000100 00001000 11000011", are serialized (in breadth-first order) and entropy-encoded using a binary arithmetic encoder.

[0082] Figure 4 The diagram shows a block representation of 3D surface 210 and an example of block 220 in Trisoup. Surface 210 intersects with block 220, which is therefore an occupied block, and block 220 exists among multiple blocks 200 in 3D space. Within block 220, the closed portion of surface 210 intersects the edges of the block at six illustrated vertices 232 of polygon 230. If an edge of block 220 contains a vertex, then that edge is said to be selected.

[0083] Figure 5 Block 220 in the Trisoup is shown. For clarity, surface 210 is omitted. Unselected edge 270, selected edge 260, and the i-th edge 250 are shown. It is assumed that the i-th edge 250 is selected. To specify the vertex on edge i... Specify a scalar value to indicate the corresponding fraction of the length of edge 250.

[0084] As shown in Figures 4 and 5, within each octet 220 of the target level in the octree, Trisoup represents the original surface 210 as a set of triangles 245. This surface is encoded and used to obtain the positions of the reconstructed (or decoded) points. First, the intersections of the surface represented by the original points with the edges of the octet are estimated by averaging the positions of the points closest to those edges in the octet. Second, all twelve edges of the octets and their associated intersections (if any) are stored as line segments and vertices, respectively. Then, each (unique) line segment is encoded as follows: The first single bit is arithmetically encoded, set to 1 if the line segment is occupied by a vertex, and 0 otherwise. If it is occupied, the relative position of the vertex on the line segment is also arithmetically encoded.

[0085] The vertices 310 of the triangle are encoded along the edges 320 of the body associated with the leaf nodes 300 of the tree, as shown in Figure 6. These vertices 310 on the edges 320 are shared among the leaf nodes 300 that share a common edge 320. This means that each edge belonging to at least one leaf node encodes at most one vertex. In this way, the continuity of the model is ensured through the leaf nodes.

[0086] As mentioned above, encoding TriSoup vertices may require two pieces of information for each edge: a vertex flag indicating whether a TriSoup vertex exists on the edge, and, if it does exist, the vertex position along the edge.

[0087] Therefore, the encoded data includes octree data and TriSoup data.

[0088] Vertex markers are encoded by an adaptive binary arithmetic encoder that uses a specific context to encode vertex markers. The position of a vertex on an edge of length n = 2^s can be encoded with unit precision by pushing s bits into a bitstream (bypass / non-entropy encoding).

[0089] Within a leaf node, if there are at least three vertices 310 on edge 320 of leaf node 300, then construct a triangle from the vertices of TriSoup. Figure 7 The reconstructed triangles 330 and 340 are depicted in the figure.

[0090] Clearly, other combinations of triangles 33° and 34° are also possible. The selection of the triangle involves a three-step process.

[0091] 1. Determine the dominant direction along one of the three axes.

[0092] 2. Sort the TriSoup vertices according to the dominant direction.

[0093] 3. Constructing triangles based on an ordered list of vertices

[0094] It is not necessary to know the exact position of the triangle in the current leaf node, and it can be derived from the vertices.

[0095] Figure 8 This will be used to explain the process. Each of the three axes is tested, and the axis that maximizes the total surface area of ​​the triangle will be chosen as the principal axis. For the sake of simplicity in the diagram, Figure 8 The test is described only on two axes.

[0096] The first test (top) along the vertical axis is performed by projecting the cube and TriSoup vertex 310 vertically onto the 2D plane. Vertices 310 are then ordered clockwise relative to the center of the projected node (square). Triangles 330 and 340 are then constructed according to a fixed rule based on the ordered vertices. Here, triangles 123 and 134 are systematically constructed when there are 4 vertices. When there are 3 vertices, the only possible triangle is 123. When there are 5 vertices, the fixed rule can be to construct triangles 123, 134, and 451. This continues until a maximum of 12 vertices are involved.

[0097] A second test (left side) is performed along the horizontal vertical axis by horizontally projecting the cube and Trisoup vertices onto a 2D plane.

[0098] The vertical projection shows the largest total 2D surface area of ​​the triangle; therefore, the principal axis is chosen as the vertical axis, and the constructed TriSoup triangles are obtained by following the order of the vertical projections, as follows: Figure 8 As shown, it is located inside the node. It's important to note that using the horizontal axis as the principal axis will result in a different construction of the triangle.

[0099] By maximizing the projection surface to fully select the principal axis, continuous reconstruction of hole-free point clouds can be achieved.

[0100] Rendering a TriSoup triangle into points can be done using ray tracing. The collection of all points rendered by ray tracing will form the decoded point cloud.

[0101] for Figure 9 The ray tracing shown allows rays to be emitted along three directions parallel to the axis. Their origins are points with integer (voxed) coordinates corresponding to the sampling precision required for rendering. The intersection point with one of the Trisoup triangles (a dashed point, if any) is then voxelized (rounded to the nearest point with the required sampling precision) and added to the list of render points.

[0102] After applying Trisoup to all leaf nodes, i.e., constructing triangles and preferably obtaining points via ray tracing, discarding copies of identical points in the list of all rendered points (i.e., retaining only one voxel among all voxels sharing the same location and volume), a set of decoded (unique) points is obtained.

[0103] Based on the above concepts, Trisoup encoding can be improved. For example, by calculating the centroid, whose coordinates are the average coordinates of all (ordered) vertices Vi, see [link to relevant documentation]. Figure 10 The centroid C is depicted using a checkerboard pattern. The centroid is used as the pivot point. The triangle is formed by ordered vertices (V1, V2, …, V…). M By pivoting around the centroid C, construct the following M triangles: , , ..., , .

[0104] The vertices are sorted clockwise, and the choice of which vertex to use is... unimportant.

[0105] This structure preserves the model's natural symmetry without granting arbitrary privileges to the triangle. Furthermore, it provides additional degrees of freedom to improve the model's accuracy, specifically the position of the centroid C.

[0106] Therefore, the position of the centroid C can be further improved by encoding the residual position in the bitstream to make the position of the centroid C closer to the original point of the point cloud.

[0107] For example, .

[0108] in, The average position is obtained by averaging the coordinates of all (ordered) vertices, and It is the encoding residual.

[0109] The encoded residual can be a 3D residual. However, it has been observed that 3D residuals are rarely advantageous because they require encoding many bits, and such a large number of bits is not fully compensated for by the improved accuracy of the model. Therefore, 1D residuals are preferred. Encode it.

[0110] For example, normal vector It can be constructed, such as Figure 11 As shown, the residual can be determined by the following formula:

[0111] = α*

[0112] Where α is the 1D signed scalar value encoded in the bitstream, see Figure 12. Normal vector You can export it through the following two steps.

[0113] 1.

[0114] 2. Then normalize

[0115] in It is the cross product (also called the vector (cross) product) between two vectors, and the edges yes:

[0116] .

[0117] It is understandable that vectors It can be parallel to the axis to simplify its determination and the calculation of its value α. Vector It can be parallel to the principal axis, serving as a good approximation of the vector calculated above.

[0118] The value α is determined by the encoder, encoded into the bitstream, and obtained by the decoder by decoding the bitstream. The value α can be binarized, and each bit can be encoded using a binary entropy encoder such as an arithmetic encoder or a context-adaptive binary encoder such as CABAC.

[0119] The value α can be binarized as

[0120] A flag indicating whether α is equal to 0

[0121] instruct >0 or The sign of <0,

[0122] instruct Flag indicating whether it equals 1 ,

[0123] Remainder encoded by the exponential Golomb encoder -2.

[0124] The value α can be obtained by the encoder by considering all points in the point cloud belonging to the current leaf node. To determine. For each point. Its relationship with the straight line ( , distance From the following formula, we get

[0125] = || ‖,

[0126] Furthermore, if this distance Below a predefined threshold (For example, =2), use point Calculate the value α. Point Relative to the average point 1D residual Obtained through scalar product (also known as inner product or dot product)

[0127] = .

[0128] Therefore, the value α is obtained by the following formula.

[0129] α =

[0130] Where S is a point A set such that their distances Below the threshold And |S| is the number of points belonging to this set.

[0131] In the current trisoup encoding, as described in conjunction with the preceding figures, the determined centroid C and the vertices of the leaf nodes ( A ray can be used to construct a triangle, and then ray tracing methods can be applied to the constructed triangle to obtain the reconstructed points.

[0132] Point cloud data captured in natural scenes often have smooth surfaces, even in concave or convex regions. However, in current trisoup encoding, modeling concave or convex regions using triangles in leaf nodes results in centroid residuals. It may be large (e.g., (It could be about half the width of a leaf node), then use the triangle constructed inside the leaf node (by using the centroid C and vertices (V0,...,V)). i The modeled surface will have large protrusions, and the resulting surface will be far from naturally smooth, exhibiting artifact effects. For example... Figure 13 As shown, point 1301 represents a vertex along the edge in a leaf node, and point 1303 represents the centroid C in a leaf node. Small point 1305 represents a point on the original surface (for clarity, only one of these small points is labeled 1305). It can be seen that a modeling surface with sharp protrusions cannot model the original surface well enough, and the reconstructed surface may not retain the smooth characteristics of the original surface. This can result in poor visual quality in the reconstructed point cloud, especially when the leaf node size is relatively large, at lower compression rates. Therefore, the compression efficiency of trisoup encoding in the current MPEG G-PCC is not optimal.

[0133] Therefore, it is preferable to improve the compression efficiency of point clouds by reducing the non-smoothness of the reconstructed surface on the decoder side.

[0134] The triangle modeling in the leaf nodes can be refined to make the reconstructed surface on the decoder side more closely resemble the original surface. Specifically, for example, after obtaining the decoded vertices and decoded centroid C on the decoder side, vectors in the leaf nodes facing the convex or concave regions can be refined. The method adjusts the position of the vertices in the direction of the reconstructed point cloud, thereby reducing the non-smoothness of the reconstructed point cloud without increasing the bitstream size.

[0135] In some embodiments, the method according to the present invention follows the steps of:

[0136] On the decoder side, after decoding the vertices of each leaf node from the bitstream, each leaf node can be iterated to construct triangles for obtaining the reconstructed point cloud, preferably by a ray tracing method.

[0137] In detail, for each leaf node, for example,

[0138] If the current (i.e., the processed) leaf node has more than 3 vertices,

[0139] First, the centroid C can be determined to construct a modeling surface, which consists of triangles formed by vertices and the centroid C. Specifically, the vertices in the leaf nodes are first determined ( , (mean point) and the unit vector of the centroid residual. Then the magnitude of the centroid residual can be decoded from the bitstream. Finally, it can be done Obtain the centroid C, where, From The vector to C can also be named .

[0140] Then, the convexity of the modeling surface in the leaf node can be determined, and in a preferred embodiment, the centroid residual can be determined. Is it large relative to the size of the leaf node?

[0141] If the centroid residual The size is relatively large compared to the leaf node size (more likely, based on the current vertex). , The surface constructed by the centroid C will have sharp protrusion artifacts in the generated modeled surface. Then, for all vertices V( in the leaf nodes) , These vertices in the vector are refined to be vectors along all three axes. For vertices on the edges of axes with maximum values, the thinning can be performed along the edges to which these vertices belong, within the boundaries of the edges in the leaf nodes, in a direction that makes the reconstructed surface more natural. (Detailed vertex thinning will be described later.)

[0142] Otherwise, all vertices in the leaf nodes ( , (This) may not be further refined.

[0143] Then, all vertices in the leaf node (including the thinned vertex V' and other non-thinned vertices V) are combined with the centroid C to construct triangles, as described above with reference to the attached figure, and ray tracing methods can be applied to each triangle to obtain the reconstructed points in the leaf node.

[0144] Otherwise, if a leaf node has 3 vertices, then there is no average point in that processed leaf node. And the centroid C, and there is only one triangle It can be constructed, and ray tracing methods can be applied to the triangle to obtain the reconstructed points in the leaf nodes.

[0145] Otherwise, if a leaf node has fewer than 3 vertices, no triangle will be constructed for the leaf node, and therefore no reconstruction point will be generated for the leaf node.

[0146] On the encoder side, after encoding the vertices of each leaf node into the bitstream, each leaf node can be iterated to encode information about the centroid position C. For subsequent attribute encoding, it may also be necessary to obtain the reconstructed geometry of the point cloud by constructing triangles based on the vertices and centroid positions C. Furthermore, to obtain a refined modeled surface in the leaf nodes, the leaf nodes can be iteratively processed using the same steps described above on the decoder side. Of course, information indicating whether the point cloud is planar is also encoded at the encoder.

[0147] In some embodiments, in order to determine the centroid residual The size of the centroid residual can be determined relative to the size of the leaf nodes. This is compared to a threshold Th, which is based on the leaf node size N. The threshold Th can be a proportion s of the leaf node size. And the ratio s can be ,For example, Furthermore, in actual implementations, to improve the accuracy of division / multiplication operations, It can be multiplied by 64 (by adjusting the...) This is achieved by using a left shift of 6 bits, therefore a comparison threshold needs to be set. Multiply by 64 to become ). And, for example, for leaf nodes, if If so, the vertices in the leaf nodes can be refined according to the above method; otherwise, the proposed surface refinement method will not be applied to the leaf nodes.

[0148] As mentioned above, if the centroid residual If the size is larger than that of the leaf node, then for all vertices in the leaf node ( , The vector in the three axes Vertices on the edges of axes with maximum values ​​are refined, either along the edges to which these vertices belong, within the boundaries of the edges in the leaf nodes, and in a direction that makes the reconstructed surface more natural. In a preferred embodiment, detailed vertex refinement may follow these steps:

[0149] First, determine the vector. Along the axis whose maximum value is axis_max;

[0150] Then, we can find the vertices (V0,…,V) on the edges parallel to axis_max in the leaf nodes. i ),

[0151] Then, based on the centroid residual vector of the leaf node Change the axis_max dimension of these vertices and obtain the vertices. It can be described as

[0152] Where, offset is relative to offset distance, It is a unit vector pointing in the direction of offset. In a preferred embodiment, Based on And the boundary of the edge, where, Represents the centroid residual vector along The projection vector of the direction, and the boundary of the edge are constraints that keep the vertex from exceeding its own leaf node. Furthermore, in such an embodiment, Can be along with Same direction, and offset can be based on The modulus value and the boundary of the edge of the leaf node are obtained. In a preferred embodiment, the offset can be... 1 / 3 of the modulus. Figure 14 The diagram illustrates a refined surface produced according to the present invention. Point 1401 on the edge is a vertex V used for modeling the surface according to the trisoup method. The arrow line parallel to the X-axis represents the projection vector of the centroid residual, and point 1401a (only one of these points is labeled for clarity) is obtained by passing through along... Refined vertex obtained by moving vertex V in the direction Point 1405 represents a point on the original surface (only one of these points is labeled for clarity). It can be seen that the refined surface is closer to the original surface in the leaf node and reduces the artifacts of the sharp protrusions.

[0153] As described above, the vertices on the edges of leaf nodes can be refined sequentially, one leaf node at a time. The refined vertices within a leaf node can then be used to construct a triangle to obtain the reconstructed point cloud. However, this refinement method is based solely on the centroid offset information of the current leaf node. Therefore, vertices on edges belonging to two adjacent leaf nodes can be refined along the directions shown in the example below. Offset by different distances:

[0154] Example (a): Centroid offset direction between two adjacent leaf nodes It's the opposite.

[0155] Or, for example (b): the centroids of two adjacent leaf nodes are offset in direction. Similarly, their offset values ​​also differ significantly.

[0156] Therefore, the above example will result in some visible holes appearing on the surface of the reconstructed point cloud, such as... Figure 15 As shown, square 1501 represents the adjacent face of two adjacent leaf nodes, and point 1503 with a dashed outline on its edge represents an unrefined vertex. and This represents the vertex offset vector (including direction and value), and the dashed area represents the hole caused by the vertex offset. Figure 15 In the diagram, the holes are generated due to the vertex offset vector. and The directions are opposite in two adjacent leaf nodes, corresponding to example (a); Figure 15 In, although the vertex offset vector and The directions are the same, but their values ​​differ greatly, which can also cause holes, corresponding to example (b) above.

[0157] These visible holes can reduce visual quality and compression efficiency. Therefore, the problem to be solved by the present invention is to further improve encoding efficiency, preferably by eliminating holes generated by vertex thinning methods.

[0158] In some embodiments, the centroid offset information of adjacent leaf nodes (encoded / processed) can be used to guide vertex thinning of leaf nodes, thus resolving the visibility hole problem. Specifically, each leaf node can be traversed, and for each leaf node, if the centroid residual... If the size is larger than the leaf node size, then it will be determined to be located parallel to... Vertex on the edge of the direction This is the same as the refinement method described above. In determining the vertices... Then, the following steps can be performed.

[0159] First, determine the vertices in the current leaf node. Whether refinement is needed can be based on the centroid offset information of its adjacent leaf nodes (encoded / processed).

[0160] Then, if the vertex in the current leaf node If further refinement is needed, then by direction. Each vertex is moved by an offset distance, where the offset is based on the centroid offset of the adjacent leaf nodes. The modulus value is determined, and the original vertex Will be updated to refine vertices Otherwise, the vertices in the current leaf node It will not be updated.

[0161] In obtaining the vertices of the leaf nodes (including refining the vertices) After (and non-refined vertices), triangles can be constructed using these vertices and the determined centroids, and then preferably, ray tracing methods are used to construct point clouds from these triangles in the leaf nodes.

[0162] The advantages of the proposed method include: it can solve the "hole problem" by eliminating the vertex offset difference between two or more adjacent leaf nodes; and it enables the thinned vertex in leaf node A (thinned by s1) and the thinned vertex in adjacent leaf nodes of A (thinned by s2) to become... Figure 15 The common points on the middle edge.

[0163] According to the present invention, the qualification for vertex refinement refers to the vertices in the current leaf node. Whether further refinement is needed. In a preferred embodiment,

[0164] First, based on conditions determined by the centroid offsets of adjacent leaf nodes and the current leaf node, a face F of the current leaf node is selected. This face F may contain vertices that need to be refined. The conditions may include:

[0165] Face F is parallel to the current leaf node direction.

[0166] Find the coded / processed neighboring leaf node that shares surface F with the current leaf node and whose centroid offset is not 0.

[0167] Centroid offset vector in adjacent leaf nodes centroid offset vector in the current leaf node The inner product between them is greater than 0. For example, Figure 16 The example shown does not meet this condition, while Figure 17 The example shown satisfies this condition. (In another embodiment, the components of the centroid offset vector in adjacent leaf nodes and the current leaf node) It's the opposite.

[0168] Then, determine whether the vertices on the edges of face F satisfy a condition, namely, face F along... Vertex on the edge of the direction The number of them is greater than 1.

[0169] Preferably, the refinement qualification of the vertices in the current leaf node can be true only if the vertices in the current leaf node satisfy the above conditions.

[0170] In some embodiments, if the vertex refinement qualification in the current leaf node is true, the vertex offset can be determined, and the offset value can be determined based on the centroid offset modulus in the encoded / processed adjacent leaf nodes, because in the current trisoup encoding, the centroid offset information is calculated sequentially for each leaf node, and when processing the current leaf node, only the centroid offset information of the encoded / processed leaf nodes can be known and retrieved.

[0171] In a preferred embodiment, the vertices on face F The offset can be determined by the centroid offset magnitudes of adjacent leaf nodes of the shared surface F. For example, the offset can be a weighted average of the centroid offset magnitudes of two adjacent leaf nodes A and B, which can be described as...

[0172] (1)

[0173] in, and Both can be set to 1 / 2. Preferably, k > b.

[0174] Meanwhile, the vertices on face F The offset can also be determined based on the boundary of the edge of the leaf node. Similar to the vertex refinement method mentioned above, the boundary of the edge is a constraint to ensure that the vertex does not exceed its leaf node.

[0175] In some embodiments, the vertex thinning method may need to follow a principle that is parallel to the direction All points on the edge of a given surface that belong to two adjacent leaf nodes must be refined by moving the same offset distance along the same direction.

[0176] In some embodiments, after obtaining vertex refinement eligibility and vertex offsets of vertices on face F in leaf nodes, along the vector (its unit vector is) The direction of ) will move each vertex by offset distance, and each vertex Refine into Then, refine the vertices. It can be expressed as follows:

[0177]

[0178] like Figure 18 As shown, leaf node A (left) is the processed leaf node, and leaf node B is the current leaf node whose vertices need to be refined. Two vertices, V3 and V4, meet the refinement criteria. Pointing from V4... The arrows represent the offset vector. * And according to equation (2), V3 and V4 are refined into and Then, refine the vertices. and It will be used together with the centroid C in the current leaf node to reconstruct the triangle.

[0179] In some embodiments, the steps for triangle modeling refinement in the proposed point cloud data trisoup encoding method are as follows:

[0180] On the decoder side, after decoding the vertices of each leaf node from the bitstream, each leaf node is iterated to construct a triangle, thereby obtaining the reconstructed point cloud through ray tracing.

[0181] In detail, for each leaf node,

[0182] If the leaf node being processed has more than 3 vertices

[0183] First, the centroid C can be determined to construct a modeling surface consisting of triangles formed by the vertices and the centroid C. Specifically, the vertices (V0,…,V) in the leaf nodes can be determined first. i The average point C) mean and the unit vector of the centroid residual Then, the modulus C of the centroid residual is decoded from the bitstream. res Finally, we can use C=C mean +C res * We obtain the centroid C, where C res * From C mean The vector to C can also be named .

[0184] Then, the convexity of the modeled surface in the leaf node is determined. In a preferred embodiment, the centroid residual C is determined. res Is it relatively large compared to the leaf node size?

[0185] If the centroid residual C res The leaf node size is relatively large (based on the current vertex (V0,…,V)). i The surface constructed from the centroid C is more likely to exhibit sharp artifacts in the generated modeled surface. Therefore, all vertices V(V1,…,V) in the leaf nodes... i Searching along ) Vertex on the edge of the direction Then, for each vertex The refined qualifications are determined using the method proposed in this invention.

[0186] If the refinement eligibility of the vertices in the current leaf node is true, then the vertex offset is determined using the method described above, and then each vertex is refined using the aforementioned method. Refine into .

[0187] Otherwise, all vertices (V0,…,V) in the leaf nodes i (This will not be refined.)

[0188] Then, triangles are constructed by combining all vertices in the leaf node (including the refined vertex V' using the proposed method and other non-refined vertices V) with the centroid C, and ray tracing methods can be applied to each triangle to obtain the reconstructed points in the leaf node.

[0189] Otherwise, if a leaf node has 3 vertices, then there is no average point in that processed leaf node. And the centroid C, and there is only one triangle It can be constructed, and ray tracing methods can be applied on the triangle to obtain the reconstructed points in the leaf nodes.

[0190] Otherwise, if a leaf node has fewer than 3 vertices, a triangle will not be constructed for the leaf node, and therefore no reconstruction point will be generated for the leaf node.

[0191] On the encoder side, after encoding the vertices of each leaf node into the bitstream, each leaf node is iterated to encode information about the centroid position C. For subsequent attribute encoding, it may also be necessary to obtain the reconstructed geometry of the point cloud by constructing triangles based on the vertices and centroid positions C. Furthermore, to obtain the refined modeled surface in the leaf nodes, the leaf nodes can be iteratively processed following the same steps described above on the decoder side.

[0192] In example embodiments of the encoder or decoder, the encoder or decoder includes a processor and a memory storage device. The memory storage device may store a computer program or application containing instructions that, when executed, cause the processor to perform operations such as those described herein. For example, the instructions may encode and output an encoded bitstream or decode a bitstream and output points of a point cloud according to the methods described herein. It should be understood that the instructions may be stored on a non-transient computer-readable medium, such as an optical disc, flash memory, random access memory, hard disk drive, etc. When the instructions are executed, the processor performs the operations and functions specified in the instructions in order to operate as a dedicated processor implementing the processes. In some examples, such a processor may be referred to as a "processor circuit" or "processor circuitry".

[0193] It is understood that the decoder and / or encoder according to this application can be implemented in many computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. The decoder or encoder can be implemented by software containing instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transitory computer-readable storage medium, including CDs, RAM, ROM, flash memory, etc.

[0194] It should be understood that the decoders and / or encoders described herein, as well as modules, routines, procedures, threads, or other software components implementing the methods / processes for configuring the encoders or decoders, can be implemented using standard computer programming techniques and languages. This application is not limited to specific processors, computer languages, computer programming conventions, data structures, or other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as part of computer-executable code stored in volatile or non-volatile memory, as part of an application-specific integrated circuit (ASIC), etc.

[0195] Certain adjustments and modifications can be made to the described embodiments. Therefore, the embodiments discussed above are considered illustrative rather than restrictive. In particular, the embodiments can be freely combined with each other.

Claims

1. A method for decoding the geometry of a 3D point cloud from a bitstream, implemented in a decoder, the method comprising: Receive a bitstream, wherein the bitstream includes vertex information; The vertex information is acquired to determine the position of the vertex, wherein at least one vertex is adjusted toward the average position of the surface points; The points of the point cloud are determined based on at least one adjusted vertex. Adjusting the average position of at least one vertex toward the surface point based on the vertex information of at least one adjacent leaf node includes: determining the vertex's eligibility for adjustment based on the vertex information of the current leaf node and the vertex information of the adjacent leaf nodes; if the vertex is eligible for adjustment, then adjusting the vertex. Wherein, the inner product of the first vector and the second vector corresponding to the vertex with adjustment qualification is a positive number, the first vector is a first vector pointing from the virtual position of the current leaf node to the centroid position of the current leaf node, and the second vector is a second vector pointing from the virtual position of the adjacent leaf node to the centroid position of the adjacent leaf node.

2. The method according to claim 1, wherein, The step of acquiring the vertex information to determine the vertex position, wherein at least one vertex is adjusted toward the average position of the surface points, includes: Determine the centroid location, wherein the centroid location is configured as an approximation of the average location of the surface points; Adjust at least one vertex toward the centroid position.

3. The method according to claim 2, wherein, Determining the centroid position includes: The virtual position is determined based on the position of a vertex of a cube associated with a leaf node of the octree structure; The centroid position is determined based on the virtual position.

4. The method according to claim 3, wherein, Determining the centroid position based on the virtual position includes: Construct a triangle based on the vertices of a cube and the virtual positions; The position of the centroid is determined based on the constructed triangle.

5. The method according to claim 4, wherein, Determining the centroid position based on the constructed triangle includes: The normal vector of the cube is determined based on the constructed triangle; The position of the centroid is determined based on the normal vector.

6. The method according to any one of claims 3 to 5, wherein, At least one vertex is adjusted only when the difference between the virtual position and the centroid position exceeds a threshold, wherein the threshold is determined based on the size of the leaf node.

7. The method according to claim 6, wherein, The threshold is based on a proportion of the leaf node size. To determine, among which, .

8. The method according to claim 7, wherein, =1 / 128。 9. The method according to claim 1, wherein, Adjust at least one of the vertices according to a fixed value.

10. The method according to claim 9, wherein, The fixed value is determined based on the type of the point cloud and is encoded into or from the bit stream.

11. The method according to claim 3, wherein, At least one vertex is adjusted based on the difference between the virtual position and the centroid position.

12. The method according to claim 11, wherein, Further, at least one of the vertices is adjusted based on the boundaries of the edges of the leaf nodes.

13. The method according to claim 1, wherein, If there is an adjustment qualification, then adjusting the vertex includes: adjusting the vertex based on a weighted sum of the modulus values ​​of the first vector and the second vector.

14. The method according to claim 13, wherein, The weight of the first vector is greater than the weight of the second vector.

15. A method for encoding 3D point clouds into a bitstream, implemented in an encoder, comprising: Obtain vertex information for each cube associated with a leaf node from the point cloud surface. The vertex information is encoded into the bitstream, wherein at least one vertex is adjusted toward the average position of the surface points; The points of the point cloud are determined based on the encoded vertex information; Adjusting the average position of at least one vertex toward the surface point based on the vertex information of at least one adjacent leaf node includes: determining the vertex's eligibility for adjustment based on the vertex information of the current leaf node and the vertex information of the adjacent leaf nodes; if the vertex is eligible for adjustment, then adjusting the vertex. Wherein, the inner product of the first vector and the second vector corresponding to the vertex with adjustment qualification is a positive number, the first vector is a first vector pointing from the virtual position of the current leaf node to the centroid position of the current leaf node, and the second vector is a second vector pointing from the virtual position of the adjacent leaf node to the centroid position of the adjacent leaf node.

16. The method according to claim 15, wherein, Encoding the vertex information into the bitstream, wherein at least one vertex is adjusted toward the average position of the surface points, includes: Determine the centroid location, wherein the centroid location is configured as an approximation of the average location of the surface points; Adjust at least one vertex toward the centroid position.

17. The method according to claim 16, wherein, Determining the centroid position includes: The virtual position is determined based on the position of a vertex of a cube associated with a leaf node of the octree structure; The centroid position is determined based on the virtual position.

18. The method according to claim 17, wherein, Determining the centroid position based on the virtual position includes: Construct a triangle based on the vertices of a cube and the virtual positions; The position of the centroid is determined based on the constructed triangle.

19. The method according to claim 18, wherein, Determining the centroid position based on the constructed triangle includes: The normal vector of the cube is determined based on the constructed triangle; The position of the centroid is determined based on the normal vector.

20. The method according to any one of claims 17 to 19, wherein, At least one vertex is adjusted only when the difference between the virtual position and the centroid position exceeds a threshold, wherein the threshold is determined based on the size of the leaf node.

21. The method according to claim 20, wherein, The threshold is based on a proportion of the leaf node size. To determine, among which, .

22. The method according to claim 21, wherein, =1 / 128。 23. The method according to claim 15, wherein, Adjust at least one of the vertices according to a fixed value.

24. The method according to claim 23, wherein, The fixed value is determined based on the type of the point cloud and is encoded into or from the bit stream.

25. The method according to claim 17, wherein, At least one vertex is adjusted based on the difference between the virtual position and the centroid position.

26. The method of claim 25, wherein, Further, at least one of the vertices is adjusted based on the boundaries of the edges of the leaf nodes.

27. The method according to claim 15, wherein, If there is an adjustment qualification, then adjusting the vertex includes: adjusting the vertex based on a weighted sum of the modulus values ​​of the first vector and the second vector.

28. The method according to claim 27, wherein, The weight of the first vector is greater than the weight of the second vector.

29. An encoder for encoding 3D point clouds into a bitstream, the encoder comprising at least one processor and a memory, wherein, The processor stores instructions that, when executed by the processor, perform the steps of the method according to any one of claims 15 to 28.

30. A decoder for decoding 3D point clouds from a bitstream, the decoder comprising at least one processor and a memory, wherein, The memory stores instructions, which, when executed by the processor, perform the steps of the method according to any one of claims 1 to 14.

31. A computer-readable storage medium comprising instructions that, when executed by a processor, perform the steps of the method according to any one of claims 1 to 28.

Images