Method, encoder and decoder for encoding and decoding 3D point clouds

Abstract

A method for decoding a geometry of a 3D point cloud from a bitstream by decoding information constituting a geometry representation of the geometry, preferably implemented in a decoder, comprising: receiving and decoding a bitstream, wherein the bitstream contains octree information comprising information about an octree structure of a volume of the point cloud and vertex information comprising information about the existence of vertices on the edges of a cube and the position of the vertices; determining triangles by connecting vertices of a cube of the octree structure; and voxelizing the triangles to determine points of the point cloud, wherein at least one triangle is extended along at least one edge for voxelization.

Description

Technical Field

[0001] This invention relates to a method for decoding 3D point clouds from a bitstream. Furthermore, an object of this invention is to provide a method for encoding 3D point clouds into a bitstream. Further, an object of this invention is to provide an encoder and decoder, a bitstream encoded according to the invention, and software. In particular, an object of this invention is to provide a method for improving the accuracy of the 3D point cloud decoding or reconstruction process. Background Technology

[0002] Point clouds have recently gained attention as a format for representing 3D data due to their versatility in representing all types of 3D objects or scenes. Therefore, many use cases can be addressed using point clouds, including...

[0003] Film post-production,

[0004] Real-time 3D immersive experience or virtual reality (VR) / augmented reality (AR) applications,

[0005] Free-viewpoint video (e.g., for watching sports),

[0006] Geographic Information System (also known as mapping),

[0007] Cultural heritage (storing scanned copies of rare items in digital form),

[0008] Autonomous driving includes 3D mapping of the environment and real-time LiDAR data acquisition.

[0009] A point cloud is a set of points in 3D space, with optional additional values ​​attached to each point. These additional values ​​are often referred to as point attributes. Therefore, a point cloud is a combination of geometry (the 3D position of each point) and attributes.

[0010] The attributes can be, for example, three-component color, material properties such as reflectivity, and / or two-component normal vectors of the surface associated with the point.

[0011] Point clouds can be captured by various types of devices, such as camera arrays, depth sensors, LiDAR, scanners, or can be generated by computers (e.g., in film post-production). Depending on the use case, a point cloud may contain thousands to billions of points for mapping applications.

[0012] The raw representation of a point cloud requires a very high number of bits per point, with at least twelve bits per spatial component X, Y, or Z, and optionally more bits for attributes, such as three times ten bits for color. Practical deployment of point cloud-based applications requires compression techniques that enable the storage and distribution of point clouds with a reasonable storage and transmission infrastructure.

[0013] For distribution to and visualization by end users, such as on AR / VR glasses or any other 3D-enabled devices, compression can be lossy (e.g., in video compression). Other use cases do require lossless compression, such as medical applications or autonomous driving, to avoid altering the decision-making outcomes derived from the analysis of compressed and transmitted point clouds.

[0014] Until recently, point cloud compression (also known as PCC) had not been solved by the mass market, and there was no standardized point cloud codec available. In 2017, the standardization working group ISO / JCT1 / SC29 / WG11, also known as the Moving Picture Experts Group or MPEG, launched a working project on point cloud compression. This resulted in two standards, namely...

[0015] MPEG-I Part 5 (ISO / IEC 23090-5) or Video-Based Point Cloud Compression (V-PCC)

[0016] MPEG-I Part 9 (ISO / IEC 23090-9) or Geometry-Based Point Cloud Compression (G-PCC)

[0017] Both the V-PCC and G-PCC standards completed their first versions at the end of 2020 and are about to be released.

[0018] The V-PCC encoding method compresses point clouds by performing multiple projections of the 3D object to obtain 2D patches that are packaged into an image (or, when processing dynamic point clouds, into a video). The acquired image or video is then compressed using existing image / video codecs, allowing the utilization of already deployed image and video solutions. By its very nature, V-PCC is only effective on dense and continuous point clouds because image / video codecs cannot compress non-smooth patches, such as those obtained from projections of sparse geometry acquired from LiDAR.

[0019] The G-PCC encoding method has two schemes for compressing geometry.

[0020] The first approach is based on an occupancy tree (octree / quadtree / binary tree) representation of the point cloud geometry. Occupied nodes are segmented until a certain size is reached, and the occupied leaf nodes provide the location of points, typically at the center of these nodes. High levels of compression for dense point clouds can be achieved by using neighbor-based prediction techniques. Sparse point clouds are also handled by directly encoding the locations of points within nodes with non-minimum sizes, stopping tree construction when only isolated points exist in a node; this technique is called Direct Coding Mode (DCM).

[0021] The second approach is based on a prediction tree, where each node represents the 3D location of a point, and the relationship between nodes is a spatial prediction from parent to child. This method can only handle sparse point clouds and has the advantages of lower latency and simpler decoding compared to occupancy-based methods. However, compared to the first occupancy-based method, the compression performance is only slightly better, and the encoding is complex, involving densely searching for the best predictor (among a long list of potential predictors) when constructing the prediction tree.

[0022] In both schemes, attribute (solution) encoding is performed after geometry (solution) encoding, resulting in two encoding passes. Therefore, low latency can be achieved by using slices that decompose the 3D space into independently coded sub-volumetric units without requiring prediction between sub-volumetric units. However, using a large number of slices can severely impact compression performance.

[0023] One important use case is the delivery of dynamic AR / VR point clouds. Dynamic means that the point cloud evolves over time. Furthermore, AR / VR point clouds are typically local 2D because they represent the surface of objects most of the time. Therefore, AR / VR point clouds are highly connected (or dense) because points are rarely isolated but have many neighbors.

[0024] Dense (or solid) point clouds represent continuous surfaces with a resolution that allows the volumes associated with points (small cubes called voxels) to contact each other without revealing any visible holes on the surface.

[0025] These point clouds are typically used in AR / VR environments and viewed by end users through devices such as TVs, smartphones, or headphones. They are either transmitted to the device or stored locally. Many AR / VR applications use dynamic point clouds, rather than static ones, which change over time. Therefore, the data volume is enormous and must be compressed. Currently, lossless compression based on octree representations of point cloud geometry can achieve slightly less than one bit per point (1 bpp). This may not be sufficient for real-time transmission, which can involve millions of points per frame at frame rates up to 50 frames per second (fps), resulting in hundreds of megabits of data per second.

[0026] Therefore, lossy compression can be used to maintain the usual requirements of acceptable visual quality while being sufficiently compressed to fit the bandwidth provided by the transmission channel, thus ensuring real-time frame transmission. In many applications, bit rates as low as 0.1 bpp (10 times higher than the compression ratio of lossless coding) have made real-time transmission possible.

[0027] VPCC, a codec based on MPEG-I Part 5 (ISO / IEC 23090-5) or Video-Based Point Cloud Compression (V-PCC), can achieve such low bit rates by using lossy compression from a video codec that compresses 2D frames derived from the projection of a point cloud onto a plane. The geometry is represented by a series of projected patches assembled into a frame, each patch being a small local depth map. However, VPCC is not universal and is limited to narrow types of point clouds that do not exhibit complex local geometries (such as trees or hair) because the resulting projected depth maps are not smooth enough to be effectively compressed by the video codec.

[0028] Pure 3D compression technology can handle any type of point cloud. Whether 3D compression can compete with VPCC (or any projection + image coding scheme) on dense point clouds remains an open question. Standardization is still moving towards providing an extension (revision) to GPCC that offers competitive lossy compression, compressing dense point clouds as well as VPCC intra-frame coding, while maintaining GPCC's versatility in handling any type of point cloud (dense point cloud, LiDAR, 3D map). This extension may use the so-called Trisoup coding scheme, which is suitable for octrees. The ISO / IEC standardization working group JTC1 / SC29 / WG7 is exploring TriSoup. TriSoup coding is also known, as reported by A. DRICOT et al., “Adaptive multi-level triangle soup for geometry–based pointcloud coding”, 2019, IEEE 21st international workshop on multimedia signal processing (MMSP), Nakagami O.: “report on triangle soup decoding”, ISO / IEC JTC1 / SC29-WG11 m52279, 2020, and US10,192,353.

[0029] However, for all lossless compression schemes, the reconstruction quality of points in the point cloud is crucial. Summary of the Invention

[0030] Therefore, the object of the present invention is to provide a method for decoding the geometry of a 3D point cloud from a bitstream and a method for encoding a 3D point cloud into a bitstream, the method of which improves accuracy.

[0031] In a first aspect, a method is provided for decoding the geometry of a 3D point cloud from a bitstream, preferably implemented in a decoder. The method includes:

[0032] Receive and decode a bitstream, wherein the bitstream includes octree information and vertex information, the octree information includes information about the octree structure of the volume of the point cloud, and the vertex information includes information about the existence and position of vertices on the edges of the cubes of the leaf nodes of the octree structure.

[0033] A triangle is determined by connecting the vertices of a cube associated with the leaf nodes of the octree structure;

[0034] Voxelization of the triangles is used to determine the points in the point cloud.

[0035] In this case, at least one triangle is extended along at least one side for voxelization.

[0036] Therefore, in the first step, a bitstream is received, and this bitstream contains information about the octree structure of the volume of the point cloud being decoded. Preferably, the geometry of the point cloud is GPCC encoded. Thus, by decoding from the bitstream, octree information about the volume of the point cloud can be provided. Furthermore, the bitstream also includes vertex information, which includes information about the presence and position of vertices on the edges of the cubes associated with the leaf nodes in the octree structure. Thus, vertex information is provided by decoding from the bitstream. Preferably, the bitstream is encoded in the encoder using the Trisoup encoding scheme.

[0037] After decoding the octree and vertex information from the bitstream described in the previous step, in a further step for reconstructing the point cloud geometry, triangles are determined for each cube by connecting the vertices on the edges of the cube. Therefore, the surface of a triangle is determined by the positions of the vertices included in the bitstream. To reconstruct the points of the point cloud from the triangles, voxelization is performed via ray tracing, where, during ray tracing, rays are emitted in three directions parallel to any one of the three axes. Their origins are integer coordinate points corresponding to the sampling precision required for rendering. The intersection point of the ray with one of the reconstructed triangles (if any) is then determined and added to the list of rendered points, i.e., to the points of the point cloud. During voxelization, the surface of the reconstructed triangle is sampled using rays to determine the points of the point cloud.

[0038] According to the invention, at least one triangle is extended along at least one side to expand the surface of the triangle in at least one direction during voxelization. Therefore, by extending the triangle during voxelization, the accuracy of the voxelization process can be improved because additional points in the original point cloud that would otherwise be ignored during voxelization can be reliably determined. Since the triangle is sampled with specific precision and sampling resolution, points in the point cloud that are just outside the triangle are now captured because the triangle is extended along at least one side to expand the surface of the triangle. Therefore, higher accuracy in reconstructing 3D point clouds is achieved, and the number of sampling errors during voxelization is reduced.

[0039] Preferably, at least one triangle is extended along more than one side to further enlarge the surface of the respective triangle. Thus, the triangle can be expanded along one, two, or all three sides to include points from the original point cloud that extend just beyond the triangle defined by the vertices on the edges of the cube.

[0040] Preferably, if a cube containing a leaf node of the octree structure can contain more than one triangle, each triangle in the cube is extended along at least one edge for voxelization. Therefore, the extension of the triangle surface can be applied to all triangles in the cube. Alternatively or additionally, in each cube of the octree structure, at least one triangle is extended along at least one edge for voxelization. Alternatively, the extension of one or more edges of the triangle will be applied only to a subset of the leaf nodes in the octree structure. This subset can be determined, for example, by the density of points in the leaf nodes of the point cloud or by requirements for accuracy and decoding speed.

[0041] Preferably, the expansion is the same for each side. Therefore, the triangle expands by the same amount in at least two directions to enlarge its surface. More preferably, the expansion is the same in all three directions. Alternatively, the expansion is different in at least two directions. Therefore, different directions can be treated differently to improve decoding accuracy.

[0042] Preferably, the extensions for each leaf node of the octree structure are the same or different. If a triangle in a leaf node of the octree structure has more than one edge or each edge has a different extension, this can be the same or different in the other leaf nodes of the octree structure. The extensions can be pre-selected or determined, for example, by applying a point cloud, the density of points in the leaf nodes, or requirements for accuracy and decoding speed.

[0043] Preferably, by - The Trumbore algorithm is used to perform voxelization.

[0044] Preferably, in In the Trumpore algorithm, the convex hull condition is relaxed to -ε ≤ u,v,w, where ε > 0, and u,v,w are the centroid coordinates of the triangle. In the original... In the Trumbore algorithm, the convex hull condition is set to 0 ≤ u, v, w. Therefore, by relaxing this requirement to -ε ≤ u, v, w, the surface of the triangle under consideration is expanded, and voxelization of points that would otherwise not have been considered during the original point cloud sampling process is now included in the reconstructed point cloud. This ultimately improves the quality and appearance of the reconstructed point cloud.

[0045] Preferably, the convex hull condition is set to -ε u ≤u,-ε v ≤v and -ε w ≤w, where ε u ,ε v ,ε w ≥0, and u, v, w are the centroid coordinates of the triangle. Therefore, for different directions, separate convex hull conditions can be provided to individually control the expansion of the triangle under consideration. Where ε u ≠ε w Alternatively or additionally, ε u ≠ε v Alternatively or additionally, ε v ≠ε w Therefore, the extension in one or more directions can be chosen independently of the other directions to determine the extension separately.

[0046] Preferably, the extension is provided by the halo parameter. Wherein, in In the case of the -Trumbore algorithm, the halo parameters are provided by ε, and for different directions by ε u ,ε v and ε w Therefore, the amount of expansion can be determined and quantified using the halo parameter.

[0047] Preferably, the halo parameter is set to less than 1 / 4 of the voxelization sampling precision or sampling resolution. Therefore, if the halo parameter is set to 1 / 4 in one direction and the sampling distance is set to D, the triangle expands by D / 4 in that direction. More preferably, the halo parameter is set to less than 1 / 8 of the voxelization sampling precision or sampling resolution. Therefore, by selecting the halo parameter, the expansion amount can be adjusted to achieve optimal results, where a larger value will result in more points being determined during voxelization.

[0048] Preferably, the halo parameters are preset. Therefore, the encoder and decoder may have already agreed on the halo parameters, and thus the halo parameters are fixed for each point cloud generated by the encoder and reconstructed by the decoder. Information about the halo parameters does not need to be encoded into the bitstream.

[0049] Alternatively, the halo parameters are encoded into the bitstream, and preferably in the bitstream's geometry parameter set (GPS). This can be done only once if the halo parameters are set for each subsequent point cloud to be decoded. Alternatively, for each point cloud, the corresponding halo parameters or halo parameter set can be encoded separately.

[0050] Alternatively, the halo parameter depends on the size of the cube's volume, i.e., the level of the octree of the current leaf node.

[0051] In another aspect of the invention, a method for encoding 3D point clouds into a bitstream is provided, preferably implemented in an encoder. The 3D point cloud encoding method includes:

[0052] Obtain octree information, wherein the octree information includes an octree structure of a volume containing multiple cubes;

[0053] For each cube associated with a leaf node, vertex information is obtained from the surface of the point cloud, wherein the vertex information includes information about the presence and position of vertices on the edges of the cube;

[0054] Encode the octree information and vertex information into a bit stream;

[0055] Using the octree and vertex information obtained from the aforementioned encoding process, the point cloud geometric data is reconstructed;

[0056] The reconstruction of the point cloud geometric data includes:

[0057] A triangle is determined by connecting the vertices of a cube associated with the leaf nodes of the octree structure;

[0058] Voxelize the triangles to determine the points in the point cloud;

[0059] In this process, at least one triangle is extended along at least one side for voxelization;

[0060] Therefore, octree information and vertex information are generated using the encoding method. This information is encoded into a bitstream. Subsequently, a reconstruction step is performed on the encoder side. In this reconstruction step, the point cloud geometry is reconstructed, and the reconstruction steps are the same as those in the decoding method described above. Then, on the encoder side, the reconstructed geometry of the point cloud is used to encode attributes (color, reflectivity, etc.), for example, by using RAHT (Region Adaptive Hierarchical Transform), predictive transform, or lifting transform to encode the attributes of the points in the point cloud.

[0061] Preferably, the geometric structure of the point cloud is encoded into a bit stream by geometry-based point cloud compression (G-PCC).

[0062] Preferably, the bitstream is a bitstream conforming to MPEG G-PCC.

[0063] Preferably, the encoding method is further constructed based on the features described above in conjunction with the decoding method.

[0064] In another aspect of the invention, an encoder for encoding 3D point clouds into a bitstream is provided. The encoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned encoding method are performed.

[0065] In another aspect of the invention, a decoder for decoding 3D point clouds from a bitstream is provided. The decoder includes a memory and a processor, wherein instructions are stored in the memory, and when the instructions are executed by the processor, the steps of the aforementioned decoding method are performed.

[0066] In another aspect of the invention, a bitstream is provided, wherein the bitstream is encoded based on the steps of the aforementioned encoding method.

[0067] In another aspect of the invention, a computer-readable storage medium is provided, comprising instructions for performing the steps of the method described above for encoding a 3D point cloud into a bitstream.

[0068] In another aspect of the invention, a computer-readable storage medium is provided, comprising instructions for performing the steps of the method described above for decoding a 3D point cloud from a bitstream. Attached Figure Description

[0069] The invention will now be described in more detail with reference to the accompanying drawings.

[0070] The accompanying diagram shows:

[0071] Figure 1 A flowchart of a method for decoding 3D point cloud geometry according to the present invention is shown.

[0072] Figure 2An example of generating an octree structure is shown.

[0073] Figure 3 Showing according to Figure 2 Octree

[0074] Figure 4 This example shows how to determine the vertices on the edges of a cube.

[0075] Figure 5 An example of generating a triangle is shown.

[0076] Figure 6 Examples of vertices on the edges of a cube are shown.

[0077] Figure 7 This shows how triangles are generated from vertices.

[0078] Figure 8 Showing according to Figure 7 An example of determining the order of triangles.

[0079] Figure 9 A schematic diagram of the voxelization process is shown.

[0080] Figure 10 The diagram shows a 2D representation of a triangle in a leaf node of an octree.

[0081] Figure 11 Show Figure 10 An example of voxelization of a triangle.

[0082] Figure 12 The diagram shows the coordinates and definition of the centroid of the triangle.

[0083] Figure 13 This shows a comparison between the vertex triangle and the original point cloud.

[0084] Figure 14 Show Figure 10 The centroid coordinates of the triangle extend in one direction.

[0085] Figure 15 Show Figure 10 The triangle extends in all three directions.

[0086] Figure 16 Show Figure 11 The representation extends in three directions, and

[0087] Figure 17 A flowchart illustrating the encoding method is shown. Detailed Implementation

[0088] Reference Figure 1 It illustrates a method for decoding geometric information from a 3D point cloud from a bitstream.

[0089] A preferred method for decoding the geometry of a 3D point cloud from a bitstream, implemented in a decoder, includes the following steps:

[0090] In step S01, a bitstream is received and decoded, wherein the bitstream contains octree information and vertex information, the octree information includes information about the octree structure of the volume of the point cloud, and the vertex information includes information about the existence and position of vertices on the edges of the cubes of the leaf nodes of the octree structure.

[0091] In step S02, a triangle is determined by connecting the vertices of a cube associated with the leaf node of the octree structure;

[0092] In step S03, voxelization is performed on the triangle to determine the points in the point cloud.

[0093] Prior to voxelization, at least one triangle is extended along at least one edge.

[0094] To determine octree information, the first step in the geometric coding process is to construct and encode the octree, such as... Figure 2 and Figure 3 As shown. The bounding box is the body 100 containing all points and is associated with the root node 112 (i.e., the single node at the top of the tree 110). This body 100 is first divided into eight sub-bodies 102, called octets, each represented by a node 114 in the tree 110. Then, octets 106 are recursively subdivided from the sub-bodies 104 until the target level is reached, wherein each octet 106 is occupied by at least one point. Figure 2 and Figure 3 The text is represented by shading.

[0095] Each octet (or node) is represented by an occupation byte containing one bit for each sub-octet. If the sub-octet is occupied by at least one node, the corresponding bit is set to one; otherwise, it is set to zero. All octet occupation bytes (118 bytes) are serialized (in breadth-first order) and entropy encoded using a binary arithmetic encoder.

[0096] Figure 4 The diagram shows a block representation of 3D surface 210 and an example of block 220 in TriSoup. Surface 210 intersects with block 220, so block 220 is an occupied block, and block 220 exists among multiple blocks 200 in 3D space. Within block 220, the closed portion of surface 210 intersects with the edges of the block at the six illustrated vertices of polygon 230. If an edge of block 220 contains a vertex, that edge is said to be selected.

[0097] Figure 5Block 220 in TriSoup is shown. For clarity, surface 210 is omitted. Unselected edge 270, selected edge 260, and the i-th edge 250 are shown. Assume the i-th edge 250 is selected. To specify the vertex v on edge i... i Specify a scalar value that indicates the corresponding fraction of the length of edge 250.

[0098] like Figure 4 and Figure 5 As shown, within each octet 220 at the target level of the octree, trisoup represents the original surface 210 as a set of triangles 245. This surface is encoded and used to obtain the positions of the reconstructed (or decoded) points. First, the intersections of the surface represented by the original points with the edges of the octets are estimated by averaging the positions of the points closest to those edges within the octets from the original points used to represent the surface. Second, all twelve edges of the octets and their associated intersections (if any) are stored as segments and vertices, respectively. Then, each (unique) segment is encoded as follows: The first single bit is arithmetically encoded, set to 1 if the segment is occupied by a vertex, and 0 otherwise. If it is occupied, the relative position of the vertex on the segment is also arithmetically encoded.

[0099] The vertex 310 of the triangle is encoded along the edge 320 of the body associated with the leaf node 300 of the tree, such as... Figure 6 As shown, these vertices 310 on edge 320 are shared among multiple leaf nodes 300 that share a common edge 320. This means that each edge belonging to at least one leaf node encodes at most one vertex. In this way, the continuity of the model is ensured through the leaf nodes.

[0100] As mentioned above, TriSoup vertex encoding requires two pieces of information for each edge:

[0101] Vertex markers indicating whether a TriSoup vertex exists on the edge, and

[0102] When it exists, the vertex position along the edge.

[0103] Therefore, the encoded data includes octree data and TriSoup data.

[0104] Vertex markers are encoded by an adaptive binary arithmetic encoder that uses a specific context to encode the vertex markers. The length is N=2. s The position of the vertex on the edge can be encoded with unit precision by pushing s bits into a bit stream (bypass / non-entropy coding).

[0105] Within a leaf node, if there are at least three vertices 310 on edge 320 of leaf node 300, then construct a triangle from the vertices of TriSoup. Figure 7 The reconstruction of triangles 330 and 340 is depicted in the text.

[0106] Clearly, other combinations of triangles 330° and 340° are possible. The selection of the triangle involves a three-step process.

[0107] 1. Determine the dominant direction along one of the three axes.

[0108] 2. Sort TriSoup vertices based on the dominant direction

[0109] 3. Constructing triangles based on an ordered list of vertices

[0110] It is not necessary to know the exact position of the triangle in the current leaf node, and it can be derived from the vertices.

[0111] Figure 8 This will be used to explain the process. Each of the three axes is tested, and the axis that maximizes the total surface area of ​​the triangle will be chosen as the principal axis. For the sake of simplicity in the diagram, Figure 8 The test is described only on two axes.

[0112] The first test (top) along the vertical axis is performed by projecting the cube and TriSoup vertex 310 vertically onto the 2D plane. Then, vertices 310 are ordered clockwise relative to the center of the projected node (square). Triangles 330, 340 are then constructed based on the ordered vertices according to fixed rules. Here, triangles 123 and 134 are systematically constructed when there are 4 vertices. When there are 3 vertices, the only possible triangle is 123. When there are 5 vertices, the fixed rule can be to construct triangles 123, 134, and 451. This continues until a maximum of 12 vertices are involved.

[0113] A second test (left side) is performed along the horizontal vertical axis by horizontally projecting the cube and Trisoup vertices onto a 2D plane.

[0114] The vertical projection shows the largest total 2D surface area of ​​the triangle; therefore, the principal axis is chosen as the vertical axis, and the constructed TriSoup triangles are obtained by following the order of the vertical projections, as follows: Figure 8 As shown, it is located inside the node. It's important to note that using the horizontal axis as the principal axis will result in a different construction of the triangle.

[0115] By maximizing the projection surface to appropriately select the principal axis, continuous reconstruction of hole-free point clouds can be achieved.

[0116] Rendering TriSoup triangles into points is performed by ray tracing. The collection of all points rendered by ray tracing forms the decoded point cloud.

[0117] for Figure 9 The ray tracing shown radiates rays along three directions parallel to the axis. Their origins are points with integer (voxelized) coordinates corresponding to the sampling precision required for rendering. The intersection point with one of the Trisoup triangles (a dashed point, if any) is then voxelized (rounded to the nearest point with the required sampling precision) and added to the list of render points.

[0118] After applying Trisoup to all leaf nodes, i.e., constructing triangles and obtaining points through ray tracing, discarding copies of identical points in the list of all rendered points (i.e., keeping only one voxel among all voxels sharing the same location and volume), a set of decoded (unique) points is obtained.

[0119] For simplicity, let's start here, the following... Figures 10 to 16 The 2D volume (square) associated with the leaf node will be depicted instead of the 3D volume (cube). The reader will remember that all methods described in this invention are applicable to 3D space.

[0120] refer to Figure 10 It shows an example of an N×N×N volume, where N = 2 s =8. There are at least three vertices V1, V2, V3 on the edge 410 of the body (depicted as a square in the figure, but actually a cube).

[0121] The edges of leaf nodes are located at positions -0.5 and N-0.5 to ensure the continuity of the TriSoup model when passing from a "volume" to adjacent volumes. In practice, this means that the faces of a cube are shared between adjacent volumes. Thus, the position of a vertex on an edge is independent of the cube to which that edge belongs.

[0122] The position p of the vertex along its corresponding edge k These are quantized positions 400 and encoded into the bitstream. These positions 400 can be quantized using unit steps, such that p... k It is an integer in the interval [0, N-1]. Figure 10 In the example, p1 = 4, p2 = 2, and p3 = 2.

[0123] TriSoup triangle 440 is constructed from vertices V1, V2, and V3, and belongs to the set of triangles of the volume that model the point cloud contained in the volume.

[0124] The process of recovering point 430 (from the decoded point cloud) from triangle 440 is called triangle voxelization. Figure 11 exhibit Figure 10 Voxelization of the TriSoup triangle. Rays are emitted along all integer coordinates 420 (white and black points), and rays intersecting the triangle point towards the partial decoding point (black point). The origin of the rays has a spacing D, which sets the sampling resolution for voxelization.

[0125] The intersection of the light rays and the triangle is achieved by using... - This is obtained using the Trumbore algorithm, which determines the location of the intersection point by using the centroid coordinates, such as... Figure 12 As shown.

[0126] Any point P in 3D space can be uniquely represented by its centroid coordinates relative to any non-degenerate 3D triangle ABC (equivalent to any triangle V1V2V3 from the TriSoup model).

[0127] Any point P in 3D space can be uniquely represented as

[0128] P = uA + vB + wC

[0129] Among them, conditional

[0130] u+v+w=1.

[0131] The points of the triangle correspond to the convex hull. Therefore,

[0132] 0≤u,v,w.

[0133] -Trumbore determines the values ​​of u and v; then w can be simply calculated using w = 1 - uv. According to Figure 12 The light ray originates from point P. start In direction Emit. Set the following notation for the 3D vector derived from the 3D points, such as... Figure 12 As shown:

[0134] and

[0135] The intersection point P of the ray and the unique plane passing through A, B, and C is obtained through the following calculation.

[0136]

[0137]

[0138]

[0139]

[0140]

[0141]

[0142]

[0143] The intersection point P belongs to a triangle if and only if 0 ≤ u, v, w.

[0144] The positions of Trisoup triangles V1V2V3, determined by the vertices in the bitstream, are slightly shifted from their natural positions at 45° in the current volume, such as... Figure 13 As shown. This location is natural because the encoder has inferred vertex V from the nearest (relative to the edge) point in the original point cloud. k The position of V. Therefore, it is very likely that it is adjacent to vertex V. k The voxelized points are the points of the point cloud. These points are natural candidate points for constructing "natural" triangle models of the point cloud.

[0145] This movement is caused by the continuity constraint of adjacent objects. This results in the loss of some points (460°) in ray tracing. Figure 13 P on miss ), because these points do not belong to the Trisoup triangle (as opposed to the triangle formed by...). Figure 11 (Compared to triangle 440, which is determined by vertices provided by the bitstream). The direct result is a decrease in quantization geometry metrics and a reduction in the rate-distortion performance of the scheme.

[0146] Therefore, by slightly relaxing the convex hull condition 0 ≤ u, v, w, a "halo" is created around the TriSoup triangle. This slightly increases the size of the triangle, causing ray tracing to intersect with the increased-sized triangle and lose fewer points P. miss .

[0147] The advantages of the proposed method are:

[0148] The decoded point cloud exhibits less distortion. In fact, quantization metrics show better PSNR, with a PSNR improvement of up to 0.5 dB for small 4×4×4 volume sizes.

[0149] Maintain complexity because the algorithm as a whole remains unchanged.

[0150] Let ε>0 be the halo parameter. For example... Figure 14 As shown, the condition 0≤u is relaxed to -ε≤u, where u is the centroid weight associated with point A. The size of the triangle is increased along the side BC opposite to point A, as shown in the dashed area 470.

[0151] By changing the convex hull 0≤u,v,w to -ε≤u,v,w, the condition can be relaxed to apply to the three centroid weights u,v, and w.

[0152] The halo 480 obtained around triangle 440, as shown Figure 15 As shown. Under the first-order approximation, the size of the halo is proportional to the parameter ε.

[0153] In one embodiment, the halo parameter can depend on the weight of each centroid of the triangle, for example, -ε. u ≤u、-ε v ≤v and -ε w ≤w. Where ε u , ε v and ε w There are three halo parameters.

[0154] exist Figure 16 The above depicts the effect on voxelization, where point P... miss It is now part of the halo 480 and is therefore decoded as a point in the decoded point cloud. Therefore, it is not lost as it would have been in the original algorithm.

[0155] Of course, the halo parameter ε must be set (or alternatively, ε). u ε v and ε w The value of ε is determined to ensure a sufficiently large halo. If ε is too small, the halo is very small and has almost no effect, thus reverting to the lost point problem in existing techniques. If ε is too large, the halo becomes large, and the overall accuracy of the Trisoup model is affected. In both cases, the distortion of the decoded point cloud is not optimal.

[0156] It has been observed that a reasonable value for the halo parameter ε is approximately ε≈1 / 4 or ε≈1 / 8.

[0157] Halo parameter ε (which can be replaced by ε) u ε v and ε w The halo parameter ε can be a fixed value. In one variation, the halo parameter ε (which can be replaced by ε) is a fixed value. u ε v and ε w The halo parameter ε is encoded into a bitstream, for example, in a set of geometric parameters (GPS). In another variant, the halo parameter ε (which can be replaced by ε) is... u ε v and ε w The halo parameter ε depends on the volume size N. In another variation, for a set of volumes representing a point cloud, the halo parameter ε is locally represented (or alternatively, ε...). u ε v and ε w ).

[0158] Reference Figure 17 This illustrates a flowchart of a method for encoding a 3D point cloud into a bitstream according to the present invention. The method includes:

[0159] In step S11, octree information is determined, which includes an octree structure of a volume containing multiple cubes;

[0160] In step S12, vertex information is obtained from the surface of the point cloud of each cube associated with the leaf node, wherein the vertex information includes information about the presence of vertices on the edges of the cube and the vertex positions;

[0161] In step S13, the octree information and vertex information are encoded into a bit stream;

[0162] In step S14, the point cloud data is reconstructed using the octree information and vertex information obtained during the above encoding process. The reconstructed point cloud data includes:

[0163] In step 141, a triangle is determined by connecting the vertices of a cube associated with the leaf node of the octree structure;

[0164] In step 142, voxelization is performed on the triangle to determine the points of the point cloud;

[0165] In this case, at least one triangle is extended along at least one side for voxelization.

[0166] Steps S11 to S13 involve known Trisoup coding, such as those from A. Dricot et al., “Adaptive Multi-Level Triangle Soup for Geometry-Based Point Cloud Coding,” 2019, IEEE 21st International Workshop on Multimedia Signal Processing (MMSP), Nakagami O., “Report on Triangle Soup Decoding,” ISO / IEC JTC1 / SC29-WG11 M52279, 2020, and US10,192,353. In addition to the usual point cloud coding, the method also includes a reconstruction step, which involves reconstructing the data with reference to previously specifically referenced... Figure 1 The decoding method described uses the same or similar steps. The reconstructed point cloud is then used to interpolate attributes (such as color), and the attributes of the points in the point cloud are encoded based on the reconstructed geometry.

Claims

1. A method for decoding 3D point cloud geometry from a bitstream, implemented in a decoder, the method comprising: Receive and decode a bitstream, wherein the bitstream contains octree information and vertex information, the octree information including information about the octree structure of the volume of the point cloud, and the vertex information including information about the existence and position of vertices on the edges of the cube of the leaf node of the octree structure. A triangle is determined by connecting the vertices of a cube associated with a leaf node of the octree structure; and The triangles are voxelized to determine the points in the point cloud. Wherein, at least one triangle is extended along at least one side for voxelization; wherein, the extension is provided by a halo parameter, and the halo parameter is encoded into the bitstream, wherein encoding the halo parameter into the bitstream includes: the halo parameter being encoded into a set of geometric parameters.

2. A method for encoding 3D point clouds into a bitstream, implemented in an encoder, the method comprising: Obtain octree information, wherein the octree information includes an octree structure of a volume containing multiple cubes; Vertex information is obtained from the surface of the point cloud of each cube associated with a leaf node, wherein the vertex information includes information about the presence and position of vertices on the edges of the cube; The octree information and the vertex information are encoded into a bitstream; The geometric data of the point cloud is reconstructed using the octree information and vertex information obtained in the aforementioned encoding process. The reconstructed geometric data of the point cloud includes: A triangle is determined by connecting the vertices of a cube associated with a leaf node of the octree structure; and Voxelize the triangles to determine the points in the point cloud; In this embodiment, at least one triangle is extended along at least one side for voxelization, wherein the extension is provided by a halo parameter, and the halo parameter is encoded into the bitstream, wherein encoding the halo parameter into the bitstream includes: the halo parameter being encoded into a geometric parameter set.

3. The method of claim 2, wherein, The encoding is Trisoup encoding.

4. The method according to any one of claims 1 to 3, wherein, The at least one triangle is extended on two or three sides for voxelization.

5. The method according to any one of claims 1 to 3, wherein, Each triangle in the cube is expanded, and at least one triangle in each cube of the point cloud with triangles is expanded.

6. The method according to any one of claims 1 to 3, wherein, The extension is the same for every edge or different for at least two edges.

7. The method according to any one of claims 1 to 3, wherein, The voxelization is performed using the Möller-Trumbore algorithm, and / or by rounding the coordinates of the point to the nearest integer.

8. The method according to claim 7, wherein, The convex hull condition is -ε ≤ u, v, w, where ε > 0, ε is used to provide the halo parameters, and u, v, w are the centroid coordinates of the triangle.

9. The method according to claim 7, wherein, The convex hull condition is -ε u ≤u, -ε v ≤v and -ε w ≤w, where, , Used to provide halo parameters in the directions corresponding to u, v, and w, where u, v, and w are the centroid coordinates of the triangle, and ε u ≠ε w and / or ε u ≠ε v and / or ε v ≠ε w .

10. The method according to any one of claims 1 to 3, wherein, The extension is provided by a halo parameter, and the halo parameter of the extension is less than 1 / 4 of the sampling precision of the voxelization and less than 1 / 8 of the sampling precision of the voxelization.

11. The method according to any one of claims 1 to 3, wherein, The extension is provided by the halo parameter, and the extension is preset.

12. An encoder for encoding 3D point clouds into a bitstream, the encoder comprising at least one processor and a memory, wherein, The memory stores instructions, which, when executed by the processor, perform the steps of the method according to any one of claims 2 to 11.

13. A decoder for decoding 3D point clouds from a bitstream, the decoder comprising at least one processor and a memory, wherein, The memory stores instructions that, when executed by the processor, perform the steps of the method according to claim 1 and any one of claims 3 to 11 referencing claim 1.

14. A computer program product for encoding a bitstream, the bitstream being encoded by the method according to any one of claims 2 to 11.

15. A computer-readable storage medium comprising instructions that, when executed by a processor, perform the steps of the method according to any one of claims 1 to 11.

Images