Methods of encoding and decoding, encoders, decoders and software

Abstract

Methods and an encoder (10) and a decoder (20) for encoding and decoding a point cloud are disclosed. A method for encoding a point cloud to generate a bitstream (14, 26) of compressed point cloud data (12, 28), wherein a geometry of the point cloud is represented by an octree-based structure comprising a plurality of nodes having a parent-child relationship by recursively splitting a volume space containing the point cloud into sub-volumes, each sub-volume being associated with a node of the octree-based structure, the method comprising the steps of determining context information for a current node (120), the context information comprising plane position context information, wherein the plane position context information for the current node (120) is determined depending on an occupancy of neighboring nodes (126) directly adjacent to the current node (120) and an occupancy of at least one sibling node (134) having a same parent node (132) as the current node (120) (S110), and entropy encoding at least an occupancy pattern of the current node based on the determined context information to generate encoded data for the bitstream (S120).

Description

Technical Field

[0001] This application generally relates to point cloud compression. Preferably, this application relates to encoding and decoding methods for improving entropy coding of point clouds, as well as encoders and decoders. Background Technology

[0002] 3D point clouds, replacing 3D meshes, have recently become a popular representation of 3D media information. The use cases associated with point cloud data are very diverse and include:

[0003] 3D assets in film production

[0004] • 3D assets used for real-time 3D immersive presence or virtual reality (VR) applications.

[0005] • 3D free-viewpoint video (e.g., for watching sports),

[0006] Geographic Information System (Cartography)

[0007] • Cultural heritage (digitally stored fragile assets),

[0008] • Autonomous driving (large-scale 3D environment mapping), etc.

[0009] A point cloud is a set of points in 3D space, where each point has associated attributes such as color and material properties. Point clouds can be used to reconstruct objects or scenes from combinations of these points. Multiple cameras and depth sensors with different settings can be used to acquire point clouds, which can consist of thousands or even billions of points, to realistically reproduce the reconstructed scene.

[0010] The location of each point in a point cloud needs to be stored (typically encoding X, Y, and Z information as 32-bit or 64-bit floating-point numbers) and its attributes (typically representing at least 24-bit RGB colors). Point clouds sometimes contain billions of points, making it easy to interpret raw point cloud data that can easily reach gigabytes. Therefore, compression techniques are urgently needed to reduce the amount of data required to characterize point clouds.

[0011] Two different methods were developed for point cloud compression:

[0012] First, in the Video-based Point Cloud Compression (VPCC) method, point cloud compression is performed by projecting the point cloud multiple times along three different axes (X, Y, Z) and at different depths, ensuring all points appear in a single projected image. The projected image is then processed into patches to eliminate redundancy and rearranged into a final image. Metadata appended to this final image is used to convert pixel locations into point locations in space. Compression is then performed using a traditional image / video MPEG encoder. The advantage of this method is that it reuses existing encoders and naturally supports dynamic point clouds (using video encoders). However, this method is almost unsuitable for sparse point clouds, and it is expected that using point cloud-specific methods can improve compression gain.

[0013] Second, in Geometry-based Point Cloud Compression (GPCC), point positions (often referred to as geometry) and point attributes (color, transparency, etc.) are encoded separately. An octree structure is used to encode the geometry. The entire point cloud is fitted to a cube, which is then repeatedly divided into eight sub-cubes until each sub-cube contains only a single point. Therefore, the point position is replaced by placeholder information for each node in the tree. Since each cube has only eight sub-cubes, 3 bits are sufficient to encode the placeholders; therefore, for a tree of depth D, 3 bits are needed. D Bits are used to encode the position of a point. While this transformation alone is insufficient to provide significant compression gain, it should be noted that due to the tree structure, many points share the same node value, and the use of an entropy encoder can significantly reduce the amount of information.

[0014] Considering that many point clouds include surfaces, a planar encoding mode has been introduced in the current GPCC design to encode such nodes more efficiently when octrees are applicable.

[0015] Therefore, a flag `isPlanar` is introduced, which indicates whether the placeholder child node belongs to the same horizontal plane. If `isPlanar` is true, an additional bit is used to flag `planePosition` to indicate whether the plane is the lower plane or the upper plane.

[0016] The `isPlanar` flag is encoded using a binary arithmetic encoder, where 3 bits of context information are used as plane context information representing the axis perpendicular to the plane. Furthermore, if the node is planar, the `planePosition` information is encoded using a binary arithmetic encoder, where 72 (=2x3x2x2) bits of context information are used as planar position context information including the following:

[0017] a) Placeholders for the two vertical neighbors adjacent to the current node (4 contexts: 00, 01, 10, 11),

[0018] b) The distance d to the nearest encoded node at the same coordinates and depth, where d is discretized into three values: "near", "not too far", or "far".

[0019] c) The planar location (if any) of the nearest encoded node at the same coordinates and depth, and

[0020] d)axisIdx (value = 0, 1 or 2), identifies the axis orthogonal to the plane.

[0021] Furthermore, an angular encoding mode is introduced for sparse point cloud data to enhance the planar encoding mode using angular context, where adjacent node occupancy tends to be zero (i.e., empty) in most cases. More specifically, the angular encoding mode replaces the context information used when encoding planePosition with reduced context information, while still using the same context model as the planar encoding mode to arithmetically encode the isPlanar flag.

[0022] However, current point cloud encoding and decoding methods do not consider information about neighboring nodes.

[0023] Therefore, the object of this invention is to provide an encoding and decoding method, as well as an encoder and decoder, to support fast and accurate data compression of point clouds. Summary of the Invention

[0024] In one aspect of the invention, a method is provided for encoding a point cloud to generate a bitstream of compressed point cloud data, wherein the geometric structure of the point cloud is characterized by an octree-based structure, the octree-based structure comprising a plurality of nodes having a parent-child relationship by recursively partitioning the volume space containing the point cloud into child volumes, each child volume being associated with a node of the octree-based structure, the method comprising the following steps:

[0025] Determine the context information of the current node, the context information including planar position context information for at least one geometric axis, wherein the planar position context information of the current node on the at least one geometric axis is determined based on the occupancy of adjacent nodes directly adjacent to the current node on the at least one geometric axis and the occupancy of at least one sibling node having the same parent node as the current node; and

[0026] Entropy encoding is performed on the placeholder information of the current node based on the determined context information to generate the encoded data of the bit stream.

[0027] The context information includes planar position context information of the current node, indicating the probability of a planar position within the current node, i.e., a prediction of the planar position within the current node. This planar position context information is determined based on the occupancy of neighboring nodes and at least one sibling node. The neighboring node is located at the same depth D in the tree structure as the current node and is directly adjacent to the current node, i.e., sharing a common surface. The neighboring node has a different parent node than the current node (at depth D-1). Furthermore, the planar position context information is determined based on the occupancy of at least one or more sibling nodes. These sibling nodes share a common parent node with the current node and are located at the same depth D in the tree structure. The occupancy indicates whether the corresponding node includes at least one point in the point cloud. The planar position context information includes information about possible planar or surface positions represented by the point cloud of the current node. Information about the occupancy of at least one sibling node and one neighboring node of the current node is considered to determine the context information used for entropy encoding. Therefore, a more reliable prediction of the occupancy of the current child node can be made, which can be used to increase the amount of data reduction through the entropy encoder. In this way, the entire tree is traversed to determine the placeholder for each node and to provide sufficient contextual information for the entropy encoder, thereby reducing the amount of data required to characterize the point cloud.

[0028] Preferably, adjacent nodes are positioned directly before or after the current node along the considered geometric axis, sharing a common surface with the current node along the considered geometric axis. The geometric axis can be one of the X, Y, or Z axes. For example, if the X-axis is considered, adjacent nodes are arranged along the X-axis directly in front of or behind the current node. Therefore, the selection of adjacent nodes depends on the considered axis to include placeholder information for adjacent nodes in the determination of contextual information.

[0029] Preferably, the at least one or more sibling nodes include one of the following:

[0030] A sibling node directly preceding the current node along the geometric axis, sharing a face with the current node, and preferably, a placeholder for a child node of the sibling node; or

[0031] A set of four sibling nodes located in a plane directly following the current node along the geometric axis, wherein the plane is perpendicular to the geometric axis under consideration.

[0032] The geometric axis can be one of the X, Y, and Z axes. The at least one or more sibling nodes can include a single sibling node. Therefore, considering the X-axis, the single sibling node is arranged along the X-axis directly before the current node. Alternatively, the at least one or more sibling nodes can include a group of four sibling nodes arranged in a plane perpendicular to the X-axis (i.e., the YZ plane). The group of four sibling nodes is arranged along the X-axis directly after the current node.

[0033] Preferably, the planar context information is determined along all three geometric axes. Therefore, the plane or surface represented by points in the point cloud is evaluated along each axis.

[0034] Preferably, for a given geometric axis, if the current node is located on the lower plane of the parent node, the planar position context information is determined based on the occupancy of a group of child nodes of the adjacent node and the occupancy of a group of four sibling nodes located in the upper plane of the parent node, wherein each child node shares a face with the current node.

[0035] In this octree structure, the parent node of the current node defines two planes perpendicular to a specific axis. Each plane consists of four nodes. Along the specific axis, the lower plane is arranged first, followed by the higher plane. For example, along the X-axis, the first four nodes of the parent node in the YZ plane form the lower plane, while the next four nodes of the same parent node form the higher plane. This order of the lower and higher planes is also the encoding order. Furthermore, the placeholders of a set of child nodes of an adjacent node at depth D+1 in the tree structure are used to determine the plane position context information. Each child node in the considered set of child nodes of the adjacent node shares a common surface with the current node. Additionally, since the current node is located on the lower plane, adjacent nodes are arranged directly before the current node along the considered axis. Child nodes do not share the same parent node as the current node. Instead, the set of four sibling nodes shares a common parent node with the current node and is arranged on the higher plane of the common parent node. Therefore, the set of four sibling nodes is arranged directly after the current node along the considered axis. To determine the planar positional context information, information about the placeholders of child nodes and sibling nodes is included to improve the prediction of the context model. Therefore, sufficient information can be obtained to provide reliable predictions about the planar positional context information to be encoded.

[0036] Preferably, if at least one child node in the group of child nodes of the adjacent node is occupied, and all sibling nodes in the group of sibling nodes are not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position.

[0037] Preferably, if all child nodes in the group of child nodes of the adjacent node are not occupied, and at least one sibling node in the group of sibling nodes is occupied, then the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a higher plane position.

[0038] Preferably, if all child nodes in the group of child nodes of the adjacent node are not occupied, and all sibling nodes in the group of sibling nodes are not occupied, then the planar position context information is unknown.

[0039] Preferably, if at least one child node in the group of child nodes of the adjacent node is occupied, and at least one sibling node in the group of sibling nodes is occupied, then the planar position context information is unknown.

[0040] Preferably, for cases where the placeholder for an encoded node is unavailable, i.e., when `adjacent_child_contextualization_enabled_flag` is set to zero, if the current node is located in the lower plane, the following applies:

[0041] If the adjacent node is occupied and all sibling nodes in the group of sibling nodes are not occupied, then the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position.

[0042] If the adjacent node is not occupied and at least one of the sibling nodes in the group of sibling nodes is occupied, then the planar position context information includes the possibility that the plane is located in the current node and the possibility that the plane is located in the higher plane position.

[0043] If the adjacent node is not occupied and all sibling nodes in the group of sibling nodes are not occupied, then the planar position context information is unknown; and

[0044] If the adjacent node is occupied and at least one of the sibling nodes in the group of sibling nodes is occupied, then the planar position context information is unknown.

[0045] Therefore, even when placeholders for coded nodes are no longer available due to data limitations, sufficient inferences can still be made regarding the planar positional context. This is achieved by considering placeholders for adjacent nodes, rather than a set of child nodes of adjacent nodes.

[0046] Preferably, for a given geometric axis, if the current node is located in the high plane of its parent node, the planar position context information is determined based on the occupancy of a set of child nodes of a sibling node in the low plane of the parent node and the occupancy of adjacent nodes opposite the current node to a sibling node, wherein each child node shares a surface with the current node. The parent node of the current node defines two planes perpendicular to a specific axis. Each plane consists of four nodes. Along the specific axis, the low plane is arranged first, followed by the high plane. For example, along the X-axis, the first four nodes of the parent node in the YZ plane form the low plane, while the next four nodes of the current parent node form the high plane. This order of low and high planes is also the encoding order. Furthermore, the occupancy of a set of child nodes of a sibling node located at depth D+1 in the tree structure is used to determine the planar position context information. Each child node in the considered set of child nodes of the sibling node shares a common surface with the current node. Additionally, if the current node is located in the high plane, the sibling node is arranged directly ahead of the current node along the considered axis, i.e., in the low plane. Here, child nodes share the same parent node as the current node. Conversely, the adjacent node has a different parent node than the current node (at depth D-1) and is arranged directly behind the current node along the considered axis and relative to the sibling node with respect to the current node. To determine the planar positional context information, information regarding the occupancy of the sibling node's child nodes and adjacent nodes is included to improve the prediction of the context model. Therefore, sufficient information can be obtained to provide reliable predictions about the planar positional context information to be encoded.

[0047] Preferably, if at least one of the child nodes in the group of child nodes of the sibling node is occupied, and the adjacent node is not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position.

[0048] Preferably, if all child nodes in the group of child nodes of the sibling node are not occupied, and the adjacent node is occupied, then the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a higher plane position.

[0049] Preferably, if all child nodes in the group of child nodes of the sibling node are not occupied, and the adjacent node is not occupied, then the planar position context information is unknown.

[0050] Preferably, if at least one child node in the group of child nodes of a sibling node is occupied, and the adjacent node is occupied, then the planar position context information is unknown.

[0051] Preferably, for cases where the placeholder for an encoded node is unavailable, i.e., when `adjacent_child_contextualization_enabled_flag` is set to zero, if the current node is located in the high plane, the following applies:

[0052] If a sibling node is occupied and the adjacent node is not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position.

[0053] If a sibling node is not occupied and an adjacent node is occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a higher plane position.

[0054] If a sibling node is not occupied, and the adjacent node is not occupied, then the planar position context information is unknown; and

[0055] If a sibling node is occupied and an adjacent node is occupied, then the planar position context information is unknown.

[0056] Therefore, even when placeholders for coded nodes are no longer available due to data limitations, sufficient inferences about planar positional context information can still be made through placeholders of the sibling node itself and adjacent nodes. Here, we consider the placeholder of a single sibling node, rather than a group of its child nodes.

[0057] Preferably, if the encoding mode is a planar encoding mode, for at least one geometric axis, preferably, for each geometric axis, the planar position context information includes one or more of the following:

[0058] The inferred plane position determined for the current node is as described above;

[0059] The distance d between the nearest encoded node that is at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure;

[0060] The planar position of the nearest encoded node that is at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure; and

[0061] Indication of the axis orthogonal to the plane.

[0062] Specifically, the inferred planar position is discretized into values ​​of "predicted low," "predicted high," and "unknown." Furthermore, the distance d is discretized into values ​​of "near," "not too far," and "far." If an encoded node exists at the same coordinates and depth D, the planar position, i.e., "high" or "low," can also be included in the context information. The axis perpendicular to the plane can have values ​​of 0, 1, or 2. The coordinates of each node are its relative position within its parent node. Specifically, the planar position context information includes one or more of the above information. Specifically, in a planar coding mode, the planar position context information can be composed of the above information.

[0063] Preferably, the planar position context information is encoded using a binary arithmetic encoder with 54 (=3×3×2×3) context bits. Therefore, compared with existing planar position context models, adjusting the planar position context information can improve encoding efficiency while reducing the amount of information required.

[0064] Preferably, if no most recently encoded node exists in the corresponding parent node at the same coordinates and at the same depth D in the octree-based structure, the planar position context information only includes the planar position of the current node based on the occupancy of at least one sibling node and adjacent nodes. Therefore, in this case, the planar position context information is determined solely based on information collected from adjacent nodes and at least one sibling node. More information is unavailable and therefore unusable. However, since information from adjacent nodes and at least one sibling node is included, efficient compression is still feasible even in this case.

[0065] Preferably, if the encoding mode is an angle encoding mode and the current node is not suitable for the angle encoding mode, then the encoding context information includes planar position context information, wherein the planar position context information only includes:

[0066] As described above, the planar position of the current node is determined by the occupancy of at least one sibling node and adjacent nodes.

[0067] Therefore, in this case, planar position context information of the planar encoding mode is used.

[0068] In one aspect of the invention, a method is provided for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, wherein the geometric structure of the point cloud is characterized by an octree-based structure, the octree-based structure comprising a plurality of nodes having a parent-child relationship by recursively partitioning the volume space containing the point cloud into child volumes, each child volume being associated with a node of the octree-based structure, the method comprising the following steps:

[0069] Entropy decoding of the bitstream is performed based on the context information of the current node, wherein the context information includes planar position context information for at least one geometric axis, and the planar position context information of the current node on the at least one geometric axis is determined based on the occupancy of the adjacent nodes directly adjacent to the current node on the at least one geometric axis and the occupancy of at least one sibling node having the same parent node as the current node.

[0070] The context information includes planar position context information of the current node, indicating the probability of a planar position within the current node, i.e., a prediction of the planar position within the current node. This planar position context information is determined based on the occupancy of neighboring nodes and at least one sibling node. The neighboring node is located at the same depth D in the tree structure as the current node and is directly adjacent to the current node, i.e., sharing a common surface. The neighboring node has a different parent node than the current node (at depth D-1). Furthermore, the planar position context information is determined based on the occupancy of at least one or more sibling nodes. These sibling nodes share a common parent node with the current node and are located at the same depth D in the tree structure. The occupancy indicates whether the corresponding node includes at least one point in the point cloud. The planar position context information includes information about possible planar or surface positions represented by the point cloud of the current node. Information about the occupancy of at least one sibling node and one neighboring node of the current node is considered to determine the context information used for entropy encoding. Therefore, a more reliable prediction of the occupancy of the current child node can be made, which can be used to increase the amount of data reduction through the entropy encoder. In this way, the entire tree is traversed to determine the placeholder for each node and to provide sufficient contextual information for the entropy encoder.

[0071] Preferably, the decoding method is further constructed based on the features described above regarding the encoding method. These features can be freely combined with the decoding method.

[0072] In one aspect of the invention, an encoder is provided for encoding a point cloud to generate a bitstream of compressed point cloud data, wherein the geometry of the point cloud is characterized by an octree-based structure, the octree-based structure comprising a plurality of nodes having parent-child relationships by recursively partitioning the volume space containing the point cloud into child volumes, each child volume being associated with a node of the octree-based structure, the encoder comprising:

[0073] processor, and

[0074] A memory storage device, wherein the memory storage device stores instructions executable by the processor, which, when executed, cause the processor to perform the methods for encoding and decoding as described above.

[0075] In one aspect of the invention, a decoder is provided for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, wherein the geometric structure of the point cloud is characterized by an octree-based structure, the octree-based structure comprising multiple nodes having parent-child relationships by recursively partitioning the volume space containing the point cloud into child volumes, each child volume being associated with a node in the octree-based structure, the decoder comprising:

[0076] processor, and

[0077] A memory storage device storing instructions executable by the processor, which, when executed, cause the processor to perform the aforementioned decoding method.

[0078] In one aspect of the invention, a non-transitory computer-readable storage medium is provided, which stores instructions that are executed by a processor, causing the processor to perform the above-described encoding and / or decoding methods when the instructions are executed by the processor. Attached Figure Description

[0079] Reference will now be made to the accompanying drawings by way of example, in which exemplary embodiments of the present application are illustrated:

[0080] Figure 1 A block diagram showing the overall view of the point cloud encoder.

[0081] Figure 2 A block diagram showing the overall view of the point cloud decoder,

[0082] Figure 3 This is a schematic diagram of an octree data structure.

[0083] Figure 4 Each node's eight child nodes were numbered.

[0084] Figure 5 A diagram illustrating contextual information.

[0085] Figure 6 Specific embodiments of the present invention are shown.

[0086] Figure 7 Specific embodiments of the present invention are shown.

[0087] Figure 8 A flowchart illustrating the steps of the first and second embodiments of the present invention is provided.

[0088] Figure 9 A schematic diagram of the encoder device, and

[0089] Figure 10This is a schematic diagram of a decoder device. Detailed Implementation

[0090] This application describes a method for encoding and decoding point clouds, as well as an encoder and decoder for encoding and decoding point clouds. The current parent node associated with a child body is further segmented into multiple child bodies, each corresponding to a child node of the current parent node. At the encoder, context information of the current node is determined, including planar position context information for at least one geometric axis. This planar position context information of the current node on the at least one geometric axis is determined based on the occupancy of directly adjacent nodes on the at least one geometric axis and the occupancy of at least one sibling node sharing the same parent node on the at least one geometric axis. The entropy encoder encodes the occupancy pattern based on the determined planar context information to produce encoded data for a bitstream. The decoder determines the same context information and performs entropy decoding on the bitstream to reconstruct the occupancy pattern.

[0091] Other aspects and features of this application will be understood by those skilled in the art through a careful reading of the following description of examples in conjunction with the accompanying drawings.

[0092] In the following description, the terms "node" and "sub-volume" may sometimes be used interchangeably. It should be understood that a node is associated with a sub-volume. A node is a specific point in a tree, which can be an internal node or a leaf node. A sub-volume is the bounded physical space represented by a node. The term "volume" can be used to refer to the largest bounded space defined to contain the point cloud. To construct a tree structure of interconnected nodes for encoding point cloud data, a "volume" is recursively divided into sub-volumes.

[0093] A point cloud is a collection of points in a three-dimensional coordinate system. These points are typically intended to represent the outer surface of one or more objects. Each point has a location in the three-dimensional coordinate system. This location can be represented by three coordinates (X, Y, Z) in a Cartesian coordinate system or any other coordinate system. These points can have other associated properties, such as color, which in some cases can be three-component values, such as R, G, B or Y, Cb, Cr. Depending on the desired application of the point cloud data, other associated properties may include transparency, reflectivity, normal vectors, etc.

[0094] Point clouds can be static or dynamic. For example, a detailed scan or mapping of an object or terrain can be static point cloud data. LiDAR-based environmental scans for machine vision applications can be dynamic point cloud data because the point cloud (at least potentially) changes over time (e.g., with each successive scan "volume"). Therefore, a dynamic point cloud is a time-ordered sequence of point clouds.

[0095] Point cloud data can be used in many applications, including archival research (scanning historical artifacts), mapping, machine vision (e.g., autonomous or semi-autonomous vehicles), and virtual reality systems. Dynamic point cloud data used for applications such as machine vision can be entirely different from static point cloud data used for archival purposes. For example, automotive vision typically involves relatively low-resolution, colorless, and highly dynamic point clouds acquired using LiDAR (or similar) sensors with high acquisition frequencies. These point clouds are not intended for human use or viewing but for machine object detection / classification in decision-making processes. For instance, a typical LiDAR frame contains tens of thousands of sequential points, while high-quality virtual reality applications require millions of points. It is anticipated that, over time, higher resolution data will be needed as computing speeds increase and new applications emerge.

[0096] While point cloud data is useful, the lack of effective and efficient compression—that is, encoding and decoding processing—can hinder adoption and deployment.

[0097] One common mechanism for encoding point cloud data is using a tree-based structure. In a tree-based structure, the bounded 3D volume of the point cloud is recursively partitioned into subvolumes. Nodes in the tree correspond to subvolumes. Whether to further partition into subvolumes can be determined based on the tree's resolution and / or whether a subvolume contains any points. Leaf nodes can have placeholder flags indicating whether their associated subvolume contains points. A split flag can indicate whether a node has child nodes (i.e., whether the current volume has been further partitioned into subvolumes). In some cases, these flags can be entropy-encoded, and in others, predictive coding can be used.

[0098] A commonly used tree structure is the octree. In this structure, the core / child cores are cubes, and each partition of a child core further generates eight child cores / child cubes. Examples of this tree structure are... Figure 3 As shown, it has nodes 30 that can characterize a volume containing a complete point cloud. This volume is divided into eight sub-volumes 32, each sub-volume... Figure 3 A node in the octree is associated with a point. A node containing a point indicates a placeholder node 34 that contains at least one point 35 of the point cloud, while an empty node 36 represents a child body that does not contain points of the point cloud. Figure 3As shown, the placeholder node can be further divided into eight sub-nodes, each of which is associated with a specific parent node 40's child node 38 to determine the placeholder pattern of the parent node 40. Figure 3 As shown, the placeholder pattern of the exemplary parent node 40 can be represented in binary form as "00100000", which indicates the third child node 38 that is occupied. In some implementations, this placeholder pattern is encoded by a binary entropy encoder to generate a bitstream of point cloud data.

[0099] Now for reference Figure 1 The diagram shows a simplified block diagram of a point cloud encoder 10 according to an aspect of this application. The point cloud encoder 10 receives point cloud data 12 and may include a tree building module for generating an octree, the octree representing the geometry of the volume space containing the point cloud and indicating the location or position of points in the point cloud within that geometry.

[0100] The basic processing used to create an octree for encoding point clouds may include:

[0101] 1. In the coordinate system, start with a bounded volume (cube) containing the point cloud;

[0102] 2. Divide the bounded solid into 8 sub-solids (eight sub-cubes);

[0103] 3. For each sub-body, if the sub-body is empty, mark it with 0; or if the sub-body contains at least one point, mark it with 1.

[0104] 4. For all sub-body marked as 1, repeat (2) to split these sub-body until the maximum split depth is reached or there is at most one point in each sub-body.

[0105] The tree can be traversed in a predetermined order (within each partitioned sub-body, width-priority or depth-priority, and depending on the scan pattern / order) to produce a bit sequence representing the placeholder pattern of each node.

[0106] The bit sequence can then be encoded using an entropy encoder 16 to produce a compressed bitstream 14. The entropy encoder 16 can encode the bit sequence using a context model 18, which specifies the probability that the entropy encoder 16 encodes bits based on context determination. After each bit or defined set of bits is encoded, the context model 18 can be adaptively updated.

[0107] Like video coding or image coding, point cloud coding can include predictive operations in which an effort is made to predict the pattern of a sub-volumetric element and the residual of the prediction is encoded, rather than the pattern itself. The prediction can be spatial (depending on previously encoded sub-volumetric elements in the same point cloud) or temporal (depending on previously encoded point clouds in a temporally ordered sequence of point clouds).

[0108] exist Figure 2 A block diagram of an exemplary point cloud decoder 20 corresponding to encoder 10 is shown. Point cloud decoder 20 includes an entropy decoder 22, which uses the same context model 24 as encoder 10. Entropy decoder 22 receives an input bitstream 26 of compressed data and performs entropy decoding on the data to produce an output sequence of decompressed bits. This sequence is then converted into reconstructed point cloud data by a tree reconstructor. The tree reconstructor reconstructs a tree structure 28 based on the decompressed data and knowledge of the scan order in which the tree data is binary. Therefore, the tree reconstructor is able to reconstruct the positions of points in the point cloud.

[0109] Reference Figure 4 This illustrates dividing parent node 112 into eight child nodes 110, each child node 110 being a 2×2×2 cube, with each cube having the same size and its side length being half the side length of the cube associated with parent node 112. Furthermore, Figure 4 Indicates the number used by child node 110 within parent node 112. This will be used in further explanation below. Figure 4 The numbering system shown. Among them, Figure 4 It also indicates the spatial orientation of the parent node 112 in the three-dimensional space represented by the geometric axes X, Y, and Z.

[0110] The occupancy pattern can include planar information about whether a node is occupied and whether points within that node belong to a surface. Typically, the real world is occupied by enclosed surfaces. This is especially true for indoor rooms and urban outdoor scenes. This fact is employed by the entropy encoder and decoder. If a surface, represented by a point cloud, can be detected, the distribution of points on that surface can be predicted, thus determining the occupancy probability of a node belonging to that surface. This can be achieved by defining contextual information used to encode and decode the isPlanar flag in the bitstream. The isPlanar flag is typically a binary value encoded for a node using planar contextual information, representing the general probability that the node belongs to a surface. In addition to the fact that a surface exists within a node, other planar information can be considered, such as planar position contextual information used to encode a planePosition flag indicating the position of a plane within the current child node. The planePosition flag can also be a binary value, with values ​​"high" and "low" indicating the corresponding position within the node. This planar information is used to encode the planar context information into the bitstream using the entropy encoder / decoder, thereby reducing the amount of data in the bitstream. Further planar location information can be correlated with planar location context information to more reliably predict surfaces in the corresponding nodes, thereby more reliably predicting occupancy in the corresponding nodes.

[0111] Furthermore, additional encoding modes using different contextual information exist for different types of point clouds. Therefore, planar encoding modes and angular encoding modes are defined. The angular encoding mode is preferably applied to sparsely distributed point clouds, with the intention of providing angular contextual information to the entropy encoder / decoder. The encoding mode can be indicated in the metadata of the bitstream.

[0112] Therefore, according to the present invention, in the case of a planar encoding mode, when a plane exists in the current node, the context information includes the planar context information and the planar position context information as described above. The planar context includes the possibility that the plane passes through the current node. Furthermore, as described in more detail below, the planar position context information of the current node is determined based on the occupancy of adjacent nodes and at least one sibling node that shares the same parent node as the current node. Additionally, in Figure 5 The planar position context information is described in the text. The planar position context information includes the following:

[0113] a) The inferred planar position in the current node, as described in more detail below;

[0114] b) The distance d between the nearest encoded node that is located at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure;

[0115] c) The planar position of the nearest encoded node at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure; and

[0116] d) Indication of axes orthogonal to the plane.

[0117] Specifically, the planar position context information is discretized into values ​​"predicted low," "predicted high," and "unknown." Furthermore, the distance d is discretized into values ​​"near," "not too far," and "far." If an encoded node already exists at the same coordinates and depth D in the tree structure, its planar position (i.e., "high" or "low") can also be included in the context information. The axis perpendicular to the plane can have values ​​of 0, 1, or 2. The coordinates of each node are its relative position within its parent node.

[0118] Therefore, the plane context information is 1 bit (plane or no plane), and the plane position context information is 3×3×2×3=54 bits of information.

[0119] To determine the prediction of planar position within the context of planar position information, refer to Figure 6 and Figure 7 , Figure 6 and Figure 7 This shows the current parent node 132 in an octree structure with eight nodes. At least one child node is occupied. Furthermore, Figure 6 and Figure 7The diagram shows adjacent nodes 126. As a child node of parent node 132, the context information of the current node 120 should be evaluated and used for entropy encoding. If the current node 120 is located at depth D in the octree structure, then parent node 132 is located at depth D-1, and adjacent node 126 is also located at depth D in the octree structure. Furthermore, the child nodes of the current parent node 132 are represented as sibling nodes sharing the common parent node 132 with the current node 120. Adjacent nodes have a different parent node than the current node. When predicting the context information of the current node 120 at a certain octree level D, the occupancy information of three adjacent neighboring nodes 126 along each of the geometric axes X, Y, and Z in the octree is utilized. (If the neighboring node is before the current node in the decoding order, then even the occupancy of the child nodes of the neighboring node 126 (at level D+1) is known and can be considered. If the neighboring node 126 is after the current node in the decoding order, then at least the occupancy of the neighboring node 126 (at level D) is known and can be used for prediction.) Since these three previous neighboring nodes are adjacent to the current node 120, there is a correlation between the occupancy of these neighboring nodes and the occupancy of the current node 120 to be encoded. This information is readily available but is not currently used in existing encoding and decoding methods. For example, if evaluating the context information of the current node 120 along the X-axis, the directly adjacent neighboring nodes along the X-axis can be considered. Therefore, during decoding, the occupancy of previous nodes is known to the decoder and can be used to determine the context information. If context information is evaluated along the Y-axis, then the neighboring nodes directly adjacent to the current node 120 along the Y-axis can be considered. This information is also known to the decoder and can be easily used to obtain reliable context information for the current node 120.

[0120] Furthermore, the placeholders for sibling nodes 130 are also known because, for the current parent node 132, the placeholders for its child nodes are determined first, and then, in subsequent steps, the context information of the child nodes is determined. Therefore, when decoding the current node 120, the placeholder information of sibling nodes is known on the decoder side. This information is not used in the current encoding and decoding method. Therefore, this invention also uses this information additionally to improve the efficiency of encoding and decoding.

[0121] When considering the correlation between the current node's occupancy and the occupancy of adjacent nodes and at least one sibling node, the position of the current node to be encoded must also be considered. The planar position indicates the plane along the considered axis and the plane perpendicular to that geometric axis. For example, along the X-axis, the lower plane is defined by child nodes 0, 1, 2, and 3, while the higher plane is defined by child nodes 4, 5, 6, and 7. Along the Y-axis, the lower plane is defined by child nodes 0, 1, 4, and 5, and the higher plane is defined by child nodes 2, 3, 6, and 7. Along the Z-axis, the lower plane is defined by child nodes 0, 2, 4, and 6, and the higher plane is defined by child nodes 1, 3, 5, and 7.

[0122] Now for reference Figure 6 , Figure 6 The following configuration is illustrated: how to determine planar position context information, particularly based on the occupancy of neighboring nodes 126 of the current node 120 and the occupancy information of sibling nodes of the current node 120 within the same parent node 132, to determine a prediction of the planar position, which can then be used in entropy coding to compress point cloud data. In this regard, the object of the invention is to consider the occupancy of neighboring nodes 126 directly adjacent to the current child node 120 and the occupancy of sibling nodes, and to incorporate this information into the context information when encoding the current node. Figure 6 This indicates that the current node 120 to be encoded is in a lower position than the current parent node 132.

[0123] Figure 6 (a) illustrates the determination of planar context information along the X-axis. The context information of the current node 120 should be evaluated, particularly its planar position context information. Therefore, the occupancy of a group of four sibling nodes 130 located in the high plane of the current parent node 132, and the occupancy of adjacent nodes 126 directly adjacent to the current node 120, are evaluated. The adjacent nodes 126 are arranged along the X-axis to be evaluated. For example, as... Figure 6 As shown in (a), if the current node 120 is located at position 0 in the current parent node 132, then the neighboring node to be considered is located at position 4 in the directly adjacent parent node, and so on. Furthermore, the neighboring nodes 126 are arranged before the current node 120 in the decoding order, i.e., along the X-axis. Therefore, when decoding the current node 120, all information about the neighboring nodes 126 is already available. Specifically, the placeholder information about the sub-nodes of the neighboring node 126 at depth D+1 in the octree structure is known. Additionally, during the encoding of the information of the current node 120, the information about the neighboring nodes 126 is also known because the positions of each point in the point cloud and therefore the placeholders of each node are entirely known to the encoder.

[0124] Specifically, to determine the context information of the current node 120, the first group of child nodes 128 (as children of the neighboring node 126) and the second group 130 (as children of the current parent node 132, i.e., siblings of the current node 120) of the adjacent node 126 are considered. The first group of four child nodes 128 of the adjacent node 126 are directly adjacent to the current node 120. Therefore, each of the first group of child nodes 128 shares a common surface with the current parent node 120. Since the current node 120 is located in the lower plane of its parent node 132, the first group of considered child nodes 128 of the adjacent node 126 is located in the higher plane of the adjacent node 126. The second group of four sibling nodes 130 of the current parent node 132 are directly opposite the adjacent nodes 128. Therefore, since the current node 120 is located in the lower plane of its parent node 132, the second group of four sibling nodes 130 is located in the higher plane of the common parent node 132. Thus, through these two groups of nodes, reliable determination of context information can be provided, and planar positions can be predicted. The surface passing through the current node 120 in the point cloud is also likely to pass through one of the planes defined by the first group 128 or the second group 130. Therefore, the context information of the current node 120 is strongly dependent on the placeholders of the first group 128 and / or the second group 130.

[0125] Generally, when the current node is located on the lower plane of its parent node, the planar position context information is determined based on the occupancy of a group of child nodes of the adjacent node (where each child node shares a plane with the current node) and the occupancy of a group of four sibling nodes located on the higher plane of the parent node. If at least one child node in the group of child nodes of the adjacent node is occupied, and all sibling nodes in the group of sibling nodes are not occupied, the planar position context information includes the probability of the plane being in the current node and the probability of its lower plane position. If all child nodes in the group of child nodes of the adjacent node are not occupied, and at least one sibling node in the group of sibling nodes is occupied, the planar position context information includes the probability of the plane being in the current node and the probability of its higher plane position. If all child nodes in the group of child nodes of the adjacent node are not occupied, and all sibling nodes in the group of sibling nodes are not occupied, the planar position context information is "unknown". If at least one child node in the group of child nodes of an adjacent node is occupied, and at least one sibling node in the group of sibling nodes is occupied, then the planar position context information is "unknown".

[0126] More specifically, according to Figure 6 (a) If the current node is located on the low plane, then for the evaluation along the X-axis and Figure 4 The numbering system shown:

[0127] • If at least one of the child nodes 4 to 7 of the adjacent node 126 is occupied, and all of the sibling nodes 4 to 7 in the group of sibling nodes 130 are not occupied, then the plane position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in a lower plane position.

[0128] • If child nodes 4 to 7 of adjacent node 126 are not occupied, and at least one of sibling nodes 4 to 7 in the group of sibling nodes 130 is occupied, then the planar position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in a higher plane position.

[0129] • If the child nodes 4 to 7 of the adjacent node 126 are not occupied, and the sibling nodes 4 to 7 in the group of sibling nodes 130 are not occupied, then the planar position context information is "unknown".

[0130] • If at least one of the child nodes 4 to 7 of the adjacent node 126 is occupied, and at least one of the sibling nodes 4 to 7 in the group of sibling nodes 130 is occupied, then the planar position context information is “unknown”.

[0131] Therefore, for encoding, when performing entropy encoding on the current node located at a low position, the probability of the position of the current node in the plane perpendicular to the X-axis is regarded as context information. This information can serve as the planar position context information in both planar encoding and angle encoding modes.

[0132] according to Figure 6 (b) For the evaluation along the Y-axis, in order to determine the planar position context information of the current node 120, if the current node is located on the low plane, the following settings are made:

[0133] • If at least one of the child nodes 2, 3, 6, and 7 of the adjacent node 126 is occupied, and all of the sibling nodes 2, 3, 6, and 7 in the group of sibling nodes 130 are not occupied, then the planar position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in a lower plane position.

[0134] • If the child nodes 2, 3, 6, and 7 of the adjacent node 126 are not occupied, and at least one of the sibling nodes 2, 3, 6, and 7 in the group of sibling nodes 130 is occupied, then the planar position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in the higher plane position.

[0135] • If the child nodes 2, 3, 6, and 7 of the adjacent node 126 are not occupied, and the sibling nodes 2, 3, 6, and 7 in the group of sibling nodes 130 are not occupied, then the planar position context information is "unknown".

[0136] • If at least one of the child nodes 2, 3, 6, and 7 of the adjacent node 126 is occupied, and at least one of the sibling nodes 2, 3, 6, and 7 in the group of sibling nodes 130 is occupied, then the planar position context information is "unknown".

[0137] Therefore, for encoding, when performing entropy encoding on the current node located at a low position, the probability of the position of the current node in the plane perpendicular to the Y-axis is regarded as context information. This information can serve as the planar position context information in both planar encoding and angle encoding modes.

[0138] according to Figure 6 (c) For the evaluation along the Z-axis, in order to determine the planar context information of the current node 120, if the current node is located on the low plane, the following settings are made:

[0139] • If at least one of the child nodes 1, 3, 5, 7 of the adjacent node 126 is occupied, and all sibling nodes 1, 3, 5, 7 of a group of sibling nodes 130 are not occupied, then the plane position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in the lower plane position.

[0140] • If the child nodes 1, 3, 5, and 7 of the adjacent node 126 are not occupied, and at least one of the sibling nodes 1, 3, 5, and 7 in a group of sibling nodes 130 is occupied, then the plane position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in the higher plane position.

[0141] • If the child nodes 1, 3, 5, and 7 of the adjacent node 126 are not occupied, and the sibling nodes 1, 3, 5, and 7 in a group of sibling nodes 130 are not occupied, then the planar position context information is "unknown".

[0142] • If at least one of the child nodes 1, 3, 5, and 7 of the adjacent node 126 is occupied, and at least one of the sibling nodes 1, 3, 5, and 7 in a group of sibling nodes 130 is occupied, then the planar position context information is "unknown".

[0143] Therefore, for encoding, when performing entropy encoding on the current node located at a low position, the probability of the position of the current node in the plane perpendicular to the Z-axis is regarded as context information. This information can serve as the planar position context information in both planar encoding and angle encoding modes.

[0144] As shown above, if the current node 120 is located on the high plane of its parent node 132, then... Figure 7As shown, the situation is different. The current node 120 is directly adjacent to the next adjacent node 126 along the considered axis. In this case, on the decoder side, the occupancy of the child nodes of the adjacent node 126 is unknown, and the context information can only be determined using the occupancy of the adjacent node 126 itself. However, the context information of the current node 120 can be determined using the occupancy information of a sibling node 134. The sibling node is arranged directly before the current node 120 on the considered axis. Specifically, the context information of the current node 120 is determined based on the occupancy of the adjacent node 126 and further based on a set of child nodes 136 of a sibling node 134, where each child node in the set of child nodes 136 is directly adjacent to the current node 120, i.e., sharing a common surface with the current node 120. Preferably, the set 136 defines a plane through four child nodes, where the plane is perpendicular to the considered geometric axis and directly adjacent to the current node 120 to be evaluated within the current parent node 132. Furthermore, the plane formed by a group of child nodes 136 of a sibling node 134 is opposite to the adjacent node 126. Since the current node 120 is located on the high plane of the current parent node 132, a sibling node 134 is located on the low plane, while the considered group of child nodes 136 is located on the high plane of a sibling node 134. For example... Figure 7 In the example of the X-axis under consideration shown in (a), if the current node 120 is located at position 4 within the current parent node 132, then the considered sibling node 134 is located at the corresponding position 0 within the same parent node 132. Similarly, the considered adjacent node 126 is located at position 0 within the adjacent parent node (not shown).

[0145] Specifically, if the current node 120 is located in the high plane of its parent node 132, the planar position context information is determined based on the occupancy of a group of child nodes 136 of a sibling node 134 located in the low plane of the parent node 132 (where each child node shares a surface with the current node 120) and the occupancy of the adjacent nodes 126 of the current node 120 relative to a sibling node 134.

[0146] Generally, for the case where the current node 120 is located in the high plane of its parent node 132: If at least one child node of a group of child nodes 136 of a sibling node 134 is occupied and the adjacent node 126 is not occupied, the plane position context information includes the probability that the plane is located in the current node 120 and the probability of its low plane position. If all child nodes of a group of child nodes of a sibling node 134 are not occupied and the adjacent node 126 is occupied, the plane position context information includes the probability that the plane is located in the current node 120 and the probability of its high plane position. If all child nodes of a group of child nodes of a sibling node 134 are not occupied and the adjacent node 126 is not occupied, the plane position context information is unknown. If at least one child node of a group of child nodes of a sibling node 134 is occupied and the adjacent node 126 is occupied, the plane position context information is unknown.

[0147] More specifically, according to Figure 7 (a) If the current node is located on the high plane, then for the evaluation along the X-axis and Figure 4 The numbering system shown:

[0148] • If at least one of the child nodes 4 to 7 of the sibling node 134 is occupied, and the adjacent node 126 is not occupied, the plane position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in the lower plane position.

[0149] • If child nodes 4 to 7 of sibling node 134 are not occupied, and adjacent node 126 is occupied, then the plane position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in the higher plane position.

[0150] • If child nodes 4 to 7 of sibling node 134 are not occupied, and adjacent node 126 is not occupied, then the planar position context information is "unknown".

[0151] • If at least one of the child nodes 4 to 7 of the sibling node 134 is occupied, and the adjacent node is occupied, then the planar position context information is "unknown".

[0152] Therefore, for encoding, when performing entropy encoding on the current node located at a high position, the probability of the position of the current node in the plane perpendicular to the X-axis is regarded as context information. This information can be used as the planar position context information in both planar encoding and angle encoding modes.

[0153] according to Figure 7 (b) For the evaluation along the Y-axis, in order to determine the planar position context information of the current node 120, if the current node is located on the high plane, the following settings are made:

[0154] • If at least one of the child nodes 2, 3, 6, and 7 of the sibling node 134 is occupied, and the adjacent node 126 is not occupied, then the plane position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in the lower plane position.

[0155] • If the child nodes 2, 3, 6, and 7 of the sibling node 134 are not occupied, and the adjacent node 126 is occupied, then the plane position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in the higher plane position.

[0156] • If the child nodes 2, 3, 6, and 7 of the sibling node 134 are not occupied, and the adjacent node 126 is not occupied, then the planar position context information is "unknown".

[0157] • If at least one of the child nodes 2, 3, 6, and 7 of the sibling node 134 is occupied, and the adjacent node 126 is occupied, then the planar position context information is "unknown".

[0158] Therefore, for encoding, when performing entropy encoding on the current node located at a high position, the probability of the position of the current node in the plane perpendicular to the Y-axis is regarded as context information. This information can be used as the planar position context information in both planar encoding and angle encoding modes.

[0159] according to Figure 7 (c) For the evaluation along the Z-axis, in order to determine the planar context information of the current node 120, if the current node is located on the high plane, the following settings are made:

[0160] • If at least one of the child nodes 1, 3, 5, and 7 of the sibling node 134 is occupied, and the adjacent node 126 is not occupied, then the plane position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in the lower plane position.

[0161] • If the child nodes 1, 3, 5, and 7 of the sibling node 134 are not occupied, and the adjacent node 126 is occupied, then the plane position context information includes the probability that the plane is located in the current node 120 and the probability that the plane is located in the higher plane position.

[0162] • If the child nodes 1, 3, 5, and 7 of the sibling node 134 are not occupied, and the adjacent node 126 is not occupied, then the planar position context information is "unknown".

[0163] • If at least one of the child nodes 1, 3, 5, and 7 of the sibling node 134 is occupied, and the adjacent node 126 is occupied, then the planar position context information is "unknown".

[0164] Therefore, for encoding, when performing entropy encoding on the current node located at a high position, the probability of the position of the current node in the plane perpendicular to the Z-axis is regarded as context information. This information can serve as the planar position context information in both planar encoding and angle encoding modes.

[0165] In summary, a reliable planar location context information model that can be used for entropy coding is established in all cases to take into account the presence and location of surfaces or planes in each node, and this knowledge is used to effectively predict node occupancy for data compression by the entropy encoder / decoder.

[0166] However, information about encoded nodes may not always be available. In particular, if placeholder information for encoded nodes is removed to reduce memory requirements, the aforementioned inference can no longer be performed. This behavior can be achieved in the current GPCC encoder / decoder by disabling the adjacent child contextualization enabled flag (adjacent_child_contextualization_enabled_flag), i.e., setting this flag to zero. The basis for inference must then be adjusted accordingly. If the current node 120 is in the low plane, the previous information about the placeholders of a set of child nodes of the adjacent node 126 is replaced by the placeholders of the adjacent node 126 itself. Similarly, if the current node 120 is in the high plane, the placeholder information of the child nodes of a sibling node 134 is replaced by the placeholders of the sibling node itself. Then, even if the adjacent_child_contextualization_enabled_flag is disabled, reliable inference can be performed based on the available adaptive placeholder information.

[0167] More specifically, for the case where the current node is in the lower plane: if the adjacent node is occupied and all sibling nodes in a group of sibling nodes are not occupied, the plane position context information includes the probability that the plane is in the current node and the probability that the plane is in the lower plane.

[0168] If the adjacent node is not occupied and at least one sibling node in a group of sibling nodes is occupied, the plane position context information includes the probability that the plane is located in the current node and the probability that the plane is located in the higher plane position.

[0169] If adjacent nodes are not occupied, and all sibling nodes in a group of sibling nodes are not occupied, then the planar position context information is unknown; and

[0170] If an adjacent node is occupied, and at least one sibling node in a group of sibling nodes is occupied, then the planar position context information is unknown.

[0171] Similarly, if the current node is located in the high plane:

[0172] If a sibling node is occupied and an adjacent node is not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a higher plane.

[0173] If a sibling node is not occupied and an adjacent node is occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in the higher plane.

[0174] If a sibling node is not occupied, and adjacent nodes are not occupied, then the planar position context information is unknown; and

[0175] If a sibling node is occupied and an adjacent node is occupied, the planar position context information is unknown.

[0176] Now for reference Figure 8 (a), Figure 8 (a) A method for encoding a point cloud to generate a bitstream of compressed point cloud data is shown, wherein the geometry of the point cloud is characterized by an octree-based structure comprising a plurality of nodes having parent-child relationships by recursively partitioning the volume space containing the point cloud into child volumes, each child volume being associated with a node of the octree-based structure, the method comprising the following steps:

[0177] S110: Determine the context information of the current node, including planar position context information, wherein the planar position context information of the current node is determined based on the occupancy of adjacent nodes directly adjacent to the current node and the occupancy of at least one sibling node having the same parent node as the current node; and

[0178] S120: Entropy encoding is performed on the placeholder pattern of the current node based on the determined context information to generate encoded data for the bit stream.

[0179] Now for reference Figure 8 (b), Figure 8 (b) A method is shown for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, wherein the geometry of the point cloud is characterized by an octree-based structure comprising a plurality of nodes having parent-child relationships by recursively partitioning the volume space containing the point cloud into child volumes, each child volume being associated with a node of the octree-based structure, the method comprising the following steps:

[0180] S210: Perform entropy decoding on the bit stream based on the context information of the current node, wherein the context information of the current node, including planar position context information, is determined based on the occupancy of the adjacent nodes directly adjacent to the current node and the occupancy of at least one sibling node having the same parent node as the current node.

[0181] Therefore, according to the present invention, information about neighboring and sibling nodes is used and evaluated to obtain a contextual information model, which predicts the position of the surface within the current child node more reliably based on available data. This information can be used to reduce the amount of data in compressed point cloud data, wherein the same contextual information model is also used in the decoder to enable decoding of the provided bitstream with high efficiency and accuracy. Thus, a significant data reduction of at least 1% can be achieved relative to previous encoding methods and existing GPCC specifications. However, this value depends on the density of the points, particularly indoor point clouds, such as LiDAR point clouds, which are assumed to be dense point clouds.

[0182] In an embodiment of the encoding or decoding method, adjacent nodes directly before or after the current node along the geometric axis share a surface with the current node.

[0183] In embodiments of the encoding or decoding method, the at least one or more sibling nodes include one of the following:

[0184] A sibling node directly preceding the current node along the geometric axis and sharing a face with the current node, and preferably, a placeholder for a child node of the sibling node; or

[0185] A set of four sibling nodes located in a plane directly following the current node along the geometric axis, wherein the plane is perpendicular to the geometric axis.

[0186] In embodiments of the encoding or decoding method, the planar context information is determined along all three geometric axes.

[0187] In an embodiment of the encoding or decoding method, if the current node is located on the lower plane of the parent node, the plane position context information is determined based on the occupancy of a group of child nodes of the adjacent node and the occupancy of a group of four sibling nodes located in the upper plane of the parent node, wherein each child node shares a plane with the current node.

[0188] In embodiments of the encoding or decoding method,

[0189] If at least one child node in the group of child nodes is occupied, and all sibling nodes in the group of sibling nodes are not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position.

[0190] If all child nodes in the group of child nodes are not occupied, and at least one sibling node in the group of sibling nodes is occupied, then the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a higher plane position.

[0191] If all child nodes in the group of child nodes are not occupied, and all sibling nodes in the group of sibling nodes are not occupied, then the planar position context information is unknown; and

[0192] If at least one child node in the group of child nodes is occupied, and at least one sibling node in the group of sibling nodes is occupied, then the planar position context information is unknown.

[0193] In embodiments of the encoding or decoding method, when the placeholder of an encoded node is unavailable,

[0194] If the adjacent node is occupied and all sibling nodes in the group of sibling nodes are not occupied, then the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position.

[0195] If the adjacent node is not occupied and at least one of the sibling nodes in the group of sibling nodes is occupied, then the planar position context information includes the possibility that the plane is located in the current node and the possibility that the plane is located in the higher plane position.

[0196] If the adjacent node is not occupied and all sibling nodes in the group of sibling nodes are not occupied, then the planar position context information is unknown; and

[0197] If the adjacent node is occupied and at least one of the sibling nodes in the group of sibling nodes is occupied, then the planar position context information is unknown.

[0198] In an embodiment of the encoding or decoding method, if the current node is located on the high plane of the parent node, the plane position context information is determined based on the occupancy of a group of child nodes of a sibling node located in the low plane of the parent node and the occupancy of adjacent nodes relative to the current node relative to the sibling node, wherein each child node shares a plane with the current node.

[0199] In embodiments of the encoding or decoding method,

[0200] If at least one child node in the group of child nodes is occupied, and the adjacent node is not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position.

[0201] If all child nodes in the group of child nodes are not occupied, and the adjacent node is occupied, then the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in the high plane position.

[0202] If all child nodes in the group of child nodes are not occupied, and the adjacent nodes are not occupied, then the planar position context information is unknown; and

[0203] If at least one child node in the group of child nodes is occupied, and the adjacent node is occupied, then the planar position context information is unknown.

[0204] In embodiments of the encoding or decoding method, in cases where encoded nodes are unavailable,

[0205] If a sibling node is occupied and the adjacent node is not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position.

[0206] If a sibling node is not occupied and an adjacent node is occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a higher plane position.

[0207] If a sibling node is not occupied, and the adjacent node is not occupied, then the planar position context information is unknown; and

[0208] If a sibling node is occupied and an adjacent node is occupied, then the planar position context information is unknown.

[0209] In embodiments of the encoding or decoding method,

[0210] If a nearest-nearest encoded node exists at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure, and is planar, then the context information further includes one or more of the following:

[0211] The distance d between the nearest encoded node that is at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure;

[0212] The planar position of the nearest encoded node that is at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure; and

[0213] An axis orthogonal to the plane.

[0214] In embodiments of the encoding or decoding method, the context information includes 54 bits or less.

[0215] In a preferred embodiment, the method for encoding / decoding a point cloud to generate a bitstream of compressed point cloud data is implemented in a LIDAR (Light Detection and Ranging) device. The LIDAR device includes a light transmission module and a sensor module. The light transmission module is configured to scan the environment with a laser, and the sensor module utilizes sensors to measure the echo of the laser reflected by objects in the environment. Furthermore, the LIDAR device includes an evaluation module configured to preferably determine a 3D representation of the environment in the point cloud by differences in laser return time and / or differences in the wavelength of the reflected laser. Therefore, the echo can include up to millions of location information points of objects or the environment, resulting in a large point cloud, which increases the need for further processing or evaluation of the point cloud by computing devices. In specific applications, such as autonomous driving, the processing of LIDAR point clouds must be near real-time due to safety requirements. Therefore, efficient and accurate compression of the point cloud data is necessary. Thus, the LIDAR device can include an encoder comprising a processor and a memory storage device. The memory storage device can store computer programs or applications containing instructions that, when executed, cause the processor to perform operations such as those described herein. For example, instructions can encode and output a bitstream encoded according to the methods described herein. Additionally or alternatively, the LIDAR device may include a decoder comprising a processor and a memory storage device. The memory storage device may include a computer program or application containing instructions that, when executed, cause the processor to perform operations as described herein. Thus, the encoder / decoder enables efficient and accurate compression of point cloud data, providing the possibility of more efficient and preferably real-time processing of the acquired point cloud data. Preferably, the encoder's processor and the decoder's processor are the same. Preferably, the encoder's memory storage device and the decoder's memory storage device are the same. Preferably, the encoder and / or decoder's processors are also configured to further process or evaluate the point cloud, even more preferably in real-time. Specifically, for an example of autonomous driving, the evaluation of the point cloud may include determining obstacles in the driving direction.

[0216] Now for reference Figure 8The diagram illustrates a simplified block diagram of an example embodiment of encoder 1100. Encoder 1100 includes a processor 1102 and a memory storage device 1104. The memory storage device 1104 may store a computer program or application containing instructions that, when executed, cause processor 1102 to perform operations such as those described herein. For example, the instructions may encode and output a bitstream encoded according to the methods described herein. It should be understood that the instructions may be stored on a non-transitory computer-readable medium, such as a compact disk, flash memory device, random access memory, hard disk drive, etc. When the instructions are executed, processor 1102 performs the operations and functions specified in the instructions to function as a dedicated processor for implementing the processes described above. In some examples, such a processor may be referred to as a "processor circuit" or "processor loop".

[0217] Still referencing Figure 9 The diagram illustrates a simplified block diagram of an example embodiment of decoder 1200. Decoder 1200 includes processor 1202 and memory storage device 1204. Memory storage device 1204 may include a computer program or application containing instructions that, when executed, cause processor 1202 to perform operations such as those described herein. It should be understood that the instructions may be stored on a computer-readable medium, such as a compact disk, flash memory device, random access memory, hard disk drive, etc. When the instructions are executed, processor 1202 performs the operations and functions specified in the instructions to function as a dedicated processor for implementing the processes and methods described above. In some examples, such a processor may be referred to as a "processor circuit" or "processor loop".

[0218] It should be understood that the decoder and / or encoder according to this application can be implemented in multiple computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. The decoder or encoder can be implemented by software containing instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transitory computer-readable storage medium, including CD, RAM, ROM, flash memory, etc.

[0219] It should be understood that the decoders and / or encoders described herein, as well as the modules, routines, processes, threads, or other software components implementing the described methods / processes for configuring the encoders or decoders, can be implemented using standard computer programming techniques and languages. This application is not limited to specific processors, computer languages, computer programming protocols, data structures, or other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as part of computer-executable code stored in volatile or non-volatile memory, as part of an application-specific integrated circuit (ASIC), etc.

[0220] This application also provides computer-readable signals encoded by an application of encoding processing according to this application.

[0221] Certain adjustments and modifications can be made to the described embodiments. Therefore, the embodiments discussed above are considered illustrative rather than restrictive. Specifically, the embodiments can be freely combined with each other.

Claims

1. A method for encoding point clouds to generate a bitstream of compressed point cloud data, wherein, The geometric structure of the point cloud is represented by an octree-based structure, which includes multiple nodes with parent-child relationships by recursively dividing the volume space containing the point cloud into child volumes. Each child volume is associated with a node in the octree-based structure. The method includes the following steps: Determine the context information of the current node, which includes planar position context information for at least one geometric axis. This planar position context information is used to encode a planar position marker indicating the position of the current node within a plane. The planar position context information of the current node on the at least one geometric axis is determined based on the occupancy of adjacent nodes directly adjacent to the current node on that axis and the occupancy of at least one sibling node sharing the same parent node as the current node. Entropy encoding is performed on the placeholder pattern of the current node based on determined context information to generate encoded data of the bit stream, wherein the placeholder pattern is characterized by a bit sequence indicating the distribution of placeholder child nodes within the corresponding parent node. Wherein, if the current node is located in the lower plane of the parent node, the plane position context information is determined based on the occupancy of a group of child nodes of the adjacent node and the occupancy of a group of four sibling nodes located in the upper plane of the parent node, wherein each child node shares a plane with the current node; If the current node is located in the high plane of the parent node, the plane position context information is determined based on the occupancy of a group of child nodes of a sibling node in the low plane of the parent node and the occupancy of adjacent nodes relative to the current node and the sibling node, wherein each child node shares a plane with the current node. The low plane of the parent node is defined by the first group of four sibling nodes of the current node, and the high plane of the parent node is defined by the second group of four sibling nodes of the current node. Along the geometric axis, the first group of four sibling nodes is located in front of the second group of four sibling nodes.

2. A method for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, wherein, The geometric structure of the point cloud is represented by an octree-based structure, which includes multiple nodes with parent-child relationships by recursively dividing the volume space containing the point cloud into child volumes. Each child volume is associated with a node in the octree-based structure. The method includes the following steps: Entropy decoding is performed on the bitstream based on the context information of the current node, wherein the context information includes planar position context information for at least one geometric axis, the planar position context information is used to decode planar position markers, the planar position markers indicate the position of the current node in the plane, and the planar position context information of the current node on the at least one geometric axis is determined based on the occupancy of the adjacent nodes directly adjacent to the current node on the at least one geometric axis and the occupancy of at least one sibling node having the same parent node as the current node; Wherein, if the current node is located in the lower plane of the parent node, the plane position context information is determined based on the occupancy of a group of child nodes of the adjacent node and the occupancy of a group of four sibling nodes located in the upper plane of the parent node, wherein each child node shares a plane with the current node; If the current node is located in the high plane of the parent node, the plane position context information is determined based on the occupancy of a group of child nodes of a sibling node in the low plane of the parent node and the occupancy of adjacent nodes relative to the current node and the sibling node, wherein each child node shares a plane with the current node. The low plane of the parent node is defined by the first group of four sibling nodes of the current node, and the high plane of the parent node is defined by the second group of four sibling nodes of the current node. Along the geometric axis, the first group of four sibling nodes is located in front of the second group of four sibling nodes.

3. The method according to claim 1 or 2, wherein, The adjacent nodes directly before or after the current node along the geometric axis share a surface with the current node.

4. The method according to claim 1 or 2, wherein, The at least one sibling node includes one of the following: A sibling node that is directly preceding the current node along the geometric axis and shares a face with the current node; or A group of four sibling nodes in a plane directly following the current node along the geometric axis, wherein the plane is perpendicular to the geometric axis.

5. The method according to claim 4, wherein, The placeholder for at least one sibling node that has the same parent node as the current node includes: the placeholder for a child node of a sibling node that is directly ahead of the current node along the geometric axis and shares a face with the current node.

6. The method according to claim 1 or 2, wherein, The context information is determined along all three geometric axes.

7. The method according to claim 1 or 2, wherein, When the current node is located in the lower plane of the parent node, If at least one child node in the group of child nodes of the adjacent node is occupied, and all sibling nodes in the group of sibling nodes are not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position. If all child nodes in the group of child nodes of the adjacent node are not occupied, and at least one sibling node in the group of sibling nodes is occupied, then the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a higher plane position. If all child nodes in the group of child nodes of the adjacent node are not occupied, and all sibling nodes in the group of sibling nodes are not occupied, then the planar position context information cannot be inferred. as well as If at least one child node in the group of child nodes of the adjacent node is occupied, and at least one sibling node in the group of sibling nodes is occupied, then the planar position context information cannot be inferred.

8. The method according to claim 1 or 2, wherein, In the case where the current node is located in the lower plane of the parent node and the placeholder mode for the encoded node is unavailable, If the adjacent node is occupied and all sibling nodes in the group of sibling nodes are not occupied, then the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position. If the adjacent node is not occupied and at least one of the sibling nodes in the group of sibling nodes is occupied, then the planar position context information includes the possibility that the plane is located in the current node and the possibility that the plane is located in the higher plane position. If the adjacent node is not occupied and all sibling nodes in the group of sibling nodes are not occupied, then the planar position context information cannot be inferred. as well as If the adjacent node is occupied and at least one of the sibling nodes in the group of sibling nodes is occupied, then the planar position context information cannot be inferred.

9. The method according to claim 1 or 2, wherein, For the case where the current node is located in the high plane of the parent node. If at least one of the child nodes in the group of child nodes of a sibling node is occupied, and the adjacent node is not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position. If all child nodes in the group of child nodes of a sibling node are not occupied, and the adjacent node is occupied, then the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a higher plane position. If all child nodes in the group of child nodes of a sibling node are not occupied, and the adjacent nodes are not occupied, then the planar position context information cannot be inferred. as well as If at least one child node in the group of child nodes of a sibling node is occupied, and the adjacent node is occupied, then the planar position context information cannot be inferred.

10. The method according to claim 1 or 2, wherein, The case where the current node is located in the high plane of the parent node and the placeholder mode for the encoded node is unavailable. If a sibling node is occupied and the adjacent node is not occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a lower plane position. If a sibling node is not occupied and an adjacent node is occupied, the planar position context information includes the probability that the plane is located in the current node and the probability that the plane is located in a higher plane position. If a sibling node is not occupied and an adjacent node is not occupied, the planar position context information cannot be inferred. as well as If a sibling node is occupied and an adjacent node is occupied, the planar position context information cannot be inferred.

11. The method according to claim 1 or 2, wherein, If the mode used to encode the point cloud is a planar encoding mode, and the most recently encoded node exists at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure, and is planar, then the context information further includes one or more of the following: The distance d between the nearest encoded node that is at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure; The planar position of the nearest encoded node that is located at the same coordinates in the corresponding parent node and at the same depth D in the octree-based structure; as well as An axis orthogonal to the plane.

12. The method according to claim 1 or 2, wherein, The planar position context information includes 54 bits or less.

13. An encoder for encoding point clouds to generate a bitstream of compressed point cloud data, wherein, The geometric structure of the point cloud is represented by an octree-based structure, which includes multiple nodes with parent-child relationships by recursively dividing the volume space containing the point cloud into child volumes. Each child volume is associated with a node in the octree-based structure. The encoder includes: Processor; and A memory storage device, wherein instructions executable by the processor are stored in the memory storage device, the instructions, when executed, cause the processor to perform the method according to claim 1 and any one of claims 3 to 12 which are dependent on claim 1.

14. A decoder for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud, wherein, The geometric structure of the point cloud is represented by an octree-based structure, which includes multiple nodes with parent-child relationships obtained by recursively partitioning the volume space containing the point cloud into child volumes. Each child volume is associated with a node in the octree-based structure. The decoder includes: Processor; and A memory storage device, wherein instructions executable by the processor are stored in the memory storage device, the instructions, when executed, cause the processor to perform the method according to claim 2 and any one of claims 3 to 12.

15. A non-transitory computer-readable storage medium storing processor-executable instructions, which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 12.

Images