Angular prior and direct encoding mode for tree representation encoding of point clouds

Abstract

A method of encoding or decoding a point cloud for representing three-dimensional positions of objects is disclosed, the point cloud being generated by a device comprising a plurality of beam emitters. The method comprises using information from the beam emitters, such as the position and angle of each beam emitter, to more efficiently represent the point cloud data.

Description

TECHNICAL FIELD

[0001] The present application relates generally to point cloud compression, and in particular to methods and apparatus for improved compression of occupancy data in tree-based encoding of point clouds and point coordinate data. BACKGROUND

[0002] Data compression is used in communications and computer networks to efficiently store, transmit, and reproduce information. There is increasing interest in representations of three-dimensional objects or spaces, which can involve large data sets, and efficient and effective compression would become very useful and valuable. In some cases, a point cloud can be used to represent a three-dimensional object or space, which is a collection of points each having a three-dimensional coordinate position (X, Y, Z) and in some cases other attributes such as color data (e.g., luminance and chrominance), transparency, reflectivity, normal vector, etc. A point cloud can be static (a fixed object, or a snapshot of an environment / object at a single point in time) or dynamic (a time-ordered sequence of point clouds).

[0003] Example applications of point clouds include terrain and mapping applications. Autonomous driving vehicles and other machine vision applications can rely on point cloud sensor data of an environment in the form of 3D scans (such as from a LiDAR scanner). Virtual reality simulations can rely on point clouds.

[0004] It can be appreciated that point clouds can involve large amounts of data, and it is important to compress (encode and decode) the data quickly and accurately. Therefore, it would be advantageous to provide methods and apparatus that more efficiently and / or effectively compress point cloud data. Such methods can save storage requirements (memory) or save bandwidth to transmit the compressed data, thereby improving the operation of 3D vision systems (such as for automotive applications), or improving the speed and rendering of virtual reality systems, for example. BRIEF DESCRIPTION OF DRAWINGS

[0005] Reference will now be made to the drawings, in which example embodiments of the present application are illustrated and described. In this regard, the present application is illustrated and described in the context of the following drawings:

[0006] Figure 1 A simplified block diagram of an example point cloud encoder is shown;

[0007] Figure 2 A simplified block diagram of an example point cloud decoder is shown;

[0008] Figure 3 and Figure 4 An example of a volume that exhibits planarity in its occupied child sub-volumes is illustrated;

[0009] Figure 5An example method of encoding point cloud data using planar encoding modes is shown in the form of a flowchart;

[0010] Figure 6 A simplified illustration of an example of a beam assembly for acquiring a point cloud is shown;

[0011] Figure 7 A simplified illustration of an example of a single beam emitter in a beam assembly is shown;

[0012] Figure 8 A graph showing points of an example point cloud represented in the (r, 0) plane is shown;

[0013] Figure 9 An example assignment of coordinate axes relative to points of a point cloud is shown;

[0014] Figure 10 An example of the arrangement of beam emitters of a beam assembly relative to a reference position of the beam assembly is shown;

[0015] Figure 11 Details of the graph in Figure 8 are shown;

[0016] Figure 12 A graph showing points of an example point cloud represented in the (r, 0) plane after a correction to the angle 0 has been performed based on characteristics of the beam assembly is shown;

[0017] Figure 13 An example of the angular size of a point cloud data volume relative to the angular difference between beams emitted by a beam assembly is shown;

[0018] Figure 14 An example of a group of sub-volumes of a volume of point cloud data relative to the angle of a beam emitter is shown;

[0019] Figure 15 A method of decoding point cloud data according to at least one embodiment of the present disclosure is shown in the form of a flowchart;

[0020] Figure 16 An angle based on an interval on the z-axis is illustrated;

[0021] Figure 17 An angle based on an updated interval on the z-axis is illustrated;

[0022] Figure 18 A method of encoding point cloud data according to at least one embodiment of the present disclosure is shown in the form of a flowchart;

[0023] Figure 19 A situation in which a lidar head is not aligned with the z-axis is illustrated;

[0024] Figure 20 a prediction point tree for representing a point cloud is illustrated;

[0025] Figure 21 A method of decoding point cloud data according to at least one embodiment of the present disclosure is shown in the form of a flowchart;

[0026] Figure 22 A method of encoding point cloud data according to at least one embodiment of the present disclosure is shown in the form of a flowchart;

[0027] Figure 23 An example simplified block diagram of an encoder is shown; and

[0028] Figure 24 An example simplified block diagram of a decoder is shown.

[0029] Similar reference characters can have been used in different drawings to indicate similar components. DETAILED DESCRIPTION

[0030] The present application describes methods of encoding and decoding point clouds, and encoders and decoders for encoding and decoding point clouds.

[0031] In one aspect, a method of encoding a point cloud to generate a bitstream of compressed point cloud data is provided, the compressed point cloud data representing three-dimensional positions of objects, the point cloud being represented by a tree and being generated by a plurality of beam emitters, the method comprising: a) encoding first and second coordinates of a point belonging to a current node of the tree and computing a radius from the first and second encoded coordinates; b) determining a beam angle of a probe beam emitter among the plurality of beam emitters that is assumed to have acquired the point; c) initializing a range of values to all possible values of a third coordinate of the point; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range of values and the upper value is selected from an upper half of the range of values; e) computing a lower limit angle based on the lower value and the radius and computing an upper limit angle based on the upper value and the radius; f) selecting an angular context based on the lower limit angle, the upper limit angle and the beam angle; and g) entropy encoding information representing the third coordinate of the point based on the angular context to generate the bitstream of compressed point cloud data.

[0032] In another aspect, a method of decoding a bitstream of compressed point cloud data for generating point cloud data is provided, the compressed point cloud data representing three-dimensional positions of objects, the compressed point cloud being represented by a tree and generated by a device comprising a plurality of beam emitters, the method comprising: a) decoding first and second coordinates of a point belonging to a current node of the tree and computing a radius from the first and second decoded coordinates; b) determining a beam angle of a probing beam emitter among the plurality of beam emitters that is assumed to have acquired the point; c) initializing a range of values to all possible values of a third coordinate of points within a volume associated with the current node; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range of values and the upper value is selected from an upper half of the range of values; e) computing a lower limit angle based on the lower value and the radius and an upper limit angle based on the upper value and the radius; f) selecting an angular context from the lower limit angle, the upper limit angle and the beam angle; and g) decoding information from the bitstream representing the third coordinate based on the angular context.

[0033] In yet another aspect, an encoder of encoding a point cloud to generate a bitstream of compressed point cloud data is provided, the compressed point cloud data representing three-dimensional positions of physical objects, the point cloud being generated by a device comprising a plurality of beam emitters, the encoder comprising: a processor; a memory; and an encoding application containing instructions executable by the processor, the instructions, when executed, causing the processor to perform: a) encoding first and second coordinates of a point belonging to a current node of the tree and computing a radius from the first and second encoded coordinates; b) determining a beam angle of a probing beam emitter among the plurality of beam emitters that is assumed to have acquired the point; c) initializing a range of values to all possible values of a third coordinate of the point; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range of values and the upper value is selected from an upper half of the range of values; e) computing a lower limit angle based on the lower value and the radius and an upper limit angle based on the upper value and the radius; f) selecting an angular context based on the lower limit angle, the upper limit angle and the beam angle; and g) entropy encoding information representing the third coordinate of the point based on the angular context to generate the bitstream of compressed point cloud data.

[0034] In yet another aspect, a decoder is provided for decoding a bitstream of compressed point cloud data to produce a reconstructed point cloud, the reconstructed point cloud representing three-dimensional positions of physical objects, the point cloud being generated by a device comprising a plurality of beam emitters, the decoder comprising: a processor; a memory; and a decoding application containing instructions executable by the processor, the instructions, when executed, causing the processor to perform: a) decoding first and second coordinates of a point belonging to a current node of a tree and computing a radius from the first and second decoded coordinates; b) determining a beam angle of a probing beam emitter of the plurality of beam emitters that is assumed to have acquired the point; c) initializing a range of values to all possible values of a third coordinate of points within a volume associated with the current node; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range of values and the upper value is selected from an upper half of the range of values; e) computing a lower limit angle based on the lower limit value and the radius and an upper limit angle based on the upper limit value and the radius; f) selecting an angular context based on the lower limit angle, the upper limit angle, and the beam angle; and g) decoding information from the bitstream representing the third coordinate based on the angular context.

[0035] In yet another aspect, a non-transitory processor-readable medium storing processor-executable instructions is provided, the instructions, when executed by a processor, causing the processor to perform: a) encoding first and second coordinates of a point belonging to a current node of a tree and computing a radius from the first and second encoded coordinates; b) determining a beam angle of a probing beam emitter of a plurality of beam emitters that is assumed to have acquired the point; c) initializing a range of values to all possible values of a third coordinate of the point; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range of values and the upper value is selected from an upper half of the range of values; e) computing a lower limit angle based on the lower limit value and the radius and an upper limit angle based on the upper limit value and the radius; f) selecting an angular context based on the lower limit angle, the upper limit angle, and the beam angle; and g) entropy encoding information representing the third coordinate of the point based on the angular context to generate the bitstream of compressed point cloud data.

[0036] In yet another aspect, a non-transitory processor-readable medium storing processor-executable instructions is provided that, when executed by a processor, cause the processor to perform: a) decoding first and second coordinates of a point belonging to a current node of a tree and computing a radius from the first and second decoded coordinates; b) determining a beam angle of a probing beam emitter of a plurality of beam emitters that is assumed to have acquired the point; c) initializing a range of values to all possible values of a third coordinate of the point within a volume associated with the current node; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range of values and the upper value is selected from an upper half of the range of values; e) computing a lower limit angle based on the lower value and the radius and an upper limit angle based on the upper value and the radius; f) selecting an angular context from the lower limit angle, the upper limit angle, and the beam angle; and g) decoding information from a bitstream representing the third coordinate based on the angular context.

[0037] In yet another aspect, a computer-readable signal containing programming instructions is provided that, when executed by a computer, cause the computer to perform: a) encoding first and second coordinates of a point belonging to a current node of a tree and computing a radius from the first and second encoded coordinates; b) determining a beam angle of a probing beam emitter of a plurality of beam emitters that is assumed to have acquired the point; c) initializing a range of values to all possible values of a third coordinate of the point; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range of values and the upper value is selected from an upper half of the range of values; e) computing a lower limit angle based on the lower value and the radius and an upper limit angle based on the upper value and the radius; f) selecting an angular context based on the lower limit angle, the upper limit angle, and the beam angle; and g) entropy encoding information representing the third coordinate of the point based on the angular context to generate a bitstream of compressed point cloud data.

[0038] In yet another aspect, a computer-readable signal containing programming instructions is provided that, when executed by a computer, cause the computer to perform: a) decoding first and second coordinates of a point belonging to a current node of a tree and computing a radius from the first and second decoded coordinates; b) determining a beam angle of a probing beam emitter of a plurality of beam emitters that is assumed to have acquired the point; c) initializing a range of values to all possible values of a third coordinate of the point within a volume associated with the current node; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range of values and the upper value is selected from an upper half of the range of values; e) computing a lower limit angle based on the lower value and the radius and an upper limit angle based on the upper value and the radius; f) selecting an angular context from the lower limit angle, the upper limit angle, and the beam angle; and g) decoding information from a bitstream representing the third coordinate based on the angular context.

[0039] Those of ordinary skill in the art will appreciate other aspects and features of the application upon reading the following detailed description, including the drawings.

[0040] Any feature described with respect to one aspect or embodiment of the application can also be used in one or more other aspects / embodiments. These and other aspects of the application will become apparent from and elucidated with respect to the embodiments described hereinafter.

[0041] Sometimes in the following description, the terms “node”, “volume” and “sub-volume” can be used interchangeably. It will be appreciated that a node is associated with a volume or sub-volume. A node is a particular point on a tree, which can be an internal node or a leaf node. A volume or sub-volume is a bounded physical space represented by a node. The term “volume” can in some cases be used to refer to the largest bounded space defined for containing a point cloud. A volume can be recursively divided into sub-volumes, with the aim of constructing a tree structure of interconnected nodes for encoding data for a point cloud. The tree structure of volumes split into sub-volumes can be referred to as “parent” and “child” relationships, where a sub-volume is a child sub-node or child sub-volume to a parent node or parent volume. Sub-volumes within the same volume can be referred to as sibling nodes or sibling sub-volumes.

[0042] In this application, the term “and / or” is intended to encompass all possible combinations and sub-combinations of the listed elements, including only one of the listed elements, any sub-combination or all of the elements, without necessarily excluding additional elements.

[0043] In this application, the phrase “at least one of … or …” is intended to encompass any one or more of the listed elements, including only one of the listed elements, any sub-combination or all of the elements, without necessarily excluding any additional elements, and without necessarily requiring all of the elements.

[0044] A point cloud is a set of points in a three-dimensional coordinate system. The points are often intended to represent the external space of one or more objects. Each point has a position (location) in the three-dimensional coordinate system. The location can be represented by three coordinates (X, Y, Z), which can be Cartesian or any other coordinate system. The points can have other associated attributes such as color, which in some cases can also be three component values such as R, G, B or Y, Cb, Cr. Other associated attributes can include transparency, reflectivity, normal vector, etc., depending on the desired application for the point cloud data.

[0045] Point clouds can be static or dynamic. For example, a detailed scan or mapping of an object or terrain can be static point cloud data. Laser-radar based scans of an environment for machine vision purposes can be dynamic in that the point cloud (at least potentially) changes over time, e.g., with each successive scan of the volume. Thus, dynamic point clouds are time-ordered sequences of point clouds.

[0046] Point cloud data can be used in many applications, giving some examples, including: preservation (scanning of historical or cultural objects), mapping, machine vision (such as autonomous or semi-autonomous vehicles), and virtual reality systems. Dynamic point cloud data for applications such as machine vision can be quite different from static point cloud data for preservation purposes. For example, automotive vision typically involves relatively small resolution, non-color, high dynamic point clouds obtained through laser-radar (or similar) sensors with high capture frequency. The goal of such point clouds is not for human consumption or viewing, but for machine object detection / classification in a decision process. As an example, a common laser-radar frame contains tens of thousands of points, while high-quality virtual reality applications require several million points. It is foreseeable that as computing speeds increase and new applications are discovered, there can be a demand for higher resolution data over time.

[0047] While point cloud data is useful, a lack of effective and efficient compression (i.e., encoding and decoding processes) can hinder adoption and deployment. A particular challenge in encoding point clouds that does not arise in other data compression (e.g., audio or video) is the encoding of the geometry of the point cloud. Point clouds tend to be sparsely populated, which makes it more challenging to efficiently encode the locations of these points.

[0048] Tree-based structure

[0049] One of the more common mechanisms to encode point cloud data is to use a tree-based structure. In a tree-based structure, the bounding three-dimensional volume of the point cloud is recursively divided into sub-volumes. The nodes of the tree correspond to the sub-volumes. The decision of whether to further divide a sub-volume can be based on the resolution of the tree, and / or whether any points are contained in the sub-volume. Nodes can have an occupancy flag that indicates whether its associated sub-volume contains points. A split flag can signal whether a node has child nodes (i.e., whether the current volume has been further split into sub-volumes). These flags can be entropy encoded in some cases, and can also use predictive encoding in some cases.

[0050] The three-dimensional volume of a point cloud can also be defined by using bounding boxes. The quadtree plus binary tree (QTBT) structure of bounding boxes allows for the partitioning of a point cloud into non-cubic volumes, which can better represent the overall shape of the point cloud or objects within the point cloud. A quadtree (QT) partition splits a volume into four sub-volumes, for example by using two planes to split the volume into four parts. A binary tree (BT) partition splits a volume into two sub-volumes, for example by using one plane to split the volume into two parts.

[0051] A commonly used tree structure is the octree. In this structure, the volumes / sub-volumes are all cubes (or more generally cuboids) and each split of a sub-volume results in another eight sub-volumes / sub-cubes. Another commonly used tree structure is the KD-tree, in which a volume (cube or rectangular cuboid) is recursively split into two parts by a plane that is orthogonal to one of the axes. The octree is a special case of the KD-tree, in which the volume is divided by three planes, each orthogonal to one of the three axes. KD-trees can generally be obtained by recursively applying binary tree (BT) splits.

[0052] The partitioning of a volume does not necessarily split into two sub-volumes (KD-tree) or eight sub-volumes (octree), but can also involve other partitions, including: splitting into non-cuboid shapes or including non-adjacent sub-volumes.

[0053] The present application can refer to octrees for ease of illustration and because they are a popular candidate tree structure for autonomous driving applications, but it will be appreciated that the methods and apparatus described herein can be implemented using other tree structures.

[0054] Reference is now made to Figure 1 which shows a simplified block diagram of a point cloud encoder 10 in accordance with aspects of the present application. The point cloud encoder 10 comprises a tree construction module 12 for receiving point cloud data and producing a tree (in this example, an octree) that represents the geometry of the volume space containing the point cloud and indicates the position or location of points from the point cloud in that geometry.

[0055] In the case of a uniformly partitioned tree structure like an octree, each node can be represented by a sequence of occupancy bits, where each occupancy bit corresponds to one of the sub-volumes in the node and signals whether that sub-volume contains at least one point. Sub-volumes that have been occupied are recursively split until a maximum depth of the tree. This can be named the serialization or binarization of the tree. As Figure 1 shown, in this example, the point cloud encoder 10 comprises a binarizer 14 for binarizing the octree to produce a bitstream of binarized data representing the tree.

[0056] The sequence of bits can then be encoded using an entropy encoder 16 to produce a compressed bitstream. The entropy encoder 16 can encode the sequence of bits using a context model 18 that specifies probabilities for encoding the bits based on context determinations made by the entropy encoder 16. The context model 18 can be adaptively updated after the encoding of each bit or defined set of bits. In some cases, the entropy encoder 16 can be a binary arithmetic encoder. In some implementations, the binary arithmetic encoder can employ context adaptive binary arithmetic coding (CABAC). In some implementations, encoders other than arithmetic encoders can be used.

[0057] In some cases, the entropy encoder 16 can not be a binary encoder, but instead can operate on non-binary data. The output octree data from the tree construction module 12 can not be evaluated in binary form, but instead can be encoded as non-binary data. For example, in the case of an octree, the eight flags (e.g., occupancy flags) within a sub-volume in their scan order can be considered to be a number of 2 8 -1 bits (e.g., an integer with a value between 1 and 255, since a value of 0 is not possible for a split sub-volume, i.e., if it is completely unoccupied, it would not have been split). In some implementations, this value can be encoded by the entropy encoder using a multi-symbol arithmetic encoder. Within a sub-volume (e.g., a cube), the sequence of flags that define this integer can be named a "pattern".

[0058] A convention commonly used in point cloud compression is that an occupancy bit value of 1 signals that the associated node or volume is "occupied," i.e., it contains at least one point, and an occupancy bit value of 0 signals that the associated node or volume is "unoccupied," i.e., it contains no points. More generally, an occupancy bit can have a value that indicates occupied or a value that indicates unoccupied. In the following description, for ease of illustration, example embodiments are described in which the convention of 1 = occupied and 0 = unoccupied is used; however, it will be understood that the present application is not limited to this convention.

[0059] A block diagram of an example point cloud decoder 50 corresponding to the encoder 10 is shown in Figure 2 The point cloud decoder 50 includes an entropy decoder 52 that uses the same context model 54 used by the encoder 10. The entropy decoder 52 receives an input bitstream of compressed data and entropy decodes the data to produce an output sequence of decompressed bits. This sequence is then converted by a tree reconstructor 56 into reconstructed point cloud data. The tree reconstructor 56 reconstructs the tree structure from the decompressed data and knowledge of the scan order in which the tree data was binarized. Thus, the tree reconstructor 56 is able to reconstruct the positions of the points from the point cloud (subject to the resolution of the tree encoding).

[0060] In European Patent Application No. 18305037.6, the present applicant describes methods and apparatus for selecting among available pattern distributions based on some occupancy information from previously encoded nodes in the vicinity of a particular node for use in encoding the occupancy pattern of the particular node. In one example implementation, the occupancy information is obtained from the pattern of occupancy of the parent of the particular node. In another example implementation, the occupancy information is obtained from one or more nodes adjacent to the particular node. The contents of European Patent Application No. 18305037.6 are incorporated herein by reference. This is referred to as determining a "neighbor configuration" and selecting a context (i.e., a pattern distribution) based at least in part on the neighbor configuration.

[0061] In European Patent Application No. 18305415.4, the present applicant describes methods and apparatus for binary entropy encoding occupancy patterns. The contents of European Patent Application No. 18305415.4 are incorporated herein by reference.

[0062] Speculative direct coding mode (IDCM)

[0063] One problem in compressing point cloud data in a tree structure is that it does not necessarily handle isolated points well. Recursive splitting of sub-volumes and positioning of points within the split sub-volumes involves computational burden and time, and recursive splitting of sub-volumes to signal the precise location of the position of one or few isolated points can be expensive in terms of bandwidth / memory storage and computational time and resources. Furthermore, isolated points "pollute" the distribution of patterns, causing many patterns to have only one occupied child, thereby changing the balance of the distribution and penalizing the encoding of other patterns.

[0064] Accordingly, it is proposed to directly encode the position information of isolated points. Direct encoding of the position of a point (e.g., coordinates within a volume or sub-volume) can be referred to as direct coding mode (DCM). Since it is very inefficient to use DCM for all points, one option is to use a dedicated flag to signal for each occupied node whether DCM is used for any point within that node; however, this option can result in too much signaling overhead, thereby leading to worse compression performance.

[0065] Another option is to determine eligibility for using DCM for an occupied node based on occupancy information from other nodes. If the occupied node is eligible for using DCM, a flag is inserted in the bitstream to indicate whether DCM is applied or not. This method can be referred to as speculative direct coding mode (IDCM).

[0066] For trees whose volumes are composed of adjacent cuboids, the eligibility condition for IDCM can be: A) none of the six adjacent cuboids (above, below, front, back, left, right) is occupied; or, in case the occupancy of the adjacent cuboids is not known: B) the number of occupied nodes in the father pattern is equal to one, i.e. the current node is isolated at the father level. In some cases, it can be beneficial to apply condition A) and B) together with condition C), which is: the number of occupied nodes in the grand-parent pattern is equal to one, i.e. the current node is also isolated at the grand-parent level. In addition, the threshold for the number of occupied father or grand-parent nodes can not be one only.

[0067] If a node is eligible for DCM, a flag is encoded to signal whether DCM is applied or not. This flag can be determined by the encoder based on the number of points belonging to the volume attached to the node. If this number is smaller than or equal to a threshold, DCM is activated, otherwise it is not.

[0068] Thus, according to another aspect of the application, the eligibility to use DCM for an occupied node is determined based on occupancy information from other nodes. If the occupied node is eligible to use DCM, a flag is inserted in the bitstream to indicate whether DCM is applied or not.

[0069] When DCM is applied, the encoding of the points can be performed by encoding the number of points using a truncated unary binarizer followed by a binary entropy encoder, but it can also depend on the threshold selected for the eligibility and use other binarization methods and be performed by independently encoding each point with respect to the X, Y and Z coordinates of the volume associated with the node. For example, if the edge size of a cube is 2 D , then each coordinate of each point needs D bits.

[0070] The value of the threshold should be known by the decoder, either implicitly or as an encoding parameter.

[0071] Planar mode

[0072] Certain types of point cloud data tend to have a strong directionality. Non-natural environments exhibit strong directionality in particular, as those tend to have features of uniform surfaces. For example, in the case of a laser radar, the walls of a road and adjacent buildings are generally planar, either horizontally or vertically. In the case of an internal scan within a room, the floor, ceiling and walls are all planar. Laser radars used for the purpose of vehicle vision and similar applications tend to be more low resolution, and also need to be compressed quickly and efficiently.

[0073] Octrees are efficient tree structures because their structure is signaled efficiently, as they are based on a uniform splitting process that splits a cube into eight sub-cubes using three orthogonal planes in all cases. However, octrees using the current signaling process cannot exploit the efficiency gained by recognizing the planar nature of some non-natural environments. KD-trees, however, can tailor the splitting to the directionality of the point cloud better. This makes them more efficient and effective structures for these types of environments. The drawback of KD-trees is that signaling their structure requires significantly more data than octrees. The fact that KD-trees are non-uniform means that some of the techniques used to improve octree compression are not available or would be computationally difficult to implement for KD-trees.

[0074] Therefore, it would be advantageous to have a mechanism for representing non-natural environments using a uniform-splitting based tree structure in a way that improves compression by exploiting horizontal and / or vertical directionality.

[0075] In some cases, point cloud compression can be improved by a planar coding mode. The planar mode is used to indicate that a volume meets certain requirements for planarity in its occupancy. Specifically, a volume is planar if all of its occupied sub-volumes are positioned in or lie in a common plane. The signaling syntax can indicate whether a volume is planar and, if so, the location of the common plane. By exploiting this knowledge of planarity, gains in compression can be realized. Applying a criterion for eligibility for the planar mode and a mechanism for context adaptive coding of the planar mode signaling helps improve compression performance.

[0076] It is generally assumed that planarity is with respect to Cartesian axes that are aligned with the structure of the volume and sub-volumes. That is, a volume is planar if all of its occupied sub-volumes are positioned in a common plane that is orthogonal to one of the axes. By convention, the z-axis is vertical, meaning that (horizontal) planes are orthogonal to the z-axis.

[0077] For eligible nodes, a flag can be added (referred to herein as isPlanar) to signal whether the node is planar. The flag can be entropy coded using a prediction technique. Given the existence of x- and y-plane modes, the syntax can consist of 3 flags: isZPlanar, isXPlanar, and isYplanar.

[0078] In some embodiments, a node is planar if all of its occupied child nodes belong to the same horizontal plane. In the case of octrees, such a plane consists of four nodes, one of which is necessarily occupied. As Figure 3 and Figure 4As shown, the horizontal plane can be located at the bottom of the current node, corresponding to a first value (e.g., 0, as shown in block 300) of the planePosition, or at the top of the current node, corresponding to a second value (e.g., 1, as shown in block 400) of the planePosition.

[0079] Thus, if the current node is planar, an additional bit planePosition is added in the syntax to indicate the plane position. This bit is also entropy coded.

[0080] Knowledge of the planar status of the current node helps to encode the occupancy pattern. For example, if the node is z-planar, the four occupancy bits associated with the child nodes belonging to the empty horizontal plane do not need to be encoded. Similarly, if the current node is not planar, then it can be assumed that each horizontal plane has at least one occupied child node, which knowledge can be used to better compress the bitstream.

[0081] In the case where the planePosition is located at the bottom, various occupancy examples are shown in FIG. 3 as blocks 302, 304, 306, and 308. Similarly, in the case where the planePosition is located at the top, various occupancy examples are shown in FIG. 4 as blocks 402, 404, 406, and 408. Figure 3 Figure 4

[0082] Figure 5 A method of encoding a node in planar mode is shown. Figure 5 The method of FIG. 5 reflects the process of encoding occupancy information for a volume. In this example, the volume is uniformly partitioned into eight sub-volumes, each having an occupancy bit, according to an octree-based encoding. For simplicity, the current example assumes that only one (e.g., horizontal) planar mode is used.

[0083] In operation 502, the encoder evaluates whether the volume qualifies for planar encoding mode. In one example, the qualification can be based on cloud density, which can be evaluated using the average number of occupied child nodes. To improve local adaptation, the qualification can be based on a probability factor tracking. If the planar encoding mode is not qualified, then the occupancy pattern for the volume is encoded without using the planar encoding mode, as shown by operation 504.

[0084] ​​If the planar mode is enabled, then in operation 506 the encoder evaluates whether the volume is planar. If not, then in operation 508 it encodes the planar mode flag (e.g., isPlanar = 0). In operation 510, the encoder then encodes the occupancy pattern based on there being at least one occupied sub-volume per plane. That is, the occupancy pattern is encoded and if the first three bits encoded for any plane (upper or lower) are zero, then the last (fourth) bit for that plane is not encoded and is presumed to be one, since the corresponding sub-volume is necessarily occupied.

[0085] If the planar mode is enabled and the volume is planar, then in operation 512 the planar mode flag (e.g., isPlanar = 1) is encoded. Because the volume is planar, the encoder then also encodes the plane position flag, planePosition. The plane position flag signals whether the occupied sub-volume of the plane is in the upper or lower half of the volume. For example, planePosition = 0 can correspond to the lower half (i.e., lower z-axis position) and planePosition = 1 can correspond to the upper half. The plane position flag is then encoded in operation 514. Then, in operation 516, the occupancy bits are encoded based on the planarity of the volume and the knowledge of the position of the occupied sub-volume. That is, up to four bits are encoded, since four can be presumed to be zero, and the fourth bit can be presumed to be one in the case that the first three encoded are zero.

[0086] Point cloud acquired by a lidar

[0087] Typically, a point cloud acquired by a lidar is acquired by a set of rotating lasers. Figure 6 An example lidar device is shown.

[0088] As Figure 6 shown, a lidar device 601 includes a vertical array of lasers 604 mounted on a rotating head, such that each laser rotates about a head axis 602. Each laser emits a laser beam 606.

[0089] Referring now to Figure 7 , a single laser 704 is shown rotating about a head axis 702 and having an angle Θ with respect to a plane 708. The plane 708 is perpendicular to the head axis 702. Naturally, due to noise and measurement accuracy, the points acquired by the laser will have an acquisition angle very close to Θ.

[0090] Figure 8 A point cloud acquired by a lidar is shown as points on a graph, with the vertical axis representing tan(Θ) or z / r, and the horizontal axis representing r. By definition, as Figure 9As shown, r is the horizontal distance from the point to the lidar head (or generally beam assembly), and the tangent of the vertical angle tan(0) is defined as the ratio of z / r, where z is the vertical height of the point 905 relative to the lidar head.

[0091] It can be clearly observed in the figure that there are some fixed laser angles tan(0). Positive angles indicate that the laser is pointing above the horizontal plane (i.e., towards the sky), while negative angles indicate that the laser is pointing below the horizontal plane (i.e., towards the ground). It is worth noting that the lasers of a lidar are deliberately oriented at different angles - for example, different lasers can intercept different parts of the same tall object, such as a building, a utility pole, a tree, etc.

[0092] From Figure 8 it can be seen that the points of a point cloud can be represented in a “quasi” 2D space. For example, it is theoretically possible to represent in a 2D space (x, y) and a finite set of vertical angles 0 corresponding to the laser beams. Or equivalently, in a two-dimensional polar coordinate space where is the azimuth angle and the vertical angle 0 is a finite set. In other words, it is possible to represent the points in spherical coordinates where 0 can only take a finite number of values. Because of the closed set, each can produce at most one point. This quasi 2D space is theoretical, but in practice, the measurement noise / accuracy and the time integration do not lead to a strictly finite set of possible 0, which is a consequence of the laser mechanical installation. On the contrary, it is observed that the values of 0 are distributed around a finite number of values 0 laser and the “noise” around these values has to be dealt with.

[0093] The above quasi 2D structure mainly depends on the position / direction of the lidar (or generally of the beam assembly (s)) and on the intrinsic characteristics of the lidar. As will be described in more detail below, it can therefore be advantageous to incorporate syntax elements (e.g., parameters, flags) into the bitstream that describe the characteristics of the lidar (or generally of the beam assembly (s)). These syntax elements can indicate at least one of a reference position of the beam assembly, a starting position of a particular beam, or a beam angle of a particular beam. These syntax elements can be incorporated into the bitstream at frame or sequence level. In any case, this data should be as close as possible to the compressed data bitstream to ensure decodability of the bitstream and to minimize the risk of losing data needed for decodability.

[0094] Noticing the above, some embodiments of the present application seek to use the knowledge of the vertical laser angles 0 laser obtained from the lidar characteristics (or characteristics of the related beam assembly) in the efficient representation of the point cloud, so that the compression is improved.

[0095] Correcting the elevation angle

[0096] Let (x) lidar ,y lidar ,z Lidar (x, y, z) represents the position of the lidar head (or a general beam assembly), oriented vertically along the z-axis in the three-dimensional XYZ (volume space) of the point cloud. The tangent of the vertical angle θ of point (x, y, z) is the elevation angle z. Lidar The ratio of the distance r of that point relative to the head. For example, this is shown in Equation 1.

[0097]

[0098] from Figure 11 As can be seen from the chart, this graph shows the relative... Figure 8 A detailed view shows that for points detected by the same beam, the angle tan(θ) is somewhat constant, but not strictly constant. In fact, for each beam, there exists a corresponding envelope in the (r,tan(θ)) graph containing the points acquired by that beam. The width of this envelope (in the tan(θ) direction) clearly depends on the distance r. This envelope width depends on noise and acquisition accuracy, and widens as r decreases. It is noteworthy that the envelopes of adjacent beams do not intersect each other.

[0099] By adding a correction term to the position of each laser relative to the head position, a correction angle θ that exhibits more stable values ​​over a wider range of r can be obtained. corr .

[0100] Figure 10 The diagram schematically illustrates the position of the beam emitter 1004 relative to a reference position of the beam assembly, used to interpret the correction angle θ to obtain the correction angle θ. corr Example method. Specifically, the position of the beam emitter 1004 is determined by coordinates (r) relative to a reference position of the beam assembly. L , z L The offset of 1012 is defined to correct for the beam 1006 from the vertical position z. Lidar +z L The beam is emitted from point 1004, and the distance from the point to the beam emitter 1004 is corrected to rr. L Where L is the beam emitter index. Therefore, the correction position of beam emitter 1004 can be called the starting position of beam 1006 or the beam starting position. Correction angle θ corr It depends on the beam emitter index L, and is provided by Equation 2 below.

[0101]

[0102] For a point that is not too close to the beam assembly, i.e., the distance r is not too small, the angle can be approximated as shown in Equation 3.

[0103]

[0104] Only vertical correction z is needed L And radial correction r L This can be disregarded. Therefore, we can start from the reference position of the beam assembly, based on the fact that (r) L , z L ) or just z L The relative displacement is used to determine the position of the beam origin.

[0105] Figure 12 Vertical correction z is shown L The effect of angle θ. Small figure (a) corresponds to... Figure 11 The smaller image (b) in the image corresponds to... Figure 11 The smaller figure (c) shows the upper envelope in smaller figure (a) and the lower envelope in smaller figure (b), which are related to the uncorrected (i.e., initial) angle θ. The lower envelope in smaller figure (a) and the upper envelope in smaller figure (b) are related to the corrected angle θ. corr Correlation. It can be clearly seen that the correction angle exhibits a generally constant behavior that is essentially independent of the distance r. The angle θ of each beam or beam emitter L... L It is limited to this constant.

[0106] To enable angle θ correction on both the encoder and decoder sides, the following values ​​can be determined by or provided to the encoder and encoded into the bitstream, allowing the decoding process to use the same beam assembly information as the encoding process:

[0107] 1. The reference position of the beam assembly, for example, the position of the lidar head (x Lidar y lidar Z Lidar );

[0108] 2. Relative position of each beam emitter (r) L z L );

[0109] 3. The constant angle θ of each beam emitter L .

[0110] The following will describe in more detail examples of syntax elements used to describe the characteristics of one or more beam components that have been used to obtain the point cloud to be encoded.

[0111] As an example, the reference position of the beam assembly can be provided by the user, and the relative position and correction angle of the beam emitter of the beam assembly can be found through an algorithm. In one example, this process can be performed as follows:

[0112] 1. First, points belonging to the same laser beam L are extracted by analyzing the histogram of uncorrected angles θ.

[0113] 2. Then, the relative position (r corr,L , z L ) of the laser beam L is determined by minimizing the standard deviation on the extracted points of the corrected angles θ L . This can be performed, for example, by a classical minimization algorithm. In the case where only the vertical offset z L is considered, this is a simple minimization on the candidate vertical relative position interval.

[0114] 3. Finally, the constant angle θ L of the laser L is the average of the corrected angles on the extracted points.

[0115] As will be appreciated, if a node corresponds to a volume that can be intersected by multiple laser beams, it will not be clear which laser beam was used to acquire the points within the node. This introduces the notion of a node being eligible for angular encoding.

[0116] In one example, the angular eligibility of a volume (or its associated node) is determined according to the following steps:

[0117] 1. Compute the distance r of the node center from the reference position of the beam assembly.

[0118] 2. Compute Δθ = E / r, where E is the length of the cube edge (or the vertical dimension of the volume of any shape) associated with the node.

[0119] 3. The node is eligible if KΔθ < Δθ min = min | tan(θ Li ) - tan(θ Li+1 ) |, where the minimum is taken over all adjacent beam pairs. This ensures that the angular size of the node is smaller than the angular increment between the two beams whose angles are closest. A factor K is used to provide a safety margin. In at least some embodiments, the value of K is 2.

[0120] Figure 13 Examples of angular eligible volumes and non-angular eligible volumes are shown. That is, the size of volume 1325 and its distance from the beam assembly 1301 is such that it is sampled by multiple beams 1306 from multiple beam emitters 1304. On the other hand, the size of volume 1320 and its distance from the beam assembly is such that it is sampled by at most a single beam 1306.

[0121] Generally, determining the angular eligibility of a volume (or its associated node) can be done by first determining a measure of the angular size of the volume as seen from a reference position of the beam assembly. As mentioned above, the reference position can be the position of the lidar head. The measure of the angular size can relate to the angle at which the volume edge is seen from the reference position, e.g., based on the length of the volume edge and the distance of the volume (reference position) to the beam assembly reference position. Thus, the measure of the angular size can be the angular size Δθ defined above, for example. Then, the measure of the angular size is compared to a measure of the difference angle between adjacent beams of the beam assembly, such as the difference angle Δθ min ) defined above. That is, the difference angle can be the difference angle between the closest adjacent beams of the beam assembly. Finally, based on the comparison result, it is determined whether the volume is angularly eligible. For example, determining the angular eligibility can be based on whether the angular size is less than half of the difference angle (or any other suitable fraction of the difference angle that provides a suitable safety margin).

[0122] Once the angular eligibility is determined, the beam L to which the angularly eligible volume belongs can be determined, e.g., by the following method:

[0123] 1. Determine z and r relative to the beam assembly position (x Lidar ,y Lidar ,z Lidar ) from the position (x, y, z) of the volume reference position (e.g., the volume center);

[0124] 2. Calculate the uncorrected angle tan(θ) = z / r

[0125] and

[0126] 3. Determine which beam L the volume belongs to by finding the beam index L for which |tan(θ) - tan(θ L )| is minimized.

[0127] Once the beam L is determined, the corrected angle θ

[0128]

[0129] or the corrected angle can be calculated by equation 5.

[0130]

[0131] The second equation can be useful in practice as it avoids the calculation of another square root reciprocal.

[0132] Generally, the following method can be followed to determine the corrected angle θ corrInitially, a first angle (e.g., elevation angle) is determined relative to the reference position within the volume and the reference position of the beam assembly. For example, the reference position within the volume could be the center of the volume. Similarly, the reference position of the beam assembly could be its center position. Based on the first elevation angle, a specific beam (or corresponding beam emitter) L is determined, assuming a point within the volume has been acquired (or actually acquired). Next, the starting position of this specific beam (beam initiation position) is determined or acquired. This can be accomplished using any suitable method described above. If the origin position of the specific beam is available, a corrected back elevation angle for a sub-volume or group of sub-volumes (e.g., a plane within the volume) can be determined relative to the origin position of the specific beam.

[0133] Corner encoding mode

[0134] An angle coding mode is introduced, utilizing the characteristics of points acquired by the lidar to obtain compression gain. This is achieved by using the tan(θ) of angle-qualified nodes. corr L) knowledge can be used to enhance or replace the predictive information of planeZPosition discussed above.

[0135] Specifically, corner coding mode uses corner information to select the context for improved compression. About Figure 14 This describes a corner encoding method.

[0136] like Figure 14 As shown, the eligible node 1420 is divided into a top horizontal plane 1430 and a bottom horizontal plane 1440, which respectively include the volumes of the top four child nodes and the bottom four child nodes. Virtual planes 1432 and 1442 are located in the middle of each of the top horizontal plane 1430 and the bottom horizontal plane 1440. The top and bottom virtual planes do not occupy the entire volume of the child nodes, but only occupy the two-dimensional surfaces in the middle of the child nodes.

[0137] Angle θ top,L and θ bottom,L These are defined as the corrected back angles, θ, from the beam 1406 emitted from laser 1404 to the centers of the top and bottom virtual planes relative to the horizontal plane 1408, respectively. L This is the corrected back angle of laser L, assuming points within the qualified node have been acquired (or actually acquired). The corrected back angle is understood as the angle relative to the horizontal plane 140° at the position passing through the beam emitter, and Z is corrected according to the reference position of the beam assembly. L and / or r L In some embodiments, an uncorrected angle may be used; however, this may result in the worst possible compression. An uncorrected angle is understood as the angle relative to the horizontal plane 1408 of the reference position passing through the beam assembly. The context can then be determined based on these angles, as described below.

[0138] The context is determined according to the following conditions:

[0139] 1, Δ top = tan(θ L ) - tan(θ top,L ) and Δ bottom = tan(θ L ) - tan(θ bottom,L ) have the same sign?

[0140] 2, |Δ top | and |Δ bottom | which is greater?

[0141] 3, D = ||Δ top |-|Δ bottom || / r is in which range? Where r is the distance between the laser and the eligible node on the horizontal plane? According to at least one embodiment, four different ranges can be used to select the context, for example, values less than 0.5, values between 0.5 and 1, values between 1 and 1.5 or values greater than 1.5.

[0142] In summary, condition 1 allows two possibilities, condition 2 allows two possibilities and condition 3 allows four possibilities. Thus, the angular context can take one of the 2 x 2 x 4 = 16 possible values.

[0143] The angular context determined as described above can be used alone or in combination with the planar context to select a context for the entropy encoder (or decoder).

[0144] Predicted point tree

[0145] Figure 20 A predicted point tree is illustrated. Each node of the tree is represented by a circle, for example, circles 2001, 2002 or 2003, and is associated with a point of the point cloud. Each arrow 2004 corresponds to a prediction branch of the tree, the arrow pointing to the father node. In this example, the root node of the tree is node 2001 since it has no father node. Node 2002 is an internal node of the tree since it has one father node and at least one child node. Node 2003 is a leaf node since it has no child node.

[0146] The point associated with a node of the predicted point tree belongs to the part of the point cloud contained in the volume 2000. The structure of the predicted point tree is determined by the encoding process in order to obtain the best structure for the compressed point cloud within the volume 2000.

[0147] The (x, y, z) coordinates of a point associated with a node of the prediction point tree are encoded using the prediction point tree encoding structure by first determining prediction coordinates (x p ,y p ,z p ) obtained from at least one point associated with an ancestor node or a sibling node; then subtracting the prediction coordinates from the coordinates to obtain residual coordinates; and finally encoding the residual coordinates. The residual coordinates can be encoded using an entropy encoder.

[0148] The prediction coordinates (x p ,y p ,z p ) can be obtained using a linear combination of the coordinates of the points related to the ancestor or sibling nodes. The coefficients of the linear combination can be selected from a set of possible linear combinations, and the selection index can be signaled. For example, if p0 is the coordinate of a point associated with a father node, p i is the coordinate of a point associated with a grandfather node, and p2 is the coordinate of a point associated with a great-grandfather node, some embodiments can use the following linear combinations.

[0149] P = 0 x p0 + 0 x p1 + 0 x p2 (no prediction)

[0150] P = 1 x p0 + 0 x p1 + 0 x p2 (delta prediction)

[0151] P = 2 x p0 - 1 x p1 + 0 x p2 (linear prediction)

[0152] P = 2 x p0 + 1 x p1 - 1 x p2 (parallelogram prediction)

[0153] The point associated with the root node, here called the root point, cannot be encoded using prediction coordinates, and its coordinates are directly encoded with respect to the volume 2000.

[0154] Golomb Rice encoding, exp-Golomb encoding, or another variable length encoding method can be used as the entropy encoder for encoding the residual coordinates. Alternatively, context-adaptive arithmetic encoding can also be used. The parameters of the entropy encoder can be selected according to the linear combination or the distance between the ancestors, such as the average distance between the father and the grandfather, or the average distance between the grandfather and the great-grandfather.

[0155] The construction of the prediction point tree for a set of points can be performed as follows. In one embodiment, the encoder first selects a point from the set of points to be associated with the root node of the prediction point tree. This single root node is first considered as a leaf node as it has no child nodes. The selected point is removed from the set of points to be processed. Then, the encoder recursively selects a point from the set of points, associates the selected point with a node of the prediction point tree, and removes the selected point from the set of points to be processed. The recursive process ends when the set of points is empty.

[0156] In one variant, the encoder first identifies point clusters and constructs a prediction point tree for each point cluster by taking the point closest to the cluster center as the root point and the other points of the cluster as its child points. Then, the set of root points is considered as a new set of points to be clustered and the process is applied recursively until the set of root points contains only one point. This variant is a bottom-up process that constructs the tree from leaf nodes to root nodes.

[0157] The prediction structure of the prediction point tree is fully determined by the prediction selection index associated with each node and the number of child nodes of each node. Then, the decoder can obtain the point coordinates from the prediction structure and the residual coordinates of each point. Therefore, the bitstream representing the prediction point tree is composed of the number of child nodes of each node, the prediction selection index, and the residual coordinates of each node.

[0158] The scan order of the tree can be performed in a predetermined encoding order. For example, the scan order of the prediction point tree can be performed after a depth-first traversal of the nodes. In another example, the scan order of the prediction point tree can follow a breadth-first traversal of the nodes.

[0159] In one variant, the number of child nodes N of a node is smaller than or equal to a maximum bound. In the case where the maximum bound is known, the number N can be encoded in the bitstream of each node, for example using Golomb Rice encoding. The maximum bound can also be encoded in the bitstream.

[0160] In another variant, the number of points associated with the child nodes of a node is fixed or derived algorithmically from information available to the decoder. In this case, the number of child nodes does not need to be signaled for each node.

[0161] Problem

[0162] The present disclosure seeks to provide a solution for improving the compression of point cloud data, as in terms of measured compression rate and decoding time.

[0163] When using IDCM, the tree is pruned at nodes and the entropy encoding of the points located in the nodes is not performed. Specifically, in the pruned branches, the angular pattern described above cannot be used and no compression advantage is provided.

[0164] When introducing the IDCM activation parameter, it can be observed that the more IDCM is used, the faster the run-time is, but the less the angular mode is used, the less the compression is. In some cases, the angular mode encoding described above provides more than 20% compression gain on the point cloud acquired by the lidar, but this gain is accompanied by a 250% increase in the decoding time. Therefore, there is a need for an improved angular encoding method that maintains the compression gain while improving the run-time.

[0165] In a tree composed of nodes associated with volumes, it can be understood that the complexity is proportional to the product of the number of nodes to process and the complexity of each node. Therefore, the complexity can be reduced by reducing the number of nodes to process when using the angular method, or by simplifying the algorithm that processes the nodes.

[0166] Solution

[0167] In a first solution, a simple version of the angular mode is used with IDCM in order to benefit from most of the tree pruning provided by IDCM, while also benefiting from the compression provided by the angular mode for all points in the point cloud, including those belonging to pruned branches.

[0168] In addition, an additional condition is introduced for IDCM eligibility. Specifically, in the previous method described above, IDCM is only used for nodes containing points that are sufficiently isolated. In this solution, IDCM will be used for nodes that are sufficiently isolated and eligible for angular mode. As mentioned above, a node is eligible for angular mode in general if it can be positively determined which laser acquired the points within the volume associated with the node.

[0169] According to at least one embodiment of the present disclosure, IDCM is used for a node if both conditions are met. In other words, a node that is sufficiently isolated but not eligible for angular mode will not be affected by IDCM.

[0170] Therefore, the frequency of application of IDCM is lower than with other methods, resulting in a slightly higher number of nodes to process. However, this increase in nodes is small when processing very sparse point clouds, which are typically acquired by lidars.

[0171] According to a first embodiment of the invention, it is assumed that the main axis of the lidar device is vertical, and therefore parallel to the z-axis. The more general case will be described below.

[0172] In addition, as can be understood from the discussion above, once IDCM is activated for a node, the laser index L associated with this node and tan(0) of the laser associated with the laser index L are known. In at least some embodiments, the corrected position (x Lidar , y Lidar , z Lidar+z L ) are also known.

[0173] This solution is illustrated in Figure 15 . Specifically, Figure 15 The method shown illustrates the decoding of a point according to at least one embodiment of the present disclosure.

[0174] In operation 1502, the x and y coordinates of the point are decoded. The x and y coordinates can be encoded and decoded in a variety of ways known to those skilled in the art. Generally, the x and y coordinates can be encoded and decoded in accordance with non-angular IDCM.

[0175] In operation 1504, the reciprocal of the azimuth radius is calculated using the following Equation 6 from the x and y coordinates decoded in 1502. Advantageously, this operation only needs to be performed once for each point to be decoded.

[0176]

[0177] Then, in operation 1506, the corrected angle tan(θ corr,L ) is calculated as described above. In some embodiments, this step can be omitted and the value of tan(θ L ) can be used instead, but this should result in less gain in the compression process. In at least one embodiment, tan(θ corr,L ) is calculated as shown in Equation 7:

[0178] tan(θ corr,L ) = tan(θ L ) + z , r inv (7)

[0179] The correction term + z L r inv is opposite to the term applied to the uncorrected angles tan(θ bottom ) and tan(θ top ) to obtain the corrected angles tan(θ bottom,L ) and tan(θ bottom,L ), which will be described below. Because this correction term is fixed, it is more efficient to correct the laser angle once for any given point, due to the knowledge of the coordinates x and y, than to correct the bottom and top angles multiple times.

[0180] In operation 1508, the z interval is initialized. In at least one embodiment, the z interval is initialized to [Z min , z max ], where z min corresponds to the lowest possible value of z within the volume associated with the current node, and z maxcorresponding to the highest possible value of z within the volume associated with the current node.

[0181] In operation 1510, it is determined whether the coordinate z is fully decoded, i.e. whether there are no remaining bits to decode. The stopping criterion can be that the z interval has a minimum length, e.g. its length is equal to 1 or equivalently z min = z max If there are remaining bits to decode, the method continues to the next point. Otherwise, the method proceeds to operation 1512 to determine tan(θ bottom ) and tan(θ top ). The angles θ bottom and θ top are illustrated with respect to Figure 16 In at least one embodiment, the calculations can be performed using the following equations 8 and 9:

[0182] tan(θ bottom ) = z bottom r inv (8)

[0183] tan(θ top ) = z top r inv (9)

[0184] By calculation, the values of z bottom and z top correspond to one quarter through the z interval and three quarters through the z interval respectively.

[0185] From the (tangent) values of θ bottom and θ top , an angular context for entropy decoding can be determined in operation 1514, and the higher order remaining bits of z (i.e. the higher bits that have not yet been decoded) can be decoded in operation 1516 using the angular context for entropy decoding.

[0186] According to at least one embodiment, the context is determined from the 16 different context possibilities as the following combinations:

[0187] 1. Is Δ top = tan(θ corr,L ) - tan(θ top,L ) and Δ bottom = tan(θ corr，L ) - tan(θ bottom,L ) of the same sign?

[0188] 2. Which is larger, |Δ top | and |Δ bottom |?

[0189] 3. D = r inv|| Δ top |-| Δ bottom What range of values for || is used to select the context? According to at least one embodiment, four different ranges can be used to select the context, for example, values less than 0.5, values between 0.5 and 1, values between 1 and 1.5, or values greater than 1.5.

[0190] However, other methods can also be used to select the context, for example, by using a subset of the combinations described above. Once the angular context is determined, it can be used to decode the next bit of z, or it can be combined with other types of context information before decoding the next bit of z.

[0191] Once the next bit of z is decoded, the z interval is updated according to the value of that decoded bit. For example, if the bit indicates that the z coordinate is in the upper half of the z interval (e.g., the bit equals 1), then the new z interval is selected as the upper half of the previous z interval. In general, the z interval is updated to the interval in which the z coordinate is found according to the information decoded so far. With respect to Figure 17 An example of an updated z interval is shown, where the z interval of Figure 16 is updated using the example where the decoded bit equals 1. The boundary z min , z max , and the angle θ bottom , θ top are updated accordingly.

[0192] After the z interval has been updated, the method returns to operation 1510 to determine whether the z coordinate is completely decoded.

[0193] The encoding process is similar to the decoding process shown in Figure 15 and described with respect to Figure 18 .

[0194] In operation 1802, the x and y coordinates of the point are encoded. The x and y coordinates can be encoded and decoded in a variety of ways known to those skilled in the art. In general, the x and y coordinates can be encoded and decoded according to a non-angular IDCM.

[0195] In operation 1804, the inverse of the azimuthal radius is calculated from the x and y coordinates using the following equation 10. Advantageously, this operation only needs to be performed once for each point to be encoded.

[0196]

[0197] In operation 1806, the correction angle tan(θ corr,L ) is calculated as described above. In some embodiments, this step can be omitted and tan(θ LThe value of ) is used instead, but this should result in lower gain during compression. In at least one embodiment, tan(θ) corr,L Calculate as shown in Equation 11.

[0198] tan(θ corr,L )=tan(θ L )+Z , r inv (11)

[0199] In operation 1808, the z-interval is initialized. In at least one embodiment, the z-interval is initialized to [z min , z max ], where z min This corresponds to the lowest possible value of z within the volume associated with the current node, and z max This corresponds to the highest possible value of z within the volume associated with the current node.

[0200] In operation 1810, it is determined whether the coordinate z has been fully encoded, i.e., whether there are no remaining bits to decode. The stopping criterion could be that the z interval has a minimum length, for example, its length is equal to 1 or equivalent to z. min =z max If there are remaining bits to decode, the method continues to the next point. Otherwise, the method proceeds to operation 1812 to determine tan(θ). bottom ) and tan(θ top Angle θ bottom and θ top Compared to Figure 16 This has been explained. In at least one embodiment, the calculations can be performed using the following equations 12 and 13:

[0201] tan(θ bottom )=z bottom r inv (12)

[0202] tan(θ top )=z top r inv (13)

[0203] Through calculation, z bottom and z top The values ​​correspond to passing through one-quarter and three-quarters of the z-interval, respectively.

[0204] According to θ bottom and θ top The (tangent) value, the angular context used for entropy decoding, can be determined in operation 1814, and the higher-order remaining bits of z (i.e. the higher bits that have not yet been encoded) can be encoded in operation 1816 using the angular context for entropy encoding.

[0205] According to at least one embodiment, the context is determined from the 16 different context possibilities as the following combination:

[0206] 1, Δ top = tan(θ corr,L ) - tan(θ top,L ) and Δ bottom = tan(θ corr，L ) - tan(θ bottom,L ) have the same sign?

[0207] 2, |Δ top | and |Δ bottom | which is larger?

[0208] 3, D = r inv ||Δ top |-|Δ bottom || has a value in which range? According to at least one embodiment, four different ranges can be used to select the context, for example, values smaller than 0.5, values between 0.5 and 1, values between 1 and 1.5 or values larger than 1.5.

[0209] However, other methods can also be used to select the context, for example by using a subset of the above combinations. Once the angular context is determined, it can be used to encode the next bit of z, or it can be combined with other types of context information before encoding the next bit of z.

[0210] Once the next bit of z is encoded, the z interval is updated according to the value of the last encoded bit. For example, if the bit indicates that the z coordinate is in the upper half of the z interval (e.g. the bit is equal to 1 ), the new z interval is chosen as the upper half of the previous z interval. In general, the z interval is updated according to the information decoded so far to the interval in which the z coordinate is found. With respect to Figure 17 An example of an updated z interval is shown, where the z interval of Figure 16 is updated by using the example that the decoded bit is equal to 1. The boundary z min , z max and the angles θ bottom , θ top are updated accordingly.

[0211] After the z interval has been updated, the method returns to operation 1810 to determine whether the z coordinate is completely encoded.

[0212] The above described method of encoding and decoding a node of a tree representing a point cloud is applicable when the axis of the laser radar head used to capture the point cloud is vertical. The following method is more generally applicable regardless of the orientation of the laser radar head.

[0213] This situation is illustrated in Figure 19 The laser 1902 on the lidar head 1904 sends a beam 1912 oriented with respect to the axis 1906. The point 1908 with coordinates (x p , y p ) corresponds to the x and y coordinates of the point being decoded or encoded. Once the coordinates (x p , y p ) become known, for example by previous encoding or decoding of the coordinates of this point, the full coordinates (x, y, z) of the point to be encoded or decoded are constrained on the perpendicular line 1910.

[0214] In this embodiment, the angle tan(0 corr,L ) is calculated from the coordinates of the point 1908 and tan0 L and the position of the laser 1902. Specifically, the angle to be calculated is the elevation angle along the z axis when the laser points to the line 1910.

[0215] When rotating, the lidar head 1904 rotates the laser 1902 and the beam 1912 describes an open cone 1914. The perpendicular line 1910 passing through the point 1908 intersects the cone 1914 at a unique point 1916 having an azimuthal coordinate (x p , y p ) with respect to the origin 1901 corresponding to the center of the lidar head 1904. The point 1916 corresponds to a particular laser position (during the rotation of the head) resulting in the intersection of the unique laser beam 1912 with the perpendicular line 1910 to which this point belongs.

[0216] To calculate tan(0 corr,L ), the origin 1901 is moved to the position of emission of the laser 1902, for example by translating z L along the axis 1906 of the head, and each coordinate (x, y, z) is changed to (x', y', z') by 3D translation. In particular, the point 1916 is transformed from (x p , y p , z p ) to (x' p , y' p , z' p ) and the new radius r' can be calculated according to equation 14.

[0217]

[0218] The corrected laser angle can be calculated according to equation 15.

[0219] tan(0 corr,L ) = z' p / r' (15)

[0220] Once this angle is computed, the method of decoding and encoding is as shown in Figure 15 and Figure 18 where the radius used is r' instead of r.

[0221] Angular mode for a prediction point tree

[0222] With reference to Figure 21 , Figure 21 A method of decoding a prediction point tree with angular mode is shown.

[0223] In operation 2102, it is determined whether there are any more nodes to decode. If so, operation 2104 determines the predicted coordinates of x, y, and z (x pred ,y pred ,z pred ). In operation 2106, the residual coordinates of x and y, x res and y res are decoded, and in operation 2108, the predicted coordinates of x and y and the residual coordinates of x and y are combined to obtain the decoded coordinates of x and y, x dec = x pred + x res and y dec = y pred + y res .

[0224] In operation 2110, the decoded x dec and y dec coordinates are used to compute the inverse radius, and in operation 2112, the corrected angle is computed using Equation 11. This requires knowledge of which laser acquired the point to obtain the values θ L and z L . These values can be decoded from special information encoded into the bitstream received by the decoder.

[0225] In operation 2114, the z interval is initialized. According to at least one embodiment, the z interval is initialized based on the predicted z coordinate z pred and the number of bits used to encode the residual coordinate z res . For example, if the predicted z coordinate z pred is 43, and 5 bits are used to encode the residual coordinate, then the residual coordinate z res can take any value between [-15, 16], then the z interval is [43 - 15, 43 + 16] = [28, 59].

[0226] In operation 2116, it is determined whether the residual z coordinate z res is fully decoded, i.e., whether there are no more z resremaining bits to be decoded. The stopping criterion can be that the z interval has a minimum length, e.g. its length is equal to 1. If there are remaining bits to be decoded, in operation 2126, the decoded z dec = z pred + z res， i.e. the decoded z res coordinate is calculated by adding the predicted z pred coordinate value and the decoded z coordinate residual value, and in operation 2126 the method returns to operation 2102. If there are no remaining bits to be decoded, the method proceeds to operation 2118 where the top and bottom corners are determined. The top and bottom corners can be selected at 3 / 4 and 1 / 4 of the z interval, respectively. Thus, in the example above, when the z interval is [28, 59], the top and bottom corners can be calculated based on z coordinates 51 and 35, respectively.

[0227] Once the top and bottom corners are determined, the corner context can be selected in operation 2120. The corner context can be selected as described with respect to the method shown in FIG. 21 A. Alternative methods of selecting the corner context can also be used. Figure 15

[0228] Then, in operation 2122, the highest order remaining bits of the z residual coordinate are decoded, and in operation 2124 the z interval is updated. Specifically, the z interval is updated to the range of possible values of z given the decoded bits. The method then returns to operation 2116.

[0229] A syntax can be encoded in the bitstream that signals the number of bits used to encode the residual z coordinate. As described above, the syntax is decoded and the length of the initial z interval can be derived from the number. The syntax can be encoded using a unary variable length and / or entropy encoder.

[0230] More specifically, the residual z coordinate can be encoded by an exp-Golomb like variable length code, where the prefix indicates the number of bits of the suffix. The suffix itself indicates the value of the residual z res coordinate. In initializing the z interval, the prefix can be decoded and the length of the interval can be derived from the decoded prefix. The center of the interval can be provided by the predicted z coordinate z pred . The suffix is decoded by an iterative process over the z interval.

[0231] For example, if z = 32 and z pred = 43, then Z res = 32 - 43 = -11 is encoded into the bitstream. The residual -11 belongs to the interval [-15, 16], which has 32 = 2 5 ​The value of z can be encoded using 5 bits. Thus, the encoder encodes the prefix 5 using unary code, where each bit of the unary code is encoded by the CABAC entropy encoder. The suffix signal determines which sub-interval the residual belongs to. For example, the first bit 0 narrows the interval to [-15, 0], the second bit 0 narrows the interval to [-15, -8], the third bit 1 narrows the interval to [-11, -8], the fourth bit 0 narrows the interval to [-11, -10], and the fifth bit 0 finally narrows the interval to [-11, -11], which satisfies the stopping criterion of an interval with length equal to 1; and z res The encoded value of z is -11.

[0232] The five bits are encoded by the entropy encoder (e.g., CABAC) using the angle context determined in the method of Figure 21 and Figure 22 .

[0233] In the decoding process discussed above, the second bit of the suffix can be inferred. For example, based on the example above, it can be assumed that the residual value of z does not belong to [-7, 8], because otherwise the number of bits represented by the prefix would be 4 or less. This inference of the second bit can be performed for encoding regardless of the sign of the residual, so the bit is not encoded.

[0234] Because the sign of the residual value is typically unpredictable, if the residual is zero, it is typically signaled by a flag to encode the sign using bypass coding, and then encode the magnitude value |z res -1| of the residual. The encoding process for a residual z res is then:

[0235] 1. If z res is 0, encode a flag f0 to signal;

[0236] 2. If f0 is false, then:

[0237] 2.1 encode the sign of z res using bypass coding;

[0238] 2.2 encode the prefix representing the number of bits using unary code, where each bit of the unary code is encoded by the CABAC entropy encoder; and

[0239] 2.3 encode the remaining bits to signal which sub-interval the residual belongs to.

[0240] Referring to Figure 22 , Figure 22 shows an encoding method corresponding to the decoding method shown in Figure 21 .

[0241] In operation 2202, it is determined whether there are still nodes to encode. If so, operation 2204 determines the predicted coordinates (x pred ,y pred ,z pred ) of x, y and z. In operation 2206, the residual coordinates x res and y res of x and y are calculated, and in operation 2208, the residual coordinates x res and y res of x and y are encoded.

[0242] In operation 2210, the encoded y and y coordinates are used to calculate the inverse radius, and in operation 2212, the correction angle is calculated using equation 11. This requires knowing which laser captured the point to get the values θ L and z L . These values can be decoded from dedicated information encoded into the bitstream received by the decoder.

[0243] In operation 2214, the z interval is initialized. According to at least one embodiment, the z interval is initialized based on the predicted z coordinate z pred and the number of bits used to encode the residual coordinate z res . For example, if the predicted z coordinate z pred is 43, and 5 bits are used to encode the residual coordinate z res , then the residual coordinate z res can take any value between [-15, 16], and the z interval is [43-15, 43+16] = [28, 59].

[0244] In operation 2216, it is determined whether the residual z coordinate has been encoded, i.e. whether there are no z res bits left to encode. The stopping criterion can be that the z interval has a minimum length, e.g. its length is equal to 1. If there are bits left to encode, the method returns to operation 2202. If not, the method proceeds to operation 2218 where the top and bottom angles are determined. The top and bottom angles can be chosen at 3 / 4 and 1 / 4 of the z interval, respectively. Thus, in the example above, when the z interval is [28, 59], the top and bottom angles can be calculated based on the z coordinates 51 and 35, respectively.

[0245] Once the top and bottom angles are determined, the angle context can be selected in operation 2220. The angle context can be selected as described with respect to the method shown in Figure 15 Alternative methods of selecting the angle context are also possible.

[0246] Then, in operation 2222, the highest order remaining bits of the z residual coordinate are encoded, and the z interval is updated in operation 2124. Specifically, given the encoded bits, the z interval is updated to the range of possible values for z. The method then returns to operation 2216.

[0247] Example encoder and decoder

[0248] Reference is now made to Figure 23 which shows a simplified block diagram of an example embodiment of an encoder 2300. The encoder 2300 includes a processor 2302, a memory 2304, and an encoding application 2306. The encoding application 2306 can include a computer program or application stored in the memory 2304 and containing instructions that, when executed, cause the processor 2302 to perform operations such as those described herein. For example, the encoding application 2306 can encode and output a bitstream encoded according to the processes described herein. It will be understood that the encoding application 2306 can be stored on a non-transitory computer readable medium such as a compact disc, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 2302 performs the operations and functions specified in the instructions to function as a specialized processor implementing the described process(es). In some examples, such a processor can be referred to as a “processor circuit” or “processor circuitry”.

[0249] Reference is now made to Figure 24 which shows a simplified block diagram of an example embodiment of a decoder 2400. The decoder 2400 includes a processor 2402, a memory 2404, and a decoding application 2406. The decoding application 2406 can include a computer program or application stored in the memory 2404 and containing instructions that, when executed, cause the processor 2402 to perform operations such as those described herein. It will be understood that the decoding application 2406 can be stored on a non-transitory computer readable medium such as a compact disc, a flash memory device, a random access memory, a hard drive, etc. When the instructions are executed, the processor 2402 performs the operations and functions specified in the instructions to function as a specialized processor implementing the described process(es). In some examples, such a processor can be referred to as a “processor circuit” or “processor circuitry”.

[0250] It will be recognized that decoders and / or encoders according to the present application can be implemented in a number of computing devices, including but not limited to servers, suitably programmed general purpose computers, machine vision systems, and mobile devices. The decoders or encoders can be implemented by software containing instructions for configuring a processor or processors to perform the functions described herein. The software instructions can be stored in any suitable non-transitory computer readable memory including CDs, RAM, ROM, flash memory, etc.

[0251] It will be understood that the decoders and / or encoders described herein, as well as the modules, routines, processes, threads, or other software components implementing the described methods / processes for configuring an encoder or decoder, can be implemented using standard computer programming techniques and languages. The present application is not limited to a particular processor, computer language, computer programming convention, data structure, other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as parts of computer-executable code stored in volatile or non-volatile memory, as parts of application-specific integrated circuits (ASICs), etc.

[0252] The present application also provides a computer-readable signal encoding data produced by the application of the described encoding process according to the present application.

[0253] Certain adjustments and modifications can be made to the described embodiments. Accordingly, the above-discussed embodiments are to be considered illustrative and not restrictive, and the scope of the application is not to be determined by the above description but only by the appended claims.

Claims

1. A method of encoding a point cloud to generate a bitstream of compressed point cloud data, the compressed point cloud data representing three-dimensional positions of objects, the point cloud being represented by a tree and generated by a plurality of beam emitters, the method comprising: a) determining a first coordinate and a second coordinate of a point belonging to a current node of the tree, and computing a radius from the encoded first coordinate and the encoded second coordinate; b) determining a beam angle of a probe beam emitter among the plurality of beam emitters, assumed to have acquired the point; c) initializing a range of values to all possible values of a third coordinate of the point; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range of values and the upper value is selected from an upper half of the range of values; e) computing a lower limit angle based on the lower value and the radius, and computing an upper limit angle based on the upper value and the radius; f) selecting an angular context based on the lower limit angle, the upper limit angle, and the beam angle; and g) entropy encoding information representing the third coordinate of the point based on the angular context to generate the bitstream of compressed point cloud data, and the method further comprising, before step a): determining that the current node is eligible for an IDCM (Inferred Direct Coding Mode) and that the current node is associated with the probe beam emitter.

2. The method of claim 1, further comprising: h) updating the range of values depending on the encoded information representing the third coordinate; wherein steps d) to h) are repeated until the range of values comprises a single possible value.

3. The method of claim 1, wherein the beam angle is corrected to account for a position of the probe beam emitter.

4. The method of claim 3, wherein the position of the probe beam emitter is included in the bitstream.

5. The method of claim 1, wherein the current node is a node of a predicted point tree.

6. The method of claim 5, further comprising: determining predicted coordinates of the point associated with the current node, wherein the first coordinate, the second coordinate, and the third coordinate are residual coordinates of the point with respect to the predicted coordinates.

7. The method of claim 6, wherein the range of values of all possible values of a third coordinate is initialized by: determining a length of the range based on information encoded in the bitstream; and determining a center of the range as the third coordinate of predicted coordinates.

8. The method of claim 1, wherein the lower value is selected to correspond to a quarter of the range and the upper value is selected to correspond to three quarters of the range.

9. The method of claim 1, wherein the beam angle of the probe beam emitter is included in the bitstream.

10. A method of decoding a bitstream of compressed point cloud data for generating point cloud data, the compressed point cloud data representing three-dimensional positions of objects, the compressed point cloud being represented by a tree and generated by a device comprising a plurality of beam emitters, the method comprising: a) determining a first coordinate and a second coordinate of a point belonging to a current node of the tree, and computing a radius from the decoded first coordinate and the decoded second coordinate; b) determining a beam angle of a probe beam emitter of the plurality of beam emitters that is assumed to have acquired the point; c) initializing a range of values to all possible values for a third coordinate of the point; d) selecting a lower value and an upper value within the range of values, wherein the lower value is selected from a lower half of the range and the upper value is selected from an upper half of the range; e) computing a lower limit angle based on the lower value and the radius, and computing an upper limit angle based on the upper value and the radius; f) selecting an angular context based on the lower limit angle, the upper limit angle, and the beam angle; and g) decoding information from the bitstream representing the third coordinate based on the angular context, and the method further comprising, prior to step a): determining that the current node is eligible for an IDCM and the current node is associated with the probe beam emitter.

11. The method of claim 10, further comprising: h) updating the range of values depending on the decoded information representing the third coordinate; wherein steps d) through h) are repeated until the range of values comprises a single possible value.

12. The method of claim 10, wherein the beam angle is corrected to account for a position of the probe beam emitter.

13. The method of claim 12, wherein the position of the probe beam emitter is included in the bitstream.

14. The method of claim 10, wherein the current node is a node of a prediction point tree.

15. The method of claim 14, further comprising: determining a predicted coordinate of the point associated with the current node, wherein the first coordinate, the second coordinate, and the third coordinate are residual coordinates of the point relative to the predicted coordinate.

16. The method of claim 15, wherein the range of values of all possible values for a third coordinate is initialized by: determining a length of the range based on information encoded in the bitstream; and determining a center of the range as the third coordinate of a predicted coordinate.

17. The method of claim 10, wherein the lower value is selected to correspond to a quarter of the range and the upper value is selected to correspond to three quarters of the range.

18. The method of claim 10, wherein the beam angle of the probe beam emitter is included in the bitstream.

19. An encoder for encoding a point cloud to generate a bitstream of compressed point cloud data, the compressed point cloud data representing three-dimensional positions of physical objects, the point cloud generated by a device comprising a plurality of beam emitters, the encoder comprising: a processor; a memory; and an encoding application containing instructions executable by the processor that, when executed, cause the processor to perform the method of any of claims 1-9.

20. A decoder for decoding a bitstream of compressed point cloud data to produce a reconstructed point cloud, the reconstructed point cloud representing three-dimensional positions of physical objects, the point cloud being generated by a device comprising a plurality of beam emitters, the decoder comprising: a processor; a memory; and a decoding application containing instructions executable by the processor, which when executed cause the processor to perform the method of any of claims 10-18.

21. A non-transitory processor-readable medium storing processor-executable instructions which, when executed by a processor, cause the processor to perform the method of any of claims 1-9 or 10-18.

22. A computer program product containing programming instructions which, when executed by a computer, cause the computer to perform the method of any of claims 1-9 or 10-18. ​

Images