Angular mode for tree-based point cloud coding

Abstract

Methods and devices for coding point cloud data using an angular coding mode. The angular coding mode can be signaled using an angular mode flag to signal that a volume is to be coded using the angular coding mode. The angular coding mode is suitable for planar volumes, all of whose occupancy child nodes are on one side of a plane bisecting the volume. A plane position flag can signal which side of the volume is occupied. Entropy coding can be used to code the plane position flag. Context determination for coding can take into account angular information of a child node or group of child nodes of the volume relative to the position of a beam combination that has sampled the point cloud. Characteristics of the beam combination can be coded into the bitstream.

Description

Technical Field

[0001] This application generally relates to point cloud compression, and more specifically to methods and apparatus for improving occupancy data compression in tree-based point cloud encoding and decoding. Background Technology

[0002] Data compression is used in communications and computer networks to efficiently store, transmit, and reproduce information. There is a growing interest in representing three-dimensional objects or spaces, which may involve large datasets, and efficient and effective compression for such representations is highly valuable. In some cases, three-dimensional objects or spaces can be represented using point clouds, which are sets of points each with three coordinate positions (X, Y, Z). In other cases, additional attributes are used, such as color data (e.g., brightness and chromaticity), transparency, reflectivity, normal vectors, etc. Point clouds can be static (a still object or environment / snapshot of the object at a single point in time) or dynamic (a sequence of point clouds in chronological order).

[0003] Example applications of point clouds include terrain and mapping applications. Autonomous vehicles and other machine vision applications may rely on point cloud sensor data in the form of 3D scans of the environment, such as from LiDAR scanners. Virtual reality simulations may also rely on point clouds.

[0004] It should be understood that point clouds can involve large amounts of data, and it is important to compress (encode and decode) this data quickly and accurately. Therefore, it would be advantageous to provide methods and devices for compressing point cloud data more efficiently and / or effectively. As an example, this approach could save on storage requirements (memory) by improving compression, or save on bandwidth for transmitting compressed data, leading to improved operation of 3D vision systems (such as for automotive applications) or, for example, improved operating speed and rendering of virtual reality systems. Attached Figure Description

[0005] The accompanying drawings, by way of example, illustrate exemplary embodiments of this application, wherein:

[0006] Figure 1 A simplified block diagram of an example point cloud encoder is shown;

[0007] Figure 2 A simplified block diagram of an example point cloud decoder is shown;

[0008] Figure 3 and Figure 4 The illustration shows an example of a volume that exhibits planarity within the sub-volume it occupies;

[0009] Figure 5 An example method for encoding point cloud data using a planar encoding / decoding mode is illustrated in the form of a flowchart;

[0010] Figure 6 An example method for decoding point cloud data using a planar encoding / decoding mode is illustrated in the form of a flowchart;

[0011] Figure 7 This shows a portion of an example of the process for encoding and decoding occupancy bits based on planarity information;

[0012] Figure 8 This shows a portion of another example of the process for encoding and decoding occupancy bits based on planarity information;

[0013] Figure 9 The diagram illustrates the possible factors in the context of determining the plane mode flag or plane position flag used for encoding and decoding;

[0014] Figure 10 An example mechanism is shown for tracking the most recently encoded occupied nodes at the same depth and in a common plane;

[0015] Figure 11 A simplified illustration of an example of beam combination used to acquire point clouds is shown;

[0016] Figure 12 A simplified illustration of an example of a single beam transmitter in a beam combination is shown;

[0017] Figure 13 An example assignment of coordinate axes relative to beam combination is shown;

[0018] Figure 14 A graph showing points included in an example point cloud represented in the (r,θ) plane is shown;

[0019] Figure 15 An example assignment of coordinate axes relative to points in the point cloud is shown;

[0020] Figure 16 It shows Figure 14 Details of the diagram;

[0021] Figure 17 An example of the arrangement of the beam transmitters of a beam combination relative to a reference position of the beam combination is shown;

[0022] Figure 18 A graph of points in an example point cloud, including the correction for angle θ, has been performed based on the characteristics of the beam combination and is expressed in the (r,θ) plane.

[0023] Figure 19 An example is shown of the angular magnitude of the volume of point cloud data relative to the angular difference between beams emitted by a beam combination;

[0024] Figure 20 An example method for encoding point cloud data using angle encoding / decoding modes is shown in the form of a flowchart;

[0025] Figure 21 An example method for decoding point cloud data using angle encoding / decoding modes is illustrated in the form of a flowchart;

[0026] Figure 22 An example of the angle of a sub-volume group of point cloud data volume relative to a beam emitter is shown;

[0027] Figure 23 An example of the angle of a subvolume of a point cloud data volume relative to a beam emitter is shown;

[0028] Figure 24 An example arrangement of the main axis of the beam combination is shown;

[0029] Figure 25 It shows Figure 24 An example of the arrangement of the beam transmitters of a beam combination relative to a reference position of the beam combination;

[0030] Figure 26 It shows Figure 24 Another example of the arrangement of the beam transmitter of the beam combination relative to the reference position of the beam combination;

[0031] Figure 27 An example method is shown in the form of a flowchart, which decodes point cloud data from a bit stream based on syntactic elements that indicate the characteristics of the beam combination that has been acquired.

[0032] Figure 28 An example method is illustrated in the form of a flowchart, which encodes point cloud data into a bit stream based on syntactic elements that indicate the characteristics of the beam combination that has been acquired.

[0033] Figure 29 Examples of the shape of point cloud data volumes and the angles of sub-volume groups relative to a beam emitter are shown.

[0034] Figure 30 A simplified block diagram of an encoder is shown; and

[0035] Figure 31 A simplified block diagram of an example decoder is shown.

[0036] Similar reference numerals may have been used in different figures to denote similar components. Detailed Implementation

[0037] This application describes methods for encoding and decoding point clouds, as well as encoders and decoders for encoding and decoding point clouds.

[0038] In one aspect, this application describes a method for encoding a point cloud to generate a bitstream of compressed point cloud data representing the three-dimensional position of an object, the point cloud being located within a volume space recursively partitioned into sub-volumes and containing points of the point cloud, wherein the volume is divided into a first sub-volume set and a second sub-volume set, the first sub-volume set being located in a first plane and the second sub-volume set being located in a second plane parallel to the first plane, and wherein an occupancy bit associated with each corresponding sub-volume indicates whether the corresponding sub-volume contains at least one point. The method may include: determining whether a volume is planar based on whether all sub-volumes containing at least one point are located in the first or second set; encoding a planar mode flag in the bitstream to signal whether a volume is planar; encoding the occupancy bits of the sub-volumes of the first set in the bitstream includes, for at least one occupancy bit, inferring its value based on whether the volume is planar and not encoding it in the bitstream; and outputting the bitstream of compressed point cloud data.

[0039] In another aspect, this application describes a method for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud representing the three-dimensional position of a physical object, the point cloud being located within a volume space recursively divided into sub-volumes and containing points of the point cloud, wherein the volume is divided into a first sub-volume set and a second sub-volume set, the first sub-volume set being located in a first plane and the second sub-volume set being located in a second plane parallel to the first plane, and wherein an occupancy bit associated with each corresponding sub-volume indicates whether the corresponding sub-volume contains at least one point. The method may include: reconstructing the points of the point cloud by reconstructing occupancy bits of a planar mode flag indicating whether a volume is planar from the bitstream, wherein a volume is planar if all sub-volumes containing at least one point are located in either the first or second set; and decoding the occupancy bits of the sub-volumes of the first set from the bitstream includes, for at least one occupancy bit, inferring its value based on whether the volume is planar and not decoding it from the bitstream.

[0040] In some implementations, determining whether a volume is planar may include: determining that the volume is planar by determining that at least one sub-volume in a first set contains at least one point and that no sub-volume in a second set contains any point, and the method may further include: encoding a planar location flag to signal that the at least one sub-volume is in a first plane based on the volume being planar. In this implementation, in some cases, encoding occupancy bits includes avoiding encoding / decoding occupancy bits associated with the second set, and inferring the value of the occupancy bits associated with the second set based on the second set which does not contain points. Encoding occupancy bits may further include: inferring that the last of the occupancy bits associated with the first set in the encoding / decoding order has an indicative value of occupancy based on determining that all other occupancy bits in the first set in the encoding / decoding order have values ​​indicating unoccupancy.

[0041] In some implementations, determining whether a volume is planar includes: determining that the volume is not planar, and on this basis, encoding the occupancy bits based on at least one occupancy bit in a first set and at least one occupancy bit in a second set having values ​​indicating occupancy.

[0042] In some embodiments, the point cloud is defined relative to a Cartesian axis in volumetric space, the Cartesian axis having a z-axis orthogonal to the vertical orientation of the horizontal plane, and the first and second planes are parallel to the horizontal plane. In other embodiments, the first and second planes are orthogonal to the horizontal plane.

[0043] In some implementations, the method includes: first determining that the volume is acceptable for planar mode encoding / decoding. Determining that the volume is acceptable for planar mode encoding / decoding may include: determining a probability of planarity, wherein the probability of planarity is greater than a threshold acceptance value.

[0044] In some implementations, encoding the planar pattern flags may include encoding the horizontal planar pattern flags and encoding the vertical planar pattern flags.

[0045] In another aspect, this application describes a method for encoding a point cloud to generate a bitstream of compressed point cloud data representing the three-dimensional position of an object, the point cloud being located within a volume space recursively divided into sub-volumes and containing points of the point cloud, wherein the volume is divided into a first sub-sub-volume set and a second sub-sub-volume set, the first sub-sub-volume set being located in a first plane and the second sub-sub-volume set being located in a second plane parallel to the first plane, and wherein an occupancy bit associated with each corresponding sub-sub-volume indicates whether the corresponding sub-sub-volume contains at least one point, the first plane and the second plane being orthogonal to an axis. The method may include: determining whether a volume is planar based on whether all sub-volumes containing at least one point are located in a first set or a second set; entropy encoding a planar mode flag in a bit stream to signal whether a volume is planar, wherein the entropy encoding includes determining the context for encoding / decoding the planar mode flag based in part on one or more of the following: (a) whether the parent volume containing the volume is planar in an occupancy, (b) the occupancy of neighboring volumes at the parent depth, the neighboring volumes being adjacent to the volume and having a face shared with the parent volume, or (c) the distance between the volume and the most recently encoded occupancy volume, the occupancy volume having the same depth as the volume and the same position on the axis as the volume; encoding occupancy bits of at least some of the sub-volumes; and outputting a bit stream of compressed point cloud data.

[0046] In another aspect, this application describes a method for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud representing the three-dimensional position of a physical object, the point cloud being located within a volume space recursively divided into sub-volumes and containing points of the point cloud, wherein the volume is divided into a first sub-sub-volume set and a second sub-sub-volume set, the first sub-sub-volume set being positioned in a first plane, and the second sub-sub-volume set being positioned in a second plane parallel to the first plane, and wherein an occupancy bit associated with each corresponding sub-sub-volume indicates whether the corresponding sub-sub-volume contains at least one point, the first plane and the second plane being orthogonal to an axis. The method may include: reconstructing points of point cloud data by reconstructing occupancy bits from a planar mode flag indicating whether a volume is planar, wherein a volume is planar if all sub-volumes containing at least one point are located in a first or second set, wherein entropy decoding includes determining the context for decoding the planar mode flag based in part on one or more of the following: (a) whether the parent volume containing the volume is planar in the occupancy, (b) the occupancy of a neighboring volume at the parent depth, the neighboring volume being adjacent to the volume and having a face shared with the parent volume, or (c) the distance between the volume and the most recently decoded occupancy volume, the occupancy volume having the same depth as the volume and the same position on the axis as the volume; and reconstructing the occupancy bits of the sub-volumes.

[0047] In some implementations, if the parent plane mode flag indicates that the parent volume is planar, then the parent volume containing the volume is planar in the occupancy.

[0048] In some implementations, the distance is either near or far, and can be based on calculating a distance metric and comparing it to a threshold.

[0049] In some implementations, the context for determining the encoding / decoding plane mode flags can be based on a combination of (a), (b), and (c).

[0050] In some implementations, determining whether a volume is planar includes: determining that the volume is planar, and therefore, entropy encoding a planar location marker to signal whether at least one point is located in a first set or a second set. Entropy encoding of the planar location marker may include: determining a context for encoding and decoding the planar location marker. Determining the context may be based in part on one or more of the following: (a') the occupancy of neighboring volumes at a parent depth; (b') the distance between the volume and the nearest encoded occupant volume; (c') the planar location of the nearest encoded occupant volume (if any); or (d') the location of a volume within a parent volume. In some cases, determining the context for encoding and decoding the planar location marker may be based on a combination of three or more of (a'), (b'), (c'), and (d').

[0051] In some implementations, the distance is near, not too far, or far, and can be based on a calculated distance metric and compared with a first threshold and a second threshold.

[0052] In another aspect, this application describes a method for encoding a point cloud (e.g., the geometry of the point cloud) acquired by means of a beam combination including a beam set to generate a bitstream of compressed point cloud data. The point cloud (e.g., the geometry of the point cloud) may be defined in a tree structure (e.g., defined by a tree structure) with multiple nodes having parent-child relationships and representing the geometry of a volume space. Furthermore, the point cloud may reside within a volume space containing points recursively divided into sub-volumes and containing point cloud data. A volume may be divided into sub-sub-volume sets. An occupancy bit associated with each corresponding sub-sub-volume may indicate whether that corresponding sub-sub-volume contains at least one point. The method may include: determining an occupancy pattern of a volume based on the occupancy status of the sub-sub-volumes of the volume for the current node associated with the volume divided into sub-sub-volumes. The method may further include: entropy encoding a representation of the occupancy pattern into a bitstream. Wherein, entropy encoding of the occupancy pattern may include: determining angular information of the sub-sub-volumes or sub-sub-volume groups of the volume. The angular information may indicate one or more elevation angles relative to a position associated with a beam combination within the volume space. Entropy-encoded occupancy patterns may further include: determining the context for encoding the representation of the occupancy pattern based on angular information. Entropy-encoded occupancy patterns may also include: using the determined context to entropy-encode the representation of the occupancy pattern.

[0053] In another aspect, this application describes a method for decoding a bitstream of piezoelectric point cloud data to generate a reconstructed point cloud (e.g., a reconstructed geometry of the point cloud). The point cloud (e.g., the geometry of the point cloud) may have been acquired by means of a beam combination including a beam set. The point cloud may be defined in a tree structure (e.g., defined by a tree structure) with multiple nodes having parent-child relationships and representing the geometry of the volume space. Furthermore, the point cloud may reside within a volume space that is recursively divided into sub-volumes and contains points containing point cloud data. The volume may be partitioned into a set of sub-subvolumes. An occupancy bit associated with each corresponding sub-subvolume may indicate whether the corresponding sub-subvolume contains at least one point. The method may include: determining angle information of the sub-subvolumes or groups of sub-subvolumes of the volume for the current node associated with the volume divided into sub-subvolumes. The angle information may indicate one or more elevation angles relative to the position associated with the beam combination within the volume space (and, for example, relative to a plane containing the position associated with the beam combination, such as a horizontal plane). The method may also include: determining the context of a representation of the occupancy pattern for decoding the volume based on the angle information. Occupancy patterns can indicate the occupancy status of sub-sub-volumes of a volume. The method may further include: decoding a bitstream using the determined context entropy to generate a reconstructed representation of the occupancy pattern.

[0054] In some implementations, determining the angle information may include: determining a first elevation angle of a reference position within the volume relative to a reference position of the beam combination. Determining the angle information may also include: based on the first elevation angle, determining a specific beam that is assumed to have acquired points within the volume. Determining the angle information may further include: determining the origin position of the specific beam. Determining the angle information may also include: determining one or more elevation angles of a sub-sub-volume or group of sub-sub-volumes of the volume relative to the origin position of the specific beam.

[0055] In some embodiments, the method may further include: first determining volume qualification for determining context based on angle information by determining a measure of the angular magnitude of the volume as seen from a reference position of the beam combination. Determining volume qualification may further include: comparing the measure of angular magnitude with a measure of the difference angle between adjacent beams of the beam combination. Determining volume qualification may further include: determining volume qualification for determining context based on the comparison result.

[0056] In some implementations, the planar volume can be a volume divided into multiple sets of sub-volumes in a corresponding parallel plane, all sub-volumes containing at least one point lying in the same plane. Then, determining the context for encoding and decoding the representation of the occupancy pattern can include: determining the context for encoding and decoding the planar position flag based on angle information. The planar position flag can signal the position of the plane containing the sub-volume containing at least one point. Further, entropy-coding the representation of the occupancy pattern can include: entropy-coding the planar position flag using the determined context.

[0057] In some implementations, determining the angle information may include determining a corresponding elevation angle for each of a plurality of parallel planes relative to the origin position of a particular beam. Determining the context for encoding / decoding plane position markers may include comparing the corresponding elevation angle of each of the plurality of parallel planes with the beam angle of the particular beam (e.g., by calculating the difference between the corresponding angles). Determining the context for encoding / decoding plane position markers may also include determining the context for encoding / decoding plane position markers based on the result of the corresponding comparison.

[0058] In some implementations, determining the context for encoding / decoding plane position markers based on the results of the corresponding comparisons may be based on at least one of the following: the difference between the beam angle of a specific beam and the corresponding elevation angle of each of the plurality of parallel planes, the magnitude of the difference between the beam angle of a specific beam and the corresponding elevation angle of each of the plurality of parallel planes, and / or the quantization result of the difference between the magnitude of the largest difference among the magnitudes of the differences between the beam angle of a specific beam and the corresponding elevation angle of each of the plurality of parallel planes and the magnitude of the smallest difference among the magnitudes of the differences between the beam angle of a specific beam and the corresponding elevation angle of each of the plurality of parallel planes.

[0059] In some implementations, determining the context for representing the occupancy pattern may include determining the context for determining the occupancy bits of a sub-sub-volume for encoding / decoding the volume. Then, entropy-coding the representation of the occupancy pattern may include entropy-coding the occupancy bits using the determined context.

[0060] In some implementations, determining the angle information may include: determining a corresponding elevation angle for each sub-sub-volume of the volume relative to the origin position of a specific beam. Determining the context for the occupancy bits of the sub-sub-volumes used to encode / decode the volume may include: comparing the corresponding elevation angle of each sub-sub-volume of the volume with the beam angle of the specific beam. Determining the context for the occupancy bits of the sub-sub-volumes used to encode / decode the volume may also include: determining the context for the occupancy bits of the sub-sub-volumes used to encode / decode the volume based on the result of the comparison.

[0061] In some implementations, the beam combination can be a rotating beam combination. The point cloud can be defined relative to a Cartesian axis in volumetric space. The Cartesian axis can have a vertically oriented z-axis that is orthogonal to the horizontal plane and substantially parallel to the rotation axis of the rotating beam combination.

[0062] In some implementations, the bit stream may include one or more parameters that describe the characteristics of the beam combination.

[0063] In another aspect, this application describes a method for bitstream decoding of compressed point cloud data to generate a reconstructed point cloud. The point cloud may have been acquired using a beamset. Further, the point cloud may lie within a volume space of points recursively divided into sub-volumes and containing the point cloud data. The method may include: decoding from the bitstream a first parameter indicating whether an angular pattern should be used to encode the point cloud data. The method may also include: if the first parameter indicates that the point cloud data should be encoded using an angular pattern, decoding from the bitstream first angle information, which is related to the beam angle of the beamset relative to the corresponding beam origin position. The method may also include: for a current volume divided into sub-volumes, determining second angle information, which is related to the angle of a sub-volume or group of sub-volumes relative to a position associated with the beamset. The method may also include: decoding the point cloud data of the volume based on the first and second angle information.

[0064] In some implementations, decoding the first angle information may include: for a beam combination including at least some beams in a beam set, decoding from a bit stream a second parameter indicating the number of beams in the beam combination, and determining the number of beams in the beam combination based on the second parameter. The method may also include: for each of a plurality of beams indicated by the second parameter, decoding from a bit stream one or more third parameters indicating the beam origin position of the corresponding beam.

[0065] In some implementations, decoding one or more third parameters indicating the beam origin position of a corresponding beam may include: decoding a parameter indicating whether the beam origin position of the beam is expressed relative to a reference position of the beam combination. Decoding one or more third parameters may further include: if the beam origin position of the beam is expressed relative to a reference position of the beam combination, decoding a parameter indicating the coordinates of the beam origin position of the beam along the main axis of the beam combination. Decoding one or more third parameters may further include: if the beam origin position of the beam is expressed relative to a reference position of the beam combination, decoding a parameter indicating whether the beam origin position is displaced from the main axis of the beam combination. Decoding one or more third parameters may further include: if the beam origin position of the beam is expressed relative to a reference position of the beam combination, and if the beam origin position is displaced from the main axis of the beam combination, decoding a parameter indicating the displacement of the beam origin position from the main axis of the beam combination.

[0066] In some implementations, decoding the first angle information may include: checking whether the bit stream includes a fourth parameter indicating the number of beam combinations that respectively include at least some beams in the beam set, and if so, decoding the fourth parameter in the bit stream. Decoding the first angle information may also include: if the fourth parameter does not exist or is equal to zero, inferring that the number of beam combinations is 1. Decoding the first angle information may also include: if the fourth parameter is equal to 1, decoding a fifth parameter indicating the number of beam combinations minus 2.

[0067] In some implementations, decoding the first angle information may include: for a beam combination including at least some beams in a beamset, checking whether the bit stream includes a sixth parameter indicating whether the reference position of the beam combination is at the origin of a coordinate system in volume space, and if so, decoding the sixth parameter. Decoding the first angle information may also include: if the sixth parameter is not present in the bit stream or if the sixth parameter indicates that the reference position of the beam combination is at the origin of the coordinate system, then considering the origin of the coordinate system as the reference position of the beam combination. Decoding the first angle information may also include: otherwise, decoding a parameter indicating the coordinates of the reference position of the beam combination from the bit stream.

[0068] In some implementations, decoding the first angle information may include: for a beam combination including at least some beams in a beamset, decoding a seventh parameter from the bit stream, the seventh parameter indicating whether the main axis of the beam combination is arranged along a coordinate axis in the volume space. Decoding the first angle information may further include: if the seventh parameter indicates that the main axis of the beam combination is arranged along a coordinate axis in the volume space, decoding a parameter indicating the specific coordinate axis along which the main axis of the beam combination is arranged, and treating the specific coordinate axis as the main axis of the beam combination (e.g., the orientation of the main axis). Decoding the first angle information may further include: additionally decoding parameters indicating an angle that expresses the orientation of the main axis of the beam combination, and determining the main axis of the beam combination based on these parameters. The main axis may be assumed to be a reference position throughout the beam combination.

[0069] In some implementations, decoding the first angle information may include: for a beam combination including at least some beams in a beam set, decoding an eighth parameter from a bit stream indicating the number of beams in the beam combination, and determining the number of beams in the beam combination based on the eighth parameter. Decoding the first angle information may also include: for each of the plurality of beams in the beam combination, decoding one or more ninth parameters from a bit stream indicating the beam angle of the respective beam.

[0070] In some implementations, decoding one or more ninth parameters indicating the beam angle of a corresponding beam may include decoding a parameter indicating whether the beam angle is expressed as the tangent of an angle. Decoding one or more ninth parameters may also include decoding a parameter expressing the beam angle.

[0071] In some embodiments, the method may further include: first determining that the volume is qualified for decoding the volume point cloud data based on first angle information and second angle information. For a beam combination including at least some beams in a beamset, determining angle qualification may include: determining a measure of the angular magnitude of the volume as seen from a reference position of the beam combination. Determining angle qualification may further include: comparing the measure of angular magnitude with a measure of the difference angle between the angles of adjacent beams of the beam combination. Determining angle qualification may further include: based on the comparison result, determining whether (or indeed) the volume is qualified for decoding the volume point cloud data based on the first angle information and second angle information.

[0072] In some implementations, determining the second angle information may include: determining a first angle relative to a reference position within the volume and a reference position of a beam combination including at least some beams in a beamset. Determining the second angle information may further include: determining a specific beam of a beam combination of beams assumed to have been acquired within the volume, based on a first elevation angle. Determining the second angle information may further include: determining the beam origin position of the specific beam. Determining the second angle information may further include: determining the angle of a sub-volume or group of sub-volumes of the volume relative to the beam origin position of the specific beam.

[0073] In another aspect, this application describes a method for encoding a point cloud acquired by means of a beamset to generate a bitstream of compressed point cloud data. The point cloud may reside within a volume space of points recursively divided into sub-volumes and containing point cloud data. The method may include: encoding a first parameter into a bitstream, the first parameter indicating whether an angular pattern should be used to encode the point cloud data. The method may further include: if the first parameter indicates that the point cloud data should be encoded using an angular pattern, encoding first angle information into a bitstream, the first angle information being related to the beam angle of the beamset relative to the corresponding beam origin position. The method may further include: if the first parameter indicates that the point cloud data should be encoded using an angular pattern, determining second angle information for the current volume divided into sub-volumes, the second angle information being related to the angle of a sub-volume or group of sub-volumes relative to a position associated with the beamset. The method may further include: if the first parameter indicates that the point cloud data should be encoded using an angular pattern, encoding the point cloud data of the current volume based on the first angle information and the second angle information.

[0074] In another aspect, this application describes encoders and decoders configured to implement such encoding / decoding and decoding methods.

[0075] In another aspect, this application describes a non-transient computer-readable medium storing computer-executable program instructions that, when executed, cause one or more processors to perform the described encoding and / or decoding methods.

[0076] In another aspect, this application describes a computer-readable signal containing program instructions that, when executed by a computer, cause the computer to perform the described encoding and / or decoding methods.

[0077] Other aspects and features of this application will be understood by those skilled in the art by reading the following description in conjunction with the accompanying drawings.

[0078] Any feature described with respect to one aspect or embodiment of the invention may also be used with respect to one or more other aspects / embodiments. These and other aspects of the invention will become apparent and elucidated with reference to the embodiments described herein.

[0079] In the following description, the terms "node," "volume," and "subvolume" are sometimes used interchangeably. It should be understood that a node is associated with a volume or subvolume. A node is a specific point in a tree, which can be an internal node or a leaf node. A volume or subvolume is the bounded physical space represented by a node. In some cases, the term "volume" can be used to refer to the largest bounded space contained within a point cloud. To construct a tree structure of interconnected nodes for encoding and decoding point cloud data, a volume can be recursively partitioned into subvolumes. The tree structure that divides a volume into subvolumes can be referred to as a "parent" and "child" relationship, where a subvolume is a child node or child-child volume of a parent node or parent volume. Subvolumes within the same volume can be referred to as sibling nodes or sibling subvolumes.

[0080] In this application, the term "and / or" is intended to cover all possible combinations and sub-combinations of the listed elements, including any one, any sub-combination, or all of the elements listed individually, and does not necessarily exclude additional elements.

[0081] In this application, the phrase “...or at least one of ...” is intended to cover any one or more of the listed elements, including any single element, any sub-combination, or all elements, without necessarily excluding any additional elements, and not necessarily all elements.

[0082] A point cloud is a set of points in a three-dimensional coordinate system. These points are typically intended to represent the outer surface of one or more objects. Each point has a location (position) in the three-dimensional coordinate system. This location can be represented by three coordinates (X, Y, Z), which can be a Cartesian coordinate system or any other coordinate system. These points may have other associated properties, such as color, which in some cases may be three-component values, such as R, G, B or Y, Cb, Cr. Other associated properties may include transparency, reflectivity, normal vector, etc., depending on the desired application of the point cloud data.

[0083] Point clouds can be static or dynamic. For example, a detailed scan or mapping of an object or terrain might be static point cloud data. LiDAR-based environmental scanning for machine vision purposes might be dynamic because the point cloud (at least potentially) changes over time, for example, by scanning one volume at a time. Therefore, a dynamic point cloud is a time-ordered sequence of point clouds.

[0084] Point cloud data can be used in many applications, including preservation (scanning of historical or cultural objects), mapping, machine vision (such as autonomous or semi-autonomous vehicles), and virtual reality systems. Dynamic point cloud data used for applications like machine vision can be quite different from static point cloud data used for preservation purposes. For example, automotive vision typically involves relatively low-resolution, non-color, highly dynamic point clouds obtained through LiDAR (or similar) sensors with high capture frequencies. These point clouds are not intended for human consumption or viewing, but rather for machine object detection / classification in the decision-making process. As an example, a typical LiDAR frame contains tens of thousands of points, while high-quality virtual reality applications require millions of points. With increasing computing speeds and the discovery of new applications, the need for even higher resolution data may emerge over time.

[0085] While point cloud data is useful, the lack of effective and efficient compression (i.e., encoding and decoding processes) can hinder adoption and deployment. A specific challenge in encoding and decoding point clouds that doesn't arise with other data compression methods (such as audio or video) is the encoding and decoding of point cloud geometry. Point clouds tend to be sparse, making efficient encoding and decoding of point locations even more challenging.

[0086] One of the more common mechanisms for encoding and decoding point cloud data is the use of a tree-based structure. In a tree-based structure, the boundary 3D volume of the point cloud is recursively divided into subvolumes. Nodes in the tree correspond to subvolumes. The decision to further subdivide subvolumes can be based on the tree's resolution and / or whether a subvolume contains any points. Nodes may have an occupancy flag indicating whether their associated subvolume contains points. Split flags can signal whether a node has child nodes (i.e., whether the current volume has been further split into subvolumes). These flags can be used for entropy encoding in some cases, and predictive encoding / decoding can be used in others.

[0087] The 3D volume of a point cloud can also be defined using bounding boxes. The quadtree plus binary tree (QTBT) structure of the bounding box allows the point cloud to be partitioned into non-cubic products, which can better represent the overall shape of the point cloud or objects within it. Quadtree (QT) partitioning divides the volume into four sub-volumes, for example, by using two planes to split the volume, thus cutting the volume into four parts. Binary tree (BT) partitioning divides the volume into two sub-volumes, for example, by using one plane to split the volume, thus cutting the volume into two parts.

[0088] A commonly used tree structure is the octree. In this structure, the volume / subvolumes are cubes, and each partition of a subvolume results in eight other subvolumes / subcubes. Another commonly used tree structure is the KD-tree, where the volume (cube or rectangular prism) is recursively partitioned into two parts by planes orthogonal to one of the axes. The octree is a special case of the KD-tree, where the volume is partitioned by three planes, each orthogonal to one of the three axes. KD-trees can often be obtained by recursively applying binary tree (BT) partitioning.

[0089] The partitioning of a volume is not necessarily two sub-volumes (KD tree) or eight sub-volumes (octree), but may involve other partitioning, including partitioning into non-cubic shapes or involving non-adjacent sub-volumes.

[0090] For ease of explanation and because octrees are a popular candidate tree structure for automotive applications, this application may refer to octrees; however, it should be understood that the methods and devices described herein may be implemented using other tree structures.

[0091] Now refer to Figure 1 The diagram shows a simplified block diagram of a point cloud encoder 10 according to various aspects of this application. The point cloud encoder 10 includes a tree building module 12 for receiving point cloud data and generating a tree (in this example, an octree) representing the geometry of a volumetric space containing the point cloud, and indicating the position or location of points relative to the point cloud within that geometry.

[0092] In the case of a uniformly partitioned tree structure, such as an octree, each node can be represented by a sequence of occupancy bits, where each occupancy bit corresponds to a sub-volume within the node, and signals indicate whether a sub-volume contains at least one node. The occupancy sub-volumes are recursively partitioned to the maximum depth of the tree. This can be referred to as tree serialization or binarization. Figure 1 As shown, in this example, the point cloud encoder 10 includes a binarizer 14 for binarizing an octree to produce a bitstream of binarized data representing the tree.

[0093] The bit sequence can then be encoded using entropy encoder 16 to produce a compressed bitstream. Entropy encoder 16 can encode the bit sequence using context model 18, which specifies the probabilities used for bit encoding / decoding based on context determination of entropy encoder 16. Context model 18 can be adaptively updated after encoding / decoding each bit or a defined set of bits. In some cases, entropy encoder 16 can be a binary arithmetic encoder. In some implementations, the binary arithmetic encoder can employ context-adaptive binary arithmetic codec (CABAC). In some implementations, codecs other than arithmetic codecs can be used.

[0094] In some cases, the entropy encoder 16 may not be a binary codec, but can operate on non-binary data. The output octree data from the tree building module 12 may not be evaluated as binary, but can be encoded as non-binary data. For example, in the case of an octree, eight flags (e.g., occupancy flags) within a subvolume in its scan order can be considered as 2... 8 -1 digit (e.g., an integer between 1 and 255, because the value 0 is impossible for splitting a sub-volume, i.e., it will not be split if it is completely unoccupied). In some implementations, this number can be encoded by an entropy encoder using a multi-symbol arithmetic codec. Within a sub-volume (e.g., a cube), the sequence of flags defining this integer can be referred to as a "pattern".

[0095] In point cloud compression, a common practice is to use an occupancy bit value of 1 to signal that the associated node or volume is "occupied," meaning it contains at least one point, and an occupancy bit value of 0 to signal that the associated node or volume is "unoccupied," meaning it does not contain any points. More generally, the occupancy bit can have a value indicating either occupancy or unoccupiedness. In the following description, for ease of explanation, exemplary embodiments may be described, wherein the convention of 1 = occupied and 0 = unoccupied is used; however, it is to be understood that this application is not limited to this convention.

[0096] exist Figure 2 A block diagram of an example point cloud decoder 50 corresponding to encoder 10 is shown. Point cloud decoder 50 includes entropy decoder 52, which uses the same context model 54 used by encoder 10. Entropy decoder 52 receives an input bitstream of compressed data and entropy decodes the data to produce an output sequence of decompressed bits. This sequence is then converted into reconstructed point cloud data by tree reconstructor 56. Tree reconstructor 56 reconstructs the tree structure using knowledge of the scan order in which the decompressed data and tree data are binarized. Tree reconstructor 56 is thus able to reconstruct the positions of points from the point cloud (depending on the resolution of the tree encoder / decoder).

[0097] In European Patent Application No. 18305037.6, the applicant describes a method and apparatus for selecting from the distribution of available patterns when encoding / decoding the occupancy patterns of a particular node based on some occupancy information from previously encoded / decoded nodes near that node. In one example embodiment, the occupancy information is obtained from the occupancy patterns of the parent node for the particular node. In another example embodiment, the occupancy information is obtained from one or more nodes adjacent to the particular node. The contents of European Patent Application No. 18305037.6 are incorporated herein by reference. This is referred to as determining a “neighbor configuration” and selecting a context (i.e., pattern distribution) based at least in part on the neighbor configuration.

[0098] In European Patent Application No. 18305415.4, the applicant describes a method and apparatus for binary entropy encoding occupancy patterns. The contents of European Patent Application No. 18305415.4 are incorporated herein by reference.

[0099] Certain types of point cloud data tend to exhibit strong directionality. Unnatural environments are particularly prone to this strong directionality because they often have uniform surfaces. For example, in the case of LiDAR, roads and the walls of adjacent buildings are typically horizontal or vertical planes. In the case of interior scanning within a room, floors, ceilings, and walls are all planar. LiDAR used in automotive vision and similar applications often has lower resolution and requires fast and efficient compression.

[0100] Octrees are efficient tree structures because they are based on a uniform partitioning process that divides a cube into eight equal sub-cubes using three orthogonal planes in every case, thus making their signaling structure efficient. However, octrees using current signaling processes cannot leverage the efficiency gained from recognizing the planar properties of some unnatural environments. KD trees, however, are better able to adjust the orientation of the partitions into point clouds. This makes them a more efficient and effective structure for these types of environments. The disadvantage of KD trees is that their signaling requires significantly more data than octrees. The non-uniformity of KD trees means that some techniques used to improve octree compression are unavailable for KD trees or computationally difficult to implement.

[0101] Therefore, it would be advantageous to have a mechanism for representing unnatural environments using a tree structure based on uniform partitioning in a way that improves compression by leveraging horizontal and / or vertical orientation.

[0102] According to one aspect of this application, the improved point cloud compression process and apparatus are characterized by a planar encoding / decoding mode. The planar mode is signaled to indicate that a volume satisfies certain requirements of planarity in its occupancy. Specifically, a volume is planar if all its occupied sub-volumes are located in or within a common plane. The syntax used for signaling can indicate whether a volume is planar, and if so, the location of the common plane. By utilizing this planarity knowledge, compression gains can be implemented. Mechanisms for applying eligibility criteria to enable the planar mode and for context-adaptive encoding / decoding for planar mode signaling contribute to improved compression performance.

[0103] In the following description, planarity is assumed to be relative to a Cartesian axis aligned with the structure of the volume and its sub-volumes. A volume is planar if all occupied sub-volumes lie in a common plane orthogonal to one of these Cartesian axes. By convention, the axis will be assumed to be vertical, meaning that the (horizontal) plane is orthogonal to the z-axis. In many of the examples below, horizontal planarity will be used to illustrate the concept; however, it will be understood that this application is not limited to horizontal planarity and may alternatively or additionally include vertical planarity relative to the x-axis, y-axis, or both x and y-axis. Furthermore, in some examples, planarity does not necessarily have to be aligned by orthogonality to a Cartesian axis. For illustration, in one example, diagonal vertical planarity may be defined as being at a 45-degree angle to both the x and y axes.

[0104] Now refer to Figure 3 and 4 Each of these examples shows sample volumes 300 and 400. In this example, horizontal planarity will be illustrated and discussed, but those skilled in the art will understand that the concept will be extended to vertical or other planarities.

[0105] Volume 300 is shown divided into eight sub-volumes. Occupied sub-volumes are indicated by shading, while unoccupied sub-volumes are shown as empty. Note that the lower four sub-volumes (in the z-axis or vertical sense) of volume 300 are occupied. This occupancy pattern is horizontal; that is, each occupied sub-volume is in the same horizontal plane, i.e., has the same z-position. Volume 400 shows another example of a horizontal plane occupancy pattern. All occupied sub-volumes of volume 400 are in the same horizontal plane. Volume 300 shows the case where the lower plane is occupied. Volume 400 shows the case where the upper plane is occupied. This can be referred to as "plane position," where the plane position signals the position of the plane sub-volume within the volume. In this case, it is a binary 'up' or 'down' signal.

[0106] The planarity of a volume is not limited to the case where all sub-volumes of the plane (e.g., all sub-volumes of the upper half of a 2x2x2 volume) are occupied. In some cases, only some sub-volumes within the plane are occupied, provided that no sub-volumes outside the plane are occupied. In fact, only one occupied sub-volume can be considered "planar." Volumes 302, 304, 306, 308, 402, 404, 406, and 408 illustrate examples of horizontal plane occupancy. Note that volumes 308 and 408 satisfy the requirement of horizontal plane occupancy because in each case, the upper or lower half of volumes 308 and 408 is empty; that is, all occupied sub-volumes (in these examples, one sub-volume) lie within the horizontal half of volumes 308 and 408. It should also be understood that, in these examples, volumes with a single occupied sub-volume will also satisfy the requirements of vertical planarity relative to the y-axis and vertical planarity relative to the x-axis. That is, volumes 308 and 408 are planar in all three directions.

[0107] Planarity can be signaled relative to the volume via a planar mode flag, such as `isPlanar`. In cases where multiple planar modes may exist, such as relative to the z-axis, y-axis, and x-axis, there may be multiple flags: `isZPlanar`, `isYPlanar`, `isXPlanar`. In this example, for ease of illustration, it is assumed that only the horizontal planar mode is enabled.

[0108] The plane mode flag indicates whether the volume is planar. If it is planar, then the second syntax element, the plane position flag, and `planePosition` can be used to signal the plane's position within the volume. In this example, the `planePosition` flag signals whether the sub-volume occupied by the plane is in the upper or lower half of the volume.

[0109] In more complex implementations involving non-orthogonal planes (e.g., planes diagonally opposite to one or more axes), more complex signaling syntax involving multiple flags or non-binary syntactic elements can be used.

[0110] Planar pattern flags and / or planar position flags can be encoded and decoded in the bitstream using any suitable encoding scheme. In some implementations, the flags can be encoded using predictive and / or context-adaptive entropy encoding to improve compression. Example techniques for determining the context for encoding and decoding flags are discussed further below.

[0111] Occupy encoding / decoding and flat mode

[0112] By signaling planarity, the encoding and decoding of occupancy bits can be modified because planarity information allows for inferences about the occupancy pattern of the shortened occupancy signal. For example, if the volume is planar, then the four sub-volumes not pointed to by the planar position flag can be assumed to be empty, and their occupancy bits do not need to be encoded and decoded. Only the maximum four bits of the occupied plane need to be encoded and decoded. Furthermore, if the first three encoded bits of the plane are zero (unoccupied), then the last (fourth) bit in the encoding and decoding order can be inferred to be 1 (occupied), because the planar signal indicates that the plane is occupied. Additionally, if the planar signal indicates that the volume is not planar, then there must be at least one occupied sub-volume in both planes, and if the first three occupancy bits of either plane are zero, then additional inferences about the occupancy of the last bit of either plane are allowed.

[0113] Therefore, signaling plane mode can provide efficiency in occupancy data encoding and decoding. However, plane mode signaling adds syntactic elements to the bitstream and may not provide efficiency in all cases. For example, in dense point clouds and at certain depths, signaling planeness may not be advantageous because, by definition, any node with more than five occupancy child nodes cannot be plane. Therefore, having eligibility criteria for enabling plane mode may be more advantageous. Providing eligibility criteria adapted to local data would be even more beneficial.

[0114] In one example, eligibility criteria might be biased towards use with sparse clouds. For instance, eligibility criteria could be based on metrics such as the average number of child nodes occupied in a volume. This running average could be determined for a certain depth of the tree and then used for eligibility of the planar pattern at the next lower depth. In one example, the planar pattern could be disabled if the average number of occupied child volumes is greater than 3. This technique has the advantage of simplicity but lacks adaptation to the local properties of clouds.

[0115] In another example, the probability factor can be determined. The probability factor indicates the likelihood that a node is planar, i.e., the probability of planarity. If the probability is low, indicating planarity in signaling would be costly with little potential gain. If the probability factor is greater than a threshold at that time, a threshold eligibility value can be set, and a planar mode can be enabled for a given volume / node.

[0116] p = prob(planar) ≥ threshold

[0117] As an example, the chosen threshold could be 0.6; however, it should be understood that other values ​​can be used for other situations. The probability is updated during the encoding and decoding process. The update process can be tuned for fast or slow updates specific to the application. Faster updates may assign greater weights or biases to the most recently encoded or decoded nodes.

[0118] The example probability update process can be expressed as:

[0119] p new = (Lp + δ(coded node)) / (L + 1)

[0120] In the above expression, p is the current probability. new For updating probabilities, δ(coded node) is the planar state of the current node, which is 0 for non-planar nodes and 1 for planar nodes, and L is a weighting factor for the rate at which tuning updates occur. The weighting factor L can be set to a power of 2 minus 1, such as 255, to allow for a simple integer implementation; more specifically, division can be implemented using a simple offset operation. Note that a planar state does not necessarily mean that a planar mode flag is encoded; probabilities track the planarity of the nearest nodes, regardless of whether a planar mode flag is enabled. For example, during the decoding process, the planar state of any node is known after decoding the occupancy bits associated with that node, regardless of whether the planar mode flag has been decoded.

[0121] For example, a probability update may occur when a node occupancy is encoded or decoded. In another example, a probability update may occur when node plane information is decoded for the current node. The updated probability is then used to determine the eligibility of the next node in the encoding / decoding sequence for plane mode signaling.

[0122] As previously mentioned, planar patterns can signal to a horizontal plane (z-axis), a vertical plane (x-axis or y-axis), or any two or all three planes. Where planar patterns can signal in more than one direction, the eligibility criteria may differ. That is, each additional planar pattern signaling to a node offers half the benefit in terms of occupancy signaling. Using the first planar pattern, if planar, half of the occupancy bits can be inferred. Using the second planar pattern, only two of the remaining four occupancy bits can be inferred, and so on. Therefore, the threshold for signaling additional planar patterns can be higher than that for the first planar pattern.

[0123] The definitions of the “first,” “second,” and “third” planar patterns can be based on probability and their order from most likely to least likely. Thresholds can then be applied to eligibility, where the first threshold is lower than the second threshold, and so on. Example thresholds are 0.6, 0.77, and 0.88, although these are merely illustrative. In another embodiment, only one threshold exists, and only the most likely planar pattern is enabled even if more than one planar pattern satisfies the threshold.

[0124] Now refer to Figure 5The example method 500 for encoding point cloud data using a planar pattern is illustrated in flowchart form. Method 500 reflects the process for encoding occupancy information for a volume. In this example, the volume is uniformly divided into eight sub-volumes, each with occupancy bits, according to an octree-based encoding / decoding. For simplicity, this example assumes that only one (e.g., horizontal) planar pattern is used.

[0125] In operation 502, the encoder evaluates whether the volume occupancy pattern is qualified for use in the planar encoding / decoding mode. As discussed above, in one example, qualification can be based on cloud density, which can be evaluated using the average number of occupied child nodes. To improve local adaptability, qualification can be based on probability factor tracking. If the planar encoding / decoding mode is unqualified, then the volume occupancy pattern is not encoded using the planar encoding / decoding mode, as indicated in operation 504.

[0126] If planar mode is enabled, then in operation 506, the encoder evaluates whether the volume is planar. If not, then in operation 508, it encodes the planar mode flag, for example, isPlanar = 0. In operation 510, the encoder then encodes the occupancy mode based on at least one occupied sub-volume for each plane. That is, the occupancy mode is encoded, and if the first three bits of the encoding for any plane (upper plane or lower plane) are zero, then the last (fourth) bit of that plane is not encoded and is inferred to be 1 because the corresponding sub-volume must be occupied.

[0127] If planar mode is enabled and the volume is planar, then in operation 512, the planar mode flag is encoded, for example, isPlanar = 1. Because the volume is planar, the encoder then also encodes the plane position flag, planePosition. The plane position flag signals whether the sub-volume occupied by the plane is in the upper or lower half of the volume. For example, planePosition = 0 might correspond to the lower half (i.e., the lower z-axis position), and planePosition = 1 might correspond to the upper half. The occupancy bits are then encoded based on the planarity of the volume and the knowledge of the position of the occupied sub-volume. That is, up to four bits are encoded, because four can be inferred to be zero, and if the first three bits are zero, the fourth bit can be inferred to be 1.

[0128] Example method 600 for decoding encoded point cloud data Figure 6The example method 600 is implemented by a decoder that receives a bitstream of encoded data. For the current volume, the decoder determines in operation 602 whether the volume is qualified for use in planar mode. The qualification assessment is the same as that performed at the encoder. If it is unqualified, then the decoder performs entropy decoding of the occupancy mode according to the signaling that is not normally used in planar mode, as indicated in operation 604.

[0129] If planar mode is enabled, then in operation 606, the decoder decodes the planar mode flag. The decoded planar mode flag indicates whether the volume is planar, as indicated in operation 608. If it is not planar, then the decoder decodes the occupancy bits to know that at least one sub-volume in each plane is occupied. This allows the decoder to infer one or both occupancy bits based on the values ​​of the other decoded bits.

[0130] If the decoded planar mode flag indicates that the volume is planar, then in operation 612, the decoder decodes the planar position flag. The decoded planar position flag indicates whether the occupied sub-volume is in the upper or lower half of the volume. Based on this knowledge, the decoder then infers the value of the four occupied bits in the unoccupied half to be zero, and it decodes up to four bits of the occupancy pattern for the occupied half, as shown in operation 614.

[0131] As mentioned earlier, the encoding and decoding of occupancy bits can include, in part, neighbor configuration entropy encoding, where the neighbor configuration reflects the occupancy of various volumes that share at least one face with the current volume. When evaluating the neighbor configuration, if the neighbor configuration (NC) is zero, meaning that no neighboring volumes are occupied, then a flag can be encoded and decoded to signal whether the current volume has a single occupied sub-volume.

[0132] Occupancy bit encoding and decoding using NC and single-node signaling can be based on Figure 7 The planar mode signaling shown is adapted. Figure 7 An example method 700 for occupancy bit encoding / decoding is shown. The portion of the flowchart illustrating method 700 reflects the effect of plane signaling on occupancy bit encoding / decoding. Although not shown, it can be assumed that appropriate qualification tests are performed and that plane flags and position flags (if applicable) are encoded / decoded. In this example, it will be assumed that only one plane mode is possible, but extensions to other modes or additional modes will be understood.

[0133] In operation 702, the codec evaluates whether NC is zero, i.e., whether all neighboring volumes are empty. If not, the codec evaluates in operation 704 whether the volume is planar. If it is not planar, the codec encodes or decodes the eight occupied bits knowing that at least one bit in each plane is 1, as shown in operation 706. If it is planar, the codec infers that the bits of the unoccupied planes are zero and encodes and decodes the other four bits, thus knowing that at least one of the occupied bits is 1.

[0134] If NC is zero, the codec determines whether a single occupied subvolume exists, indicated in the bitstream by a single node flag, as shown in operation 710. If the single node is true, then in operation 712, the data is bypassed and encoded to signal any remaining xyz position data regarding the location of the single node within the volume, which has not yet been obtained from the encoded plane mode and plane position flag. For example, if the plane mode is enabled to encode horizontal plane information, then the x and y flags are bypassed and encoded to signal the single node location, but the z position is known from the plane position flag.

[0135] If a single node is false, then the occupied bits are encoded / decoded, as indicated by operation 714, provided that at least two sub-volumes are occupied. This may include determining whether a volume is planar, and if so, determining its planar location, and then encoding / decoding the occupied bits accordingly. For example, if it is planar, then unoccupied planes can be inferred to contain all zero bits, and the bits of occupied planes can be encoded / decoded, with at most two of them inferred based on the knowledge that at least two sub-volumes are occupied.

[0136] Now refer to Figure 8 The diagram illustrates a flowchart of an example method 800 for encoding and decoding occupancy bits using three possible planar modes. The portion of method 800 shown in the flowchart reflects the effect of planar signaling on occupancy bit encoding and decoding. Although not shown, it can be assumed that appropriate qualification tests are performed and that planar and position flags (if applicable) are encoded / decoded. The codec first evaluates whether all three planar modes indicate that the volume is planar relative to all three axes, as shown in operation 802. If so, then they collectively indicate the position of a single occupied sub-volume, and all occupancy bits can be inferred, as shown in operation 804.

[0137] If the occupancy bits are not all planar, the codec evaluates the neighbor configuration (NC) in operation 806 to see if it is zero. If the NC is not zero, then in operation 808, the occupancy bits are encoded and decoded based on planar mode signaling. As discussed above, occupancy bit encoding and decoding can be masked by planar mode signaling, allowing some possible inferences to be made to shorten the occupancy encoding and decoding process.

[0138] If NC is zero, then the single-node flag can be encoded / decoded. The codec first evaluates whether at least one of the plane mode flags indicates that the plane mode is false. If so, it means that it cannot be a single-node case because more than one sub-volume is occupied. Therefore, if this is not the case, i.e., no plane is false in operation 810, then the single-node flag is evaluated and encoded / decoded in operation 812. If the single-node flag is set, the x, y, z bits of the single-node position can be bypassed and encoded / decoded to a degree that they have not yet been inferred from the plane position data, as shown in operation 814.

[0139] If operation 810 determines that at least one planar mode flag indicates the volume is non-planar, or if in operation 812 the signal node flag indicates it is not a single node, then the codec in operation 816 evaluates whether there are two planar mode flags indicating the volume is planar in both directions, and if so, all occupied bits can be inferred in operation 804. If not, then the occupied bits are encoded and decoded in operation 818 to determine the planarity (if any) and that at least two bits are non-zero.

[0140] Those skilled in the art will understand that the feature employed in current test modes for point cloud encoding and decoding is the Inferential Direct Encoding and Decoding (IDCM) mode, which is used to handle very isolated points. Since isolated points have little correlation with neighboring nodes, their locations are directly encoded and decoded, rather than encoding and decoding occupancy information of cascaded individual child nodes. This mode is qualified under the condition of isolated nodes, and in the qualified case, IDCM activation is signaled by an IDCM flag. Upon activation, the local locations of one or more points belonging to that node are encoded and decoded, and then that node becomes a leaf node, effectively halting the recursive partitioning and tree-based encoding and decoding process for that node.

[0141] The process of signaling planeness can be incorporated into the encoding / decoding process with IDCM mode by signaling planeness (if qualified) before signaling IDCM. First, the qualification of a node for IDCM can be affected by plane information. For example, if a node is not planar, it may be unqualified for IDCM. Second, with IDCM active, plane knowledge helps in encoding / decoding the location of points in the volume associated with the node. For example, the following rules can be applied:

[0142] ● If the node is in the x-plane, then the plane's position, planeXPosition, is known; therefore, the highest bit of the point's x-coordinate is known from the plane's position. This bit is not encoded or decoded in IDCM; the decoder will derive it from the plane's position.

[0143] ● If the node is in the y-plane, then the plane position (planeYPosition) is known, therefore the highest bit of the point's y-coordinate is known from the plane position. This bit is not encoded or decoded in IDCM; the decoder will derive it from the plane position.

[0144] ● If the node is in the z-plane, the plane position (planeZPosition) is known, therefore the highest bit of the point's z-coordinate is known from the plane position. This bit is not encoded or decoded in IDCM; the decoder will derive it from the plane position.

[0145] If a node is planar in multiple directions, the inference of the highest position of the x, y, and z coordinates still holds. For example, if a node is in both the x and y planes, the highest positions of the x and y coordinates are inferred from planeXPosition and planeYPosition.

[0146] Entropy coding of planar pattern syntax

[0147] Planar pattern syntax (such as planar pattern flags or planar position flags) can represent significant portions of a bitstream. Therefore, to make planar patterns effective in compressed point cloud data, it may be advantageous to ensure that planar information is entropy-coded using effective contextual determination.

[0148] Whether a node / volume is planar is signaled using the planar pattern flag `isPlanar`. In this discussion of examples, it will be assumed that the planar pattern applies to horizontal planarity, i.e., relative to the z-axis. In this example, the flag might be referred to as `isZPlanar`. The entropy encoding of the flag can employ a binary arithmetic codec, such as a context-adaptive binary arithmetic codec (CABAC). The context (or internal probability) can be determined using one or more predictors.

[0149] The planar pattern flag of the current node or sub-volume signals whether the sub-sub-volumes within the sub-volume are planar. The current node or sub-volume exists within the parent volume. As an example, predictive factors used to determine the context for encoding and decoding the planar pattern flag may include one or more of the following:

[0150] (a) Planarity of the parent volume;

[0151] (b) Occupation of adjacent volumes; and

[0152] (c) Distance to the nearest occupied encoded / decoded node at the same depth and z-axis position.

[0153] Figure 9 The diagram illustrates three example factors related to the current node 900 within the parent node 902.

[0154] Factor (a) refers to whether the parent node 902 is planar. Regardless of whether it uses planar mode encoding or decoding, if the parent node 902 satisfies the planarity criterion (in this case, horizontal planarity), then the parent node 902 is considered planar. Factor (a) is binary: "the parent is planar" or "the parent is not planar".

[0155] Factor (b) refers to the occupancy status of the neighboring volume 904 at the parent depth adjacent to the face (aligned with the plane) of the parent volume. In the case of horizontal planeness, if the current node 900 is in the upper half of the parent node 902, then the neighboring volume 904 is vertically above the parent node 902. If the current node 900 is in the lower half of the parent node 902, then the neighboring volume 904 will be vertically below it. In the case of vertical planeness, the neighboring volume will be adjacent to one side, depending on the x-axis or y-axis planeness and the position of the current node. Factor (b) is also binary: the neighbor is either occupied or unoccupied.

[0156] Factor (c) refers to how far away the nearest encoded / decoded node 906 is, assuming the encoded / decoded node and the current node 900 are at the same depth and in a common plane (i.e., at the same z-axis position as the current node 900). The encoded / decoded node 906 is not necessarily in an adjacent volume and may be some distance away, depending on cloud density. The codec tracks encoded / decoded nodes and identifies the nearest node that satisfies these criteria. The distance d between the current node 900 and the encoded / decoded node 906 can be determined by the relative positions of nodes 900 and 906. In some embodiments, for ease of calculation, the L1 norm can be used to determine the distance, i.e., the absolute value of increment x plus the absolute value of increment y. In some embodiments, the L2 norm can be used to determine the distance, i.e., the square root of the sum of squares (of the squares of the squares of increment x and the squares of increment y).

[0157] In some implementations, the distance d can be discretized into two values, "near" and "far". The division between "near" d and "far" d can be appropriately chosen. By classifying the distance as near or far, the factor (c) is also binary. It is to be understood that in some implementations, the distance can be discretized into three or more values.

[0158] If all three example factors are used in context determination, then 2x2x2 = 8 separate contexts can be maintained for encoding and decoding of the planar pattern flags.

[0159] If the planar mode flag is encoded for the current node 900 and the current node 900 is planar, then the planar position flag can be encoded, such as planeZPosition. The planar position flag signals which half of the current node 900 contains the occupied sub-volume. In the case of horizontal planarity, the planar position flag signals the lower or upper half.

[0160] Entropy encoding of planar location markers can also employ binary arithmetic codecs, such as CABAC. Context (or internal probabilities) can be determined using one or more predictive factors; possible examples include:

[0161] (a') Occupation of adjacent volume 904;

[0162] (b') Distance to the nearest occupied encoded / decoded node 906 at the same depth and z-axis position;

[0163] (c') If the most recently occupied encoded / decoded node 906 at the same depth and z-axis position is planar, then its planar position; and

[0164] (d') The position of the current node 900 within its parent node 902.

[0165] Factor (a') is the same as factor (b) discussed above in the context of planar pattern marking. Factor (b') is the same as factor (c) discussed above in the context of planar pattern marking. In some example implementations, factor (b') can discretize the distance into three categories: "near", "not too far", and "far". As discussed above, the distance can be determined using the L1 norm or L2 norm or any other suitable metric.

[0166] Factor (c') refers to whether the most recently occupied encoded / decoded node 906 is planar, and if so, whether it is on the upper or lower plane, i.e., its planar position. As a result, even distant, planar encoded / decoded nodes can be strong predictors of the planarity or planar position of the current node. That is, factor (c') can have three outcomes: not planar, has the same planar position as the current node 900, or has a different planar position than the current node 900. If the current node 900 and the most recently occupied encoded / decoded node 906 have the same planar position, then the sub-volumes they occupy are aligned in a common horizontal plane at the same z-axis position.

[0167] Factor (d') refers to whether the current node 900 is located in the upper or lower half of its parent node 902 (in the case of horizontal planarity). Since the parent may be planar due to qualification requirements, if the current node 900 is planar, then a planar position is more likely to be "outside" the parent node 902 rather than towards the center. Therefore, the position of the current node 900 within its parent node 902 has a significant impact on the probability of a planar position within that current node 900.

[0168] In an implementation that combines all four factors, if the most recently occupied encoded node 906 at the same z-axis and the depth (as the current node 900) is planar, there may be 2 x 3 x 2 x 2 = 24 prediction combinations; otherwise, when the most recently occupied encoded node 906 at the same z-axis and the same depth is not planar, a specific context is used instead. Therefore, in this example, 24 + 1 = 25 contexts can be used to encode and decode the planar position flags using a binary arithmetic codec.

[0169] Although the above examples involve three factors for context determination in the case of planar pattern markers and four factors for context determination in the case of planar position markers, it should be understood that this application includes the use of each factor for context determination as well as all combinations and sub-combinations of such factors.

[0170] Now refer to Figure 10 The diagram illustrates an example implementation of a mechanism for managing the determination of recently occupied codec nodes during the context determination process. In this example mechanism, the codec device uses memory, such as volatile or persistent memory units, to implement a buffer 1000 containing information associated with occupied codec nodes. Specifically, the buffer 1000 allocates space to track codec nodes in the tree that have the same z-axis position and depth. In this particular example, the buffer 1000 tracks information associated with up to four codec nodes that have the same z-axis position and depth.

[0171] Each row of example buffer 1000 corresponds to the z-axis position and depth. The four columns correspond to the four most recently encoded occupied nodes with that z-axis position. For example, example row 1002 contains data associated with four encoded occupied nodes. The stored data for each encoded node may include the x and y positions of the encoded occupied node, regardless of whether the node is planar; if so, it is a planar position.

[0172] During the encoding / decoding of the current node 1004, the encoding / decoding device accesses buffer 1000 to identify the nearest occupied encoded / decoded node from the four storage nodes in example row 1002, based on the same z-axis position used for the current node 1004 in example row 1002. As discussed above, the distance metric can be based on the L1 norm, L2 norm, or any other distance metric. The x and y positions stored for each node in buffer 1000 assist in directly determining the nearest node, especially in the case of the L1 norm.

[0173] Once the nearest node is identified, such as nearest node 1006, its distance from the current node 1004, as well as its planarity and / or planar position, are used in the context determination(s). Buffer 1000 is then updated by adding the current node 1004 to the first position 1008 of buffer 1000 and shifting all other node data in this example row 1002 of buffer 1000 to the right, resulting in the last item in buffer 1000 being discarded. In some examples, based on distance determination, the identified nearest node may retain a high potential relevance to the current codec; therefore, before adding the current node 1004 to buffer 1000, the contents of example row 1002 are first rearranged to move the nearest node 1006 to the first position 1008 and shift the node to the right to accommodate it. For example, in this example, the node data in the first position 1008 and the second position are offset to the second and third positions, respectively. In this way, the codec device avoids prematurely excluding the most recently identified nearest node from buffer 1000.

[0174] It should be understood that the described buffer 1000 is an example implementation of a mechanism for managing data associated with the most recent node, but this application is not necessarily limited to this example, and many other mechanisms for tracking the most recent node information can be used. Furthermore, it should be understood that retaining only a fixed number of recently occupied codec nodes in the buffer 1000 means that the identified node may not actually be the most recently occupied codec node, but merely the most recently obtained codec node from the buffer 1000; however, even when the buffer is limited to four candidates, as shown in the example above, the impact on performance is negligible.

[0175] Occupied mode angle mode encoding and decoding

[0176] Next, specific modes for encoding and decoding point clouds acquired (e.g., sampled) by beam combinations including beam sets will be described. For example, a beam can be a light beam. Without limitation, a lidar is used as an example of such a beam combination for acquiring point clouds; however, it is understood that other beam combinations are also feasible and are understood to be covered by this description. Similarly, without limitation, a laser beam is used, but other (optical) beams are also feasible and are understood to be covered by this description. For example, beam combinations can involve any rotating beam combination, including but not limited to lidar. Alternatively, beam combinations can be associated with scanning beam combinations or solid-state lidar. In any case, these beam combinations are assumed to emit beams at certain predefined (and known) angles (particularly elevation angles). It should be understood that any beam combination described throughout this application can be mounted, for example, on a car or any other vehicle.

[0177] Typically, point clouds acquired by lidar are obtained using spin laser beams. An example of such spin laser beams is... Figure 11 The diagram is schematically illustrated. The most common arrangement is an array (e.g., 16, 32, or 64 lasers) of vertically arranged lasers 1104 emitting corresponding laser beams 1106. The lasers 1104 are mounted on a spin head 1101 that rotates about a rotation axis 1102. Alternatively, the lasers 1104 themselves can rotate about the rotation axis 1102. The head 1101 including the lasers 1104 may be referred to as a beam combiner.

[0178] Naturally, the point acquired by a specific laser beam has a fixed (or fixed to noise and / or measurement accuracy) vertical angle θ (e.g., elevation angle) relative to a plane perpendicular to the head axis. This is in Figure 12 The diagram schematically illustrates that laser 1204 emits laser beam 1206 at an acquisition angle (e.g., elevation angle) θ relative to a plane 1208 perpendicular to head axis 1202. Figure 13 The possible definitions of the coordinate axes for head 1301 are shown.

[0179] Figure 14 The r–tan(θ) graph shows points in a point cloud obtained by an example lidar with 64 laser beams. Some fixed laser angles tan(θ) are clearly visible in the graph. Positive angles indicate the lasers are pointing above the horizontal plane (i.e., towards the sky), while negative angles indicate the lasers are pointing below the horizontal plane (i.e., towards the ground). It is noteworthy that the lidar lasers are intentionally directed at different angles—for example, so that different lasers pick up different parts of objects of the same height (such as buildings, utility poles, trees, etc.). This is determined by the constraints and as... Figure 15As shown, r is the horizontal distance from point 1505 to lidar head 1301 (or a typical beam combination), and the tangent of the vertical angle tan(θ) is defined as the ratio z / r, where z is the vertical elevation angle of point 1505 relative to lidar head 1301.

[0180] from Figure 14 It can be seen that points in a point cloud can be represented in a “quasi” 2D space. For example, it is theoretically possible to represent them in 2D space (x,y) and a finite set of perpendicular angles θ corresponding to the laser beam. Or equivalently, in 2D polar coordinate space (φ,r), where φ is the azimuth and the perpendicular angle θ is a finite set. In other words, points can be represented in spherical coordinates (φ,r,θ), where θ can only take a finite number of values. This is a consequence of occlusion, resulting in at most one point per (φ,θ). This quasi-2D space is theoretical, but in practice, measurement noise / accuracy and time integration do not result in a strictly finite set of possible θ values, a consequence of the mechanical mounting of the laser. Instead, the observed values ​​of θ are spread across a finite number of values ​​θ. laser The surrounding area must be considered, and the "noise" around these values ​​must be addressed.

[0181] The aforementioned quasi-2D structure primarily depends on the position / orientation of one or more lidar units (or beam combinations in general) and the inherent characteristics of lidar units(s). As will be described in more detail below, it may be advantageous to include syntactic elements (e.g., parameters, flags) in the bitstream describing the characteristics of lidar units(s) (or beam combinations / combinations in general). These syntactic elements may indicate at least one of the reference position of the beam combination, the origin position of a particular beam, or the beam angle of a particular beam. Including these syntactic elements in the bitstream can be done at the frame or sequence level. In any case, this data should be placed as close as possible to the compressed data bitstream to ensure the decodeability of the bitstream and minimize the risk of losing the data required for decodeability.

[0182] Note the above; some embodiments of this application seek to use the vertical laser angle {θ} in the efficient representation of point clouds. laser The knowledge gained from the characteristics of lidar (or the characteristics of related beam combinations) allows for improved compression. Furthermore, since strong and robust compression capabilities have already been achieved through the use of octree representations and the proximity configuration of occupied nodes, some embodiments seek to utilize this knowledge in the context of encoding and decoding octree representations of point clouds.

[0183] One possible solution for leveraging the quasi-2D structure of point clouds is based on variations in the representation space. In these variations-based solutions, the high-level syntax used for encoding and decoding the point cloud can signal information about the spatial variations and the characteristics of the lidar in the varying space. A first drawback of this variation-based solution is the difficulty in constructing reversible discrete spatial variations without requiring additional precision in the varying space. Therefore, lossless compression of 3D point clouds is difficult to achieve good performance. An example of spatial variation is the representation of a point cloud in a monochrome image, where image frames belong to a bounded space [0, 2π] * [-π / 2, +π / 2] of pairs (φ, θ), and the intensity of a pixel is provided by a distance r. Of course, not all pixels of the "image" are occupied. Therefore, the point cloud can be viewed as a depth map in angular space (φ, θ). This raises several issues, such as the required precision to separate all points on φ and θ such that no two of them fall on the same pixel. Furthermore, it remains to be proven that this representation can be used efficiently to compress point clouds. This might require specific local predictions (e.g., benefiting from predictions around a fixed angle θ). laser (The scattering of θ) and other compression techniques. It is clearly incompatible with the octree representation of point clouds. Finally, the main drawback of 2D frame representation is its inability to handle point clouds acquired by more than one LiDAR. This is a very practical use case, as some manufacturers envision mounting multiple LiDARs on a vehicle (e.g., on the side of the vehicle) rather than a single LiDAR attached to the roof. Therefore, native 3D spatial representation seems to be the only reasonable representation to address multi-LiDAR acquisition.

[0184] Within the framework of an octree representation of point cloud geometry, this application proposes defining and using an angle θ associated with a point in the point cloud or a node in the octree to improve the compression of occupancy bits attached to a parent node and to signal the occupancy of child nodes of that parent node. The corresponding encoding / decoding mode using the angle θ associated with a point in the point cloud or a node in the octree will be referred to in this application as an angle encoding / decoding mode. According to the angle encoding / decoding mode, the angle θ associated with a point or node is related to a fixed angle {θ} associated with a beam (e.g., a laser beam). laser The comparison is then used to construct an estimate of the occupancy of the child nodes. In fact, the selection of the context for directly or indirectly (planar mode) encoding and decoding the occupancy bits of child nodes in a binary entropy encoder is performed based on the estimate. Possible implementations of this scheme will now be described in more detail below. Generally, it can be said that the angle encoding and decoding mode involves determining the context of the occupancy mode for entropy encoding the current volume (or node) based on angle information of the sub-sub-volumes (or child nodes) or groups of sub-sub-volumes, where the angle information indicates one or more angles (e.g., elevation angle) relative to the position associated with the beam combination providing the beam.

[0185] Let (x)head ,y head ,z head (x, y, z) is the position of the (LiDAR) head (or general beam combination), oriented vertically along the z-axis in the three-dimensional space XYZ (volume space) of the point cloud. The tangent of the vertical angle θ of the point (x, y, z) is the elevation angle z. head The ratio of the distance r of that point relative to the head.

[0186]

[0187] from Figure 16 The r–tan(θ) plot shows that, relative to Figure 14 In a detailed view, for points detected by the same beam, the angle tan(θ) is somewhat constant, but not strictly constant. In fact, from... Figure 16 Figures (b) and (c) show that for each beam, there exists a corresponding envelope in the (r, tan(θ)) plot, which contains the points acquired by that beam. The width of this envelope (in the tan(θ) direction) clearly depends on the distance r. This envelope width is due to noise and acquisition accuracy, and it widens as r decreases. Notably, the envelopes of adjacent beams do not intersect each other.

[0188] While assigning points to their corresponding envelopes already provides additional knowledge that can be used to improve compression performance, obtaining a corrected angle θ that exhibits more stable values ​​across a wide range of distances r is even more important. corr This would be advantageous. For example, this could be achieved by adding a correction term to the position of each beam emitter (e.g., a laser) relative to the head position (or typically relative to a reference position of the beam combination).

[0189] Figure 17 The schematic diagram illustrates the position of beam transmitter 1704 relative to reference position 1710 of beam assembly 1701, used to explain the correction angle θ to obtain the corrected angle θ. corr An example method. Specifically, the position (location) of the beam emitter (e.g., laser) L is determined by an offset (r) relative to a reference position 1710 (e.g., head position) of the beam combination. L ,z L )1712 is used for correction, so that the corresponding beam 1706 is from the vertical position z head +z L The emission is performed, and the distance (at the point) to the beam emitter laser is corrected to rr. L Therefore, the corrected position of the beam transmitter can be referred to as the origin position of the corresponding beam 1706 or the beam origin position. Correction angle θ corrDepends on the beam emitter index (e.g., laser index) L and is provided by the following formula

[0190] For points not too close to the beam combination (e.g., the head of a lidar system), where the distance r is not too small, this angle can be approximated by the following formula.

[0191]

[0192] And only vertical correction z is required. L And radial correction r L This can be ignored. Therefore, starting from the reference position of the beam combination, the beam origin position can be based on (r). L ,z L ) or just z L The relative displacement expressed is used to determine this.

[0193] Figure 18 Vertical correction z is shown L The effect on angle θ. Panel (a) corresponds to Figure 16 Panel (b) in the diagram, and panel (b) corresponds to... Figure 16 Panel (c) is shown in the diagram. The upper envelope in panel (a) and the lower envelope in panel (b) are related to the uncorrected (i.e., initial) angle θ. The lower envelope in panel (a) and the upper envelope in panel (b) are related to the corrected angle θ. corr Correlation. It can be clearly seen that the corrected angle exhibits a generally constant behavior that is essentially independent of the distance r. The angle θ of the corresponding beam or beam emitter (e.g., laser) L. L =θ laser It is limited to this constant.

[0194] To enable angle θ correction on both the encoder and decoder sides, the following values ​​can be determined or provided to the encoder and encoded as a bit stream, allowing the decoding process to use the same beam combination information (e.g., lidar information) as the encoding process:

[0195] ● For example, the reference position of the beam combination (e.g., the center position), such as the head position (x) of the lidar. head y head z head )

[0196] ● The relative position (r) of each beam emitter (e.g., laser) L ,z L )

[0197] ● A constant angle θ for each beam emitter (e.g., a laser). L

[0198] Examples of syntactic elements that describe the characteristics of one or more beam combinations of point clouds to be encoded or decoded will be described in more detail below.

[0199] As an example, the reference position of the beam combiner (e.g., the position of the lidar head) can be provided by the user, and the relative position and correction angle of the beam emitters (e.g., lasers) of the beam combiner can be found algorithmically. In one example, this can be done as follows:

[0200] 1. First, by analyzing the histogram of the uncorrected angle θ, points belonging to the same laser beam L are extracted.

[0201] 2. Then, the relative position (r) of the laser beam L. L ,z L By minimizing the correction angle θ at the extraction point corr,L The standard deviation is used to determine this. For example, this can be performed using a classic minimization algorithm. This involves only vertically offsetting z... L If taken into account, this is a simple minimization of the interval between candidate vertical relative positions.

[0202] 3. Finally, the constant angle θ of laser L L It is the correction angle θ at the extraction point. corr,L The average value.

[0203] Next, an example method for determining the correction angle of a specific point or volume, sub-volume, or group of sub-volumes in a point cloud will be described. Specifically, a method for determining the tangent tan(θ) of the correction angle of a point in a point cloud relative to a beam (e.g., a laser beam) L is described. corr,L The method is similar. Similarly, the corrected angle can be determined for a node (or its associated volume) of an octree by simply assimilating the node to the point at its center. A similar consideration applies to sub-volumes or groups of sub-volumes.

[0204] Regarding volumes (associated with the corresponding nodes in the octree), it is important to understand that there are conditions regarding the size and location of the volume to definitively determine which beam has acquired the volume (e.g., a point within the volume). Ambiguity arises if two beams intersect with the volume. To address this issue, this application proposes introducing the concept of (angular) qualification of volumes in angular encoding / decoding modes.

[0205] In one example, the angular compliance of a volume (or its associated nodes) is determined as follows:

[0206] 1. Calculate the reference position (e.g., center) of the volume and the reference position of the beam combination (e.g., the position of the lidar head (x)). head ,y head ,zhead The distance r)

[0207] 2. Calculate the magnitude of the tangent angle Δθ of the volume using Δθ = E / r, where E is a typical length of the volume (e.g., the length of the edge of a cube).

[0208] 3. If 2Δθ≤Δθ min,laser =min|tan(θ) L1 )-tan(θ L2 If the minimum value is used for all beam pairs, then the volume is acceptable, meaning the node's angular size is less than the angular increment between the two nearest beams. Factor 2 was chosen as a non-restrictive example to provide some security margin. Other factors (e.g., factors greater than or equal to 1) are also possible.

[0209] Figure 19 Examples of volume 1920 with acceptable angle and volume 1925 with unacceptable angle are shown. That is, the size of volume 1925 and its distance from beam combination 1901 are such that it is sampled by multiple beams 1906 from multiple beam transmitters 1904. On the other hand, the size of volume 1920 and its distance from beam combination 1901 are such that it is sampled by at most a single beam 1906.

[0210] Typically, determining the angular compliance of a volume (or its associated nodes) can be done by first determining a metric of the angular size of the volume as seen from a reference position of the beam combination. As mentioned above, the reference position can be the location of the head. The metric of the angular size can be related to the angle seen from the reference position of the volume edge and can be determined, for example, based on the length of the volume edge and the distance between the volume (and its reference position) and the reference position of the beam combination. Therefore, the metric of the angular size can be, for example, the angular size Δθ defined above. The metric of the angular size is then compared with a metric of the difference angle between adjacent beams of the beam combination, such as the difference angle Δθ defined above. min,laser That is, the difference angle can be the difference angle between the closest adjacent beams in the beam combination. Finally, the angular adequacy of the volume is determined based on the comparison results. For example, angular adequacy can be determined based on whether the angle size is less than half of the difference angle (or any other suitable portion of the difference angle to provide an appropriate safety margin). As will be described in more detail below, the angular adequacy of a volume may imply that the context of the occupancy mode used for encoding / decoding volumes (sub-volumes) can be determined or selected based on angular information.

[0211] Once the angular compliance has been determined, the beam L belonging to the angularly compliant volume can be determined, for example, by the following methods:

[0212] 1. Determine the position relative to the beam combination (x, y, z) from the reference position (x, y, z) of the volume (e.g., the center of the volume). head ,y head ,z head z and r

[0213] 2. Calculate the uncorrected angle tan(θ) = z / r

[0214]

[0215] 3. By finding |tan(θ) - tan(θ) L The smallest beam index L determines which nearest beam L a volume / point belongs to.

[0216] Once the beam (e.g., laser beam) L has been determined, the corrected angle can be calculated using the following formula.

[0217]

[0218] or

[0219]

[0220] The second equation may actually be useful because it avoids calculating the reciprocal of the other square root.

[0221] Typically, the corrected angle θ of the volume associated with a node of an octree can be determined by following these methods. corr Initially, a first angle (e.g., elevation angle) is determined relative to a reference position within the volume. For example, the reference position within the volume could be the center of the volume. Similarly, the reference position of the beam combination could be the center of the beam combination. Based on the first elevation angle, a specific beam (or corresponding beam transmitter) L, assumed to have been acquired (or actually acquired), is determined as a point within the volume. Next, the origin position of this specific beam (beam origin position) is determined or acquired. This can be accomplished using any suitable method described above. If the origin position of the specific beam is available, the corrected elevation angles of a sub-volume or group of sub-volumes (e.g., a plane within the volume) can be determined relative to the origin position of the specific beam.

[0222] Figure 20An example of a method 2000 for encoding a point cloud acquired by means of a beam combination including a beam set to generate a bitstream of compressed point cloud data is illustrated in flowchart form. The point cloud is defined in a tree structure (e.g., an octree structure) with multiple nodes having parent-child relationships and representing the geometry of the volume space, as described elsewhere in this application. The point cloud resides within a volume space of points recursively divided into sub-volumes and containing point cloud data. The point cloud may be defined relative to a Cartesian axis in the volume space, which may be a vertically oriented z-axis orthogonal to the horizontal plane and substantially parallel to the principal axis or principal orientation axis (e.g., rotation axis) of the beam combination (e.g., lidar). In the volume space, a volume is divided into a set of sub-sub-volumes, and the volume is associated with a node of the tree, whose sub-volumes are associated with the child nodes of that node. An occupancy bit associated with each corresponding sub-sub-volume indicates whether the corresponding sub-sub-volume contains at least one point. It can be said that the occupancy bit of the sub-sub-volume defines the occupancy pattern. In method 2000, operations 2010 to 2040 are performed on the current node associated with the volume that has been split into sub-sub-volumes.

[0223] In operation 2010, the occupancy pattern of a volume is determined based on the occupancy status of its sub-sub-volumes. Operations 2020 to 2040 involve encoding the representation entropy of the occupancy pattern into a bit stream. In operation 2020, angular information of the sub-sub-volumes or sub-sub-volume groups of a volume is determined. This angular information indicates one or more angles (e.g., elevation angles) relative to a position associated with a beam combination within the volume space. When viewed from the position associated with the beam combination, the elevation angle can indicate the elevation angle of the corresponding sub-sub-volume or sub-sub-volume group above a plane substantially orthogonal to the principal direction (e.g., principal axis) of the beam combination (such as the common rotation axis of a beam set, such as the rotation axis of a lidar). In some embodiments, this plane can be a horizontal plane (e.g., the xy plane). For example, these angles can be expressed as their tangents. The position associated with the beam combination can be the (reference) position of the beam combination or the origin position of a specific beam of the beam combination. In operation 2030, the context for encoding the representation of the occupancy pattern is determined based on the angular information. Finally, in operation 2040, the occupancy pattern representation is entropy encoded using the determined context.

[0224] Figure 21 An example of method 2100 for decoding a bitstream of compressed point cloud data to generate a reconstructed point cloud is illustrated in flowchart form. Consistent with method 2000, it is assumed that the point cloud has been acquired using beam combining, which includes a beam set. Furthermore, the point cloud is assumed to be defined in the same manner as described above for method 2000. Operations 2110 to 2130 are performed on the current node associated with the volume split into sub-sub-volumes.

[0225] In operation 2110, angular information of the sub-sub-volumes or groups of sub-sub-volumes of the volume is determined. This angular information indicates one or more angles (e.g., elevation angle) relative to a position associated with a beam combination within the volume space. In operation 2120, the context for decoding the representation of the volume's occupancy pattern is determined based on the angular information. As mentioned above, the occupancy pattern indicates the occupancy status of the volume's sub-sub-volumes. Finally, in operation 2130, the bitstream is entropy decoded to generate a reconstructed representation of the occupancy pattern using the determined context.

[0226] As mentioned above, angle information can be related to corrected angles (e.g., corrected elevation angles), which, in a sense, are measured with reference to the beam origin position of a specific beam that is assumed to have been acquired (e.g., sampled) a point in the current volume. Therefore, determining angle information can include: determining a first (e.g., initial) elevation angle of a reference position within the volume relative to the reference position of the beam combination. Once this first elevation angle is available, it can be used to determine a specific beam that can be assumed to have been acquired within the volume. Knowing the specific beam, its origin position can be determined, for example, by using a lookup table or by an algorithm. Finally, one or more elevation angles of sub-sub-volumes or groups of sub-sub-volumes of the volume can be determined relative to the origin position of the specific beam. These elevation angles will then correspond to corrected elevation angles.

[0227] Next, an embodiment using angular information to determine the context for entropy encoding of planar position markers in the planar encoding / decoding mode described above will be described.

[0228] It has been observed that directly encoding and decoding planar information (i.e., planar pattern flags (e.g., the flag isPlanar) and planar position flag bits for planar positions (e.g., the flag planePosition)) without compression can lead to inefficient planar patterns. Therefore, compressing the syntax associated with the planar patterns can prove advantageous. As mentioned above, such compression of the planar position flags can be achieved by selecting the context of a binary entropy encoder for the planar position flags based on a prediction method.

[0229] For z-plane mode, if a node is planar (isZPlanar == true), its planar position (bottom or top) must be signaled by an additional bit, planeZPosition. This bit is also encoded and decoded using a binary arithmetic codec, and its context depends on the prediction information, such as... Figure 9 As shown. This predictive information consists of any, some, or all of the following:

[0230] ● The occupancy of the vertical neighbors of the parent node and the current node.

[0231] ● The distance d to the nearest encoded / decoded node at the same z and the same depth, where d can be discretized into three values: "near", "not too far", or "far".

[0232] ● The planar position (bottom or top, if any) of the nearest encoded / decoded node with the same z-axis and the same depth (same as the current node), and

[0233] ● The vertical coordinate z of the current node relative to its parent node.

[0234] Using all the above prediction information, if the most recently encoded node at the same z-axis and depth (same as the current node) is planar, this results in 2 × 3 × 2 × 2 = 24 possible prediction combinations; otherwise, when the most recently encoded node at the same z-axis and depth is not planar, additional specific context will be used instead. Therefore, a binary arithmetic codec can use 24 + 1 = 25 contexts to encode and decode planar locations.

[0235] This embodiment proposes to enhance or replace the prediction information of bit planeZPosition by using the angle information (e.g., tan(θcorr,L)) of nodes with qualified angles.

[0236] Therefore, the context determination in operation 2030 or operation 2120 of this embodiment may depend on whether the volume is (determined to be) a planar volume. To this end, methods 2000 and 2100 may include corresponding determination steps. As described above, a planar volume is a volume divided into multiple sets of sub-volumes in a corresponding parallel plane, where all sub-volumes containing at least one point lie in the same plane. A parallel plane can be understood as parallel to a horizontal plane or orthogonal to the main axis of the beam combination (e.g., the rotation axis of a lidar head). In the case of an octree, for example, there would be a first set of four sub-volumes and a second set of four sub-volumes, and a planar position marker would indicate whether the sub-volume containing the point is located in the first or second set.

[0237] If the volume is (determined to be) a planar volume, then determining the context for encoding and decoding the representation of the occupancy pattern in operation 2030 or operation 2120 includes determining the context for encoding and decoding the planar position marker based on the angle information defined above. This planar position marker (e.g., the marker planeZPosition) signals the position of the plane containing the sub-volume containing at least one point. Furthermore, entropy encoding the representation of the occupancy pattern in operation 2040 or operation 2130 includes entropy encoding the planar position marker using the determined context.

[0238] In some implementations, the prediction information given above can be replaced, partially replaced, or enhanced by angle information. Therefore, determining the context for encoding and decoding the planar position markers can be based at least in part on one or more of the following: (a') the occupancy of neighboring volumes at the depth of the volume's parent volume, (b') the distance between the volume and the most recently encoded occupied volume, (c') the planar position of the most recently encoded occupied volume (if any), or (d') the position of the volume within the parent volume, excluding the angle information. The distance can be near, not too far, or far, and is based on calculating a distance metric and comparing it with a first threshold and a second threshold. Typically, determining the context for encoding and decoding the planar position markers can be based on a combination of three or more of (a'), (b'), (c'), and (d') and the angle information.

[0239] The angular information used to determine the context may include the corresponding elevation angle of each of the multiple parallel planes of the volume relative to the origin position of a specific beam already determined for the volume. Therefore, determining the angular information may include determining these elevation angles of the respective planes. The elevation angle may be determined with respect to a reference position (e.g., a center position) within each parallel plane relative to the origin position of the specific beam. Determining the context for encoding and decoding the plane position markers may include comparing the corresponding elevation angle of each of the multiple parallel planes with the beam angle θ of the specific beam, assuming the volume has already been sampled. L (or tan(θ) L The comparison is performed, and the context for encoding and decoding the planar position markers based on the results of the comparison is determined.

[0240] For the octree case, the above is equivalent to calculating two angles for the bottom and top virtual planes respectively; that is, the bottom and top corrected vertical angles. In some implementations, the determination of the two angles may depend on whether the node's child nodes are angularly qualified. These two angles correspond to the corrected angle tan(θ) associated with the point located at the center of the two bottom and top virtual planes. bottom,L ) and tan(θ top,L ), and as Figure 22 As illustrated in the figure, the angularly compliant volume 2220 has an upper plane 2230 (corresponding to a sub-volume set) and a lower plane 2240 (corresponding to a sub-volume set). When viewed from the origin position of the beam transmitter 2204 transmitting beam 2206, the beam 2206 has sampling points in volume 2220, with a correction angle tan(θ). top,L The elevation angle of the center position 2232 of the upper plane 2230 above the horizontal plane 2208 is measured. Similarly, when viewed from the origin position of the beam transmitter 2204, the correction angle tan(θ) is calculated. bottom,L) Measure the elevation angle of the center position 2242 of the lower plane 2240 above the horizontal plane 2208.

[0241] Once the elevation angle of the virtual plane of the volume is tan(θ) top,L ) and tan(θ bottom,L These values ​​are calculated and can then be used to determine the context for entropy encoding of planar position markers. For example, determining the context can be based on at least one of the following: the beam angle tan(θ) of a particular beam. L The quantization result is the difference between the difference between the beam angle of a specific beam and the corresponding elevation angle of each of the multiple parallel planes, the magnitude of the difference between the beam angle of a specific beam and the corresponding elevation angle of each of the multiple parallel planes, and the magnitude of the difference between the largest and smallest of the magnitudes of the differences between the beam angle of a specific beam and the corresponding elevation angle of each of the multiple parallel planes. In some embodiments, this may involve comparing the differences between the beam angle of a specific beam and the corresponding elevation angle of each of the multiple parallel planes with each other and / or comparing the magnitudes of the differences between the beam angle of a specific beam and the corresponding elevation angle of each of the multiple parallel planes with each other. For example, the quantization may be ternary quantization.

[0242] In the case of octrees, the context used for entropy encoding of planar positional markers can use two angles tan(θ). bottom,L ) and tan(θ top,L (This may be determined along with the prediction information (a') to (d') given above.

[0243] For example, the angle context index can be calculated from the bottom / top vertical angle tan(θ) as follows: bottom,L ) and tan(θ top,L )Sure:

[0244] ● The first element of the context index depends on the difference tan(θ) L )-tan(θ bottom ) and tan(θ L )-tan(θ top Do they have the same symbol?

[0245] ●The second position of the context index depends on |tan(θ) L )-tan(θ bottom )|and|tan(θ L )-tan(θ top Which of the following has the highest amplitude?

[0246] ●The context index also depends on |tan(θ) L )-tan(θbottom )|and|tan(θ L )-tan(θ top The quantization of the difference D = max - min in | is performed (e.g., ternary quantization). Quantization can be performed relative to the inverse radius r associated with the node center; that is, the value D*r can be quantized.

[0247] In the pseudocode, the angle context index in the example above can be determined as follows:

[0248]

[0249] In the example above, the quantity D*r is quantized into four values ​​{0, 1, 2, 3} corresponding to the value of D*r, which are less than 0.5, between 0.5 and 1, between 1 and 1.5, or greater than 1.5. Therefore, the angular context index can use 2*2*4 = 16 values.

[0250] As mentioned above, the angle context can replace the prediction information used in the plane mode context determination (e.g., prediction information (a') to (d')), or enhance the prediction information to create a new (i.e., a larger) plane / angle context set.

[0251] Next, an embodiment of using angle information to determine the context of the occupancy bits of the sub-sub-volumes used for entropy coding will be described.

[0252] The occupancy of child nodes (sub-sub-volumes) of the current node (current volume) is signaled by corresponding occupancy bits, which are encoded and decoded by an arithmetic entropy encoder, where the "context" is selected based on the occupancy of neighboring nodes. In the case of an octree, the occupancy of the eight child nodes is signaled by eight corresponding occupancy bits.

[0253] Therefore, in this embodiment, determining the context for encoding and decoding the representation of the occupancy pattern includes determining the context for encoding and decoding the occupancy bits of the sub-sub-volumes of the volume. Further, entropy encoding the representation of the occupancy pattern includes entropy encoding the occupancy bits using the determined context.

[0254] Typically, the context selection of occupants is performed on the arithmetic codec itself through a so-called OBUF mechanism. This mechanism provides indices of binary entropy encoders from a (very) finite number of codecs selected for encoding and decoding occupants. For example, the index of the selected codec can be obtained based on the following information:

[0255] ● The value of the bits previously occupied by encoding / decoding

[0256] ●Nearby information,

[0257] ● Inter-frame / intra-frame prediction.

[0258] In practice, the index can be provided by the output of a lookup table (LUT) whose entries reflect the above information. Typically, each occupancy bit can be encoded or decoded by an entropy encoder selected from a predefined set of entropy encoders for that occupancy bit.

[0259] This embodiment proposes determining the context of the occupied bit based at least in part on angle information. In some implementations, this may be equivalent to modifying the OBUF LUT entry indexed by the codec by adding the angle information to the information list used by the entry.

[0260] Figure 23 An example of determining the contextual angle information used to determine the occupancy bits of child nodes for entropy coding of the current node is shown. For an angle-qualified current volume (associated with the current node) 2320, once the specific beam L 2306, assumed to have been acquired (e.g., sampled), has been determined, the correction angle tan(θ) is calculated. j,L For each child node j 2350 of the current node (e.g., eight child nodes in the case of an octree), these angles are calculated. These angles can be calculated with respect to the center position 2352 of the child node (or its associated sub-volume). As mentioned earlier, these angles can be the elevation angles relative to the horizontal plane 2308 when viewed from the origin position of beam 2306 (or equivalently, the position of beam transmitter 2304). Then, for each child node j, the angular distance Δ to the specific beam L is calculated. j It can be determined by the following formula

[0261] Δ j =|tan(θ) L )-tan(θ j，L )|

[0262] Typically, in this embodiment, determining the angle information in operation 2020 or operation 2110 includes determining the corresponding elevation angle for each sub-sub-volume of the volume relative to the origin position of a specific beam. Then, the context for encoding and decoding the occupancy bits of the sub-sub-volumes of the volume can be achieved by comparing the corresponding elevation angle of each sub-sub-volume of the volume with the beam angle of the specific beam (e.g., by calculating the difference or angular distance Δ between the corresponding angles). j ), and determine the context for encoding and decoding the occupancy bits of the sub-sub-volumes of the volume based on the results of the corresponding comparison.

[0263] Used in context determination for comparison results (e.g., angular distance Δ) j Two example methods (such as enhancements to OBUF LUT entries) will be described below.

[0264] In the first example, selecting a codec (context) for a occupancy bit can involve the difference Δ between the elevation angle of each sub-sub-volume in the volume-based sub-sub-volumes and the beam angle of a particular beam. j The corresponding metrics are used to sort the occupants. Then, the corresponding codec (context) can be selected based on the sequential index of the corresponding occupants in the sub-volume.

[0265] Specifically, the method described in the first example can be followed as follows:

[0266] 1. Depends on Δ j The values ​​are sorted for the sub-volumes, for example, from smallest to largest. Therefore, each sub-volume j is assigned an index 0 ≤ o(j) ≤ 7.

[0267] 2. Optionally, the sequential index o(j) may be restricted by a value less than 7.

[0268] 3. The sort index o(j) is directly used for the OBUF LUT entries, expecting a smaller index (i.e., a smaller Δ in this example). j This leads to a higher probability of occupancy.

[0269] In the second example, selecting a codec (context) for a occupant bit can involve the elevation angle θ of each sub-sub-volume in the sub-sub-volumes that quantize the volume. j,L Beam angle θ with a specific beam L L The difference Δ between j The corresponding metric is used to select the appropriate codec (context) based on the corresponding quantization value of the difference metric.

[0270] Specifically, the method described in the second example can be followed as follows:

[0271] 1. For each sub-volume j, the quantization value Δ j (or Δ) j *r) to obtain the discrete value Q(Δ) j The quantized value can be restricted to a given threshold.

[0272] 2. Quantized value Q(Δ) j ) can be directly used for OBUF LUT entries, with the expectation that smaller values ​​will result in a higher probability of occupancy.

[0273] It should be understood that any statements regarding angles in this application also apply to the tangent of an angle, and vice versa. Specifically, calculations involving the tangent of a given angle can also be performed based on the angle itself, and vice versa, possibly after appropriate trigonometric transformations. In some implementations, using the tangent of an angle may be advantageous to avoid using expensive trigonometric functions or their approximations.

[0274] The influence of volume shape on angle determination

[0275] The shape of an angularly appropriate volume affects the ability of angular information to distinguish between occupied and unoccupied sub-volume sets. More precisely, when considering the elevation angle used to determine the context of the entropy encoder, a volume stretched along the vertical direction tends to provide better distinction as the entropy encoder encodes and decodes occupancy information representing the occupancy of a sub-volume set.

[0276] Now refer to Figure 29 Each panel (a), (b), and (c) is Figure 22 The two-dimensional representation (in the plane (r,z)). Here, the two-dimensional representation is for simplicity, to represent the shape of the angularly qualified volume 2920 relative to the elevation angle tan(θ) obtained by measuring the elevation angle above the horizontal plane 2908 from the origin position of the beam transmitter 2904 at the center position 2932 of the upper plane (and the center position 2942 of the lower plane respectively). top (respectively tan(θ)) bottom,L The emitted beam 2906 has a sampling point 2905 in volume 2920. Panel (a) depicts volume 2920 as a substantially cubic shape, panel (b) depicts volume 2920 as elongated in the radial direction (relative to the main axis of the lidar), and panel (c) depicts volume 2920 as elongated in the direction of the lidar's main axis. One should consider volumes 2920 covering substantially the same cubic metric to compare the effect of their shapes on angle determination. Geometrically, volumes 2920 covering substantially the same cubic metric with a higher ratio H / W (where H is the volume height along the main axis and W is the volume width along the radial direction) result in a greater effect on the beam angle tan(θ). L tan(θ) top ) and tan(θ bottom,L The higher angular distinction is used to predict the occupancy set of sub-volumes. Therefore, when using angular mode encoding and decoding, it may be advantageous to construct a tree such that the volume associated with the nodes of the tree has a ratio H / W that is strictly greater than 1.

[0277] In the first example, the high ratio H / W of the volume associated with the nodes of the tree representing the geometry of the point cloud can be obtained by starting with the initial volume (or bounding box) associated with the root node (which itself has a high ratio H / W) and then recursively splitting that initial volume according to the tree structure (such as an octree structure or any other tree structure) such that all sub-volumes associated with child nodes have the same ratio H / W until the minimum size is reached.

[0278] In the second example, a high ratio H / W of the volume associated with a node of a tree representing the geometry of the point cloud can be achieved by starting with an initial volume (or bounding box) associated with the root node (with any arbitrary ratio H / W) and then recursively splitting that initial volume along radial directions (e.g., along the x and y axes), such as sub-volumes associated with child nodes, by satisfying the ratio condition H / W >= r > 1, where r is a given ratio parameter, and finally recursively continuing to split after the tree structure (e.g., an octree structure or any other tree structure) such that all subsequent sub-volumes maintain the same ratio condition H / W = r, until the minimum sub-volume size is reached. For example, the minimum sub-volume size can be reached when the radial (e.g., x-size and / or y-size) size of the sub-volume has become the minimum acceptable size (e.g., size 1), and further splitting of said sub-volume can only be performed along the principal axis (e.g., the z-axis) without guaranteeing the ratio condition. Further splitting can be performed until the sub-volume size becomes minimum (e.g., size 1 along all three axes).

[0279] Without loss of generality, the dimensions H and W associated with the volume represent the height (along the principal axis of the head) and the width along the radial axis of the volume. In practice, H can be the length of a segment obtained by projecting the volume onto the principal axis (e.g., the z-axis), such that H is well-defined for any volume with a convex shape. Similarly, W can be the length L of two segments obtained by projecting the volume onto an axis orthogonal to the principal axis (radial axis, e.g., the x and y axes). x and L y The maximum value in W makes W well-limited for any volume with a convex shape.

[0280] In the second example embodiment, the ratio condition H / W = r > 1 is obtained when the initial volume, which may be a (rectangular) cuboid (e.g., shoebox-shaped), is recursively split using binary tree (BT) and / or quadtree (QT) partitions along radial axes (e.g., x and y axes). By definition, a BT partition along a radial axis splits the volume into two sub-volumes by cutting the volume with a plane orthogonal to the radial axis (x or y). A QT partition along a radial axis splits the volume into four sub-volumes by cutting the volume with two planes, each plane being orthogonal to one of the radial axes. In a particular embodiment, BT and QT partitions can be performed such that the splitting planes are parallel to the faces of the cuboid volume, and such that the sub-volumes of the volume are all cuboids of the same size. The recursive splitting of the initial volume using BT and / or QT partitions along the radial axes is performed until cuboid sub-volumes with the desired H / W = r > 1 ratio are obtained.

[0281] In another embodiment, the initial volume is recursively split using BT and / or QT partitions along the radial axis until the cuboid sub-volume reaches the minimum radial size (e.g., both x and y sizes are 1), and then further partitioning is performed only along the principal axis (e.g., the z-axis) until the subsequent cuboid sub-volume reaches the minimum overall size (e.g., the size is 1 along all three axes).

[0282] In another embodiment, the initial volume is recursively split using only BT partitions along one radial axis (e.g., the x-axis or y-axis) until the resulting cuboid sub-volumes have equal sizes in both radial directions (e.g., the size in x is equal to the size in y). Then, a first further QT partition is performed along both radial axes (e.g., the x and y axes) until the subsequent cuboid sub-volumes reach the minimum radial size (e.g., both x and y sizes are 1). Then, a second further partition is performed only along the principal axis (e.g., the z-axis) until the subsequent cuboid sub-volumes reach the minimum overall size (e.g., the size is 1 along all three axes).

[0283] In another embodiment, the initial volume is recursively split using only BT partitions along a single radial axis (e.g., the x-axis or y-axis) until the resulting cuboid sub-volumes have equal sizes in both radial directions (e.g., the size in x is equal to the size in y). Then, a first further QT partition is performed along both radial axes (e.g., the x and y axes) until the subsequent cuboid sub-volumes reach the desired ratio condition H / W = r > 1. Then, a second further partition is performed along all axes (e.g., x, y, and z) until the minimum radial size is reached. Finally, a third further partition is performed only along the principal axis (e.g., the z-axis) until the subsequent cuboid sub-volumes reach the minimum overall size (e.g., the size is 1 along all three axes).

[0284] In another embodiment of the second example, the target ratio r can be signaled in the bit stream. Both the encoder and decoder can split the volume associated with a node along the radial axis, for example, by using BT and / or QT splits, until the H / W ratio of the obtained sub-volume becomes at least r (e.g., H / W >= r). Once the sub-volume reaches the target ratio H / W >= r, the sub-volume can be further split using octree splits or any other tree split (e.g., binary tree splits along the main axis). The decoder can receive the target ratio r from the bit stream, decode the target ratio r, and use it for the initial volume splits. If the target ratio r is not present in the bit stream, the value of the target ratio r can be inferred as a predetermined value (e.g., r = 1), or the initial volume splits can begin directly with any tree split without any value for the sub-volume ratio H / W.

[0285] Angle Pattern Syntax

[0286] As mentioned above, angle information (e.g., tan(θ)) corr,L This can be used to improve the compression capabilities of point clouds, for example, represented by octrees. One issue that may arise in the context of angle pattern encoding / decoding is what information should be placed in a dedicated syntax within the bitstream to fully describe the quasi-2D space to which points in the point cloud acquired by beamforming (e.g., lidar) belong, and to aid in determining angle information. Another issue is where this information should be placed within the bitstream.

[0287] Since using the information provided by the syntax to improve the compression capability of the point cloud codec is an important goal, the proposed syntax should be compact enough not to unnecessarily negatively impact the size of the bitstream. Furthermore, the syntax should be flexible enough to describe complex acquisition scenarios involving multiple beam combinations with different positions and orientations (not necessarily vertical) (e.g., LiDAR). This embodiment proposes a syntax (e.g., a high-level syntax) to address the aforementioned problems and requirements.

[0288] First, the possible characteristics of beam combinations (such as lidar or lidar heads) will refer to Figures 24 to 26 Description. Without being intended to be limiting, reference will be made to lidar or lidar heads, but other embodiments of beam combinations used to acquire point clouds should be understood to be equally covered by this description.

[0289] Point clouds can be captured (e.g., acquired) by one or more lidar heads. For example, multiple lidars (lidar heads) can be attached to a vehicle, each of which acquires (e.g., samples, detects) a portion of the scene. In the case of a single lidar head, it is natural to align the 3D xyz space (e.g., the volume space mentioned above) with the lidar orientation. For example, the vertical axis z can be arranged parallel to the rotation axis of the lidar head, and the center of the lidar head can be located at the origin O = (0,0,0) of the xyz coordinates. However, this alignment should not be imposed. Worse still, it cannot be applied to more than one lidar when dealing with multiple lidars acquiring the same point cloud. Therefore, it is proposed to include at least the following lidar position information in the syntax:

[0290] ●Number of (LiDAR) heads

[0291] ●For each head

[0292] ○ Head position (x head ,y head ,z head )

[0293] ○ Head orientation (primary axis, primary orientation axis), such as the orientation of the head's rotation axis.

[0294] Now refer to Figure 24 The diagram schematically illustrates a lidar head 2401 (as a non-limiting example of beamforming). The center position 2410 of the lidar head 2401 and its principal axis (e.g., rotation axis) 2402 are also shown. The orientation of the lidar head 2401 may or may not be parallel to the xyz coordinate axes of 3D space, where the point cloud is defined. If not, the principal axis of the head (e.g., rotation axis) can be derived from two angles in spherical coordinates (e.g.,...). A (θ) or (single) 3D vector signals in the bit stream. It should be understood that the main axis 2402 is assumed to pass through the reference position 2410 of the beam combination.

[0295] To ensure syntactic compactness for individual lidar head use cases, it is advantageous to use a shortcut when signaling the head's location at the origin, i.e., x head =y head =z head =0, and / or the principal axis is parallel to one of the xyz coordinate axes. An example of this shortcut will be described below.

[0296] Regarding the beam (e.g., laser), in order to calculate the aforementioned corrected angle, it is proposed to include the following information in the syntax for each head:

[0297] ●For each head

[0298] ○ The number of beams (e.g., lasers) attached to or included by the head.

[0299] ○ For each beam

[0300] ■ Relative position of the beam (e.g., the relative position of the beam origin and the reference / center position of the head)

[0301] ■ Beam Angle

[0302] As discussed above, the relative position of the beams can be important for calculating the correction angle, which ensures that the angle of the point acquired from a given beam is constant and independent of the distance r from the head. The primary correction factor is the relative position d of the beam (beam origin) with respect to the head position along the principal axis (e.g., the rotation axis). L This is in Figure 25 As shown, the beam emitter (e.g., laser) 2504 of the emitting beam 2506 is displaced by a displacement vector 2512 from the center position 2510 of the head 2501. In this case, the displacement vector 2512 extends along the main axis (e.g., rotation axis) 2502 of the head 2501, such that the displacement vector 2512 can be determined by a distance d along the main axis 2502. LExpression. Therefore, the syntax can include the relative displacement or distance d of each beam transmitter of a given head. L Indication (e.g., the signed length of vector 2512).

[0303] However, in some implementations, the 3D(d) of the beam transmitter is involved. L ,r L More complex 3D corrections for relative positions can be used, such as... Figure 26 As depicted herein, the beam emitter (e.g., a laser) 2604 of the transmitting beam 2606 is displaced from the center position 2610 of the head 2601 by a relative displacement vector 2612. This relative displacement vector 2612 can be determined by a distance d along the principal axis. L and the radial distance r perpendicular to the principal axis L Expression. In view of this, it is proposed that in the case of relative position along the axis of rotation, i.e., r L =0, the syntactic structure should allow for compact expressions in order to signal d. L Instead of (d) L ,r L That's sufficient, and / or in the case of a relative position of zero, i.e., d L =r L =0, and the beam emitter is actually located at the head position. The latter case may occur on certain types of lidar, or if calibration has already been applied during the processing of the raw lidar data, or if the user does not know the relative positions of the (multiple) beam emitters and wants the default to zero.

[0304] Regarding the beam angle, because the compression algorithm may use the tangent of the angle instead of the angle itself, a signal is sent in the syntax to notify tan(θ). L ) instead of θ L This may be advantageous, as it avoids the need for trigonometric function calculations.

[0305] Finally, it may be advantageous to completely bypass the process of inferring information about beam combinations from the bit stream, for example if the angle encoding / decoding mode is not used, or if information related to the beam set of the acquired point cloud is not included in the bit stream.

[0306] Method 2700, proposed in view of the above considerations, Figure 27The diagram is presented in flowchart form. This method decodes a bitstream of compressed point cloud data to generate a reconstructed point cloud. The point cloud is assumed to be acquired using a beamset, for example, by sampling (e.g., probing) one or more objects using a beamset. Further, the point cloud is assumed to reside within a volumetric space of points recursively divided into sub-volumes containing the point cloud data. While this may be the case in some implementations, the point cloud data is not strictly confined to a tree structure (e.g., an octree).

[0307] In operation 2710, a first parameter indicating whether an angular mode should be used for encoding / decoding the point cloud data is decoded from the bit stream. An example of the first parameter is the angular_flag parameter defined below. In operation 2720, based on the decoded first parameter, it is checked whether an angular mode should be used for encoding / decoding the point cloud data. If yes (yes in operation 2720), the method proceeds to operation 2740. Otherwise (no in operation 2720), the method proceeds to operation 2730. In operation 2730, the point cloud data is decoded without using an angular mode, for example, without using angular information. In operation 2740, first angular information related to the beam angle of the beam set relative to the corresponding beam origin position is decoded from the bit stream. The first angular information may be related to parameters, flags, or other syntactic elements included in the bit stream. In operation 2750, for the current volume split into sub-volumes, second angular information related to the angle of the sub-volumes or sub-volume groups of the volume relative to the position associated with the beam set is determined. This can be done, for example, in the same or similar manner as operation 2110 described above. For example, all positions can be defined relative to volume space (such as a coordinate system relative to volume space). For example, the position associated with the beam set can be a reference position of the beam set (e.g., the center position of the beam combination) or a beam origin position among the beam origin positions. In operation 2760, the volumetric point cloud data is decoded based on first angle information and second angle information. This can involve selecting a context based on the first angle information and second angle information and performing entropy decoding on the point cloud data using the selected context. For example, decoding can be done in the same or similar manner as operation 2130 described above. For some applications, it is expected that the angle pattern will be systematically applied (e.g., for some automotive use cases), in which case angular_flag can be omitted from the bit stream, and it is inferred that the angle encoding and decoding pattern should be used. In other applications, conversely, it is expected that the angle encoding and decoding pattern will not be applied, in which case agular_flag can be omitted from the bit stream, and it is inferred that the angle encoding and decoding pattern is not used. The understanding of the inference can be defined in the point cloud codec's configuration file, for example, by describing which codec modes (and, for example, flags) are used or not. Furthermore, the angle mode can depend on the plane mode. If the configuration file or plane mode flag indicates that the plane codec mode should not be used, it means that the angle codec mode is also not used, and the relevant syntax element is not present in the bitstream.

[0308] Now refer to Figure 28The diagram illustrates a corresponding method 2800 for encoding a point cloud acquired using a beamset to generate a bitstream of compressed point cloud data. It should be understood that method 2800 is a counterpart to method 2700 and reflects the corresponding operations of that method. Therefore, also for encoding, the point cloud resides within a volume space of points recursively divided into sub-volumes containing the point cloud data.

[0309] In operation 2810, a first parameter indicating whether an angle pattern should be used to encode and decode point cloud data is encoded as a bit stream. In operation 2820, it is checked whether the point cloud data should be encoded and decoded using an angle pattern based on the first parameter. If yes (yes in operation 2820), the method proceeds to operation 2840. Otherwise (no in operation 2820), the method proceeds to operation 2830. In operation 2830, the point cloud data is encoded without using an angle pattern, for example, without using angle information. In operation 2840, first angle information related to the beam angle of the beam set relative to the corresponding beam origin position is encoded as a bit stream. In operation 2850, for the current volume split into sub-volumes, second angle information related to the angle of the sub-volumes or sub-volume groups of the volume relative to the position associated with the beam set is determined. In operation 2860, for the current volume, the point cloud data of the volume is encoded based on the first angle information and the second angle information. This may involve selecting a context based on the first angle information and the second angle information and using the selected context to entropy encode the point cloud data. For example, encoding can be performed in the same or similar manner as operation 2040 described above.

[0310] In view of the characteristics of the head (e.g., lidar) and beam (e.g., laser) required to calculate the correction angle, this application further proposes the following implementation example of operation 2740, namely, decoding the first angle information from the bit stream.

[0311] In one implementation example, decoding the first angle information includes: decoding a second parameter from the bit stream for a beam combination (e.g., a head) that includes at least some beams in the beam set, the second parameter indicating the number of beams in the beam combination. Based on this parameter, the number of beams in the beam combination can be determined. Then, for each of the plurality of beams indicated by the second parameter, one or more third parameters indicating the position of the beam origin of the corresponding beam are decoded. These steps can be performed for each of the plurality of beam combinations that collectively provide the beam set. The num_of_lasers parameter described below is an example of the second parameter.

[0312] On the encoder side, encoding the first angle information in this embodiment includes encoding a second parameter indicating the number of beams in the beam combination, and for each beam, encoding one or more third parameters indicating the position of the beam origin of the corresponding beam.

[0313] In the above implementation examples, one or more third parameters may include parameters indicating whether the beam origin position of the beam is expressed relative to a reference position of the beam combination, parameters indicating the coordinates of the beam origin position along the main axis of the beam combination, parameters indicating whether the beam origin position is (radially) shifted from the main axis of the beam combination, and parameters expressing the (radial) shift of the beam origin position from the main axis of the beam combination. One or more third parameters may also include parameters indicating the main axis of the beam combination.

[0314] Therefore, in this embodiment, decoding one or more third parameters indicating the beam origin position of a corresponding beam may include: decoding a parameter indicating whether the beam origin position of the beam is expressed relative to a reference position of the beam combination. The `laser_relative_position_to_head_flag` described below is an example of this parameter. If the beam origin position of the beam is expressed relative to a reference position of the beam combination, then a parameter indicating the coordinates of the beam origin position along the main axis of the beam combination is decoded. The `laser_relative_position_dL` parameter described below is an example of this parameter. Further, a parameter indicating whether the beam origin position is displaced from the main axis (radial) of the beam combination is decoded. The `relative_position_rL` flag described below is an example of this parameter. Then, if the beam origin position is displaced from the main axis (radial) of the beam combination, then a parameter expressing the displacement of the beam origin position from the main axis (radial) of the beam combination is decoded. The `laser_relative_rL` parameter described below is an example of this parameter.

[0315] On the encoder side, the above can correspond to encoding parameters indicating whether the beam origin position of the indicative beam is relative to the reference position of the beam combination. If the beam origin position is relative to the reference position of the beam combination, the parameter indicating the beam origin position along the main axis of the beam combination is encoded as a bit stream. Further, the parameter indicating whether the beam origin position is displaced (radially) from the main axis of the beam combination is encoded as a bit stream. Then, if the beam origin position is displaced (radially) from the main axis of the beam combination, the parameter expressing the (radial) displacement of the beam origin position relative to the main axis of the beam combination is encoded as a bit stream.

[0316] In this implementation example, it may also be necessary to decode the information used to determine or infer the reference position (e.g., center position) and orientation (e.g., main axis) of the beam combination. This can be done as described below for the corresponding implementation example.

[0317] In another embodiment that can be combined with the foregoing examples, decoding the first angle information may include decoding information (e.g., parameters) to infer or determine the number of beam combinations (e.g., heads) constituting the beam set. Therefore, in this embodiment, decoding the first angle information includes checking whether the bit stream includes a fourth parameter indicating the number of beam combinations that respectively include at least some beams in the beam set. If so, the fourth parameter is decoded from the bit stream. If the fourth parameter is absent or equal to zero in the bit stream, the number of beam combinations is inferred to be 1, i.e., all beams in the beam set are part of the same single beam combination (e.g., head). On the other hand, if the fourth parameter is equal to 1, a fifth parameter indicating the number of beam combinations minus 2 is decoded from the bit stream. The num_of_heads_flag described below is an example of the fourth parameter. The num_of_heads_minus2 parameter described below is an example of the fifth parameter.

[0318] On the encoder side, the above is equivalent to the following: If the number of beam combinations is 1, the fourth parameter indicating a single beam combination is encoded as a bit stream with an appropriate value (e.g., bit value), or the fourth parameter is not encoded as a bit stream. If the number of beam combinations is greater than 1, the fourth parameter is encoded as a bit stream, and its value does not indicate a single beam combination. Furthermore, the fifth parameter indicating the number of beam combinations minus 2 is also encoded as a bit stream.

[0319] In another embodiment that can be combined with one or more of the above embodiments, for a beam combination including at least some beams in a beamset, decoding the first angle information may include decoding information (e.g., parameters) to infer or determine a reference position (e.g., center position) of the beam combination. Therefore, in this embodiment, decoding the first angle information includes: for a beam combination including at least some beams in a beamset, checking whether the bit stream includes a sixth parameter indicating whether the reference position of the beam combination is at the origin O = (0,0,0) of a coordinate system in volume space. If so, the sixth parameter is decoded from the bit stream. If the sixth parameter is not present in the bit stream, or if the sixth parameter indicates that the reference position of the beam combination is at the origin O of the coordinate system, then the origin O of the coordinate system is considered the reference position of the beam combination. Otherwise, the parameter indicating the coordinates of the reference position of the beam combination is decoded from the bit stream. The `head_absolute_is_000_flag` described below is an example of the sixth parameter. The head_absolute_x, head_absolute_y, and head_absolute_z parameters described below are examples of parameters that indicate the coordinates of the reference position of the beam combination.

[0320] On the encoder side, the above is equivalent to the following: If the reference position of the beam combination is at the origin O of the coordinate system, the sixth parameter is encoded as a bit stream, where an appropriate value (e.g., a bit value) indicates this fact, or the sixth parameter is not encoded as a bit stream. If the reference position of the beam combination is not at the origin O of the coordinate system, the sixth parameter is encoded as a bit stream, where an appropriate value (e.g., a bit value) indicates this fact, and the parameter indicates the coordinates of the reference position of the beam combination.

[0321] In another embodiment that can be combined with one or more of the above-described embodiments, for a beam combination including at least some beams in a beamset, decoding the first angle information may include decoding information (e.g., parameters) for inferring or determining the orientation (e.g., the main axis) of the beam combination. Therefore, in this embodiment, decoding the first angle information includes: for a beam combination including at least some beams in a beamset, decoding a seventh parameter from a bit stream, the seventh parameter indicating whether the main axis of the beam combination is arranged along a coordinate axis in volume space. If the seventh parameter indicates that the main axis of the beam combination is arranged along a coordinate axis in volume space, then a parameter indicating that the main axis of the beam combination is arranged along a specific coordinate axis is decoded from the bit stream. This specific coordinate axis is then considered as the orientation of the main axis of the beam combination. Specifically, the main axis may be considered to be parallel to the specific coordinate axis and passing through a reference position (e.g., the center position) of the beam combination. Otherwise, if the seventh parameter indicates that the main axis of the beam combination is not arranged along a coordinate axis in volume space, then a parameter indicating the angle expressing the orientation of the main axis of the beam combination is decoded from the bit stream, and the main axis of the beam combination is determined based on these parameters. The `head_parallel_to_axis_flag` described below is an example of the seventh parameter. The `axis_index` parameter described below is an example of a parameter indicating the main axis of the beamset along a specific coordinate axis along which it is arranged. Finally, the `head_orientation_phi` and `head_orientation_theta` parameters described below are examples of parameters indicating the angle expressing the orientation of the main axis. If the seventh parameter is not present in the bitstream, it is inferred that the main axis of the beamset is arranged along a coordinate axis in volume space.

[0322] On the encoder side, the above is equivalent to the following: If the beam combiner's main axis is arranged along the coordinate axes of the volume space, the seventh parameter is encoded as a bit stream, where an appropriate value (e.g., bit value) indicates this fact. Further, a parameter indicating that the beam combiner's main axis is arranged along the specific coordinate axis of its arrangement is also encoded as a bit stream. Otherwise, if the beam combiner's main axis is not arranged along the coordinate axes of the volume space, the seventh parameter is encoded as a bit stream, where a value (e.g., bit value) indicates this fact. Additionally, a parameter indicating the angle expressing the orientation of the beam combiner's main axis is also encoded as a bit stream. Alternatively, when the beam combiner's main axis is arranged along the coordinate axes of the volume space, only the parameter indicating the specific coordinate axis of the beam combiner's main axis is encoded as a bit stream.

[0323] In another embodiment that can be combined with one or more of the above-described embodiments, for a beam combination including at least some beams in a beamset, decoding the first angle information may include decoding information (e.g., parameters) to infer or determine the beam angle of the beam provided by the beam combination. Therefore, in this embodiment, decoding the first angle information may include: for a beam combination including at least some beams in a beamset, decoding an eighth parameter from the bit stream, the eighth parameter indicating the number of beams in the beam combination. The number of beams in the beam combination is then determined based on the eighth parameter. For each of the plurality of beams in the beam combination, one or more ninth parameters are decoded from the bit stream, indicating the beam angle of the corresponding beam. The num_of_lasers parameter described below is an example of the eighth parameter.

[0324] On the encoder side, the above is equivalent to the following: The eighth parameter is encoded as a bit stream to indicate the number of beams in the beam combination. For each of the multiple beams in the beam combination, one or more ninth parameters are then encoded as a bit stream, indicating the beam angle of the corresponding beam.

[0325] In this implementation example, decoding one or more ninth parameters indicating the beam angle of a corresponding beam may include: decoding a parameter indicating whether the beam angle is expressed as the tangent of an angle, and then decoding a parameter expressing the beam angle. The angle_is_tangent_flag described below is an example of a parameter indicating whether the beam angle is expressed as the tangent of an angle. The laser_angle parameter described below is an example of a parameter expressing the beam angle.

[0326] On the encoder side, if the beam angle is expressed as tangent, this is equivalent to encoding a parameter indicating whether the beam angle is expressed as the tangent of an angle into a bit stream, with an appropriate value for that angle (e.g., a bit value) indicating this fact, and then encoding the parameter expressing the beam angle. Directly encoding the parameter expressing the beam angle can be advantageous, in which case the beam angle parameter represents an angle.

[0327] Furthermore, in this embodiment, the use of angle information for encoding and decoding the point cloud data of the current volume can depend on whether the volume is angularly acceptable for encoding and decoding. Therefore, method 2700 and / or 2800 may include the following steps: first, determining whether (or indeed) the volume is acceptable for encoding and decoding the point cloud data of the volume based on first angle information and second angle information. This may involve: for a beam combination including at least some beams in a beam set, determining a measure of the angular size of the volume as seen from a reference position of the beam combination. The determined measure of angular size is compared with a measure of the difference angle between the angles of adjacent beams in the beam combination (e.g., the minimum difference angle between any pair of beams). The actual determination of whether the volume is acceptable for decoding the point cloud data of the volume based on the first angle information and second angle information can then be performed based on the result of the comparison. Typically, this can be done with reference to the above. Figure 19 The determination of the eligibility of the described angle is carried out in the same or similar manner.

[0328] As mentioned above, determining the second angle information in operation 2750 can be performed in the same or similar manner as in operation 2110. Similarly, determining the second angle information in operation 2850 can be performed in the same or similar manner as in operation 2020. Therefore, determining the second angle information may include: determining a first angle of a reference position within a volume relative to a reference position of a beam combination (including at least some beams in a beam set). For example, the reference position within a volume may be the center position of the volume. For example, the reference position of a beam combination may be the center position of a rotating beam combination. This reference position may be determined, for example, based on the first angle information described above. Based on a first elevation angle, a specific beam of a beam combination that is assumed to have acquired points within the volume is determined. For this specific beam, the beam origin position is determined, for example, based on the first angle information described above. Then, the beam origin position of the specific rotating beam can be used as the position associated with the beam combination defined in operation 2750 or operation 2850. Finally, the angle of a sub-volume or sub-volume group of the volume relative to the beam origin position of the specific beam is determined based on the beam origin position. Typically, this can be done in the same or similar way as the angle correction described above.

[0329] Given the aforementioned requirements for calculating correction angles, the syntax described in Tables 1 and 2 below is proposed. This ensures a compact syntax for the simplest case of a head located at the origin O and parallel to one of the x, y, or z axes. It also ensures a compact syntax for beams that do not require relative position correction or only require correction along the principal axis (e.g., the head rotation axis). In the tables below, u(n) indicates the use of an n-bit unsigned integer, and s(n) indicates the use of an n-bit signed integer. It has been observed that relative position (d... L ,r L(3 decimal places precision and angle θ) L or tangent tan(θ) L The 18-bit decimal precision is sufficient to achieve optimal compression performance when using beam angle.

[0330]

[0331]

[0332] Table 1

[0333] Alternative syntax for certain additional markers will allow for complete bypassing of the angle process, or for inferring additional values ​​for some syntactic elements. Examples of such syntax are shown in Table 2 below.

[0334]

[0335]

[0336] Table 2

[0337] The information used to signal in the proposed syntax does indeed depend on the physical characteristics of the beam combination (e.g., lidar) and its position / orientation within the scene represented by the point cloud (e.g., in a volumetric space containing the point cloud). However, it does not depend on the local structure of the point cloud. Therefore, the proposed syntax can be a high-level syntax at the frame or sequence level. In practical examples, the proposed syntax can be placed into...

[0338] ●Geometric Parameter Set (GPS)

[0339] ● Sequence Parameter Set (SPS)

[0340] ●Geometric Shape Slice Header

[0341] ●Out-of-band SEI messages

[0342] For example, these syntactic elements can be placed in any high-level syntax and header, such as equivalents of video parameter sets (e.g., in multi-functional video codecs, newer MPEG video codecs after HEVC), frame headers, or future sensor parameter sets.

[0343] Next, semantic examples to be used in the proposed syntax (such as those used in the table above) will be described.

[0344] An `angular_flag` of 0 indicates that header and beam (e.g., laser) information is absent from the syntax. `angular_flag` is an example of the first parameter defined above. An `angular_flag` of 1 indicates that header and beam information is present in the syntax. When equal to 0, the encoding / decoding method for nodes (e.g., volumes) does not use angular information.

[0345] A value of 0 for `num_of_heads_flag` indicates that the number of (LiDAR) heads is inferred as 1. If `num_of_heads_flag` is 1, it indicates that `num_of_heads_minus2` exists in the syntax, and the number of LiDARs / heads is equal to `num_of_heads_minus2+2`. When it does not exist in the bitstream, the number of LiDARs / heads is inferred as 1.

[0346] `head_absolute_is_000_flag` equal to 1 indicates that the absolute position of the head is inferred as x = y = z = 0. When `head_absolute_is_000_flag` equals 0, it indicates the existence of `head_absolute_x`, `head_absolute_y`, and `head_absolute_z` in the bit stream, signaling the absolute head position. When `head_absolute_is_000_flag` is not present in the bit stream, the absolute position of the head can be inferred as x = y = z = 0. The absolute value is interpreted as the coordinate in the xyz coordinate system represented by the point cloud.

[0347] head_absolute_x indicates the absolute position of the head (capture device) on the x-axis. For example, a 32-bit signed integer with 3 decimal places of precision makes the absolute x-position of the head head_absolute_x / 8.

[0348] head_absolute_y indicates the absolute position of the head (capture device) on the y-axis. For example, a 32-bit signed integer with 3 decimal places of precision makes the absolute y position of the head head_absolute_y / 8.

[0349] head_absolute_z indicates the absolute position of the head (capture device) on the z-axis. For example, a 32-bit signed integer with 3 decimal places of precision makes the absolute z position of the head head_absolute_z / 8.

[0350] `head_parallel_to_axis_flag` equal to 1 indicates that the head is parallel to the axis orientation. In this case, `axis_index` exists in the stream. `head_parallel_to_axis_flag` equal to 0 indicates that the head is not parallel to the axis orientation, and in this case, `head_orientation_phi` and `head_orientation_theta` exist in the stream instead of `axis_index`. If `head_parallel_to_axis_flag` does not exist in the bitstream, it is inferred to be equal to 1.

[0351] The axis_index indicates which axis the head is parallel to. For example, an axis_index of 0 indicates that the head is parallel to the x-axis, 1 indicates that the head is parallel to the y-axis, and 2 indicates that the head is parallel to the z-axis.

[0352] `head_orientation_phi` and `head_orientation_theta` indicate the orientation of the head in spherical coordinates. For example, `head_orientation_phi` is the azimuth angle in the xy plane, and `head_orientation_theta` is the polar angle relative to the z-axis. Both can be signed 32-bit integers with 18 decimal places of precision.

[0353] `num_of_lasers` specifies the number of lasers (or beams) for a given head. It can be a 10-bit integer in the range of 1 to 1023.

[0354] An angle_is_tangent_flag equal to 1 indicates that the information provided by laser_angle is the tangent of an angle. An angle_is_tangent_flag equal to 0 indicates that the information provided by laser_angle is an angle. When it does not exist in the bitstream, angle_is_tangent_flag is inferred to be equal to 1.

[0355] `laser_angle` is the angle (tangent) associated with a given laser. This is the elevation angle relative to the plane perpendicular to the head direction and to which the laser (or beam emitter) is located. It can be a signed 32-bit integer with 18 decimal places of precision.

[0356] A `laser_relative_position_to_head_flag` value of 0 indicates that the laser / beam emitter position is the same as the head's absolute position. A `laser_relative_position_to_head_flag` value of 1 indicates that the laser / beam emitter position is different from the head's absolute position; in this case, `laser_relative_position_dL` and `relative_position_rL_flag` are present in the bit stream. If `laser_relative_position_to_head_flag` is not present in the bit stream, it is inferred to be equal to 0.

[0357] `laser_relative_position_dL` indicates the position of the laser (beam emitter) relative to the head along the head orientation direction. It can be a 16-bit signed integer with 3 decimal places of precision.

[0358] A relative_position_rL_flag value of 0 indicates that laser_relative_rL does not exist in the bitstream and is inferred to be 0. A relative_position_rL_flag value of 1 indicates that laser_relative_rL exists in the bitstream. When relative_position_rL_flag does not exist in the bitstream, it is inferred to be equal to 0.

[0359] `laser_relative_rL` indicates the (radial) distance between the laser (beam emitter) and a line parallel to the head's orientation direction that passes through the absolute position of the head. It can be a 16-bit signed integer with 3 decimal places of precision.

[0360] Now refer to Figure 30 This diagram illustrates a simplified block diagram of an example embodiment of encoder 3000. Encoder 3000 includes processor 3002, memory 3004, and encoding application 3006. Encoding application 3006 may include a computer program or application stored in memory 3004 and containing instructions that, when executed, cause processor 3002 to perform operations such as those described herein. For example, encoding application 3006 may encode, decode, and output a bitstream encoded according to the processes described herein. It is to be understood that encoding application 3006 may be stored on a non-transitory computer-readable medium, such as an optical disk, flash memory device, random access memory, hard disk drive, etc. When the instructions are executed, processor 3002 performs the operations and functions specified in the instructions to operate as a dedicated processor implementing the described processes(s). In some examples, such a processor may be referred to as a "processor circuit" or a "processor circuit system".

[0361] Still referencing Figure 31 The diagram illustrates a simplified block diagram of an example embodiment of decoder 3100. Decoder 3100 includes processor 3102, memory 3104, and decoding application 3106. Decoding application 3106 may include a computer program or application stored in memory 3104 and containing instructions that, when executed, cause processor 3102 to perform operations such as those described herein. It should be understood that decoding application 3106 may be stored on a computer-readable medium, such as an optical disc, flash memory device, random access memory, hard disk drive, etc. When the instructions are executed, processor 3102 performs the operations and functions specified in the instructions to operate as a dedicated processor implementing (multiple) the described processes. In some examples, such a processor may be referred to as a "processor circuit" or a "processor circuit system".

[0362] It should be understood that the decoder and / or encoder according to this application can be implemented in multiple computing devices, including but not limited to servers, appropriately programmed general-purpose computers, machine vision systems, and mobile devices. The decoder or encoder can be implemented in software containing instructions for configuring one or more processors to perform the functions described herein. The software instructions can be stored on any suitable non-transient computer-readable storage medium, including CDs, RAM, ROM, flash memory, etc.

[0363] It should be understood that the decoders and / or encoders described herein, as well as modules, routines, procedures, threads, or other software components implementing the described methods / procedures for configuring the encoder or decoder, can be implemented using standard computer programming techniques and languages. This application is not limited to specific processors, computer languages, computer programming conventions, data structures, or other such implementation details. Those skilled in the art will recognize that the described processes can be implemented as part of computer-executable code stored in volatile or non-volatile memory, as part of an application-specific integrated circuit (ASIC), etc.

[0364] This application also provides a computer-readable signal that is encoded by applying an encoding process according to this application.

[0365] Impact on compressibility

[0366] Tests were performed using a planar model in three directions (x, y, and z) with example implementations, where multiple example point clouds exhibited different characteristics and were compared to the current implementation of the Animation Experts Group (MPEG) test model. The different types of point clouds used in the experiments included point clouds related to outdoor scenes, including urban built environments, indoor built environments, 3D maps, LiDAR road scans, and natural landscapes. In the case of natural landscapes, the compression gain was negligible. Compression gains of 2% to 4% were observed in the case of 3D maps, reaching up to 10% in outdoor built scenes, 6% to 9% in the case of LiDAR data, and exceeding 50% in some indoor built scenes.

[0367] Further observations revealed that, compared to the planar mode alone, the angled mode offered over 15% additional gain for lossless encoding and decoding, and 5% to 8% additional gain for lossy encoding and decoding. Compared to the MPEG test model, the combination of planar and angled modes provided 22% to 24% (lossless) and 11% to 17% (lossy) gains.

[0368] Furthermore, it was observed that using an angled pattern with a tree, where the volume associated with the octree node has a ratio H / W = 1 for all volumes, provides an additional gain of approximately 5% or more compared to using an angled pattern with a cubic octree that separates all volumes associated with the octree node, the tree being constructed such that the volume associated with the tree node has a typical ratio H / W, with a height of at least 4 compared to the radial width.

[0369] Certain adaptations and modifications can be made to the described embodiments. Therefore, the embodiments discussed above are considered illustrative rather than limiting.

Claims

1. A method for encoding the geometry of a point cloud acquired by beam combining including a beam set to generate a bitstream of compressed point cloud data, said geometry being constrained in a tree structure having multiple nodes with parent-child relationships and representing three-dimensional positions of objects, the point cloud residing within a volume space recursively partitioned into sub-volumes and containing points of said point cloud, wherein the volume is partitioned into sub-sub-volume sets, and wherein an occupancy bit associated with each corresponding sub-sub-volume indicates whether said corresponding sub-sub-volume contains at least one of the points, the method comprising, For the current node associated with the volume that has been split into sub-sub-volumes: The occupancy pattern for the volume is determined based on the occupancy status of the sub-sub-volumes of the volume; as well as The occupancy pattern representation is entropy encoded into the bit stream. The occupancy pattern entropy encoding includes: Determine angular information for a sub-sub-volume or sub-sub-volume group for the volume, the angular information indicating one or more elevation angles relative to a position associated with the beam combination within the volume space; The context for encoding the representation of the occupancy pattern is determined based on the angle information; and The determined context is used to entropy encode the representation of the occupancy pattern.

2. A method for decoding a bitstream of compressed point cloud data to generate a reconstructed geometry of the point cloud, the point cloud being acquired by beam combining including a beam set, the geometry of the point cloud being defined in a tree structure having multiple nodes with parent-child relationships and representing the three-dimensional positions of objects, the point cloud residing within a volume space recursively partitioned into sub-volumes and containing points of the point cloud, wherein the volume is partitioned into sub-sub-volume sets, and wherein an occupancy bit associated with each corresponding sub-sub-volume indicates whether the corresponding sub-sub-volume contains at least one of the points, the method comprising... For the current node associated with the volume that has been split into sub-sub-volumes: Determine angular information for a sub-sub-volume or sub-sub-volume group for the volume, the angular information indicating one or more elevation angles relative to a position associated with the beam combination within the volume space; Based on the angle information, a context for decoding the representation of the occupancy pattern of the volume is determined, the occupancy pattern indicating the occupancy status of the sub-sub-volumes of the volume; as well as The bitstream is entropy decoded using the determined context to generate a reconstructed representation of the occupancy pattern.

3. The method according to claim 1 or 2, wherein determining the angle information includes: Determine a first elevation angle of the reference position within the volume relative to the reference position of the beam combination; Based on the first elevation angle, determine the beam that is assumed to have acquired the point within the volume; Determine the origin position of the beam; as well as Determine one or more elevation angles of the sub-sub-volumes or sub-sub-volume groups of the volume relative to the origin position of the beam.

4. The method according to claim 1 or 2, further comprising: First, the volume is determined to be acceptable using the following steps to determine the context based on the angle information: Determine a measure of the angular size of the volume as seen from a reference position of the beam combination; The metric of the angle magnitude is compared with the metric of the difference angle between adjacent beams of the beam combination; as well as Based on the comparison results, the volume is determined to be qualified for use in determining the context based on the angle information.

5. The method of claim 3, wherein if the volume is determined to be a planar volume, the planar volume is a volume divided into a plurality of sub-volume sets in corresponding parallel planes, for the planar volume, all sub-volumes containing at least one point are located in the same plane: Determining the context for encoding and decoding the representation of the occupancy pattern includes: Based on the angle information, a context for encoding and decoding a planar position marker is determined, wherein the planar position marker signals the position of the plane in which the sub-volume containing at least one point is located; as well as Entropy encoding / decoding of the representation of the occupancy pattern includes: entropy encoding / decoding of the planar location marker using the determined context.

6. The method according to claim 5, Determining the angle information includes: A corresponding elevation angle is determined for each of the plurality of parallel planes relative to the origin position of the beam; and Determining the context used for encoding and decoding the planar position markers includes: The corresponding elevation angle of each of the plurality of parallel planes is compared with the beam angle of the beam; as well as The context for encoding and decoding the planar location markers is determined based on the results of the corresponding comparisons.

7. The method of claim 6, wherein determining the context for encoding / decoding the planar location marker based on the result of a corresponding comparison includes at least one of the following: The difference between the beam angle of the beam and the corresponding elevation angle of each of the plurality of parallel planes; The magnitude of the difference between the beam angle of the beam and the corresponding elevation angle of each of the plurality of parallel planes; and The quantified result of the difference between the amplitude of the maximum difference between the difference between the beam angle of the beam and the amplitude of the minimum difference between the difference between the beam angle of the beam and the amplitude of the minimum difference between the difference between the corresponding elevation angle of the beam and the amplitude of the corresponding elevation angle of the parallel planes.

8. The method of claim 3, wherein Determining the context for encoding and decoding the representation of the occupancy pattern includes: Determine the context for encoding and decoding the occupancy bits of the sub-sub-volumes of the volume; and Entropy encoding / decoding of the representation of the occupancy pattern includes: entropy encoding / decoding of the occupancy bits using the determined context.

9. The method according to claim 8, Determining the angle information includes: A corresponding elevation angle is determined for each of the sub-sub-volumes of the volume, relative to the origin position of the beam; and The context for determining the occupancy bits used to encode and decode the sub-sub-volumes of the volume includes: The elevation angle of each of the sub-sub-volumes of the volume is compared with the beam angle of the beam; as well as Based on the results of the corresponding comparison, the context for encoding and decoding the occupancy bits of the sub-sub-volumes of the volume is determined.

10. The method according to claim 1 or 2, The beam combination mentioned above is a rotating beam combination; and The point cloud is defined relative to a Cartesian axis in the volume space, the Cartesian axis having a vertically oriented z-axis that is orthogonal to the horizontal plane and substantially parallel to the rotation axis of the rotating beam assembly.

11. The method of claim 1 or 2, wherein the bit stream includes one or more parameters describing the characteristics of the beam combination.

12. An encoder for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being defined in a tree structure having multiple nodes having parent-child relationships and representing the geometry of a volume space, the volume space being recursively subdivided into sub-volumes and containing the points of the point cloud, the encoder comprising: processor; Memory; and An coded application comprising instructions executable by the processor, the instructions, when executed, causing the processor to perform the method according to claim 1.

13. A decoder for decoding a bitstream of compressed point cloud data to produce a reconstructed point cloud, the point cloud being defined in a tree structure having multiple nodes with parent-child relationships and representing the geometry of a volume space, the volume space being recursively subdivided into sub-volumes and containing the points of the point cloud, the decoder comprising: processor; Memory; and A decoding application comprising instructions executable by the processor, which, when executed, cause the processor to perform the method according to claim 2.

14. A non-transient processor-readable medium storing processor-executable instructions, which, when executed by a processor, causes the processor to perform the method of any one of claims 1 to 11.

15. A computer-readable signal comprising program instructions, which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 11.

Images