In-tree geometry quantization of point clouds

Abstract

An exemplary method includes receiving (502) a plurality of points representing a point cloud; representing a position of the points in each dimension of a three-dimensional space as a sequence of bits (504), wherein the position of the point is encoded according to a tree data structure; dividing (506) at least one of the sequence of bits into a first portion of bits and a second portion of bits; quantizing (508) each of the second portion of bits according to a quantization step size, wherein the quantization step size is determined according to an exponential function having a quantization parameter value as input and the quantization step size as output; and generating (510) a data structure representing the point cloud and including the quantized second portion of bits.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application Serial No. 62 / 958,688, filed on January 8, 2020, and U.S. Provisional Application Serial No. 62 / 958,689, filed on January 8, 2020, which are hereby incorporated by reference in their entireties. Technical Field

[0003] The present disclosure generally relates to storing and processing information about a point cloud comprising a plurality of points. Background Art

[0004] Various types of sensors (such as light detection and ranging (LIDAR) systems, 3D cameras, 3D scanners, etc.) can capture data indicating the location of points in three-dimensional space (e.g., positions in the X, Y, and Z planes). Furthermore, such systems can capture attribute information in addition to spatial information for corresponding points, such as color information (e.g., RGB values), intensity attributes, reflectivity attributes, motion-related attributes, modality attributes, or various other attributes. In some cases, additional attributes can be assigned to corresponding points, such as a timestamp when the point was captured. The points captured by such sensors can constitute a "point cloud," which includes a set of points, each with associated spatial information and one or more associated attributes. In some cases, a point cloud can include thousands, hundreds of thousands, millions, or even more points. Furthermore, in some cases, a point cloud can be generated, for example, in software, separate from the point cloud being captured by one or more sensors. In either case, such a point cloud can include a large amount of data, and storing and transmitting these point clouds can be costly and time-consuming. Summary of the Invention

[0005] In one aspect, a method includes receiving, by a computer system, a plurality of points representing a point cloud in three-dimensional space; for each point, representing, by the computer system, a position of the point in each dimension of the three-dimensional space as a sequence of bits, wherein the position of the point is encoded according to a tree data structure; dividing, by the computer system, at least one of the bit sequences into a first portion of bits and a second portion of bits; quantizing, by the computer system, each of the second portion of bits according to a quantization step size, wherein the quantization step size is determined according to an exponential function having a quantization parameter value as input and the quantization step size as output; generating, by the computer system, a data structure representing the point cloud in the three-dimensional space, the data structure including the second portion of the quantized bits; and performing, by the computer system, at least one of: transmitting the data structure or storing the data structure.

[0006] Implementations of this aspect may include one or more of the following features.

[0007] In some implementations, the tree data structure can be an octree.

[0008] In some embodiments, partitioning at least one of the bit sequences may include partitioning at least one of the bit sequences such that each of the first portions of bits includes first d bits from a corresponding bit sequence in the bit sequence, where d is a scaled depth of the tree data structure; and partitioning at least one of the bit sequences such that each of the second portions of bits includes remaining portions of bits from a corresponding bit sequence in the bit sequence.

[0009] In some implementations, quantizing each of the second portion of bits can include dividing the value indicated by each of the second portion of bits by the quantization step size.

[0010] In some implementations, the method may further include indicating at least one of the quantization step size or the quantization parameter value in the data structure.

[0011] In some implementations, the exponential function can be selected such that for positive integer quantization parameter values, a majority of the quantization step sizes output by the exponential function are integers.

[0012] In some embodiments, the exponential function can be selected so that for positive integer quantization parameter values, N of the quantization step sizes output by the exponential function are non-integer, where N = (p×log2p)-p+1, and where p is the number of quantization parameters for each doubling of the quantization step size.

[0013] In some implementations, the exponential function can be QS=0.125*(mod[QP,8])*2^(floor[QP / 8]), where QS is the quantization step size and QP is the quantization parameter value.

[0014] In some implementations, the method can further include approximating at least some of the second portion of the quantized bits to integer values.

[0015] On the other hand, a method includes receiving, by a computer system, a plurality of points representing a point cloud in a three-dimensional space; and generating, by the computer system, a data structure representing the point cloud in the three-dimensional space. The data structure includes a header portion and a data portion following the header portion. For each point, the data portion indicates the position of the point encoded according to a tree data structure and quantized according to a corresponding quantization parameter. The tree data structure has a plurality of tree levels. The data portion includes an offset value that indicates, at one or more of the tree levels, a difference between a quantization parameter used to quantize the position of the point at the one or more tree levels and a base quantization parameter. The data portion includes, at each of the one or more tree levels, a data flag indicating the presence of one or more offset values at the tree level. The method also includes, by the computer system, at least one of: transmitting the data structure or storing the data structure.

[0016] Implementations of this aspect may include one or more of the following features.

[0017] In some implementations, for each of one or more tree levels, the data tag can be included at the beginning of the tree level.

[0018] In some implementations, the tree data structure can be an octree.

[0019] In some implementations, the header portion does not include data indicating the presence of the offset value.

[0020] On the other hand, a method includes receiving, by a computer system, a data structure representing a point cloud in a three-dimensional space. The point cloud has a plurality of points. The data structure includes a header portion and a data portion following the header portion. For each point, the data portion indicates a position of the point encoded according to a tree data structure and quantized according to a corresponding quantization parameter. The tree data structure has a plurality of tree levels. The data portion includes an offset value that indicates, at one or more of the tree levels, a difference between a quantization parameter used to quantize the position of the point at the one or more tree levels and a base quantization parameter. The data portion includes, at each of the one or more tree levels, a data flag indicating the presence of one or more offset values at the tree level. The method also includes determining, by the computer system, a position of each of the points based on the data structure; and outputting, by the computer system, the determined position of each of the points.

[0021] Implementations of this aspect may include one or more of the following features.

[0022] In some specific implementations, determining the position of each of the points based on the data structure may include determining that one or more of the offset values exist at a particular tree level based on a data marker at the tree level; determining the quantization parameter for quantizing the position of the point at the tree level based on the offset value at the tree level; and determining the position of each of the points at the tree level based on the determined quantization parameter.

[0023] In some implementations, the tree data structure can be an octree.

[0024] In some implementations, the method may further include rendering three-dimensional content based on the determined position of each of the points. The three-dimensional content may be at least one of virtual reality content or augmented reality content.

[0025] In another aspect, a method includes receiving, by a computer system, a data structure representing a plurality of points in a point cloud in three-dimensional space; decoding, by the computer system, the data structure to determine one or more bit sequences, wherein each of the bit sequences represents a position of one of the points in one of the dimensions of the three-dimensional space, and wherein the data structure is decoded according to a tree data structure; dividing, by the computer system, at least one of the bit sequences into a first portion of bits and a second portion of bits; scaling, by the computer system, each of the second portion of bits according to a quantization step size, wherein the quantization step size is determined according to an exponential function having a quantization parameter value as input and the quantization step size as output; generating, by the computer system, a reconstructed point position of at least one of the points by combining, for each of the at least one of the points, the first portion of bits and the scaled second portion of bits corresponding to the point; and outputting, by the computer system, the reconstructed point position of the at least one of the points.

[0026] Implementations of this aspect may include one or more of the following features.

[0027] In some implementations, the tree data structure can be an octree.

[0028] In some implementations, partitioning at least one of the bit sequences may include partitioning at least one of the bit sequences such that each of the first portions of bits includes first d bits from a corresponding bit sequence in the bit sequence, where d is a scaled depth of the tree data structure, and partitioning at least one of the bit sequences such that each of the second portions of bits includes a remainder of bits from a corresponding bit sequence in the bit sequence.

[0029] In some implementations, scaling each of the second portion of bits can include multiplying the value indicated by each of the second portion of bits by the quantization step size.

[0030] In some implementations, at least one of the quantization step size or the quantization parameter value can be indicated in the data structure.

[0031] In some implementations, the exponential function can be selected such that for positive integer quantization parameter values, a majority of the quantization step sizes output by the exponential function are integers.

[0032] In some embodiments, the exponential function can be selected so that for positive integer quantization parameter values, N of the quantization step sizes output by the exponential function are non-integer, where N = (p×log2p)-p+1, and where p is the number of quantization parameters for each doubling of the quantization step size.

[0033] In some implementations, the exponential function can be Where QS is the quantization step size, and QP is the quantization parameter value.

[0034] In some specific implementations, the method may further include rendering three-dimensional content based on the reconstructed point position of the at least one of the points, wherein the three-dimensional content is at least one of virtual reality content or augmented reality content.

[0035] Other specific implementations relate to systems, devices, and non-transitory computer-readable media having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform the operations described herein.

[0036] The details of one or more embodiments are set forth in the accompanying drawings and the detailed description below. Other features and advantages will be apparent from the detailed description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A system is shown that includes a sensor that captures information about points of a point cloud and an encoder that compresses attribute and / or spatial information of the point cloud, where the compressed attribute and / or spatial information is sent to a decoder.

[0038] Figure 2 An exemplary recursive partitioning of space and corresponding tree data structure is shown.

[0039] Figures 3A to 3E An exemplary encoding of points according to a tree data structure is shown.

[0040] Figure 4 Exemplary distortions in a point cloud during the encoding and decoding processes are shown.

[0041] Figure 5A An exemplary process for generating a data structure representing a point cloud is shown.

[0042] Figure 5B An exemplary process for rendering three-dimensional content based on a data structure representing a point cloud is shown.

[0043] Figure 6A An exemplary process for generating a data structure representing a point cloud is shown.

[0044] Figure 6B An exemplary process for rendering three-dimensional content based on a data structure representing a point cloud is shown.

[0045] Figure 7 Shows compressed point cloud information being used in 3D applications.

[0046] Figure 8 Compressed point cloud information is shown being used in a virtual reality application according to some embodiments.

[0047] Figure 9 An exemplary computer system is shown that may implement an encoder or decoder.

[0048] This specification includes references to "one embodiment" or "an embodiment." The appearance of the phrase "in one embodiment" or "in an embodiment" does not necessarily refer to the same embodiment. The particular features, structures, or characteristics may be combined in any suitable manner consistent with the present disclosure.

[0049] The term "comprising" is open-ended. As used in the appended claims, the term does not exclude additional structures or steps. Consider the following recited claim: "An apparatus comprising one or more processor units..." Such a claim does not exclude the apparatus from including additional components (e.g., a network interface unit, graphics circuitry, etc.).

[0050] Various units, circuits, or other components may be described or stated as being “configured to” perform one or more tasks. In such contexts, “configured to” is used to imply a structure (e.g., circuitry) that performs the task or tasks during operation by indicating that the unit / circuit / component includes the structure. Thus, the unit / circuit / component is said to be configured to perform the task even when the specified unit / circuit / component is not currently operational (e.g., not turned on). Units / circuits / components used with the “configured to” language include hardware—e.g., circuitry, memory storing program instructions executable to implement the operation, etc. Reference to a unit / circuit / component being “configured to” perform one or more tasks is expressly intended not to invoke 35 USC §112(f) with respect to that unit / circuit / component. Furthermore, “configured to” may include a general structure (e.g., general circuitry) manipulated by software and / or firmware (e.g., an FPGA or a general-purpose processor executing software) to operate in a manner capable of performing the task or tasks to be solved. "Configured to" may also include adapting a manufacturing process (eg, a semiconductor fabrication facility) to produce a device (eg, an integrated circuit) suitable for implementing or performing one or more tasks.

[0051] "First," "Second," etc. As used herein, these terms act as labels for the nouns that precede them and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.). For example, a buffer circuit may be described herein as performing a write operation for a "first" value and a "second" value. The terms "first" and "second" do not necessarily imply that the first value must be written before the second value.

[0052] "Based on." As used herein, this term is used to describe one or more factors that influence a determination. This term does not exclude additional factors that influence the determination. That is, the determination may be based solely on these factors, or at least in part on these factors. Consider the phrase "A is determined based on B." In this case, B is a factor that influences the determination of A, and such a phrase does not exclude that the determination of A may also be based on C. In other examples, A may be determined based solely on B. DETAILED DESCRIPTION

[0053] As data acquisition and display technologies become more advanced, the ability to capture point clouds comprising tens of thousands of points in 2D or 3D space has increased (such as via LIDAR systems). Furthermore, the development of advanced display technologies (such as virtual reality or augmented reality systems) has increased the potential uses of point clouds. However, point cloud files are typically very large, and storing and transmitting these point cloud files can be costly and time consuming. For example, communication of a point cloud over a private network or a public network (such as the Internet) can require a considerable amount of time and / or network resources, such that some uses of the point cloud data (such as real-time uses) may be limited. Additionally, the storage requirements of the point cloud files may consume a significant amount of storage capacity of the device storing the point cloud files, which may also limit potential applications using the point cloud data.

[0054] In some embodiments, an encoder can be used to generate a compressed point cloud to reduce the cost and time associated with storing and transmitting large point cloud files. In some embodiments, a system can include an encoder that compresses attribute information or spatial information of a point cloud file, allowing the point cloud file to be stored and transmitted more quickly and in a manner that occupies less storage space than an uncompressed point cloud. In some embodiments, compressing the attributes of points in a point cloud can enable the point cloud to be transmitted over a network in real time or near real time. For example, a system can include a sensor that captures attribute information about points in an environment in which the sensor is located, where the captured points and corresponding attributes constitute a point cloud. The system can also include an encoder that compresses the attribute information of the captured point cloud. The compressed attribute information of the point cloud can be transmitted over a network in real time or near real time to a decoder that decompresses the compressed attribute information of the point cloud. The decompressed point cloud can be further processed, for example, to make control decisions based on the surrounding environment at the sensor location. The control decisions can then be transmitted back to a device at or near the sensor location, where the device receiving the control decisions implements the control decisions in real time or near real time. In some embodiments, a decoder may be associated with an augmented reality system, and the decompressed attribute information may be displayed or otherwise used by the augmented reality system. In some embodiments, compressed attribute information about a point cloud may be transmitted along with compressed spatial information about the points of the point cloud. In other embodiments, the spatial information and attribute information may be encoded separately and / or transmitted separately to the decoder.

[0055] In some embodiments, the system may include a decoder that receives one or more point cloud files including compressed attribute information from a remote server or other storage device storing the one or more point cloud files via a network. For example, a 3D display, a holographic display, or a head-mounted display may be manipulated in real time or near real time to display different portions of a virtual world represented by a point cloud. To update the 3D display, holographic display, or head-mounted display, a system associated with the decoder may request point cloud files from a remote server based on user manipulation of the display, and these point cloud files may be transmitted from the remote server to the decoder and decoded by the decoder in real time or near real time. The display may then be updated with updated point cloud data (such as updated point attributes) in response to the user manipulation.

[0056] In some embodiments, a system may include one or more LIDAR systems, 3D cameras, 3D scanners, etc., and such sensor devices may capture spatial information, such as the X, Y, and Z coordinates of points in the sensor device's field of view. In some embodiments, the spatial information may be relative to a local coordinate system or may be relative to a global coordinate system (e.g., a Cartesian coordinate system may have fixed reference points, such as fixed points on the Earth, or may have non-fixed local reference points, such as sensor locations).

[0057] In some embodiments, such sensors may also capture attribute information about one or more points, such as color attributes, reflectivity attributes, velocity attributes, acceleration attributes, time attributes, modality, and / or various other attributes. In some embodiments, in addition to LIDAR systems, 3D cameras, 3D scanners, etc., other sensors may capture attribute information to be included in the point cloud. For example, in some embodiments, a gyroscope or accelerometer may capture motion information to be included in the point cloud as an attribute associated with one or more points of the point cloud. For example, a vehicle equipped with a LIDAR system, a 3D camera, or a 3D scanner may include the vehicle's direction and velocity in the point cloud captured by the LIDAR system, 3D camera, or 3D scanner. For example, when points in the vehicle's field of view are captured, these points may be included in a point cloud, wherein the point cloud includes the captured points and associated motion information corresponding to the state of the vehicle at the time the points were captured.

[0058] Figure 1 A system is shown that includes a sensor that captures information about points of a point cloud and an encoder that compresses attribute information of the point cloud, where the compressed attribute information is sent to a decoder.

[0059] System 100 includes a sensor 102 and an encoder 104. Sensor 102 captures a point cloud 110, which includes points representing structures 106 in a view 108 of sensor 102. For example, in some embodiments, structure 106 may be a mountain, a building, a sign, the surroundings of a street, or any other type of structure. In some embodiments, a captured point cloud, such as captured point cloud 110, may include spatial information and attribute information about the points included in the point cloud. For example, point A in captured point cloud 110 includes X, Y, and Z coordinates, as well as attributes 1, 2, and 3. In some embodiments, the attributes of a point may include attributes such as R, G, and B color values, velocity at the point, acceleration at the point, reflectivity of the structure at the point, a timestamp indicating when the point was captured, a string value indicating the modality at the time the point was captured, such as "walking," or other attributes. Captured point cloud 110 may be provided to encoder 104, which generates a compressed version of the point cloud (compressed attribute information 112), which is transmitted to decoder 116 via network 114. In some embodiments, a compressed version of a point cloud (such as compressed attribute information 112) may be included in a common compressed point cloud that also includes compressed spatial information for the points of the point cloud, or in some embodiments, the compressed spatial information and compressed attribute information may be transmitted as separate files.

[0060] In some embodiments, encoder 104 can be integral to sensor 102. For example, encoder 104 can be implemented in hardware or software included in a sensor device, such as sensor 102. In other embodiments, encoder 104 can be implemented on a separate computing device adjacent to sensor 102.

[0061] In some cases, the number of bits required to encode the attribute information comprises a significant portion of the bitstream for the point cloud.

[0062] In some embodiments, the locations of points in point cloud 110 can be encoded according to a tree data structure. For example, the three-dimensional space can be recursively subdivided into successively smaller parts. Each of these parts can be represented by a corresponding node in the tree data structure. The presence of a point in each part can be indicated in the corresponding node of the tree data structure.

[0063] As an example, refer to Figure 2, the positions of points in the point cloud 110 can be encoded according to an octree 200, where each node 202 of the octree 200 has exactly eight child nodes 202. The three-dimensional space 204 can be recursively subdivided into successively smaller eighths. Each of these eighths can be represented by a corresponding node 202 of the octree 200. The presence of a point in each eighth can be indicated in the corresponding node 202 of the octree (e.g., using a bit "1" to indicate the presence of a point in the corresponding eighth, or using a bit "0" to indicate the absence of a point in the corresponding eighth).

[0064] Figures 3A to 3E An example of encoding the positions of points according to a tree data structure is shown in FIG. In this example, a plurality of points 300 on a single two-dimensional plane are encoded according to a quadtree (a tree data structure in which each node has exactly four children). However, it should be understood that points in a three-dimensional point cloud can be encoded in a similar manner according to an octree.

[0065] like Figure 3A As shown, a plurality of points 300 are located on a two-dimensional plane. The presence of a point 300 at a particular location is indicated by a white box in a grid 302, while the absence of a point 300 at a particular location is indicated by a black box in the grid 302. The rows of the grid 302 are indexed according to corresponding binary sequences 304 (shown on the right side of the grid 302, where the black boxes indicate bit "0" and the colored boxes indicate bit "1"). Similarly, the columns of the grid 302 are indexed according to corresponding binary sequences 306 (shown at the bottom of the grid 302, where the black boxes indicate bit "0" and the colored boxes indicate bit "1").

[0066] like Figure 3B As shown, the grid 302 can be divided into four equal-sized portions 308a-308d (e.g., squares). The division can be represented by a tree data structure 310 having a parent node and four child nodes branching from the parent node. The presence of one or more points 300 in each portion 308a-308d can be indicated by a bit "1" in the corresponding child node. Here, since each portion 308a-308d includes at least one point 300, each child node is assigned a bit "1."

[0067] like Figure 3CAs shown, each portion 308a-308d can be further subdivided into four equally sized sub-portions (e.g., squares). For example, portion 308a can be subdivided into portions 310a-310d. This division can be represented by a tree data structure 310, which has four child nodes branching from the node corresponding to portion 308a. The presence of a point or multiple points 300 in each sub-portion 310a-310d can be represented by a bit "1" in the corresponding child node. Here, since each sub-portion in sub-portions 310a, 310b, and 310d includes at least one point 300, the child nodes associated with those sub-portions are each assigned a bit "1." The child node associated with sub-portion 310c is assigned a bit "0." Similarly, other portions 308c-308d can also be subdivided into four equal sub-portions, and the presence of a point in each sub-portion can be indicated by the tree data structure 310 in a manner similar to that described above.

[0068] like Figure 3D As shown, each sub-portion 310a-310d can be further subdivided into four equally sized sub-portions (e.g., squares). For example, sub-portion 310a can be subdivided into sub-portions 312a-312d. This division can be represented by a tree data structure 310, which has four child nodes branching from the node corresponding to sub-portion 310a. The presence of a point or multiple points 300 in each sub-portion 312a-312d can be indicated by a bit "1" in the corresponding child node. Here, since each sub-portion in sub-portions 312a and 312b includes at least one point 300, the child nodes associated with those sub-portions are each assigned a bit "1". The child nodes associated with sub-portions 310c and 310d are each assigned a bit "0". Similarly, the other sub-portions of the grid can each be subdivided into four equal sub-portions, and the presence of a point in each sub-portion can be indicated by the tree data structure 310 in a manner similar to that described above.

[0069] The spatial position of a particular point 300 may be represented by one or more binary sequences. For example, the spatial position of point 300a relative to the horizontal axis may be represented by the binary sequence "011" (e.g., corresponding to its column index), and the spatial position of point 300a relative to the vertical axis may be represented by the binary sequence "101" (e.g., corresponding to its row index).

[0070] although Figures 3A to 3D 3. The tree data structure 310 is shown as having four levels, but in practice, the tree data structure may include any number of levels (corresponding to any number of spatial subdivisions). Figures 3A to 3DA tree data structure 310 is shown having four child nodes branching from each parent node (e.g., a quadtree), but in practice, the tree data structure can have any number of child nodes branching from each parent node (e.g., 8 child nodes branching from each parent node, as with an octree).

[0071] Quantization step size :

[0072] In some implementations, during the encoding process, the positions of the points can be partially quantized or scaled so that the positions (or approximate positions) of the points can be represented using a reduced number of bits. As an example, the encoder can divide each of the binary sequences representing the positions of the points into two parts: (i) a first part comprising the first d bits of the binary sequence, where d is a particular depth (e.g., representing bits that have already been encoded by the encoder), and (ii) a second part comprising the remaining bits of the binary sequence. The second part can be quantized according to a selected quantization parameter.

[0073] To illustrate, the binary representation of the x position of a point can be represented as {b4, b3, b2, b1, b0}, where x is between 0 and 31 (inclusive). For example, x = 21 = 0b10101 (i.e., the binary sequence "10101"). After encoding the first two tree levels, at depth d = 2, the first two bits of 0b10101 are discarded, leaving 0b101 (i.e., decimal value 5). The remainder can be quantized according to a step size of 2 (e.g., ). This has the effect of reducing the range of the remainder from decimal values 0 to 7 to decimal values 0 to 3. Since the range has been reduced by a factor of two, only two tree levels are needed to represent the remainder, rather than the three levels of the remainder before quantization. The level reduction can be expressed as Floor(Log2(QS)), where QS is the quantization step size. A decoder that is informed to quantize with a step size of 2 at tree depth 2 will decode a four-bit representation of the corresponding point (5-Floor(Log2(2))=4). The decoder will scale the quantized portion of the position information (the two least significant bits, 0b10) by the quantization step size QS (0b10*2=0b100) and append the scaled value to the two most significant bits (0b10:0b100), resulting in a reconstructed position of 0b10100.

[0074] The performance of this "in-tree" geometric quantization has various effects. For example, this quantization eliminates levels from the subtree of fully quantized nodes, thereby avoiding signaling occupation of the eliminated nodes. In addition, this quantization reduces the number of coded points in the tree data structure. Moreover, this quantization compresses sparse surface combinations into a less sparse form. As a result, information about a point cloud (e.g., the presence and / or location of one or more points in the point cloud) can be stored using less data (e.g., compared to techniques that do not utilize quantization).

[0075] However, in some cases, this quantization can introduce spatial distortions in the point cloud during the encoding and decoding processes, especially when the quantization step size is not an integer value.

[0076] For example, quantization can map a point with nodes of size nS to a quantized node size nQ, where nQ ≤ nS. Therefore, the quantization step size QS is As an example, a three-dimensional node of size 128×128×128 (2,097,152 voxels) can be reduced to a node of size 64×64×64 (262,144 voxels) by using a quantitative step size QS of 2.

[0077] In some implementations, the quantitative step size QS may be determined based on an integer quantization parameter QP of an exponential function f(QP)=QS. For example, the relationship The quantitative step size QS is determined. In this example, increasing QP by 6 (eg, from 4 to 10) doubles the quantitative step size QS.

[0078] In this example, for a given integer value of the quantization parameter QP, the quantization step size QS is not an integer, except for powers of two. This can introduce distortions during the encoding and decoding processes. For example, in some implementations, the geometry encoder can only encode an integer number of point positions (e.g., as specified by a particular codec). However, scaling the quantized position values by a non-integer step size can result in positions with fractional parts that must be rounded to integers according to the codec's specifications. This discarding of fractional parts can result in visually objectionable artifacts during encoding and decoding.

[0079] For example, Figure 4As shown, point cloud 400 has been quantized according to quantization parameter QP=11, which results in a non-integer quantization step size QS (e.g., QS=2.24492). In this example, the geometry encoder can only encode an integer number of point positions. Therefore, any position of a point with a fractional part is rounded to an integer. During the decoding process, this rounding can cause uneven distortion in the point cloud. For example, some points have been shifted from their expected positions, resulting in lines or gaps between points and reduced fidelity.

[0080] In some implementations, one or more techniques can be used to eliminate or otherwise reduce these distortions.

[0081] In some implementations, the output of the codec can be fixed point. The number of fixed point bits can be configured in a set of parameters associated with the encoding process and the decoding process (e.g., in a sequence parameter set (SPS) according to the High Efficiency Video Coding (HEVC) standard). If geometric quantization is used with a non-quadratic quantization step size Q When used together, the fractional part can be non-zero.

[0082] As another example, given a positive integer quantization parameter QP, the exponential function f(QP)=QS can be selected so that some or all quantization step sizes QS are integers. In some implementations, the function f(QP)=QS can be selected so that for a positive integer quantization parameter QP, no more than five of the quantization step sizes QP output by the function are non-integer. As an example, QS can be expressed using the relationship In this example, given a positive integer quantization parameter QP, only QP = 1, 2, 3, 5, and 7 will result in QS having non-integer values. The value of QS is linearly distributed within each quadratic interval. Therefore, during the encoding and decoding processes, spatial distortion is eliminated or otherwise reduced due to the elimination or reduction of rounding of point position values. In addition, since the quantization step size is essentially limited to a power of two, this essentially eliminates the need to perform point-by-point multiplications during the decoding process, as they can be replaced by shifts. In some specific implementations, no additional overhead is required.

[0083] An integer implementation of the scaling process can be implemented as follows. First, the fixed-point quantization step size is determined as Shift = QP>>2; qsTimesFour = (4+(QP&3))<<Shift, where ">>" and "<<" represent right and left shift operators, and "&" represents a bitwise AND operator. Second, scaling is performed as (Pos×qsTimesFour)>>2, where Pos is the quantitative portion of the point position. In some embodiments, scaling can include an offset to perform "rounding" or approximation: (Pos×qsTimesFour+2)>>2.

[0084] In the example described above, an increase of four in the quantization parameter QP doubles the quantization step size QS. Within each quadratic doubling interval, the intermediate quantization step sizes are linearly distributed. However, other relationships are possible (e.g., increasing or decreasing the number of intermediate quantization steps between each doubling of the quantization step size).

[0085] For example, in some implementations, increasing the quantization parameter QP by eight doubles the quantization step size QS. This can be done using the relationship is implemented such that the intermediate quantization step size increases between each doubling of the quantization step size. In this example, given a positive integer quantization parameter QP, and assuming a 21-bit geometry, there are only 17 QP integer values in the range of 0 to 168 that result in QS having non-integer values (e.g., less than 11%). Thus, during the encoding and decoding processes, spatial distortion is eliminated or otherwise reduced due to the elimination or reduction of rounding of point position values. Additionally, an integer implementation of the scaling process may be implemented using: Shift = QP>>3; qsTimesEight = (8+(QP&7))<<Shift; where scaling is performed as (Pos×qsTimesEight)>>3, or with fractional positions rounded to (Pos×qsTimesEight+4)>>3.

[0086] As another example, in some implementations, increasing the quantization parameter QP by two doubles the quantization step size QS. This can be done using the relationship is implemented so that the intermediate quantization step size between each doubling of the quantization step size is reduced. In addition, an integer implementation of the scaling process can be implemented using the following: Shift = QP>>1; qsTimesTwo = (2+(QP&1))<<Shift; where the scaling is performed as (Pos×qsTimesTwo)>>1, or with fractional positions rounded to (Pos×qsTimesTwo+1)>>1.

[0087] Table 1 shows exemplary values of the quantization parameter QP and the quantization step size QS according to the exemplary relationship described above. In column QS, when the quantization parameter QP is increased by two so that the quantization step size QS is doubled (e.g., according to the relationship ), sub-column "2" shows the value of the quantization step size QS for a given specific quantization parameter QP. Furthermore, in column QS, when the quantization parameter QP is increased by four so that the quantization step size QS doubles (e.g., according to the relationship ), sub-column "4" shows the value of the quantization step size QS for a given specific quantization parameter QP. Furthermore, in column QS, when the quantization parameter QP is increased by eight so that the quantization step size QS doubles (e.g., according to the relationship ), sub-column "8" shows the value of the quantization step size QS for a given specific quantization parameter QP.

[0088]

[0089] Table 1 - QS values for quantization parameters (QP), with 2, 4, or 8 QPs per QS multiplication interval.

[0090] As another example, in some implementations, the quantization step size QS is doubled when the quantization parameter QP is increased by one (e.g., such that all intermediate steps between each doubling of the step size are eliminated). This can be explained using the relationship QS=2 QP For example, this may be useful to further reduce the complexity of the implementation. In addition, the integer implementation of the scaling process can be implemented as Pos<<QP.

[0091] Other relationships between the quantization parameter QP and the quantization step size QS are also possible. For example, in some implementations, increasing the quantization parameter QP by 16, 32, 64, or any other number can double the quantization step size QS. In some implementations, the number of quantization parameters that generate fractional step sizes is (p×log2p)−p+1, where p is the number of QPs per doubling of QS.

[0092] During decoding, the location of the point can be determined by reversing one or more of the above encoding techniques.

[0093] For example, as described above, a data structure can be encoded to represent the positions of multiple points in a point cloud in three-dimensional space. Specifically, the position of each point in each dimension can be represented by a corresponding bit sequence. In addition, at least some of the bit sequences can indicate the position of the bit based on partial quantization or scaling (e.g., as described above).

[0094] During the decoding process, the partially quantized or scaled bit sequence can be divided into two parts: (i) a first part, which includes the first d bits of the binary sequence, where d is a certain depth (e.g., representing bits that have not yet been quantized by the encoder), and (ii) a second part, which includes the remaining bits of the binary sequence, where the second part has been quantized according to the selected quantization parameter during the encoding process.

[0095] The second portion of bits can be scaled to reverse the quantization (e.g., by prefixing leading zero bits corresponding to the Floor(Log2(QS)) level reduction from the quantization, and by multiplying the value indicated by the second portion of bits by the quantization step size previously used to quantize the particular portion of bits during the encoding process). The scaled second portion of bits can be recombined with the first portion of bits (e.g., by appending the scaled second portion of bits to the end of the first portion of bits) to reconstruct the quantized position of the point. In turn, the reconstructed position of the point can be used to render a point cloud (e.g., as part of virtual reality content or augmented reality content).

[0096] Signaling quantization step size :

[0097] In some implementations, the position of a point can be scaled (e.g., "inverse quantized") at one or more levels of a tree data structure. For example, the tree data structure can include one or more offset values that indicate a difference or "offset" between a quantization parameter QP of a portion of the tree data structure (e.g., a quantization parameter QP of one or more levels of the tree data structure) relative to a "base" or "default" quantization parameter QP (e.g., a base or default quantization parameter QP of the tree data structure).

[0098] In some implementations, the geometry tree level at which scaling occurs can be signaled at each tree data structure level (e.g., each "geometry slice" level). For example, the tree level can be signaled using the parameter geom_octree_qp_offset_depth, which in turn is used to derive the geometry scaling depth (e.g., parameter GeomScalingDepth) and the scaling node size (e.g., parameter ScalingNodeSizeLog2). In some implementations, this parameter can be signaled in the header of the tree data structure (e.g., a portion of the data structure that precedes the encoded data about the positions of the points in the point cloud). In some cases, this may be preferred. For example, in some implementations, if the value of the parameter is known before encoding, signaling the parameter in the header can potentially reduce resource consumption (e.g., by reducing the need to perform additional memory copies or concatenation techniques).

[0099] However, in some implementations, it may be preferable to indicate at the beginning of each tree level (e.g., using a data flag) whether an offset value is present for that particular tree level (and, accordingly, whether scaling should be performed relative to that tree level). In some implementations, this can be indicated using a data flag included at the beginning of each tree level of the tree data structure (e.g., a data flag geom_octree_qp_offsets_present_flag). After the flag has been set, no signaling is sent for any subsequent tree levels in the tree data structure (e.g., any lower tree levels in the tree data structure). This data flag has the effect of signaling a unique depth at which an offset exists. Furthermore, with the data flag included, a variable specifying the geometric scaling depth (e.g., GeomScalingDepth) is no longer required, and the scaling node size (e.g., ScalingNodeSizeLog2) can be derived from the current node size when the flag is asserted.

[0100] To illustrate, an exemplary syntax for a tree data structure (eg, a "geometry slice") is shown below.

[0101]

[0102] Table 2: Example syntax for geometry slicing.

[0103] In addition, an exemplary syntax for encoding each geometry node is shown below:

[0104]

[0105] Table 3: Example syntax of the geometry node.

[0106] In Tables 2 and 3, bold lines indicate syntax that may be used to represent the depth at which the offset value is signaled.

[0107] As described above, the data within a geometry slice includes a sequence of geometry nodes. In the header of the geometry slice, the geometry tree level at which the scaling occurs can be signaled at each tree data structure level. Instead of signaling, a data flag (e.g., node_qp_offsets_present_flag) can be used at the beginning of each tree level to indicate whether an offset value is present for that particular tree level (and, accordingly, whether the scaling should be performed relative to that tree level). Since the flag appears infrequently, at most once per level, some implementations will encode the flag using the entropy encoder's bypass mode.

[0108] The semantics of this flag can be: node_qp_offsets_present_flag equal to 1 indicates that geom_node_qp_offset_eq0_flag is present in every geometry node at the current tree level; node_qp_offsets_present_flag equal to 0 indicates that geom_node_qp_offset_eq0_flag is not present in any geometry node at the current tree level. When not present, geom_node_qp_offset_eq0_flag is inferred to be equal to 0.

[0109] In this example, the bitstream of data does not include any markings indicating the first node in a particular tree level (e.g., that the first node has a particular depth). However, this may be interpreted in the codec's specification. For example, in some implementations, the codec may specify that there is always a first level containing one node, and that the occupancy information for all nodes in a level indicates the number of nodes in the next level).

[0110] In some implementations, the geometry node syntax may be modified such that the data flag node_qp_offsets_present is decoded when !nodeQpOffsetsSignalled and nodeIdx=0.

[0111] Exemplary Process

[0112] Figure 5A An exemplary process 500 for generating a data structure representing a point cloud in three-dimensional space is shown. The process 500 may be performed at least in part using one or more devices (e.g., Figure 8 One or more computer systems shown).

[0113] According to process 500 , a computer system receives a plurality of points representing a point cloud in three-dimensional space (block 502 ).

[0114] The computer system represents the position of the point in each dimension of the three-dimensional space as a bit sequence (block 504). The position of the point is represented according to a tree data structure. In some implementations, the tree data structure can be an octree.

[0115] The computer system partitions at least one of the bit sequences into a first portion of bits and a second portion of bits (block 506). In some implementations, this may include partitioning the at least one of the bit sequences such that the first portion of bits includes the first d bits from the corresponding bit sequence in the bit sequence, where d is the scaled depth of the tree data structure. Additionally, this may include partitioning the at least one of the bit sequences such that each of the second portions of bits includes the remaining portion of bits (e.g., bits other than the first d bits) from the corresponding bit sequence in the bit sequence.

[0116] The computer system quantizes each of the second portion of bits according to a quantization step size (block 508). The quantization step size is determined according to an exponential function having a quantization parameter value as input and a quantization step size as output. In some implementations, quantizing each of the second portion of bits may include dividing the numerical value indicated by the second portion of bits by the quantization step size.

[0117] In some implementations, the exponential function can be selected such that for positive integer quantization parameter values, a majority of the quantization step sizes output by the exponential function are integers. In some implementations, the exponential function can be selected such that for positive integer quantization parameter values, N of the quantization step sizes output by the exponential function are non-integer numbers, where N = (p × log2p) - p + 1, and where p is the number of quantization parameters for each doubling of the quantization step size. In some implementations, the exponential function can be Where QS is the quantization step size, and QP is the quantization parameter value.

[0118] The computer system generates a data structure representing a point cloud in three-dimensional space (block 510).The data structure includes a second portion of quantized bits.

[0119] The computer system transfers and / or stores the data structure (block 512).

[0120] In some implementations, the computer system may also approximate at least some of the second portion of the quantized bits to integer values and include the approximated second portion of bits in the data structure.

[0121] In some implementations, the computer system can also indicate at least one of a quantization step size or a quantization parameter value in a data structure.

[0122] Figure 5B An exemplary process 520 for rendering three-dimensional content based on a data structure representing a point cloud is shown. The process 520 may be performed at least in part using one or more devices (e.g., Figure 9 One or more computer systems shown).

[0123] According to process 520 , a computer system receives a data structure representing a plurality of points in a point cloud in three-dimensional space (block 522 ).

[0124] The computer system decodes the data structure to determine one or more bit sequences (block 524). Each bit sequence represents the position of a point in one dimension of three-dimensional space. The data structure is decoded according to a tree data structure. In some implementations, the tree data structure can be an octree.

[0125] The computer system partitions at least one of the bit sequences into a first portion of bits and a second portion of bits (block 526). In some implementations, at least some of the bit sequences can be partitioned such that each of the first portions of bits includes the first d bits from a corresponding bit sequence in the bit sequence, where d is a scaled depth of the tree data structure. Additionally, at least some of the bit sequences can be partitioned such that each of the second portions of bits includes the remaining portions of bits from a corresponding bit sequence in the bit sequence.

[0126] The computer system scales each of the second portion of bits according to a quantization step size (block 528). The quantization step size is determined according to an exponential function that has a quantization parameter value as input and a quantization step size as output. In some implementations, scaling each of the second portion of bits may include multiplying the numerical value indicated by each of the second portion of bits by the quantization step size.

[0127] In some implementations, the exponential function can be selected such that for positive integer quantization parameter values, a majority of the quantization step sizes output by the exponential function are integers.

[0128] In some embodiments, the exponential function can be selected so that for positive integer quantization parameter values, N of the quantization step sizes output by the exponential function are non-integer, where N = (p×log2p)-p+1, and where p is the number of quantization parameters for each doubling of the quantization step size.

[0129] In some implementations, the exponential function can be Where QS is the quantization step size, and QP is the quantization parameter value.

[0130] The computer system generates a reconstructed point position for at least one of the points by combining, for each of the at least one of the points, a first portion of bits and a second portion of bits corresponding to a scale of the point (block 530 ).

[0131] The computer system outputs a reconstructed point location of at least one of the points (block 532 ).

[0132] In some implementations, the quantization step size and / or the quantization parameter value can be indicated or included in a data structure.

[0133] In some implementations, the process may further include rendering three-dimensional content based on the reconstructed point position of at least one of the points. The three-dimensional content may be virtual reality content and / or augmented reality content.

[0134] Figure 6A An exemplary process 600 for generating a data structure representing a point cloud in three-dimensional space is shown. Process 600 may be performed at least in part using one or more devices (e.g., Figure 9 One or more computer systems shown).

[0135] According to process 600 , a computer system receives a plurality of points representing a point cloud in three-dimensional space (block 602 ).

[0136] The computer system generates a data structure representing a point cloud in three-dimensional space (block 604). The data structure includes a header portion and a data portion following the header portion. For each point, the data portion indicates the location of the point, which is encoded according to a tree data structure and quantized according to a corresponding quantization parameter. The tree data structure has multiple tree levels. In some implementations, the tree data structure can be an octree.

[0137] Furthermore, the data portion includes an offset value indicating, at one or more of the tree levels, a difference between a quantization parameter used to quantize a position of a point at the one or more tree levels and a base quantization parameter. The data portion includes, at each of the one or more tree levels, a data flag indicating the presence of one or more offset values at the tree level.

[0138] In some implementations, for each of one or more tree levels, the data tag can be included at the beginning of the tree level.

[0139] In some implementations, the header portion does not include data indicating the presence of the offset value.

[0140] The computer system transfers or stores the data structure (block 606).

[0141] Figure 6B An exemplary process 620 for rendering three-dimensional content based on a data structure representing a point cloud is shown. The process 620 may be performed at least in part using one or more devices (e.g., Figure 9 One or more computer systems shown).

[0142] According to process 620, a computer system receives a data structure representing a point cloud in three-dimensional space (block 622). The point cloud has a plurality of points. The data structure includes a header portion and a data portion following the header portion. For each point, the data portion indicates the location of the point, which is encoded according to a tree data structure and quantized according to a corresponding quantization parameter. The tree data structure has a plurality of tree levels. In some implementations, the tree data structure may be an octree.

[0143] Furthermore, the data portion includes an offset value indicating, at one or more of the tree levels, a difference between a quantization parameter used to quantize a position of a point at the one or more tree levels and a base quantization parameter. The data portion includes, at each of the one or more tree levels, a data flag indicating the presence of one or more offset values at the tree level.

[0144] The computer system determines the position of each point based on the data structure (block 624). In some implementations, this may include determining that one or more of the offset values exist at a particular tree level based on the data marker at the tree level; determining the quantization parameter for quantizing the position of the point at the tree level based on the offset value at the tree level; and determining the position of each of the points at the tree level based on the determined quantization parameter.

[0145] The computer system outputs the determined position of each point (block 626).

[0146] In some implementations, the method may further include rendering three-dimensional content based on the determined position of each of the points. The three-dimensional content may be at least one of virtual reality content or augmented reality content.

[0147] Figure 7 Shown is a compressed point cloud being used in a 3D application.

[0148] In some embodiments, a sensor (such as sensor 102), an encoder (such as encoder 104), and a decoder (such as decoder 116) may be used to transmit a point cloud in a 3D application. For example, at block 702, a sensor (such as sensor 102) may capture a 3D image, and at block 704, the sensor or a processor associated with the sensor may perform 3D reconstruction based on the sensed data to generate a point cloud.

[0149] At block 706, an encoder (such as encoder 104) may compress the point cloud, and at block 708, the encoder or post-processor may package the compressed point cloud and transmit it via network 710. At block 712, the data packet may be received at a destination location including a decoder (such as decoder 116). At block 714, the decoder may decompress the point cloud, and at block 716, the decompressed point cloud may be rendered. In some embodiments, the 3D application may transmit the point cloud data in real time, such that the display at block 716 represents the image being observed at block 702. For example, at block 616, a camera in the canyon may allow a remote user to experience walking through a virtual canyon.

[0150] Figure 8 A compressed point cloud is shown being used in a virtual reality (VR) or augmented reality (AR) application.

[0151] In some embodiments, the point cloud may be generated in software (e.g., as opposed to being captured by a sensor). For example, at block 802, virtual reality or augmented reality content is generated. The virtual reality or augmented reality content may include point cloud data and non-point cloud data. For example, a non-point cloud character may traverse terrain represented by a point cloud. At 804, the point cloud data may be compressed, and at block 806, the compressed point cloud data and the non-point cloud data may be packaged and transmitted via a network 808. For example, the virtual reality or augmented reality content generated at block 802 may be generated at a remote server and transmitted to a VR or AR content consumer via network 808. At block 810, the data packets may be received and synchronized at the VR or AR consumer's device. At block 812, a decoder operating at the VR or AR consumer's device may decompress the compressed point cloud, and the point cloud and non-point cloud data may be rendered in real time, for example, on a head-mounted display of the VR or AR consumer's device. In some embodiments, the point cloud data may be generated, compressed, decompressed, and rendered in response to the VR or AR consumer manipulating the head-mounted display to look in different directions.

[0152] In some embodiments, point cloud compression as described herein can be used in various other applications such as geographic information systems, live sports broadcasts, museum displays, autonomous navigation, etc.

[0153] Exemplary Computer System

[0154] Figure 9 An exemplary computer system 900 is shown that can implement an encoder or decoder or any other of the components described herein (e.g., as described above with reference to FIG. Figures 1 to 8). The computer system 900 can be configured to perform any or all of the above-described embodiments. In various embodiments, the computer system 900 can be any of various types of devices, including, but not limited to, a personal computer system, a desktop computer, a laptop computer, a notebook computer, a tablet computer, an all-in-one computer, a tablet or netbook computer, a mainframe computer system, a handheld computer, a workstation, a network computer, a camera, a set-top box, a mobile device, a consumer device, a video game controller, a handheld video game device, an application server, a storage device, a television, a video recording device, a peripheral device (such as a switch, a modem, a router), or generally any type of computing or electronic device.

[0155] The various embodiments of the point cloud encoder or decoder described herein may be executed on one or more computer systems 900, which may interact with various other devices. Figures 1 to 8 Any component, action, or functionality described may be implemented in a system configured as Figure 9 900. In the illustrated embodiment, the computer system 900 includes one or more processors 910 coupled to a system memory 920 via an input / output (I / O) interface 930. The computer system 900 also includes a network interface 940 coupled to the I / O interface 930, and one or more input / output devices 950, such as a cursor control device 960, a keyboard 970, and one or more displays 980. In some cases, it is contemplated that the embodiments may be implemented using a single instance of the computer system 900, while in other embodiments, multiple such systems or multiple nodes comprising the computer system 900 may be configured to host different parts or instances of the embodiments. For example, in one embodiment, some elements may be implemented via one or more nodes of the computer system 900 that are different from those that implement other elements.

[0156] In various embodiments, computer system 900 can be a uniprocessor system including one processor 910, or a multiprocessor system including a plurality of processors 910 (e.g., two, four, eight, or another suitable number). Processor 910 can be any suitable processor capable of executing instructions. For example, in various embodiments, processor 910 can be a general-purpose or embedded processor that implements any of a variety of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, or MIPS ISAs, or any other suitable ISAs. In a multiprocessor system, each of processors 910 can typically, but not necessarily, implement the same ISA.

[0157] The system memory 920 can be configured to store point cloud compression or point cloud decompression program instructions 922 and / or sensor data accessible by the processor 910. In various embodiments, the system memory 920 can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash type memory, or any other type of memory. In the illustrated embodiment, the program instructions 922 can be configured to implement an image sensor control application that incorporates any of the functionality described above. In some embodiments, the program instructions and / or data can be received, sent, or stored on different types of computer-accessible media or on similar media separate from the system memory 920 or computer system 900. Although the computer system 900 is described as implementing the functionality of the functional blocks of the preceding figures, any functionality described herein can be implemented via such a computer system.

[0158] In one embodiment, the I / O interface 930 can be configured to coordinate I / O communications between the processor 910, the system memory 920, and any peripheral devices in the device (including the network interface 940 or other peripheral device interfaces, such as input / output devices 950). In some embodiments, the I / O interface 930 can perform any necessary protocol, timing, or other data conversion to convert data signals from one component (e.g., the system memory 920) into a format suitable for use by another component (e.g., the processor 910). In some embodiments, the I / O interface 930 can include support for devices attached, for example, via various types of peripheral buses (such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard). In some embodiments, the functionality of the I / O interface 930 can be split into two or more separate components, such as a northbridge and a southbridge, for example. Furthermore, in some embodiments, some or all of the functionality of the I / O interface 930 (such as the interface to the system memory 920) can be incorporated directly into the processor 910.

[0159] The network interface 940 can be configured to allow data to be exchanged between the computer system 900 and other devices (e.g., carriers or proxy devices) attached to the network 985, or between nodes of the computer system 900. In various embodiments, the network 985 can include one or more networks, including but not limited to a local area network (LAN) (e.g., an Ethernet or an enterprise network), a wide area network (WAN) (e.g., the Internet), a wireless data network, some other electronic data network, or some combination thereof. In various embodiments, the network interface 940 can support communication via a wired or wireless general data network (such as any suitable type of Ethernet network), for example; via a telecommunications / telephone network (such as an analog voice network or a digital fiber optic communication network); via a storage area network (such as a Fibre Channel SAN), or via any other suitable type of network and / or protocol.

[0160] In some embodiments, input / output devices 950 may include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other device suitable for inputting or accessing data by one or more computer systems 900. Multiple input / output devices 950 may be present in computer system 900 or may be distributed across various nodes of computer system 900. In some embodiments, similar input / output devices may be separate from computer system 900 and may interact with one or more nodes of computer system 900 through a wired or wireless connection, such as through network interface 940.

[0161] like Figure 9 As shown, memory 920 may include program instructions 922, which may be executable by the processor to implement any of the elements or actions described above. In one embodiment, the program instructions may execute the method described above. In other embodiments, different elements and data may be included. It should be noted that the data may include any of the data or information described above.

[0162] Those skilled in the art will appreciate that computer system 900 is merely illustrative, and is not intended to limit the scope of the embodiments. Specifically, computer system and equipment may include any combination of hardware or software that can perform the functions indicated, including computers, network equipment, internet equipment, PDAs, wireless telephones, pagers, etc. Computer system 900 may also be connected to other devices not shown, or may otherwise operate as an independent system. In addition, the functions provided by the shown components may be combined in fewer components or distributed in additional components in some embodiments. Similarly, in some embodiments, the functions of some components in the shown components may not be provided, and / or other additional functions may be available.

[0163] Those skilled in the art will also recognize that, although various items are shown as being stored in memory or on storage device during use, for the purpose of memory management and data integrity, these items or parts thereof can be transmitted between memory and other storage devices. Alternatively, in other embodiments, some or all of these software components can be executed in the memory on another device, and communicate with the illustrated computer system via inter-computer communication. Some or all of system components or data structures can also be stored on computer accessible media or portable articles (for example as instructions or structured data) to be read by suitable drivers, and its multiple examples are described above. In some embodiments, the instructions stored on the computer accessible media separated from the computer system 900 can be transferred to the computer system 900 via transmission media or signal (such as electrical signals, electromagnetic signals or digital signals transmitted via communication media such as networks and / or wireless links). Various embodiments can also include receiving, sending or storing instructions and / or data implemented according to the above description on a computer accessible medium. Generally speaking, computer-accessible media may include non-transitory computer-readable storage media or memory media, such as magnetic or optical media, e.g., disks or DVD / CD-ROMs, volatile or non-volatile media, such as RAM (e.g., SDRAM, DDR, RDRAM, SRAM, etc.), ROM, etc. In some embodiments, computer-accessible media may include transmission media or signals, such as electrical, electromagnetic, or digital signals transmitted via a communication medium, such as a network and / or wireless link.

[0164] In different embodiments, the methods described herein can be implemented in software, hardware, or a combination thereof. In addition, the order of the method frames can be changed, and various elements can be added, reordered, combined, omitted, modified, etc. For those skilled in the art who benefit from this disclosure, it is obvious that various modifications and changes can be made. The various embodiments described herein are intended to be illustrative and not restrictive. Many variations, modifications, additions, and improvements are possible. Therefore, multiple examples can be provided for the components described as a single example in this article. The boundaries between various components, operations, and data repositories are arbitrary to a certain extent, and specific operations are shown in the context of specific exemplary configurations. Other allocations of functions are contemplated and may fall within the scope of the appended claims. Finally, the structure and function presented as discrete components in the exemplary configuration may be implemented as a combined structure or component. These and other variations, modifications, additions, and improvements may fall within the scope of the embodiments defined in the following claims.

Claims

1. A method comprising: Receiving, by a computer system, a data structure representing a plurality of points in a point cloud in three-dimensional space; decoding, by the computer system, the data structure to determine one or more bit sequences, wherein each of the bit sequences represents a position of one of the points in one of the dimensions of the three-dimensional space, and wherein the data structure is decoded according to a tree data structure; dividing, by the computer system, at least one of the bit sequences into a first portion of bits and a second portion of bits; scaling, by the computer system, each of the second portion of bits according to a quantization step size, wherein the quantization step size is determined according to an exponential function having as input a quantization parameter value and having as output the quantization step size; generating, by the computer system, a reconstructed point position of at least one of the points by combining, for each of at least one of the points, the first portion of the bits and a second portion of the bits corresponding to a scale of the point; as well as The reconstructed point position of the at least one of the points is output by the computer system. The method of claim 1 , wherein the tree data structure is an octree.

3. The method of claim 1 , wherein partitioning at least one of the bit sequences comprises: partitioning at least one of the bit sequences such that each of the first portions of bits includes the first d bits from a corresponding one of the bit sequences, where d is a scaled depth of the tree data structure, and At least one of the bit sequences is partitioned such that each of the second portions of bits includes a remainder of bits from a corresponding one of the bit sequences. 4 . The method of claim 1 , wherein scaling each of the second portions of bits comprises multiplying a numerical value indicated by each of the second portions of bits by the quantization step size. 5 . The method of claim 1 , wherein at least one of the quantization step size or the quantization parameter value is indicated in the data structure.

6. The method of claim 1, wherein the exponential function is selected such that for positive integer quantization parameter values, a majority of the quantization step sizes output by the exponential function are integers.

7. The method of claim 1 , wherein the exponential function is selected so that for positive integer quantization parameter values, N of the quantization step sizes output by the exponential function are non-integer, where N = (p×log2p)−p+1, and where p is the number of quantization parameters for each doubling of the quantization step size.

8. The method of claim 1, wherein the exponential function is Where QS is the quantization step size, and QP is the quantization parameter value.

9. The method of claim 1, further comprising rendering three-dimensional content based on the reconstructed point position of the at least one of the points, wherein the three-dimensional content is at least one of virtual reality content or augmented reality content.

10. A device comprising: one or more processors; and A memory storing instructions which, when executed by the one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 9.

11. A non-transitory computer-readable storage medium having instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 9.

Images