Apparatus and media for point cloud geometry compression using octrees with multiple scan orders

Abstract

The encoder is configured to compress point cloud geometry information using an octree geometry compression technique that utilizes node groups. Nodes within a node group are scanned according to a breadth-first scan order. The order node group to be evaluated can be selected according to the breadth-first scan order or a depth-first scan order based on whether the breadth-first scan order or the depth-first scan order is indicated in a flag in a previous node group that was evaluated. In some embodiments, the evaluation order of node groups can be implicit without being signaled via a flag. The decoder is configured to reconstruct a point cloud based on a bitstream encoded by the encoder.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to compression and decompression of point clouds comprising a plurality of points, each point having associated spatial and / or attribute information. BACKGROUND

[0002] Various types of sensors, such as light detection and ranging (LIDAR) systems, 3D cameras, 3D scanners, and the like, can capture data indicative of the position of points in three-dimensional space (e.g., position in the X, Y, and Z planes). Additionally, such systems can capture attribute information in addition to spatial information for respective points, such as color information (e.g., RGB values), intensity attributes, reflectivity attributes, motion-related attributes, modal attributes, or various other attributes. In some cases, additional attributes can be assigned to respective points, such as a timestamp of when the point was captured. The points captured by such sensors can constitute a “point cloud” comprising a set of points each having associated spatial information and one or more associated attributes. In some cases, a point cloud can comprise thousands of points, hundreds of thousands of points, millions of points, or even more. Additionally, in some cases, a point cloud can be generated, e.g., in software, as opposed to being captured by one or more sensors. In either case, such point clouds can comprise a large amount of data, and storing and transmitting these point clouds can be costly and time-consuming. SUMMARY

[0003] In some embodiments, a decoder stores program instructions that, when executed by one or more processors, cause the processors to receive a bitstream that includes a plurality of node groups for nodes of an octree structure generated for a point cloud that has been compressed. The nodes of the octree structure define spatial locations of points of the point cloud in a three-dimensional space, and each node group includes a subset of a set of nodes of the octree structure of the point cloud. Using the received bitstream, the program instructions of the decoder, when executed by the one or more processors, further cause the one or more processors to reconstruct a geometry of the point cloud based on the nodes included in respective node groups of the bitstream. To perform the reconstruction, the program instructions cause the one or more processors to scan the nodes within a given node group according to a breadth-first scan order and to select a next subsequent node group to evaluate according to a depth-first scan order. Scanning the nodes within a node group according to a breadth-first scan order can improve decoding efficiency for decoding the entropy encoded occupancy information of the nodes of the node group, for example, by allowing an entropy encoder to inform an entropy decoding process using contexts determined for neighboring nodes. At the same time, limiting the breadth-first scan order to the nodes included in a node group can reduce memory requirements at the decoder to buffer a large number of nodes being evaluated, such as determining entropy decoding contexts. Additionally, selecting the next node group to evaluate in a depth-first scan order can improve resilience of the compressed / encoded point cloud. For example, leaf nodes can be reached relying on fewer higher level node groups than would be the case if the node groups were selected for evaluation according to a breadth-first scan order. The leaf nodes can define locations of output points to be included in the reconstructed point cloud. Thus, relying on fewer node groups to reach the leaf nodes can reduce a probability of an inability to determine the output points due to, for example, a loss of an intervening node group in transmission of the compressed point cloud.

[0004] In some embodiments, an encoder stores program instructions that, when executed by one or more processors, cause the processors to receive a point cloud to compress, where the point cloud includes a plurality of points in a three-dimensional space. To compress the point cloud, the program instructions of the encoder cause the one or more processors to generate an octree structure that includes occupancy information for nodes of the octree structure, where the occupancy information indicates whether a cube or sub-cube at a given level of the octree structure is occupied by one or more points of the point cloud. The program instructions of the encoder further cause the one or more processors to group the nodes of the octree into a plurality of node groups, where each node group includes a subset of the nodes of the high octree structure. Additionally, the program instructions of the encoder cause the one or more processors to generate a bitstream that includes: encoded occupancy information for the nodes of the octree included in the node groups, where the nodes within a given node group are organized according to a breadth-first scan order, and where the occupancy information for the nodes of subsequent node groups to evaluate is selected between the node groups according to a depth-first scan order.

[0005] In some embodiments, the decoder additionally or alternatively stores program instructions that, when executed by one or more processors, cause the one or more processors to receive a bitstream comprising a slice of an octree structure of a point cloud, wherein the octree structure of the point cloud comprises a set of nodes defining spatial positions of points in the point cloud, and wherein the slice comprises a subset of the set of nodes of the octree structure and a flag indicating an end of the slice. The bitstream further comprises two or more additional slices of the octree structure of the point cloud, wherein each of the two or more additional slices comprises an additional respective subset of the set of nodes of the octree structure and an additional respective flag indicating a respective end of the two or more additional slices. In some embodiments, the slice or the two or more additional slices can each comprise a plurality of groups of nodes. Additionally, the bitstream further comprises, for at least the two or more additional slices, an indication of a location of at least one node of the respective slice in the octree structure, wherein the indicated location is indicated using a reference to a particular tree level of the octree structure and a spatial position within the tree level or a reference to a particular tree level of the octree structure and a node index position within the tree level. Using the received bitstream, the program instructions of the decoder further cause the one or more processors to reconstruct a geometry of the point cloud, wherein to reconstruct the geometry, the respective node subsets of the two or more additional slices are located in the octree structure by the decoder based on their respective indicated locations included in the bitstream. For example, once at least one node of a slice is correctly located in the octree structure, the locations of other nodes of the slice in the octree structure can be determined by the decoder based on the relationship of the nodes to the node of the slice whose location in the octree structure is indicated. In this way, if a slice is lost, or if slices are decoded in parallel, the nodes of the slice can be placed in the correct locations in the octree structure based on the indicated location of the at least one node of the slice.

[0006] In some implementations, the encoder additionally or alternatively stores program instructions that, when executed by one or more processors, cause the processors to receive a point cloud to be compressed, wherein the point cloud comprises multiple points in three-dimensional space. The encoder's program instructions also cause the one or more processors to generate an octree structure including occupancy information of nodes in an octree structure, wherein the occupancy information indicates whether a cube or sub-cube of the octree at a given level is occupied by one or more points of the point cloud. Additionally, the encoder's program instructions cause the one or more processors to organize the nodes of the octree structure into multiple slices, the multiple slices comprising: a first slice of the octree structure comprising a subset of the set of nodes of the octree structure; and two or more additional slices of the octree structure, wherein each of the two or more additional slices comprises an additional corresponding subset of the set of nodes of the octree structure. Furthermore, the encoder's program instructions cause the one or more processors to encode the occupancy information of the octree nodes included in the first slice and the two or more additional slices, and generate a bitstream of a compressed version of the point cloud. The bitstream includes: encoded occupancy information of nodes included in the first slice and a flag indicating the end of the first slice, and encoded occupancy information of nodes included in the two or more additional slices and additional corresponding flags indicating the respective ends of the two or more additional slices. Additionally, the bitstream includes: for at least the two or more additional slices, an indication of the position of at least one node in the corresponding slice within the octree structure. This indication is made using a reference to a specific tree level of the octree structure and a spatial location within that tree level, or a reference to a specific tree level of the octree structure and a node index position within that tree level. Attached Figure Description

[0007] Figure 1 A system according to some embodiments is shown, which includes a sensor that captures information about points in a point cloud and an encoder that compresses attribute information and / or spatial information of the point cloud, wherein the compressed point cloud information is sent to a decoder.

[0008] Figure 2 An example of a simplified octree structure according to some implementation schemes is shown.

[0009] Figure 3 An exemplary breadth-first scan order of nodes in an octree structure according to some implementations is shown.

[0010] Figure 4 An exemplary depth-first scan order of nodes in an octree structure according to some implementations is shown.

[0011] Figure 5The nodes of an octree structure grouped into multiple node groups are shown according to some implementation schemes.

[0012] Figure 6 The diagram illustrates a depth-first scan order for selecting node groups to be evaluated, according to some implementations, wherein nodes within the corresponding node groups are scanned according to a breadth-first scan order.

[0013] Figure 7 An octree structure with nodes grouped into multiple node groups according to some implementations is shown, where a flag indicates the evaluation order of subsequent node groups for selecting the node group to be evaluated.

[0014] Figure 8 The diagram illustrates a node group evaluated according to a hybrid depth-first / breadth-first scan order according to some embodiments, wherein at the end of the node group, the value of a flag or the absence of a flag indicates whether the next node group to be evaluated is selected according to a depth-first scan order or a breadth-first scan order, and wherein the nodes within the corresponding node group are scanned according to a breadth-first scan order.

[0015] Figure 9 The diagram illustrates a group of nodes organized into multiple slices according to some embodiments, wherein each group of nodes contains a flag indicating whether the group of nodes is the last group of nodes for a given slice.

[0016] Figure 10 The diagram illustrates a group of nodes organized into multiple slices according to some embodiments, wherein each group of nodes contains a flag indicating whether it is the last group of nodes for a given slice, and wherein the slices are arranged sequentially in the order in which they will be received, for example, by the decoder.

[0017] Figure 11A and Figure 11B The diagram shows groups of nodes organized into multiple slices, where each group of nodes contains a flag indicating whether it is the last group of nodes in a given slice. Figure 11A and Figure 11B The entropy context savepoints for one or more nodes for a given slice, according to some implementations, are also shown.

[0018] Figure 12A and Figure 12B The diagram shows groups of nodes organized into multiple slices, where each group of nodes contains a flag indicating whether it is the last group of nodes in a given slice. Figure 12A and Figure 12B Entropy context storage points for one or more nodes indicated for a given slice are also shown according to some implementations, wherein at least some slices are entropy-dependent slices that depend on a common upper slice.

[0019] Figure 13 An exemplary octree structure is shown, in which subtrees terminate early according to some implementation schemes.

[0020] Figure 14A Exemplary cubes and sub-cubes that can be used at different levels of an octree according to some embodiments are shown, wherein a node is indicated as occupied if it is included in the volume of the cube or sub-cube corresponding to a node at a given level of the octree structure, and a node is indicated as unoccupied if it is not included in the volume of the cube or sub-cube corresponding to that node.

[0021] Figure 14B An exemplary neighborhood configuration of a cube or sub-cube corresponding to a given node of an octree is shown according to some embodiments, wherein the neighborhood configuration is used to determine an entropy encoding / decoding context for encoding and decoding occupancy information of the given node.

[0022] Figure 15A The components of an encoder according to some implementation schemes are shown.

[0023] Figure 15B The components of a decoder according to some implementation schemes are shown.

[0024] Figure 16 This illustrates compressed point cloud information used in 3D teledisplay applications according to some implementation schemes.

[0025] Figure 17 This illustrates compressed point cloud information used in virtual reality applications according to some implementation schemes.

[0026] Figure 18 An exemplary computer system that can implement an encoder or decoder according to some implementation schemes is shown.

[0027] This specification includes references to "one embodiment" or "implementation". The use of the phrase "in one embodiment" or "in an embodiment" does not necessarily refer to the same embodiment. Specific features, structures, or characteristics may be combined in any suitable manner consistent with this disclosure.

[0028] The term "comprising" is open-ended. As used in the appended claims, it does not exclude additional structures or steps. Consider the following cited claim: "An apparatus comprising one or more processor units..." Such claims do not exclude the inclusion of additional components (e.g., network interface units, graphics circuitry, etc.).

[0029] "Configured as" refers to various units, circuits, or other components that can be described or stated as being "configured as" to perform one or more tasks. In such a context, "configured as" is used to imply a structure (e.g., a circuit) that includes a unit / circuit / component that performs one or more tasks during operation. Thus, a unit / circuit / component is allegedly configured to perform the task even when the specified unit / circuit / component is currently inoperable (e.g., not switched on). Units / circuits / components used with the language "configured as" include hardware—e.g., circuits, memory storing program instructions that can be executed to perform the operation, etc. Referring to a unit / circuit / component as being "configured as" to perform one or more tasks is explicitly intended to exclude reference to 35 U.S.SC §112(f) for that unit / circuit / component. Furthermore, "configured as" can include general structures (e.g., general-purpose circuits) manipulated by software and / or firmware (e.g., FPGAs or general-purpose processors executing software) in a manner capable of performing one or more tasks to be solved. "Configured to" may also include adjusting the manufacturing process (e.g., a semiconductor manufacturing facility) to manufacture equipment (e.g., an integrated circuit) suitable for performing one or more tasks.

[0030] "First," "second," etc. As used herein, these terms serve as labels for the nouns preceding them and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.). For example, a buffer circuit may be described herein as performing write operations on a "first" value and a "second" value. The terms "first" and "second" do not necessarily imply that the first value must be written before the second value.

[0031] "Based on." As used herein, this term describes one or more factors that influence the determination. This term does not exclude additional factors influencing the determination. That is, the determination may be based solely on these factors or at least partially on them. Consider the phrase "A is determined based on B." In this case, B is the factor influencing the determination of A, and such phrases do not exclude the possibility that the determination of A may also be based on C. In other instances, A may be determined solely on B. Detailed Implementation

[0032] As data acquisition and display technologies become more advanced, the ability to capture point clouds containing tens of thousands of points in 2D or 3D space has been enhanced (e.g., via LiDAR systems). Furthermore, advancements in advanced display technologies (such as virtual reality or augmented reality systems) have increased the potential uses of point clouds. However, point cloud files are typically very large, and storing and transmitting them can be costly and time-consuming. For example, communication of point clouds over private or public networks (such as the Internet) can require considerable time and / or network resources, potentially limiting some uses of the point cloud data (such as real-time use). Additionally, the storage requirements of point cloud files can consume a significant amount of storage capacity on the devices storing them, which may also limit the use of point cloud data by potential applications.

[0033] In some implementations, an encoder can be used to generate compressed point clouds to reduce the cost and time associated with storing and transmitting large point cloud files. In some implementations, the system may include an encoder that compresses the attribute information and / or spatial information (also referred to herein as geometric information) of the point cloud file, enabling the point cloud file to be stored and transmitted faster than an uncompressed point cloud, and in a manner that allows the point cloud file to occupy less storage space than an uncompressed point cloud. In some implementations, compression of the spatial information and / or attributes of points in the point cloud enables the point cloud to be transmitted over a network in real-time or near real-time. For example, the system may include a sensor that captures spatial information and / or attribute information about points in the environment in which the sensor is located, wherein the captured points and corresponding attributes constitute a point cloud. The system may also include an encoder that compresses the attribute information of the captured point cloud. The compressed attribute information of the point cloud can be transmitted over a network in real-time or near real-time to a decoder that decompresses the compressed attribute information of the point cloud. The decompressed point cloud can be further processed, for example, to make control decisions based on the surrounding environment at the sensor location. The control decision can then be transmitted back to a device at or near the sensor location, where the receiving device implements the control decision in real time or near real time. In some embodiments, the decoder can be associated with an augmented reality system, and the decompressed spatial and / or attribute information can be displayed or otherwise used by the augmented reality system. In some embodiments, compressed attribute information about the point cloud can be transmitted together with compressed spatial information about the points in the point cloud. In other embodiments, the spatial information and attribute information can be encoded and / or transmitted separately to the decoder.

[0034] In some implementations, the system may include a decoder that receives, via a network, one or more point cloud files, including compressed spatial information about the point clouds, from a remote server or other storage device storing one or more point cloud files. For example, a 3D display, holographic display, or head-mounted display can be manipulated in real-time or near real-time to display different parts of a virtual world represented by point clouds. To update the 3D display, holographic display, or head-mounted display, the system associated with the decoder can request point cloud files from a remote server based on user actions on the display, and these point cloud files can be transferred from the remote server to the decoder and decoded by the decoder in real-time or near real-time. The display can then be updated with updated point cloud data (such as updated point cloud geometry) in response to user actions.

[0035] In some implementations, a system may include one or more LiDAR systems, 3D cameras, 3D scanners, etc., and such sensor devices can capture spatial information, such as the X, Y, and Z coordinates of points in the sensor device's view. In some implementations, the spatial information may be relative to a local coordinate system or may be relative to a global coordinate system (e.g., a Cartesian coordinate system may have a fixed reference point, such as a fixed point on Earth, or may have a non-fixed local reference point, such as the sensor's location).

[0036] In some implementations, such sensors may also capture attribute information about one or more points, such as color attributes, reflectivity attributes, velocity attributes, acceleration attributes, temporal attributes, modal and / or various other attributes. In some implementations, in addition to LIDAR systems, 3D cameras, 3D scanners, etc., other sensors may capture attribute submission information to be included in the point cloud file. For example, in some implementations, a gyroscope or accelerometer may capture motion information to be included in the point cloud file as attributes associated with one or more points in the point cloud. For example, a vehicle equipped with a LIDAR system, 3D camera, or 3D scanner may include the vehicle's orientation and speed in the point cloud captured by the LIDAR system, 3D camera, or 3D scanner. For example, when points are captured within the vehicle's field of view, these points may be included in the point cloud, which includes the captured points and associated motion information corresponding to the vehicle's state at the time the points were captured.

[0037] In some implementations, attribute information may include string values, such as different modalities. For example, attribute information may include string values ​​indicating modalities such as "walking," "running," "driving," etc. In some implementations, the encoder may include "string value" to integer indices, where certain strings are associated with certain corresponding integer values. In some implementations, the point cloud may indicate a string value for a point by including an integer associated with that string value as an attribute of that point. Both the encoder and decoder may store common string values ​​as integer indices, allowing the decoder to determine the string value for a point based on finding the integer value of the string attribute of that point in a string value to integer index that matches or is similar to the decoder's string value to integer index.

[0038] In some implementations, in addition to compressing attribute information about the points in the point cloud, the encoder also compresses and encodes the spatial information of the point cloud. For example, to compress spatial information, an octree can be generated, where the corresponding occupied / unoccupied state of each cube and / or sub-cube of the octree is encoded. Note that the cubes / sub-cubes of the octree can correspond to nodes in the octree structure, where each node is indicated as occupied or unoccupied based on whether any point in the point cloud falls within the volume represented by the cube or sub-cube corresponding to a given node. This sequence of encoded occupied / unoccupied states of the sub-cubes can be encoded as occupancy symbols for the nodes of the octree, which transmit spatial information about the points in the point cloud to the decoder.

[0039] In some implementations, the encoder and / or decoder can determine the neighborhood occupancy configuration of a given cube corresponding to a node whose occupancy information in the occupancy tree is being encoded or decoded. The neighborhood occupancy configuration can indicate the occupancy status of neighboring cubes adjacent to the given cube corresponding to the node whose occupancy information is being encoded or decoded. For example, a cube whose neighboring cubes are occupied is more likely to also include occupied child cubes than a cube whose neighboring cubes are unoccupied. Figure 14B As shown and discussed in more detail below, for a given cube corresponding to a node whose occupancy information is being encoded, there are various possible neighborhood occupancy configurations.

[0040] In some implementations, the encoder and / or decoder can map specific neighborhood occupancy configurations to specific encoding contexts, where different encoding contexts are used to encode (or decode) occupancy information of nodes in an octree structure corresponding to cubes / sub-cubes with different neighborhood occupancy configurations. For example, Figure 14BTen possible neighborhood occupancy configurations are illustrated. In some embodiments, the encoder or decoder may utilize fewer than 10 encoding / decoding contexts. For example, in some embodiments, a set of more frequent neighborhood occupancy configurations may each be associated with a separate encoding / decoding context, and one or more sets of less frequent neighborhood occupancy configurations may be grouped together and associated with one or more shared encoding / decoding contexts. For example, in some embodiments, the number of encoding / decoding contexts may be reduced to six encoding contexts, where the five most frequent neighborhood occupancy configurations are each assigned a different encoding context, and the remaining five less frequent neighborhood occupancy configurations share a common encoding context.

[0041] Figure 1 A system according to some embodiments is shown, which includes a sensor that captures information about points in a point cloud and an encoder that compresses spatial and / or attribute information of the point cloud, wherein the compressed spatial and / or attribute information is sent to a decoder.

[0042] System 100 includes a sensor 102 and an encoder 104. Sensor 102 captures a point cloud 110, which includes points representing a structure 106 in a view 108 of sensor 102. For example, in some embodiments, structure 106 may be a mountain, building, signage, the environment surrounding a street, or any other type of structure. In some embodiments, the captured point cloud (such as captured point cloud 110) may include spatial and attribute information about the points included in the point cloud. For example, point A in captured point cloud 110 includes X, Y, Z coordinates and attributes 1, 2, and 3. In some embodiments, the location of points in the point cloud in 3D space may define the geometry of the point cloud, such as the spatial location of the points. In some embodiments, the attributes of the points may include attributes such as R, G, B color values, velocity at the point, acceleration at the point, reflectivity of the structure at the point, a timestamp indicating when the point was captured, a string value indicating the modality at which the point was captured, such as "walking," or other attributes. The captured point cloud 110 may be provided to the encoder 104, which generates a compressed version of the point cloud (compressed point cloud information 112), which is transmitted via network 114 to the decoder 116. In some embodiments, the compressed version of the point cloud (such as compressed point cloud information 112) may be included in a common compressed point cloud, which also includes compressed spatial information about the points in the point cloud; or in some embodiments, the compressed spatial information and compressed attribute information may be transmitted as separate files. In some embodiments, the compressed point cloud information 112 may include a bitstream for a compressed point cloud as described in more detail herein.

[0043] In some implementations, encoder 104 may be integrated with sensor 102. For example, encoder 104 may be implemented in hardware or software included in a sensor device (such as sensor 102). In other implementations, encoder 104 may be implemented on a separate computing device adjacent to sensor 102.

[0044] Figure 2 An exemplary simplified octree structure according to some implementation schemes is shown.

[0045] It should be noted that, for ease of illustration, a binary tree has been shown to represent a simplified octree. However, in three-dimensional space, each node of the illustrated octree structure may include more child nodes than shown in the two-dimensional diagram. Furthermore, it should be noted that while the description herein is based on an octree structure, any other suitable tree structure may be used in some embodiments.

[0046] An octree structure 200 includes nodes 202 and octree levels 204. The octree structure includes parent nodes and child nodes. For example, node 1 of octree level 0 includes two child nodes (nodes 2 and 3) of octree level 1. Similarly, node 2 has child node 4 in octree level 2, and node 3 has child node 5 in octree level 2. In some embodiments, octree nodes may have 1 to 8 child nodes. Furthermore, each node of the octree represents a spatial volume defined by a three-dimensional cube or sub-cubes, where each child node is a sub-cube of the cube corresponding to its parent node. In an octree in three-dimensional space, each cube of the parent node is divided into eight sub-cubes, which correspond to child nodes in the progression from one octree structure level to the next. In this way, the volume associated with a lower-level node is contained within the larger volume of a higher-level node. For example, Figure 2 All nodes shown have a volume that can be contained within the volume covered by node 1. As an example, Figure 14A The diagram illustrates cubes and sub-cubes at different subdivision levels (e.g., octree structure levels). Note that, for simplicity, Figure 14A It is shown in 2D, but the cube described here can be a three-dimensional cube.

[0047] To represent spatial information about a point cloud, the spatial location of the point cloud can be overlaid onto the volumes of the individual nodes of an octree structure. A given node is considered occupied if it or a child node of a given node includes at least one point in the point cloud within its corresponding volume. Otherwise, the node is considered unoccupied. For example, a point cloud may include sparse regions containing spatial volumes that do not contain any points. For these portions of the point cloud, the cube and sub-cube volumes corresponding to nodes in the octree structure can be unoccupied. An octree node with an occupied cube at a given level of the octree structure can continue to be divided into child nodes with sub-cubes (these sub-cubes correspond to nodes at the next lower level of the octree structure) until the sub-cube of a child node includes a single point in the point cloud. In this way, the spatial location of a single point in the point cloud can be defined by the location of the child node containing that single point in the octree. For example, leaf node 206 can define the location of individual points in 3D space.

[0048] For each parent node, the occupancy information of its child nodes can be encoded into a coded word. For example, if a parent node has two occupied child nodes, three occupied child nodes, etc., this information constitutes the parent node's occupancy information, which can be encoded into a coded word. In some implementations, an entropy encoder can be used to encode / decode occupancy information, such as the coded word of a node in an octree structure.

[0049] In a given region of an octree structure, nodes that are close to each other (e.g., adjacent nodes) may have similar occupancy states. For example, if the point cloud defines a flat surface (such as a wall), nodes in an octree structure falling along the wall may have similar occupancy states. These similarities in occupancy states can be beneficial for improving the efficiency of encoding occupancy information, for example, when using an entropy encoder / decoder and using the occupancy states of adjacent nodes to inform the entropy encoding / decoding context used to encode or decode the occupancy information of a given node in an octree structure using an entropy encoder / decoder.

[0050] For example, Figure 14BExemplary neighborhood configurations are shown for a given node whose occupancy information is being entropy-encoded / decoded (e.g., the node being encoded / decoded corresponds to a light-colored cube), where darker-colored cubes indicate occupied neighboring cubes corresponding to neighboring nodes of the given node in the octree structure. For example, in neighborhood configuration 1402, the node corresponding to the light-colored cube may not have any occupied neighboring nodes / cubes. In neighborhood configuration 1404, only one neighboring node / cube on one side of the node / cube whose occupancy information is being encoded can be occupied. In 1406, cubes on either side of the node / cube whose occupancy information is being encoded / decoded can be occupied. Similarly, other neighborhood configurations are shown for 1408, 1410, 1412, 1414, 1416, 1418, and 1420.

[0051] Entropy encoders / decoders can improve entropy coding performance for nodes whose occupancy information is being encoded / decoded by knowing the neighborhood configuration / occupancy information of neighboring nodes. However, to determine the occupancy information of neighboring nodes, the encoder or decoder may have to store the determined occupancy information in a memory buffer for neighboring nodes to determine the entropy coding context of subsequent evaluation nodes adjacent to the neighboring node whose occupancy information has been determined. While knowing the occupancy configuration of neighboring nodes can improve entropy coding efficiency, there may be a trade-off in the amount of storage space required to maintain a large amount of buffered occupancy information for neighboring nodes. Furthermore, requiring decoding dependencies between nodes can increase complexity and / or slow down the encoding and decoding of occupancy information, where nodes dependent on the entropy context from other encoded or decoded nodes must wait until the other encoded / decoded nodes are evaluated before the dependent node can be evaluated. Additionally, if a portion of the octree structure is lost during transmission, nodes that are dependent on the entropy context of other nodes lost in transmission may not be able to decode their occupancy information.

[0052] Therefore, as described in this paper, grouping nodes into groups that have entropy context dependencies within the node group but limited dependencies outside the node group can balance entropy coding efficiency with memory buffer requirements and / or complexity, as well as processing limitations caused by entropy context dependencies.

[0053] Furthermore, as further described herein, organizing nodes into slices that correspond in size to data transmission units (such as Internet Protocol (IP) packets) and include header information (which defines the starting point of the slice within the octree structure and includes an indication of the initial entropy context to be used for the slice) provides resilience, allowing the point cloud to be reconstructed at the decoder without the missing slice if a slice is lost during transmission. However, the portion of the point cloud corresponding to the missing slice can be rendered at a lower resolution than it would have had if the slice were not lost. This contrasts with other octree encoders / decoders—if a portion of the octree structure is lost during transmission, other octree encoders / decoders need to retransmit the complete octree structure. Additionally, because slices can be placed independently within the octree structure, and because slices have entropy contexts that do not necessarily depend on the immediately preceding slice being encoded / decoded, the occupancy information of nodes for multiple slices of the octree structure can be determined in parallel. Furthermore, in some implementations, such slices can be used to enable scalable decoding. For example, if a lower resolution version of the point cloud is acceptable, not all slices may need to be decoded. However, because slices can reside independently in an octree structure and have an entropy context that does not necessarily depend on the slice immediately preceding them, a scaled version of the point cloud can be reconstructed without using all slices of the octree structure to reconstruct a scaled version of the point cloud.

[0054] Figure 3 An exemplary breadth-first scan order of nodes in an octree structure according to some implementations is shown.

[0055] As used in this article, the breadth-first scan order follows the following... Figure 3 The sequence shown involves sequentially evaluating nodes (or groups of nodes) at a given octree level (such as level 1) until the end of that octree level is reached. At this point, the breadth-first scan continues to the next lower octree level in the octree structure. For example, in Figure 3 In this process, nodes are evaluated in the order indicated by the numbers and letters shown in the figure.

[0056] In other words, nodes (or groups of nodes) in the same level of the octree structure are scanned in breadth-first order, and if the end of a node (or group of nodes) in the same level of the octree structure is reached, the scanning order continues to scan the sequential nodes (or groups of nodes) in the next lower level of the octree structure.

[0057] Figure 4 An exemplary depth-first scan order of nodes in an octree structure according to some implementations is shown.

[0058] As used in this paper, nodes (or groups of nodes) are evaluated in the sequential hierarchy of the octree structure in a depth-first scan order until a leaf node is reached. At this point, the scan order backs up and then continues down to sibling nodes at lower levels. For example, as... Figure 4 As shown, when the leaf node is reached at position 4, the scan order backs up to the already evaluated node 3, and then continues down to the already evaluated node 5, which is a sibling node of the already evaluated node 4. It can also be seen that after node 8, the depth-first scan order backs up to node 1, because all nodes below node 2 have been evaluated. At this point, the depth-first scan order continues along the sibling path by continuing to nodes below node 9.

[0059] In other words, following the depth-first scan order, the next node (or node group) that is a child node at a lower level of the octree structure than the parent node included in the current node (or node group) being evaluated is selected as the next node (or node group) to be evaluated. If the octree structure does not include the child nodes of the nodes included in the current node (or node group) being evaluated at a lower level of the octree structure, the depth-first scan order returns to the nodes (or node groups) that include the parent nodes of the nodes being evaluated, where these parent nodes are at a higher level of the octree structure than the nodes being evaluated. If the nodes (or node groups) that include the parent nodes have already been evaluated, the depth-first scan order continues down to the nodes (or node groups) that include the sibling nodes of the nodes being evaluated at the same level of the octree structure.

[0060] Figure 5 The nodes of an octree structure grouped into multiple node groups are shown according to some implementation schemes.

[0061] As described above, in some embodiments, the nodes of the octree structure can be organized into node groups, and in some embodiments, the node groups can be organized together to form slices, wherein the slices correspond in size to the payload of data transmission units (such as IP packets).

[0062] In some implementations, a node group may define a spatial volume, which includes the volume of the cube or sub-cube corresponding to the nodes included in the node group. It should be noted that a node group can represent a given volume in space, regardless of whether the nodes in the node group are occupied or unoccupied.

[0063] For example, Figure 5An octree structure 500 is illustrated, wherein the nodes of the octree structure are grouped into two node groups 502 at level 2, four node groups 504 at level 3, and eight node groups 506 at level 4. It should be noted that in some embodiments, node groups at a given level of the octree structure may correspond to the same volume size in space, but node groups at different levels of the octree structure may correspond to different volume sizes in space. For example, since there are more nodes in level 4 than in level 2, node group 506 may correspond to a smaller volume than node group 502.

[0064] In some implementations, the entropy context used to encode / decode node occupancy information can be determined using neighboring nodes within a node group. However, the search for neighboring nodes used to compute the entropy context can be limited to those nodes included in the same node group as the given node whose occupancy information is being encoded / decoded. As discussed above, this simplifies dependencies between nodes and limits the amount of node occupancy information that needs to be stored in a memory buffer to determine the entropy encoding / decoding context.

[0065] Figure 6 The diagram illustrates a depth-first scan order for selecting node groups to be evaluated, according to some implementations, wherein nodes within the corresponding node groups are scanned according to a breadth-first scan order.

[0066] In some implementations, nodes within a node group can be scanned according to a breadth-first scan order, but the next subsequent node group to be evaluated can be selected based on either a breadth-first scan order or a depth-first scan order.

[0067] For example, Figure 6The arrows at the top of the node groups illustrate scanning nodes within a node group according to a breadth-first scan order. However, the dashed lines indicate selecting the next sequential node group to be evaluated according to a depth-first scan order. For example, after evaluating node group 2, the depth-first scan order selects node group 3, which is at a lower level of the octree structure to be evaluated next (instead of node group 9, which is at the same octree level as node group 2). In this way, fewer node groups may need to be evaluated before reaching the leaf nodes. For example, the leaf node of node group 4 may be reached after evaluating four node groups, whereas if node groups were evaluated in a breadth-first scan order, node groups 0, 1, 2, 9, 3, 6, A, and B (e.g., eight node groups) would have to be evaluated before reaching the leaf node of node group 4. Therefore, there would be almost twice as many higher-level nodes above the leaf nodes, any of which, if lost during transmission, would adversely affect the determination of the location of the leaf node used for reconstruction. It can be seen that this arrangement offers increased flexibility compared to a purely breadth-first scan order. However, at least some of the entropy coding efficiency of using neighboring nodes to determine the entropy context is not lost due to the fact that a breadth-first scan order is still enabled within the node group.

[0068] In some implementations, a hybrid scan order of breadth-first within a node group and depth-first for selecting the next node group to be evaluated can be used at both the encoder and decoder, for example, without having to signal the scan order.

[0069] Figure 7 An octree structure with nodes grouped into multiple node groups according to some implementations is shown, where a flag indicates the evaluation order of subsequent node groups for selecting the node group to be evaluated.

[0070] In some implementations, a dynamic scan order can be used to select the next node group to be evaluated. For example, a flag can be placed at the end of a node group to indicate the scan order used to select the next node group to be evaluated, instead of always selecting the next node group based on a depth-first scan order. It should be noted that, although... Figure 7 Flags for each node group are shown, but in some implementations, flags can be asserted if a first scan order (e.g., depth-first scan order) will be used, and unasserted flags indicate a deviation from the default scan order (e.g., breadth-first scan order). For example, it can be inferred that a flag, when unasserted, indicates one scan order, and when asserted, it indicates another scan order.

[0071] Figure 8The diagram illustrates node groups evaluated according to a hybrid depth-first / breadth-first scan order according to some embodiments, wherein a marker at the end of a node group indicates whether the next node group to be evaluated is selected according to a depth-first scan order or a breadth-first scan order, and wherein the nodes within the corresponding node group are scanned according to a breadth-first scan order.

[0072] As in Figure 8 As can be seen, at level 2, a flag can indicate that a breadth-first scan order should be followed after node group 2, rather than continuing to node group 4 at the next lower level below node group 2, such that the next node group to be evaluated (e.g., node group 3) is the next sequential node group at the same level as node group 2 in the octree. However, at level 3, after evaluating node group 3, a flag (or the absence of a flag) can indicate that the next node group to be evaluated should be selected according to a depth-first scan order, for example, node group 4 at the next lower level of the octree structure.

[0073] In some implementations, the node group selection order can be chosen to balance the trade-off between flexibility and memory buffer / dependency requirements. Additionally, in some implementations, different node groups may include different numbers of occupied nodes that require different amounts of bits for encoding. Therefore, in some implementations, the encoder may alternate between selecting the next node group to evaluate based on a depth-first scan order or a breadth-first scan order to select sequential node groups to include in a slice that fill the slice but do not overfill it.

[0074] Figure 9 The diagram illustrates a group of nodes organized into multiple slices according to some embodiments, wherein each group of nodes contains a flag indicating whether the group of nodes is the last group of nodes for a given slice.

[0075] In some implementations, the node groups described above can be grouped into slices. Furthermore, each node group may include a flag indicating whether it is the last node group in a given slice (this flag may be referred to as a slice end flag). For example, Figure 9 A slice is shown consisting of a set of node groups, where different slices are represented by different shading / fill patterns. Marker 902 (shown as a hollow circle) indicates that the node group is not the last node group in the slice, while mark 904 (shown as a solid circle) indicates the last node group in the slice. It should be noted that in some embodiments, a slice may include node groups at more than one octree level of the octree structure 200.

[0076] In some implementations, the value of the next node group scan order flag may or may not be derived from the encoder and rules present in the encoder. The encoder or decoder checks the rule at the end of each node group and determines whether the rule enforces a specific scan order. If the rule does not apply, a flag indicating the specific order is signaled. In one implementation, a rule states that if a slice end flag is asserted at a juxtaposition in the level or previous level, the tree traversal continues to the next node group in depth-first order. Another rule may state that if the node group scan order flag is asserted as depth-first at a juxtaposition in the level or previous level, or is derived as depth-first by another rule, the tree traversal continues to the next node group in depth-first order. Such rules allow, for example, the maintenance of a hierarchical structure of slices that avoid overlap to increase resilience to lost or damaged slices without the overhead of signaling an explicit scan order. The presence of such rules can be conveyed by signaling additional flags in a parameter set that transmits parameters shared by one or more slices.

[0077] In some implementations, in order to minimize any overhead from signaling the last node group flag in a slice, there may be a rule indicating that the last node group flag in a slice does not exist unless the associated scan order flag (or rule) changes the tree traversal order.

[0078] In some embodiments, a header may be associated with a slice, wherein the header transmits parameters to be applied to the group of nodes included in the slice. For example, the header may define the size of the group of nodes included in the slice. Additionally, the header may include an indication of where a reference node of the slice (e.g., the first node of the slice) should be located in the octree structure 200. In this way, if an intermediate slice is lost, subsequent slices can be placed in the octree structure even without the intermediate slice. In some embodiments, the reference location of the slice may be transmitted in the header using a reference to a specific tree level of the octree structure and a spatial location within that tree level, or using a reference to a specific tree level of the octree structure and a node index location within that tree level. In some embodiments, a spatial location or index may be signaled relative to the index of a node in a group of nodes or relative to the location of the group of nodes within a specific tree level of the octree structure. In some embodiments, a spatial location or index may be signaled relative to the leaf nodes of a particular slice.

[0079] Additionally, in some implementations, the node group size can be set at the octree level, where node groups at a given octree level have the same size. In some implementations, the node group size can be set at the beginning of each octree level within a given slice.

[0080] In some implementations, the header of each slice may include an index value that specifies the order of the slice relative to other slices in the octree structure 200. For example, the first slice may be slice 1, the second slice may be slice 2, and so on.

[0081] In some implementations, the first slice may be referred to as the initial slice and may include the complete set of header parameters. However, subsequent slices may have header information that depends on the preceding slice. Therefore, the headers of subsequent slices may have light weights and may indicate that the slice is a dependent slice and also indicate what slice headers the slice depends on.

[0082] In some implementations, the entropy context can be reset for a slice; in this case, the reset can be signaled in the slice header. Additionally, such slices can be called entropy-independent slices because they do not depend on the entropy state provided by the previous slice. Conversely, some slices may use the initial entropy context inherited from the previous slice. Such slices can be called entropy-dependent slices.

[0083] Figure 10 The diagram illustrates a group of nodes organized into multiple slices according to some embodiments, wherein each group of nodes contains a flag indicating whether it is the last group of nodes for a given slice, and wherein the slices are arranged sequentially in the order in which they will be received, for example, by the decoder.

[0084] Figure 11A and Figure 11B The diagram shows groups of nodes organized into multiple slices, where each group of nodes contains a flag indicating whether it is the last group of nodes in a given slice. Figure 11A and Figure 11B The entropy context savepoints indicated by one or more nodes for a slice, according to some implementations, are also shown.

[0085] For example, if slice 1102 is missing and slice 1104 is entropy independent, the portion of slice 1104 that is not below slice 1102 can still be decoded. For example, the portion of slice 1104 that is a child of slice 1106 can still be decoded. However, the nodes of slice 1104 that are children of slice 1102 may not be decoded. Additionally, the nodes in the lower right of slice 1104 (which follow the nodes that are children of slice 1102) may not be decoded because they have an entropy dependency on the undecoded nodes below 1102.

[0086] In some implementations, an entropy-dependent slice may have an entropy context of the slice’s starting node, which is recovered from a saved entropy context from the previous slice.

[0087] In some implementations, the entropy context / state saved to a memory buffer for use in subsequent slices can be implicit. For example, the entropy state of the last node in a slice or the entropy state of the last node at a given octree level can be saved. Additionally, in some implementations, the encoder can mark nodes whose entropy states will be saved for use in subsequent slices. For example, flag 1108. It can be used to send signals: the entropy context of a specific node in slice 1106 will be saved to be used as the inherited entropy of slice 1102 or slice 1104.

[0088] In some implementations, for a slice spanning more than one octree level, the entropy state of the last node in the penultimate octree level can overwrite the entropy state originally stored for the slice. Additionally, in some implementations, a flag can be set to indicate that the entropy state of the last node in the octree level will overwrite the entropy context of a given slice; if the flag is not set, the entropy context of the last node in the octree level is not used to overwrite the entropy context stored for the slice. In some implementations, this process can be omitted if the end of the tree level coincides with the end of the slice.

[0089] In some implementations, an entropy-dependent slice can recover its entropy state from the entropy state stored in the preceding slice. However, in other implementations, an entropy-dependent slice can recover its entropy state from a slice other than the immediately preceding slice. In some implementations, this can be signaled as an index value of the slice from which the entropy context of the slice from which the entropy-dependent slice is recovered, or it can be signaled relatively (e.g., if there are two preceding slices). For example, slice 1104 can inherit the entropy state stored for node 1108 of slice 1106. Therefore, if slice 1102 is lost, slice 1104 can still recover its entropy state. In another implementation, the slice from which the entropy state is recovered is inferred to be the slice containing the parent node of the first node in that slice.

[0090] Figure 12A and Figure 12B The diagram shows groups of nodes organized into multiple slices, where each group of nodes contains a flag indicating whether it is the last group of nodes in a given slice. Figure 12A and Figure 12B Entropy context storage points for one or more nodes indicated for slices according to some embodiments are also shown, wherein at least some slices are entropy-dependent slices that depend on a common previous slice.

[0091] As another example of entropy-dependent slices, if slices 1202, 1204, 1206, and 1208 are all entropy-dependent slices that inherit their entropy states from slice 1210, then if slice 1202 is lost, slices 1204 and 1208 can still be decoded because the initial entropy states of these slices are known and because they do not contain tree dependencies on the nodes of slice 1202.

[0092] Furthermore, various combinations of parallel decoding are possible because slices 1206 and 1208 do not depend on the immediately preceding slice. For example, once the entropy context of slice 1210 is added to the memory buffer, decoding of subsequent dependent slices (such as 1206 and 1208) can begin. Additionally, if the entropy context of a slice is not the last node in that slice, such as for... Figure 11A As shown by flag 1108, once the flagged node is reached and the entropy context of the flagged node is added to the memory buffer, the saved entropy context / entropy reset can be used to begin decoding subsequent slices in parallel with decoding the rest of the reference slice.

[0093] In some implementations, attribute information can be sent using signals independently of the geometric tree data.

[0094] Figure 13 An exemplary octree structure is shown, in which subtrees terminate early according to some implementation schemes.

[0095] In some implementations, each geometric slice can generate zero or more output points. For example, Figure 13 The leaf nodes shown can generate output points. Additionally, early-terminated subtrees (e.g., IDCM) can also generate one or more output points.

[0096] In some implementations, a single attribute slice is generated for each geometric slice with output points. In other implementations, a level-of-detail or region-adaptive grading transformation is generated only for the output points generated from the geometric slices with output points, or the level-of-detail or region-adaptive grading transformation is applied only to these output nodes.

[0097] In some implementations, attribute slices are generated for geometric slices that do not include output points, or for points on a geometric slice other than the output points. These can be considered virtual points. For example, if a geometric slice will be lost and a higher-level geometric slice will be used to generate the output point, attribute slices can be generated for both geometric slices, where if the lower-level geometric slice is not lost, the attribute slice of the higher-level geometric slice is simply ignored at the decoder. However, if the lower-level geometric slice is lost, the attribute slice of the higher-level geometric slice (e.g., virtual output points) can be used to determine the attributes of the lower-resolution output generated when the lower-level geometric slice is lost. Similar techniques can be used for partial decoding.

[0098] Figure 15A The components of an encoder according to some implementation schemes are shown.

[0099] Encoder 1502 can be with Figure 1 The encoder 104 shown is similar to an encoder. Encoder 1502 includes a spatial encoder 1504, an octree generator 1510, a prediction / correction evaluator 1506, an input data interface 1514, and an output data interface 1508. Encoder 1502 also includes a context memory 1516 (e.g., a memory buffer) and a configuration memory 1518.

[0100] In some implementations, a spatial encoder (such as spatial encoder 1504) can compress spatial information associated with points in a point cloud, enabling the spatial information to be stored or transmitted in a compressed format. In some implementations, the spatial encoder (such as spatial encoder 1504) can utilize an octree to compress spatial information about points in a point cloud, as discussed in more detail above.

[0101] In some implementations, compressed spatial information may be stored or transmitted together with compressed attribute information, or it may be stored or transmitted separately. In either case, the decoder that receives compressed attribute information for points in a point cloud may also receive compressed spatial information for those points in the point cloud, or may obtain spatial information for those points in the point cloud.

[0102] Octree generators (such as Octree Generator 1510) can use spatial information about points in a point cloud to generate octrees that subdivide the point cloud into cubes and subcubes.

[0103] A prediction / correction estimator (such as prediction / correction estimator 1506 of encoder 1502) can determine the predicted attribute value for a point in the point cloud using the attribute values ​​of the K nearest neighbors of the point to which the predicted attribute value is directed, based on an inverse distance interpolation method. The prediction / correction estimator can also compare the predicted attribute value of the point being evaluated with the original attribute value of that point in the uncompressed point cloud to determine an attribute correction value. In some embodiments, the prediction / correction estimator (such as prediction / correction estimator 1506 of encoder 1502) can adaptively adjust the prediction strategy for predicting the attribute values ​​of points in a given neighborhood of a point based on a measurement of the variability of the attribute values ​​of points in the neighborhood.

[0104] Outgoing data encoders (such as outgoing data encoder 1508, such as encoder 1502) can encode attribute correction values ​​and assigned attribute values ​​included in compressed attribute information files for point clouds.

[0105] In some embodiments, the encoder (such as encoder 1502) may also include an incoming data interface such as incoming data interface 1514. In some embodiments, the encoder may receive incoming data from one or more sensors that capture points in a point cloud or capture attribute information associated with points in the point cloud. For example, in some embodiments, the encoder may receive data from a LiDAR system, a 3D camera, a 3D scanner, etc., and may also receive data from other sensors such as gyroscopes, accelerometers, etc. Additionally, the encoder may receive other data, such as the current time, from a system clock, etc. In some embodiments, these different types of data may be received by the encoder via an incoming data interface (such as incoming data interface 1514 of encoder 1502).

[0106] In some embodiments, the encoder (such as encoder 1502) may also include a configuration interface (such as configuration interface 1512) through which one or more parameters used by the encoder to compress the point cloud can be adjusted. In some embodiments, the configuration interface (such as configuration interface 1512) may be a programmatic interface, such as an API. The configuration used by the encoder (such as encoder 1502) may be stored in a configuration memory (such as configuration memory 1518).

[0107] In some implementations, the encoder (such as encoder 1502) may include a ratio Figure 15A Show more or fewer parts.

[0108] Figure 15B The components of a decoder according to some implementation schemes are shown.

[0109] Decoder 1520 can be with Figure 1The decoder 116 shown is similar to the decoder in the diagram. Decoder 1520 includes an encoded data interface 1526, a spatial decoder 1522, a prediction evaluator 1524, a context memory 1530, a configuration memory 1532, and a decoded data interface 1528.

[0110] A decoder (such as decoder 1520) may receive an encoded compressed point cloud and / or an encoded compressed attribute information file for points in the point cloud, such as an encoded bitstream as discussed above. For example, the decoder (such as decoder 1520) may receive a compressed attribute information file and / or a compressed spatial information file. The decoder may receive the compressed attribute information file and / or the compressed spatial information file via an encoded data interface (such as encoded data interface 1526). The decoder may use the encoded compressed point cloud to determine spatial information for points in the point cloud. For example, spatial information for points in the point cloud included in the compressed point cloud bitstream may be generated by a spatial decoder (such as spatial decoder 1522). In some embodiments, a compressed point cloud, which has previously been encoded by an encoder (such as encoder 1502), may be received from a storage device or other intermediate source via an encoded data interface (such as encoded data interface 1526). In some embodiments, the encoded data interface (such as encoded data interface 1526) may decode the spatial information. For example, various encoding techniques as described herein may be used to encode the spatial information.

[0111] The attribute prediction estimator of the decoder (such as prediction estimator 1524) may select the starting point of the minimum spanning tree based on the assigned starting point included in the compressed attribute information file. In some embodiments, the compressed attribute information file may include one or more assigned values ​​for one or more corresponding attributes of the starting point. In some embodiments, the attribute prediction estimator (such as prediction estimator 1524) may assign values ​​to one or more attributes of the starting point in the decompression model of the point cloud being decompressed based on the assigned values ​​for the starting point included in the compressed attribute information file. The prediction estimator (such as prediction estimator 1524) may also use the assigned values ​​of the attributes of the starting point to determine the attribute values ​​of neighboring points. For example, the prediction estimator may select the nearest neighbor to the starting point as the next point to be evaluated, where the next nearest neighbor is selected based on the shortest distance from the starting point to the neighbor in the minimum spanning tree. It should be noted that because the minimum spanning tree is generated on the decoder based on the same or similar spatial information used to generate the minimum spanning tree on the encoder, the decoder can determine the same evaluation order as evaluating the points for the point cloud being decompressed determined at the encoder by identifying the next nearest neighbor in the minimum spanning tree.

[0112] A decoder (such as decoder 1520) can provide a decompressed point cloud, generated based on a received compressed point cloud and / or a received compressed attribute information file, to a receiving device or application via a decoding data interface (such as decoding data interface 1528). The decompressed point cloud may include points of the point cloud and attribute values ​​for attributes of those points. In some embodiments, the decoder may decode some attribute values ​​for the attributes of the point cloud without decoding other attribute values ​​for other attributes of the point cloud. For example, a point cloud may include color attributes for the points of the point cloud, and may also include other attributes for the points of the point cloud, such as velocity. In such cases, the decoder may decode one or more attributes (such as velocity attributes) of the points of the point cloud without decoding other attributes (such as color attributes) of the points of the point cloud.

[0113] In some implementations, the decompressed point cloud and / or decompressed attribute information file can be used to generate a visual display, such as for a head-mounted display. Additionally, in some implementations, the decompressed point cloud and / or decompressed attribute information file can be provided to a decision engine that uses the decompressed point cloud and / or decompressed attribute information file to make one or more control decisions. In some implementations, the decompressed point cloud and / or decompressed attribute information file can be used for a variety of other applications or for a variety of other purposes.

[0114] Example Applications for Point Cloud Compression and Decompression

[0115] Figure 16 This illustrates a compressed point cloud used in 3D teledisplay applications according to some implementation schemes.

[0116] In some implementations, sensors (such as sensor 102), encoders (such as encoder 104 or encoder 1502), and decoders (such as decoder 116 or decoder 1520) can be used to transmit point clouds in 3D applications. For example, at 1602, a sensor (such as sensor 102) may capture a 3D image, and at 1604, the sensor or a processor associated with the sensor may perform 3D reconstruction based on the sensed data to generate a point cloud.

[0117] At 1606, an encoder (such as encoder 104 or 1502) can compress the point cloud, and at 1608, the encoder or post-processor can package the compressed point cloud and transmit it via network 1610. At 1612, data packets can be received at a target location including a decoder (such as decoder 116 or decoder 1520). At 1614, the decoder can decompress the point cloud, and at 1616, the decompressed point cloud can be rendered. In some implementations, 3D teledisplay applications can transmit point cloud data in real time, such that the display at 1616 can represent the image being observed at 1602. For example, at 1616, a camera in a canyon can allow a remote user to experience traversing a virtual canyon.

[0118] Figure 17 The diagram illustrates compressed point clouds used in virtual reality (VR) or augmented reality (AR) applications according to some implementation schemes.

[0119] In some implementations, point clouds can be generated in software (e.g., as opposed to being captured by sensors). For example, at 1702, virtual reality or augmented reality content is generated. The virtual reality or augmented reality content may include point cloud data and non-point cloud data. For example, as an example, non-point cloud characters may traverse terrain represented by point clouds. At 1704, the point cloud data may be compressed, and at 1706, the compressed point cloud data and non-point cloud data may be packaged and transmitted via network 1708. For example, the virtual reality or augmented reality content generated at 1702 may be generated at a remote server and transmitted to a VR or AR content consumer via network 1708. At 1710, the data packet may be received and synchronized at the VR or AR consumer's device. At 1712, a decoder operating at the VR or AR consumer's device may decompress the compressed point cloud, and the point cloud and non-point cloud data may be rendered in real time, for example, on the VR or AR consumer's device's head-mounted display. In some implementations, point cloud data can be generated, compressed, decompressed, and rendered in response to VR or AR consumers manipulating head-mounted displays to look in different directions.

[0120] In some implementations, point cloud compression, as described herein, can be used in a variety of other applications such as geographic information systems, live sports broadcasts, museum displays, and autonomous navigation.

[0121] Some embodiments of this disclosure may also be described in accordance with the following provisions:

[0122] 1. A non-transitory computer-readable medium storing program instructions, said program instructions, when executed by one or more processors, causing said one or more processors to:

[0123] Receive bit stream, the bit stream comprising:

[0124] A slice of an octree structure of a point cloud, wherein the octree structure of the point cloud includes a set of nodes defining the spatial locations of points in the point cloud, and wherein the slice includes a subset of the set of nodes of the octree structure and a marker indicating the end of the slice.

[0125] Two or more additional slices of the octree structure of the point cloud, wherein each of the two or more additional slices includes an additional corresponding subset of the node set of the octree structure and an additional corresponding flag indicating the corresponding end of the two or more additional slices; and

[0126] Wherein at least two or more of the additional slices include:

[0127] An indication of the position of at least one node in the corresponding slice within the octree structure, wherein the indicated position is indicated using the following:

[0128] References to specific tree levels of the octree structure and spatial locations within those tree levels; or

[0129] References to specific tree levels of the octree structure and to the index positions of nodes within those tree levels; and

[0130] The geometry of the point cloud is reconstructed based on the bitstream, wherein, in order to reconstruct the geometry, the corresponding subsets of nodes of the two or more additional slices are located in the octree structure based on their corresponding indicated positions.

[0131] 2. The non-transitory computer-readable medium according to Clause 1, wherein the two or more additional slices further comprise:

[0132] An index value indicating the order of the corresponding slice relative to other slices in the octree structure.

[0133] 3. The non-transitory computer-readable medium according to Clause 2, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

[0134] Determine if a given slice is missing in the said order; and

[0135] Based on the position indication included in the bitstream for the next non-missing slice among the two or more slices, the next non-missing slice among the slices is positioned in the octree structure.

[0136] The program instructions, when executed by the one or more processors, cause the one or more processors to:

[0137] The geometry of the point cloud is reconstructed at different resolutions for a portion of the point cloud corresponding to the given missing slice.

[0138] 4. The non-transitory computer-readable medium according to Clause 1, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to:

[0139] Receive instructions to reconstruct a scaled-resolution version of the point cloud; and

[0140] The point cloud is reconstructed using slices from a complete set of slices of the octree structure that is less than the point cloud itself.

[0141] The point cloud geometry corresponding to nodes in the unused slices of the octree structure is reconstructed at a resolution different from the resolution in the case where the nodes in the unused slices are used during the reconstruction of the point cloud.

[0142] 5. The non-transitory computer-readable medium according to Clause 1, wherein the slice and the two or more additional slices each correspond in size to the size of a data transmission unit for transmitting a portion of the bitstream to a decoder over a network.

[0143] 6. The non-transitory computer-readable medium as described in Clause 5, wherein:

[0144] The data transmission unit used to transmit the bit stream over the network is an Internet Protocol (IP) packet;

[0145] The slice and the two or more additional slices are determined at the application layer, separate from the IP layer, such that each slice and each of the two or more additional slices has a size suitable for the payload of the IP packet; and

[0146] In response to a lost IP packet comprising a slice of the octree structure of the point cloud, the program instructions, when executed on the one or more processors, cause the one or more processors to continue reconstructing the point cloud without retransmitting the lost IP packet.

[0147] 7. The non-transitory computer-readable medium according to Clause 1, wherein occupancy information of the nodes of the octree structure included in the slice and the two or more additional slices has been encoded using an entropy encoder, and

[0148] The program instructions, when executed by the one or more processors, cause the one or more processors to:

[0149] Based on the occupancy information of the neighboring nodes of the given node whose entropy-encoded occupancy information is being decoded, an encoding context is determined for decoding the entropy-encoded occupancy information of the corresponding node in the given node.

[0150] The at least two or more additional slices include:

[0151] A header indicating the entropy context to be used for entropy decoding of the entropy-encoded occupancy information of a given node in a given slice of the two or more additional slices:

[0152] It will be reset to the default entropy context; or

[0153] It will be set to the entropy context saved for the previous slice.

[0154] The program instructions therein cause the one or more processors to:

[0155] The entropy context determined for the slice or a given slice among the two or more additional slices is saved to a memory buffer to be used as the saved entropy context for subsequent slices among the two or more additional slices.

[0156] 8. The non-transitory computer-readable medium as described in Clause 7, wherein at least two of the two or more additional slices reference the same entropy context saved to the memory buffer for the preceding slice.

[0157] 9. The non-transitory computer-readable medium according to Clause 8, wherein the program instructions, when executed by the one or more processors, cause:

[0158] Entropy decoding is performed on the entropy-encoded occupancy information of at least two slices that reference the same entropy context of the preceding slice, at least in part in parallel.

[0159] 10. The non-transitory computer-readable medium according to Clause 8, wherein the program instructions, when executed by the one or more processors, cause:

[0160] Entropy decoding is performed on the entropy-encoded occupancy information of a given slice that references the same entropy context of the preceding slice among the two or more slices, regardless of whether the other slice among the two or more slices that references the same entropy context is a missing slice.

[0161] 11. The non-transitory computer-readable medium according to Clause 7, wherein the program instructions, when executed by the one or more processors, cause:

[0162] For each slice whose entropy-encoded occupancy information is decoded, at least one node saves the entropy context to the memory buffer.

[0163] 12. The non-transitory computer-readable medium as described in Clause 11, wherein the entropy context is stored at the last node of a given slice for which the encoded occupancy information has been decoded.

[0164] 13. The non-transitory computer-readable medium according to Clause 11, wherein the entropy context is stored for a given slice of the two or more additional slices, wherein the given slice spans more than one tree level of the octree structure, and wherein, for the given slice, the entropy encoding context of the node in the penultimate tree level is stored in the memory buffer.

[0165] 14. The non-transitory computer-readable medium according to Clause 7, wherein the program instructions, when executed by the one or more processors, cause:

[0166] For the last node of each tree level of the octree structure whose entropy-encoded occupancy information has been decoded, the entropy context is saved to the memory buffer.

[0167] 15. The non-transitory computer-readable medium according to Clause 14, wherein the program instructions, when executed by the one or more processors, cause:

[0168] The entropy context saved to the memory buffer for the last node of a given tree level overrides the previous entropy context saved to the memory buffer for a given slice spanning more than one tree level.

[0169] 16. The non-transitory computer-readable medium as described in Clause 7, wherein the header identifier of the given slice in which the entropy-encoded occupancy information is being decoded will recover the previous slice of the entropy context therefrom.

[0170] 17. The non-transitory computer-readable medium as described in Clause 7, wherein the previous slice from which the entropy context is to be recovered is implicitly determined based on the encoding rules applied at both the encoder and the decoder.

[0171] 18. The non-transitory computer-readable medium according to Clause 14, wherein the last node of a given tree level is marked in the bit stream with a flag, wherein the flag indicates that the entropy context of the marked node will be saved to the memory buffer.

[0172] 19. The non-transitory computer-readable medium according to Clause 1, wherein each of the two or more additional slices comprises two or more node groups, wherein the node groups represent the spatial volume of the octree structure, and wherein the header of each of the respective two or more additional slices indicates the node group size of the node groups included in the respective slice.

[0173] 20. The non-transitory computer-readable medium according to Clause 1, wherein each of the two or more additional slices comprises two or more node groups, wherein the node group represents the spatial volume of the octree structure, and wherein the size of the node group is signaled in the bitstream for each tree level of the octree structure.

[0174] 21. A non-transitory computer-readable medium storing program instructions, which, when executed by one or more processors, cause the one or more processors to:

[0175] Receive a point cloud to be compressed, wherein the point cloud comprises multiple points in three-dimensional space;

[0176] Generate the octree structure including occupancy information of nodes in the octree structure, wherein the occupancy information indicates whether, at a given level of the octree structure, a cube or sub-cube of the octree at that given level is occupied by one or more points of the point cloud; and

[0177] The nodes of the octree structure are organized into multiple slices, and the multiple slices

[0178] The film includes:

[0179] The first slice of the octree structure, the first slice comprising a subset of the node set of the octree structure; and

[0180] The octree structure has two or more additional slices, wherein each of the two or more additional slices includes an additional corresponding subset of the set of nodes of the octree structure;

[0181] Encode the occupancy information of the nodes of the octree included in the first slice and the two or more additional slices; and

[0182] Generate a bit stream, the bit stream comprising:

[0183] The first slice includes the encoded occupancy information of the nodes and a flag indicating the end of the first slice;

[0184] The encoded occupancy information of the nodes included in the two or more additional slices and the additional corresponding flags indicating the corresponding end of the two or more additional slices; and

[0185] For at least two or more of the additional slices:

[0186] An indication of the position of at least one node in the corresponding slice within the octree structure, wherein the indicated position is indicated using the following:

[0187] References to specific tree levels of the octree structure and spatial locations within those tree levels; or

[0188] References to specific tree levels of the octree structure and the index positions of nodes within those tree levels.

[0189] 22. The non-transitory computer-readable medium according to Clause 21, wherein the generated bit stream further comprises:

[0190] A corresponding index value indicating the corresponding order of the plurality of slices relative to the other slices among the plurality of slices.

[0191] 23. The non-transitory computer-readable medium as described in Clause 21, wherein the generated bit stream further comprises:

[0192] A corresponding header for the plurality of slices, wherein the header indicates the volume size of the group of nodes included in the corresponding slice.

[0193] 24. The non-transitory computer-readable medium according to Clause 21, wherein the slice and the two or more additional slices each correspond in size to the size of a data transmission unit for transmitting the bitstream to the decoder over a network.

[0194] 25. The non-transitory computer-readable medium according to Clause 21, wherein the occupancy information of the node is encoded using an entropy encoder, and wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to:

[0195] Based on the occupancy information of the neighboring nodes of the given node whose entropy-encoded occupancy information is being encoded, an encoding context is determined for encoding the occupancy information of the corresponding nodes among the nodes of the plurality of slices; and

[0196] The bitstream for each of the plurality of slices includes an indication of an initial entropy context, which will be used to decode the entropy-encoded occupancy information of a given node of the corresponding slice.

[0197] 26. The non-transitory computer-readable medium according to Clause 25, wherein the indication for the initial entropy context used to decode the entropy-encoded occupancy information of the given node of the corresponding slice indicates:

[0198] The entropy context saved to the memory buffer for the previous slice will be used as the initial entropy context for the given slice; or

[0199] The reset entropy context will be used as the initial entropy context for the given slice.

[0200] 27. A non-transitory computer-readable medium as described in Clause 26, wherein the corresponding indications for two or more slices indicate:

[0201] The same entropy context saved for the previous slice will be used as the initial entropy context for the two or more slices.

[0202] 28. The non-transitory computer-readable medium according to Clause 26, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to:

[0203] The bitstream of the corresponding node among the nodes of the plurality of slices includes a corresponding flag, wherein the corresponding flag indicates that the entropy context for decoding the entropy-encoded occupancy information of the corresponding node among the nodes will be added to the memory buffer storing the saved entropy context for use as the initial entropy context for subsequent slices.

[0204] 29. The non-transitory computer-readable medium according to Clause 26, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to:

[0205] The last node in the tree hierarchy of the octree structure is indicated by a flag in the bitstream, wherein the flag indicates that the entropy context for decoding the last node in the tree hierarchy will be added to the memory buffer.

[0206] 30. An apparatus, the apparatus comprising:

[0207] Memory, the memory storing program instructions as described in Clause 1; and

[0208] One or more processors.

[0209] 31. The device according to clause 30, further comprising:

[0210] monitor,

[0211] Wherein the program instructions described in Clause 1, when executed by the one or more processors, cause the one or more processors to:

[0212] The point cloud having the reconstructed geometry is rendered on the display of the device.

[0213] 32. An apparatus, the apparatus comprising:

[0214] Memory, the memory storing program instructions as described in clause 21; and

[0215] One or more processors.

[0216] 33. The device according to clause 32 further includes:

[0217] One or more sensors are configured to capture a plurality of points constituting the point cloud, wherein each point includes spatial information and attribute information about the point.

[0218] The point cloud to be compressed is received from one or more sensors.

[0219] Example Computer System

[0220] Figure 18 The following are examples of implementations available for use in implementing the methods described herein (e.g., as referenced above). Figures 1-17 An exemplary computer system 1800 is described above as an encoder or decoder. In various embodiments, the computer system 1800 may be any of a variety of types of devices, including but not limited to: personal computer systems, desktop computers, laptops, notebook computers, tablet computers, all-in-one computers, tablet computers, netbook computers, handheld computers, workstations, network computers, cameras, set-top boxes, mobile devices, consumer devices, video game controllers, handheld video game devices, application servers, storage devices, televisions, video recording devices, peripheral devices (such as switches, modems, routers), or any type of computing or electronic device in general.

[0221] In the illustrated embodiment, computer system 1800 includes one or more processors 1810 coupled to system memory 1820 via input / output (I / O) interface 1830. Computer system 1800 also includes a network interface 1840 coupled to I / O interface 1830 and one or more input / output devices 1850, such as cursor control device 1860, keyboard 1870, and display 1880.

[0222] System memory 1820 may be configured to store point cloud compression or decompression program instructions 1822 and / or sensor data accessible by processor 1810. In various embodiments, system memory 1820 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory. In an illustrated embodiment, program instructions 1822 may be configured to implement an image sensor control application combining any of the above-described functionalities. In some embodiments, program instructions and / or data may be received, transmitted, or stored on a different type of computer-accessible medium or similar medium separate from system memory 1820 or computer system 1800.

[0223] In one embodiment, I / O interface 1830 may be configured to coordinate I / O communication between processor 1810, system memory 1820, and any peripheral devices (including network interface 1840 or other peripheral device interfaces, such as input / output devices 1850) within the device. In some embodiments, I / O interface 1830 may perform any necessary protocol, timing, or other data conversions to convert data signals from one component (e.g., system memory 1820) into a format suitable for use by another component (e.g., processor 1810). In some embodiments, I / O interface 1830 may include support for devices attached, for example, via various types of peripheral buses (e.g., variants of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard). In some embodiments, the functionality of I / O interface 1830 may be divided among two or more separate components, such as a north bridge and a south bridge. Furthermore, in some embodiments, some or all of the functionality of I / O interface 1830 (such as an interface to system memory 1820) may be directly incorporated into processor 1810.

[0224] Network interface 1840 can be configured to allow data exchange between computer system 1800 and other devices (e.g., bearers or agent devices) attached to network 1885, or between nodes of computer system 1800. In various embodiments, network 1885 may include one or more networks, including but not limited to local area networks (LANs) (e.g., Ethernet or enterprise networks), wide area networks (WANs) (e.g., the Internet), wireless data networks, some other electronic data network, or some combination thereof. In various embodiments, network interface 1840 may support communication, for example, via wired or wireless general-purpose data networks (such as any suitable type of Ethernet network); via telecommunications / telephone networks (such as analog voice networks or digital fiber optic communication networks); via storage area networks (such as Fibre Channel SANs); or via any other suitable type of network and / or protocol.

[0225] In some implementations, input / output device 1850 may include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other device suitable for inputting or accessing data by one or more computer systems 1800. Multiple input / output devices 1850 may be present in computer system 1800 or distributed across various nodes of computer system 1800. In some implementations, similar input / output devices may be separate from computer system 1800 and may interact with one or more nodes of computer system 1800 via wired or wireless connections (such as via network interface 1840).

[0226] like Figure 18 As shown, memory 1820 may include program instructions 1822, which may be executable by a processor to perform any of the elements or actions described above. In one embodiment, the program instructions may perform the methods described above. In other embodiments, different elements and data may be included.

[0227] The computer system 1800 may also be connected to other devices not shown, or may operate as a standalone system. Furthermore, the functionality provided by the illustrated components may, in some embodiments, be combined into fewer components or distributed among additional components. Similarly, in some embodiments, the functionality of some of the illustrated components may not be provided, and / or other additional functions may be available.

[0228] Those skilled in the art will also recognize that while various items are shown as being stored in memory or on storage devices during use, these items, or portions thereof, may be transferred between memory and other storage devices for memory management and data integrity purposes. Alternatively, in other embodiments, some or all of these software components may be executed in memory on another device and communicate with the illustrated computer system via inter-computer communication. Some or all of system components or data structures may also be stored (e.g., as instructions or structured data) on a computer-accessible medium or portable article of manufacture for reading by a suitable drive, various examples of which are described above. In some embodiments, instructions stored on a computer-accessible medium separate from computer system 1800 may be transmitted to computer system 1800 via a transmission medium or signal (such as electrical, electromagnetic, or digital signals transmitted via communication media such as networks and / or wireless links). Various embodiments may also include receiving, transmitting, or storing instructions and / or data implemented according to the above description on a computer-accessible medium. Generally, computer-accessible media may include non-transitory computer-readable storage media or memory media, such as magnetic or optical media, like discs or DVD / CD-ROMs, and volatile or non-volatile media, such as RAM (e.g., SDRAM, DDR, RDRAM, SRAM, etc.), ROM, etc. In some embodiments, computer-accessible media may include transmission media or signals, such as electrical signals, electromagnetic signals, or digital signals transmitted via communication media such as networks and / or wireless links.

[0229] In various implementations, the methods described herein may be implemented in software, hardware, or a combination thereof. Furthermore, the order of the blocks of the method may be changed, and various elements may be added, reordered, combined, omitted, modified, etc. Various modifications and changes will be apparent to those skilled in the art who benefit from this disclosure. The various implementations described herein are intended to be illustrative and not restrictive. Many variations, modifications, additions, and improvements are possible. Thus, multiple examples may be provided for a component described herein as a single example. The boundaries between various components, operations, and data repositories are somewhat arbitrary, and specific operations are shown in the context of a particular exemplary configuration. Other assignments of functionality are contemplated, which may fall within the scope of the appended claims. Finally, the structures and functions of discrete components presented in exemplary configurations may be implemented as combined structures or components. These and other variations, modifications, additions, and improvements may fall within the scope of the implementations as defined in the following claims.

Claims

1. A non-transitory computer-readable medium storing program instructions, said program instructions, when executed by one or more processors, causing said one or more processors to: Receive bit stream, the bit stream comprising: Multiple node groups of nodes in an octree structure of a point cloud, wherein the nodes of the octree structure define the spatial location of points in the point cloud, and wherein each node group comprises a subset of the set of nodes of the octree structure of the point cloud. as well as The geometry of the point cloud is reconstructed based on the nodes included in the corresponding node group of the bit stream, wherein, in order to reconstruct the geometry, the program instructions cause the one or more processors to: The nodes within a given group of nodes are scanned according to a breadth-first scan order; and The next subsequent node group to be evaluated is selected based on the depth-first scan order.

2. The non-transitory computer-readable medium of claim 1, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to: The next subsequent node group to be evaluated is selected based on a flag in the given node group indicating the selection of the next subsequent node group to be evaluated according to the depth-first scan order.

3. The non-transitory computer-readable medium of claim 2, wherein, in order to reconstruct the geometry of the point cloud, the program instructions cause the one or more processors to: The next subsequent node group to be evaluated is selected according to the breadth-first scan order, wherein the next subsequent node group does not include a flag indicating that the next subsequent node group is selected according to the depth-first scan order.

4. The non-transitory computer-readable medium according to any one of claims 1 to 3, wherein, in order to scan the nodes within a given group of nodes according to the breadth-first scan order, the program instructions cause the one or more processors to: Scan the sequential nodes in the node group at the same level of the octree structure, and if the end of the node at the same level of the octree structure is reached before reaching the end of the node group, continue scanning the sequential nodes in the node group at the next lower level of the octree structure.

5. The non-transitory computer-readable medium according to any one of claims 1 to 3, wherein, in order to select the next subsequent group of nodes to be evaluated according to the depth-first scan order, the program instructions cause the one or more processors to: The next node group selected includes child nodes at a lower level of the octree structure than the parent node included in the current node group being evaluated. If the octree structure does not include child nodes of the node included in the current node group being evaluated at a lower level of the octree structure, the depth-first scan order returns to the node group including the parent node of the node in the current node group being evaluated, where the parent node is at a higher level of the octree structure than the node in the current node group being evaluated. If the node group including the parent node has already been evaluated, the depth-first scan order continues down to the node group including the sibling nodes of the node in the current node group being evaluated at the same level of the octree structure.

6. The non-transitory computer-readable medium according to any one of claims 1 to 3, wherein each of the plurality of node groups represents a spatial volume, the spatial volume comprising a plurality of octet cubes of the octet structure at a given octet level of the octet structure.

7. The non-transitory computer-readable medium of claim 6, wherein the bit stream further comprises: The slice header includes the slice header of a subset of the node group, and The slice header indicates the size of the node group of the subset of the node groups included in the slice.

8. The non-transitory computer-readable medium according to any one of claims 1 to 3, wherein occupancy information of the nodes in the octree structure included in the plurality of node groups has been encoded using an entropy encoder, and The program instructions, when executed by the one or more processors, cause the one or more processors to: Based on the occupancy information of the neighboring nodes of the given node whose entropy-encoded occupancy information is being decoded, an encoding context is determined for decoding the occupancy information of the corresponding node in the given node. The neighboring nodes considered for determining the encoding context of a given node are limited to other nodes included in the same node group as the given node.

9. A non-transitory computer-readable medium storing program instructions, which, when executed by one or more processors, cause the one or more processors to: Receive a point cloud to be compressed, wherein the point cloud comprises multiple points in three-dimensional space; Generate an octree structure including occupancy information of nodes in the octree structure, wherein the occupancy information indicates whether a cube or sub-cube at a given level of the octree structure is occupied by one or more points of the point cloud. The nodes of the octree are grouped into multiple node groups, wherein each node group includes a subset of the nodes of the octree structure; as well as Generate a bit stream, the bit stream comprising: The coded occupancy information of the nodes of the octree included in the node group, wherein the nodes within a given node group are organized according to a breadth-first scan order; and The method involves selecting the occupancy information of the nodes in the subsequent node group to be evaluated based on the depth-first scan order.

10. The non-transitory computer-readable medium of claim 9, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to: The bitstream includes a flag indicating that the subsequent node group to be evaluated will be selected according to the depth-first scan order, wherein otherwise the subsequent node group to be evaluated will be selected according to the breadth-first scan order.

11. The non-transitory computer-readable medium of claim 9 or 10, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to: Each subsequent node group to be evaluated is selected according to the depth-first scan order used to select the subsequent node group to be evaluated, wherein the nodes within a given node group are entropy decoded according to the breadth-first scan order.

12. The non-transitory computer-readable medium of claim 9 or 10, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to: The node group set is selected based on either a depth-first scan order or a breadth-first scan order, such that the node group set includes multiple nodes corresponding to the size of a slice used to transmit a portion of the bitstream.

13. The non-transitory computer-readable medium of claim 9 or 10, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to: In the bitstream for a given slice, a flag indicating the last node group of the slice is sent by signal, wherein the slice comprises multiple node groups, and the header includes information about the node groups of the given slice.

14. The non-transitory computer-readable medium of claim 13, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to: The node group size of the node group included in the given slice is sent by signal in the slice header used for the given slice.

15. An apparatus, the apparatus comprising: The memory stores program instructions; and One or more processors, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to: Receive bit stream, the bit stream comprising: Multiple node groups of nodes in an octree structure of a point cloud, wherein the nodes of the octree structure define the spatial location of points in the point cloud, and wherein each node group comprises a subset of the set of nodes of the octree structure of the point cloud. as well as The geometry of the point cloud is reconstructed based on the nodes included in the corresponding node group of the bit stream, wherein, in order to reconstruct the geometry, the program instructions cause the one or more processors to: The nodes within a given group of nodes are scanned according to a breadth-first scan order; and The next subsequent node group to be evaluated is selected based on the depth-first scan order.

16. The apparatus of claim 15, further comprising: monitor, The program instructions, when executed by the one or more processors, cause the one or more processors to: The point cloud with the reconstructed geometry is rendered on the display of the device.

17. The device of claim 15 or 16, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to: The flag indicating the next subsequent node group to be evaluated is based on the previous evaluated node group and the depth-first scan order.

18. The device of claim 15 or 16, wherein the program instructions, when executed by the one or more processors, cause the one or more processors to: The next subsequent node group to be scanned is selected according to the breadth-first scan order, wherein the next subsequent node group does not include a flag indicating that the next subsequent node group is selected according to the depth-first scan order.

19. The device of claim 15 or 16, wherein the program instructions, when executed by the one or more processors, further cause the one or more processors to: Receive a point cloud to be compressed, wherein the point cloud to be compressed includes multiple points in three-dimensional space; Generate an octree structure including occupancy information of the nodes of the octree, wherein the occupancy information indicates whether a cube or sub-cube at a given level of the octree structure is occupied by one or more points of the point cloud to be compressed. The nodes of the octree are grouped into multiple node groups, wherein each node group includes a subset of the nodes of the octree structure; as well as Generate a bit stream, the bit stream comprising: The coded occupancy information of the nodes of the octree included in the node group, wherein the nodes within a given node group are organized according to the breadth-first scan order; and The occupancy information of nodes to be included in the bit stream is selected from among the node groups according to the depth-first scan order.

20. The apparatus of claim 19, further comprising: One or more sensors are configured to capture a plurality of points constituting the point cloud to be compressed, wherein each point includes spatial information and attribute information of the point. The point cloud to be compressed is received from one or more sensors.

Images