Improving the Performance of Geometry Point Cloud Compression (GPCC) Planar Mode Using Inter Prediction

Abstract

An exemplary device for processing a point cloud includes a memory configured to store at least a portion of the point cloud and one or more processors implemented in a circuit, where the one or more processors are configured to obtain planarity information of a reference block of the point cloud, determine a context based on the planarity information of the reference block, context-adaptively code a syntax element indicating whether a current node is coded using the planar mode based on the context, and code the current node using the planar mode based on the current node being coded using the planar mode.

Description

[Technical field]

[0001]

[0001] This application claims priority to U.S. Patent Application No. 17 / 659,219, filed April 14, 2022, and U.S. Provisional Application No. 63 / 176,098, filed April 16, 2021, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 17 / 659,219, filed April 14, 2022, claims the benefit of U.S. Provisional Application No. 63 / 176,098, filed April 16, 2021.

[0002] This disclosure relates to point cloud encoding and decoding. [Brief description of the drawings]

[0003] [Figure 1]

[0003] FIG. 1 is a block diagram illustrating an example encoding and decoding system in which the techniques of this disclosure may be implemented. [Diagram 2]

[0004] 1 is a block diagram illustrating an example geometry point cloud compression (G-PCC) encoder. [Diagram 3]

[0005] FIG. 1 is a block diagram illustrating an example G-PCC decoder. [Figure 4]

[0006] 1 is a flowchart illustrating an example motion estimation technique for InterEM. [Diagram 5]

[0007] 11 is a flowchart illustrating an example technique for estimating local node motion vectors. [Figure 6]

[0008] FIG. 1 is a conceptual diagram illustrating an example range-finding system that may be used with one or more techniques of this disclosure. [Figure 7]

[0009] FIG. 1 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of the present disclosure may be used. [Figure 8]

[0010] FIG. 1 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of this disclosure may be used. [Figure 9]

[0011] 1 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of the present disclosure may be used. [Figure 10A]

[0012] 1 is a conceptual diagram illustrating the range update process in binary arithmetic coding. [Figure 10B] 1 is a conceptual diagram illustrating the range update process in binary arithmetic coding. [Figure 11]

[0013] A conceptual diagram showing the output process in binary arithmetic coding. [Figure 12]

[0014] 1 is a block diagram showing a context-adaptive binary arithmetic coder in a G-PCC encoder. [Figure 13]

[0015] 1 is a block diagram showing a context-adaptive binary arithmetic coder in a G-PCC decoder. [Figure 14]

[0016] 1 is a flow diagram illustrating an example technique for predicting points of a point cloud, in accordance with one or more aspects of the present disclosure. Summary of the Invention

[0004]

[0017] In general, this disclosure describes techniques for coding nodes of a point cloud using inter prediction, such as for the currently developed Geometry Point Cloud Compression (G-PCC) standard. However, the exemplary techniques are not limited to the G-PCC standard. The reference block of a node may be derived by motion compensation using estimated motion information (rotation and translation). Good estimation of motion information may lead to high correlation between the current node and the reference node in terms of geometry structure, such as occupancy, plane information, etc. Thus, utilizing this geometry information of the reference node may improve the coding performance of the current node. This disclosure includes several techniques for utilizing information of the reference block in coding the plane information of the current node. In general, this information may be used in selecting the node's eligibility for the planar coding mode, the context in coding the plane flag and the plane index.

[0005]

[0018] According to one or more techniques of this disclosure, a G-PCC coder may include a memory configured to store at least a portion of a point cloud and one or more processors implemented in a circuit, where the one or more processors are configured to obtain planar information of a reference block of the point cloud, determine a context based on the planar information of the reference block, context-adaptively code a syntax element indicating whether a current node is coded using the planar mode based on the context, and code the current node using the planar mode based on the current node being coded using the planar mode.

[0006]

[0019] In one example, a method for processing a point cloud includes obtaining planar information of a reference block of the point cloud, determining a context based on the planar information of the reference block, context-adaptive coding a syntax element indicating whether a current node is coded using the planar mode based on the context, and coding the current node using the planar mode based on the current node being coded using the planar mode.

[0007]

[0020] In another example, a computer-readable storage medium stores instructions that, when executed by one or more processors, cause the one or more processors to obtain planar information of a reference block of the point cloud; determine a context based on the planar information of the reference block; context-adaptively code a syntax element indicating whether a current node is coded using the planar mode based on the context; and code the current node using the planar mode based on the current node being coded using the planar mode.

[0008]

[0021] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009]

[0022] 1 is a block diagram illustrating an example encoding and decoding system 100 that may implement the techniques of this disclosure. The techniques of this disclosure are generally directed to coding (encoding and / or decoding) point cloud data, i.e., supporting point cloud compression. In general, point cloud data includes any data for processing a point cloud. Coding may be effective in compressing and / or decompressing point cloud data.

[0010]

[0023] As shown in FIG. 1, the system 100 includes a source device 102 and a destination device 116. The source device 102 provides encoded point cloud data to be decoded by the destination device 116. In particular, in the example of FIG. 1, the source device 102 provides the point cloud data to the destination device 116 via a computer-readable medium 110. The source device 102 and the destination device 116 may comprise any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, terrestrial or marine vehicles, spacecraft, aircraft, robots, LIDAR devices, satellites, and the like. In some cases, the source device 102 and the destination device 116 may be equipped for wireless communication.

[0011]

[0024] In the example of FIG. 1, the source device 102 includes a data source 104, a memory 106, a G-PCC encoder 200, and an output interface 108. The destination device 116 includes an input interface 122, a G-PCC decoder 300, a memory 120, and a data consumer 118. According to the present disclosure, the G-PCC encoder 200 of the source device 102 and the G-PCC decoder 300 of the destination device 116 may be configured to apply techniques of the present disclosure related to utilizing information of a reference block in coding plane information of a current node (e.g., a current block). Thus, the source device 102 represents an example of an encoding device, while the destination device 116 represents an example of a decoding device. In other examples, the source device 102 and the destination device 116 may include other components or configurations. For example, the source device 102 may receive data (e.g., point cloud data) from an internal source or an external source. Similarly, destination device 116 may interface with external data consumers rather than including the data consumers in the same device.

[0012]

[0025] The system 100 shown in FIG. 1 is only an example. In general, other digital encoding and / or decoding devices may implement the techniques of this disclosure related to utilizing information of a reference block in coding the plane information of a current node. The source device 102 and the destination device 116 are only examples of devices in which the source device 102 generates coded data for transmission to the destination device 116. This disclosure refers to a "coding" device as a device that performs coding (encoding and / or decoding) of data. Thus, the G-PCC encoder 200 and the G-PCC decoder 300 represent examples of coding devices, in particular, encoders and decoders, respectively. In some examples, the source device 102 and the destination device 116 may operate substantially symmetrically such that each of the source device 102 and the destination device 116 includes an encoding component and a decoding component. Thus, the system 100 may support one-way or two-way transmission between the source device 102 and the destination device 116, for example, streaming, playback, broadcasting, telephony, navigation, and other applications.

[0013]

[0026] Generally, the data source 104 represents a source of data (i.e., raw, unencoded point cloud data) and may provide a continuous series of "frames" of data to the G-PCC encoder 200, which encodes the data for the frames. The data source 104 of the source device 102 may include a point cloud capture device, such as any of a variety of cameras or sensors, e.g., a 3D scanner or light detection and ranging (LIDAR) device, one or more video cameras, an archive containing previously captured data, and / or a data feed interface for receiving data from a data content provider. Alternatively or additionally, the point cloud data may be computer-generated from a scanner, camera, sensor, or other data. For example, the data source 104 may generate computer graphics-based data as source data, or may create a combination of live data, archived data, and computer-generated data. In each case, the G-PCC encoder 200 encodes the captured data, pre-captured data, or computer-generated data. The G-PCC encoder 200 may reorder the frames from a reception order (sometimes referred to as a "display order") into a coding order for coding. The G-PCC encoder 200 may generate one or more bitstreams including the encoded data. The source device 102 may then output the encoded data onto a computer-readable medium 110 via an output interface 108, for receipt and / or retrieval by, for example, an input interface 122 of a destination device 116.

[0014]

[0027] The memory 106 of the source device 102 and the memory 120 of the destination device 116 may represent general-purpose memories. In some examples, the memory 106 and the memory 120 may store raw data, e.g., raw data from the data source 104 and raw decoded data from the G-PCC decoder 300. Additionally or alternatively, the memory 106 and the memory 120 may store software instructions executable by, e.g., the G-PCC encoder 200 and the G-PCC decoder 300, respectively. Although the memory 106 and the memory 120 are shown in this example separately from the G-PCC encoder 200 and the G-PCC decoder 300, it should be understood that the G-PCC encoder 200 and the G-PCC decoder 300 may also include internal memories for functionally similar or equivalent purposes. Additionally, the memory 106 and the memory 120 may store encoded data, e.g., the output from the G-PCC encoder 200 and the input to the G-PCC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, e.g., to store raw, decoded, and / or encoded data. For example, memory 106 and memory 120 may store data representing a point cloud.

[0015]

[0028] The computer-readable medium 110 may represent any type of medium or device capable of transporting encoded data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium for enabling the source device 102 to transmit the encoded data directly to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. The output interface 108 may modulate a transmission signal including the encoded data, and the input interface 122 may demodulate a received transmission signal according to a communication standard such as a wireless communication protocol. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication from the source device 102 to the destination device 116.

[0016]

[0029] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0017]

[0030] In some examples, the source device 102 may output the encoded data to a file server 114 or another intermediate storage device that may store the encoded data generated by the source device 102. The destination device 116 may access the stored data from the file server 114 via streaming or download. The file server 114 may be any type of server device capable of storing the encoded data and transmitting the encoded data to the destination device 116. The file server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. The destination device 116 may access the encoded data from the file server 114 through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both, that is suitable for accessing the encoded data stored on the file server 114. The file server 114 and the input interface 122 may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

[0018]

[0031] The output interface 108 and the input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples in which the output interface 108 and the input interface 122 comprise wireless components, the output interface 108 and the input interface 122 may be configured to transfer data, such as encoded data, according to a cellular communication standard, such as 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, etc. In some examples in which the output interface 108 comprises a wireless transmitter, the output interface 108 and the input interface 122 may be configured to transfer data, such as encoded data, according to other wireless standards, such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee), the Bluetooth standard, etc. In some examples, the source device 102 and / or the destination device 116 may include respective system-on-chip (SoC) devices. For example, the source device 102 may include a SoC device for performing functions attributable to the G-PCC encoder 200 and / or the output interface 108, and the destination device 116 may include a SoC device for performing functions attributable to the G-PCC decoder 300 and / or the input interface 122.

[0019]

[0032] The techniques of this disclosure may be applied to encoding and decoding in support of any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors, and processing devices such as local or remote servers, geographic mapping, or other applications.

[0020]

[0033] The input interface 122 of the destination device 116 receives the encoded bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded bitstream may include signaling information defined by the G-PCC encoder 200, such as syntax elements having values ​​that describe characteristics and / or processing of the coded units (e.g., slices, pictures, groups of pictures, sequences, etc.), which are also used by the G-PCC decoder 300. The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display for presenting an image based on the point cloud.

[0021]

[0034] The G-PCC encoder 200 and the G-PCC decoder 300 may each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the present technique is implemented partially in software, a device may store instructions for the software on a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to implement the techniques of the present disclosure. Each of the G-PCC encoder 200 and the G-PCC decoder 300 may be included in one or more encoders or decoders, any of which may be integrated as part of a composite encoder / decoder (codec) in the respective device. A device including the G-PCC encoder 200 and / or the G-PCC decoder 300 may comprise one or more integrated circuits, microprocessors, and / or other types of devices.

[0022]

[0035] The G-PCC encoder 200 and the G-PCC decoder 300 may operate according to a coding standard, such as the Video Point Cloud Compression (V-PCC) standard or the Geometry Point Cloud Compression (G-PCC) standard. This disclosure may generally refer to coding (e.g., encoding and decoding) a picture to include the process of encoding or decoding data. An encoded bitstream generally includes a set of values ​​for syntax elements that represent coding decisions (e.g., coding modes).

[0023]

[0036] This disclosure may generally refer to "signaling" certain information, such as syntax elements. The term "signaling" may generally refer to communication of values ​​for syntax elements and / or other data used to decode encoded data. That is, the G-PCC encoder 200 may signal values ​​for syntax elements in a bitstream. Generally, signaling refers to generating values ​​in a bitstream. As described above, the source device 102 may transport a bitstream to the destination device 116 in substantially real-time or non-real-time, which may occur when storing syntax elements in the storage device 112 for later retrieval by the destination device 116.

[0024]

[0037] ISO / IEC MPEG (JTC1 / SC29 / WG11) is studying the potential need for a standard for point cloud coding techniques with compression capabilities significantly beyond those of current methods, with the goal of producing such a standard. The group is collaborating on this exploration in a collaborative effort known as the 3D Graphics Team (3DG) to evaluate compression technology designs proposed by experts in the field.

[0025]

[0038] Point cloud compression activities are categorized into two different approaches. The first approach is "Video Point Cloud Compression" (V-PCC), which segments a 3D object and projects the segments into multiple 2D planes (represented as "patches" in a 2D frame), which are further coded by a legacy 2D video codec, such as the High Efficiency Video Coding (HEVC) (ITU-T H.265) codec. The second approach is "Geometry-Based Point Cloud Compression" (G-PCC), which directly compresses the 3D geometry, i.e., the positions of a set of points in 3D space and the associated attribute values ​​(for each point associated with the 3D geometry). G-PCC addresses compression of point clouds in both category 1 (static point clouds) and category 3 (dynamically collected point clouds). A recent draft of the G-PCC standard is available in G-PCC DIS, ISO / IEC JTC1 / SC29 / WG11 w19088, Brussels, Belgium, January 2020, and the codec description is available in G-PCC Codec Description v6, ISO / IEC JTC1 / SC29 / WG11 w19091, Brussels, Belgium, January 2020.

[0026]

[0039] A point cloud includes a set of points in 3D space and may have attributes associated with the points. The attributes may be color information such as R, G, B, or Y, Cb, Cr, or reflectance information, or other attributes. Point clouds may be captured by various cameras or sensors, such as LIDAR sensors and 3D scanners, or may be computer generated. Point cloud data is used in a variety of applications, including but not limited to architecture (modeling), graphics (3D models for visualization and animation), and the automotive industry (LIDAR sensors used to aid navigation).

[0027]

[0040] The 3D space occupied by the point cloud data may be enclosed by a virtual bounding box. The positions of the points in the bounding box may be represented with a certain precision, and thus the positions of one or more points may be quantized based on that precision. At the smallest level, the bounding box is divided into voxels, which are the smallest units of space represented by a unit cube. A voxel in a bounding box may be associated with zero, one, or more points. The bounding box may be divided into multiple cubic / rectangular regions, sometimes called tiles. Each tile may be coded into one or more slices. The partitioning of the bounding box into slices and tiles may be based on the number of points in each partition or other considerations (e.g., a particular region may be coded as a tile). The slice regions may be further partitioned using partitioning decisions similar to those in video codecs.

[0028]

[0041] Figure 2 provides an overview of a G-PCC encoder 200. Figure 3 provides an overview of a G-PCC decoder 300. The illustrated modules are logical and do not necessarily correspond one-to-one to the code implemented in the reference implementation of the G-PCC codec, i.e., the TMC13 test model software studied by ISO / IEC MPEG (JTC1 / SC29 / WG11).

[0029]

[0042] In both the G-PCC encoder 200 and the G-PCC decoder 300, the point cloud position is coded first. The attribute coding depends on the decoded geometry. In Figures 2 and 3, the grey shaded modules are the options typically used for category 1 data. The shaded modules are the options typically used for category 3 data. All other modules are common between category 1 and category 3.

[0030]

[0043] For category 3 data, the compressed geometry is typically represented as an octree from the root down to the leaf level of the individual voxels. For category 1 data, the compressed geometry is typically represented by a pruned octree (i.e. an octree from the root down to the leaf level of blocks larger than a voxel) and a model that approximates the surface within each leaf of the pruned octree. In this way, both category 1 and category 3 data share the octree coding mechanism, but category 1 data may additionally approximate the voxels within each leaf with a surface model. The surface model used is a triangulation with 1-10 triangles per block, resulting in a triangle soup. Category 1 geometry codecs are therefore known as Trisoup geometry codecs and category 3 geometry codecs are known as Octree geometry codecs.

[0031]

[0044] At each node of the octree, the occupancy is signaled (if not inferred) for one or more of its child nodes (up to 8 nodes). Several neighborhoods are specified, including (a) nodes that share faces with the current octree node, (b) nodes that share faces, edges or vertices with the current octree node, etc. Within each neighborhood, the occupancy of a node and / or its children can be used to predict the occupancy of the current node or its children. For sparsely distributed points in several nodes of the octree, the codec also supports a direct coding mode, where the 3D positions of the points are directly coded. A flag can be signaled to indicate that the direct mode is signaled. At the lowest level, the number of points associated with an octree node / leaf node can also be coded.

[0032]

[0045] When geometry is coded, attributes corresponding to the geometry points are coded. When there are multiple attribute points corresponding to one reconstructed / decoded geometry point, an attribute value representing the reconstructed point can be derived.

[0033]

[0046] In G-PCC, there are three attribute coding methods: Region Adaptive Hierarchical Transform (RAHT) coding, Interpolation-based Hierarchical Nearest Neighbor Prediction (Prediction Transform), and Interpolation-based Hierarchical Nearest Neighbor Prediction with Update / Lifting Step (Lifting Transform). RAHT and lifting are typically used for category 1 data, while prediction is typically used for category 3 data. However, either method can be used for any data, and just as in the case of geometry codecs in G-PCC, the attribute coding method used to code point clouds is specified in the bitstream.

[0034]

[0047] The coding of attributes may be done in levels of detail (LODs), with each level of detail obtaining a finer representation of the point cloud attributes. Each level of detail may be designated based on a distance metric from neighboring nodes or based on a sampling distance.

[0035]

[0048] In the G-PCC encoder 200, the residual obtained as the output of the coding method for the attribute is quantized. The residual may be obtained by subtracting the attribute value from a prediction derived based on points in the neighborhood of the current point and based on the attribute value of a previously coded point. The quantized residual may be coded using context-adaptive arithmetic coding.

[0036]

[0049] In the example of FIG. 2, the G-PCC encoder 200 may include a coordinate transformation unit 202, a color transformation unit 204, a voxelization unit 206, an attribute transfer unit 208, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, a geometry reconstruction unit 216, a RAHT unit 218, an LOD generation unit 220, a lifting unit 222, a coefficient quantization unit 224, and an arithmetic coding unit 226.

[0037]

[0050] As shown in the example of FIG. 2, the G-PCC encoder 200 may obtain a set of positions of points in a point cloud and a set of attributes. The G-PCC encoder 200 may obtain a set of positions of points in a point cloud and a set of attributes from a data source 104 (FIG. 1). The positions may include coordinates of the points in the point cloud. The attributes may include information about the points in the point cloud, such as a color associated with the points in the point cloud. The G-PCC encoder 200 may generate a geometry bitstream 203 that includes an encoded representation of the positions of the points in the point cloud. The G-PCC encoder 200 may also generate an attribute bitstream 205 that includes an encoded representation of the set of attributes.

[0038]

[0051] The coordinate transformation unit 202 may apply a transformation to the coordinates of the points to convert the coordinates from an initial domain to a transformation domain. This disclosure may refer to the transformed coordinates as transformed coordinates. The color transformation unit 204 may apply a transformation to convert color information of the attributes to a different domain. For example, the color transformation unit 204 may convert color information from an RGB color space to a YCbCr color space.

[0039]

[0052] Further, in the example of FIG. 2, voxelization unit 206 may voxelize the transformed coordinates. Voxelization of the transformed coordinates may include quantization and removing some points of the point cloud. In other words, multiple points of the point cloud may be contained within a single "voxel", which may then be treated as one point in some respects. Further, octree analysis unit 210 may generate an octree based on the voxelized transformed coordinates. Further, in the example of FIG. 2, surface approximation analysis unit 212 may analyze the points to potentially determine a surface representation of the set of points. Arithmetic coding unit 214 may entropy code syntax elements representing the octree and / or surface information determined by surface approximation analysis unit 212. G-PCC encoder 200 may output these syntax elements in geometry bitstream 203. Geometry bitstream 203 may also include other syntax elements, including syntax elements that are not arithmetically coded.

[0040]

[0053] The geometry reconstruction unit 216 may reconstruct transformation coordinates of points in the point cloud based on the octree, the data indicating the surface determined by the surface approximation analysis unit 212, and / or other information. The number of transformation coordinates reconstructed by the geometry reconstruction unit 216 may differ from the original number of points of the point cloud due to voxelization and surface approximation. This disclosure may refer to the obtained points as reconstructed points. The attribute transfer unit 208 may transfer attributes of the original points of the point cloud to the reconstructed points of the point cloud.

[0041]

[0054] Further, the RAHT unit 218 may apply RAHT coding to the attributes of the reconstructed points. In some examples, under RAHT, attributes of a 2×2×2 block of point positions are taken and transformed along one direction to obtain four low (L) frequency nodes and four high (H) frequency nodes. Then, the four low frequency nodes (L) are transformed in a second direction to obtain two low (LL) frequency nodes and two high (LH) frequency nodes. The two low frequency nodes (LL) are transformed along a third direction to obtain one low (LLL) frequency node and one high (LLH) frequency node. The low frequency node LLL corresponds to the DC coefficient, and the high frequency nodes H, LH, and LLH correspond to the AC coefficients. The transformation in each direction may be a 1-D transformation with two coefficient weights. The low frequency coefficients may be taken as the coefficients of the 2x2x2 block for the next higher level of the RAHT transform, and the AC coefficients are coded without modification, and such a transformation continues up to the top root node. The tree traversal for coding is used from top to bottom to calculate the weights to be used for the coefficients, and the transformation order is from bottom to top. The coefficients may then be quantized and coded.

[0042]

[0055] Alternatively or additionally, the LOD generation unit 220 and the lifting unit 222 may apply LOD processing and lifting, respectively, to the attributes of the reconstructed points. The LOD generation is used to split the attributes into different refinement levels. Each refinement level provides a refinement of the attributes of the point cloud. The first refinement level provides a coarse approximation and includes a small number of points, the subsequent refinement levels typically include more points, and so on. The refinement levels may be constructed using a distance-based metric or may also use one or more other classification criteria (e.g., subsampling from a particular order). Thus, all reconstructed points may be included in a refinement level. Each level of detail is created by taking the union of all points up to a particular refinement level, e.g., LOD1 is obtained based on refinement level RL1, LOD2 is obtained based on RL1 and RL2, ...LODN is obtained by the union of RL1, RL2, ...RLN. In some cases, LOD generation may be followed by a prediction scheme (e.g., predictive transformation), where attributes associated with each point in the LOD are predicted from a weighted average of previous points, and the residual is quantized and entropy coded. Lifting schemes are built on top of predictive transformation mechanisms, where an update operator is used to update the coefficients, and adaptive quantization of the coefficients is performed.

[0043]

[0056] The RAHT unit 218 and the lifting unit 222 may generate coefficients based on the attributes. The coefficient quantization unit 224 may quantize the coefficients generated by the RAHT unit 218 or the lifting unit 222. The arithmetic coding unit 226 may apply arithmetic coding to syntax elements representing the quantized coefficients. The G-PCC encoder 200 may output these syntax elements in the attribute bitstream 205. The attribute bitstream 205 may also include other syntax elements, including syntax elements that are not arithmetically coded.

[0044]

[0057] In the example of FIG. 3, the G-PCC decoder 300 may include a geometry arithmetic decoding unit 302, an attribute arithmetic decoding unit 304, an octree synthesis unit 306, an inverse quantization unit 308, a surface approximation synthesis unit 310, a geometry reconstruction unit 312, a RAHT unit 314, a LoD generation unit 316, an inverse lifting unit 318, an inverse transform coordinate unit 320, and an inverse transform color unit 322.

[0045]

[0058] The G-PCC decoder 300 may obtain the geometry bitstream 203 and the attribute bitstream 205. A geometry arithmetic decoding unit 302 of the decoder 300 may apply arithmetic decoding (e.g., context-adaptive binary arithmetic coding (CABAC) or other types of arithmetic decoding) to syntax elements in the geometry bitstream 203. Similarly, an attribute arithmetic decoding unit 304 may apply arithmetic decoding to syntax elements in the attribute bitstream 205.

[0046]

[0059] The octree synthesis unit 306 may synthesize an octree based on syntax elements parsed from the geometry bitstream 203. Starting with the root node of the octree, the occupancy of each of the eight child nodes at each octree level is signaled in the bitstream. When the signaling indicates that a child node at a particular octree level is occupied, the occupancy of the child node's children is signaled. The signaling of the nodes at each octree level is signaled before proceeding to a subsequent octree level. At the final level of the octree, each node corresponds to a voxel position, and when a leaf node is occupied, one or more points may be specified to be occupied at the voxel position. In some cases, some branches of the octree may end before the final level due to quantization. In such cases, the leaf node is considered an occupied node that has no child nodes. In cases where surface approximations are used in the geometry bitstream 203, the surface approximation synthesis unit 310 may determine a surface model based on syntax elements parsed from the geometry bitstream 203 and based on the octree.

[0047]

[0060] Further, the geometry reconstruction unit 312 may perform reconstruction to determine coordinates of points in the point cloud. For each position in a leaf node of the octree, the geometry reconstruction unit 312 may reconstruct the node position by using a binary representation of the leaf node in the octree. At each respective leaf node, the number of points at the respective leaf node is signaled, which indicates the number of overlapping points at the same voxel position. When geometry quantization is used, the point positions are scaled to determine a reconstructed point position value.

[0048]

[0061] The inverse transform coordinate unit 320 may apply an inverse transform to the reconstructed coordinates (positions) of points in the point cloud to convert them from the transformed domain back to the original domain. The positions of points in the point cloud may be in the floating-point domain, but the point positions in the G-PCC codec are coded in the integer domain. The inverse transform may be used to convert the positions back to the original domain.

[0049]

[0062] 3, the inverse quantization unit 308 may inverse quantize the attribute value. The attribute value may be based on a syntax element obtained from the attribute bitstream 205 (e.g., including the syntax element decoded by the attribute arithmetic decoding unit 304).

[0050]

[0063] Depending on how the attribute values ​​are coded, the RAHT unit 314 may perform RAHT coding to determine color values ​​for the points of the point cloud based on the dequantized attribute values. RAHT decoding is performed from the top of the tree to the bottom. At each level, the low and high frequency coefficients derived from the dequantization process are used to derive component values. At the leaf nodes, the derived values ​​correspond to the attribute values ​​of those coefficients. The weight derivation process for the points is similar to the process used in the G-PCC encoder 200. Alternatively, the LOD generation unit 316 and the inverse lifting unit 318 may use a level-of-detail based technique to determine color values ​​for the points of the point cloud. The LOD generation unit 316 decodes each LOD that gives a progressively finer representation of the attributes of the points. With a predictive transform, the LOD generation unit 316 derives a prediction of the point from a weighted sum of the points that are at the previous LOD or that were previously reconstructed at the same LOD. The LOD generation unit 316 may add a prediction to the residual (obtained after inverse quantization) to obtain a reconstructed value of the attribute. When a lifting scheme is used, the LOD generation unit 316 may also include an update operator for updating the coefficients used to derive the attribute value. The LOD generation unit 316 may also apply inverse adaptive quantization in this case.

[0051]

[0064] 3, the inverse transform color unit 322 may apply an inverse color transform to the color values. The inverse color transform may be the inverse of the color transform applied by the color transform unit 204 of the encoder 200. For example, the color transform unit 204 may transform the color information from the RGB color space to the YCbCr color space. Thus, the inverse color transform unit 322 may transform the color information from the YCbCr color space to the RGB color space.

[0052]

[0065] The various units in FIG. 2 and FIG. 3 are shown to aid in understanding the operations performed by the encoder 200 and the decoder 300. The units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. A fixed function circuit refers to a circuit that provides a specific function and is pre-configured with respect to the operations that may be performed. A programmable circuit refers to a circuit that may be programmed to perform various tasks and to provide flexible functionality in the operations that may be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware instructions. A fixed function circuit may execute software instructions (e.g., to receive parameters or output parameters), but the type of operations that the fixed function circuit performs is generally invariant. In some examples, one or more of the units may be separate circuit blocks (fixed function or programmable), and in some examples, one or more of the units may be an integrated circuit.

[0053]

[0066] The planar coding mode, first proposed in (Sebastien Lasserre, David Flynn, "Planar mode in octree-based geometry coding", ISO / IEC JTC1 / SC29 / WG11 MPEG / m48906, Gothenburg, Sweden, July 2019), was adopted at the 128th MPEG meeting in Geneva, Switzerland (Sebastien Lasserre, Jonathan Taquet, "CE13.22 report on planar coding mode", ISO / IEC JTC1 / SC29 / WG11 MPEG / m50008, Geneva, Switzerland, October 2019). The angular coding mode, first proposed in (Sebastien Lasserre, Jonathan Taquet, “[GPCC][CE 13.22 related] An improvement of the planar coding mode”, ISO / IEC JTC1 / SC29 / WG11 MPEG / m50642, Geneva, Switzerland, October 2019, hereafter “m50642”), was adopted at the 129th MPEG meeting in Brussels, Belgium (Sebastien Lasserre, Jonathan Taquet, “[GPCC] CE 13.22 report on angular mode”, ISO / IEC JTC1 / SC29 / WG11 MPEG / m51594, Brussels, Belgium, January 2020, hereafter “m51594”) to improve the coding efficiency of the planar mode by using the sensor characteristics of typical LIDAR sensors. The angle coding mode is optionally used in conjunction with the planar mode to improve the coding of vertical (z) plane position syntax elements by employing knowledge of the position and angle at which the laser beam is sensed in a typical LIDAR sensor. Additionally, the angle coding mode may be optionally used to improve the coding of vertical z position bits in the IDCM.In a separate contribution (Geert Van der Auwera, Bappaditya Ray, Louis Kerofsky, Adarsh ​​K. Ramasubramonian, Marta Karczewicz, "[GPCC][New Proposal] Angular mode simplifications and HLS refinements", ISO / IEC JTC1 / SC29 / WG11 MPEG / m53693, Teleconference (formerly Alpbach Conference), April 2020), context derivation for angular coding modes was simplified and HLS coding of sensor data parameters was made more efficient. The description of angular modes in the following sections is based on the original MPEG contributions [m50642, m51594] and on the GPCC DIS text (G-PCC DIS, ISO / IEC JTC1 / SC29 / WG11 w19617, Teleconference, November 2020, hereafter "GPCC DIS").

[0054]

[0067] The azimuthal coding mode was first proposed in (Sebastien Lasserre, Jonathan Taquet, "The azimuthal coding mode", ISO / IEC JTC1 / SC29 / WG11 MPEG / m51596, Brussels, Belgium, January 2020, hereafter "m51596") and adopted at the 130th MPEG Teleconference (Sebastien Lasserre, Jonathan Taquet, "Report on azimuthal coding mode", ISO / IEC JTC1 / SC29 / WG11 MPEG / m52958, Teleconference (formerly Alpbach Conference), April 2020, hereafter "m52958"). The azimuth coding mode is similar to the angle mode, extending the angle mode to the coding of the (x) and (y) planar position syntax elements of the planar mode and improving the coding of the x or y position bits in the IDCM. In a separate contribution at the 131st MPEG Teleconference (Geert Van der Auwera, Bappaditya Ray, Adarsh ​​K. Ramasubramonian, Marta Karczewicz, [GPCC][New Proposal] Planar and azimuthal coding mode simplifications, ISO / IEC JTC1 / SC29 / WG11 MPEG / m54694, Teleconference, July 2020, hereafter "m54694"), the number of contexts used in the azimuth mode was significantly reduced.

[0055]

[0068] Note: "Angle mode" may also refer to azimuth mode in the following sections.

[0056]

[0069] The specifications related to planar coding modes are summarized in the GPCC DIS as follows: 8.2.3.1 Node Eligibility for Planar Coding Mode XXX Split and rearrange [XXX, this process is missing the planar rate update after decoding is_planar_flag] The explicit coding of the occupancy plane is conditional on the probability of XXX. The array PlanarRate with elements PlanarRate[k] for k=0..2 is an estimate of the probability that the occupancies of the nodes form a single plane perpendicular to the kth axis. The variable LocalDensity is an estimate of the average number of occupied children in a node. The variable NumNodesUntilPlanarUpdate counts the number of nodes that will be parsed before updating the PlanarRate and LocalDensity. [XXX Entropy state continues] At the start of parsing a geometry_octree syntax structure, PlanarRate and LocalDensity are initialized as follows:

[0057]

number

[0058] At the beginning of parsing each geometry_octree_node syntax structure, NumNodesUntilPlanarUpdate is decremented. If NumNodesUntilPlanarUpdate is less than 0, PlanarRate and LocalDensity are updated as follows:

[0059] The number of occupied sibling nodes is determined and used to update the LocalDensity estimate.

[0060]

number

[0061] The number of nodes until the next update is:

[0062]

number

[0063] The occupancy information of the parent nodes is used to determine the presence of a single occupied plane along each axis and to update the corresponding plane probability estimate, PlanarRate[k].

[0064]

number

[0065] At the start of parsing each geometry_octree_node syntax structure, for each axis it is determined whether the current node is eligible to signal planarity information. The output of this process is an array PlanarEligible with elements PlanarEligible[k], for k=0..2. First, PlanarRate is used to determine the ordering planeOrder[k] of the three planes from most likely to least likely according to Table 18. Then, PlanarEligible is set as follows:

[0066]

number

[0067] [Table 1]

[0068] Syntax elements may be signaled in the bitstream: is_planar_flag[axisIdx] equal to 1 indicates that the positions of the children of the current node form a single plane perpendicular to the axisIdxth axis. is_planar_flag[axisIdx] equal to 0, when present, indicates that the positions of the children of the current node occupy both planes perpendicular to the axisIdxth axis. The context index (ctxIdx) for coding is_planar_flag is specified in Table 37 in the GPCC DIS, where it is set equal to axisIdx. 8.2.3.2 Buffer that tracks closest nodes along an axis The arrays PlanarPrevPos, PlanarPlane, and IsPlanarNode record information about previously decoded geometry tree nodes for use in determining the ctxIdx for the syntax element plane_position. When geometry_planar_enabled_flag is equal to 0 or planar_buffer_disabled_flag is equal to 1, the arrays are not used by the decoding process. In this process, the variable axisIdx is used to represent one of the three coded axes, and the variable axisPos represents the node's position along the axisIdx-th axis. The value of axisPos is in the range 0..0x3fff. The array IsPlanarNode with values ​​IsPlanarNode[axisIdx][axisPos] indicates whether the most recently decoded node with the axisIdx-th position component equal to axisPos is a plane in a plane perpendicular to the axisIdx-th axis. An array PlanarPrevPos with values ​​PlanarPrevPos[axisIdx][axisPos] storing the maximum position component of the most recently decoded node that has the axisIdx-th position component equal to axisPos. The array PlanarPlane with value PlanarPlane[axisIdx][axisPos] indicates the value of plane_position[axisIdx] for the most recently decoded node with the axisIdx-th position component equal to axisPos. At the beginning of each geometry tree level, the elements of the arrays PlanarPrevPos and IsPlanarNode are initialized to zero. After decoding each geometry_planar_mode_data syntax structure with parameters childIdx and axisIdx, the arrays PlanarPrevPos, PlanarPlane and IsPlanarNode are updated as follows:

[0069] The variable axisPos, which represents the position along the axisIdx-th axis, is derived as follows.

[0070]

number

[0071] The array entry corresponding to the node is updated as follows:

[0072]

number

[0073] 8.2.3.3 Determining the ctxIdx for the syntax element plane_position The inputs to this process are:

[0074] A variable, axisIdx, that identifies the axis normal to the plane, and The position (sN,tN,vN) of the current node within the geometry tree level. The output of this process is the variable ctxIdx. The variable neighOccupied indicates whether there are any neighbors to the current node along both axisIdx-th axes. It is derived as follows:

[0075]

number

[0076] When planar_buffer_disabled_flag is equal to 1, the value of ctxIdx is set equal to adjPlaneCtxInc and no further processing is performed by this process. Otherwise, the remainder of this section applies. The variable axisPos indicates the 14 least significant position bits of the current node along the axisIdxth axis.

[0077]

number

[0078] The variable dist represents the distance between the current node and the most recently decoded [Ed. decoded is probably the wrong term?] node position with the same value of axisPos along the axisIdx-th axis. It is derived as follows:

[0079]

number

[0080] The context index ctxIdx is derived as follows.

[0081]

number

[0082] 8.2.3.4 Determination of planePosIdxAzimuthalS and planePosIdxAzimuthalT for coding of horizontal plane positions [Ed. Correction on how this interacts with ctxIdx above. NB: ctxIdx is not plane independent] The determination of planePosIdxAngularS for the arithmetic coding of plane_position[0] and planePosIdxAngularT for the arithmetic coding of plane_position[1] is obtained as follows. When geometry_angular_enabled_flag is equal to 0, the values ​​of both planePosIdxAzimuthalS and planePosIdxAzimuthalT are set equal to planePosIdx. Otherwise the following applies:

[0083]

number

[0084] The determination of the contextAngular for the arithmetic coding of plane_position[2] is performed as described in XREF. 8.2.3.5 Determining planePosIdxAngular for coding vertical plane positions [Ed. Correction on how this interacts with ctxIdx above. NB: ctxIdx is not plane independent] The determination of planePosIdxAngular for the arithmetic coding of plane_position[2] is obtained as follows: When geometry_angular_enabled_flag is equal to 0, the value of planePosIdxAngular is set equal to planePosIdx. Otherwise, the following applies:

[0085]

number

[0086] The determination of the contextAngular for the arithmetic coding of plane_position[2] is performed as described in Section 8.2.4.4. Angle and azimuth modes in GPCC DIS Angle Mode Syntax The syntax elements that convey the Lidar laser sensor information required for the angle coding mode to have any coding efficiency benefits are italicized in Table 2. The semantics of these syntax elements are specified in the GPCC DIS as follows: geometry_planar_enabled_flag equal to 1 indicates that the planar coding mode is activated. geometry_planar_enabled_flag equal to 0 indicates that the planar coding mode is not activated. When not present, geometry_planar_enabled_flag is inferred to be 0. geom_planar_th[i], for i in the range 0..2, specifies the value of the activation threshold for the planar coding mode along the i-th most likely direction for which the planar coding mode is efficient. geom_planar_th[i] is an integer in the range 0..127. geom_idcm_rate_minus1 specifies the rate at which a node may be eligible for direct coding. When not present, geom_idcm_rate_minus1 is inferred to be 31. The array IdcmEnableMask is derived as follows:

[0087]

number

[0088] geometry_angular_enabled_flag equal to 1 indicates that the angular coding mode is activated. geometry_angular_enabled_flag equal to 0 indicates that the angular coding mode is not activated. A geom_slice_angular_origin_present_flag equal to 1 specifies that a slice relative angular origin is present in the geometry slice header. A geom_slice_angular_origin_present_flag equal to 0 specifies that an angular origin is not present in the geometry slice header. When not present, the geom_slice_angular_origin_present_flag is assumed to be 0. geom_angular_origin_bits_minus1 + 1 is the bit - unit length of the syntax element geom_angular_origin_xyz[k]. geom_angular_origin_xyz[k] specifies the k - th component of the (x, y, z) coordinates of the origin used in the processing of the angular coding mode. When not present, for k = 0..2, the value of geom_angular_origin_xyz[k] is assumed to be 0. geom_angular_azimuth_scale_log2 and geom_angular_radius_scale_log2 specify factors used to scale positions coded in a spherical coordinate system during conversion to Cartesian coordinates. geom_angular_azimuth_step_minus1 + 1 specifies the unit change of the azimuth angle. The differential prediction residuals used in the angular prediction tree coding can be partially represented as multiples of geom_angular_azimuth_step_minus1 + 1. The value of geom_angular_azimuth_step_minus1 shall be less than (1<<geom_angular_azimuth_scale_log2). numbers_lasers_minus1 + 1 specifies the number of lasers used for the angular coding mode. laser_angle_init, and laser_angle_diff[i], for i=0..number_lasers_minus1, specify the tangent of the elevation angle of the ith laser relative to the horizontal plane defined by the first and second coded axes. The array LaserAngle[i], for i=0..number_lasers_minus1, is derived as follows:

[0089]

number

[0090] It is a bitstream conformance requirement that the value of LaserAngle[i], for i=1..number_lasers_minus1, be greater than or equal to LaserAngle[i-1]. laser_correction_init, and laser_correction_diff[i], for i=1..number_lasers_minus1, specify the correction along the second intrinsic axis of the ith laser position relative to GeomAngularOrigin[2]. laser_phi_per_turn_init_minus1, and laser_phi_per_turn_diff[i], for i=1..number_lasers_minus1, specify the number of samples generated by the ith laser of the rotation sensing system located at the origin used in the angle coding mode of processing. The arrays LaserCorrection[i] and LaserPhiPerTurn[i], for i=1..number_lasers_minus1, are derived as follows:

[0091]

number

[0092] It is a bitstream conformance requirement that the value of LaserPhiPerTurn[i], for i=0..number_lasers_minus1, be non-zero. The arrays DeltaPhi[i] and InvDeltaPhi[i], for i=0..number_lasers_minus1, are derived as follows:

[0093]

number

[0094] planar_buffer_disabled_flag equal to 1 indicates that using a buffer to track the nearest node is not used in the process of coding the planar mode flags and planar positions in planar mode. planar_buffer_disabled_flag equal to 0 indicates that using a buffer to track the nearest node is used. When not present, planar_buffer_disabled_flag is inferred to be !geometry_planar_enabled_flag.

[0095] [Table 2-1] [Table 2-2] [Table 2-3] [Table 2-4]

[0096] The data syntax for the planar and direct modes is contained in Tables 3 and 4, respectively.

[0097] [Table 3-1] [Table 3-2]

[0098] [Table 4]

[0099] 8.2.4.1 Node Angle Eligibility Derivation Process XXX Input / Output If geometry_angular_enabled_flag is equal to 0, angular_eligible is set equal to 0. If not, the following applies: The variable deltaAngle, which specifies the minimum angular separation between the lasers, is derived as follows:

[0100]

number

[0101] Finally, angular_eligible is derived as follows: [Ed, sNchild needs to be verified]

[0102]

number

[0103] 8.2.4.2 Derivation process of the laser index laserIndex associated with a node XXX Input / Output If the angular eligibility angular_eligible is equal to 0, the laserIndex index is set to the preset value UNKOWN_LASER. Otherwise, if the angular eligibility, angular_eligible, is equal to 1, then the following applies as a continuation of the process described in 8.2.5.1: First, the inverse of the radial distance of the current node from the lidar, rInv, is determined as follows:

[0104]

number

[0105] The angle theta32 is then determined as follows:

[0106]

number

[0107] [Ed XXX:laserIndex[Parent] is meaningless, we need to add another state array] Finally, the angle eligibility and associated laser are determined based on the parent node Parent as follows:

[0108]

number

[0109] 8.2.4.3 Derivation process of contextAzimuthalS and contextAzimuthalT for planar coding modes XXX Input / Output As a continuation of the process described in 8.2.5.2 the following applies: First, the two angles are inferred from the node positions relative to the angle origin.

[0110]

number

[0111] Second, the azimuth angle predictor is obtained from the array phiBuffer.

[0112]

number

[0113] Two azimuth contexts are initialized as follows:

[0114]

number

[0115] Then, if the predictor predPhi is not equal to 0x80000000, the following is applied to refine the two azimuth contexts:

[0116]

number

[0117] 8.2.4.4 ContextAngular Derivation Process for Flat Coding Mode XXX Input / Output If the laser index laserIndex is equal to UNKOWN_LASER, then contextAngular is set to the preset value UNKOWN_CONTEXT. Otherwise, if the laser index laserIndex is not equal to UNKOWN_LASER, the following applies as a continuation of the process described in 8.2.5.2: First, two angular differences for the lower and upper planes, thetaLaserDeltaBot and thetaLaserDeltaTop, are determined.

[0118]

number

[0119] The angular context is then inferred from the difference between the two angles.

[0120]

number

[0121]

[0070] Motion Prediction in GPCC. There are two kinds of motion involved in the G-PCC InterEM software (Exploratory Model for Inter Prediction in G-PCC, ISO / IEC JTC1 / SC29 WG11 N18096, Macau, China, October 2018): global motion matrix and local nodal motion vectors. Global motion parameters are defined as the rotation matrix and translation vectors that will be applied to all points in the predicted (reference) frame (except for points to which local motion modes are applied). The local nodal motion vectors of a node of an octree are motion vectors that are applied only to points within the node in the predicted (reference) frame. The details of the motion estimation algorithm in InterEM are described below.

[0122]

[0071] Figure 4 is a flow chart showing an example motion estimation technique for InterEM. Given an input prediction (reference) frame and a current frame, global motion is first estimated at a global scale. After applying global motion to the prediction, local motion is estimated at a finer scale node level in the octree. Finally, the estimated local node motion is applied in motion compensation.

[0123]

[0072] Details of the above techniques are described below.

[0124] Method for estimating global motion matrix and translation vectors. Figure 5 is a flow chart illustrating an example technique for estimating local nodal motion vectors. As shown in Figure 5, the motion vectors are estimated in a recursive manner. The cost function used to select the best suitable motion vector may be based on a rate-distortion cost.

[0125]

[0074] If the current node is not split into 8 children, the motion vector that can produce the lowest cost between the current node and the predicted node is determined. If the current node is split into 8 children, a motion estimation algorithm is applied, and the total cost under the splitting condition is obtained by adding the estimated cost value of each child node. The decision to split or not to split is reached by comparing the cost between splitting and not splitting, and if split, each subnode is assigned its respective motion vector (or can be further split to its children), and if not split, the current node is assigned a motion vector.

[0126]

[0075] Two parameters that affect the performance of motion vector estimation are block size (BlockSize) and minimum prediction unit size (MinPUSize). BlockSize defines the upper limit and MinPUSize defines the lower limit of the node size for applying motion vector estimation.

[0127]

[0076] The techniques described above may present one or more drawbacks. The reference block of a node is derived by motion compensation using estimated motion information (rotation and translation). Good estimation of motion information leads to high correlation between the current node and the reference node in terms of geometric structures, such as occupancy, plane information, etc. Thus, utilizing this geometric information of the reference node will improve the coding performance of the current node. According to one or more techniques of the present disclosure, a G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may utilize information of the reference block in coding the plane information of the current node. As an example, the G-PCC coder may utilize information of the reference block in selecting the context when coding the eligibility of the node for the planar coding mode, the plane flag and the plane index. One or more techniques disclosed in this document may be applied independently or in combination.

[0128]

[0077] Planar eligibility of a node using inter prediction. In one example, PlanarRate may be updated by a factor (R) that depends on the planar information of the reference block. In this example, PlanarRate in section 8.2.3.1 in GPCC DIS may be specified as follows (additions are shown in bold italics): The occupancy information of the parent nodes is used to determine the presence of a single occupied plane along each axis and to update the corresponding plane probability estimate, PlanarRate[k].

[0129]

number

[0130] where R[k] is a scaling factor that depends on whether the reference block is planar in the kth direction. In one example, if the reference block is planar in the kth direction, R[k] may be set higher than 1. Otherwise, R[k] may be set lower than 1.

[0131] In another example, PlanarEligible may not be allowed if the reference block is not planar. Then, PlanarEligible is set as follows:

[0132]

number

[0133] Alternatively, the reference planar rate (PlanarRateRef) may be determined separately based on the planarity of the reference block. Furthermore, the eligibility of the current node for the planar mode may be determined based on comparing the reference planar rate to a threshold. For example:

[0080] Then, PlanarEligible is set as follows:

[0134]

number

[0135] In some cases, the plane index position is derived based on the plane index position of the reference node. In some cases, this determination may be based on a threshold comparison.

[0136]

[0082] Plane Copy Mode (PCM). A G-PCC coder may signal a flag (PCM flag) in the bitstream to indicate whether the current node and the reference node share the same plane mode in all directions (e.g., k=0..2). When this flag is 1, the decoder may not need to decode the plane flag for each direction, and the decoder uses only the corresponding value in the reference node (e.g., the plane flag may be copied from the reference node). When the flag is 0, the plane flag in each direction may be signaled.

[0137]

[0083] In a further level PCM flag, it may also indicate whether the current node and the reference node share a plane position index. In this example, if the PCM flag is 1, in the kth direction, the decoder does not need to decode the plane position index, and the decoder may use the plane index in the reference block in the same kth direction.

[0138] In some examples, the PCM flag may be conditionally signaled. For example, the occupancy of a reference block is 0 and the PCM flag may not be signaled and may be implicitly set to be 0. In some examples, a flag in the slice header or SPS header may be defined to activate or deactivate the PCM mode.

[0139]

[0085] Context selection in signaling the planar flag using inter prediction. There may be three contexts for signaling the planar flag (is_planar_flag). The index selection is simply selected by the direction index (axisIdx). According to one or more techniques of this disclosure, the context for encoding the planar flag may be extended using the planar mode of the reference node. One example of the extension may be described as follows: The inputs to this process are:

[0140] - a variable childIdx that identifies a child of the current node, - a variable axisIdx that identifies the axis perpendicular to the plane, and - The position (sN,tN,vN) of the current node within the geometry tree level. The output of this process is the variable ctxIdx. The value of ctxIdx is set equal to (2*axisIdx+PlanarModeRef[axisIdx]) and no further processing is performed.

[0141] In this example, PlanarModeRef[axisIdx] indicates whether the reference node is planar in the axisIdx direction.

[0142]

[0087] CtxIdx Determination for Syntax Element plane_position Using Inter Mode: In GPCC DIS, the context index used to code plane_position is a function of axisIdx, a plane position prediction based on neighborhood occupancy, and a buffer lookup to determine the closest already coded node at the same buffer row index including isPlanar, planePosition, and a distance measure, as follows (Section 8.2.3.3): The context index ctxIdx is derived as follows.

[0143]

number

[0144]

[0088] In one example of the present disclosure, the occupancy and planar mode of the reference block may be used as additional parameters for determining the context index for planar position coding.

[0145]

[0089] Let PlanarModeRef and RefPlane be the planar mode and plane position in the reference block.

[0146]

[0090] If PlanarModeRef[axisIdx] is 0, then RefPlane[axisIdx] is set equal to -1.

[0147] In one example, RefPlane[axisIdx] may be used to replace prevPlane.

[0148]

number

[0149] In another example, ctxIdx may be updated as follows:

[0150]

number

[0151] where N is the number of contexts supported using only axisIdx, adjPlaneCtxInc, distCtxInc, and prevPlane. In the current draft GPCC DIS, N is 36.

[0152]

[0093] In another example, the occupancy and planar position of a reference block may be used to replace neighboring occupancies for deriving context.

[0153]

[0094] In yet another example, the context index for the planar flag may be derived using only the directional index and planar mode of the reference block, as follows:

[0154]

number

[0155]

[0095] Derivation process of context contextAngular for planar coding mode using inter prediction. Let M be the number of contexts supported for contextAngular to code planar position. After contextAngular is derived as in section 3.3.5, it can be updated using the planar mode and planar position of the reference block.

[0156]

[0096] In one example, the contextAngular in section 3.3.5 can be updated as follows:

[0157]

[0097] In some cases, if PlanarModeRef[axisIdx] is 0, then RefPlane[axisIdx] is set equal to -1. contextAngular values ​​may be assigned as follows: The angular context is then inferred from the difference between the two angles.

[0158]

number

[0159]

[0098] In another example, if PlanarModeRef[axisIdx] is 0, then RefPlane[axisIdx] is set equal to 0.

[0160]

[0099] The contextAngular value can be assigned as follows: The angular context is then inferred from the difference between the two angles.

[0161]

number

[0162]

[0100] Derivation process of contexts contextAzimuthalS and contextAzimuthalT combined with inter prediction for planar coding mode.

[0163]

[0101] After the contexts contextAzimuthalS and contextAzimuthalT are derived in 8.2.4.3, they can be updated with the use of the planar mode of the inter reference blocks.

[0164]

[0102] In some cases, RefPlane[axisIdx] is set equal to -1 if PlanarModeRef[axisIdx] is 0. The following modifications may be made:

[0165]

number

[0166]

[0103] In some cases, if PlanarModeRef[axisIdx] is 0, then RefPlane[axisIdx] is set equal to 0. The following modifications may be made:

[0167]

number

[0168]

[0104] Context-Occupancy Coding with Inter Prediction.

[0169]

[0105] In the reference software for InterEM, the context derivation for the occupancy bits is derived as follows:

[0170]

number

[0171]

[0106] In the above calculation, the context associated with inter prediction (ctxInter) is the sum of !!mappedPred, bitPred, and bitPredStrong. According to one or more techniques of this disclosure, ctxIdxMapIdx may be modified as follows:

[0172]

number

[0173] Motion-Based Threshold: A threshold may be defined based on motion vectors / motion parameters, which may be used in one or more of the decisions disclosed in this document.

[0174]

[0108] For example, the threshold may be determined based on a magnitude / parameter associated with the rotation (e.g., the angle of the rotation) or the translation (e.g., the magnitude of the translation), where if the angle associated with the rotation is x and the magnitude of the translation is y, then the threshold may be derived as a function of x and y (e.g., a linear combination a*x+b*y, where a and b are fixed values).

[0175]

[0109] In other alternatives, the thresholds associated with each axis may be derived separately; for example, the translation associated with an axis may be used to derive the thresholds associated with the axis.

[0176]

[0110] Based on a threshold, one or more decisions may be made. For example, if a point is associated with zero motion, a threshold associated with the zero motion may be used to determine plane eligibility of the node (as described above with respect to PCM), and when the motion associated with the point is greater, a different threshold may be used for the plane eligibility determination.

[0177]

[0111] In another example, when a threshold (or motion parameter) exceeds a certain fixed value, one or more of the decisions disclosed in this document may be overridden.

[0178]

[0112] For different decisions, different thresholds can be used, which can also be signaled in the bitstream.

[0179]

[0113] The examples in the various aspects of the present disclosure may be used individually or in any combination.

[0180] FIG. 6 is a conceptual diagram illustrating an example range measurement system 700 that may be used with one or more techniques of the present disclosure. In the example of FIG. 6, the range measurement system 700 includes an illuminator 702 and a sensor 704. The illuminator 702 may emit light 706. In some examples, the illuminator 702 may emit the light 706 as one or more laser beams. The light 706 may be at one or more wavelengths, such as infrared wavelengths or visible light wavelengths. In other examples, the light 706 is not a coherent laser light. When the light 706 encounters an object, such as an object 708, the light 706 results in return light 710. The return light 710 may include backscattered light and / or reflected light. The return light 710 may pass through a lens 711 that directs the return light 710 to result in an image 712 of the object 708 on the sensor 704. The sensor 704 generates a signal 714 based on the image 712. Image 712 may comprise a set of points (eg, represented by dots in image 712 of FIG. 6).

[0181] In some examples, the illuminator 702 and sensor 704 may be mounted on a spinning structure (e.g., a spinning LIDAR sensor) such that the illuminator 702 and sensor 704 capture a 360-degree view of the environment. In other examples, the range measurement system 700 may include one or more optical components (e.g., mirrors, collimators, diffraction gratings, etc.) that enable the illuminator 702 and sensor 704 to detect the range of an object within a particular range (e.g., up to 360 degrees). Although the example of FIG. 6 shows only a single illuminator 702 and sensor 704, the range measurement system 700 may include multiple sets of illuminators and sensors.

[0182]

[0116] In some examples, the illuminator 702 generates a structured light pattern. In such examples, the range measurement system 700 may include multiple sensors 704 on which respective images of the structured light pattern are formed. The range measurement system 700 may use the parallax between the images of the structured light pattern to determine the distance to an object 708 from which the structured light pattern is backscattered. The structured light based range measurement system may have a high level of accuracy (e.g., sub-millimeter range accuracy) when the object 708 is relatively close (e.g., 0.2 meters to 2 meters) to the sensor 704. This high level of accuracy may be useful in facial recognition applications such as unlocking a mobile device (e.g., mobile phone, tablet computer, etc.) and for security applications.

[0183]

[0117] In some examples, the range measurement system 700 is a time-of-flight (ToF) based system. In some examples where the range measurement system 700 is a ToF based system, the illuminator 702 generates a pulse of light. In other words, the illuminator 702 may modulate the amplitude of the emitted light 706. In such examples, the sensor 704 detects return light 710 from the pulse of light 706 generated by the illuminator 702. The range measurement system 700 may then determine the distance to the object 708 from which the light 706 is backscattered based on the delay between when the light 706 is emitted and detected and the known speed of light in air). In some examples, instead of (or in addition to) modulating the amplitude of the emitted light 706, the illuminator 702 may modulate the phase of the emitted light 706. In such an example, the sensor 704 may detect the phase of the returning light 710 from the object 708 and determine the distance to a point on the object 708 using the speed of light and based on the time difference between when the illuminator 702 generates the light 706 at a particular phase and when the sensor 704 detects the returning light 710 at a particular phase.

[0184]

[0118] In other examples, the point cloud may be generated without the use of the illuminator 702. For example, in some examples, the sensor 704 of the range measurement system 700 may include two or more optical cameras. In such examples, the range measurement system 700 may use the optical cameras to capture stereo images of an environment including the object 708. The range measurement system 700 may include a point cloud generator 716 that may calculate disparity between locations in the stereo images. The range measurement system 700 may then use the disparity to determine distances to locations shown in the stereo images. From these distances, the point cloud generator 716 may generate a point cloud.

[0185]

[0119] The sensor 704 may also detect other attributes of the object 708, such as color and reflectance information. In the example of Figure 6, the point cloud generator 716 may generate a point cloud based on the signal 714 generated by the sensor 704. The range measurement system 700 and / or the point cloud generator 716 may form part of the data source 104 (Figure 1). Thus, the point cloud generated by the range measurement system 700 may be encoded and / or decoded according to any of the techniques of this disclosure.

[0186] FIG. 7 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of the present disclosure may be used. In the example of FIG. 7, a vehicle 800 includes a range measurement system 802. The range measurement system 802 may be implemented in the manner described with respect to FIG. 107. Although not shown in the example of FIG. 7, the vehicle 800 may also include a data source, such as the data source 104 (FIG. 1), and a G-PCC encoder, such as the G-PCC encoder 200 (FIG. 1). In the example of FIG. 7, the range measurement system 802 emits a laser beam 804 that reflects off a pedestrian 806 or other object in the road. The data source of the vehicle 800 may generate a point cloud based on the signal generated by the range measurement system 802. The G-PCC encoder of the vehicle 800 may encode the point cloud to generate a bit stream 808, such as a geometry bit stream (FIG. 2) and an attribute bit stream (FIG. 2). The bit stream 808 may include many fewer bits than the unencoded point cloud obtained by the G-PCC encoder.

[0187]

[0121] An output interface of the vehicle 800 (e.g., output interface 108 (FIG. 1)) may transmit the bitstream 808 to one or more other devices. The bitstream 808 may include many fewer bits than the unencoded point cloud obtained by the G-PCC encoder. Thus, the vehicle 800 may be able to transmit the bitstream 808 to other devices more quickly than the unencoded point cloud data. Furthermore, the bitstream 808 may require less data storage capacity.

[0188]

[0122] In the example of FIG. 7, the vehicle 800 may transmit a bit stream 808 to another vehicle 810. The vehicle 810 may include a G-PCC decoder, such as the G-PCC decoder 300 (FIG. 1). The G-PCC decoder of the vehicle 810 may decode the bit stream 808 to reconstruct a point cloud. The vehicle 810 may use the reconstructed point cloud for various purposes. For example, the vehicle 810 may determine based on the reconstructed point cloud that a pedestrian 806 is on the road ahead of the vehicle 800 and therefore, for example, to begin decelerating even before the driver of the vehicle 810 realizes that the pedestrian 806 is on the road. Thus, in some examples, the vehicle 810 may perform an autonomous navigation operation based on the reconstructed point cloud.

[0189] Additionally or alternatively, vehicle 800 may transmit bitstream 808 to server system 812. Server system 812 may use bitstream 808 for various purposes. For example, server system 812 may store bitstream 808 for subsequent reconstruction of a point cloud. In this example, server system 812 may use the point cloud along with other data (e.g., vehicle telemetry data generated by vehicle 800) to train an autonomous driving system. In another example, server system 812 may store bitstream 808 for subsequent reconstruction for forensic accident investigation.

[0190] FIG. 8 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of the present disclosure may be used. Extended reality (XR) is a term used to cover a range of technologies including augmented reality (AR), mixed reality (MR), and virtual reality (VR). In the example of FIG. 8, a user 900 is located at a first location 902. The user 900 wears an XR headset 904. As an alternative to the XR headset 904, the user 900 may use a mobile device (e.g., a mobile phone, a tablet computer, etc.). The XR headset 904 includes a depth detection sensor, such as a range measurement system, that detects the position of a point on an object 906 at the location 902. A data source of the XR headset 904 may use a signal generated by the depth detection sensor to generate a point cloud representation of the object 906 at the location 902. The XR headset 904 may include a G-PCC encoder (e.g., the G-PCC encoder 200 of FIG. 1) configured to encode the point cloud to generate a bitstream 908.

[0191]

[0125] The XR headset 904 may transmit the bitstream 908 (e.g., over a network such as the Internet) to an XR headset 910 worn by a user 912 at a second location 914. The XR headset 910 may decode the bitstream 908 to reconstruct a point cloud. The XR headset 910 may use the point cloud to generate an XR visualization (e.g., an AR, MR, VR visualization) representing the object 906 at the location 902. Thus, in some examples, the user 912 may have a 3D immersive experience of the location 902, such as when the XR headset 910 generates a VR visualization. In some examples, the XR headset 910 may determine the position of a virtual object based on the reconstructed point cloud. For example, the XR headset 910 may determine that an environment (e.g., location 902) includes a flat surface based on the reconstructed point cloud, and then determine that a virtual object (e.g., a cartoon character) should be placed on the flat surface. The XR headset 910 may generate an XR visualization of the virtual object in the determined position. For example, the XR headset 910 may show a cartoon character sitting on a flat surface.

[0192]

[0126] Figure 9 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of the present disclosure may be used. In the example of Figure 9, a mobile device 1000 (e.g., a wireless communication device), such as a mobile phone or tablet computer, includes a range measurement system, such as a LIDAR system, that detects the position of points on an object 1002 in the environment of the mobile device 1000. A data source of the mobile device 1000 may use signals generated by a depth detection sensor to generate a point cloud representation of the object 1002. The mobile device 1000 may include a G-PCC encoder (e.g., the G-PCC encoder 200 of Figure 1) configured to encode the point cloud to generate a bitstream 1004. In the example of Figure 9, the mobile device 1000 may transmit the bitstream to a remote device 1006, such as a server system or another mobile device. The remote device 1006 may decode the bitstream 1004 to reconstruct the point cloud. The remote device 1006 may use the point cloud for various purposes. For example, the remote device 1006 may use the point cloud to generate a map of the environment of the mobile device 1000. For example, the remote device 1006 may generate a map of the interior of a building based on the reconstructed point cloud. In another example, the remote device 1006 may generate imagery (e.g., computer graphics) based on the point cloud. For example, the remote device 1006 may use the points of the point cloud as vertices of a polygon and use color attributes of the points as a basis for shading the polygon. In some examples, the remote device 1006 may use the reconstructed point cloud for facial recognition or other security applications.

[0193]

[0127] Figures 10A and 10B show an example of this process for bin n. In example 201 of Figure 10A, given a certain context state (σ), the range in bin n is determined by the LPS (p σ) probability. Example 201 illustrates updating the range in bin n+1 when the value of bin n is equal to MPS. In this example, the low remains the same, but the value of range in bin n+1 is reduced to the value of RangeMPS in bin n. Example 203 in FIG. 10B illustrates updating the range in bin n+1 when the value of bin n is not equal to MPS (i.e., it is equal to LPS). In this example, the low is moved to the lower range value of RangeLPS in bin n. Additionally, the value of range in bin n+1 is reduced to the value of RangeLPS in bin n.

[0194]

[0128] In some examples, the range may be represented by 9 bits and the low by 10 bits. There is a renormalization process to keep the range and low values ​​with sufficient accuracy. Whenever the range is smaller than 256, renormalization is performed. Thus, the range is always equal to or greater than 256 after renormalization. Depending on the range and low values, the BAC outputs a "0" or a "1" to the bitstream, or updates an internal variable (called BO: bits-outstanding) to keep for future output. Figure 11 shows an example of BAC output depending on the range. For example, when the range and low are above a certain threshold (e.g., 512), a "1" is output to the bitstream. When the range and low are below a certain threshold (e.g., 512), a "0" is output to the bitstream. When the range and low are between certain thresholds, nothing is output to the bitstream. Instead, the BO value is incremented and the next bin is coded.

[0195]

[0129] As explained above, arithmetic coding methods can be used to provide high compression efficiency. This can be achieved by first converting non-binary syntax elements to binary representations (e.g., 0, 1) using a process called binarization. The resulting converted entries are called bins or bin strings. These bins or bin strings are then fed to the arithmetic coding process. Figure 11 shows an exemplary context-adaptive binary arithmetic coding (CABAC) encoding stage. The exemplary CABAC encoding stage can be implemented in a G-PCC encoder, such as by arithmetic coding unit 214 and / or arithmetic coding unit 226 of G-PCC encoder 200 of Figure 2.

[0196]

[0130] In some examples of G-PCC, context-adaptive binary arithmetic coding (CABAC) may be used to generate bins through the binarization process. For each coded bin value, an appropriate context model is selected. These context models are used to code each bin value into an output bit based on the bin probability value. The CABAC engine bypasses context modeling and bin coding when a bin is as likely to be 0 or 1. This is the bypass coding stage described below. Otherwise, an appropriate context model is specified when the bin value is coded, modeling based on the probability of the bin value. The context is adapted as the encoder codes more bins. Finally, the context-coded bin values ​​or the raw bitstream are transmitted or otherwise provided to the decoder.

[0197] FIG. 12 is a block diagram of an example arithmetic coding unit 214 that may be configured to perform CABAC according to the techniques of this disclosure. A syntax element 1180 is input to the arithmetic coding unit 214. If the syntax element is already a binary-valued syntax element (e.g., a flag or other syntax element that has only values ​​of 0 and 1), the binarization step may be skipped. If the syntax element is a non-binary-valued syntax element (e.g., a syntax element that may have values ​​other than 1 or 0), the non-binary-valued syntax element is binarized by the binarizer 1200. The binarizer 1200 performs a mapping of the non-binary-valued syntax element to a sequence of binary decisions. These binary decisions are often called "bins." For example, at a transform coefficient level, the value of the level may be divided into consecutive bins, each bin indicating whether the absolute value of the coefficient level is greater than a certain value. For example, bin 0 (sometimes called a significance flag) indicates whether the absolute value of the transform coefficient level is greater than 0. Bin 1 indicates whether the absolute value of the transform coefficient level is greater than 1, etc. For each non-binary value syntax element, a unique mapping may be created.

[0198]

[0132] Each bin generated by the binarizer 1200 is fed to the binary arithmetic coding side of the arithmetic coding unit 214. That is, for a given set of non-binary valued syntax elements, each bin type (e.g., bin 0) is coded before the next bin type (e.g., bin 1). The coding can be performed in either normal mode or bypass mode. In bypass mode, the bypass coding engine 1260 performs arithmetic coding using a fixed probability model, for example, using Golomb-Rice or Exponential-Golomb coding. The bypass mode is generally used for more predictable syntax elements.

[0199]

[0133] Coding in normal mode involves performing CABAC. Normal mode CABAC is for coding bin values ​​where the probability of the bin's value is predictable given the previously coded bin's value. The probability that the bin is an LPS is determined by the context modeler 1220. The context modeler 1220 outputs the bin value and a probability state for the context (e.g., a probability state σ that includes the value of the LPS and the probability that the LPS occurs). The context may be an initial context for the series of bins or may be determined based on the coded values ​​of the previously coded bins. An identity of the context may be represented and / or determined based on the value of the variable ctxInc (a context increment, such as a value of ctxInc that represents the increment to be applied to the previous context). As described above, the context modeler 1220 may update the state based on whether the received bin was an MPS or an LPS. After the context and the probability state σ are determined by the context modeler 1220, the normal coding engine 1240 performs BAC on the bin values.

[0200]

[0134] Figure 13 is a block diagram of an example arithmetic decoding unit 302 that may be configured to perform CABAC according to the techniques of this disclosure. The arithmetic decoding unit 302 of Figure 13 performs CABAC in a manner opposite to that of the arithmetic encoding unit 214 described in Figure 12. Coded bits from the bitstream 2180 are input to the arithmetic decoding unit 302. The coded bits are provided to either the context modeler 2200 or the bypass decoding engine 2220 based on whether they were entropy coded using a normal mode or a bypass mode. If the coded bits were coded in a bypass mode, the bypass decoding engine will use, for example, Golomb-Rice or exponential-Golomb decoding to extract the bins of the binary value syntax elements or non-binary syntax elements.

[0201]

[0135] If the coded bits are coded in normal mode, the context modeler 2200 may determine a probability model for the coded bits, and the normal decoding engine 2240 may decode the coded bits to generate bins of the non-binary valued syntax element (or the syntax element itself, in case of binary value). After the context and probability state σ are determined by the context modeler 2200, the normal decoding engine 2240 performs BAC to decode the bin values. In other words, the normal decoding engine 2240 may determine the probability state of the context and decode the bin value based on the previously coded bin and the current range. After decoding the bin, the context modeler 2200 may update the probability state of the context based on the window size and the value of the decoded bin.

[0202]

[0136] Figure 14 is a flow diagram illustrating an example technique for predicting points of a point cloud according to one or more aspects of the present disclosure. The technique of Figure 14 may be implemented by a G-PCC coder, such as the G-PCC encoder 200 of Figure 2. However, other devices may implement the technique of Figure 14, such as the G-PCC decoder 300 of Figure 3.

[0203]

[0137] The G-PCC encoder 200 may obtain planar information of a reference block of a point cloud (1402). For example, the arithmetic coding unit 214 of the G-PCC encoder 200 may determine whether the reference block is coded using a planar mode in a particular direction (e.g., PlanarModeRef[axisIdx], which may indicate whether the reference block / node is planar in the axisIdx direction).

[0204]

[0138] The G-PCC encoder 200 may determine a context based on the plane information of the reference block (1404). For example, the arithmetic coding unit 214 may determine a context index (ctxIdx) based on the plane information of the reference block. As an example, the arithmetic coding unit 214 may determine ctxIdx as (2*axisIdx+PlanarModeRef[axisIdx]).

[0205]

[0139] The G-PCC encoder 200 may context-adaptively code a syntax element indicating whether the current node is coded using a planar mode based on the context (1406). For example, the arithmetic coding unit 214 may perform context-adaptive binary arithmetic coding (CABAC) of the is_planar_flag syntax element for the current node based on ctxIdx. As described above, an is_planar_flag syntax element equal to 1 may indicate that the positions of the children of the current node form a single plane perpendicular to the axisIdx-th axis. An is_planar_flag[axisIdx] equal to 0, when present, may indicate that the positions of the children of the current node occupy both planes perpendicular to the axisIdx-th axis.

[0206]

[0140] The G-PCC encoder 200 may code the current node using a planar mode based on the current node being coded using the planar mode (1408). For example, the G-PCC encoder 200 may code the children of the current node as forming a single plane.

[0207] In some examples, when the current node is coded using a planar mode, the arithmetic coding unit 214 may determine a second context based on the reference plane and context-adaptively code a syntax element indicating a plane for the current node based on the second context. The syntax element indicating a plane for the current node may be a plane_position syntax element. In some examples, to determine the second context based on the reference plane, the arithmetic coding unit 214 may determine a context index according to the following formula: ctxIdx=(12×axisIdx+4×adjPlaneCtxInc+2×distCtxInc+prevPlane+3)+(RefPlane[axisIdx]+1)×N, where ctxIdx is the context index, axisIdx is the axis index, adjPlaneCtxInc is the adjusted plane context increment, distCtxInc is the distance context increment, prevPlane is the previous plane, and RefPlane[axisIdx] is the reference plane.

[0208] In some examples, the arithmetic coding unit 214 may determine an angular context for the current node based on a reference plane, and may determine a plane for the current node based on the angular context. To code the current node using the planar mode, the G-PCC encoder 200 may code the current node based on the plane.

[0209]

[0143] In some examples, the arithmetic coding unit 214 may determine an azimuth context for the current node based on a reference plane, and determine a plane for the current node based on the azimuth context. To code the current node using a plane mode, the G-PCC encoder 200 may code the current node based on a plane. In some examples, to determine the azimuth context, the arithmetic coding unit 214 may determine the azimuth context according to the following formula: contextAzimuthal=contextAnglePhi+8×(RefPlane[axisIdx]+1), where contextAzimuthal is the azimuth context, contextAnglePhi is an intermediate value used to derive the azimuth context, and RefPlane[axisIdx] is the reference plane. Multiple azimuth contexts may be derived based on contextAnglePhi, including contextAzimuthalS and contextAzimuthalT.

[0210]

[0144] In some examples, the current node may be selectively coded using a planar copy mode (PCM). For example, the G-PCC encoder 200 may determine whether to copy planar information for the current node from the reference node. The G-PCC encoder 200 may signal whether the current node is coded using PCM. For example, the arithmetic coding unit 214 may code a syntax element, such as a binary flag (e.g., PCM_flag), indicating whether the current node is coded using the planar copy mode. If the current node is coded using the planar copy mode, the G-PCC decoder 300 may copy the planar information of the current node from the reference node. For example, the G-PCC decoder 300 may utilize the planar position of the reference node as the planar position of the current node. Similarly, if the current node is not coded using the planar copy mode, the G-PCC encoder 200 may encode (and the G-PCC decoder 300 may decode) the planar information for the current node from the bitstream. In this manner, PCM may improve coding efficiency.

[0211]

[0145] The following numbered clauses may describe one or more aspects of the disclosure.

[0212]

[0146] Clause 1A. A method for coding point cloud data, the method comprising: obtaining plane information of a reference block of the point cloud data; and coding a current block of the point cloud data based on the obtained plane information.

[0213]

[0147] Clause 2A. The method of clause 1A, wherein coding a current block of point cloud data comprises determining a planar rate for a current orientation of the current block based at least in part on whether the reference block is coded using a planar mode in the current orientation.

[0214]

[0148] Clause 3A. The method of clause 1A, wherein coding the current block of point cloud data comprises determining whether the current block is plane eligible based on whether the reference block is planar.

[0215]

[0149] Clause 4A. The method of clause 1A, further comprising coding, in the coded bitstream, a syntax element having a value indicating whether the current block and the reference block share the same planar mode in all directions.

[0216]

[0150] Clause 5A. The method of clause 1A, further comprising coding, in the coded bitstream, a syntax element having a value indicating whether the current block and the reference block share the same plane position index.

[0217]

[0151] Clause 5A. The method of clause 1A, wherein coding the current block of point cloud data comprises determining a context for context-adaptive coding of a planar flag of the current block based on whether the reference block is planar.

[0218]

[0152] Clause 6A. The method of clause 1A, wherein coding the current block of point cloud data comprises determining an angular context for the current block based on a planar mode of a planar position of the reference block.

[0219]

[0153] Clause 7A. A device for processing a point cloud, the device comprising one or more means for performing the method described in any of clauses 1A to 6A.

[0220]

[0154] Clause 8A. The device of clause 7A, wherein the one or more means comprises one or more processors implemented in circuitry.

[0221]

[0155] Clause 9A. A device as described in any of clauses 7A or 8A, further comprising a memory for storing data representing the point cloud.

[0222]

[0156] Clause 10A. A device according to any one of clauses 7A to 9A, wherein the device comprises a decoder.

[0223]

[0157] Clause 11A. A device according to any one of clauses 7A to 10A, wherein the device comprises an encoder.

[0224]

[0158] Clause 12A. A device as described in any of clauses 7A to 11A, further comprising a device for generating a point cloud.

[0225]

[0159] Clause 13A. A device as described in any of clauses 7A to 12A, further comprising a display for presenting an image based on the point cloud.

[0226]

[0160] Clause 14A. A computer-readable storage medium having instructions stored thereon which, when executed, cause one or more processors to perform a method according to any of clauses 1A to 6A.

[0227]

[0161] It should be appreciated that, depending on the example, some acts or events of any of the techniques described herein may be performed in a different sequence, added, merged, or omitted entirely (e.g., not all described acts or events may be required to practice the techniques). Moreover, in some examples, acts or events may be performed simultaneously rather than sequentially, for example, through multithreaded processing, interrupt processing, or multiple processors.

[0228]

[0162] In one or more examples, the described functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or may include a communication medium, which includes any medium that facilitates transfer of a computer program from one place to another, for example according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0229]

[0163] By way of example and not limitation, such computer-readable storage media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if the instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks typically reproduce data magnetically and discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer readable media.

[0230]

[0164] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the terms "processor" and "processing circuitry" as used herein may refer to any of the above structures, or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functions described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a composite codec. Also, the techniques may be fully implemented in one or more circuits or logic elements.

[0231]

[0165] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs) or sets of ICs (e.g., chipsets). In this disclosure, various components, modules, or units have been described to highlight functional aspects of devices configured to implement the disclosed techniques, but those components, modules, or units do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors described above, along with suitable software and / or firmware.

[0232]

[0166] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. 1. A device for processing a point cloud, comprising: a memory configured to store at least a portion of the point cloud; one or more processors implemented in the circuit; wherein the one or more processors: Obtaining plane information of a reference node of the point cloud; determining a context based on the planarity information of the reference node; context-adaptively coding a syntax element indicating whether a current node of the point cloud is coded using a planar mode based on the context; coding the current node using the planar mode based on the current node being coded using the planar mode; and A device configured to:

2. The device of claim 1 , wherein the syntax element indicating whether the current node is coded using the planar mode comprises an is_planar_flag syntax element.

3. the context is a first context, wherein the one or more processors: In response to determining that the current node is coded using the planar mode, determining a second context based on a reference plane; context-adaptively coding a syntax element indicating a plane for the current node based on the second context; [0023] 20. The method according to claim 1, further comprising: wherein, to code the current node using the planar mode, the one or more processors are configured to code the current node based on the planar mode. The device of claim 1 .

4. The device of claim 3 , wherein the syntax element indicating the plane for the current node comprises a plane_position syntax element.

5. To determine the second context based on the reference plane, the one or more processors are configured to determine a context index according to the following formula: [0010] where ctxIdx is the context index, axisIdx is the axis index, adjPlaneCtxInc is the adjusted plane context increment, distCtxInc is the distance context increment, prevPlane is the previous plane, RefPlane[axisIdx] is the reference plane, and N is the number of contexts supported using only axisIdx, adjPlaneCtxInc, distCtxInc, and prevPlane. The device of claim 3.

6. the one or more processors: determining an angular context for the current node based on a reference plane; determining a plane for the current node based on the angular context; wherein, to code the current node using the planar mode, the one or more processors are configured to code the current node based on the planar mode. The device of claim 1 .

7. The one or more processors: determining an azimuth context for the current node based on a reference plane; determining a plane for the current node based on the azimuth context; wherein, to code the current node using the planar mode, the one or more processors are configured to code the current node based on the planar mode. The device of claim 1 .

8. To determine the azimuth context, the one or more processors are configured to determine the azimuth context according to the following equation: [0025] where contextAzimuthal is the azimuth angle context, contextAnglePhi is an intermediate value used to derive the azimuth angle context, and RefPlane[axisIdx] is the reference plane. The device of claim 7.

9. The one or more processors: coding a syntax element indicating whether the current node is coded using a plane copy mode; if the current node is coded using the plane copy mode, copying plane information of the current node from the reference node; The device of claim 1 , further configured to:

10. To copy the plane information, the one or more processors: Use the planar position of the reference node as the planar position of the current node The device according to claim 9, configured so as to

11. The device of claim 9 , wherein the syntax element indicating whether the current node is coded using the planar copy mode comprises a binary flag.

12. Spinning LIDAR Sensor 2. The device of claim 1 , further comprising:

13. The device of claim 1 , wherein the device is a wireless communication device.

14. 1. A method for coding point cloud data, the method comprising: Obtaining plane information of a reference node of the point cloud; determining a context based on the planarity information of the reference node; context-adaptively coding a syntax element indicating whether a current node of the point cloud is coded using a planar mode based on the context; coding the current node using the planar mode based on the current node being coded using the planar mode; and A method comprising:

15. When executed, the program causes one or more processors to Obtaining plane information of a reference node of the point cloud; determining a context based on the planarity information of the reference node; context-adaptively coding a syntax element indicating whether a current node of the point cloud is coded using a planar mode based on the context; coding the current node using the planar mode based on the current node being coded using the planar mode; and A computer-readable storage medium storing instructions to cause a

Images