Occupancy coding with dynamic-based optimal binary decoder (OBUF) occupancy coding using inter prediction in geometry-based point cloud compression
By determining the occupancy of the same-position child nodes in the reference frame in point cloud compression, the context selection of inter prediction blocks is improved, and the encoding efficiency of point cloud data is improved by using dynamic OBUF and inter prediction technology, and the existing technology is solved The problem of low point cloud data encoding efficiency.
Patent Information
- Application Number
- CN202380030680.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-04-13
- Filing Date
- 2023-04-14
- Publication Date
- 2024-11-12
AI Technical Summary
In the prior art, in point cloud compression, the context selection accuracy of inter prediction blocks is suboptimal, resulting in low point cloud data encoding efficiency, especially in octave geometry decoding.
By determining the occupancy of the same-position child nodes in the reference frame, and determining the context of the occupancy bits of the current child node based on it, arithmetic decoding is improved using dynamic OBUF and inter-frame prediction to improve the encoding efficiency of point cloud data.
The encoding efficiency of point cloud data is improved, especially in the occupancy decoding of octree geometry, which enhances the decoding performance of occupancy-related syntax elements, reduces the amount of data, and preserves the geometric structure of point clouds.
Smart Images

Figure CN118946905A_ABST
Abstract
Description
[0001] This application claims priority to U.S. Patent Application No. 18 / 300,023, filed on April 13, 2023, which claims the benefit of U.S. Provisional Patent Application No. 63 / 331,175, filed on April 14, 2022, and U.S. Provisional Patent Application No. 63 / 363,160, filed on April 18, 2022, each of which is incorporated herein by reference in its entirety. U.S. Patent Application No. 18 / 300,023, filed on April 13, 2023, claims the benefit of U.S. Provisional Patent Application No. 63 / 331,175, filed on April 14, 2022, and U.S. Provisional Patent Application No. 63 / 363,160, filed on April 18, 2022. Technical Field
[0002] The present disclosure relates to point cloud encoding and decoding. Background Art
[0003] A point cloud is a collection of points in a 3-dimensional space. A point may correspond to a point on an object in the 3-dimensional space. Therefore, a point cloud may be used to represent the physical content of the 3-dimensional space. Point clouds may be useful in a variety of situations. For example, a point cloud may be used in the context of an autonomous vehicle to represent the location of an object on a road. In another example, a point cloud may be used in the context of representing the physical content of an environment for the purpose of locating virtual objects in augmented reality (AR) or mixed reality (MR) applications. Point cloud compression is a process for encoding and decoding a point cloud. Encoding a point cloud may reduce the amount of data required to store and transmit the point cloud. Summary of the invention
[0004] In summary, the present disclosure describes techniques for geometry-based point cloud compression (G-PCC). In particular, the present disclosure describes techniques that can improve the performance of decoding one or more syntax elements related to occupancy (when also decoding with inter-frame prediction) for G-PCC related codecs (including the next G-PCC standard developed by the 3DG group (WG 7) within MPEG). Typically, the occupancy of an inter-frame predicted block (e.g., a block in another frame) can be used in context selection for occupancy decoding of a current node in an octree geometry. In particular, the proposed techniques of the present disclosure can be applied to octree occupancy decoding based on a dynamic optimal binary decoder (OBUF) with instant updating.
[0005] In a more specific example, the G-PCC decoder may determine the occupancy of a reference subnode that is co-located with the current subnode being decoded in the current frame in the reference frame. The G-PCC decoder may determine a context (e.g., a probability model) for entropy decoding of the occupied bits for the current subnode based on the occupancy of the reference subnode. For inter-predicted frames, the occupancy of the co-located nodes in the reference frame may have a strong correlation with the occupancy of the nodes in the current frame. By using the occupancy information of the reference node in the context selection for the current node, a more accurate context may be selected, thereby improving decoding efficiency when arithmetically decoding occupancy-related syntax elements (e.g., occupied bits).
[0006] In one example, the present disclosure describes a device configured to decode point cloud data, the device comprising: a memory configured to store the point cloud data; and one or more processors in communication with the memory, the one or more processors configured to: determine the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of the point cloud data for inter-frame prediction of a current node in a current frame of the point cloud data; determine a context for decoding a current occupied bit of a current subnode of the current node based on the occupancy of the reference subnode; and use the context to arithmetically decode the current occupied bit.
[0007] In another example, the present disclosure describes a method for decoding point cloud data, the method comprising determining the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of the point cloud data for inter-frame prediction of a current node in a current frame of the point cloud data; determining a context for decoding currently occupied bits of a current subnode of the current node based on the occupancy of the reference subnode; and using the context to arithmetically decode the currently occupied bits.
[0008] In another example, the present disclosure describes a non-temporary computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to decode point cloud data to perform the following operations: determine the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of the point cloud data used for inter-frame prediction of a current node in a current frame of the point cloud data; determine a context for decoding currently occupied bits of a current subnode of the current node based on the occupancy of the reference subnode; and use the context to arithmetically decode the currently occupied bits.
[0009] In another example, the present disclosure describes a device configured to decode point cloud data, the device comprising: a unit for determining occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of the point cloud data for inter-frame prediction of a current node in a current frame of the point cloud data; a unit for determining a context for decoding currently occupied bits of a current subnode of the current node based on the occupancy of the reference subnode; and a unit for arithmetically decoding the currently occupied bits using the context.
[0010] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 is a block diagram illustrating an example encoding and decoding system that may perform the techniques of this disclosure.
[0012] Figure 2 is a block diagram illustrating an example geometry point cloud compression (G-PCC) encoder that may perform techniques of this disclosure.
[0013] Figure 3 is a block diagram illustrating an example G-PCC decoder that may perform the techniques of this disclosure.
[0014] Figure 4 is a flow chart illustrating an example of occupancy coding in G-PCC for use with the techniques of this disclosure.
[0015] Figure 5 is a conceptual diagram illustrating an example of adjacent sub-neighbor context identification for use in conjunction with the techniques of this disclosure.
[0016] Figure 6 is a conceptual diagram illustrating example nodes adjoining child nodes by front faces, edges, or corners for use in conjunction with techniques of this disclosure.
[0017] Figure 7 is a flow chart illustrating an example of dynamic OBUF based occupancy decoding for use with the techniques of this disclosure.
[0018] Figure 8 Example code for dynamic OBUF-based context selection in occupancy coding for use in conjunction with the techniques of this disclosure is shown.
[0019] Fig. 9 is a conceptual diagram illustrating an example of a Word4 construction for use in conjunction with the techniques of the present disclosure.
[0020] Fig.10is a conceptual diagram illustrating example neighbor nodes to obtain Sparse and Word7 for child node 0 for use in conjunction with the techniques of the present disclosure.
[0021] Fig.11 is a flow chart illustrating an example of motion estimation for G-PCC for use with the techniques of this disclosure.
[0022] Fig.12 is a flow chart illustrating an example algorithm for estimating local node motion vectors in a G-PCC for use with the techniques of the present disclosure.
[0023] Fig.13 is a conceptual diagram illustrating an example of context selection based on occupancy of a co-located reference child node for use in conjunction with the techniques of this disclosure.
[0024] Fig.14 is a conceptual diagram illustrating an example Word4 derivation using inter-prediction occupancy for use with the techniques of this disclosure.
[0025] Fig.15 is a flow chart illustrating an example encoding method according to techniques of this disclosure.
[0026] Fig.16 is a flow chart illustrating an example decoding method according to techniques of this disclosure.
[0027] Fig.17 is a conceptual diagram illustrating an example ranging system that may be used with one or more techniques of this disclosure.
[0028] Fig.18 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of this disclosure may be used.
[0029] Fig.19 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of this disclosure may be used.
[0030] Fig. 20 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of this disclosure may be employed. DETAILED DESCRIPTION
[0031] The goal of the geometry-based point cloud compression (G-PCC) technique is to reduce the amount of data required to represent a point cloud while preserving as much of its geometric structure as possible. G-PCC can be used in a variety of applications (such as 3D graphics, virtual reality, and autonomous driving) where efficiently sending or storing large point clouds can be beneficial.
[0032] One technique used in geometry-based point cloud compression is voxelization. Voxelization involves dividing the 3D space into a regular grid of small cubes (e.g., voxels or nodes), each of which represents a small volume of space. Points in the point cloud are then assigned to the voxels they fall into. This voxelized representation can be used to efficiently encode the geometry of the point cloud, since only the voxel coordinates and the points within the voxel need to be sent or stored.
[0033] Occupancy coding is a technique used to further reduce the amount of data required to represent a point cloud. In occupancy coding, a binary value is assigned to each voxel / node to indicate whether it contains any points from the point cloud (occupied) or does not contain any points from the point cloud (unoccupied). The occupancy information is then encoded and sent along with the location of the occupied voxels.
[0034] By using occupancy decoding, the number of points to be sent can be reduced, since only the positions of occupied voxels need to be sent, rather than the positions of all points in the point cloud. However, this technique may result in a loss of detail, since some unoccupied voxels may be discarded, which may affect the overall quality of the reconstructed point cloud.
[0035] To further improve compression efficiency, occupancy coding is often used in conjunction with other compression techniques such as octree-based decomposition and prediction-based coding. Octree-based decomposition involves recursively subdividing a frame of point clouds into octants (half-sized cubes). Prediction-based coding involves predicting the positions of some points based on the positions of neighboring points, and sending only the difference between the predicted position and the actual position. In some examples of G-PCC, prediction-based coding may include inter-frame prediction coding, where a point cloud in a current frame may be decoded relative to a point cloud in another reference frame of point cloud data.
[0036] The present disclosure describes techniques that can improve the performance of decoding one or more syntax elements related to occupancy (when also decoding with inter-frame prediction). Typically, occupancy of an inter-frame predicted block (e.g., a block in another frame) can be used in context selection for occupancy decoding for a current node in an octree geometry. In particular, the proposed techniques of the present disclosure can be applied to octree occupancy decoding based on a dynamic optimal binary decoder (OBUF) with on the fly updates. Previous techniques for context selection for occupancy bits only use information from the current frame of point cloud data. As such, the accuracy of context selection can be suboptimal.
[0037] In a more specific example of the present disclosure, the G-PCC decoder may determine the occupancy of a reference subnode in a reference frame that is collocated with the current subnode being decoded in the current frame. The G-PCC decoder may determine a context (e.g., a probability model) for entropy decoding of the occupied bits for the current subnode based on the occupancy of the reference subnode. For inter-predicted frames, the occupancy of the collocated node in the reference frame may have a strong correlation with the occupancy of the node in the current frame. By using the occupancy information of the reference node in the context selection for the current node, a more accurate context may be selected, thereby improving the decoding efficiency when arithmetic decoding of syntax elements is performed.
[0038] Figure 1 1 is a block diagram illustrating an example encoding and decoding system 100 that can perform the techniques of the present disclosure. The techniques of the present disclosure generally relate to decoding (encoding and / or decoding) point cloud data, i.e., to support point cloud compression. In general, point cloud data includes any data used to process a point cloud. The decoding can be effective in compressing and / or decompressing point cloud data.
[0039] like Figure 1 As shown in FIG. 1 , system 100 includes a source device 102 and a destination device 116. Source device 102 provides encoded point cloud data to be decoded by destination device 116. In particular, in Figure 1 In the example of , source device 102 provides point cloud data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 may include any of a wide variety of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets (such as smart phones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, land or sea vehicles, spacecraft, aircraft, robots, LIDAR devices, satellites, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication.
[0040] exist Figure 1In the example of, source device 102 includes data source 104, memory 106, G-PCC encoder 200 and output interface 108. Destination device 116 includes input interface 122, G-PCC decoder 300, memory 120 and data consumer 118. According to the present disclosure, the G-PCC encoder 200 of source device 102 and the G-PCC decoder 300 of destination device 116 can be configured to apply the technology of the present disclosure. Therefore, source device 102 represents an example of an encoding device, and destination device 116 represents an example of a decoding device. In other examples, source device 102 and destination device 116 may include other components or arrangements. For example, source device 102 can receive data (e.g., point cloud data) from an internal or external source. Similarly, destination device 116 can be docked with an external data consumer instead of including the data consumer in the same device.
[0041] like Figure 1 The system 100 shown in is only an example. Generally, other digital encoding and / or decoding devices can perform the technology of the present disclosure. Source device 102 and destination device 116 are only examples of such devices in which source device 102 generates decoded data for transmission to destination device 116. The present disclosure refers to a "decoding" device as a device that performs decoding (encoding and / or decoding) of data. Therefore, G-PCC encoder 200 and G-PCC decoder 300 represent examples of decoding devices (especially, encoder and decoder, respectively). In some examples, source device 102 and destination device 116 can operate in a substantially symmetrical manner so that each of source device 102 and destination device 116 includes encoding and decoding components. Therefore, system 100 can support one-way or two-way transmission between source device 102 and destination device 116, for example, for streaming, playback, broadcasting, phone, navigation and other applications.
[0042] Typically, the data source 104 represents a source of data (i.e., raw, unencoded point cloud data), and a sequential series of "frames" of data can be provided to the G-PCC encoder 200, which encodes the data for the frames. The data source 104 of the source device 102 can include a point cloud capture device, such as any of a variety of cameras or sensors, for example, a 3D scanner or a light detection and ranging (LIDAR) device, one or more cameras, an archive containing previously captured data, and / or a data feed interface for receiving data from a data content provider. Alternatively or in addition, the point cloud data can be computer generated from a scanner, camera, sensor or other data. For example, the data source 104 can generate computer graphics-based data as source data, or generate a combination of real-time data, archived data and computer-generated data. In each case, the G-PCC encoder 200 encodes captured, pre-captured or computer-generated data. The G-PCC encoder 200 can rearrange the frames from the received order (sometimes referred to as "display order") to a decoding order for decoding. G-PCC encoder 200 may generate one or more bitstreams including the encoded data. Source device 102 may then output the encoded data onto computer-readable medium 110 via output interface 108 for receipt and / or retrieval by input interface 122 of destination device 116, for example.
[0043] The memory 106 of the source device 102 and the memory 120 of the destination device 116 can represent general memory. In some examples, the memory 106 and the memory 120 can store original data, for example, the original data from the data source 104 and the original decoded data from the G-PCC decoder 300. In addition or alternatively, the memory 106 and the memory 120 can store software instructions executable by, for example, the G-PCC encoder 200 and the G-PCC decoder 300, respectively. Although the memory 106 and the memory 120 are shown as being separated from the G-PCC encoder 200 and the G-PCC decoder 300 in this example, it should be understood that the G-PCC encoder 200 and the G-PCC decoder 300 can also include internal memory for functionally similar or equivalent purposes. In addition, the memory 106 and the memory 120 can store, for example, the encoded data output from the G-PCC encoder 200 and input to the G-PCC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, for example, to store raw decoded and / or encoded data.For example, memory 106 and memory 102 may store data representing a point cloud.
[0044] The computer-readable medium 110 may represent any type of medium or device capable of transmitting the encoded data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium so that the source device 102 can directly send the encoded data to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, the output interface 108 can modulate the transmission signal including the encoded data, and the input interface 122 can demodulate the received transmission information. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a portion of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other device that may be useful for facilitating communication from the source device 102 to the destination device 116.
[0045] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.
[0046] In some examples, source device 102 may output the encoded data to file server 114 or another intermediate storage device that may store the encoded data generated by source device 102. Destination device 116 may access the stored data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing the encoded data and sending the encoded data to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded data from file server 114 via any standard data connection (including an Internet connection). This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both that is suitable for accessing the encoded data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a download transmission protocol, or a combination thereof.
[0047] Output interface 108 and input interface 122 may represent wireless transmitters / receivers, modems, wired networking components (e.g., Ethernet cards), wireless communication components that operate according to any of a variety of IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (such as encoded data) according to a cellular communication standard (such as 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data (such as encoded data) according to other wireless standards (such as IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee) TM ),Bluetooth TM Standards, etc.) to transmit data (such as encoded data). In some examples, source device 102 and / or destination device 116 may include respective system-on-chip (SoC) devices. For example, source device 102 may include a SoC device for performing functions attributed to G-PCC encoder 200 and / or output interface 108, and destination device 116 may include a SoC device for performing functions attributed to G-PCC decoder 300 and / or input interface 122.
[0048] The techniques of this disclosure may be applied to encoding and decoding to support any of a variety of applications, such as communications between autonomous vehicles, communications between scanners, cameras, sensors, and processing devices (such as local or remote servers), geographic mapping, or other applications.
[0049] The input interface 122 of the destination device 116 receives an encoded bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded bitstream may include signaling information defined by the G-PCC encoder 200 (which is also used by the G-PCC decoder 300), such as syntax elements with values describing characteristics of decoding units (e.g., slices, pictures, groups of pictures, sequences, etc.) and / or processing of decoding units (e.g., slices, pictures, groups of pictures, sequences, etc.). The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display that presents imagery based on a point cloud.
[0050] G-PCC encoder 200 and G-PCC decoder 300 can each be implemented as any one of a variety of suitable encoder and / or decoder circuit systems, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device can store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute instructions in hardware to perform the technology of the present disclosure. Each of G-PCC encoder 200 and G-PCC decoder 300 can be included in one or more encoders or decoders, and any of the encoders or decoders can be integrated as a part of the combined encoder / decoder (CODEC) in the respective devices. The device including G-PCC encoder 200 and / or G-PCC decoder 300 may include one or more integrated circuits, microprocessors, and / or other types of devices.
[0051] The G-PCC encoder 200 and the G-PCC decoder 300 may operate according to a coding standard, such as the Video Point Cloud Compression (V-PCC) standard or the Geometry Point Cloud Compression (G-PCC) standard. The present disclosure may generally refer to the coding (e.g., encoding and decoding) of a picture as a process that includes encoding or decoding data. The encoded bitstream typically includes a series of values for syntax elements that represent coding decisions (e.g., coding modes).
[0052] The present disclosure may generally relate to "signaling" certain information (such as syntax elements). The term "signaling" may generally refer to the transmission of values for syntax elements and / or other data used to decode encoded data. That is, the G-PCC encoder 200 may signal values for syntax elements in a bitstream. Generally, signaling refers to generating values in a bitstream. As described above, the source device 102 may transmit the bitstream to the destination device 116 substantially in real time or not in real time (such as may occur when storing syntax elements to the storage device 112 for subsequent retrieval by the destination device 116).
[0053] The ISO / IEC MPEG 3DG (JTC 1 / SC 29 / WG 7) is studying the potential need for standardization of point cloud coding techniques with compression capabilities that significantly exceed those of current approaches. The community is working together on this exploratory activity in a collaborative effort called the 3D Graphics Group (3DG) to evaluate compression technology designs proposed by their experts in this field.
[0054] Point cloud compression activities are categorized into two different approaches. The first approach is "Video Point Cloud Compression" (V-PCC), which segments 3D objects and projects the segments in multiple 2D planes (which are represented as "patches" in 2D frames), which are further decoded by a traditional 2D video codec such as the High Efficiency Video Coding (HEVC) (ITU-T H.265) codec. The second approach is "Geometry-based Point Cloud Compression" (G-PCC), which directly compresses 3D geometry, i.e., the positions of a set of points in 3D space, and the associated attribute values (for each point associated with the 3D geometry). G-PCC addresses the compression of point clouds in both category 1 (static point clouds) and category 3 (dynamically acquired point clouds).
[0055] A point cloud contains a collection of points in 3D space and may have attributes associated with the points. The attributes may be color information (such as R, G, B or Y, Cb, Cr), or reflectivity information or other attributes. Point clouds may be captured by various cameras or sensors (such as LIDAR sensors and 3D scanners), and may also be computer generated. Point cloud data is used in various applications including but not limited to construction (modeling), graphics (3D models for visualization and animation), and the automotive industry (LIDAR sensors to aid navigation).
[0056] The 3D space occupied by the point cloud data can be surrounded by a virtual bounding box. The position of the points in the bounding box can be represented by a certain precision; therefore, the position of one or more points can be quantized based on precision. At the smallest level, the bounding box is divided into voxels, which are the smallest spatial units represented by unit cubes. The voxels in the bounding box can be associated with zero, one, or more than one points. The bounding box can be divided into multiple cubes / cube regions, which can be called tiles. Each tile can be decoded into one or more slices. The division of the bounding box into slices and tiles can be based on the number of points in each partition, or based on other considerations (for example, a particular area can be decoded as a tile). The slice area can be further divided using a segmentation decision similar to the segmentation decision in the video codec.
[0057] Figure 2 An overview of the G-PCC encoder 200 is provided. Figure 3 An overview of a G-PCC decoder 300 is provided. The modules shown are logical and do not necessarily correspond one-to-one with code implemented in a reference implementation of the G-PCC codec.
[0058] In both the G-PCC encoder 200 and the G-PCC decoder 300, the point cloud positions are first decoded. The attribute decoding depends on the decoded geometry. Figure 2 and Figure 3 , the gray-shaded modules are options typically used for category 1 data. The diagonally shaded modules are options typically used for category 3 data. All other modules are common between category 1 and category 3.
[0059] For category 3 data, the compressed geometry is typically represented as an octree from the root down to the leaf level for each voxel. For category 1 data, the compressed geometry is typically represented by a pruned octree (i.e., an octree from the root down to the leaf level for blocks larger than a voxel) plus a model that approximates the surface within each leaf of the pruned octree. In this way, both category 1 and category 3 data share the octree decoding mechanism, while category 1 data can additionally utilize a surface model to approximate the voxels within each leaf. The surface model used is a triangulation that includes 1-10 triangles per block, resulting in a triangle soup. Therefore, category 1 geometry codecs are called Trisoup geometry codecs, while category 3 geometry codecs are called Octree geometry codecs.
[0060] At each node of the octree, occupancy is signaled for one or more of its children (up to eight nodes) (when not inferred). Multiple neighborhoods are specified, including: (a) nodes that share a front face with the current octree node, (b) nodes that share a front face, an edge, or a vertex with the current octree node, etc. Within each neighborhood, the occupancy of a node and / or its children can be used to predict the occupancy of the current node or its children. For points that are sparsely populated in certain nodes of the octree, the codec also supports a direct decoding mode in which the 3D position of the point is encoded directly. A flag may be signaled to indicate that a direct mode is signaled. At the lowest level, the number of points associated with an octree node / leaf node may also be decoded.
[0061] Once the geometry is decoded, the attributes corresponding to the geometry points are decoded. When there are multiple attribute points corresponding to one reconstructed / decoded geometry point, the attribute value representing the reconstructed point can be derived.
[0062] There are three attribute coding methods in G-PCC: Region Adaptive Hierarchical Transform (RAHT) coding, interpolation-based hierarchical nearest neighbor prediction (prediction transform), and interpolation-based hierarchical nearest neighbor prediction (lifting transform) with update / lifting step size. RAHT and lifting are typically used for category 1 data, while prediction is typically used for category 3 data. However, either method can be used for any data, and just like the geometry codec in G-PCC, the attribute coding method used to decode the point cloud is specified in the bitstream.
[0063] The decoding of the attributes may be performed in levels of detail (LOD), wherein with each level of detail, a finer representation of the point cloud attributes may be obtained. Each level of detail may be specified based on a distance metric to neighboring nodes or based on a sampling distance.
[0064] At the G-PCC encoder 200, the residual obtained as an output of the coding method for the attribute is quantized. The residual may be obtained by subtracting the attribute value from a prediction derived based on points in the neighborhood of the current point and based on the attribute value of a previously encoded point. The quantized residual may be coded using context adaptive arithmetic coding.
[0065] exist Figure 2 In the example, the G-PCC encoder 200 may include a coordinate transformation unit 202, a color transformation unit 204, a voxelization unit 206, an attribute transfer unit 208, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, a geometry reconstruction unit 216, a RAHT unit 218, an LOD generation unit 220, a lifting unit 222, a coefficient quantization unit 224 and an arithmetic coding unit 226.
[0066] As in Figure 2 As shown in the example of , the G-PCC encoder 200 can obtain a set of positions and a set of attributes of points in the point cloud. The G-PCC encoder 200 can obtain the position set and the attribute set of points in the point cloud from the data source 104 ( Figure 1 ) obtains a set of positions and a set of attributes of points in a point cloud. The positions may include the coordinates of the points in the point cloud. The attributes may include information about the points in the point cloud, such as the color associated with the points in the point cloud. The G-PCC encoder 200 may generate a geometry bitstream 203 including an encoded representation of the positions of the points in the point cloud. The G-PCC encoder 200 may also generate an attribute bitstream 205 including an encoded representation of the attribute set.
[0067] The coordinate transformation unit 202 may apply a transformation to the coordinates of the point to transform the coordinates from the initial domain to the transformed domain. The present disclosure may refer to the transformed coordinates as transformed coordinates. The color transformation unit 204 may apply a transformation to transform the color information of the attribute to a different domain. For example, the color transformation unit 204 may transform the color information from the RGB color space to the YCbCr color space.
[0068] In addition, Figure 2 In the example of , the voxelization unit 206 can voxelize the transformed coordinates. Voxelization of the transformed coordinates can include quantizing and removing some points of the point cloud. In other words, multiple points of the point cloud can be grouped into a single "voxel", which in some aspects can be considered as a point thereafter. In addition, the octree analysis unit 210 can generate an octree based on the voxelized transformed coordinates. In addition, in Figure 2 In the example of FIG. 1 , the surface approximation analysis unit 212 may analyze the points to potentially determine a surface representation of the set of points. The arithmetic coding unit 214 may entropy encode syntax elements representing the octree and / or information of the surface determined by the surface approximation analysis unit 212. The G-PCC encoder 200 may output these syntax elements in a geometry bitstream 203. The geometry bitstream 203 may also include other syntax elements, including syntax elements that are not arithmetically encoded.
[0069] The arithmetic coding unit 214 may include a context selection unit 215. The context selection unit 215 may be configured to perform one or more techniques of the present disclosure. For example, as will be explained in more detail below, the context selection unit 215 may be configured to determine the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of point cloud data for inter-frame prediction of a current node in a current frame of point cloud data; determine a context for decoding a current occupied bit of a current subnode of the current node based on the occupancy of the reference subnode; and use the context to arithmetically encode the current occupied bit. By using the occupancy information of the reference node in the context selection for the current node, a more accurate context can be selected, thereby improving decoding efficiency when arithmetically decoding occupancy-related syntax elements (e.g., occupied bits).
[0070] The geometry reconstruction unit 216 may reconstruct the transformed coordinates of the points in the point cloud based on the octree, the data indicating the surface determined by the surface approximation analysis unit 212, and / or other information. Due to voxelization and surface approximation, the number of transformed coordinates reconstructed by the geometry reconstruction unit 216 may be different from the number of original points of the point cloud. The present disclosure may refer to the resulting points as reconstructed points. The attribute transfer unit 208 may transfer the attributes of the original points of the point cloud to the reconstructed points of the point cloud.
[0071] In addition, the RAHT unit 218 may apply RAHT decoding to the attributes of the reconstructed points. In some examples, under RAHT, the attributes of the block of 2x2x2 point positions are obtained and transformed along one direction to obtain four low-frequency (L) and four high-frequency (H) nodes. Subsequently, the four low-frequency nodes (L) are transformed in the second direction to obtain two low-frequency nodes (LL) and two high-frequency nodes (LH). The two low-frequency nodes (LL) are transformed along the third direction to obtain a low-frequency node (LLL) and a high-frequency node (LLH). The low-frequency node LLL corresponds to the DC coefficient, and the high-frequency nodes H, LH and LLH correspond to the AC coefficient. The transformation in each direction may be a one-dimensional transformation with two coefficient weights. The low-frequency coefficients may be considered as coefficients of the next higher level 2x2x2 block for the RAHT transform, and the AC coefficients are encoded without change; such transformations continue until the top root node. The tree traversal for encoding is used from top to bottom to calculate the weights to be used for the coefficients; the transformation order is from bottom to top. The coefficients may then be quantized and decoded.
[0072] Alternatively or in addition, the LOD generation unit 220 and the lifting unit 222 may apply LOD processing and lifting to the attributes of the reconstructed points, respectively. LOD generation is used to segment the attributes into different refinement levels. Each refinement level provides a refinement of the attributes of the point cloud. The first refinement level provides a rough approximation and contains few points; subsequent refinement levels typically contain more points, and so on. The refinement levels can be constructed using a distance-based metric, or one or more other classification criteria (e.g., subsampling from a specific order) can also be used. Therefore, all reconstructed points can be included in the refinement level. Each level of detail is generated by taking the union of all points to a specific refinement level: for example, LOD1 is obtained based on refinement level RL1, LOD2 is obtained based on RL1 and RL2,..., LODN is obtained by the union of RL1, RL2,..., RLN. In some cases, LOD generation can be followed by a prediction scheme (e.g., predictive transform), where the attributes associated with each point in the LOD are predicted based on a weighted average of previous points, and the residual is quantized and entropy coded. The lifting scheme builds on the predictive transform mechanism, where an update operator is used to update coefficients, and performs adaptive quantization of the coefficients.
[0073] The RAHT unit 218 and the lifting unit 222 may generate coefficients based on the attributes. The coefficient quantization unit 224 may quantize the coefficients generated by the RAHT unit 218 or the lifting unit 222. The arithmetic coding unit 226 may apply arithmetic coding to the syntax elements representing the quantized coefficients. The G-PCC encoder 200 may output these syntax elements in the attribute bitstream 205. The attribute bitstream 205 may also include other syntax elements, including syntax elements that are not arithmetically encoded.
[0074] exist Figure 3 In the example, the G-PCC decoder 300 may include a geometry arithmetic decoding unit 302, an attribute arithmetic decoding unit 304, an octree synthesis unit 306, an inverse quantization unit 308, a surface approximation synthesis unit 310, a geometry reconstruction unit 312, a RAHT unit 314, a LoD generation unit 316, an inverse lifting unit 318, an inverse transform coordinate unit 320 and an inverse transform color unit 322.
[0075] The G-PCC decoder 300 may obtain the geometry bitstream 203 and the attribute bitstream 205. The geometry arithmetic decoding unit 302 of the G-PCC decoder 300 may apply arithmetic decoding (e.g., context adaptive binary arithmetic coding (CABAC) or other types of arithmetic decoding) to the syntax elements in the geometry bitstream 203. Similarly, the attribute arithmetic decoding unit 304 may apply arithmetic decoding to the syntax elements in the attribute bitstream 205.
[0076] The geometry arithmetic decoding unit 302 may include a context selection unit 303. The context selection unit 303 may be configured to perform one or more techniques of the present disclosure. For example, as will be explained in more detail below, the context selection unit 303 may be configured to determine the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of point cloud data for inter-frame prediction of a current node in a current frame of point cloud data; determine a context for decoding a current occupied bit of a current subnode of the current node based on the occupancy of the reference subnode; and use the context to arithmetically decode the current occupied bit. By using the occupancy information of the reference node in the context selection for the current node, a more accurate context can be selected, thereby improving decoding efficiency when arithmetically decoding occupancy-related syntax elements (e.g., occupied bits).
[0077] The octree synthesis unit 306 can synthesize an octree based on the syntax elements parsed from the geometry bitstream 203. Starting from the root node of the octree, the occupancy of each of the eight child nodes at each octree level is signaled in the bitstream. When the signaling indicates that the child node at a specific octree level is occupied, the occupancy of the child nodes of this child node is signaled. The signaling of the node at each octree level is signaled before the subsequent octree level. At the last level of the octree, each node corresponds to a voxel position; when a leaf node is occupied, one or more points can be designated as occupied at the voxel position. In some cases, due to quantization, some branches of the octree may terminate earlier than the last level. In such a case, the leaf node is considered to be an occupied node without a child node. In an example in which surface approximation is used in the geometry bitstream 203, the surface approximation synthesis unit 310 can determine the surface model based on the syntax elements parsed from the geometry bitstream 203 and based on the octree.
[0078] In addition, the geometry reconstruction unit 312 can perform reconstruction to determine the coordinates of the points in the point cloud. For each position at a leaf node of the octree, the geometry reconstruction unit 312 can reconstruct the node position by using the binary representation of the leaf node in the octree. At each respective leaf node, the number of points at the respective leaf node is signaled; this indicates the number of identical points at the same voxel position. When geometry quantization is used, the point positions are scaled for determining the reconstructed point position values.
[0079] The inverse transform coordinate unit 320 may apply an inverse transform to the reconstructed coordinates to convert the reconstructed coordinates (positions) of the points in the point cloud from the transform domain back to the original domain. The point positions in the point cloud may be in the floating point domain, but the point positions in the G-PCC codec are decoded in the integer domain. An inverse transform may be used to convert the positions back to the original domain.
[0080] In addition, Figure 3 In the example of , the inverse quantization unit 308 may inverse quantize the property value. The property value may be based on syntax elements obtained from the property bitstream 205 (eg, including syntax elements decoded by the property arithmetic decoding unit 304).
[0081] Depending on how the attribute value is encoded, the RAHT unit 314 may perform RAHT decoding to determine the color value for the point of the point cloud based on the inverse quantized attribute value. RAHT decoding is performed from the top to the bottom of the tree. At each level, the low-frequency coefficients and high-frequency coefficients derived from the inverse quantization process are used to derive the component (constitute) value. At the leaf node, the derived value corresponds to the attribute value of the coefficient. The weight derivation process for the point is similar to the process used at the G-PCC encoder 200. Alternatively, the LOD generation unit 316 and the inverse lifting unit 318 may use a detail level-based technique to determine the color value for the point of the point cloud. The LOD generation unit 316 decodes each LOD that gives a progressively fine representation of the attribute of the point. Using the prediction transform, the LOD generation unit 316 derives the prediction of the point from the weighted sum of the points in the previous LOD or previously reconstructed in the same LOD. The LOD generation unit 316 can add the prediction to the residual (which is obtained after inverse quantization) to obtain the reconstructed value of the attribute. When a lifting scheme is used, the LOD generation unit 316 may also include an update operator to update the coefficients used to derive the property value. In this case, the LOD generation unit 316 may also apply inverse adaptive quantization.
[0082] In addition, Figure 3 In the example of , the inverse color transform unit 322 can apply an inverse color transform to the color value. The inverse color transform can be the inverse process of the color transform applied by the color transform unit 204 of the G-PCC encoder 200. For example, the color transform unit 204 can transform the color information from the RGB color space to the YCbCr color space. Therefore, the inverse color transform unit 322 can transform the color information from the YCbCr color space to the RGB color space.
[0083] Show Figure 2 and Figure 3 Various units are provided to help understand the operations performed by the G-PCC encoder 200 and the G-PCC decoder 300. The units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions, and are pre-set with respect to the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks, and provide flexible functions with the operations that can be performed. For example, a programmable circuit can execute software or firmware that enables the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (for example, to receive parameters or output parameters), but the type of operation performed by the fixed-function circuit is generally unchangeable. In some examples, one or more of the units in the unit may be a distinguished circuit block (fixed function or programmable), and in some examples, one or more of the units in the unit may be an integrated circuit.
[0084] G-PCC plane decoding mode
[0085] Examples of tools used in G-PCC include plane decoding mode. G-PCC may also include angle decoding mode, which can enhance the decoding efficiency of plane mode by using the sensor characteristics of a typical LIDAR sensor. The angle decoding mode is optionally used with the plane mode, and the decoding of the vertical (z) plane position syntax element is improved by adopting the knowledge of the position and angle of the laser beam sensed in the typical LIDAR sensor. In addition, the angle decoding mode can be optionally used to improve the decoding of the vertical z position bit and the horizontal x, y position in the inferred direct decoding mode (IDCM). The first node in the octree can be decoded as an IDCM mode. When the node is decoded using the IDCM mode, the occupied bits of the child nodes of the first node are not transmitted with a signal. Instead, the position of the point contained in the first node is explicitly transmitted with a signal. These positions transmitted with a signal are not the explicit positions of the points in the point cloud, nor are they the positions of the points relative to the octree origin or the slice origin. Instead, the positions of the points transmitted with a signal in the first node are those relative to the node position. One or more of the x, y, z coordinates of the position of a point in a node are decoded. In one example, at most two positions may be decoded in an IDCM node. In an example contribution to G-PCC, context derivation for angle coding modes is simplified, and high-level syntax (HLS) coding of sensor data parameters is made more efficient.
[0086] Azimuth coding mode can also be used in G-PCC. Azimuth coding mode is similar to angle mode and extends angle mode to the coding of (x) and (y) plane position syntax elements for plane mode and improves the coding of x or y position bits in IDCM. In a contribution to G-PCC, the number of contexts used in azimuth mode is significantly reduced.
[0087] Occupancy decoding for octree geometry
[0088] The encoding of occupancy for a point cloud based on a geometry octree can be in Figure 4 outlined in . Figure 4 is a flow chart illustrating an example process 400 for occupancy coding in a G-PCC.
[0089] Typically, in one example of G-PCC, the occupancy of a node is encoded using context-based entropy coding, where a context index is adaptively derived based on occupancy information of neighboring nodes and plane information of the current node. A G-PCC decoder (e.g., G-PCC encoder 200 or G-PCC decoder 300) may determine occupancy information of neighboring nodes relative to the current node (obtain neighbors 402). The G-PCC decoder may then calculate a neighbor configuration N64 (integer) of the neighboring node (calculating N 64 404). N64 refers to 64 possible neighbor configurations based on a 6-bit pattern (e.g., nodes that share 6 front faces of the current node). In general, a neighbor configuration refers to the spatial relationship between neighboring points in a point cloud. In some examples, a neighbor configuration refers to the configuration of neighboring points of a point within a certain distance threshold, which is typically determined by the resolution of the point cloud.
[0090] By encoding the neighborhood configuration of a point, a compressed point cloud can be reconstructed with greater accuracy, since neighboring points can provide additional information about occupancy, surface geometry, and curvature (if applicable) in the vicinity of the point. Neighbor configuration can also help reduce the amount of data required to represent a point cloud by more efficiently predicting the occupancy of a current point based on the occupancy of neighboring points, or enabling better selection of the context for which to predict the occupancy of the current point.
[0091] Then, the G-PCC decoder uses the neighbor information and N64 to perform occupancy prediction for the current node (calculate intra-frame prediction 406). Typically, G-PCC uses the information of the neighbors of the current node to predict whether the current node is occupied. The G-PCC decoder can also calculate the occupancy of improved sub-neighbors (calculate improved sub-neighbors 408). The improved sub-neighbors can be sub-nodes of neighbor nodes. The G-PCC decoder can then use the plane information of the current node to determine whether the occupied bits of the current node need to be encoded or can be explicitly derived in the decoder. If the occupied bits of the current node need to be encoded, the context is obtained using N64, the calculated intra-frame prediction of the current node and the improved sub-neighbor information.
[0092] For example, the G-PCC may first determine whether the current node can be decoded using three plane markings (410). If yes at 410, the occupancy information for the current node is not decoded (e.g., not signaled or decoded). If no at 410, the G-PCC decoder then determines whether the N64 value is 0 (412). N64 equal to zero means that all neighbor nodes are unoccupied. If yes at 412, the G-PCC decoder then determines whether the current node has a single child node or a single plane (414). If yes at 414, the occupancy information for the current node is not decoded. If no at 414, the G-PCC decoder determines whether the current node can be decoded using two plane markings (416). If yes at 416, the occupancy information for the current node is not decoded.
[0093] If the current node is not decoded in planar mode (e.g., all of 410, 414, and 416 are no), and the N64 value is not 0 (no at 412), the G-PCC decoder determines geometric invariance (418) to update the N64 value. The neighborhood configuration includes neighbors in all three directions relative to the node. However, two configurations can be considered to be invariant to each other based on whether the two configurations look exactly the same as the node with respect to their respective dimensions. For example, assume that node N has a position (x, y, z). In the first case, N has only one neighbor at (x+1, y, z). In the other case, assume that N has only one neighbor at (x, y+1, z). Although the neighborhood configurations in each of these two cases are different, they are considered to be geometrically invariant because the configuration about the x-axis of the first case is the same as the configuration about the y-axis of the second case. Similarly, a neighbor configuration with two nodes (x+1, y, z) and (x, y, z+1) can be considered invariant to a neighbor configuration with two nodes (x+1, y, z) and (x, y+1, z). The geometric invariance of the configuration can be used to reduce the number of possible values of N64, thereby reducing the number of context states.
[0094] The G-PCC decoder then determines again whether the N64 value is 0 (420). If it is yes at 420, this means that all neighbor nodes are occupied, and the G-PCC decoder can use zero (Z) occupancy decoding (which uses the optimal binary decoder (OBUF) with instant update) to decode the occupied bits of the current node (422). Typically, a specific arithmetic decoder is selected from the arithmetic decoder set using Z occupancy decoding of OBUF to decode the occupied bits. If it is yes at 420, this means that at least one of the neighbor nodes is occupied, and the G-PCC decoder can use non-zero (NZ) occupancy decoding using OBUF to decode the occupied bits of the current node (424).
[0095] Decoding using OBUF may include two general steps: In a first step, for the occupancy of a given child node to be encoded, a binary arithmetic decoder is selected from a set of binary arithmetic decoders. This decision to select a specific arithmetic decoder depends on the index of the child node and the dependency state derived from the neighborhood information. The dependency state X can take one of several values, where the value corresponds to one or more neighborhood configurations. Each arithmetic decoder is associated with a probability, and the probabilities of the binary arithmetic decoders are selected so that they together cover the probability range of [0,1], that is, a set of increasing probability values is associated with the decoder so that the first value is close to 0 and the last value is close to 1. The decision to select a binary arithmetic decoder for the i-th child node is obtained using the i-th lookup table that maps the index i and the dependency state X to the index of the arithmetic decoder. Once a bit is decoded, the lookup table corresponding to the index i and the given dependency state X is updated to take into account the decoded bit; as a result, the index of the arithmetic decoder corresponding to the index i and the dependency state X can be modified (typically the index increases or decreases by 1).
[0096] Below is an excerpt from Section 9.2.10 of Draft 22 of the G-PCC standard.
[0097] Use in Section 9.2.10 of G-PCCFDIS, ISO / IEC JTC1 / SC29 / WG11m55637 (Draft 22) Bit-by-bit occupancy decoding 9.2.10 Bit Occupancy Decoding 9.2.10.1 Overview When octree_bitwise_coding is 1, subclause 9.2.1.0 applies. The neighborhood ranking node occupancy bitmap shall be coded as a sequence of separate occupancy_bit syntax elements. The coding uses constraints on occupancy to infer the values of certain occupancy_bit syntax elements. The entropy coding of each coded bit is contextualized by a combination of the coded bit index, the previously coded occupancy_bit syntax element, the reduced occupied neighborhood pattern, the number of spatially neighboring subnodes in a neighboring node, and a ternary prediction based on the existence of neighboring nodes. 9.2.10.2 Association between node occupancy bitmap and occupancy_bit The bits of the neighborhood ranking node occupancy bitmap shall be decoded in the order specified in Table 19. Each occupancy_bit[cbIdx] syntax element decodes the bit OccBitCodingOrder[cbIdx]. The expression OccBitIdx[cbIdx] is the bit position in the node occupancy bitmap of the bit decoded by occupancy_bit[cbIdx]. For example, when OccNeighPat is 17, occupancy_bit[6] corresponds to the second bit (b1) of the node occupancy bitmap. OccBitIdx[cbIdx]:=OccBitIdxFromNpBit[OccBitCodingOrder[cbIdx]] The expression OccBitLocC[cbIdx][k] is the position of the child node relative to the node represented by occupancy_bit[cbIdx]. OccBitLocC[cbIdx][k]:=OccLocC[OccBitIdx[cbIdx]][k] The expression OccBitMap is the node occupancy bitmap. OccBitMap:= OccBitMap=0 for(cbIdx=0;cbIdx<8;cbIdx++) OccBitMap=OccBitMap|(occupancy_bit[cbIdx]< <OccBitIdx[cbIdx]) Table 19 — Order used to decode the bits of the neighborhood ranking node occupancy bitmap into occupancy_bit[cbIdx] -cbIdx 0 1 2 3 4 5 6 7 -OccBitCodingOrder[cbIdx] 1 7 5 3 2 6 4 0 9.2.10.3 Existence of occupancy_bit If the value of an occupancy bitmap bit can be inferred to be set or not set, it shall not be decoded. The expression OccBitPresent[cbIdx] specifies whether occupancy_bit[cbIdx] is present. 9.2.10.4 Inference of unset bits The occupied bitmap bits shall be inferred to be 0 when any of the following conditions exist: Depending on the node's decoded axis, this bit indicates an invalid child node (9.2.2.3), or • This bit represents the child nodes in the unoccupied plane signaled by the plane occupancy decoding. When the expression OccBitInferUnset[cbIdx] is equal to 1, occupancy_bit[cbIdx] should be inferred to be 0. 9.2.10.5 Inference of set bits As specified by the expression OccBitInferSet[cbIdx], the occupancy bitmap bit occupancy_bit[cbIdx] shall be inferred to be 1 when: The bit is the last bit present, and all previously decoded bits are 0, or The bit is the second-to-last bit present, all previously decoded bits are 0, and the node is required to have two children, or The bit is in a plane identified as occupied by plane occupancy decoding, the bit is the last bit in the plane, and all previous bits in the plane are zero. OccBitInferSet[cbIdx]:= PlanarEligible[0]&&PopCnt(OccKnownZero&(0x0F<<4×OccBitLocC[cbIdx][0]))===3 ||PlanarEligible[1]&&PopCnt(OccKnownZero&(0x33<<2×OccBitLocC[cbIdx][1]))===3 ||PlanarEligible[2]&&PopCnt(OccKnownZero&(0x55<<1×OccBitLocC[cbIdx][2]))===3 ||cbIdx==6&&PopCnt(OccKnown)==0&&OccMinChildren==2 ||cbIdx==7&&PopCnt(OccKnown)==0 The expression OccKnownMask is a bit mask that identifies the bits in the node occupancy bitmap that have known values before decoding occupancy_bit[cbIdx]. OccKnownMask:= OccKnownMask=0 for(i=0;i <cbIdx;i++) OccKnownMask|=1< <OccBitIdx[i] for(i=0;i<8;i++) OccKnownMask|=OccBitInferUnset[i]< <OccBitIdx[i] The expression OccKnown is a partially decoded node occupancy bitmap that includes the bits decoded before occupancy_bit[cbIdx]. OccKnown:= OccKnown=0 for(i=0;i <cbIdx;i++) OccKnown|=occupancy_bit[i]< <OccBitIdx[i] The expression OccKnownZero is a bitmap of occupied bits that are known to be zero. OccKnownZero:=(0xFF^OccKnown)&OccKnownMask 9.2.10.6 Putting it in context 9.2.10.6.1 Overview The contextualization of the occupancy_bit syntax element is a two-stage process. First, a context discriminator is used to select a half-CPM. Then, the half-CPM is used to select the CPM for coding the syntax element. Half-CPM is an 8-bit unsigned integer that models the probability of a coded zero-valued occupancy_bit syntax element. The values 0, 128, and 256 represent the probability of the zero bin as impossible, equally probable, and certain, respectively. Due to the operation of the update process of the probability model, the values 0 and 256 can never be obtained. 9.2.10.6.2 State variables Context selection is specified using the following state variables: The array OccCtxSel; OccCtxSel[selNeigh][cbIdx][selSib][selAdj][selPred] is a half CPM that is contextualized by selNeigh, cbIdx, selSib, selAdj, and selPred. 9.2.10.6.3 Initial state Half CPM shall be initialized at the beginning of each GDU. When slice_entropy_continuation is 1, initialization shall be performed according to the parsing state recovery procedure (11.6.2.2). Otherwise (slice_entropy_continuation is 0), all elements of OccCtxSel should be set to 127. 9.2.10.6.4 Determination of CtxIdxOccBit for syntax element occupancy_bit The expression OccCtxSelVar specifies the half CPM for the syntax element occupancy_bit[CbIdx] using the following: SelNeigh, the reduced occupied neighborhood context discriminator (9.2.10.6.6); SelSib, sibling occupancy context discriminator (9.2.10.6.7); SelAdj, the adjacent sub-neighbor context discriminator (9.2.10.6.8); SelPred, the neighbor prediction occupancy context discriminator (9.2.10.6.9). OccCtxSelVar:=OccCtxSel[SelNeigh][CbIdx][SelSib][SelAdj][SelPred] The CPM index CtxIdxOccBit shall be the value of the half CPM except for the bottom three bits: CtxIdxOccBit:=OccCtxSelVar>>3 9.2.10.6.5 Update after each coded occupancy_bit syntax element After each coded occupancy_bit syntax element, its half CPM shall be updated. The updates specified by Table 20 provide values for increasing or decreasing the probability of a zero bin based on the upper four bits of the value of the half CPM: if(OccBitPresent[CbIdx]) if(occupancy_bit[CbIdx]) OccCtxSelVar+=OccCtxSelUpdate[255–OccCtxSelVar>>4] else OccCtxSelVar-=OccCtxSelUpdate[OccCtxSelVar>>4] Table 20 — Values of OccCtxSelUpdate[i] i 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 OccCtxSelUpdate[i] 0 1 1 2 4 7 9 11 14 16 19 23 22 18 13 6 9.2.10.6.6 Reduced Occupied Neighborhood Context Discriminator The reduced occupied neighbourhood context discriminator shall distinguish between different reduced occupied neighbourhood patterns (OccNeighPatR) depending on the coded bit index (CbIdx) as specified by Table 21 as the expression SelNeigh. Table 21 — Discrimination values SelNeigh and OccNeighPatR for occupancy_bit[CbIdx] 9.2.10.6.7 Peer Occupancy Context Discriminator The sibling occupancy context discriminator shall distinguish between the placement of previously coded / inferred siblings for a node coded as specified by the expression SelSib: If a node does not exist in the occupied neighborhood mode, it shall be identified by the number of child nodes present as indicated by the syntax element occupancy_bit[i], where i <CbIdx。 If there is at least one node present in occupied neighborhood mode, it shall be identified by the combination of the present child nodes identified by the syntax element occupancy_bit[i], where i <CbIdx。 SelSib:=OccNeighPat? occPrevBits:PopCnt(occPrevBits) For i < CbIdx, the expression occPrevBits is the concatenation of occupancy_bit[i]. occPrevBits := occPrevBits = 0 for (i = 0; i < CbIdx; i++) occPrevBits |= occupancy_bit[i] << i 9.2.10.6.8 Neighboring Sub - Neighbor Context Discriminator The neighboring sub - neighbor context discriminator for the child node decoded by occupancy_bit[CbIdx] is specified by the expression SelAdj. When neighboring sub - neighbors are placed in context (occtree_adjacent_child_enabled is 1), it differentiates between contexts as follows: · The number of child nodes that are adjacent to the decoded child node through the positive side from the available previously decoded nodes at the same tree level (9.2.7.2); and · Whether any of the available previously decoded nodes adjacent to the decoded child node at the same tree level do not have a child node that is also adjacent to the decoded child node. An example is shown in Fig.10 . The child node C0 of the decoded node N is adjacent to a single child node. There are two available previously decoded nodes adjacent to C0, and one of them does not contain a child node that is also adjacent to C0. SelAdj := occtree_adjacent_child_enabled ? 2 × Min(2, adjCntC) + ((cbIdx ≤ 4 || adjCntC == 1) && adjUnocc) : 0 The expression adjOccN[k] identifies the nodes that are spatially adjacent along the k - th axis within the occupied neighborhood availability window. The values for the expressions ds, dt, and dv are specified in Table 22 for each axis k. The expression adjOccC[k] identifies whether there is a child node that is spatially adjacent along the k - th axis within the occupied neighborhood availability window. The values for the expressions ds, dt, and dv are specified in Table 22 for each axis k. adjOccC[k]:=adjOccN[k]&&OccNodePresent[Dpth+1][cs+ds][ct+dt][cv+dv]≠0 in cs:=NsC+OccBitLocC[CbIdx][0] ct:=NtC+OccBitLocC[CbIdx][1] cv:=NvC+OccBitLocC[CbIdx][2] Table 22 — Relative neighbor positions (ds, dt, dv) used in the calculation of adjOccN[k] and adjOccC[k] k ds dt dv 0 -1 0 0 1 0 -1 0 2 0 0 -1
[0098] * Figure 5 5 is a conceptual diagram showing an example of adjacent sub-neighborhood context identification 500. It should be noted that in the above standard part, Figure 5 Cited as " Fig.10 ”.
[0099] The expressions adjCntN and adjCntC are the number of spatially adjacent nodes and child nodes within the occupied neighborhood availability window, respectively. adjCntN:=adjOccN[0]+adjOccN[1]+adjOccN[2] adjCntC:=adjOccC[0]+adjOccC[1]+adjOccC[2] The expression adjUnocc identifies whether there are spatially neighboring nodes within the occupied neighborhood availability window that do not have child nodes that are spatially neighboring to the decoded child node. adjUnocc:=adjCntN≠adjCntC 9.2.10.6.9 Neighbor Prediction Occupancy Context Discriminator —Overview For eligible nodes (9.2.10.9.6.2), the neighbor prediction occupancy context discriminator shall distinguish between three predictions for the existence of a child node decoded by occupancy_bit[CbIdx]. The discriminator is specified by the expression SelPred. The three predictions are that the node exists, does not exist, or is not predicted. SelPred:=SelPredEligible? OccIntraPred:0 — Eligibility The discriminator should only form predictions for eligible nodes specified by the expression SelPredEligible. An eligible node should have both of the following: Three axes of freedom; and The maximum log2 node dimension is less than occtree_intra_pred_max_nodesize_log2 (occtree_intra_pred_max_nodesize_log2 small_log2). SelPredEligible:= OccFreeAxisCnt==3&&MaxVec(NodeSizeLog2) <occtree_intra_pred_max_nodesize_log2 —Occupancy prediction The occupancy prediction generates a ternary prediction for the existence of the child node identified by the occupancy_bit[CbIdx] of the decoded node. The prediction is specified by the expression OccIntraPred. This is based on how many of the nodes neighboring the decoded node are also volumetrically adjacent to the identified child node by front, edge, or corner (as determined by Fig.11 shown): If there are two or fewer adjacent nodes, the absence of a child node should be predicted. If there are at least a threshold number of adjacent nodes, then the child node should be predicted to exist. The threshold is specified by the expression OccIntraThreshold. The threshold is four nodes, unless there are more than 13 adjacent nodes; in this case, the threshold is 5 nodes. Otherwise, existence is unpredictable. The size of a child node's volume is half the size of its neighbor nodes in each dimension. OccIntraPred:=(OccAdjCnt≤2)+2×(OccAdjCnt≥OccIntraThreshold) OccIntraThreshold:=4+(OccNeighCnt≥14)
[0100] Figure 6 is a conceptual diagram showing an example node 600 that is adjacent to a child node through a front face, an edge, or a corner. It should be noted that in the above standard section, Figure 6 Cited as " Fig.11 ”.
[0101] The expression OccAdj[ds][dt][dv] identifies whether the coded node of the neighboring node with relative tree position (ds, dt, dv) will be volume-adjacent to the identified child node. OccAdj[ds][dt][dv]:=(OccBitIdx[CbIdx]&adjMask)==adjLoc in adjMask:=Morton(ds≠0,dt≠0,dv≠0) adjLoc:=Morton(ds>0,dt>0,dv>0) The expression OccAdjCnt is the number of neighbors that adjoin the identified child node volume. OccAdjCnt:=SumN26[neighAdj] in neighAdj[ds][dt][dv]:=OccNeigh[Ns+ds][Nt+dt][Nv+dv]&&OccAdj[ds][dt][dv] The expression OccNeighCnt is the number of nodes that are neighbors to the node being decoded. OccNeighCnt:=SumN26[neighRel] in neighRel[ds][dt][dv]:=OccNeigh[Ns+ds][Nt+dt][Nv+dv] The expression SumN26[expr] sums the results of applying expr to the relative tree position of each of the 26 possible neighboring nodes. SumN26[expr]:= SumN26=0 for(ds=-1;ds≤1;ds++) for(dt=-1;dt≤1;dt++) for(dv=-1;dv≤1;dv++) if(ds≠0&&dt≠0&&dv≠0) SumN26+=expr[ds][dt][dv]
[0102] New occupancy decoding for octree based on dynamic OBUF
[0103] In one example of G-PCC, a new platform for encoding occupancy for octrees based on dynamic OBUFs is proposed. This example simplifies the octree decoding encoder for some examples of G-PCC. An overview of this technique is in Figure 7 As shown in . Figure 7 700 is a flow chart showing an example of a dynamic OBUF based occupancy decoding process. Figure 4 Compared to the process 400 shown in FIG. 4 , intra prediction (calculate intra prediction 406), calculate improved sub-neighbor process 408, and geometric invariance 418 are removed. Figure 4 The other processes with the same reference numerals remain the same. However, the zero (Z) and non-zero occupancy decoding process after 420 has been changed. In particular, a new derivation of context with dynamic OBUF is added for encoding nodes with N64 not equal to zero (No at 420), where N64 is a neighbor configuration calculated based on the occupancy of neighboring nodes.
[0104] like Figure 7 As shown, if yes at 420, simplified Z occupancy decoding is used 702. If no at 420, the G-PCC decoder can obtain complete neighborhood occupancy information (704), calculate context information based on the neighborhood occupancy information (706), and then use dynamic OBUF to perform NZ occupancy decoding (708). In general, the G-PCC decoder can determine the context for decoding the occupancy bits of the current node based on the occupancy information of the neighboring nodes in the same frame.
[0105] A new derivation of the context using dynamic OBUF can be as follows Figure 8 It is implemented as in . Figure 8 The context selection process 800 based on dynamic OBUF in occupancy decoding is shown. In this method, Word4 and Word7 values are first derived based on the occupancy information of neighboring nodes. Next, the G-PCC decoder uses Word4 and Word7 to determine the primary information context ctx1 and the secondary information context ctx2. Ctx1 and ctx2 are then used to derive the context index to encode the occupancy bits using the getEvolve function. Figure 8In , occupancy represents the occupancy of the current node, and i represents the index of the current bit to be encoded in the occupancy. Sparse information indicating how sparse the occupied nodes are in the neighborhood of the point is also derived using the occupancy of the neighboring nodes. _ctcMapOccupancy represents the entire context table in which the occupancy is to be encoded. _MapOccypancySparse and _MapOccupancy indicate context index regions used to encode occupied bits located in sparse and non-sparse areas. In an implementation, _MapOccupancySparse and _MapOccupancy are represented as tables of objects for each bit index.
[0106] Word4 derivation:
[0107] For each child node i, Word4[i] is constructed in 4-bit format. This construction is based on the number of nodes that share an edge or vertex with child node i but do not share a front face ( Fig. 9 )’s 4 parent neighbors. Fig. 9 is a conceptual diagram showing an example of the structure of Word4.
[0108] Sparse and Word7 derivation :
[0109] To construct sparse bits and Word7 bits, three neighbor sets are used, which include SL, SF and SB neighbors, as in Fig.10 As given in . Fig.10 is a conceptual diagram illustrating a process 1000 for an example neighbor node to obtain Sparse and Word7 for a child node b0.
[0110] exist Fig.10 In , SL(occL), SF(occF), and SB(occB) each represent the occupancy of 4 nodes from the left front neighbor and the bottom neighbor of the parent node. NN (a variable indicating the number of neighbors) can be calculated as follows: int NN=! ! (occL&1)+! ! (occL&2)+! ! (occL&4)+! ! (occL&8); NN+=! ! (occF&1)+! ! (occF&2)+! ! (occF&4)+! ! (occF&8); NN+=! ! (occB&1)+! ! (occB&2)+! ! (occB&4)+! ! (occB&8);
[0111] If NN is less than 1, the child node i is in the sparse region. Otherwise, it is marked as non-sparse. If the node i is in the sparse region, its Word7 bit is represented by 9 bits; otherwise, it is represented by 12 bits.
[0112] G-PCC Motion Prediction
[0113] There are two kinds of motion for decoding point clouds in the G-PCC InterEM software: global motion matrix and local node motion vectors. The global motion parameters are defined as the rotation matrix and translation vector, which will be applied to all points in the prediction (reference) frame (except for the points to which the local motion mode is applied). The local node motion vector of the node of the octree is the motion vector that is applied only to the points within the node in the prediction (reference) frame. The details of the motion estimation algorithm in InterEM are described below.
[0114] Fig.11 is a flow chart showing an example of motion estimation for InterEM mode. Fig.11 In the present invention, the current frame 1100 and the predicted frame 1102 are frames of point cloud data used as input for the motion prediction (e.g., inter-frame prediction) process. The current frame 1100 is the frame being decoded, and the predicted frame 1102 is a reference frame (e.g., a previously decoded frame) of the point cloud data used as a reference. The G-PCC encoder 200 uses the predicted frame 1102 and the current frame 1104 as inputs to perform a global motion estimation process 1110 at a global scale. The G-PCC encoder 200 then applies the estimated global motion to the predicted frame 1102 (1120). The G-PCC encoder 200 then uses the current frame 1104 and the predicted frame with the applied global motion as inputs to perform local node motion estimation 1130 at a finer scale (node level in the octree). The G-PCC encoder 200 then performs motion compensation (1140) on the current frame 1104, including encoding motion vectors and points. The details of the algorithm are briefly described below.
[0115] Method for estimating global motion matrix and translation vector
[0116] Fig.12 is a flow chart illustrating an example algorithm for estimating a local node motion vector for the current node 1200. Fig.12 In , the motion vectors are estimated in a recursive manner. The cost function used to select the most appropriate motion vector is based on the rate-distortion cost.
[0117] If the current node 1200 is not split into 8 child nodes, the motion vector that may result in the lowest cost between the current node and the predicted node is determined (1210). If the current node 1200 is divided into 8 child nodes (1220), a motion estimation algorithm is applied and motion is found for each child node (1230). The total cost under the split condition is obtained by adding the estimated cost values of each child node (1240). The decision on whether to split or not is achieved by comparing the costs between splitting and not splitting (1250); if split, each child node is assigned its own motion vector (or can be further split into its child nodes), and if not split, a motion vector is assigned to the current node.
[0118] Two parameters that affect the performance of motion vector estimation are the block size (BlockSize) and the minimum prediction unit size (MinPUSize). BlockSize defines the upper limit of the node size for applying motion vector estimation, and MinPUSize defines the lower limit.
[0119] Example
[0120] Inter prediction has shown many benefits in occupancy coding in G-PCC (InterEM). However, in previously proposed occupancy coding approaches, inter prediction information is not utilized. This disclosure proposes some techniques for utilizing inter prediction information in occupancy coding using newer occupancy coding methods. This disclosure describes several methods for improving occupancy coding using inter prediction.
[0121] In the present disclosure, the occupancy of the collocated nodes in the motion compensated frame is denoted as preOcc, which includes 8 bits. The i-th bit in preOcc represents the occupancy bit of the i-th child node in the reference node. The above summarizes the motion estimation and motion compensation for deriving the motion compensated reference frame. In general, the number of points in the prediction node is counted for each child node to form preOcc. For each child node, if there is at least one point, the corresponding bit in preOcc is set to 1. Otherwise, it is set to 0. In the present disclosure, p i It is the same-position reference child node of the i-th child node in the current node.
[0122] 8.1 Using the occupancy of a reference node to have adaptive context selection for the occupancy of the current node
[0123] In this example of the present disclosure, the occupied bits of the i-th child node in the reference node can be used to determine the context for entropy coding / decoding (e.g., arithmetic coding / decoding) of the occupied bits of the i-th node in the current node. In one example, the G-PCC encoder 200 and / or the G-PCC decoder 300 can be configured to determine the occupied bits of the i-th child node in the reference node. In some examples, the reference child node is co-located with the current child node. The reference node is in a reference frame of the point cloud data for inter-frame prediction of the current node in the current frame of the point cloud data.
[0124] The G-PCC encoder 200 and / or the G-PCC decoder 300 may also be configured to determine a context (e.g., from among multiple contexts) for decoding currently occupied bits of the i-th node of a current node based on the occupied bits of the i-th child node in a reference mode, and to perform arithmetically decoding on the currently occupied bits using the determined context.
[0125] Fig.13 is a conceptual diagram illustrating an example of context selection based on occupancy of a co-located reference child node for use in conjunction with the techniques of this disclosure. Fig.13 A simplified version of a current node 1300 in a current frame of point cloud data and a reference node 1350 in a reference node point cloud data is shown. The current node 1300 includes 4 child nodes, and the reference node 1350 includes 4 reference child nodes that are co-located with the 4 child nodes of the current node 1300. That is, the reference child nodes are in the same position of the reference frame as the child nodes in the current frame. It should be understood that Fig.13 is a simplified example, and the node of point cloud data may include more sub-nodes.
[0126] The context used for arithmetic decoding of the occupied bits of child node 1302 is based on the value of the occupied bits of the co-located reference child node 1352. Similarly, the occupied bits of child node 1304 are based on the value of the occupied bits of the co-located reference child node 1354. Fig.13 , reference subnode 1352 has an occupied bit value of 1, indicating that reference subnode 1352 is occupied. Reference subnode 1354 has an occupied bit value of 0, indicating that reference subnode 1354 is not occupied. According to the techniques of the present disclosure, the occupied bits of subnode 1302 will be decoded with a different context than the occupied bits of subnode 1304, assuming that their respective co-located reference subnodes have different values for their occupied bits.
[0127] In other examples, the occupancy of the reference child nodes may not correspond to the signaled syntax elements. For example, the reference frame may have an octree structure that is not aligned with the current frame, or, due to motion compensation, the points may be modified and they will no longer correspond to the octree structure.
[0128] Instead, in another example of the present disclosure, the G-PCC decoder 300 can analyze the co-located region corresponding to the current child node. The child node can have a width, height, and depth w, h, d at a position x0, y0, z0. Then, the reference child node is considered to be a node with the same w, h, and d and the same position in the reference frame (here, the position x0, y0, z0 is relative to a common origin). The occupancy of the reference child node is then obtained by counting the number of points of the reference frame in the reference child node region.
[0129] As such, in order to determine the occupancy of the reference subnode, the G-PCC decoder may determine the occupancy of the reference subnode by checking the number of points included in the reference subnode.
[0130] In one example, a framework for integrating the technique into a G-PCC for using dynamic OBUF is given below. The addition to the previous code is in the tag <add> and< / add> The removal of the previous code is shown between the tags <del> and< / del> shown between.
[0131] At the encoder (geometry_octree_encoder.cpp): Map occupancy is defined as follows: ctxZ is the context table that encodes occupancy when NN64 is equal to 0. ctxZ is defined in the geometry_octree.h file as follows: AdaptiveBitModel_ctxZ[8][7] <add> [2]< / add> ; At the decoder, the geometry_octree_decoder.cpp file can be modified accordingly as follows:
[0132] It should be noted that in intra-coded frames, predOcc is always 0. Therefore, in one example, the proposed technique does not change in intra-coded frames.
[0133] 8.2 Use the occupancy of the sibling nodes of the same node in the reference frame to select the context for the current node Occupancy coding
[0134] In this example of the present disclosure, the occupancy of the sibling nodes of the co-located node in the reference frame is used to determine the context in which the occupancy of the current node is to be encoded. In one example, the G-PCC encoder 200 and / or the G-PCC decoder can be configured to determine the occupancy bits of the sibling nodes of the co-located node in the reference frame; based on the occupancy bits of the sibling nodes of the co-located node in the reference frame, determine the context for decoding the current occupied bit current node; and use the context to perform arithmetic decoding on the current occupied bit.
[0135] In some examples, Word7 and Word4 values are constructed using neighbor information. They may contain redundant information.
[0136] In this example of the present disclosure, the Word4 value may be constructed using the same technique in previous contributions for intra-coded frames. However, in inter-coded frames, the occupancy of child nodes in a reference node may be used to construct the Word4 value.
[0137] Let p i is the same-position reference child node of the i-th child node in the current node. In one example, p i The occupancy of the four neighbor nodes (set P4) of pi can be used to construct the value of Word4[i]. Also in this example, P4 may include the four sibling nodes of pi, including i A child node that shares a vertex with p i 3 child nodes that share the front face. This example can be as in Fig.14Visualized in.
[0138] The 4 bits of Word4[i] may be formatted as follows: Word4[i]=p x p y p z p v , where p x 、p y and p z In the x, y and z directions respectively i The occupancy of the child nodes that share the front face. v Indicates that p i The occupancy of child nodes that share a vertex.
[0139] In another example, the Word4 value in an intra-coded slice may be derived using neighbor information in the current slice, while the Word4 value is derived using occupancy of reference nodes in an inter-coded frame.
[0140] Combination of 8.1 and 8.2
[0141] It should be noted that the techniques in Section 8.1 and Section 8.2 above can be used in combination. i The occupancy of is used to determine the context for occupancy decoding of the i-th child node, while the neighbor information of the co-located child node in the reference frame can be used to derive the Word4[i] value in the inter-coded frame.
[0142] In one example, the Word4 value derivation may be modified for inter-coded nodes as follows. V P i (co-located child node in the reference frame) instead. Alternatively, the Word4 value can be expanded to contain 5 bits, where the first 4 bits are bits obtained from the Word4 construction and the fifth bit is obtained from p i In another example, the Word4 value can be extended to include more co-located child nodes in the reference frame; with each additional node, the number of bits used to derive Word4 can be increased. Similar modifications can also be applied to the construction of Word7 values.
[0143] In another example, the use of co-located nodes in the reference frame may depend on whether the child node is a sparse region. For example, when the child node is in a sparse region, only the co-located nodes of the child node (p0) may be selected to derive occupancy. When the child node is in a non-sparse region, the co-located nodes of the child node and one or more of its neighbors may be used to derive occupancy.
[0144] The sparsity of a region containing a child node may be inferred by examining the occupancy of one or more neighbors of the child node, a co-located node in a reference frame, and one or more neighbors of the co-located node in the reference frame.
[0145] Fig.15 is a flow chart illustrating an example encoding method according to techniques of this disclosure. Fig.15 The technique may be performed by one or more structural components of the G-PCC encoder 200 (including the arithmetic coding unit 214).
[0146] In one example of the present disclosure, the G-PCC encoder 200 is configured to determine the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of point cloud data for inter-frame prediction of a current node in a current frame of point cloud data (1500). The G-PCC encoder 200 may also determine a context for encoding a current occupied bit of a current subnode of the current node based on the occupancy of the reference subnode (1502), and use the context to arithmetic encode the current occupied bit (1504).
[0147] In one example, the reference child node is co-located with the current child node.
[0148] In another example, the reference child node is an i-th child node of the reference node, and wherein, to determine the occupancy of the reference child node, the G-PCC encoder 200 is configured to determine a value for an i-th entry in an occupancy syntax element of the reference node.
[0149] In another example, the reference subnode is an i-th child node of the reference node, and wherein, to determine the occupancy of the reference subnode, the G-PCC encoder 200 is configured to determine the occupancy of the reference subnode by determining the number of points included in the reference subnode.
[0150] In another example, the G-PCC encoder 200 is configured to determine the occupancy of the current node using a dynamic optimal binary decoder with on-the-fly update (OBUF) and current occupancy bits of current child nodes of the current node.
[0151] In another example, the G-PCC encoder 200 is configured to encode the geometry of the point cloud data using the current occupied bits of the current child node of the current node. In one example, the geometry is an octree geometry. In another example, in order to encode the geometry, the G-PCC encoder 200 is configured to encode the geometry using inter-frame prediction. In another example, in order to encode the geometry, the G-PCC encoder 200 is configured to encode the geometry using intra-frame prediction, wherein the occupancy of the reference child node is zero. In another example, the G-PCC encoder 200 is configured to encode the attributes of the point cloud data, wherein the attributes and the geometry form the encoded point cloud data.
[0152] In another example, in order to determine a context for encoding currently occupied bits of a current child node of a current node, the G-PCC encoder 200 is configured to determine a context for encoding currently occupied bits of a current child node of the current node based on an occupancy of a reference child node and one or more sibling occupancies of one or more reference sibling nodes, wherein the reference sibling nodes are located in a spatial neighborhood of the reference child node.
[0153] Fig.16 is a flow chart illustrating an example decoding method according to techniques of this disclosure. Fig.16 The technique may be performed by one or more structural components of the G-PCC decoder 300, including the geometry arithmetic decoding unit 302.
[0154] In one example of the present disclosure, the G-PCC decoder 300 is configured to determine an occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of point cloud data for inter-frame prediction of a current node in a current frame of point cloud data (1600). The G-PCC decoder 300 may also determine a context for decoding a current occupied bit of a current subnode of the current node based on the occupancy of the reference subnode (1602), and use the context to arithmetically decode the current occupied bit (1604).
[0155] In one example, the reference child node is co-located with the current child node.
[0156] In another example, the reference child node is an i-th child node of the reference node, and wherein, to determine the occupancy of the reference child node, the G-PCC decoder 300 is configured to determine a value of an i-th entry in an occupancy syntax element for the reference node.
[0157] In another example, the reference subnode is an i-th child node of the reference node, and wherein, to determine the occupancy of the reference subnode, the G-PCC decoder 300 is configured to determine the occupancy of the reference subnode by determining the number of points included in the reference subnode.
[0158] In another example, the G-PCC decoder 300 is configured to determine the occupancy of the current node using a dynamic optimal binary decoder with immediate update (OBUF) and current occupancy bits of current child nodes of the current node.
[0159] In another example, the G-PCC decoder 300 is configured to decode the geometry of the point cloud data using the current occupied bits of the current child node of the current node. In one example, the geometry is an octree geometry. In another example, in order to decode the geometry, the G-PCC decoder 300 is configured to decode the geometry using inter-frame prediction. In another example, in order to decode the geometry, the G-PCC decoder 300 is configured to decode the geometry using intra-frame prediction, wherein the occupancy of the reference child node is zero. In another example, the G-PCC decoder 300 is configured to decode the attributes of the point cloud data, wherein the attributes and the geometry form the decoded point cloud data.
[0160] In another example, in order to determine a context for decoding currently occupied bits of a current child node of a current node, the G-PCC decoder 300 is configured to determine a context for decoding currently occupied bits of a current child node of the current node based on an occupancy of a reference child node and one or more sibling occupancies of one or more reference sibling nodes, wherein the reference sibling nodes are located in a spatial neighborhood of the reference child node.
[0161] In version 5 InterEM (mpegx.int-evry.fr / software / MPEG / PCC / CE / mpeg-pcc-tmc13 / - / tree / mpeg137 / mtg / InterEM-v5.0-dev), IDCM eligibility for inter may not have the best trade-off between decoding performance and runtime complexity. For example, when angle mode is disabled, the decoding runtime may double compared to intra decoding, while the performance is not that high. In the current InterEM version 5, IDCM eligibility is defined as follows:
[0162] As can be seen above, IDCM eligibility can be set to false when angular mode is disabled (i.e., when isAngularModeEnabled is false) and occupancy is predictable (i.e., occupancyIsPredictable is true). The code above shows this as "if (occupancyIsPredictable && !isAngularModeEnabled) return false". As such, in version 5, when angular mode is enabled, IDCM eligibility does not depend on inter-frame prediction. occupancyIsPredictable is true when the mismatch between the occupancy of the parent node's sibling nodes and the co-located nodes in the reference is less than 6 bits. If occupancyIsPredictable is true and angular mode is disabled, octree occupancy is used. This means that when inter-frame prediction is enabled, fewer nodes may be decoded using IDCM, which may not be desirable.
[0163] According to one or more techniques or disclosures, a G-PCC decoder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) may utilize alternative IDCM eligibility criteria for inter-coded frames. For example, the G-PCC decoder may utilize IDCM eligibility criteria that allow IDCM eligibility to be true when angle mode is enabled or when angle mode is disabled. For example, the G-PCC encoder 200 and the G-PCC decoder 300 may determine whether the current node is eligible for IDCM regardless of whether angle mode is enabled for the current node. As such, in one example, an inter-coded frame when angle mode is enabled and an intra-coded frame (angle enabled or angle disabled) are identical, as follows (with respect to the removal of the previous code in the tag <del> and< / del> shown between):
[0164] As can be seen from the examples above, IDCM eligibility can be determined based on strength, with different parameters for strength levels 0-3. IDCM can be enabled for strength levels 1-3, regardless of whether angle mode is enabled. By allowing IDCM eligibility when angle mode is enabled, more nodes can be decoded using IDCM. In this way, the techniques of the present disclosure can desirably improve decoding efficiency (e.g., because fewer bits may be required to decode a node using IDCM than if IDCM were not used).
[0165] In another example, when angles are disabled, IDCM eligibility for inter-coded frames may be defined as follows:
[0166] As can be seen from the examples above, whether or not angle mode is enabled, IDCM eligibility can be determined based on strength, occupancy predictability, and / or the number of peer nodes. By allowing IDCM eligibility when angle mode is enabled, more nodes can be decoded using IDCM. In this way, the techniques of the present disclosure can desirably improve decoding efficiency (e.g., because fewer bits may be required to decode a node using IDCM than if IDCM were not used).
[0167] In one example above, for inter prediction, under the conditions of angularMode=0 and occupancyIsPredictable=0, only IDCM mode 1 is allowed.
[0168] In addition, the number of contexts based on inter-frame prediction is 4, which significantly increases the total number of contexts used for occupancy decoding. The present disclosure proposes to use fewer inter-frame prediction contexts while maintaining decoding performance. For example, a G-PCC decoder can utilize two or fewer contexts.
[0169] In InterEM version 5, the four inter prediction contexts are derived in GeometryOctreeDecoder::decodeOccupancyNeighNZ() (which decodes the occupancy of a node when N64 is non-zero) as follows:
[0170] According to one or more techniques of the present disclosure, the G-PCC decoder can utilize the occupied bits of the co-located child node that can be used as context for occupied decoding of the current child node. For example, the G-PCC decoder can operate according to the following (the removal relative to the previous code is shown between the tags <Del> and < / Del>): And the code in the function GeometryOctreeDecoder::decodeOccupancyNeighZ() (which decodes the occupancy of a node when N64 is zero) can be modified accordingly as follows:
[0171] Fig.17 1 is a conceptual diagram illustrating an example ranging system 1700 that can be used with one or more techniques of this disclosure. Fig.17 In an example of , ranging system 1700 includes illuminator 1702 and sensor 1704. Illuminator 1702 can emit light 1706. In some examples, illuminator 1702 can emit light 1706 as one or more laser beams. Light 1706 can be at one or more wavelengths, such as infrared wavelengths or visible light wavelengths. In other examples, light 1706 is not a coherent laser. When light 1706 encounters an object such as object 1708, light 1706 produces return light 1710. Return light 1710 may include backscattered light and / or reflected light. Return light 1710 may pass through lens 1711, which guides return light 1710 to produce an image 1712 of object 1708 on sensor 1704. Sensor 1704 generates signal 1714 based on image 1712. Image 1712 may include a set of points (e.g., as shown by Fig.17 1712).
[0172] In some examples, illuminator 1702 and sensor 1704 can be mounted on a rotating structure so that illuminator 1701 and sensor 1702 capture a 360-degree view of the environment (e.g., a rotating LIDAR sensor). In other examples, ranging system 1700 can include one or more optical components (e.g., mirrors, collimators, diffraction gratings, etc.) that enable illuminator 1702 and sensor 1704 to detect the range of objects within a certain range (e.g., up to 360 degrees). Although Fig.17 The example shows only a single illuminator 1702 and sensor 1704, but the ranging system 1700 can include multiple sets of illuminators and sensors.
[0173] In some examples, the illuminator 1702 generates a structured light pattern. In such an example, the ranging system 1700 may include a plurality of sensors 1704 on which respective images of the structured light pattern are formed. The ranging system 1700 may use the apparent difference between the images of the structured light pattern to determine the distance to an object 1708 from which the structured light pattern is backscattered. When the object 1708 is relatively close to the sensor 1704 (e.g., 0.2 meters to 2 meters), the ranging system based on structured light can have a high level of accuracy (e.g., accuracy in the sub-millimeter range). This high level of accuracy is useful in facial recognition applications such as unlocking mobile devices (e.g., mobile phones, tablet computers, etc.) and for security applications.
[0174] In some examples, the ranging system 1700 is a time-of-flight (ToF) based system. In some examples where the ranging system 1700 is a ToF based system, the illuminator 1702 generates pulses of light. In other words, the illuminator 1702 can modulate the amplitude of the emitted light 1706. In such an example, the sensor 1704 detects the return light 1710 from the pulse of light 1706 generated by the illuminator 1702. The ranging system 1700 can then determine the distance to the object 1708 from which the light 1706 is backscattered based on the delay between when the light 1706 is emitted and detected and the known speed of light in air. In some examples, the illuminator 1702 can modulate the phase of the emitted light 1706 instead of (or in addition to) modulating the amplitude of the emitted light 1706. In such an example, sensor 1704 can detect the phase of return light 1710 from object 1708 and determine the distance to a point on object 1708 using the speed of light and based on the time difference between when illuminator 1702 generates light 1706 at a particular phase and when sensor 1704 detects return light 1710 at the particular phase.
[0175] In other examples, the point cloud may be generated without using the illuminator 1702. For example, in some examples, the sensor 1704 of the ranging system 1700 may include two or more optical cameras. In such an example, the ranging system 1700 may use the optical cameras to capture a stereoscopic image of an environment including the object 1708. The ranging system 1700 may include a point cloud generator 1716 that may calculate the apparent differences between the positions in the stereoscopic images. The ranging system 1700 may then use the apparent differences to determine the distance from the positions shown in the stereoscopic images. Based on these distances, the point cloud generator 1716 may generate a point cloud.
[0176] Sensor 1704 may also detect other properties of object 1708, such as color and reflectivity information. Fig.17 In the example of , the point cloud generator 1716 can generate a point cloud based on the signal 1714 generated by the sensor 1704. The ranging system 1700 and / or the point cloud generator 1716 can form a data source 104 ( Figure 1 ). Therefore, the point cloud generated by the ranging system 1700 can be encoded and / or decoded according to any of the techniques of the present disclosure. Inter-frame prediction and residual prediction as described in the present disclosure can reduce the size of the encoded data.
[0177] Fig.18 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of the present disclosure may be used. Fig.18In the example of FIG. 1 , vehicle 1800 includes a ranging system 1802. The ranging system 1802 can be used to measure the distance between the vehicle and the vehicle. Fig.17 Although not in Fig.18 , but the vehicle 1800 may also include a data source (such as data source 104 ( Figure 1 )) and a G-PCC encoder (such as G-PCC encoder 200 ( Figure 1 )).exist Fig.18 In the example of FIG. 1 , a range finding system 1802 emits a laser beam 1804 that is reflected from a pedestrian 1806 or other object in the road. A data source of the vehicle 1800 can generate a point cloud based on the signal generated by the range finding system 1802. A G-PCC encoder of the vehicle 1800 can encode the point cloud to generate a bit stream 1808, such as a geometry bit stream ( Figure 2 ) and attribute bitstream ( Figure 2 ). Inter-frame prediction and residual prediction as described in the present disclosure can reduce the size of the geometry bitstream. Compared to the unencoded point cloud obtained by the G-PCC encoder, the bitstream 1808 can include much fewer bits.
[0178] The output interface of the vehicle 1800 (eg, the output interface 108 ( Figure 1 )) can send the bitstream 1808 to one or more other devices. Compared with the unencoded point cloud obtained by the G-PCC encoder, the bitstream 1808 may include much fewer bits. Therefore, compared with the unencoded point cloud data, the vehicle 1800 may be able to send the bitstream 1808 to other devices faster. In addition, the bitstream 1808 may require less data storage capacity on the device.
[0179] exist Fig.18 In the example of FIG. 1 , vehicle 1800 may send a bitstream 1808 to another vehicle 1810. Vehicle 1810 may include a G-PCC decoder, such as G-PCC decoder 300 ( Figure 1 ). The G-PCC decoder of vehicle 1810 can decode bitstream 1808 to reconstruct the point cloud. Vehicle 1810 can use the reconstructed point cloud for various purposes. For example, vehicle 1810 can determine that pedestrian 1806 is in the road in front of vehicle 1800 based on the reconstructed point cloud, and therefore begin to slow down, for example, even before the driver of vehicle 1810 realizes that pedestrian 1806 is in the road. Therefore, in some examples, vehicle 1810 can perform autonomous navigation operations based on the reconstructed point cloud.
[0180] Additionally or alternatively, the vehicle 1800 may send a bitstream 1808 to a server system 1812. The server system 1812 may use the bitstream 1808 for various purposes. For example, the server system 1812 may store the bitstream 1808 for subsequent reconstruction of the point cloud. In this example, the server system 1812 may use the point cloud along with other data (e.g., vehicle telemetry data generated by the vehicle 1800) to train an autonomous driving system. In other examples, the server system 1812 may store the bitstream 1808 for subsequent reconstruction for use in a collision investigation suitable for a court of law.
[0181] Fig.19 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of the present disclosure may be used. Extended reality (XR) is a term used to encompass a range of technologies including augmented reality (AR), mixed reality (MR), and virtual reality (VR). Fig.19 In the example of FIG. 1 , user 1900 is located at a first location 1902. User 1900 wears an XR headset 1904. As an alternative to the XR headset 1904, user 1900 may use a mobile device (e.g., a mobile phone, a tablet computer, etc.). The XR headset 1904 includes a depth detection sensor (such as a range finding system) that detects the location of a point on an object 1906 at location 1902. A data source for the XR headset 1904 may use a signal generated by the depth detection sensor to generate a point cloud representation of the object 1906 at location 1902. The XR headset 1904 may include a G-PCC encoder (e.g., Figure 1 The G-PCC encoder 200 is configured to encode the point cloud to generate a bitstream 1908. Inter-frame prediction and residual prediction as described in the present disclosure can reduce the size of the bitstream 1908.
[0182] The XR headset 1904 may send a bitstream 1908 to an XR headset 1910 worn by a user 1912 at a second location 1914 (e.g., via a network such as the Internet). The XR headset 1910 may decode the bitstream 1908 to reconstruct the point cloud. The XR headset 1910 may use the point cloud to generate an XR visualization (e.g., AR, MR, VR visualization) representing an object 1906 at the location 1902. Thus, in some examples, such as when the XR headset 1910 generates a VR visualization, the user 1912 may have a 3D immersive experience of the location 1902. In some examples, the XR headset 1910 may determine the location of a virtual object based on the reconstructed point cloud. For example, the XR headset 1910 may determine that the environment (e.g., location 1902) includes a flat surface based on the reconstructed point cloud, and then determine that the virtual object (e.g., a cartoon character) will be positioned on the flat surface. The XR headset 1910 may generate an XR visualization in which the virtual object is located at the determined location. For example, the XR headset 1910 may display a cartoon character located on a flat surface.
[0183] Fig. 20 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of this disclosure may be used. Fig. 20 In an example of FIG. 1 , a mobile device 2000 (e.g., a wireless communication device) (such as a mobile phone or tablet computer) includes a range finding system (such as a LIDAR system) that detects the location of points on an object 2002 in the environment of the mobile device 2000. A data source of the mobile device 2000 can use a signal generated by a depth detection sensor to generate a point cloud representation of the object 2002. The mobile device 2000 can include a G-PCC encoder (e.g., Figure 1 A G-PCC encoder 200 is configured to encode the point cloud to generate a bitstream 2004. Fig. 20In an example, mobile device 2000 can send a bitstream to remote device 2006 (such as a server system or other mobile device). Inter-frame prediction and residual prediction as described in the present disclosure can reduce the size of bitstream 2004. Remote device 2006 can decode bitstream 2004 to reconstruct point cloud. Remote device 2006 can use point cloud for various purposes. For example, remote device 2006 can use point cloud to generate a map of the environment of mobile device 2000. For example, remote device 2006 can generate a map of the interior of a building based on the reconstructed point cloud. In another example, remote device 2006 can generate an image (e.g., computer graphics) based on point cloud. For example, remote device 2006 can use the point of point cloud as the vertex of a polygon, and use the color attribute of the point as the basis for coloring the polygon. In some examples, remote device 2006 can use the reconstructed point cloud for facial recognition or other security applications.
[0184] Examples in various aspects of the present disclosure may be used individually or in any combination. Additional aspects are described below:
[0185] Aspect 1A - A method for decoding a point cloud, the method comprising: determining the occupied bits of the i-th child node in a reference node; determining a context for decoding the current occupied bits of the i-th node of a current node based on the occupied bits of the i-th child node in the reference node; and using the context to arithmetically decode the current occupied bits.
[0186] Aspect 2A - A method for decoding a point cloud, the method comprising: determining occupied bits of sibling nodes of a co-located node in a reference frame; determining a context for decoding currently occupied bits of a current node based on the occupied bits of sibling nodes of the co-located node in the reference frame; and using the context to arithmetically decode the currently occupied bits.
[0187] Aspect 3A - The method according to any of Aspects 1A to 2A, further comprising generating a point cloud.
[0188] Aspect 4A - An apparatus for processing a point cloud, the apparatus comprising one or more units for performing the method according to any one of Aspects 1A to 3A.
[0189] Aspect 5A - The apparatus of aspect 4A, wherein the one or more units include one or more processors implemented in the circuit system.
[0190] Aspect 6A-The apparatus according to any of aspects 4A or 5A, further comprising a memory for storing data representing the point cloud.
[0191] Aspect 7A - The apparatus according to any of Aspects 4A to 6A, wherein the apparatus comprises a decoder.
[0192] Aspect 8A - An apparatus according to any of Aspects 4A to 7A, wherein the apparatus comprises an encoder.
[0193] Aspect 9A - The apparatus according to any of Aspects 4A to 8A, further comprising means for generating a point cloud.
[0194] Aspect 10A-The device according to any of Aspects 4A-9A, further comprising a display for presenting an image based on the point cloud.
[0195] Aspect 11A - A computer-readable storage medium having stored thereon instructions which, when executed, cause one or more processors to perform a method according to any of Aspects 1A-3A.
[0196] Aspect 1B - A device configured to decode point cloud data, the device comprising: a memory configured to store point cloud data; and one or more processors communicating with the memory, the one or more processors being configured to: determine the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of the point cloud data for inter-frame prediction of a current node in a current frame of the point cloud data; determine a context for decoding a current occupied bit of a current subnode of the current node based on the occupancy of the reference subnode; and use the context to arithmetically decode the current occupied bit.
[0197] Aspect 2B—The apparatus of Aspect 1B, wherein the reference child node is co-located with the current child node.
[0198] Aspect 3B - An apparatus according to Aspect 1B, wherein the reference child node is the i-th child node of the reference node, and wherein, to determine the occupancy of the reference child node, the one or more processors are configured to: determine the value of the i-th entry in the occupancy syntax element for the reference node.
[0199] Aspect 4B—An apparatus according to Aspect 1B, wherein the reference subnode is the i-th subnode of the reference node, and wherein, in order to determine the occupancy of the reference subnode, one or more processors are configured to: determine the occupancy of the reference subnode by determining the number of points included in the reference subnode.
[0200] Aspect 5B - an apparatus according to any aspect of Aspect 1B to Aspect 4B, wherein one or more processors are further configured to: determine the occupancy of the current node using a dynamic optimal binary decoder (OBUF) with instant updates and the current occupied bits of the current child nodes of the current node.
[0201] Aspect 6B - The apparatus according to any of Aspects 1B to 5B, wherein the one or more processors are further configured to: use the current occupied bits of the current child node of the current node to decode the geometry of the point cloud data.
[0202] Aspect 7B—The apparatus of Aspect 6B, wherein the geometry is an octree geometry.
[0203] Aspect 8B - The apparatus of aspect 6B, wherein, to decode the geometry, the one or more processors are configured to: decode the geometry using inter-frame prediction.
[0204] Aspect 9B - The apparatus of aspect 6B, wherein, to decode the geometry, the one or more processors are configured to: decode the geometry using intra prediction, wherein the occupancy of the reference child node is zero.
[0205] Aspect 10B-A device according to Aspect 6B, wherein the one or more processors are further configured to decode attributes of the point cloud data, wherein the attributes and the geometry form decoded point cloud data, and the device further comprises: a display configured to display the decoded point cloud data.
[0206] Aspect 11B - an apparatus according to any aspect of Aspect 1B - Aspect 10B, wherein, in order to determine a context for decoding currently occupied bits of a current child node of a current node, one or more processors are also configured to: determine a context for decoding currently occupied bits of a current child node of a current node based on the occupancy of a reference child node and one or more sibling occupancies of one or more reference sibling nodes, wherein the reference sibling nodes are located in a spatial neighborhood of the reference child node.
[0207] Aspect 12B - A method for decoding point cloud data, the method comprising: determining the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of the point cloud data for inter-frame prediction of a current node in a current frame of the point cloud data; determining a context for decoding a current occupied bit of a current subnode of the current node based on the occupancy of the reference subnode; and using the context to arithmetically decode the current occupied bit.
[0208] Aspect 13B-The method according to Aspect 12B, wherein the reference child node is co-located with the current child node.
[0209] Aspect 14B—The method according to Aspect 12B, wherein the reference child node is an i-th child node of the reference node, and wherein determining the occupancy of the reference child node comprises: determining a value of an i-th entry in an occupancy syntax element for the reference node.
[0210] Aspect 15B—The method according to Aspect 12B, wherein the reference subnode is the i-th subnode of the reference node, and wherein determining the occupancy of the reference subnode comprises: determining the occupancy of the reference subnode by determining the number of points included in the reference subnode.
[0211] Aspect 16B-The method according to any of Aspects 12B-15B further includes: determining the occupancy of the current node using a dynamic optimal binary decoder (OBUF) with instant update and the current occupancy bit of the current child node of the current node.
[0212] Aspect 17B - The method according to any one of Aspects 12B to 16B, further comprising: using the current occupied bits of the current child node of the current node to decode the geometry of the point cloud data.
[0213] Aspect 18B—The method according to Aspect 17B, wherein the geometry is an octree geometry.
[0214] Aspect 19B - The method according to aspect 17B, wherein decoding the geometry includes: decoding the geometry using inter-frame prediction.
[0215] Aspect 20B - The method according to aspect 17B, wherein decoding the geometry comprises: decoding the geometry using intra prediction, wherein the occupancy of the reference child node is zero.
[0216] Aspect 21B-The method according to Aspect 17B further includes decoding attributes of the point cloud data, wherein the attributes and the geometric shape form the decoded point cloud data, and the method further includes: displaying the decoded point cloud data.
[0217] Aspect 22B - a method according to any aspect of Aspect 12B - Aspect 21B, wherein determining a context for decoding currently occupied bits of a current child node of a current node comprises: determining a context for decoding currently occupied bits of a current child node of a current node based on an occupancy of a reference child node and one or more sibling occupancies of one or more reference sibling nodes, wherein the reference sibling nodes are located in a spatial neighborhood of the reference child node.
[0218] Aspect 23B - A non-temporary computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to decode point cloud data to perform the following operations: determine the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of the point cloud data used for inter-frame prediction of a current node in a current frame of the point cloud data; determine a context for decoding currently occupied bits of a current subnode of the current node based on the occupancy of the reference subnode; and use the context to arithmetically decode the currently occupied bits.
[0219] Aspect 24B - A device configured to decode point cloud data, the device comprising: a unit for determining the occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of the point cloud data for inter-frame prediction of a current node in a current frame of the point cloud data; a unit for determining a context for decoding a current occupied bit of a current subnode of the current node based on the occupancy of the reference subnode; and a unit for arithmetically decoding the current occupied bit using the context.
[0220] It is to be appreciated that, depending on the example, certain actions or events of any of the techniques described herein may be performed in a different order, may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary for the practice of the techniques). Additionally, in some examples, actions or events may be performed concurrently rather than sequentially, such as through multithreading, interrupt handling, or multiple processors.
[0221] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium or sent through it and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium or a communication medium, the communication medium including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-temporary, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in the present disclosure. A computer program product may include a computer-readable medium.
[0222] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, flash memory, or may be used to store desired program codes in the form of instructions or data structures and any other medium that can be accessed by a computer. In addition, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, optical fiber cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to send instructions from a website, server or other remote source, then the coaxial cable, optical fiber cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave) is included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals or other temporary media, but are instead directed to non-temporary tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser optical disks, optical disks, digital versatile disks (DVDs), floppy disks and blue-ray disks, wherein disks usually copy data magnetically, while optical disks use lasers to optically copy data. Combinations of the above should also be included within the scope of computer-readable media.
[0223] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuit systems. Accordingly, the terms "processor" and "processing circuit system" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. In addition, the techniques may be fully implemented in one or more circuits or logic elements.
[0224] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in the present disclosure to emphasize the functional aspects of a device configured to perform the disclosed techniques, but are not necessarily required to be implemented by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit, or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.
[0225] Various examples have been described. These and other examples are within the scope of the following claims.
Claims
1. A device configured to decode point cloud data, the device include: A memory configured to store the point cloud data; as well as One or more processors in communication with the memory, the one or more processors configured to: determining occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of point cloud data for inter-frame prediction of a current node in a current frame of the point cloud data; determining a context for decoding currently occupied bits of a current child node of the current node based on the occupancy of the reference child node; and The currently occupied bits are arithmetically decoded using the context.
2. The device according to claim 1, in, The reference child node is co-located with the current child node.
3. The device according to claim 1, in, The reference subnode is an i-th subnode of the reference node, and wherein, to determine the occupancy of the reference subnode, the one or more processors are configured to: A value of an i-th entry in an occupied syntax element for the reference node is determined.
4. The device according to claim 1, in, The reference subnode is an i-th subnode of the reference node, and wherein, to determine the occupancy of the reference subnode, the one or more processors are configured to: The occupancy of the reference subnode is determined by determining the number of points included in the reference subnode.
5. The device according to claim 1, in, The one or more processors are further configured to: The occupancy of the current node is determined using a dynamic optimal binary decoder with immediate update (OBUF) and the current occupancy bits of the current child nodes of the current node.
6. The device according to claim 1, in, The one or more processors are further configured to: The geometry of the point cloud data is decoded using the currently occupied bits of the current child node of the current node.
7. The device according to claim 6, in, The geometry is an octree geometry.
8. The device according to claim 6, in, To decode the geometric shape, the one or more processors are configured to: The geometry is decoded using inter-frame prediction.
9. The device according to claim 6, in, To decode the geometric shape, the one or more processors are configured to: The geometry is decoded using intra prediction, wherein the occupancy of the reference child node is zero.
10. The device according to claim 6, in, The one or more processors are further configured to decode attributes of the point cloud data, wherein the attributes and the geometric shape form decoded point cloud data, the apparatus further comprising: A display is configured to display the decoded point cloud data.
11. The device according to claim 1, in, To determine the context for decoding the currently occupied bits of the current child node of the current node, the one or more processors are further configured to: The context for decoding the current occupied bits of the current child node of the current node is determined based on the occupancy of the reference child node and one or more sibling occupancies of one or more reference sibling nodes, wherein the reference sibling nodes are located in the spatial neighborhood of the reference child node.
12. A method for decoding point cloud data, the method include: determining occupancy of a reference subnode in a reference node, wherein the reference node is in a reference frame of point cloud data for inter-frame prediction of a current node in a current frame of the point cloud data; determining a context for decoding currently occupied bits of a current child node of the current node based on the occupancy of the reference child node; and The currently occupied bits are arithmetically decoded using the context.
13. The method according to claim 12, in, The reference child node is co-located with the current child node.
14. The method according to claim 12, in, The reference subnode is an i-th subnode of the reference node, and wherein determining the occupancy of the reference subnode comprises: A value of an i-th entry in an occupied syntax element for the reference node is determined.
15. The method according to claim 12, in, The reference subnode is an i-th subnode of the reference node, and wherein determining the occupancy of the reference subnode comprises: The occupancy of the reference subnode is determined by determining the number of points included in the reference subnode.
16. The method according to claim 12, further comprising: include: The occupancy of the current node is determined using a dynamic optimal binary decoder with immediate update (OBUF) and the current occupancy bits of the current child nodes of the current node.
17. The method according to claim 12, further comprising: include: The geometry of the point cloud data is decoded using the currently occupied bits of the current child node of the current node.
18. The method according to claim 17, in, The geometry is an octree geometry.
19. The method according to claim 17, in, Decoding the geometric shape includes: The geometry is decoded using inter-frame prediction.
20. The method according to claim 17, in, Decoding the geometric shape includes: The geometry is decoded using intra prediction, wherein the occupancy of the reference child node is zero.
21. The method according to claim 17, further comprising: include: decoding an attribute of the point cloud data, wherein the attribute and the geometric shape form decoded point cloud data, the method further comprising: The decoded point cloud data is displayed.
22. The method according to claim 12, in, Determining the context for decoding the currently occupied bits of the current child node of the current node comprises: The context for decoding the current occupied bits of the current child node of the current node is determined based on the occupancy of the reference child node and one or more sibling occupancies of one or more reference sibling nodes, wherein the reference sibling nodes are located in the spatial neighborhood of the reference child node.
23. A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors of a device configured to decode point cloud data to: Determine the occupancy of the reference subnodes in the reference node, in, The reference node is in a reference frame of the point cloud data used for inter-frame prediction of a current node in a current frame of the point cloud data; determining a context for decoding currently occupied bits of a current child node of the current node based on the occupancy of the reference child node; as well as The currently occupied bits are arithmetically decoded using the context.
24. A device configured to decode point cloud data, the device include: means for determining occupancy of a reference child node in a reference node in a reference frame of point cloud data for inter-frame prediction of a current node in a current frame of the point cloud data; means for determining a context for decoding currently occupied bits of a current child node of the current node based on the occupancy of the reference child node; and Means for arithmetically decoding the currently occupied bits using the context.
